Upload jar to Solr collection – Lucidworks

Goal

How to upload a custom JAR/plugin to the Solr collection?

Environment

Solr

Guide

Uploading custom JAR files to Apache Solr, whether in standalone or cloud mode, involves adding the JAR files to the Solr classpath so that Solr can utilize them. Custom JAR files are often used to extend Solr's functionality by providing custom components, functions, or plugins.

Here's how you can upload custom JAR files to Solr in both standalone and cloud mode:

Standalone Mode:

Locate Solr's lib Directory: In Solr standalone mode, you'll need to locate the "lib" directory where Solr's dependencies and custom JARs are stored. This directory is typically located inside the Solr installation directory.
Copy Your Custom Plugin JAR: Copy your custom plugin JAR file into the "lib" directory. The JAR should contain the plugin's class files and related resources.
Configure SolrConfig: You need to configure Solr to use the custom plugin by modifying the "solrconfig.xml" file. This file is located in the "server/solr/configsets/_default/conf" directory by default.
- Open the "solrconfig.xml" file in a text editor.
- Add or modify relevant sections to reference your custom plugin. This may involve configuring request handlers, search components, or other parts of the Solr configuration that use the plugin.
```
<config>
  

  <lib dir="${solr.install.dir:../../../..}/lib" regex="my-custom-plugin-.*\.jar" />

  
</config>
```
Replace "my-custom-plugin-.*\.jar" with a regular expression that matches the name of your custom plugin JAR file.
Restart Solr: After adding the custom plugin JAR and making the necessary configuration changes, restart the Solr instance for the changes to take effect.

Cloud Mode (SolrCloud):

In SolrCloud mode, custom plugins need to be shared across all nodes of the cluster.

When running Solr in SolrCloud mode and you want to use custom code (such as custom analyzers, tokenizers, query parsers, and other plugins), it can be cumbersome to add jars to the classpath on all nodes in your cluster. Using the Blob Store API and special commands with the Config API, you can upload jars to a special system-level collection and dynamically load plugins from them at runtime without needing to restart any nodes.

A drawback of this feature is that it could be used to load malicious executable code into the system. However, it is possible to restrict the system to load only trusted jars using PKI to verify that the executables loaded into the system are trustworthy.

The following steps will allow you enable security for this feature. The instructions assume you have started all your Solr nodes with the -Denable.runtime.lib=true.

Step 1: Generate an RSA Private Key

The first step is to generate an RSA private key. The example below uses a 512-bit key, but you should use the strength appropriate to your needs.

$ openssl genrsa -out priv_key.pem 512

Step 2: Output the Public Key

The public portion of the key should be output in DER format so Java can read it.

$ openssl rsa -in priv_key.pem -pubout -outform DER -out pub_key.der

Step 3: Load the Key to ZooKeeper

The .der files that are output from Step 2 should then be loaded to ZooKeeper under a node /keys/exe so they are available throughout every node. You can load any number of public keys to that node and all are valid. If a key is removed from the directory, the signatures of that key will cease to be valid. So, before removing the a key, make sure to update your runtime library configurations with valid signatures with the update-runtimelib command.

At the current time, you can only use the ZooKeeper zkCli.sh (or zkCli.cmd on Windows) script to issue these commands (the Solr version has the same name, but is not the same). If you have your own ZooKeeper ensemble running already, you can find the script in $ZK_INSTALL/bin/zkCli.sh (or zkCli.cmd if you are using Windows).

To load the keys, you will need to connect to ZooKeeper with zkCli.sh, create the directories, and then create the key file, as in the following example.

# Connect to ZooKeeper
# Replace the server location below with the correct ZooKeeper connect string for your installation.
$ .bin/zkCli.sh -server localhost:9983

# After connection, you will interact with the ZK prompt.
# Create the directories
[zk: localhost:9983(CONNECTED) 5] create /keys
[zk: localhost:9983(CONNECTED) 5] create /keys/exe

# Now create the public key file in ZooKeeper
# The second path is the path to the .der file on your local machine
[zk: localhost:9983(CONNECTED) 5] create /keys/exe/pub_key.der /myLocal/pathTo/pub_key.der

After this, any attempt to load a jar will fail. All your jars must be signed with one of your private keys for Solr to trust it. The process to sign your jars and use the signature is outlined in Steps 4-6.

Step 4: Sign the jar File

Next you need to sign the sha1 digest of your jar file and get the base64 string.

$ openssl dgst -sha1 -sign priv_key.pem myjar.jar | openssl enc -base64

The output of this step will be a string that you will need to add the jar to your classpath in Step 6 below.

Step 5: Load the jar to the Blob Store

Load your jar to the Blob store, using the Blob Store API. This step does not require a signature; you will need the signature in Step 6 to add it to your classpath.

curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @{filename}
http://localhost:8983/solr/.system/blob/{blobname}

The blob name that you give the jar file in this step will be used as the name in the next step.

Step 6: Add the jar to the Classpath

Finally, add the jar to the classpath using the Config API as detailed above. In this step, you will need to provide the signature of the jar that you got in Step 4.

curl http://localhost:8983/solr/techproducts/config -H 'Content-type:application/json'  -d '{
  "add-runtimelib": {
    "name":"blobname",
    "version":2,
    "sig":"mW1Gwtz2QazjfVdrLFHfbGwcr8xzFYgUOLu68LHqWRDvLG0uLcy1McQ+AzVmeZFBf1yLPDEHBWJb5KXr8bdbHN/
           PYgUB1nsr9pk4EFyD9KfJ8TqeH/ijQ9waa/vjqyiKEI9U550EtSzruLVZ32wJ7smvV0fj2YYhrUaaPzOn9g0=" }
}'