Issue
How do I move an existing Fusion instance from one node to another within a cluster?
Environment
Fusion 3
Resolution
Note: Do not use this for migrations between environments. Follow the Fusion documentation on the site for that. This is only for quick moves within an existing cluster.
There are a few reasons why you may want to move an existing Fusion instance from one node to another. Maybe you want to move within your network or change data centers or you may want to move to a bigger machine when upgrading your current one is no longer feasible.
In such a case, you don’t want to set up a fresh Fusion instance and do the hard work of manually moving all of your data. No, there’s a much simpler way of handling this process and here I’m going to detail out the steps to be executed:
1) Shutdown your cluster: This means shutting down everything right from zookeeper to your ui. The reason to do this is to avoid inconsistencies between nodes that could result in an unstable cluster after the move is finished.
2) Backup: Compress the current Fusion instance directory (entire directory rooted at $FUSION_HOME) and keep it somewhere safe. Also backup the zookeeper data folder and configuration (conf/zookeeper/zoo.cfg). This includes the myid file and the version-2 folder. ZK backup is needed in case you’re running an external ensemble. If you’re running the bundled version, then backing up fusion should take care of this.
3) Copy the Fusion instance to new node: You can copy either the compressed fusion directory or the entire folder. In most cases, copying the compressed file is much faster than copying the directory.
4) Decompress the Fusion instance in a directory from where you’ll be running it.
5) Modify fusion.properties (Not needed if ZK is external and/or not being migrated): Change the default.zk.connect string to point to the IP of your new node where ZK will be running.
6) Modify zoo.cfg: You now want to modify the zoo.cfg file for all zk instances and replace the IP of old machine in the server.x entry with the IP and ports of the new machine.
7) Modify the connectString for the default searchCluster (Not needed if ZK is external and/or not being migrated): To do this, first start zookeeper on all Fusion nodes except the old one. The location for this search cluster in zk will vary by version and can be found after the download upload instructions.
To modify the connect string, we’ll be using the zkcli script that is bundled with solr. This can be found under $FUSION_HOME/apps/solr-dist/server/scripts/cloud-scripts.
Use either the zkcli.sh or the zkcli.bat file depending on your OS.
Download the file: ./zkcli.sh -z <zk1>:<port1>,<zk2>:<port2>,... -cmd getfile <path_to_default_cluster> <path_to_dump_file>.json
Eg. ./zkcli.sh -z localhost:9983 -cmd getfile /lwfusion/3.1.2/core/search-clusters/default default_search_cluster.json
Modify the file: In the downloaded json file, find the connectString key and replace the old IP in it’s value with the IP of your new Fusion node. Be sure to specify the chroot if your cluster is configured so.
Eg.
{
"id" : "default",
"connectString" : "localhost:9983/lwfusion/3.1.2/solr",
"zkClientTimeout" : 30000,
"zkConnectTimeout" : 60000,
"cloud" : true,
"bufferFlushInterval" : 1000,
"bufferSize" : 100,
"concurrency" : 10,
"authConfig" : {
"authType" : "none"
},
"validateCluster" : true
}
Upload the file: ./zkcli.sh -z <zk1>:<port1>,<zk2>:<port2>,... -cmd putfile <path_to_default_cluster> <path_to_dump_file>.json
Eg. ./zkcli.sh -z localhost:9983 -cmd putfile /lwfusion/3.1.2/core/search-clusters/default default_search_cluster.json
Locations of the default cluster:
2.4.x: /lucid/search-clusters/default
3.x: /lwfusion/<fusion_version>/core/search-clusters/default
8) Start entire cluster: That’s all folks! Now go ahead and start all the fusion nodes in your cluster except for the old one. Verify that the new Fusion nodes is consistent with the others and stable and you’re good to go.