Goal
How can I perform Solr replica re-balancing? A Solr cluster may become imbalanced -- possibly due to data migration, manual replica adds/removals/moves, or some other architectural updates. For example, if I have a two node cluster and create a new single-shard collection with a replication factor of two, I would expect one replica to be created on each Solr node. Under certain conditions, however, a Solr cluster may become imbalanced and create both replicas on the same node. Once the cluster is in this state, all new replicas have the potential to be improperly distributed.
Environment
Tested in Solr 8.11, but applies to any versions of Solr for which the "MOVEREPLICAS" Solr API call is available.
Guide
An imbalanced cluster contains replicas that are not properly distributed among the Solr nodes, leading to multiple new replicas potentially being created on a single Solr node. Once my cluster is in this condition:
- Option 1: Manually fix the replica distribution each time a collection / replica is created -- When I create a new collection with a replication factor of two and both replicas get created on the same node, use the command below to manually move one of the replicas from the source node to the target node.
- Option 2: Rebalance the existing replicas across nodes -- Once the existing Solr cluster is balanced properly, new replicas will be distributed properly going forward. This can be done by looking at the existing Solr nodes and replicas and manually moving as-needed to balance them equally. Once back in the balanced state, new collections/replicas will be created in a balanced fashion.
The command below can be used to move Solr replicas from a source node to a target node.
Note: More information on Solr 8.11 replica management can be found here: https://solr.apache.org/guide/8_11/replica-management.html