Set a custom replica count for a connector plugin using pluginValues – Lucidworks

Goal

Ensure that a specific connector plugin runs with a custom replica count that persists across restarts of the connectors-backend service.

Environment

Fusion 5.9.0 and later
Kubernetes-based deployments using the Helm chart (e.g., AKS, EKS, GKE)

Guide

Understand the default behavior

By default, when the connectors-backend service is restarted, all connector plugins are re-deployed using the configuration defined in the backend’s Helm values or ConfigMap. If no custom replica count is defined per plugin, the system reverts to the global or default configuration. This can override manual changes made directly to the deployment.

To persist a custom replica count (e.g., 1) for a specific plugin such as ldap-acls, the plugin configuration must be explicitly defined using the pluginValues field in your Helm values file or connectors-backend ConfigMap.

Add `pluginValues` configuration

Update your Helm values or Kubernetes ConfigMap to include a pluginValues section under connector-plugin. The following example shows how to define a single replica for a specific plugin while allowing other plugins to use their default configurations:

connector-plugin:
  pluginValues:
    - id: "ldap-acls"
      replicaCount: 1
      resources:
        limits:
          cpu: "7000m"
          memory: "20000Mi"
        requests:
          cpu: "400m"
          memory: "2000Mi"

Example :

---
# plugins will use the main resources by default
resources:
  limits:
    cpu: "2"
    memory: "2Gi"
  requests:
    cpu: "1"
    memory: "1Gi"
# if specific resources are needed
pluginValues:
  - id: "sharepoint-optimized" (1)
    resources: (2)
      limits:
        cpu: "2"
        memory: "3Gi"
      requests:
        cpu: "250m"
        memory: "2Gi"
    replicaCount: 1 (3)
  - id: "fs"
    resources:
      limits:
        cpu: "600m"
        memory: "3Gi"
      requests:
        cpu: "250m"
        memory: "2Gi"
  - id: "jdbc"
    replicaCount: 2
  - id: "web-v2"
    replicaCount: 3

Note : replicaCount 1 by default, the number of replicas per plugin.

Replace "ldap-acls" with the ID of the plugin you want to configure.

This configuration will persist across connectors-backend pod restarts, and only the specified plugin will have its replica count overridden. All other plugins will continue to use the default replicaCount unless explicitly specified.

Deploy the changes

If modifying the values via Helm:

Update your values file (typically values.yaml)
Run the following command to apply the changes:

helm upgrade <release-name> lucidworks/fusion-values -f values.yaml

If editing directly via Kubernetes ConfigMap:

Export the current ConfigMap:

kubectl get configmap <connectors-backend-configmap-name> -n <namespace> -o yaml > connectors-backend.yaml

Edit the file to include the pluginValues section under the appropriate path
Apply the updated ConfigMap:

kubectl apply -f connectors-backend.yaml

Restart the connectors-backend deployment:

kubectl rollout restart deployment <connectors-backend-deployment-name> -n <namespace>

Verify the deployment

Use the following command to confirm that the correct number of pods is running for the plugin:

kubectl get pods -n <namespace> | grep connector-plugin-ldap-acls

You should see only the number of replicas specified in your configuration (e.g., 1 pod).

Reference

For more details, refer to the Fusion documentation on deploying Fusion 5 on Kubernetes with plugin-specific configurations .