Increase Zookeeper file size limit to support large dictionary uploads

Goal

Upload dictionary files larger than 1MB to Zookeeper in Fusion by increasing the Zookeeper jute.maxbuffer size limit.

Environment

• Fusion 5.9.2 
• Kubernetes 1.27
• Applicable to any Kubernetes-based deployment of Fusion where Solr configuration files (e.g., Hunspell dictionaries) exceed the default Zookeeper file size limit of 1MB.

Guide

Uploading large files to Zookeeper (such as .dic files used for Hunspell filters) may fail due to the default jute.maxbuffer limit of 1MB. To resolve this, increase the limit to 4MB across all services that interact with Zookeeper and Solr configuration.

Increase buffer size in Zookeeper

1. Edit the Zookeeper ConfigMap:

kubectl edit configmap <zk-service-name> -n <namespace>

2. Under the run: section, set the JVMFLAGS to:

JVMFLAGS="-Xmx$ZK_HEAP_SIZE -Xms$ZK_HEAP_SIZE -XX:+ExitOnOutOfMemoryError -Djute.maxbuffer=4000000"

3. Save and restart the StatefulSet:

kubectl rollout restart sts <zk-statefulset-name> -n <namespace>

Update Fusion Admin configuration

1. Edit the deployment for Fusion Admin:

kubectl edit deployment <fusion-admin-deployment> -n <namespace>

2. Add or update the following environment variables:

- name: JAVA_OPTS
  value: -Djute.maxbuffer=4000000

- name: JAVA_TOOL_OPTIONS
  value: -Dspring.profiles.active=k8s -Dspring.cloud.kubernetes.secrets.paths=/etc/secrets -XX:PermSize=256m -XX:MaxPermSize=512m -Djute.maxbuffer=4000000

3. Restart the deployment:

kubectl rollout restart deployment <fusion-admin-deployment> -n <namespace>

Update Solr configuration

1. Edit the Solr StatefulSet:

kubectl edit statefulset <solr-service-name> -n <namespace>

2. Add or modify the following environment variables:

- name: SOLR_OPTS
  value: -Djute.maxbuffer=4000000

- name: SOLR_JAVA_MEM
  value: -Xmx4g -Xms4g -Djute.maxbuffer=4000000 -XX:+ExitOnOutOfMemoryError

3. Restart the StatefulSet:

kubectl rollout restart sts <solr-service-name> -n <namespace>

Validation

• Attempt to upload a file larger than 1MB (e.g., a Hunspell .dic dictionary) using the Fusion Admin UI or the Solr Config API.

• If upload succeeds but the file still cannot be viewed in the UI, verify that Djute.maxbuffer=4000000 is also present in the JAVA_TOOL_OPTIONS of the Fusion Admin container.

Notes

• Files above 1MB may still fail to render in the UI due to frontend or browser-side limits, even if upload succeeds.

• Hunspell dictionaries sourced from OpenOffice or LibreOffice can commonly exceed the 1MB threshold.

• For large-scale or frequently updated dictionaries, consider external storage solutions like a blob store as an architectural alternative.