Skip to main content
Sign in

Solr pod disk storage is full

  • Updated

Issue

Solr pods may stop processing data or queries when the allocated disk storage in their Persistent Volume Claim (PVC) reaches capacity. Attempts to increase PVC size may not reflect in the running pods.

 

Diagnosis

To confirm the issue:

  • Check Solr pod logs for errors related to insufficient disk space.

  • Verify PVC size and usage:

kubectl get pvc -n <fusion-namespace>
kubectl describe pvc <pvc-name> -n <fusion-namespace>

  • Check pod filesystem usage:

kubectl exec -n <fusion-namespace> <solr-pod-name> -- df -h

  • If the PVC has been expanded but pods are not recognizing the new size, this confirms the issue.

 

Environment

Fusion 5.5.0 and above, deployed on Kubernetes.

 

Cause

Kubernetes StatefulSets running Solr may not automatically mount the resized PVC storage. In these cases, pods continue to use the original allocation until they are recycled or the StatefulSet is recreated.

 

Resolution

Step 1: Free temporary space

If immediate relief is needed, you can delete unnecessary logs or temporary files within the Solr pod:

kubectl exec -n <fusion-namespace> <solr-pod-name> -- rm -rf /var/solr/logs/*

This provides only short-term relief and should not be considered a full resolution.

 

Step 2: Resize the PVC

Increase the PVC size to a larger value:

kubectl edit pvc <pvc-name> -n <fusion-namespace>

Update the spec.resources.requests.storage field with the new desired size.

 

Step 3: Restart Solr pods

Recycle the Solr pods to pick up the new PVC size:

kubectl delete pod <solr-pod-name> -n <fusion-namespace>

 

Step 4: Recreate the StatefulSet if needed

If the pod does not claim the resized volume, delete and recreate the StatefulSet without cascading deletion of PVCs:

kubectl delete statefulset <solr-statefulset> -n <fusion-namespace> --cascade=false

Then reapply the StatefulSet manifest to restore the pods.

 

Step 5: Validate Solr health

After pods restart, verify that:

  • PVC size reflects the new allocation:

kubectl exec -n <fusion-namespace> <solr-pod-name> -- df -h

  • All Solr collections have healthy replicas:

kubectl exec -n <fusion-namespace> <solr-pod-name> -- solr healthcheck

Related articles

Powered by Zendesk