Unable to delete query profile due to API error

Issue

Attempts to delete a query profile using the Query Profile API fail with a 500 Internal Server Error, and query profiles do not appear in the Fusion UI. The error response typically includes a message similar to the following:

{
  "timestamp": "2024-10-01T12:12:15.629290",
  "service": "query",
  "error": "service",
  "path": "/query-profiles/<profile_id>",
  "message": "Error deserializing item with id <profile_id>: "
}

When this occurs, the API may also fail to list existing query profiles, returning similar deserialization errors.

Diagnosis

This issue generally indicates a corrupted query profile object stored in ZooKeeper. The corrupted entry prevents Fusion services from properly loading or removing query profile data, resulting in a 500 error response.

To confirm this:

• Attempt to delete the affected profile using the Query Profile API or App API:

curl --request DELETE \
     --url http://<fusion_host>:<port>/api/query-profiles/<profile_id> \
     -u 'user:password' \
     --header 'Accept: application/json' \
     --header 'Content-Type: application/json'

or

curl --request DELETE \
     --url http://<fusion_host>:<port>/api/apps/<app_name>/query-profiles/<profile_id> \
     -u 'user:password' \
     --header 'Accept: application/json' \
     --header 'Content-Type: application/json'

• If both APIs return the same 500 error and query profiles are missing in the UI or Object Explorer, the profile data in ZooKeeper is likely corrupted.

Environment

Fusion 5.9.x and above
Applies to Fusion deployments on Kubernetes (AKS, GKE, or EKS) or self-hosted clusters using ZooKeeper for configuration storage.

Cause

A corrupted query profile node in ZooKeeper can cause the query service to fail when deserializing the profile. The corruption may occur after partial updates, incomplete deployments, or manual edits to stored configuration data.

Resolution

If the Query Profile API fails to delete the corrupted profile, you can manually remove the affected node from ZooKeeper.

Step 1: Access ZooKeeper CLI

Open a shell session on a Fusion pod with access to ZooKeeper and launch the ZooKeeper CLI:

/opt/fusion/zookeeper/bin/zkCli.sh -server <zookeeper-host>:2181

Step 2: Locate the query profile node

List the nodes under the Fusion configuration path to identify the query profile entry:

ls /lucidworks/query-profiles

Locate the node corresponding to the problematic profile ID.

Step 3: Delete the corrupted profile node

Once identified, delete the node directly from ZooKeeper:

delete /lucidworks/query-profiles/<profile_id>

If recursive deletion is required (for nested nodes), use:

rmr /lucidworks/query-profiles/<profile_id>

Note: Exercise caution when performing direct ZooKeeper modifications. Back up ZooKeeper data before making changes.

Step 4: Validate in Fusion UI

After deletion, refresh the Fusion UI and navigate to:
Apps → System → Object Explorer → Query Profiles

All remaining query profiles should now appear correctly, and the Fusion application should resume normal behavior.