Text Tagger Troubleshooting

The Text Tagger in Fusion's Query Workbench enhances search results by automatically adding relevant tags to documents based on input text. By analysing the content, the Text Tagger identifies keywords and phrases, assigning tags that can help refine search queries and improve the relevance of search results. This functionality is pivotal in creating a more intuitive and relevant search experience for users.

For more detailed insights, you can refer to the documentation here: Text Tagger. 

Issue

Many time customer face issue regarding the text tagger not working or the rules which they added and not getting any effect when they search with those queries. So there are certain aspects where we should look at to check if the things are in place to get this feature work as expected.

Note that the text tagger stage uses Text tagger feature of Apache solr. You can learn about the text tagger handler on the Solr's Document page : Solr Text Tagger Handler 

If the Text Tagger is not functioning as expected, consider the following mentioned troubleshooting steps.

Environment

Fusion 5.5.0 and above

Troubleshooting Steps and Resolution

1. Check Collection Sharding
   

   The underlying SolrTextTagger currently supports only single-shard collections. Ensure that your designated collection (e.g., COLLECTION_NAME_query_rewrite) is single-sharded before enabling the Text Tagger stage. If you find that this collection points to a multi-sharded collection, switch it to a single-sharded collection. You can change the collection name in the text tagger configuration:
   
   
   
   

2. Test with Default Query Pipeline
   

   Use the default query pipeline in the Query Workbench. If adding the Text Tagger stage allows the query you are trying to tag to function correctly, it might indicate an issue with your previous pipeline configuration.
   
   

3. Verify the /tag Handler
   

   Lastly, confirm that the /tag handler is included in the solrconfig and is enabled for the query rewrite collection. This handler is critical and should be present in the query_rewrite collection to ensure proper operation. You may get something similar warning while the Text Tagger is trying to query the handler as bellow:
   

   Request SolrQuery: COLLECTION_NAME_query_rewrite, tag, 
   {q=[*:*], editSessionId=[..], json.nl=[map], 
   fl=[id,surface_form,action,output,boost,slop,type,is_deleted,doc_type,
   triggerdoc_b,removeid_s], overlaps=[ALL], fq=[.., type:(tail),
   doc_type:(query_rewrite), surface_form:([* TO *])]}
   to tagger failed due to: <p>
   
   Searching for Solr?<br/>
   You must type the correct path.<br/>
   Solr will respond.
   </p>

   
   If similar warning is surfacing in the logs and you find that the /tag handler is missing from the rewrite you may want to add handler config manually.
   
   
   <requestHandler name="/tag" class="solr.TaggerRequestHandler">
   <lst name="defaults">
   <int name="tagsLimit">5000</int>
   <str name="field">tagger_text</str>
   <str name="overlaps">LONGEST_DOMINANT_RIGHT</str>
   <str name="wt">json</str>
   </lst>
   </requestHandler>

By following these troubleshooting steps, you can identify common issues that may prevent the Text Tagger from working effectively in your Fusion environment.