Spelling suggestions not appearing for misspelled query terms

Issue

Spelling suggestions are not returned for certain misspelled query terms in a Fusion collection, even though the spellcheck component is configured and enabled.

Diagnosis

Verify that spellcheck is working, and is using the DirectSolrSpellChecker  implementation.  Documentation - https://solr.apache.org/guide/7_7/spell-checking.html#directsolrspellchecker 

To confirm that the issue is related to the thresholdTokenFrequency value, test a misspelled query using the /spell handler or via a spellcheck.q parameter. If no suggestions are returned, verify the indexed frequency of the expected suggestion term. If the document frequency of the suggestion term is below the configured thresholdTokenFrequency value, suggestions will be suppressed.

For example, if the term "keyboard" exists in only 138 documents out of 240,000 in the collection, and thresholdTokenFrequency is set to 0.01 (1%), then it will not be considered a viable suggestion.

Environment

Fusion (Any)
Solr component: DirectSolrSpellChecker  configured in solrconfig.xml
Applicable to Fusion deployments using the spellcheck component via managed search collections

Cause

The thresholdTokenFrequency parameter defines the minimum frequency a suggested word must meet in the index to be considered a viable correction. When set as a percentage, terms below that frequency are excluded from results. This can cause valid but infrequent words to be ignored by the spellcheck component.

Resolution

To return suggestions for infrequently used but valid words, reduce the value of the thresholdTokenFrequency parameter in the solrconfig.xml file of the collection using the relevant spellcheck component.

Example configuration:

<lst name="spellchecker">
  <str name="name">spellcheck_en_GB</str>
  <str name="field">spelling_en_GB</str>
  <str name="classname">solr.DirectSolrSpellChecker</str>
  <str name="distanceMeasure">internal</str>
  <str name="accuracy">0.5</str>
  <str name="maxEdits">2</str>
  <str name="minPrefix">1</str>
  <str name="maxInspections">5</str>
  <str name="minQueryLength">4</str>
  <str name="maxQueryFrequency">0.01</str>
  <str name="thresholdTokenFrequency">10</str> <!-- Use an absolute value -->
  <str name="onlyMorePopular">true</str>
</lst>

Guidance

• If using a percentage-based threshold, ensure that the document frequency of expected suggestions exceeds that percentage.

• Alternatively, set an absolute value for thresholdTokenFrequency (e.g., 10) to allow suggestions with lower frequency.

• After making changes, reload the Solr core or restart the Fusion collection to apply updates.

• Validate the behavior using the /spell handler or query pipeline endpoint with the spellcheck.q parameter.

Note: Always test thoroughly before deploying changes to production to ensure that suggestions are relevant and accurate.