Goal
How can I setup multi-word AutoSuggest in Fusion?
Environment
Fusion 3.1.2 and above
Guide
To setup multi-word AutoSuggest in Fusion we need to make some changes in solrconfig.xml where we need to add suggester, please verify if you have already suggester defined inside your solrconfig, if there you can update the existing one.
This will be done in 3 parts:
- Configure solr to dish out suggestions
- Configure a Fusion pipeline to retrieve suggestions from solr
- Create new query-profile
Configure Solr to dish out suggestions
First, we need to ensure that the suggester component is configured in the solrconfig.xml file.
Configuring it is a simple matter of choosing your collection in Fusion, navigating to the Solr configs and editing the solrconfig.xml file.
This should be something of the type:
<lstname="suggester">
<strname="name">mySuggester</str>
<strname="lookupImpl">AnalyzingInfixLookupFactory</str>
<strname="dictionaryImpl">DocumentDictionaryFactory</str>
<strname="field">title_t</str>
<strname="suggestAnalyzerFieldType">text_general</str>
<strname="buildOnStartup">false</str>
</lst>
</searchComponent>
Now for multi-word suggestions, you want to ensure that the lookupImpl parameter is set to AnalyzingInfixLookupFactory. This not only gives multi-word suggestions but also provides prefix suggestions, helpful when your customers are expecting suggestions not only for phrases beginning with the keyword they entered but also for phrases that have the entered keywords in the middle.
The default Dictionary configured with this will be the DocumentDicitionaryFactory and should be the best option in almost all cases.
It enables context filtering while paired with the AnalyzingInfixLookupFactory or the BlendedInfixLookupFactory.
Now just add a request handler to utilize the suggester component and return suggestions.
This could be something like:
Be sure to include the suggest.dictionary parameter either here on in your query without which Solr will throw an error saying that there were no dictionaries configured.
Now save this.
Configure a Fusion pipeline to retrieve suggestions from solr:
This is quite simple as well. From your collection home, simply select query-pipelines.
We need to make a small change to the Solr Query stage in order to use our newly configured suggest handler.
So choose your pipeline, choose the solr query stage, and in the AllowedRequestHandlers add the suggest handler and save. By default, only select is available.
Now, to test it, assuming that you set the buildOnStartup parameter to false in your configuration, you’ll first have to build the dictionary before you start using it.
To do this, simply append “&suggest.build=true” to your very first query to the /suggest handler and the dictionary will be built. This is not required for subsequent queries unless your autosuggest index has changed. Do note, that building the dictionary is an expensive operation and could take a significant amount of time (worst I’ve seen is over 90s) in cases where you’ve got millions of terms in the suggest field. This could have a more than significant impact on the performance of your queries. You may also want to keep from setting the buildOnStartup parameter to true as this will cause the same slowdown when you bring your solr node up because the core won’t be active until the dictionary has been built.
Create new query-profile:
Here’s an example request:
http://localhost:8764/api/apps/APP_NAME/query/QUERY_PROFILE_ID?suggest.build=true
Here I have an index of lucidworks I want to suggest on. For the first request, the suggest.build parameter has been set to true to build the index as stated above.
You can execute query like below to test it:
http://localhost:8764/api/apps/APP_NAME/query/QUERY_PROFILE_ID?suggest.q=Lucidworks%20fusion
Example response:
<lst name="responseHeader">
<bool name="zkConnected">true</bool>
<int name="status">0</int>
<int name="QTime">36</int>
<int name="totalTime">99</int>
</lst>
<lst name="suggest">
<lst name="mySuggester">
<lst name="Lucidworks fusion">
<int name="numFound">2</int>
<arr name="suggestions">
<lst>
<str name="term">Why
<b>Lucidworks</b>
<b>Fusion</b>
and ChatGPT? |
<b>Lucidworks</b></str>
<long name="weight">0</long>
<str name="payload"/>
</lst>
<lst>
<str name="term">
<b>Lucidworks</b>
<b>Fusion</b>| Site Search Platform |
<b>Lucidworks</b></str>
<long name="weight">0</long>
<str name="payload"/>
</lst>
</arr>
</lst>
</lst>
</lst>
</response>
You can see the suggested term in the term field in suggestions.
The number of suggestions will vary based on the query and the settings you’ve applied.
You can include a suggestion response as part of your normal response just as easily (for a did you mean? Use case). In this case, configure the search component normally but instead of creating a new request handler for the suggester, you can just include the component in a pre-existing handler you have already configured.
There are many more nuances and details about Suggesters not covered in this article and they can be found in the very well written solr guide here: https://lucene.apache.org/solr/guide/6_6/suggester.html
Additionally, if you’re brave and want to dig into the internal workings of the suggester then check out this blog by Mike McCandless:
http://blog.mikemccandless.com/2013/06/a-new-lucene-suggester-based-on-infix.html