Configuring Fusion for multi-word suggestions – Lucidworks

Issue:

How can i setup multi-word AutoSuggest in Fusion 3.1.2?

Environment:

Fusion 3.1.2

Resolution:

This will be done in 2 parts:
Configuring solr to dish out suggestions
Configuring a Fusion pipeline to retrieve suggestions from solr

1) Configuring Solr to dish out suggestions:
First, we need to ensure that the suggester component is configured in the solrconfig.xml file.
Configuring it is a simple matter of choosing your collection in Fusion, navigating to the Solr configs and editing the solrconfig.xml file.

This should be something of the type:
<searchComponent name="suggest" class="solr.SuggestComponent">
<lst name="suggester">
<str name="name">mySuggester</str>
<str name="lookupImpl">AnalyzingInfixLookupFactory</str>
<str name="dictionaryImpl">DocumentDictionaryFactory</str>
<str name="field">title_t</str>
<str name="suggestAnalyzerFieldType">text_general</str>
<str name="buildOnStartup">false</str>
</lst>
</searchComponent>

Now for multi-word suggestions, you want to ensure that the lookupImpl parameter is set to AnalyzingInfixLookupFactory. This not only gives multi-word suggestions but also provides prefix suggestions, helpful when your customers are expecting suggestions not only for phrases beginning with the keyword they entered but also for phrases that have the entered keywords in the middle.

The default Dictionary configured with this will be the DocumentDicitionaryFactory and should be the best option in almost all cases.
It enables context filtering while paired with the AnalyzingInfixLookupFactory or the BlendedInfixLookupFactory.

Now just add a request handler to utilize the suggester component and return suggestions.
This could be something like:
<requestHandler name="/suggest" class="solr.SearchHandler"
startup="lazy" >
<lst name="defaults">
<str name="suggest">true</str>
<str name="suggest.count">10</str>
<str name="suggest.dictionary">mySuggester</str>
</lst>
<arr name="components">
<str>suggest</str>
</arr>
</requestHandler>

Be sure to include the suggest.dictionary parameter either here on in your query without which Solr will throw an error saying that there were no dictionaries configured.
Now save this.

2) Configuring a Fusion pipeline to retrieve suggestions from solr:
This is quite simple as well. From your collection home, simply select query-pipelines.
We need to make a small change to the Solr Query stage in order to use our newly configured suggest handler.
So choose your pipeline, choose the solr query stage, and in the AllowedRequestHandlers add the suggest handler and save. By default, only select is available.

Now, to test it, assuming that you set the buildOnStartup parameter to false in your configuration, you’ll first have to build the dictionary before you start using it.
To do this, simply append “&suggest.build=true” to your very first query to the /suggest handler and the dictionary will be built. This is not required for subsequent queries unless your autosuggest index has changed. Do note, that building the dictionary is an expensive operation and could take a significant amount of time (worst I’ve seen is over 90s) in cases where you’ve got millions of terms in the suggest field. This could have a more than significant impact on the performance of your queries. You may also want to keep from setting the buildOnStartup parameter to true as this will cause the same slowdown when you bring your solr node up because the core won’t be active until the dictionary has been built.

Here’s an example request:
http://localhost:8764/api/apollo/query-pipelines/default/collections/default/suggest?q=fusion%20potential&wt=json&suggest.build=true

Here I have an index of street names I want to suggest on. For the first request, the suggest.build parameter has been set to true to build the index.

Example response:
{
"responseHeader": {
"zkConnected": true,
"status": 0,
"QTime": 32
},
"suggest": {
"mySuggester": {
"fusion potential": {
"numFound": 10,
"suggestions": [
{
"term": "Signature of Shallow Potentials in Deep Sub-barrier Fusion Reactions",
"weight": 0,
"payload": ""
},
{
"term": "Assessing the adequacy of the bare optical potential in near-barrier\n fusion calculation",
"weight": 0,
"payload": ""
},
{
"term": "Extended Optical Model Analyses of Elastic Scattering and Fusion Cross\n Section Data for the 7Li+208Pb System at Near-Coulomb-Barrier Energies using\n the Folding Potential",
"weight": 0,
"payload": ""
},
{
"term": "Fusion-fission reactions with modified Woods-Saxon potential",
"weight": 0,
"payload": ""
},
{
"term": "Surface diffuseness anomaly in heavy-ion fusion potentials",
"weight": 0,
"payload": ""
},
{
"term": "Dynamical collective potential energy landscape: its impact on the\n competition between fusion and quasi-fission in a heavy fusing system",
"weight": 0,
"payload": ""
},
{
"term": "Microscopic DC-TDHF study of heavy-ion potentials and fusion cross\n sections",
"weight": 0,
"payload": ""
},
{
"term": "The structure of minimizers of the frame potential on fusion frames",
"weight": 0,
"payload": ""
},
{
"term": "Effects of mass renormalization on the surface properties of heavy-ion\n fusion potential",
"weight": 0,
"payload": ""
},
{
"term": "Adiabatic Heavy Ion Fusion Potentials for Fusion at Deep Sub-barrier\n Energies",
"weight": 0,
"payload": ""
}
]
}
}
}
}

You can see the suggested term in the term field in suggestions.

The number of suggestions will vary based on the query and the settings you’ve applied.

You can include a suggestion response as part of your normal response just as easily (for a did you mean? Use case).
In this case, configure the search component normally but instead of creating a new request handler for the suggester, you can just include the component in a pre-existing handler you have already configured.
There are many more nuances and details about Suggesters that I have not covered in this article and they can be found in the very well written solr guide here: https://lucene.apache.org/solr/guide/6_6/suggester.html

Additionally, if you’re brave and want to dig into the internal workings of the suggester then check out this blog by Mike McCandless:
http://blog.mikemccandless.com/2013/06/a-new-lucene-suggester-based-on-infix.html

Related articles