Issue
I want to query external collections using Solr SubQuery query-pipeline stage. How do I do that?
Environment
Fusion
This will work with all versions of Fusion, but the specific code may be slightly different.
Description
The Solr SubQuery query-pipeline stage is used to query external collections.
Resolution
The Solr SubQuery query-pipeline stage is used to query external collections. The Solr SubQuery stage runs a query and adds the results to the pipeline Request object. The query must return results in JSON format so that they can be added to the query pipeline.
Add JS stage:
logger.info("Request:" + request.getParams());
logger.info("Response: " + response.initialEntity.toString());
logger.info("Context: " + ctx.toString());
Here, we are trying to leverage the parameter from the Parent Query and pass it as a subquery to signal collection (we can do it with any external collection also) called “manendra_signals” with HTTP GET method.
We can also pass additional parameters based on which we need to query the external collection. The result from the query is added to KEY specified in the stage in this case “subquery-results” in the Fusion ctx object. Details about the Request, Response and CTX objects can be found here:
https://doc.lucidworks.com/fusion/5.7/395/custom-java-script-stages-global-variables
You will have the output something like this (ctx > subquery-results) in api.logs
2023-09-30T08:56:58,415 - INFO [qtp662441761-1182:internal.scripts.Script$Recompilation$11$76AAAAAA$\^eval\_@4] - {collectionId=manendra, requestId=kwJw1O8I9e, stageId=1lu} - Context_Test:{jobId=mjVKY5LXLJ, subquery-results=SearchResponse{response=Response{numFound=496, start=0, maxScore=1.0, docs=[{date_year=2023-09-30T00:00:00.00Z, params_f.id.facet.mincount_s=1, params_echoParams_s=all, response_type=results, params_f.genre_s.facet.missing_s=false, type=response, flag_s=event, platform=fusion, params_wt_s=json, http_method=POST, params_context_s=app:manendra, fusion_query_id=tWu2szlYT5, 2023-09-30T08:56:58,412 - INFO [qtp662441761-1182:internal.scripts.Script$Recompilation$11$76AAAAAA$\^eval\_@4] - {collectionId=manendra, requestId=kwJw1O8I9e, stageId=1lu} - Response_Test:{"debug":{"rawquerystring":"*:*","querystring":"*:*","parsedquery":"+MatchAllDocsQuery(*:*)"," parsedquery_toString":"+*:*","explain":{"0553293354":"\n1.0 = *:*\n"},"facet-debug":{"elapse":0,"sub-facet":[{"processor":"SimpleFacets","elapse":0,"action":"field facet","maxThreads":0,"sub-facet":[{"elapse":0,"requestedMethod":"not specified","appliedMethod":"FC","inputDocSetSize":1,"field":"id","numBuckets":11},{"elapse":0,"requestedMetho d":"not specified","appliedMethod":"FC","inputDocSetSize":1,"field":"genre_s","numBuckets":3}]}]},"QParser":"Extende dDismaxQParser","altquerystring":null,"boost_queries":null,"parsed_boost_queries":[],"boostfuncs":null,"filter_que ries":["id:(\"0553293354\")"],"parsed_filter_queries":["id:0553293354"],"timing":{"time":0.0,"prepare":{"time":0.0, "query":{"time":0.0},"facet":{"time":0.0},"facet_module":{"time":0.0},"mlt":{"time":0.0},"highlight":{"time":0.0}, "stats":{"time":0.0},"expand":{"time":0.0},"terms":{"time":0.0},"debug":{"time":0.0}},"process":{"time":0.0,"quer y":{"time":0.0},"facet":{"time":0.0},"facet_module":{"time":0.0},"mlt":{"time":0.0},"highlight":{"time":0.0},"stats ":{"time":0.0},"expand":{"time":0.0},"terms":{"time":0.0},"debug":{"time":0.0}}}},"response":{"numFound":1,"st art":0,"maxScore":1.0,"docs":[{"name_s":"Foundation","_lw_parser_id_s":"116509dd-80f5-4b0e-9bb6-e352ed47c8 c4","price_s":"7.99","price_t":"7.99","_lw_file_content_type_s":"text/csv","genre_s":"scifi","_lw_data_source_coll ection_s":"manendra","_lw_file_modified":["2023-09-08T09:13:49Z"],
Now, to use the result from the Context object we will need to add a JS stage after this and parse the required elements as needed and then add them to the Fusion response object.
Sample JS code (this varies on user requirement)
function (req, res, ctx)
{
//here we are extracting the sub-query result from the context object
var ctxresponse=ctx.getProperty("subquery-results");
//we are appending the sub query result to the main response object res.initialEntity.appendString("subquery-results", ctxresponse.response); logger.info("responsenewP:" +res.initialEntity.getUnderlyingObject());
}
Note: Custom Javascript is not part of Lucidworks Support break/fix scope and it would be better if handled via Professional Services.
The original response object from Fusion is not affected by the Solr Subquery stage and can only be manipulated via a JS stage.
Reference Link: