Apply conditional recency-based boosting to specific datasources in a Fusion query pipeline

Goal

Apply a recency-based boost function to documents from a specific datasource in a single Fusion query pipeline response, allowing recent documents within that datasource to rank higher among their peers without affecting the ranking of documents from other datasources.

Environment

• Any Fusion version, managed or self-hosted.  Examples from 4.2.6 Fusion SP3 and Solr 7.7.  Specific functions may be different in later releases.

• Query pipeline response including documents from multiple datasources.

Guide

Understanding the requirement

In scenarios where a query pipeline including documents from multiple datasources, it may be necessary to apply recency-based ranking only to a specific datasource. For example, documents from one datasource should be boosted if they are recent, while documents from other datasources remain unaffected by recency.

Configuring the conditional boost function

This can be achieved by using Solr’s boost parameter and function query syntax within Fusion’s Additional Parameters stage. The core approach is to use the if and termfreq Solr functions to conditionally apply a recency-based function only to the desired datasource.

Links to Documentation:

• Solr functions - https://solr.apache.org/guide/7_7/function-queries.html
• Solr boost parameter - https://solr.apache.org/guide/7_7/the-extended-dismax-query-parser.html#extended-dismax-parameters
• Fusion Additional Query Parameters stage - https://doc.lucidworks.com/docs/5.9/fusion/reference/config-ref/pipeline-stages/query-stages/set-query-params-stage

Example boost function syntax

In this example, we want to boost documents from a specific datasource (DS1) while also factoring in document recency using a date field (created_date).

To apply a recency-based boost only to documents where _lw_data_source_s is DS1, add the following boost expression in the Additional Parameters stage of the pipeline:

boost=if(termfreq(_lw_data_source_s,'DS1'),
         recip(ms(NOW,created_date),3.16e-11,2,1),
         1)

Explanation:

• termfreq(_lw_data_source_s,'DS1')
  Ensures the boost is applied only to documents originating from the specified datasource.

• recip(ms(NOW,created_date),3.16e-11,2,1)
  Applies a recency-based boost using the document’s date field, giving higher scores to more recent documents.

• 1
  Documents from other datasources receive a neutral boost, leaving their scores unchanged.

Tuning the boost function

You may need to tune the parameters of the recip function to strike the right balance between document recency and overall relevance.

In some cases, you may want to ensure that older documents are not penalized with a reduced score. The following alternative formulations can help achieve that behavior:

sum(recip(ms(NOW,created_date),3.16e-11,1,1),1)

This approach adds the recency boost on top of the base score, ensuring that older documents do not receive a score lower than the original.

max(recip(ms(NOW,created_date),3.16e-11,2,1),1)

This formulation enforces a minimum score of 1, preventing older documents from being negatively impacted by the recency calculation.

Best Practices

• Experiment with different recip parameters to control how quickly the recency boost decays over time.

• Validate the behavior by running queries restricted to a single datasource before enabling the boost in production.

• Monitor ranking changes to ensure relevance signals are balanced appropriately.

Implementing in Fusion

1. Open the desired query pipeline in Fusion.

2. Add or edit an Additional Parameters stage.

3. Set the boost parameter with the chosen function query.

4. Save and test your pipeline to observe the effect on search result ordering.

Example configuration

boost=if(termfreq(_lw_data_source_s,'DS1'),
         recip(ms(NOW,created_date),3.16e-11,2,1),
         1)

• This applies a time-decay boost only to documents from DS1. All other documents are scored as usual.

Considerations

• The impact of the boost is multiplicative; scores from the targeted datasource will be boosted relative to others.

• If more complex conditions are required (such as boosting only when certain related documents are present), custom indexing logic may be needed to add marker fields for conditional boosting.

• Always verify the field names match your schema and that your date field (e.g., created_date) is correctly formatted for Solr function queries.

Notes

Note: Boost functions can significantly alter result ordering. Careful testing and tuning are recommended before deploying to production environments.

Note: These instructions apply to Fusion 4.x on Solr 7.x. For later Fusion or Solr versions, consult the latest function query and pipeline documentation for any changes.