Issue
A query pipeline targeting multiple collections fails with a 403 Forbidden error. The error message indicates that a specific shard URL is neither a live node nor in the configured allowUrls list, even if the node appears healthy in the Solr Admin UI.
Example Error String:
The 'shards' parameter value 'null' contained value(s) not allowed: URL http://internal-standard-solrcloud-1... is neither a live node of the cluster nor in the configured 'allowUrls' [].Diagnosis
The query pipeline targets multiple collections
The query request contains the parameter
shards.preference=replica.type:PULL.Inspect the replica types for all collections in the query. Some collections may only contain
NRT(Near Real Time) replicas, while the query is explicitly requestingPULLreplicas.Solr health checks show all collections are "Green" and nodes are active, but distributed search fails during the fan-out phase because it cannot resolve a valid path for the requested replica type across all shards.
Environment
Lucidworks Fusion
SolrCloud
Multi-collection distributed search
Cause
This issue is caused by a conflict between the Shards Preference setting and the physical Replica Type distribution in the Solr cluster.
When shards.preference=replica.type:PULL is used, Solr attempts to route sub-queries to PULL replicas. If one or more collections in a multi-collection query (fan-out) only possess NRT replicas, the routing logic may fail to identify a "live node" that satisfies the constraint, leading to a misleading 403 allowUrls security exception.
Resolution
Align Replica Types
To resolve this, ensure that all collections targeted by the query pipeline have a consistent replica architecture:
Add PULL replicas to the collections that currently only use NRT replicas.
Alternatively, if PULL replicas are not required for performance scaling, change the existing NRT replicas to PULL replicas (if the shard leader is maintained elsewhere).
Adjust Query Preference
If modifying the infrastructure is not an option, adjust the query pipeline parameters:
Remove
shards.preference=replica.type:PULLfrom the query stage.This allows Solr to default to any available replica type (NRT, TLOG, or PULL), resolving the routing failure.