> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/avnlp/vectordb/llms.txt
> Use this file to discover all available pages before exploring further.

# Hybrid search

> Combine dense and sparse retrieval with fusion for best-of-both-worlds search

Hybrid search runs both dense and sparse retrieval in parallel, then fuses results using Reciprocal Rank Fusion (RRF) or weighted combination. You get the best of both worlds: semantic understanding for concepts plus keyword precision for specific terms.

## How it works

Hybrid search combines dense semantic embeddings with sparse lexical embeddings for enhanced retrieval quality.

### Search process

1. **Dual embedding** - Query is embedded using both dense and sparse embedders
2. **Parallel retrieval** - Dense embedding captures semantic query intent, sparse embedding captures exact term matches
3. **Score fusion** - Results are fused using RRF or weighted scoring: `alpha * dense_score + (1-alpha) * sparse_score`
4. **Ranked results** - Documents are ranked by the fused score

### Fusion mechanisms

**Reciprocal Rank Fusion (RRF)**

RRF combines rankings from multiple sources using the formula:

```
score(d) = Σ 1/(k + rank_i(d))
```

Where k is a constant (typically 60) and rank\_i is the rank from source i. RRF is robust to score miscalibration between sources since it uses ranks rather than raw scores.

**Alpha weighting**

For databases with native hybrid support (like Pinecone), alpha controls the balance between dense and sparse:

* `alpha = 1.0` - Pure dense/semantic search
* `alpha = 0.0` - Pure sparse/keyword search
* `alpha = 0.5` - Balanced hybrid (default)

## Key features

* Dense + sparse fusion with configurable weights
* RRF handles score normalization automatically
* Built-in evaluation metrics: Recall\@k, MRR, NDCG, Precision\@k
* No FastEmbed dependency—uses native SentenceTransformers sparse encoders

## Implementation

<CodeGroup>
  ```python Haystack theme={null}
  from vectordb.haystack.hybrid_indexing import MilvusHybridSearchPipeline

  pipeline = MilvusHybridSearchPipeline(
      "src/vectordb/haystack/hybrid_indexing/configs/milvus_triviaqa.yaml"
  )
  result = pipeline.run(query="machine learning algorithms", top_k=10)

  for doc in result["documents"]:
      print(f"Score: {doc.score:.3f} | {doc.content[:100]}...")
  ```

  ```python LangChain theme={null}
  from vectordb.langchain.hybrid_indexing import PineconeHybridSearchPipeline

  pipeline = PineconeHybridSearchPipeline("config.yaml")
  results = pipeline.search(
      query="neural network architectures",
      top_k=10,
      filters={"category": "technology"},
  )

  print(f"Found {len(results['documents'])} documents")
  for doc in results["documents"]:
      print(doc.page_content[:100])
  ```
</CodeGroup>

## Configuration

### Required settings

<ParamField path="pinecone.api_key" type="string" required>
  Pinecone API authentication key
</ParamField>

<ParamField path="pinecone.index_name" type="string" required>
  Target index name for hybrid search
</ParamField>

### Optional settings

<ParamField path="pinecone.alpha" type="float" default={0.5}>
  Fusion weight (0.0=sparse only, 1.0=dense only, 0.5=balanced hybrid)
</ParamField>

<ParamField path="pinecone.namespace" type="string">
  Namespace within the index for document isolation
</ParamField>

<ParamField path="embedder" type="object">
  Dense embedder configuration for semantic vector generation
</ParamField>

<ParamField path="sparse" type="object">
  Sparse embedder configuration for lexical vectors
</ParamField>

### Example configuration

```yaml theme={null}
pinecone:
  api_key: "${PINECONE_API_KEY}"
  index_name: "hybrid-search"
  namespace: "default"
  alpha: 0.7  # Favor semantic over keyword

embeddings:
  model: "sentence-transformers/all-MiniLM-L6-v2"

sparse:
  enabled: true
```

## Search parameters

<ParamField path="query" type="string" required>
  Search query text to embed with both dense and sparse embedders
</ParamField>

<ParamField path="top_k" type="integer" default={10}>
  Maximum number of results to return
</ParamField>

<ParamField path="filters" type="dict">
  Optional metadata filters for pre-filtering candidates
</ParamField>

## When to use hybrid search

Hybrid search excels when you need both:

**Semantic understanding** (dense)

* Understanding query intent
* Matching synonyms and paraphrases
* Conceptual similarity

**Exact term matching** (sparse)

* Product SKUs or model numbers
* Technical specifications
* Legal or medical terminology
* Proper nouns and acronyms

### Example scenarios

```python theme={null}
# Technical query benefiting from hybrid approach
results = pipeline.search(
    query="GDPR compliance requirements for ML models",
    top_k=10,
)
# Dense: understands "compliance" and "requirements"
# Sparse: exact match on "GDPR" and "ML"

# Product search with specific identifiers
results = pipeline.search(
    query="lightweight running shoes model XR-2024",
    top_k=5,
)
# Dense: understands "lightweight running shoes"
# Sparse: exact match on "XR-2024"
```

## Sparse vector format

Sparse embeddings use SPLADE models to create sparse vectors that emphasize specific terms, similar to traditional BM25 but with learned term importance.

Sparse embeddings are represented as dictionaries mapping token indices to importance weights:

```python theme={null}
{
  "indices": [42, 156, 892, 1024],
  "values": [0.85, 0.62, 0.43, 0.31]
}
```

## Database-specific implementations

### Pinecone

Pinecone requires separate `sparse_values` field for sparse embeddings, distinct from the standard `values` field used for dense vectors. Native fusion with alpha parameter.

### Weaviate

Weaviate uses native BM25 without external embeddings for the sparse component. Hybrid search with configurable fusion weights.

### Milvus

Milvus supports hybrid search with partition-based isolation and configurable fusion strategies.

### Qdrant

Qdrant uses payload-based filtering with optimized indexes for hybrid retrieval.

### Chroma

Chroma provides flexible hybrid search with tenant and database scoping.

## Fusion strategy comparison

| Strategy        | Best for                            | Parameters         |
| --------------- | ----------------------------------- | ------------------ |
| RRF             | Default choice, score normalization | k (default: 60)    |
| Alpha weighting | Known source reliability            | alpha (0.0-1.0)    |
| Weighted merge  | Prior knowledge of source quality   | weights per source |

## Performance tips

<Tip>
  Start with `alpha=0.5` (balanced) and tune based on your evaluation metrics. For semantic-heavy queries, increase to 0.7-0.8. For keyword-heavy queries, decrease to 0.3-0.4.
</Tip>

* Over-fetch candidates (2-3x top\_k) before fusion for better quality
* Use metadata filters to reduce search space before hybrid scoring
* Cache sparse embeddings for repeated queries to reduce latency
* Monitor RRF k parameter impact on result diversity

## Related features

<CardGroup cols={2}>
  <Card title="Semantic search" icon="magnifying-glass" href="/features/semantic-search">
    Dense vector similarity search
  </Card>

  <Card title="Sparse search" icon="hashtag" href="/features/sparse-search">
    Keyword/lexical matching with SPLADE/BM25
  </Card>

  <Card title="Reranking" icon="arrow-down-1-9" href="/features/reranking">
    Cross-encoder second-stage scoring
  </Card>

  <Card title="Diversity filtering" icon="filter" href="/features/diversity-filtering">
    Post-retrieval redundancy reduction
  </Card>
</CardGroup>