> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/avnlp/vectordb/llms.txt
> Use this file to discover all available pages before exploring further.

# Metadata filtering

> Filter search results using structured document attributes

Metadata filtering applies structured constraints to vector search results, enabling you to narrow results by fields like category, date, author, or custom attributes while still ranking by semantic relevance.

## Overview

Vector search finds semantically similar content but cannot filter by structured properties. Metadata filtering adds boolean predicates (equals, range, contains) to retrieve precisely targeted documents.

<Info>
  Metadata filtering is essential for production RAG applications when you need queries like "find documents about ML published in 2024" or "retrieve technical articles by author X".
</Info>

## How it works

The metadata filtering pipeline follows these steps:

1. **Query embedding** - Convert query text to vector representation
2. **Filtered vector search** - Execute similarity search with metadata constraints
3. **Post-processing** - Apply additional client-side filters if configured
4. **RAG generation** - Generate answer using filtered documents (optional)

## Supported operators

VectorDB supports the following filter operators across all databases:

| Operator     | Description                        | Example                            |
| ------------ | ---------------------------------- | ---------------------------------- |
| `equals`     | Exact match                        | `category = "electronics"`         |
| `not_equals` | Not equal                          | `status != "archived"`             |
| `gt`         | Greater than                       | `price > 100`                      |
| `gte`        | Greater than or equal              | `date >= "2024-01-01"`             |
| `lt`         | Less than                          | `score < 0.5`                      |
| `lte`        | Less than or equal                 | `rating <= 4.5`                    |
| `in`         | Value in list                      | `category in ["tech", "science"]`  |
| `not_in`     | Value not in list                  | `author not in ["user1", "user2"]` |
| `contains`   | Substring match (case-insensitive) | `title contains "machine"`         |
| `startswith` | Prefix match (case-insensitive)    | `name startswith "Dr"`             |
| `endswith`   | Suffix match (case-insensitive)    | `filename endswith ".pdf"`         |

<Note>
  String operators (`contains`, `startswith`, `endswith`) are case-insensitive for consistent behavior across databases.
</Note>

## Database-specific syntax

Each database uses its own native filter format:

<CodeGroup>
  ```python Pinecone theme={null}
  # Pinecone uses JSON filter syntax
  filters = {
      "$and": [
          {"category": {"$eq": "technical"}},
          {"date": {"$gte": "2024-01-01"}}
      ]
  }
  ```

  ```python Weaviate theme={null}
  # Weaviate uses GraphQL-style filters
  from weaviate.classes.query import Filter

  filters = Filter.by_property("category").equal("technical") & \
            Filter.by_property("date").greater_or_equal("2024-01-01")
  ```

  ```python Qdrant theme={null}
  # Qdrant uses structured filter objects
  from qdrant_client.models import Filter, FieldCondition, MatchValue

  filters = Filter(
      must=[
          FieldCondition(key="category", match=MatchValue(value="technical")),
          FieldCondition(key="date", range={"gte": "2024-01-01"})
      ]
  )
  ```

  ```python Milvus theme={null}
  # Milvus uses SQL-like expressions
  filters = 'category == "technical" and date >= "2024-01-01"'
  ```

  ```python Chroma theme={null}
  # Chroma uses where clause dictionaries
  filters = {
      "$and": [
          {"category": {"$eq": "technical"}},
          {"date": {"$gte": "2024-01-01"}}
      ]
  }
  ```
</CodeGroup>

## Configuration

Define metadata filters in your pipeline configuration:

```yaml theme={null}
filters:
  conditions:
    - field: "category"
      value: "technical"
      operator: "equals"
    - field: "price"
      value: 500
      operator: "lt"
```

## Usage example

<CodeGroup>
  ```python LangChain theme={null}
  from vectordb.langchain.metadata_filtering.search.pinecone import (
      PineconeMetadataFilteringSearchPipeline,
  )

  pipeline = PineconeMetadataFilteringSearchPipeline("config.yaml")

  # Search with filters
  results = pipeline.search(
      "machine learning frameworks",
      top_k=10,
      filters={"category": {"$eq": "technical"}},
  )

  print(f"Found {len(results['documents'])} documents")
  for doc in results["documents"]:
      print(f"- {doc.metadata['title']} (category: {doc.metadata['category']})")
  ```

  ```python Haystack theme={null}
  from vectordb.haystack.metadata_filtering.search.pinecone import (
      PineconeMetadataFilteringSearchPipeline,
  )

  pipeline = PineconeMetadataFilteringSearchPipeline("config.yaml")

  results = pipeline.run(
      query="machine learning frameworks",
      top_k=10,
  )

  for doc in results["documents"]:
      print(doc.content[:100])
  ```
</CodeGroup>

## Performance optimization

### Selectivity analysis

Filter order matters for query performance. VectorDB includes selectivity analysis to optimize filter execution:

```python theme={null}
# High selectivity filters (fewer matches) should run first
filters = [
    {"user_id": "specific-user"},  # High selectivity
    {"category": "news"},          # Lower selectivity
]
```

### Pre-filter vs post-filter

Databases apply filters at different stages:

* **Pre-filter** - Filter before vector search (faster, smaller search space)
* **Post-filter** - Filter after vector search (preserves ranking quality)

<Tip>
  Use pre-filtering for highly selective filters (user\_id, tenant\_id) and post-filtering for broader criteria (category, date ranges).
</Tip>

## Timing metrics

Track filter performance with built-in timing metrics:

```python theme={null}
results = pipeline.search(query, filters=filters)

print(f"Filter time: {results['metrics']['filter_time_ms']}ms")
print(f"Search time: {results['metrics']['search_time_ms']}ms")
print(f"Total time: {results['metrics']['total_time_ms']}ms")
```

## Related features

<CardGroup cols={2}>
  <Card title="JSON indexing" icon="code" href="/data/json-indexing">
    Filter by nested JSON paths
  </Card>

  <Card title="Namespaces" icon="folder" href="/data/namespaces">
    Logical data partitioning
  </Card>

  <Card title="Multi-tenancy" icon="users" href="/data/multi-tenancy">
    Tenant-isolated retrieval
  </Card>

  <Card title="Semantic search" icon="magnifying-glass" href="/features/semantic-search">
    Vector similarity search
  </Card>
</CardGroup>