> ## Documentation Index > Fetch the complete documentation index at: https://mintlify.com/avnlp/vectordb/llms.txt > Use this file to discover all available pages before exploring further. # Haystack integration overview > Learn about Haystack-based retrieval and RAG pipeline implementations for vector databases This module provides Haystack-based retrieval and RAG pipeline implementations across all five supported vector database backends. Every feature is organized as a self-contained directory with configuration files, indexing scripts, and search scripts for each backend. ## What you get * Seventeen retrieval and RAG patterns implemented using Haystack's pipeline and component abstractions * Full portability across Pinecone, Weaviate, Chroma, Milvus, and Qdrant (with feature-specific notes on backend support) * YAML-driven configuration with environment variable substitution so credentials stay out of code * Evaluation support via shared `utils/evaluation.py` metrics * Shared reusable components and helper factories that all feature pipelines draw from ## Module structure Each feature directory follows the same layout: ``` feature_name/ ├── configs/ │ ├── chroma_triviaqa.yaml │ ├── milvus_triviaqa.yaml │ ├── pinecone_triviaqa.yaml │ ├── qdrant_triviaqa.yaml │ ├── weaviate_triviaqa.yaml │ └── (one config per backend × dataset combination) ├── indexing/ │ ├── chroma.py │ ├── milvus.py │ ├── pinecone.py │ ├── qdrant.py │ └── weaviate.py ├── search/ │ ├── chroma.py │ ├── milvus.py │ ├── pinecone.py │ ├── qdrant.py │ └── weaviate.py └── README.md ``` ## Feature catalog Dense vector similarity search - the baseline pattern Combines dense + sparse embeddings with RRF fusion Reusable pipeline components for routing, compression, and query enhancement Pipeline architecture patterns and composition ### Core retrieval patterns | Feature | Description | Use when | | ------------------------ | -------------------------------------- | ------------------------------------------ | | **semantic\_search** | Dense vector similarity search | Starting point and baseline | | **hybrid\_indexing** | Dense + sparse embeddings with RRF | Both semantic and keyword precision needed | | **sparse\_indexing** | Sparse-only retrieval | Pure keyword/lexical precision | | **reranking** | Two-stage retrieval with cross-encoder | Better final ranking needed | | **mmr** | Maximal Marginal Relevance | Relevant + diverse results | | **diversity\_filtering** | Similarity count filtering | Less redundant context | ### Advanced retrieval | Feature | Description | Use when | | ------------------------------- | -------------------------------------------- | ------------------------------------ | | **metadata\_filtering** | Structured constraints | Need to filter by attributes | | **json\_indexing** | JSON-native documents | Structured fields + semantic content | | **query\_enhancement** | Multi-query, HyDE, step-back | Better query recall | | **contextual\_compression** | Abstractive, extractive, relevance filtering | Shorter, cleaner context | | **parent\_document\_retrieval** | Index children, retrieve parents | Long docs with fragment search | ### Production features | Feature | Description | Use when | | ------------------------ | --------------------------------------------- | --------------------------- | | **cost\_optimized\_rag** | Token/compute budget control | Cost reduction needed | | **agentic\_rag** | Multi-step iterative RAG with self-reflection | Complex multi-hop questions | | **multi\_tenancy** | Tenant-scoped isolation | Per-customer data isolation | | **namespaces** | Logical segmentation | Environment separation | ## Embedding configuration All Haystack feature pipelines read embedding configuration from the YAML config: ```yaml theme={null} embeddings: model: "sentence-transformers/all-MiniLM-L6-v2" # Required device: "cpu" # Optional batch_size: 32 # Optional ``` For hybrid and sparse features: ```yaml theme={null} sparse: model: "naver/splade-cocondenser-ensembledistil" ``` ## RAG configuration Generation is controlled by the `rag` section: ```yaml theme={null} rag: enabled: true model: "llama-3.3-70b-versatile" api_key: "${GROQ_API_KEY}" api_base_url: "https://api.groq.com/openai/v1" temperature: 0.7 max_tokens: 2048 ``` Set `enabled: false` to run retrieval-only pipelines without generation. ## Recommended onboarding path Start with `semantic_search` on your target backend with a small dataset limit (100-200 records) and verify the pipeline loads, indexes, and retrieves successfully. Use `evaluation_queries()` and `evaluate_retrieval()` to establish baseline metrics. Add one improvement feature at a time (e.g., `reranking` or `hybrid_indexing`) and measure whether quality improves on your evaluation set. Once the retrieval baseline is strong, adopt `multi_tenancy` or `namespaces` for data isolation, and `cost_optimized_rag` for budget controls. Use `agentic_rag` or `query_enhancement` for hard multi-hop questions where single-pass retrieval falls short. ## Supported backends * **Chroma**: Local embedded vector database with SQLite persistence * **Milvus**: High-performance distributed vector database * **Pinecone**: Managed vector database with native sparse vector support * **Qdrant**: Vector database with advanced filtering and multitenancy * **Weaviate**: GraphQL-based vector database with semantic search Each backend has consistent API patterns through Haystack's abstraction layer. ## Next steps Start with the baseline semantic search pattern Learn about dense + sparse hybrid retrieval Explore reusable pipeline components Understand pipeline architecture patterns