Advanced RAG with Hybrid Search

Build a RAG pipeline with BM25, vector search, RRF, and reranking.

Introduction

30 Second Summary

When you ask an AI chatbot a specific question about your documents, have you ever noticed it sometimes pulls back vague or irrelevant information? That happens because basic retrieval systems only look at one dimension of similarity, missing results that a smarter approach would catch.

In this project, you will build an advanced RAG pipeline that layers three powerful retrieval techniques on top of basic vector search: hybrid search, cross-encoder reranking, and multi-query expansion.

What You'll Build

A complete advanced RAG system powered by NVIDIA NIM that answers questions over a set of documents using three layered retrieval techniques, each progressively improving retrieval quality.

By the end of this project, you'll have:

  • A hybrid retriever combining BM25 keyword search with NVIDIA dense vector search, merged using Reciprocal Rank Fusion.
  • A reranking layer using NVIDIA's cross-encoder model that dramatically improves retrieval precision.
  • A comparison script that runs the same queries through basic and advanced pipelines, printing results side-by-side so you can see the difference.
  • Secret Mission: Add retrieval evaluation metrics to quantify exactly how much better your advanced pipeline performs.

Are there any prerequisites?

This project assumes you have a basic understanding of RAG (loader, chunker, vector store, retrieval chain) and Python fundamentals.

You'll also need an NVIDIA API key from the NVIDIA API Catalog. The free tier is all you need for this project.

Before We Start

To build an advanced RAG pipeline, you need a working development environment with the right libraries and credentials configured.

In this step, you'll install several Python packages that each handle a different part of the pipeline. These include NVIDIA NIM endpoints for embeddings and reranking, a BM25 implementation for keyword search, and FAISS for vector storage.

In this step, get ready to:
  • Install the Python packages for embeddings, keyword search, vector storage, and reranking.
  • Configure your NVIDIA API key in a .env file.
  • Verify your connection to the NVIDIA API.
Create your project folder and install packages

You need a dedicated folder for your project files and a set of Python libraries that power each retrieval technique.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir advanced-rag-pipeline
cd advanced-rag-pipeline
  • Confirm the folder exists by running:
pwd

You should see a path ending in advanced-rag-pipeline in the output.

  • Install the project dependencies by running:
pip install langchain-nvidia-ai-endpoints rank_bm25 faiss-cpu langchain langchain-community python-dotenv

What does this command install?

  • langchain-nvidia-ai-endpoints provides ChatNVIDIA, NVIDIAEmbeddings, and NVIDIARerank classes for accessing NVIDIA NIM models.
  • rank_bm25 implements the BM25 keyword search algorithm in pure Python.
  • faiss-cpu is Facebook's library for fast vector similarity search.
  • langchain and langchain-community provide the text splitter and FAISS integration.
  • python-dotenv loads environment variables from your .env file.

You should see Successfully installed followed by a list of package names at the end of the output.

Installation failed?

  • Make sure you're connected to the internet.
  • If you see a permissions error, try running pip install --user followed by the same package list.
  • If pip is not found, try pip3 instead.

help me fix my pip install error for langchain-nvidia-ai-endpoints

Configure your NVIDIA API key

Your pipeline needs an API key to authenticate with NVIDIA's hosted models for embeddings, reranking, and text generation.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code (or Cursor) and press Enter to open your IDE.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Select the advanced-rag-pipeline folder on your Desktop, then click Open.
  • Create a new file called .env in the root of your advanced-rag-pipeline folder by clicking the new file icon in the Explorer sidebar.
  • Add the following content to your .env file:
NVIDIA_API_KEY=your-api-key-here

What is NVIDIA_API_KEY?

This environment variable authenticates your requests to the NVIDIA API Catalog. NVIDIA API keys start with nvapi-.

The python-dotenv library loads this value automatically so your scripts can access it without hardcoding the key.

  • Replace your-api-key-here with your actual NVIDIA API key.
  • Save .env.

Don't have an NVIDIA API key?

Visit the NVIDIA API Catalog at build.nvidia.com, sign in, and generate a free API key from any model's page.

help me get an NVIDIA API key for the API Catalog

✔️ Awesome, I've got everything!

Great. Make sure you've saved your .env file before moving on.

ⓧ I'd like to double check the full code

Here is the complete .env file for comparison:

NVIDIA_API_KEY=your-api-key-here
Verify your API connection

Before building anything, confirm that your API key works and your packages are installed correctly.

  • In your terminal, make sure you are inside the advanced-rag-pipeline folder.
  • Verify your connection to NVIDIA by running this command:
python -c "from langchain_nvidia_ai_endpoints import ChatNVIDIA; from dotenv import load_dotenv; load_dotenv(); models = ChatNVIDIA.get_available_models(); print(f'Connected! Found {len(models)} available models.')"

What does this command do?

  • It imports ChatNVIDIA from the NVIDIA endpoints package.
  • load_dotenv() reads your .env file so the API key is available.
  • ChatNVIDIA.get_available_models() calls the NVIDIA API and returns a list of models you can access.
  • If you see a model count printed, your key and packages are working correctly.

You should see output like Connected! Found followed by a number of available models.

Seeing an authentication error?

  • Double check that your .env file is in the advanced-rag-pipeline folder and that you saved it.
  • Confirm your API key starts with nvapi- and has no extra spaces.
  • Make sure you are running the command from inside the advanced-rag-pipeline directory where your .env file lives.

help me fix my NVIDIA API authentication error when calling ChatNVIDIA.get_available_models()

Your environment is ready. Next up, you'll create a knowledge base of documents and build the ingestion pipeline that chunks and indexes them for retrieval.

Set Up Your Environment

With your packages installed and your NVIDIA_API_KEY saved in .env, the next thing your pipeline needs is a single configuration file that all other modules can import from.

Centralizing settings like model names, chunk sizes, and retrieval parameters in one place means you can tune your pipeline later without hunting through multiple files.

In this step, get ready to:
  • Create a configuration file with all pipeline constants.
  • Verify your NVIDIA API connection is working.
Create the configuration file

Your pipeline will use three NVIDIA models: one for embeddings, one for reranking, and one for chat. It also needs several retrieval parameters like chunk sizes and how many documents to return at each stage.

  • Create a new file called config.py in your project folder.
  • Add the following code to config.py:
import os
from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv()

# NVIDIA API authentication
NVIDIA_API_KEY = os.getenv("NVIDIA_API_KEY")

# Model identifiers
EMBEDDING_MODEL = "nvidia/llama-nemotron-embed-1b-v2"
RERANK_MODEL = "nvidia/llama-nemotron-rerank-1b-v2"
LLM_MODEL = "meta/llama-3.1-8b-instruct"

# Chunking parameters
CHUNK_SIZE = 500
CHUNK_OVERLAP = 50

# Retrieval parameters
TOP_K_RETRIEVAL = 20
TOP_K_FINAL = 5
RRF_K = 60
NUM_QUERY_EXPANSIONS = 3

What does this code do?

This file loads your API key from the .env file and defines every tunable parameter for the pipeline in one place.

  • load_dotenv() reads the .env file and sets its values as environment variables so your code can access them securely.
  • EMBEDDING_MODEL points to NVIDIA's Nemotron embedding model for converting text into vectors.
  • RERANK_MODEL points to the cross-encoder reranker that will score query-document relevance.
  • TOP_K_RETRIEVAL = 20 means you pull 20 candidates from search before reranking narrows them to the final 5 (TOP_K_FINAL).
  • RRF_K = 60 is the constant in the Reciprocal Rank Fusion formula that prevents top-ranked items from dominating.
  • Save config.py.

Seeing an import error for dotenv?

Make sure the python-dotenv package is installed. You can confirm by running pip show python-dotenv in your terminal. If it is missing, reinstall it.

help me fix a python-dotenv import error

Verify your NVIDIA API connection

Before building the rest of the pipeline, confirm your API key is valid and the NVIDIA NIM endpoint is reachable.

  • Verify connectivity by running this command in your terminal:
python -c "from config import NVIDIA_API_KEY; from langchain_nvidia_ai_endpoints import ChatNVIDIA; models = ChatNVIDIA.get_available_models(); print('Connected! Found', len(models), 'models available.')"
  • You should see output like Connected! Found 50 models available. (the exact number may vary).

What does this command do?

This one-liner imports your config (which triggers load_dotenv() and loads your API key), then calls ChatNVIDIA.get_available_models() to query the NVIDIA API Catalog. A successful response means your key is valid and your network can reach the endpoint.

Seeing an authentication or connection error?

  • Open your .env file and confirm your key starts with nvapi- and has no extra spaces or quotes around it.
  • Make sure the file is named exactly .env (not env.txt or .env.local) and sits in the same folder as config.py.
  • If you see a timeout, check your internet connection. The NVIDIA API Catalog endpoint requires outbound HTTPS access.

help me fix my NVIDIA API connection error

✔️ Awesome, I've got everything!

Great. Double check you've saved config.py and that your verification command printed a model count.

ⓧ I'd like to double check the full code

import os
from dotenv import load_dotenv

load_dotenv()

NVIDIA_API_KEY = os.getenv("NVIDIA_API_KEY")
EMBEDDING_MODEL = "nvidia/llama-nemotron-embed-1b-v2"
RERANK_MODEL = "nvidia/llama-nemotron-rerank-1b-v2"
LLM_MODEL = "meta/llama-3.1-8b-instruct"
CHUNK_SIZE = 500
CHUNK_OVERLAP = 50
TOP_K_RETRIEVAL = 20
TOP_K_FINAL = 5
RRF_K = 60
NUM_QUERY_EXPANSIONS = 3

Your configuration is locked in and your API connection is live. Next up, you'll build the document ingestion pipeline that chunks and embeds your knowledge base.

Build the Document Ingestion Pipeline

Your environment is configured and your NVIDIA API connection is verified. Now it's time to give your pipeline some data to work with.

Before any retrieval can happen, you need documents indexed in a way that both keyword search and vector search can access them. In this step, you'll build a knowledge base of technical documents, split them into retrieval-sized chunks, and index those chunks in two different search systems.

In this step, get ready to:
  • Create a sample knowledge base with overlapping technical documents.
  • Build a chunking function that splits documents into retrieval-sized pieces.
  • Index chunks in both a FAISS vector store and a BM25 keyword index.
Create the sample knowledge base

Your retrieval pipeline needs documents to search over. You'll create a set of 8 technical articles about RAG-related topics, with enough overlapping terminology to make retrieval genuinely challenging.

  • Create a new file called documents.py in your project folder and paste the following code:
DOCUMENTS = [
    {
        "title": "Vector Databases Overview",
        "content": """Vector databases are specialized database systems designed to store, index, and query high-dimensional vector embeddings efficiently. Unlike traditional databases that work with structured data like numbers and strings, vector databases handle dense numerical representations of unstructured data such as text, images, and audio. Popular vector databases include FAISS, Pinecone, Weaviate, Qdrant, and Milvus. These systems use approximate nearest neighbor (ANN) algorithms like HNSW (Hierarchical Navigable Small World) graphs to enable fast similarity search across millions or billions of vectors. The choice of distance metric (cosine similarity, euclidean distance, or dot product) depends on how the embeddings were trained. Vector databases are fundamental to modern AI applications including recommendation systems, semantic search, and retrieval-augmented generation (RAG) pipelines."""
    },
    {
        "title": "BM25 Algorithm Explained",
        "content": """BM25 (Best Matching 25) is a ranking function used by search engines to estimate the relevance of documents to a given search query. It is part of the family of probabilistic information retrieval models. BM25 considers term frequency (how often a word appears in a document), inverse document frequency (how rare a word is across all documents), and document length normalization. The formula includes two free parameters: k1 controls term frequency saturation (typically set to 1.2-2.0) and b controls document length normalization (typically 0.75). BM25 excels at exact keyword matching and performs well for queries containing specific technical terms, product names, error codes, or identifiers that dense embedding models might miss. It is computationally cheap and requires no GPU or model inference, making it ideal for the first stage of a multi-stage retrieval pipeline."""
    },
    {
        "title": "Retrieval-Augmented Generation Architecture",
        "content": """Retrieval-Augmented Generation (RAG) combines information retrieval with text generation to produce factually grounded responses. The basic RAG architecture has three stages: indexing, retrieval, and generation. During indexing, documents are chunked, embedded, and stored in a vector database. At query time, the user's question is embedded and used to find similar document chunks via vector similarity search. The retrieved chunks are then inserted into the LLM's prompt as context for generating an answer. Advanced RAG systems add multiple layers to fix production failures: hybrid search to catch exact-match terms that pure vector search misses, cross-encoder reranking to eliminate noisy results, query transformation to bridge the question-answer distribution mismatch, and knowledge graph traversal to handle multi-hop questions requiring facts from multiple documents."""
    },
    {
        "title": "Cross-Encoder Reranking",
        "content": """Cross-encoder reranking is a second-pass relevance scoring technique applied after initial retrieval. Unlike bi-encoders that encode queries and documents independently, cross-encoders process the query and document together as a single input pair, allowing full cross-attention between all tokens. This joint encoding produces much more accurate relevance scores but at significantly higher computational cost. In practice, a bi-encoder retrieves 20-100 candidate documents cheaply, then a cross-encoder rescores each candidate individually to produce a final ranking. Cross-encoder reranking typically improves precision by 15-20 points on MRR metrics. The added latency is usually 150-300ms for 20 candidates, which is acceptable for interactive applications with total response times of 2-4 seconds. Popular reranking models include Cohere Rerank, NVIDIA NeMo Retriever rerankers, and open-source models like BGE-reranker and ms-marco-MiniLM."""
    },
    {
        "title": "Reciprocal Rank Fusion",
        "content": """Reciprocal Rank Fusion (RRF) is a method for combining multiple ranked lists into a single unified ranking. The key insight of RRF is that it uses rank positions rather than raw scores, making it robust across retrieval systems with incomparable score scales. The formula is: RRF_score(d) = sum over all retrievers of 1/(k + rank(d)), where k is a constant (typically 60) that prevents the top-ranked item from dominating. Documents that rank highly in multiple retrieval systems receive boosted scores. RRF is preferred over score normalization because BM25 scores, cosine similarity values, and cross-encoder logits have fundamentally different distributions. RRF only requires rank positions, making it a fair fusion method regardless of the underlying scoring mechanism. In production RAG systems, RRF is commonly used to merge BM25 keyword results with dense vector search results, capturing both exact-term matches and semantic similarity."""
    },
    {
        "title": "Query Transformation Techniques",
        "content": """Query transformation rewrites the user's query before retrieval to improve recall without changing the index. The core problem is that user questions are typically short and interrogative, while documents are long and declarative, creating a distribution mismatch in embedding space. Multi-query expansion generates several alternative phrasings of the same question and retrieves for each, taking the union of results. This overcomes vocabulary mismatch when different words describe the same concept. HyDE (Hypothetical Document Embeddings) generates a synthetic answer first, embeds that answer-shaped text, and retrieves documents similar to it. Step-back prompting derives a more abstract version of a specific question to retrieve broader context. Each transformation targets a different failure mode: multi-query for vocabulary mismatch, HyDE for register mismatch between questions and passages, and step-back for overly specific queries. Production systems typically apply transformations selectively based on query classification rather than using them on every query."""
    },
    {
        "title": "Embedding Models for RAG",
        "content": """Embedding models convert text into dense vector representations for similarity search. For RAG applications, the choice of embedding model significantly impacts retrieval quality. Key considerations include embedding dimensionality (higher dimensions capture more information but require more storage), maximum token length (determines how much text can be embedded in one pass), and whether the model was trained with query-passage asymmetry (using different prefixes for queries vs documents). Modern embedding models like NVIDIA's NeMo Retriever embeddings, OpenAI's text-embedding-3-large, and Cohere's embed-v3 support configurable dimensions through Matryoshka representation learning, allowing tradeoffs between accuracy and storage. For multilingual RAG, models trained on diverse language data perform cross-lingual retrieval where questions in one language find answers in another. Embedding models are typically trained using contrastive learning with bi-encoder architectures, maximizing similarity for relevant query-passage pairs while minimizing it for negative pairs."""
    },
    {
        "title": "Chunking Strategies for RAG",
        "content": """Chunking is the process of splitting documents into smaller segments for embedding and retrieval. The chunking strategy significantly affects RAG quality because chunks that are too large dilute relevant information with noise, while chunks that are too small lose context. Fixed-size chunking splits text at character or token boundaries with optional overlap, offering simplicity but potentially breaking sentences mid-thought. Recursive character splitting tries multiple separators in order (paragraphs, sentences, words) to find natural boundaries. Semantic chunking groups consecutive sentences based on embedding similarity, creating chunks where related content stays together. Document-aware chunking respects structural boundaries like headings, code blocks, and tables. For most RAG applications, recursive character splitting with 500-1000 token chunks and 10-20% overlap provides a good baseline. The overlap ensures that information spanning chunk boundaries is captured in at least one chunk."""
    }
]

What does this knowledge base do?

The DOCUMENTS list contains 8 technical articles about RAG-related concepts like vector databases, BM25, reranking, and chunking.

These documents deliberately overlap in vocabulary. For example, "BM25" appears in the BM25 article, the RRF article, and the RAG architecture article. This makes retrieval harder and gives you a realistic test bed for comparing search strategies.

  • Save documents.py.

Seeing a syntax error?

Make sure you have the triple-quote """ at both the start and end of each content value. Missing a closing triple-quote causes Python to treat the next dictionary entry as part of the string.

help me fix a syntax error in my documents.py knowledge base file

Build the document chunking function

Full documents are too large to embed and retrieve effectively. You need to split them into smaller chunks that each focus on a single idea.

  • Create a new file called ingestion.py in your project folder and paste the following code:
from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings
from langchain_community.vectorstores import FAISS
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_core.documents import Document
from rank_bm25 import BM25Okapi
from config import NVIDIA_API_KEY, EMBEDDING_MODEL, CHUNK_SIZE, CHUNK_OVERLAP
from documents import DOCUMENTS


def create_chunks():
    """Split documents into chunks."""
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=CHUNK_SIZE,
        chunk_overlap=CHUNK_OVERLAP,
        separators=["\n\n", "\n", ". ", " ", ""]
    )

    all_chunks = []
    for doc in DOCUMENTS:
        chunks = text_splitter.split_text(doc["content"])
        for i, chunk in enumerate(chunks):
            all_chunks.append(Document(
                page_content=chunk,
                metadata={"source": doc["title"], "chunk_index": i}
            ))
    return all_chunks

What does this code do?

The imports bring in everything you'll need for the full ingestion pipeline: embeddings, vector store, text splitter, BM25, and your config values.

RecursiveCharacterTextSplitter tries splitting on paragraph breaks first, then sentences, then words. This keeps ideas together instead of cutting mid-sentence. Each chunk is 500 characters with 50 characters of overlap so information at boundaries isn't lost.

The metadata dictionary on each chunk tracks which document it came from. This lets you trace retrieval results back to their source later.

  • Save ingestion.py.

Import error on rank_bm25?

Make sure you installed the package with pip install rank_bm25 in Step 1. The package name on PyPI uses an underscore, not a hyphen.

help me fix an import error for rank_bm25 in my ingestion.py file

Add the vector store and BM25 index builders

Chunks need to be indexed before you can search them. You'll build two separate indices: a FAISS vector store for semantic similarity search, and a BM25 index for exact keyword matching.

  • In ingestion.py, add the following function below create_chunks():
def build_vector_store(chunks):
    """Build FAISS vector store from chunks."""
    embeddings = NVIDIAEmbeddings(
        model=EMBEDDING_MODEL,
        api_key=NVIDIA_API_KEY,
        truncate="END"
    )
    vector_store = FAISS.from_documents(chunks, embeddings)
    return vector_store

What does this code do?

NVIDIAEmbeddings connects to the nvidia/llama-nemotron-embed-1b-v2 model via the NVIDIA API. It converts each chunk's text into a dense vector representation.

The truncate="END" parameter tells the API to trim text from the end if it exceeds the model's maximum token length, rather than raising an error. FAISS.from_documents() embeds all chunks and stores them in an in-memory vector index for fast similarity search.

  • Now add the BM25 index builder below build_vector_store():
def build_bm25_index(chunks):
    """Build BM25 index from chunks."""
    tokenized_corpus = [
        chunk.page_content.lower().split() for chunk in chunks
    ]
    bm25 = BM25Okapi(tokenized_corpus)
    return bm25

What does this code do?

BM25Okapi builds a keyword-based search index over your chunks. It needs a tokenized corpus, so each chunk's text is lowercased and split into individual words.

This gives you a second retrieval path that excels at finding exact terms like "BM25", "k=60", or "HNSW" that embedding models sometimes miss.

  • Save ingestion.py.
  • Verify the ingestion pipeline works by running this in your terminal:
python -c "from ingestion import create_chunks; chunks = create_chunks(); print(f'Created {len(chunks)} chunks')"

You should see output showing the number of chunks created (around 20-25 chunks depending on how the splitter breaks the text).

Seeing a ModuleNotFoundError?

Make sure both documents.py and ingestion.py are saved in the same folder as config.py. Python looks for imports in the current directory, so run your command from the project folder.

help me fix a ModuleNotFoundError when importing from ingestion.py

✔️ Awesome, I've got everything!

Great. Double check you've saved both documents.py and ingestion.py.

ⓧ I'd like to double check the full code

Here is the complete documents.py:

DOCUMENTS = [
    {
        "title": "Vector Databases Overview",
        "content": """Vector databases are specialized database systems designed to store, index, and query high-dimensional vector embeddings efficiently. Unlike traditional databases that work with structured data like numbers and strings, vector databases handle dense numerical representations of unstructured data such as text, images, and audio. Popular vector databases include FAISS, Pinecone, Weaviate, Qdrant, and Milvus. These systems use approximate nearest neighbor (ANN) algorithms like HNSW (Hierarchical Navigable Small World) graphs to enable fast similarity search across millions or billions of vectors. The choice of distance metric (cosine similarity, euclidean distance, or dot product) depends on how the embeddings were trained. Vector databases are fundamental to modern AI applications including recommendation systems, semantic search, and retrieval-augmented generation (RAG) pipelines."""
    },
    {
        "title": "BM25 Algorithm Explained",
        "content": """BM25 (Best Matching 25) is a ranking function used by search engines to estimate the relevance of documents to a given search query. It is part of the family of probabilistic information retrieval models. BM25 considers term frequency (how often a word appears in a document), inverse document frequency (how rare a word is across all documents), and document length normalization. The formula includes two free parameters: k1 controls term frequency saturation (typically set to 1.2-2.0) and b controls document length normalization (typically 0.75). BM25 excels at exact keyword matching and performs well for queries containing specific technical terms, product names, error codes, or identifiers that dense embedding models might miss. It is computationally cheap and requires no GPU or model inference, making it ideal for the first stage of a multi-stage retrieval pipeline."""
    },
    {
        "title": "Retrieval-Augmented Generation Architecture",
        "content": """Retrieval-Augmented Generation (RAG) combines information retrieval with text generation to produce factually grounded responses. The basic RAG architecture has three stages: indexing, retrieval, and generation. During indexing, documents are chunked, embedded, and stored in a vector database. At query time, the user's question is embedded and used to find similar document chunks via vector similarity search. The retrieved chunks are then inserted into the LLM's prompt as context for generating an answer. Advanced RAG systems add multiple layers to fix production failures: hybrid search to catch exact-match terms that pure vector search misses, cross-encoder reranking to eliminate noisy results, query transformation to bridge the question-answer distribution mismatch, and knowledge graph traversal to handle multi-hop questions requiring facts from multiple documents."""
    },
    {
        "title": "Cross-Encoder Reranking",
        "content": """Cross-encoder reranking is a second-pass relevance scoring technique applied after initial retrieval. Unlike bi-encoders that encode queries and documents independently, cross-encoders process the query and document together as a single input pair, allowing full cross-attention between all tokens. This joint encoding produces much more accurate relevance scores but at significantly higher computational cost. In practice, a bi-encoder retrieves 20-100 candidate documents cheaply, then a cross-encoder rescores each candidate individually to produce a final ranking. Cross-encoder reranking typically improves precision by 15-20 points on MRR metrics. The added latency is usually 150-300ms for 20 candidates, which is acceptable for interactive applications with total response times of 2-4 seconds. Popular reranking models include Cohere Rerank, NVIDIA NeMo Retriever rerankers, and open-source models like BGE-reranker and ms-marco-MiniLM."""
    },
    {
        "title": "Reciprocal Rank Fusion",
        "content": """Reciprocal Rank Fusion (RRF) is a method for combining multiple ranked lists into a single unified ranking. The key insight of RRF is that it uses rank positions rather than raw scores, making it robust across retrieval systems with incomparable score scales. The formula is: RRF_score(d) = sum over all retrievers of 1/(k + rank(d)), where k is a constant (typically 60) that prevents the top-ranked item from dominating. Documents that rank highly in multiple retrieval systems receive boosted scores. RRF is preferred over score normalization because BM25 scores, cosine similarity values, and cross-encoder logits have fundamentally different distributions. RRF only requires rank positions, making it a fair fusion method regardless of the underlying scoring mechanism. In production RAG systems, RRF is commonly used to merge BM25 keyword results with dense vector search results, capturing both exact-term matches and semantic similarity."""
    },
    {
        "title": "Query Transformation Techniques",
        "content": """Query transformation rewrites the user's query before retrieval to improve recall without changing the index. The core problem is that user questions are typically short and interrogative, while documents are long and declarative, creating a distribution mismatch in embedding space. Multi-query expansion generates several alternative phrasings of the same question and retrieves for each, taking the union of results. This overcomes vocabulary mismatch when different words describe the same concept. HyDE (Hypothetical Document Embeddings) generates a synthetic answer first, embeds that answer-shaped text, and retrieves documents similar to it. Step-back prompting derives a more abstract version of a specific question to retrieve broader context. Each transformation targets a different failure mode: multi-query for vocabulary mismatch, HyDE for register mismatch between questions and passages, and step-back for overly specific queries. Production systems typically apply transformations selectively based on query classification rather than using them on every query."""
    },
    {
        "title": "Embedding Models for RAG",
        "content": """Embedding models convert text into dense vector representations for similarity search. For RAG applications, the choice of embedding model significantly impacts retrieval quality. Key considerations include embedding dimensionality (higher dimensions capture more information but require more storage), maximum token length (determines how much text can be embedded in one pass), and whether the model was trained with query-passage asymmetry (using different prefixes for queries vs documents). Modern embedding models like NVIDIA's NeMo Retriever embeddings, OpenAI's text-embedding-3-large, and Cohere's embed-v3 support configurable dimensions through Matryoshka representation learning, allowing tradeoffs between accuracy and storage. For multilingual RAG, models trained on diverse language data perform cross-lingual retrieval where questions in one language find answers in another. Embedding models are typically trained using contrastive learning with bi-encoder architectures, maximizing similarity for relevant query-passage pairs while minimizing it for negative pairs."""
    },
    {
        "title": "Chunking Strategies for RAG",
        "content": """Chunking is the process of splitting documents into smaller segments for embedding and retrieval. The chunking strategy significantly affects RAG quality because chunks that are too large dilute relevant information with noise, while chunks that are too small lose context. Fixed-size chunking splits text at character or token boundaries with optional overlap, offering simplicity but potentially breaking sentences mid-thought. Recursive character splitting tries multiple separators in order (paragraphs, sentences, words) to find natural boundaries. Semantic chunking groups consecutive sentences based on embedding similarity, creating chunks where related content stays together. Document-aware chunking respects structural boundaries like headings, code blocks, and tables. For most RAG applications, recursive character splitting with 500-1000 token chunks and 10-20% overlap provides a good baseline. The overlap ensures that information spanning chunk boundaries is captured in at least one chunk."""
    }
]

Here is the complete ingestion.py:

from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings
from langchain_community.vectorstores import FAISS
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_core.documents import Document
from rank_bm25 import BM25Okapi
from config import NVIDIA_API_KEY, EMBEDDING_MODEL, CHUNK_SIZE, CHUNK_OVERLAP
from documents import DOCUMENTS


def create_chunks():
    """Split documents into chunks."""
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=CHUNK_SIZE,
        chunk_overlap=CHUNK_OVERLAP,
        separators=["\n\n", "\n", ". ", " ", ""]
    )

    all_chunks = []
    for doc in DOCUMENTS:
        chunks = text_splitter.split_text(doc["content"])
        for i, chunk in enumerate(chunks):
            all_chunks.append(Document(
                page_content=chunk,
                metadata={"source": doc["title"], "chunk_index": i}
            ))
    return all_chunks


def build_vector_store(chunks):
    """Build FAISS vector store from chunks."""
    embeddings = NVIDIAEmbeddings(
        model=EMBEDDING_MODEL,
        api_key=NVIDIA_API_KEY,
        truncate="END"
    )
    vector_store = FAISS.from_documents(chunks, embeddings)
    return vector_store


def build_bm25_index(chunks):
    """Build BM25 index from chunks."""
    tokenized_corpus = [
        chunk.page_content.lower().split() for chunk in chunks
    ]
    bm25 = BM25Okapi(tokenized_corpus)
    return bm25

Your ingestion pipeline is ready. Next up, you'll use both of these indices to build a hybrid search function that combines keyword matching with semantic similarity.

Implement Hybrid Search with Reciprocal Rank Fusion

Your documents are chunked, embedded, and indexed. You have two retrieval systems ready to go: a FAISS vector store for semantic similarity and a BM25 index for keyword matching.

The problem is that each system has a blind spot. Vector search misses exact keyword matches (like specific parameter names or error codes), while BM25 misses semantic relationships between words. In this step, you'll combine both approaches using Reciprocal Rank Fusion to get the best of both worlds.

In this step, get ready to:
  • Build vector search and BM25 search wrapper functions.
  • Implement the RRF algorithm to merge ranked results from different retrievers.
  • Create a hybrid retrieval function that fuses both search approaches.
Create the retrieval file with search functions

You need two search functions that query your indices. One wraps the FAISS vector store for semantic similarity. The other wraps the BM25 index for keyword matching.

  • Create a new file called retrieval.py in your project folder.
  • Add the imports and the vector search function by pasting this code:
from langchain_nvidia_ai_endpoints import NVIDIARerank, ChatNVIDIA
from langchain_core.documents import Document
from config import (
    NVIDIA_API_KEY, RERANK_MODEL, LLM_MODEL,
    TOP_K_RETRIEVAL, TOP_K_FINAL, RRF_K, NUM_QUERY_EXPANSIONS
)


def vector_search(vector_store, query, k=TOP_K_RETRIEVAL):
    """Perform dense vector search."""
    results = vector_store.similarity_search(query, k=k)
    return results

What does this code do?

The imports pull in everything you'll need across this file (some will be used in later steps).

  • vector_search() wraps the FAISS similarity_search method to return the top k most semantically similar chunks to a query.
  • TOP_K_RETRIEVAL defaults to 20, meaning you cast a wide net before narrowing down later.

You won't see a change yet. The next function completes the search pair.

  • Below vector_search(), add the BM25 search function:
def bm25_search(bm25_index, chunks, query, k=TOP_K_RETRIEVAL):
    """Perform BM25 keyword search."""
    tokenized_query = query.lower().split()
    scores = bm25_index.get_scores(tokenized_query)

    scored_chunks = list(zip(scores, chunks))
    scored_chunks.sort(key=lambda x: x[0], reverse=True)

    return [chunk for _, chunk in scored_chunks[:k]]

What does this code do?

  • tokenized_query = query.lower().split() breaks the query into lowercase words, matching how the BM25 index was tokenized in ingestion.py.
  • get_scores() returns a relevance score for every chunk in the corpus. Higher scores mean stronger keyword overlap.
  • The function pairs each score with its chunk, sorts by score descending, and returns the top k results.
  • Save retrieval.py.
  • Verify the file is importable by running this in your terminal:
python -c "from retrieval import vector_search, bm25_search; print('Search functions imported successfully')"

You should see Search functions imported successfully printed in your terminal.

Seeing ModuleNotFoundError?

  • If you see ModuleNotFoundError: No module named 'retrieval', make sure you saved the file as retrieval.py (not retrieval.txt) in the same folder as config.py.
  • If you see ModuleNotFoundError: No module named 'langchain_nvidia_ai_endpoints', run pip install langchain-nvidia-ai-endpoints to install the missing package.
  • Make sure your terminal is in the same directory as your project files.

help me fix this ModuleNotFoundError when importing retrieval.py

Implement the RRF algorithm

Now you have two retrievers that return ranked lists. The challenge is combining them fairly. BM25 scores and cosine similarity values have completely different scales, so you cannot simply add them together.

RRF solves this by ignoring raw scores entirely. It only uses rank positions, giving each document a score of 1/(k + rank) from each retriever. Documents that rank highly in both lists get boosted.

  • Below bm25_search(), add the RRF function:
def reciprocal_rank_fusion(ranked_lists, k=RRF_K):
    """Fuse multiple ranked lists using RRF."""
    rrf_scores = {}

    for ranked_list in ranked_lists:
        for rank, doc in enumerate(ranked_list):
            doc_id = doc.page_content
            if doc_id not in rrf_scores:
                rrf_scores[doc_id] = {"score": 0.0, "doc": doc}
            rrf_scores[doc_id]["score"] += 1.0 / (k + rank + 1)

    sorted_results = sorted(
        rrf_scores.values(),
        key=lambda x: x["score"],
        reverse=True
    )
    return [item["doc"] for item in sorted_results]

What does this code do?

  • rrf_scores is a dictionary that accumulates a fusion score for each unique document across all ranked lists.
  • doc.page_content serves as the document ID. If the same chunk appears in both the vector and BM25 results, its scores stack.
  • The formula 1.0 / (k + rank + 1) gives rank-1 documents a score of 1/61 (when k=60). The k constant prevents top-ranked items from dominating the final ranking.
  • Documents appearing in multiple lists accumulate higher scores, surfacing results that both retrievers agree on.

Why k=60?

Research shows that sweeping k from 10 to 200 yields less than 1% improvement over the k=60 default. It is one of those rare hyperparameters that almost never needs tuning.

You won't see a change yet. The next function wires RRF into the full hybrid pipeline.

Create the hybrid retrieval function

The final piece ties vector search and BM25 search together. This function runs both retrievers, then merges their results using RRF.

  • Below reciprocal_rank_fusion(), add the hybrid retrieval function:
def hybrid_retrieve(vector_store, bm25_index, chunks, query, k=TOP_K_RETRIEVAL):
    """Combine vector and BM25 search with RRF."""
    vector_results = vector_search(vector_store, query, k=k)
    bm25_results = bm25_search(bm25_index, chunks, query, k=k)

    fused_results = reciprocal_rank_fusion(
        [vector_results, bm25_results], k=RRF_K
    )
    return fused_results[:k]

What does this code do?

  • vector_search() and bm25_search() each return their top 20 candidates independently.
  • reciprocal_rank_fusion() merges both lists into one unified ranking based on rank positions.
  • The final [:k] slice returns only the top results from the fused list, ready for the next stage of the pipeline.
  • Save retrieval.py.
  • Verify the full module imports cleanly by running:
python -c "from retrieval import hybrid_retrieve; print('Hybrid retrieval ready')"

You should see Hybrid retrieval ready in your terminal.

Seeing an ImportError?

  • If you see ImportError: cannot import name 'hybrid_retrieve' from 'retrieval', make sure you saved the file after adding the function.
  • Check for indentation errors. Each function should start at the leftmost column (no extra spaces before def).
  • If you see a SyntaxError, look for missing colons or unmatched parentheses near the line number Python reports.

help me debug this ImportError in retrieval.py

Why hybrid search instead of just vector search?

Vector search captures meaning but misses exact terms. If someone asks about "BM25 k1 parameter", a vector search might return chunks about information retrieval in general. BM25 catches the exact keyword "k1" that vector embeddings dilute.

By fusing both with RRF, you get semantic understanding AND exact keyword precision. Research shows hybrid search achieves higher recall across all query types compared to either approach alone.

✔️ Awesome, I've got everything!

Great. Double check you've saved retrieval.py with all four functions.

ⓧ I'd like to double check the full code

from langchain_nvidia_ai_endpoints import NVIDIARerank, ChatNVIDIA
from langchain_core.documents import Document
from config import (
    NVIDIA_API_KEY, RERANK_MODEL, LLM_MODEL,
    TOP_K_RETRIEVAL, TOP_K_FINAL, RRF_K, NUM_QUERY_EXPANSIONS
)


def vector_search(vector_store, query, k=TOP_K_RETRIEVAL):
    """Perform dense vector search."""
    results = vector_store.similarity_search(query, k=k)
    return results


def bm25_search(bm25_index, chunks, query, k=TOP_K_RETRIEVAL):
    """Perform BM25 keyword search."""
    tokenized_query = query.lower().split()
    scores = bm25_index.get_scores(tokenized_query)

    scored_chunks = list(zip(scores, chunks))
    scored_chunks.sort(key=lambda x: x[0], reverse=True)

    return [chunk for _, chunk in scored_chunks[:k]]


def reciprocal_rank_fusion(ranked_lists, k=RRF_K):
    """Fuse multiple ranked lists using RRF."""
    rrf_scores = {}

    for ranked_list in ranked_lists:
        for rank, doc in enumerate(ranked_list):
            doc_id = doc.page_content
            if doc_id not in rrf_scores:
                rrf_scores[doc_id] = {"score": 0.0, "doc": doc}
            rrf_scores[doc_id]["score"] += 1.0 / (k + rank + 1)

    sorted_results = sorted(
        rrf_scores.values(),
        key=lambda x: x["score"],
        reverse=True
    )
    return [item["doc"] for item in sorted_results]


def hybrid_retrieve(vector_store, bm25_index, chunks, query, k=TOP_K_RETRIEVAL):
    """Combine vector and BM25 search with RRF."""
    vector_results = vector_search(vector_store, query, k=k)
    bm25_results = bm25_search(bm25_index, chunks, query, k=k)

    fused_results = reciprocal_rank_fusion(
        [vector_results, bm25_results], k=RRF_K
    )
    return fused_results[:k]

Your hybrid retriever is ready. Next up, you'll add a cross-encoder reranking layer that takes these fused results and dramatically improves precision by rescoring each candidate.

Add Cross-Encoder Reranking

Your hybrid retriever now pulls candidates from two different search systems and fuses them with RRF. The problem is that 20 candidates is still too many noisy results to feed directly to an LLM.

A cross-encoder reranker solves this by scoring each candidate against the query with full cross-attention, then keeping only the top 5 most relevant chunks. This is the precision layer that ensures your LLM only sees the highest-quality context.

In this step, get ready to:
  • Initialize NVIDIA's cross-encoder reranking model.
  • Build a reranking function that narrows 20 hybrid candidates down to the top 5.
  • Integrate the reranker so only the highest-relevance chunks reach the LLM.
Understand why reranking matters

Your hybrid search retrieves 20 candidate chunks using both keyword matching and vector similarity. Those initial retrieval methods are fast but imprecise.

A cross-encoder processes the query and each candidate together as a single input pair, allowing the model to compare them token by token. This produces far more accurate relevance scores than the first-pass retrievers.

Why not just use the cross-encoder for everything?

Cross-encoders score each query-document pair individually, which is computationally expensive. Running a cross-encoder over your entire corpus for every query would be too slow.

The solution is a two-stage pipeline: cheap first-pass retrieval (BM25 + vector) narrows thousands of chunks to 20 candidates, then the expensive cross-encoder rescores just those 20. The added latency is typically 150-300ms for 20 candidates, which is acceptable for interactive applications.

Add the rerank function to your retrieval module

The NVIDIARerank class is already imported at the top of your retrieval.py file, along with the config values it needs. You just need to add the function that uses it.

  • In retrieval.py, scroll to the end of the file after the hybrid_retrieve() function.
  • Add the following rerank() function below it:
def rerank(query, documents, top_n=TOP_K_FINAL):
    """Rerank documents using NVIDIA cross-encoder."""
    reranker = NVIDIARerank(
        model=RERANK_MODEL,
        api_key=NVIDIA_API_KEY,
        top_n=top_n,
        truncate="END"
    )
    reranked = reranker.compress_documents(
        query=query,
        documents=documents
    )
    return reranked

What does this code do?

  • NVIDIARerank initializes the nvidia/llama-nemotron-rerank-1b-v2 cross-encoder model via the NVIDIA API.
  • top_n=TOP_K_FINAL tells the reranker to return only the top 5 most relevant documents (set in your config.py).
  • truncate="END" handles documents that exceed the model's token limit by trimming from the end rather than raising an error.
  • compress_documents() scores each candidate against the query and returns only the top-ranked results.
  • Save retrieval.py.

How reranking fits the pipeline

In your complete pipeline, the flow will be: hybrid search returns 20 candidates, then rerank() scores those 20 candidates against the original query and keeps only the 5 most relevant. The LLM only ever sees those top 5 chunks as context.

Seeing an import error for NVIDIARerank?

  • Confirm that the import line at the top of retrieval.py reads from langchain_nvidia_ai_endpoints import NVIDIARerank, ChatNVIDIA.
  • Make sure you installed langchain-nvidia-ai-endpoints in Step 1. If not, run pip install langchain-nvidia-ai-endpoints again.

Why is my reranker not working?

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

Here is the full retrieval.py as it should look now:

from langchain_nvidia_ai_endpoints import NVIDIARerank, ChatNVIDIA
from langchain_core.documents import Document
from config import (
    NVIDIA_API_KEY, RERANK_MODEL, LLM_MODEL,
    TOP_K_RETRIEVAL, TOP_K_FINAL, RRF_K, NUM_QUERY_EXPANSIONS
)


def vector_search(vector_store, query, k=TOP_K_RETRIEVAL):
    """Perform dense vector search."""
    results = vector_store.similarity_search(query, k=k)
    return results


def bm25_search(bm25_index, chunks, query, k=TOP_K_RETRIEVAL):
    """Perform BM25 keyword search."""
    tokenized_query = query.lower().split()
    scores = bm25_index.get_scores(tokenized_query)

    scored_chunks = list(zip(scores, chunks))
    scored_chunks.sort(key=lambda x: x[0], reverse=True)

    return [chunk for _, chunk in scored_chunks[:k]]


def reciprocal_rank_fusion(ranked_lists, k=RRF_K):
    """Fuse multiple ranked lists using RRF."""
    rrf_scores = {}

    for ranked_list in ranked_lists:
        for rank, doc in enumerate(ranked_list):
            doc_id = doc.page_content
            if doc_id not in rrf_scores:
                rrf_scores[doc_id] = {"score": 0.0, "doc": doc}
            rrf_scores[doc_id]["score"] += 1.0 / (k + rank + 1)

    sorted_results = sorted(
        rrf_scores.values(),
        key=lambda x: x["score"],
        reverse=True
    )
    return [item["doc"] for item in sorted_results]


def hybrid_retrieve(vector_store, bm25_index, chunks, query, k=TOP_K_RETRIEVAL):
    """Combine vector and BM25 search with RRF."""
    vector_results = vector_search(vector_store, query, k=k)
    bm25_results = bm25_search(bm25_index, chunks, query, k=k)

    fused_results = reciprocal_rank_fusion(
        [vector_results, bm25_results], k=RRF_K
    )
    return fused_results[:k]


def rerank(query, documents, top_n=TOP_K_FINAL):
    """Rerank documents using NVIDIA cross-encoder."""
    reranker = NVIDIARerank(
        model=RERANK_MODEL,
        api_key=NVIDIA_API_KEY,
        top_n=top_n,
        truncate="END"
    )
    reranked = reranker.compress_documents(
        query=query,
        documents=documents
    )
    return reranked

Your pipeline now has a precision layer that ensures only the most relevant chunks reach your LLM. Next, you'll add multi-query expansion to cast a wider net during retrieval and catch documents that a single query phrasing might miss.

Add Multi-Query Expansion

Your pipeline can now retrieve with hybrid search and rerank for precision. There is still one failure mode left to address: vocabulary mismatch.

A user might ask "How does the k constant affect fusion?" while the relevant document says "k is a parameter that prevents the top-ranked item from dominating." Different words, same intent. Multi-query expansion solves this by generating alternative phrasings of the question and retrieving for each one, casting a wider net before reranking narrows the results back down.

In this step, get ready to:
  • Build a query expansion function that uses an LLM to generate alternative phrasings.
  • Verify that the function returns multiple rephrased questions from a single input query.
Add the expand_query() function to retrieval.py

User questions are typically short and use different vocabulary than the documents they need to find. Multi-query expansion generates several alternative phrasings of the same question, so the retriever can match documents regardless of which words the user chose.

  • In retrieval.py, scroll to the bottom of the file after the rerank() function.
  • Add the following expand_query() function below it:
def expand_query(query):
    """Generate alternative query phrasings using LLM."""
    llm = ChatNVIDIA(
        model=LLM_MODEL,
        api_key=NVIDIA_API_KEY,
        temperature=0.7,
        max_tokens=256
    )

    prompt = f"""Generate {NUM_QUERY_EXPANSIONS} alternative phrasings of the following question.
Each phrasing should capture the same intent but use different words.
Return only the alternative questions, one per line, without numbering.

Question: {query}"""

    response = llm.invoke(prompt)
    alternatives = [
        line.strip() for line in response.content.strip().split("\n")
        if line.strip()
    ]
    return alternatives[:NUM_QUERY_EXPANSIONS]

What does this code do?

This function uses ChatNVIDIA with the meta/llama-3.1-8b-instruct model to rephrase the user's question into 3 alternative versions.

  • temperature=0.7 adds creativity so each phrasing uses genuinely different vocabulary.
  • The prompt asks the LLM to return only the questions, one per line, with no numbering. This makes parsing the response straightforward.
  • alternatives[:NUM_QUERY_EXPANSIONS] enforces a hard cap of 3 results, even if the LLM returns extra lines.

Why does multi-query expansion help retrieval?

In the next step, you will run hybrid retrieval for each alternative phrasing and combine all the results with RRF before reranking. This catches documents that would be missed by a single query phrasing alone.

For example, asking "What is the RRF formula?" might miss a chunk that only uses the phrase "reciprocal rank fusion scoring method." An alternative phrasing bridges that gap.

  • Save retrieval.py.
  • Verify there are no syntax errors by running this in your terminal:
python -c "from retrieval import expand_query; print('expand_query loaded successfully')"

You should see expand_query loaded successfully printed in the terminal.

Seeing an ImportError or SyntaxError?

  • Make sure the new function is pasted after the closing line of rerank(), not inside it. Check your indentation.
  • Confirm that NUM_QUERY_EXPANSIONS is included in the import block at the top of retrieval.py. It should already be there from the previous step.
  • Check that the triple-quoted f-string in the prompt has matching opening and closing """ marks.

Help me debug my expand_query function in retrieval.py

✔️ Awesome, I've got everything!

Great. Double check you have saved your file.

ⓧ I'd like to double check the full code

Here is the complete retrieval.py file at this point:

from langchain_nvidia_ai_endpoints import NVIDIARerank, ChatNVIDIA
from langchain_core.documents import Document
from config import (
    NVIDIA_API_KEY, RERANK_MODEL, LLM_MODEL,
    TOP_K_RETRIEVAL, TOP_K_FINAL, RRF_K, NUM_QUERY_EXPANSIONS
)


def vector_search(vector_store, query, k=TOP_K_RETRIEVAL):
    """Perform dense vector search."""
    results = vector_store.similarity_search(query, k=k)
    return results


def bm25_search(bm25_index, chunks, query, k=TOP_K_RETRIEVAL):
    """Perform BM25 keyword search."""
    tokenized_query = query.lower().split()
    scores = bm25_index.get_scores(tokenized_query)

    scored_chunks = list(zip(scores, chunks))
    scored_chunks.sort(key=lambda x: x[0], reverse=True)

    return [chunk for _, chunk in scored_chunks[:k]]


def reciprocal_rank_fusion(ranked_lists, k=RRF_K):
    """Fuse multiple ranked lists using RRF."""
    rrf_scores = {}

    for ranked_list in ranked_lists:
        for rank, doc in enumerate(ranked_list):
            doc_id = doc.page_content
            if doc_id not in rrf_scores:
                rrf_scores[doc_id] = {"score": 0.0, "doc": doc}
            rrf_scores[doc_id]["score"] += 1.0 / (k + rank + 1)

    sorted_results = sorted(
        rrf_scores.values(),
        key=lambda x: x["score"],
        reverse=True
    )
    return [item["doc"] for item in sorted_results]


def hybrid_retrieve(vector_store, bm25_index, chunks, query, k=TOP_K_RETRIEVAL):
    """Combine vector and BM25 search with RRF."""
    vector_results = vector_search(vector_store, query, k=k)
    bm25_results = bm25_search(bm25_index, chunks, query, k=k)

    fused_results = reciprocal_rank_fusion(
        [vector_results, bm25_results], k=RRF_K
    )
    return fused_results[:k]


def rerank(query, documents, top_n=TOP_K_FINAL):
    """Rerank documents using NVIDIA cross-encoder."""
    reranker = NVIDIARerank(
        model=RERANK_MODEL,
        api_key=NVIDIA_API_KEY,
        top_n=top_n,
        truncate="END"
    )
    reranked = reranker.compress_documents(
        query=query,
        documents=documents
    )
    return reranked


def expand_query(query):
    """Generate alternative query phrasings using LLM."""
    llm = ChatNVIDIA(
        model=LLM_MODEL,
        api_key=NVIDIA_API_KEY,
        temperature=0.7,
        max_tokens=256
    )

    prompt = f"""Generate {NUM_QUERY_EXPANSIONS} alternative phrasings of the following question.
Each phrasing should capture the same intent but use different words.
Return only the alternative questions, one per line, without numbering.

Question: {query}"""

    response = llm.invoke(prompt)
    alternatives = [
        line.strip() for line in response.content.strip().split("\n")
        if line.strip()
    ]
    return alternatives[:NUM_QUERY_EXPANSIONS]

You now have all the individual retrieval building blocks: vector search, BM25 search, RRF fusion, reranking, and query expansion. Next, you will wire them together into a complete pipeline and compare it against basic RAG to see the difference.

Build the Complete Pipeline and Compare

You've built all three advanced retrieval techniques individually. Each one addresses a different failure mode of basic vector search. Now it's time to chain them together into a single pipeline.

In this step, you'll wire query expansion, hybrid search, RRF fusion, and reranking into one function. Then you'll build a basic RAG baseline and run the same queries through both to see the measurable difference.

In this step, get ready to:
  • Wire the full advanced retrieval pipeline together in a single function.
  • Create the generation layer that feeds retrieved context to an LLM.
  • Run test queries through both pipelines and compare results side-by-side.
Wire together the advanced retrieval pipeline

Your retrieval.py file has all the building blocks: vector search, BM25 search, RRF fusion, reranking, and query expansion. The final step is a single function that orchestrates them in sequence.

  • In retrieval.py, scroll to the bottom of the file (after the expand_query() function).
  • Add the advanced_retrieve() function by copying the following code at the bottom of the file:
def advanced_retrieve(vector_store, bm25_index, chunks, query):
    """Full advanced pipeline: expand -> hybrid search -> RRF -> rerank."""
    all_queries = [query] + expand_query(query)

    all_ranked_lists = []
    for q in all_queries:
        vector_results = vector_search(vector_store, q, k=TOP_K_RETRIEVAL)
        bm25_results = bm25_search(bm25_index, chunks, q, k=TOP_K_RETRIEVAL)
        all_ranked_lists.append(vector_results)
        all_ranked_lists.append(bm25_results)

    fused = reciprocal_rank_fusion(all_ranked_lists, k=RRF_K)
    candidates = fused[:TOP_K_RETRIEVAL]

    reranked = rerank(query, candidates, top_n=TOP_K_FINAL)
    return reranked

What does this function do?

  • expand_query(query) generates 3 alternative phrasings. Combined with the original, you now have 4 queries to search with.
  • For each query, the function runs both vector search and BM25 search. That produces 8 ranked lists total (4 queries x 2 retrievers).
  • All 8 lists are fused with RRF into a single ranking, then the top 20 candidates are reranked down to 5 with the cross-encoder.
  • Save retrieval.py.

✔️ Awesome, I've got everything!

Great. Make sure you've saved retrieval.py with the new function at the bottom.

ⓧ I'd like to double check the full code

from langchain_nvidia_ai_endpoints import NVIDIARerank, ChatNVIDIA
from langchain_core.documents import Document
from config import (
    NVIDIA_API_KEY, RERANK_MODEL, LLM_MODEL,
    TOP_K_RETRIEVAL, TOP_K_FINAL, RRF_K, NUM_QUERY_EXPANSIONS
)


def vector_search(vector_store, query, k=TOP_K_RETRIEVAL):
    """Perform dense vector search."""
    results = vector_store.similarity_search(query, k=k)
    return results


def bm25_search(bm25_index, chunks, query, k=TOP_K_RETRIEVAL):
    """Perform BM25 keyword search."""
    tokenized_query = query.lower().split()
    scores = bm25_index.get_scores(tokenized_query)

    scored_chunks = list(zip(scores, chunks))
    scored_chunks.sort(key=lambda x: x[0], reverse=True)

    return [chunk for _, chunk in scored_chunks[:k]]


def reciprocal_rank_fusion(ranked_lists, k=RRF_K):
    """Fuse multiple ranked lists using RRF."""
    rrf_scores = {}

    for ranked_list in ranked_lists:
        for rank, doc in enumerate(ranked_list):
            doc_id = doc.page_content
            if doc_id not in rrf_scores:
                rrf_scores[doc_id] = {"score": 0.0, "doc": doc}
            rrf_scores[doc_id]["score"] += 1.0 / (k + rank + 1)

    sorted_results = sorted(
        rrf_scores.values(),
        key=lambda x: x["score"],
        reverse=True
    )
    return [item["doc"] for item in sorted_results]


def hybrid_retrieve(vector_store, bm25_index, chunks, query, k=TOP_K_RETRIEVAL):
    """Combine vector and BM25 search with RRF."""
    vector_results = vector_search(vector_store, query, k=k)
    bm25_results = bm25_search(bm25_index, chunks, query, k=k)

    fused_results = reciprocal_rank_fusion(
        [vector_results, bm25_results], k=RRF_K
    )
    return fused_results[:k]


def rerank(query, documents, top_n=TOP_K_FINAL):
    """Rerank documents using NVIDIA cross-encoder."""
    reranker = NVIDIARerank(
        model=RERANK_MODEL,
        api_key=NVIDIA_API_KEY,
        top_n=top_n,
        truncate="END"
    )
    reranked = reranker.compress_documents(
        query=query,
        documents=documents
    )
    return reranked


def expand_query(query):
    """Generate alternative query phrasings using LLM."""
    llm = ChatNVIDIA(
        model=LLM_MODEL,
        api_key=NVIDIA_API_KEY,
        temperature=0.7,
        max_tokens=256
    )

    prompt = f"""Generate {NUM_QUERY_EXPANSIONS} alternative phrasings of the following question.
Each phrasing should capture the same intent but use different words.
Return only the alternative questions, one per line, without numbering.

Question: {query}"""

    response = llm.invoke(prompt)
    alternatives = [
        line.strip() for line in response.content.strip().split("\n")
        if line.strip()
    ]
    return alternatives[:NUM_QUERY_EXPANSIONS]


def advanced_retrieve(vector_store, bm25_index, chunks, query):
    """Full advanced pipeline: expand -> hybrid search -> RRF -> rerank."""
    all_queries = [query] + expand_query(query)

    all_ranked_lists = []
    for q in all_queries:
        vector_results = vector_search(vector_store, q, k=TOP_K_RETRIEVAL)
        bm25_results = bm25_search(bm25_index, chunks, q, k=TOP_K_RETRIEVAL)
        all_ranked_lists.append(vector_results)
        all_ranked_lists.append(bm25_results)

    fused = reciprocal_rank_fusion(all_ranked_lists, k=RRF_K)
    candidates = fused[:TOP_K_RETRIEVAL]

    reranked = rerank(query, candidates, top_n=TOP_K_FINAL)
    return reranked
Create the generation layer

Retrieval alone doesn't answer questions. You need a generation layer that takes the retrieved chunks, formats them as context, and sends them to an LLM for a grounded answer.

  • Create a new file called rag.py in your project folder.
  • Add the following code to build the generation function:
from langchain_nvidia_ai_endpoints import ChatNVIDIA
from config import NVIDIA_API_KEY, LLM_MODEL


def generate_answer(query, context_docs):
    """Generate an answer using retrieved context."""
    llm = ChatNVIDIA(
        model=LLM_MODEL,
        api_key=NVIDIA_API_KEY,
        temperature=0.1,
        max_tokens=512
    )

    context = "\n\n".join([doc.page_content for doc in context_docs])

    prompt = f"""Answer the following question based only on the provided context.
If the context doesn't contain enough information, say so.

Context:
{context}

Question: {query}

Answer:"""

    response = llm.invoke(prompt)
    return response.content

What does this code do?

  • ChatNVIDIA initializes the meta/llama-3.1-8b-instruct model with low temperature (0.1) for consistent, factual responses.
  • The function joins all retrieved chunks into one context string and builds a prompt that instructs the LLM to answer only from the provided context.
  • This grounding instruction prevents the model from hallucinating answers that aren't supported by retrieved documents.
  • Save rag.py.

Seeing an import error?

Make sure the file is named exactly rag.py (not RAG.py or Rag.py). Python imports are case-sensitive.

Help me debug my import error in rag.py

✔️ Awesome, I've got everything!

Great. Make sure you've saved rag.py in the same folder as your other project files.

ⓧ I'd like to double check the full code

from langchain_nvidia_ai_endpoints import ChatNVIDIA
from config import NVIDIA_API_KEY, LLM_MODEL


def generate_answer(query, context_docs):
    """Generate an answer using retrieved context."""
    llm = ChatNVIDIA(
        model=LLM_MODEL,
        api_key=NVIDIA_API_KEY,
        temperature=0.1,
        max_tokens=512
    )

    context = "\n\n".join([doc.page_content for doc in context_docs])

    prompt = f"""Answer the following question based only on the provided context.
If the context doesn't contain enough information, say so.

Context:
{context}

Question: {query}

Answer:"""

    response = llm.invoke(prompt)
    return response.content
Create the comparison script and run it

The final piece is main.py. This script builds the indices, runs the same queries through a basic vector-only pipeline and your full advanced pipeline, then prints the results side-by-side so you can compare them directly.

  • Create a new file called main.py in your project folder.
  • Add the following code to wire the full comparison together:
from ingestion import create_chunks, build_vector_store, build_bm25_index
from retrieval import vector_search, advanced_retrieve
from rag import generate_answer


def main():
    print("=" * 60)
    print("ADVANCED RAG PIPELINE WITH NVIDIA NIM")
    print("=" * 60)

    # Build indices
    print("\n[1] Building document indices...")
    chunks = create_chunks()
    print(f"    Created {len(chunks)} chunks from documents")

    vector_store = build_vector_store(chunks)
    print("    Built FAISS vector store")

    bm25_index = build_bm25_index(chunks)
    print("    Built BM25 index")

    # Test queries
    test_queries = [
        "What is the RRF formula and how does the k parameter work?",
        "How does BM25 handle document length normalization?",
        "What are the latency tradeoffs of cross-encoder reranking?",
    ]

    for query in test_queries:
        print("\n" + "=" * 60)
        print(f"QUERY: {query}")
        print("=" * 60)

        # Basic RAG (vector only)
        print("\n--- BASIC RAG (Vector Search Only) ---")
        basic_docs = vector_search(vector_store, query, k=5)
        basic_answer = generate_answer(query, basic_docs)
        print(f"Sources: {[d.metadata['source'] for d in basic_docs[:3]]}")
        print(f"Answer: {basic_answer[:200]}...")

        # Advanced RAG
        print("\n--- ADVANCED RAG (Hybrid + Rerank + Query Expansion) ---")
        advanced_docs = advanced_retrieve(
            vector_store, bm25_index, chunks, query
        )
        advanced_answer = generate_answer(query, advanced_docs)
        print(f"Sources: {[d.metadata['source'] for d in advanced_docs[:3]]}")
        print(f"Answer: {advanced_answer[:200]}...")


if __name__ == "__main__":
    main()

What does this script do?

  • First, it builds all indices (chunks, FAISS vector store, and BM25 index) from your document collection.
  • The basic RAG path uses only vector_search() with k=5 to retrieve documents, then generates an answer from those.
  • The advanced RAG path uses advanced_retrieve() which runs the full pipeline: query expansion, hybrid search across all expanded queries, RRF fusion of all ranked lists, then cross-encoder reranking.
  • For each query, it prints the top sources and the first 200 characters of the answer from both pipelines so you can compare quality directly.
  • Save main.py.
  • Run the pipeline by executing this command in your terminal:
python main.py
  • You should see output showing the pipeline building indices, then for each of the 3 test queries, a side-by-side comparison of basic RAG sources and answers versus advanced RAG sources and answers.

What should I look for in the output?

Compare the Sources lines between basic and advanced. The advanced pipeline should pull from more targeted documents. For example, the RRF query should retrieve the "Reciprocal Rank Fusion" document as its top source in the advanced path.

The advanced answers should be more precise and directly address the question. The basic path might pull from tangentially related documents and produce vaguer responses.

Getting an API error or timeout?

The advanced pipeline makes multiple API calls (embedding, query expansion, reranking, generation). If you see a timeout, wait a moment and try again. The NVIDIA API free tier has rate limits.

Check that your NVIDIA_API_KEY in .env is valid and starts with nvapi-.

Help me debug an API error when running my advanced RAG pipeline

✔️ Awesome, I've got everything!

Great. Double check you've saved main.py and that the output shows both basic and advanced results for all 3 queries.

ⓧ I'd like to double check the full code

from ingestion import create_chunks, build_vector_store, build_bm25_index
from retrieval import vector_search, advanced_retrieve
from rag import generate_answer


def main():
    print("=" * 60)
    print("ADVANCED RAG PIPELINE WITH NVIDIA NIM")
    print("=" * 60)

    # Build indices
    print("\n[1] Building document indices...")
    chunks = create_chunks()
    print(f"    Created {len(chunks)} chunks from documents")

    vector_store = build_vector_store(chunks)
    print("    Built FAISS vector store")

    bm25_index = build_bm25_index(chunks)
    print("    Built BM25 index")

    # Test queries
    test_queries = [
        "What is the RRF formula and how does the k parameter work?",
        "How does BM25 handle document length normalization?",
        "What are the latency tradeoffs of cross-encoder reranking?",
    ]

    for query in test_queries:
        print("\n" + "=" * 60)
        print(f"QUERY: {query}")
        print("=" * 60)

        # Basic RAG (vector only)
        print("\n--- BASIC RAG (Vector Search Only) ---")
        basic_docs = vector_search(vector_store, query, k=5)
        basic_answer = generate_answer(query, basic_docs)
        print(f"Sources: {[d.metadata['source'] for d in basic_docs[:3]]}")
        print(f"Answer: {basic_answer[:200]}...")

        # Advanced RAG
        print("\n--- ADVANCED RAG (Hybrid + Rerank + Query Expansion) ---")
        advanced_docs = advanced_retrieve(
            vector_store, bm25_index, chunks, query
        )
        advanced_answer = generate_answer(query, advanced_docs)
        print(f"Sources: {[d.metadata['source'] for d in advanced_docs[:3]]}")
        print(f"Answer: {advanced_answer[:200]}...")


if __name__ == "__main__":
    main()

Secret mission

Add Retrieval Evaluation Metrics

Your advanced pipeline feels better, but how much better? In this secret mission, you'll build a quantitative evaluation harness that measures exactly how much your hybrid + rerank + expansion pipeline outperforms basic vector search. You'll implement MRR@5 and Recall@5, the same metrics used in industry retrieval benchmarks.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

This project runs entirely locally, and the NVIDIA API Catalog free tier has no ongoing costs. There is nothing that will charge you if you leave it as-is.

Resources you used:

  • Local project folder containing main.py, config.py, retrieval.py, ingestion.py, rag.py, documents.py, evaluate.py, and .env.
  • Installed pip packages (langchain-nvidia-ai-endpoints, rank-bm25, faiss-cpu, langchain, langchain-community, python-dotenv).
  • NVIDIA API key stored in your local .env file.

Keep everything running

No action needed. Choose this if you plan to keep experimenting with your advanced RAG pipeline or want to tweak retrieval parameters.

  • Your project files stay on your Desktop.
  • Your pip packages remain installed in your Python environment.
  • Your NVIDIA API key stays in your .env file for future use.

Pause - I'll come back to this later

There are no running processes or active services to pause. Your project is already in a paused state since it only runs when you execute the scripts.

  • Your files and packages will be waiting for you when you return.
  • To resume, open a terminal in your project folder and run python main.py again.

Delete - I don't want to use this again

Remove all project files and uninstall the packages you installed for this project.

  • Delete your project folder by running these commands in your terminal:

macOS

cd ~/Desktop
rm -rf advanced-rag-pipeline

Windows

cd ~\Desktop
rmdir /s /q advanced-rag-pipeline
  • Uninstall the pip packages by running:
pip uninstall langchain-nvidia-ai-endpoints rank-bm25 faiss-cpu langchain langchain-community python-dotenv -y
  • Your NVIDIA API key was stored only in the .env file you just deleted. No further action needed for the key.

Nice Work!

Nice Work!

You just built a production-grade advanced RAG pipeline powered by NVIDIA NIM. This is the kind of retrieval architecture that separates demo-quality RAG from systems that actually work in production.

Here's what you learned:

  • Built a hybrid retriever combining BM25 keyword search with NVIDIA dense vector embeddings, fused together using Reciprocal Rank Fusion to merge ranked lists fairly regardless of score scales.
  • Added a cross-encoder reranking layer using NVIDIA's Nemotron reranker to dramatically improve retrieval precision on your top candidates.
  • Implemented multi-query expansion to overcome vocabulary limitations, then compared basic vs. advanced RAG side-by-side to see the measurable difference.
  • Secret Mission: Added quantitative evaluation metrics (MRR@5 and Recall@5) to measure exactly how much better the advanced pipeline performs.

Ready to quiz yourself?