Build a PDF Question-Answering System

Build a local RAG pipeline that answers questions about your PDF documents.

Introduction

30 Second Summary

You have a folder of PDF reports sitting on your computer, and someone asks you a question buried somewhere on page 47. Instead of skimming every page, what if you could just type the question and get the exact answer pulled straight from the text?

In this project, you will build a local command-line tool that reads your PDFs, understands their content, and answers questions in plain English using Retrieval-Augmented Generation.

What You'll Build

You'll type a natural-language question in your terminal, and within seconds the tool retrieves the exact relevant passages from your PDFs and generates a grounded answer you can trust.

By the end of this project, you'll have:

  • A working RAG pipeline where you ask any question about your PDFs and get an accurate answer pulled from the actual text, not hallucinated.
  • Full visibility into retrieval quality by viewing exactly which text chunks were retrieved and their similarity scores, so you can debug why an answer is good or bad.
  • Firsthand proof of how chunking strategy controls answer accuracy. You'll watch a naive approach fail at chunk boundaries, then fix it with overlapping chunks.
  • Secret Mission: Add page-number metadata and source citations so every answer tells you exactly which PDF page to check.

Are there any prerequisites?

You should be comfortable with Python basics like functions, loops, and file I/O. You'll also need a free Google AI Studio API key, which you can generate at aistudio.google.com.

Before We Start

Before any code hits the screen, let's lock in what you're about to build and why it matters. Understanding the problem RAG solves will make every step that follows click into place.

Set Up Your Environment

Before you can build a RAG pipeline, you need the right tools installed and talking to each other.

In this step, you'll create an isolated Python project, install the three core libraries, and confirm your Gemini API connection works.

In this step, get ready to:
  • Create a project folder with a virtual environment.
  • Install PyMuPDF, ChromaDB, and google-genai.
  • Verify your Gemini API connection with a test script.
Create your project folder and virtual environment

A virtual environment keeps your project's libraries separate from the rest of your system. This prevents version conflicts between different Python projects on your machine.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir pdf-rag
cd pdf-rag
  • Create a virtual environment by running:
python3 -m venv venv
  • Confirm the virtual environment was created by running:
ls

You should see a venv folder listed in the output.

Don't see the venv folder?

Make sure you ran the python3 -m venv venv command from inside the pdf-rag folder, not from your Desktop.

If you see python3: command not found, try python instead (some systems use this name).

Help me debug this

  • Open VS Code in the project folder by running:
code .

Getting command not found for code?

If code is not recognized, open VS Code manually.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Visual Studio Code and press Enter to open it.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Select the pdf-rag folder on your Desktop.

Help me open VS Code in my project folder

  • Open VS Code's integrated terminal by pressing Ctrl+` (backtick key, usually above Tab).

From this point on, run all terminal commands in VS Code's integrated terminal.

  • Activate the virtual environment by running:

macOS

source venv/bin/activate

Windows

venv\Scripts\Activate.ps1

Your terminal prompt now shows (venv) at the beginning. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

On Windows PowerShell, you may see an execution policy error. Run this first, then try the activation command again:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

On macOS, make sure you typed source before the path. Without it, the activation won't work.

Help me activate my virtual environment

Install the project dependencies

Your RAG pipeline depends on three libraries. PyMuPDF extracts text from PDF files. ChromaDB stores and searches text embeddings. The google-genai SDK connects you to Google's Gemini models for embeddings and text generation.

  • Install all three libraries by running:
pip install pymupdf==1.28.0 chromadb==1.5.9 google-genai==2.12.1

What does this command do?

The == syntax pins each library to an exact version. This guarantees everyone following this project gets the same behaviour, regardless of when they run it.

The install downloads each library plus its dependencies. You'll see progress bars and a success message when it finishes.

Before you run the next command, what do you think will happen if one of the libraries failed to install?

  • Verify all three libraries installed correctly by running:
python -c "import pymupdf; import chromadb; from google import genai; print('All imports OK')"

You should see All imports OK printed. If any library was missing, Python would throw an ImportError instead.

Seeing an ImportError?

Check that (venv) appears in your terminal prompt. If it doesn't, the packages were installed globally instead of into your virtual environment. Activate the venv and re-run the pip install command.

If you see a permissions error, make sure you're using the pip inside the virtual environment, not your system pip.

Help me fix this ImportError

Set your API key and verify the Gemini connection

The google-genai SDK needs an API key to authenticate with Google AI Studio. You'll store it as an environment variable so your code can read it without hardcoding secrets.

  • Set your API key as an environment variable by running:

macOS

export GOOGLE_API_KEY="your-api-key-here"

Windows

$env:GOOGLE_API_KEY = "your-api-key-here"

Where do I find my API key?

Your API key is in Google AI Studio. Go to aistudio.google.com, click Get API key in the left sidebar, and copy the key you already created.

Replace your-api-key-here with your actual key. Keep the quotes around it.

  • In VS Code's Explorer sidebar, right-click the empty area and select New File.
  • Name the file rag.py and press Enter.
  • Paste the following code into rag.py:
import os
from google import genai

GOOGLE_API_KEY = os.environ.get("GOOGLE_API_KEY", "your-api-key-here")
client = genai.Client(api_key=GOOGLE_API_KEY)

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Say hello!",
)
print(response.text)

What does this code do?

This script reads your API key from the GOOGLE_API_KEY environment variable you just set. If the variable isn't found, it falls back to the placeholder string.

It creates a Gemini client and sends a simple prompt. If the connection works, Gemini's response prints to your terminal.

  • Save rag.py with Cmd+S (macOS) or Ctrl+S (Windows).

Before you run this, what do you expect to see if your API key is valid?

  • Run the test script:
python rag.py

You should see a friendly greeting from Gemini printed in your terminal. The exact wording varies each time, but any response proves your connection works.

Seeing an API key error?

If you see a message about an invalid API key, double-check that you replaced your-api-key-here with your actual key in the export command. The quotes must wrap the key.

If the environment variable isn't being picked up, make sure you set it in the same terminal session where you're running python rag.py. Opening a new terminal loses the variable.

Help me debug my API key setup

✔️ Awesome, I've got everything!

Great. Double check you've saved rag.py and confirm the Gemini greeting printed successfully.

ⓧ I'd like to double check the full code

Here's the complete rag.py file you should have at this point:

import os
from google import genai

GOOGLE_API_KEY = os.environ.get("GOOGLE_API_KEY", "your-api-key-here")
client = genai.Client(api_key=GOOGLE_API_KEY)

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Say hello!",
)
print(response.text)

Your environment is locked in. You have a working project folder, all three libraries installed, and a confirmed Gemini API connection. Next up, you'll extract text from real PDFs and start breaking it into chunks.

Extract PDF Text and Chunk It

Your RAG pipeline needs raw text before it can search or answer anything. PDFs are binary files, so you need a library to extract readable text from each page.

In this step, you will write extraction functions and a naive chunking strategy that splits text into fixed-size pieces. This first attempt is deliberately simple. You will see its limitations firsthand when text gets cut mid-sentence at chunk boundaries.

In this step, get ready to:
  • Create a PDF folder and add documents to process.
  • Write extraction functions that pull text from every page of every PDF.
  • Write a naive chunking function and observe where it cuts sentences.
Create your PDF folder and add documents

Your RAG system needs a folder of PDFs to ingest. Any documents will work. Reports, research papers, or documentation are ideal because they contain structured information you can query later.

  • Create the PDF folder by running this command in your terminal:
mkdir pdfs
  • Add 2-3 PDF files into the pdfs/ folder. You can drag files from your Downloads or copy them from Finder (macOS) or File Explorer (Windows).
  • Confirm your files are in place by running:
ls pdfs

You should see your PDF filenames listed in the output.

Pick PDFs with substance

Choose documents with at least a few pages of text. Single-page PDFs or image-only scans will not produce enough text to demonstrate chunking boundaries effectively.

Don't see any files listed?

Make sure you placed PDF files inside the pdfs/ folder that is inside your project folder. If you created the folder in the wrong location, delete it and re-run mkdir pdfs from your project directory.

help me find where my pdfs folder was created

Write the PDF extraction functions

PyMuPDF reads PDF pages and returns their text content as plain strings. You will write two functions: one that extracts text from a single PDF, and one that processes every PDF in a folder.

  • Replace the entire contents of rag.py with the following imports, configuration, and extraction function:
import os
import pymupdf
import chromadb
from google import genai
from google.genai import types

# --- Configuration ---
GOOGLE_API_KEY = os.environ.get("GOOGLE_API_KEY", "your-api-key-here")
client = genai.Client(api_key=GOOGLE_API_KEY)

CHUNK_SIZE = 500
CHUNK_OVERLAP = 100
COLLECTION_NAME = "pdf_knowledge_base"
PDF_FOLDER = "./pdfs"


# --- PDF Extraction ---
def extract_text_from_pdf(pdf_path):
    """Extract all text from a PDF file."""
    doc = pymupdf.open(pdf_path)
    text = ""
    for page in doc:
        text += page.get_text()
    doc.close()
    return text

What does this code do?

  • The configuration constants define your chunk size (500 characters), overlap (100 characters, used later), collection name for the vector database, and the path to your PDF folder.
  • extract_text_from_pdf() opens a PDF with PyMuPDF, loops through every page, concatenates the text, then closes the file. It returns one big string of all the text in that PDF.
  • Add the extract_all_pdfs() function and a main block below the code you just pasted:

def extract_all_pdfs(pdf_folder):
    """Extract text from all PDFs in a folder."""
    all_text = {}
    for filename in os.listdir(pdf_folder):
        if filename.lower().endswith(".pdf"):
            filepath = os.path.join(pdf_folder, filename)
            all_text[filename] = extract_text_from_pdf(filepath)
            print(f"  Extracted {len(all_text[filename]):,} characters from {filename}")
    return all_text


if __name__ == "__main__":
    print("Extracting text from PDFs...")
    pdf_texts = extract_all_pdfs(PDF_FOLDER)

What does this code do?

  • extract_all_pdfs() scans a folder for any file ending in .pdf, calls the single-file extractor on each one, and returns a dictionary mapping filenames to their full extracted text.
  • The if __name__ == "__main__": block runs the extraction when you execute the script directly. You will expand this block in the next substep.
  • Save rag.py and run the script to verify extraction works:
python rag.py

You should see "Extracting text from PDFs..." followed by a line for each PDF showing how many characters were extracted.

Seeing "No module named pymupdf"?

Make sure your virtual environment is activated. You should see the environment name in parentheses at the start of your terminal prompt. If not, activate it again before running the script.

help me fix my pymupdf import error

Add chunking and observe boundary cuts

Extracted text can be thousands of characters long. An embedding model has a token limit, and searching a single giant string is not useful. Chunking splits text into smaller pieces that can each be embedded and retrieved independently.

Your first chunking strategy is deliberately naive: fixed 500-character slices with zero overlap. This will cut sentences in the middle, which you will see in the output.

  • In rag.py, add the chunk_text() function above the if __name__ == "__main__": block:

# --- Chunking ---
def chunk_text(text, chunk_size=CHUNK_SIZE, overlap=0):
    """Split text into fixed-size chunks with no overlap."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        if chunk.strip():
            chunks.append(chunk)
        start += chunk_size
    return chunks

What does this code do?

  • The function walks through the text in steps of chunk_size (500 characters). Each slice becomes one chunk.
  • The overlap=0 parameter means chunks do not share any text with their neighbors. If a sentence spans the boundary between chunk 4 and chunk 5, it gets sliced in half.
  • The if chunk.strip(): check skips empty chunks that would occur if the text ends with whitespace.
  • Now replace the if __name__ == "__main__": block with this expanded version that chunks, summarizes, and displays boundary content:
if __name__ == "__main__":
    print("Extracting text from PDFs...")
    pdf_texts = extract_all_pdfs(PDF_FOLDER)
    for filename, text in pdf_texts.items():
        chunks = chunk_text(text)
        print(f"\n{filename}: {len(text):,} chars, {len(chunks)} chunks")
        for i, chunk in enumerate(chunks[:3]):
            print(f"  Chunk {i}: {len(chunk)} chars")
            print(f"    Preview: {chunk[:80]!r}")
        if len(chunks) >= 5:
            print(f"\nBoundary chunks (4 and 5):")
            print(f"  Chunk 4 end: {chunks[3][-100:]!r}")
            print(f"  Chunk 5 start: {chunks[4][:100]!r}")

What does the main block do?

  • For each PDF, it chunks the extracted text and prints the filename, total character count, and number of chunks created.
  • It previews the first 3 chunks so you can see what each chunk contains.
  • The boundary display shows the last 100 characters of chunk 4 and the first 100 characters of chunk 5. This is where you will see a sentence sliced in half.
  • Save rag.py.

Before you run this, what do you think the boundary output will look like? Will chunk 4 end neatly at the end of a sentence, or will it cut off mid-word?

  • Run the script to see your chunks:
python rag.py

You should see each PDF's filename, total characters, and chunk count. The first 3 chunks show previews. The boundary display reveals text cut mid-sentence between chunks 4 and 5.

Why does the boundary cut matter?

When a sentence is split across two chunks, neither chunk contains the full thought. If someone asks a question and the answer sits right at that boundary, the retrieval system will pull back an incomplete fragment. You will hit this exact problem in the next step when you wire up question-answering.

Seeing "No PDFs found"?

The script looks for files in ./pdfs relative to where you run the command. Make sure your terminal is in the project folder (the one containing rag.py) and that your pdfs/ folder contains actual PDF files.

help me debug why my PDFs aren't being found

✔️ Awesome, I've got everything!

Great. Double check you have saved rag.py and that your output shows chunk previews and boundary content.

ⓧ I'd like to double check the full code

Scroll up and confirm your rag.py contains each of these sections in order:

  • Imports (os, pymupdf, chromadb, genai, types).
  • Configuration section with GOOGLE_API_KEY, client, CHUNK_SIZE, CHUNK_OVERLAP, COLLECTION_NAME, and PDF_FOLDER.
  • extract_text_from_pdf() function that opens a PDF and returns all text.
  • extract_all_pdfs() function that loops through a folder and extracts each PDF.
  • chunk_text() function with overlap=0.
  • if __name__ == "__main__": block that extracts, chunks, prints summaries, and displays boundary content.

You have a working extraction and chunking pipeline. Next, you will turn these chunks into vectors and store them in a database so you can search them by meaning.

Embed Chunks and Store in ChromaDB

You now have raw text chunks sitting in Python lists. They're searchable by exact keyword match, but not by meaning.

To ask a question like "What were the key findings?" and get back the right chunks, you need to convert each chunk into a numerical vector that captures its semantic meaning. Then you store those vectors in a database purpose-built for finding the closest matches. That's what this step builds.

In this step, get ready to:
  • Write an embedding function that converts text into vectors using Gemini.
  • Build a vector store that chunks, embeds, and stores all PDFs in ChromaDB.
  • Write a retrieval function and test it with a real query against your PDFs.
Write the embedding function

An embedding is a list of numbers (a vector) that represents the meaning of a piece of text. Texts with similar meanings produce vectors that are close together in number space.

  • In rag.py, find the end of the chunk_text() function.
  • Add the following embed_texts() function below it:
# --- Embedding and Storage ---
def embed_texts(texts, task_type="RETRIEVAL_DOCUMENT"):
    """Embed a list of texts using Gemini."""
    result = client.models.embed_content(
        model="gemini-embedding-001",
        contents=texts,
        config=types.EmbedContentConfig(
            task_type=task_type,
            output_dimensionality=768,
        ),
    )
    return [e.values for e in result.embeddings]

What does this code do?

This function sends text to Gemini's embedding model and gets back a list of 768-dimensional vectors.

  • The task_type parameter tells Gemini how the text will be used. RETRIEVAL_DOCUMENT optimizes embeddings for stored documents. RETRIEVAL_QUERY optimizes for search queries.
  • Using different task types for documents vs queries is called asymmetric search. It improves retrieval accuracy because queries and documents have different linguistic patterns.
  • The output_dimensionality=768 sets the vector size. Larger dimensions capture more nuance but use more memory.
  • Next, add the first part of the create_vector_store() function directly below embed_texts():
def create_vector_store(pdf_texts):
    """Chunk, embed, and store all PDF texts in ChromaDB."""
    chroma_client = chromadb.PersistentClient(path="./chroma_db")

    # Clean slate for re-runs
    existing = [c.name for c in chroma_client.list_collections()]
    if COLLECTION_NAME in existing:
        chroma_client.delete_collection(name=COLLECTION_NAME)

    collection = chroma_client.create_collection(name=COLLECTION_NAME)

    all_chunks = []
    all_ids = []
    all_metadatas = []

    for filename, text in pdf_texts.items():
        chunks = chunk_text(text)
        for i, chunk in enumerate(chunks):
            all_chunks.append(chunk)
            all_ids.append(f"{filename}_chunk_{i}")
            all_metadatas.append({"source": filename, "chunk_index": i})

What does this code do?

This is the setup portion of your vector store builder.

  • The PersistentClient saves the database to disk at ./chroma_db. Your embeddings survive between script runs.
  • The "clean slate" block deletes any existing collection with the same name so you can safely re-run without duplicates.
  • The loop walks every PDF's text, chunks it, and builds parallel lists of chunks, unique IDs, and metadata dictionaries.

Before continuing with the rest of the function, verify that the embedding function connects to the Gemini API correctly.

  • Save rag.py.
  • Test the embedding function by running this command:
python -c "from rag import embed_texts; print(len(embed_texts(['hello world'])[0]))"

You should see 768 printed. That confirms your API key works and Gemini returned a 768-dimensional vector.

Seeing an error instead of 768?

  • If you see an authentication error, double check that your GOOGLE_API_KEY environment variable is set in the same terminal session.
  • If you see an import error, make sure your virtual environment is activated and all dependencies are installed.

Help me debug my Gemini embedding function

Embed chunks in batches and store them in ChromaDB

The Gemini API has rate limits, so you need to send chunks in batches rather than all at once. The second half of create_vector_store() handles this.

  • In rag.py, find the last line of create_vector_store() you just wrote:
            all_metadatas.append({"source": filename, "chunk_index": i})
  • Add the batch embedding loop directly below it, still inside the function:
    # Embed and store in batches
    batch_size = 50
    for i in range(0, len(all_chunks), batch_size):
        batch_end = min(i + batch_size, len(all_chunks))
        batch_chunks = all_chunks[i:batch_end]
        batch_ids = all_ids[i:batch_end]
        batch_metadatas = all_metadatas[i:batch_end]

        embeddings = embed_texts(batch_chunks, task_type="RETRIEVAL_DOCUMENT")

        collection.add(
            ids=batch_ids,
            embeddings=embeddings,
            documents=batch_chunks,
            metadatas=batch_metadatas,
        )
        print(f"  Embedded batch {i // batch_size + 1} ({batch_end}/{len(all_chunks)} chunks)")

    print(f"\nStored {len(all_chunks)} chunks in ChromaDB")
    return collection

What does this code do?

  • The loop processes 50 chunks at a time. Each batch gets embedded via the Gemini API, then stored in ChromaDB with collection.add().
  • Each chunk is stored alongside its embedding vector, the original text (documents), and metadata tracking which file it came from and its position in that file.
  • The progress print lets you watch batches complete. With a few hundred chunks, this takes about 10-20 seconds.
  • Now replace the entire if __name__ == "__main__": block at the bottom of rag.py with this updated version:
if __name__ == "__main__":
    print("Extracting text from PDFs...")
    pdf_texts = extract_all_pdfs(PDF_FOLDER)
    print(f"Extracted text from {len(pdf_texts)} PDF(s)\n")

    print("Chunking and embedding...")
    collection = create_vector_store(pdf_texts)
  • Save rag.py.
  • Run the script to embed and store all your chunks:
python rag.py

You should see batch progress messages followed by a summary like Stored 42 chunks in ChromaDB (your number depends on your PDFs).

Script hangs or you see rate limit errors?

  • If the script hangs on the first batch, check your internet connection. The Gemini API requires network access.
  • If you see a 429 rate limit error, reduce batch_size from 50 to 20 in create_vector_store() and re-run.

Help me fix rate limit or timeout errors when embedding chunks

Write a retrieval function and test it

Your chunks are now stored as vectors. To search them, you embed a question the same way, then ask ChromaDB to find the vectors closest to your question's vector.

  • In rag.py, add the following retrieve() function below create_vector_store():
# --- Retrieval ---
def retrieve(collection, query, n_results=5):
    """Retrieve the most relevant chunks for a query."""
    query_embedding = embed_texts([query], task_type="RETRIEVAL_QUERY")[0]

    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=n_results,
        include=["documents", "metadatas", "distances"],
    )
    return results

What does this code do?

  • The query is embedded with task_type="RETRIEVAL_QUERY" to match the asymmetric search pattern. Documents were stored with RETRIEVAL_DOCUMENT.
  • The collection.query() call searches the vector database and returns the N closest chunks. The distances field shows how far each result is from your query. Lower distance means a better match.
  • Now add the retrieval test to your __main__ block. Find this line:
    collection = create_vector_store(pdf_texts)
  • Add the following test code directly below it, still inside the if __name__ block:
    print("\nTesting retrieval...")
    test_query = "What is this document about?"
    results = retrieve(collection, test_query)
    for doc, metadata, distance in zip(
        results["documents"][0],
        results["metadatas"][0],
        results["distances"][0],
    ):
        print(f"  [{metadata['source']} chunk {metadata['chunk_index']}] distance={distance:.4f}")
        print(f"  {doc[:100]!r}\n")
  • Save rag.py.

Before you run this, what do you think the distance scores will look like for a broad question like "What is this document about?" Will most chunks score similarly, or will there be a big gap between the best match and the rest?

  • Run the script again to test retrieval:
python rag.py

You should see the embedding progress again, then the top retrieved chunks printed with their source file, chunk index, and distance scores. Lower distances indicate higher relevance.

Not seeing retrieval results?

  • If you see an error about the collection, make sure the retrieve() function is placed above the if __name__ block, not inside another function.
  • If you see IndexError, check that your pdfs/ folder still contains PDF files with extractable text.

Help me debug my ChromaDB retrieval function

Why do documents and queries use different task types?

When you search Google, your query ("best pizza near me") looks nothing like the documents it matches (a restaurant's full menu page). Asymmetric search accounts for this mismatch.

The RETRIEVAL_DOCUMENT task type optimizes the vector for representing stored content. The RETRIEVAL_QUERY task type optimizes for matching questions to relevant content. Using both gives better results than treating documents and queries identically.

✔️ Awesome, I've got everything!

Great. Double check you have saved rag.py and confirmed the retrieval output shows chunks with distance scores.

ⓧ I'd like to double check the full code

Your rag.py should now contain these sections in order. Scroll through your file and confirm each is present:

  • Imports and Configuration (GOOGLE_API_KEY, client, CHUNK_SIZE, CHUNK_OVERLAP, COLLECTION_NAME, PDF_FOLDER).
  • PDF Extraction section: extract_text_from_pdf() and extract_all_pdfs().
  • Chunking section: chunk_text() with no overlap.
  • Embedding and Storage section: embed_texts() and create_vector_store().
  • Retrieval section: retrieve().
  • The if __name__ == "__main__" block that extracts, stores, and tests retrieval.

Your RAG pipeline now extracts, chunks, embeds, stores, and retrieves. Next up, you will wire retrieval to Gemini's language model so it can actually answer questions using the chunks it finds.

Ask Questions and Hit the Chunk Boundary Problem

You have a vector database full of embedded chunks, and your retrieve() function can find the most relevant ones for any query.

Now you need the final piece: wiring retrieval to an LLM so it can read those chunks and generate a real answer. In this step, you will write a function that retrieves relevant chunks, feeds them to Gemini as context, and returns a grounded answer.

In this step, get ready to:
  • Write an answer generation function that combines retrieval with Gemini.
  • Test with a simple factual question and confirm a correct answer.
  • Test with a harder question and observe what happens at chunk boundaries.
Write the answer generation function

The ask_question() function connects your retrieval system to Gemini's generation capability. It retrieves the top chunks, formats them with source labels and similarity scores, then passes everything to the LLM with a prompt that constrains it to answer only from the provided context.

  • In rag.py, find the retrieve() function and scroll below it.
  • Add the first half of the ask_question() function by copying this code below the retrieve() function:
# --- Answer Generation ---
def ask_question(collection, question):
    """Retrieve relevant chunks and generate a grounded answer."""
    results = retrieve(collection, question)

    # Build context from retrieved chunks
    context_parts = []
    for doc, metadata, distance in zip(
        results["documents"][0],
        results["metadatas"][0],
        results["distances"][0],
    ):
        context_parts.append(
            f"[Source: {metadata['source']}, Chunk {metadata['chunk_index']}] "
            f"(similarity: {1 - distance:.3f})\n{doc}"
        )

    context = "\n\n---\n\n".join(context_parts)

What does this code do?

This first half handles context assembly. It calls your existing retrieve() function to get the top-5 chunks, then formats each one with a source label and a similarity score.

The similarity score is calculated as 1 - distance because ChromaDB returns distance (lower is more similar). Converting to similarity (higher is better) makes it easier to interpret at a glance.

  • Continue the ask_question() function by adding the prompt construction and API call directly below the line you just pasted:
    prompt = f"""Based on the following context extracted from PDF documents, answer the question.
If the answer cannot be found in the context, say "I don't have enough information to answer that."
Be specific and cite which source the information comes from.

Context:
{context}

Question: {question}

Answer:"""

    response = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=prompt,
    )

    return response.text

What does the prompt do?

The prompt uses a technique called context injection. It gives Gemini the retrieved chunks as "Context" and explicitly tells it to answer only from that material.

The instruction to say "I don't have enough information" when the context is insufficient prevents the model from hallucinating answers beyond what your PDFs actually contain.

Test with a simple factual question

Your function is ready. Now replace the existing main block with a test that asks a straightforward question clearly contained within a single chunk.

  • In rag.py, find the existing if __name__ == "__main__": block at the bottom of the file.
  • Replace the entire main block with this code:
if __name__ == "__main__":
    print("=== PDF RAG Pipeline ===\n")

    print("Extracting text from PDFs...")
    pdf_texts = extract_all_pdfs(PDF_FOLDER)
    print(f"Extracted text from {len(pdf_texts)} PDF(s)\n")

    print("Chunking and embedding...")
    collection = create_vector_store(pdf_texts)

    print("\n--- Test 1: Simple factual question ---")
    answer1 = ask_question(collection, "What is this document about?")
    print(f"Answer: {answer1}\n")

What does this test?

The question "What is this document about?" is broad enough that the answer will live comfortably within a single 500-character chunk. This proves your end-to-end pipeline works: extract, chunk, embed, retrieve, generate.

  • Save rag.py.
  • Run the script by entering this command in your terminal:
python rag.py

You should see your PDFs being processed, followed by a coherent answer to "What is this document about?" that references actual content from your files.

Getting an error or empty answer?

  • If you see a NameError for ask_question, make sure the function is defined above the if __name__ block, not inside it.
  • If the answer says "I don't have enough information", your chunks may be too small to contain meaningful content. Check that your PDFs have actual text content (not just scanned images).
  • If you see a rate limit error, wait 60 seconds and try again.

Help me debug my ask_question function

Test with a boundary-spanning question

Your pipeline works for simple questions. Now try a question whose answer is likely to span the boundary between two chunks.

  • In rag.py, add the following code at the end of the main block, after the print(f"Answer: {answer1}\n") line:
    print("--- Test 2: Boundary-spanning question ---")
    boundary_question = "What are the key findings?"
    results = retrieve(collection, boundary_question)
    print("Retrieved chunks:")
    for doc, metadata, distance in zip(
        results["documents"][0], results["metadatas"][0], results["distances"][0]
    ):
        print(f"  [{metadata['source']} chunk {metadata['chunk_index']}] {doc[-80:]!r}")
    answer2 = ask_question(collection, boundary_question)
    print(f"Answer: {answer2}\n")

What does this test reveal?

This test prints the last 80 characters of each retrieved chunk. If a chunk was cut off mid-sentence by the naive 500-character boundary, you will see it end abruptly without completing its thought.

The question "What are the key findings?" typically requires information that spans multiple sentences. If those sentences happen to straddle a chunk boundary, the retrieved chunk only contains half the answer.

  • Save rag.py.

Before you run this, think about what you expect to happen. Your chunks are 500 characters with zero overlap. If a key finding starts near character 480 of one chunk, what happens to the rest of the sentence?

  • Run the script again:
python rag.py

Look at the "Retrieved chunks" output. You should see at least one chunk that ends mid-sentence, cut off at exactly 500 characters. The answer for Test 2 is incomplete, vague, or missing information compared to Test 1.

Why is this happening?

This is the chunk boundary problem. Your naive chunking slices text at fixed 500-character intervals regardless of where sentences or paragraphs end. When a key piece of information straddles two chunks, neither chunk contains the complete thought.

The retrieval system finds the most relevant chunk, but that chunk is literally cut off mid-word. Gemini can only answer from what it receives, so it gives an incomplete or inaccurate answer. This is the gap you will fix in the next step.

Both answers look equally good?

  • If both questions produce good answers, your PDFs may be structured so that key information falls neatly within chunk boundaries. Try a more specific question that targets a detail you noticed was split in Step 2 (when you printed chunks at boundaries).
  • Look at the retrieved chunk endings printed in the output. Even if the answer seems OK, you should still see chunks ending mid-sentence. That truncation will bite harder on longer, more complex questions.

Why are my RAG answers not showing the chunk boundary problem?

✔️ Awesome, I've got everything!

Great. Double check you have saved rag.py and confirmed that Test 1 gives a correct answer while Test 2 shows a weaker or incomplete answer.

ⓧ I'd like to double check the full code

Your rag.py file is now quite long. Scroll through and confirm you have each section in this order:

  • The imports and configuration block at the top (import os through PDF_FOLDER = "./pdfs").
  • The extract_text_from_pdf() and extract_all_pdfs() functions under # --- PDF Extraction ---.
  • The chunk_text() function under # --- Chunking --- (still with overlap=0).
  • The embed_texts() and create_vector_store() functions under # --- Embedding and Storage ---.
  • The retrieve() function under # --- Retrieval ---.
  • The new ask_question() function under # --- Answer Generation ---.
  • The if __name__ == "__main__": block with both Test 1 and Test 2.

You have just experienced the core limitation of naive chunking firsthand. Your RAG pipeline works perfectly when information sits cleanly inside a single chunk, but falls apart when answers span a boundary. In the next step, you will fix this by adding overlap between chunks so that boundary information always appears in at least two chunks.

Fix Chunking with Overlap and Re-embed

In the previous step, you saw the chunk boundary problem firsthand. A question whose answer spans two adjacent chunks returned an incomplete response because the relevant text was split across a hard boundary.

The fix is overlap. By making adjacent chunks share characters at their edges, information that sits at a boundary appears in at least two chunks. This ensures retrieval can always find a chunk that contains the full context.

In this step, get ready to:
  • Add an overlap parameter to your chunking function so adjacent chunks share 100 characters.
  • Re-embed all chunks with the new overlap strategy and rebuild the vector database.
  • Re-ask the boundary-spanning question and confirm the answer is now correct and complete.
Update chunk_text() to use overlapping chunks

Right now, each chunk starts exactly where the previous one ended. The overlap strategy makes each new chunk start chunk_size - overlap characters after the previous one, so 100 characters of context are shared between neighbors.

  • In rag.py, find the chunk_text() function. It currently looks like this:
def chunk_text(text, chunk_size=CHUNK_SIZE, overlap=0):
    """Split text into fixed-size chunks with no overlap."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        if chunk.strip():
            chunks.append(chunk)
        start += chunk_size
    return chunks
  • Replace the entire chunk_text() function with this updated version:
def chunk_text(text, chunk_size=CHUNK_SIZE, overlap=CHUNK_OVERLAP):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        if chunk.strip():
            chunks.append(chunk)
        start += chunk_size - overlap
    return chunks

What changed?

  • The default overlap parameter changed from 0 to CHUNK_OVERLAP (which is 100). Every call to chunk_text() now uses overlap by default.
  • The advance line changed from start += chunk_size to start += chunk_size - overlap. Each chunk now starts only 400 characters after the previous one, so 100 characters are shared.

Why does overlap fix the boundary problem?

With zero overlap, a sentence that starts at character 490 and ends at character 520 gets cut in half. Chunk A has the first 10 characters and chunk B has the rest. Neither chunk contains the full sentence.

With 100-character overlap, chunk B starts at character 400 instead of 500. That boundary sentence now appears fully inside chunk B. Any information near an edge appears in at least two chunks.

Replace the main block with the interactive Q&A loop

The test-specific main block from the previous step served its purpose. Now you need a reusable main block that re-embeds with overlap and lets you query interactively.

  • In rag.py, find the entire if __name__ == "__main__": block at the bottom of the file.
  • Delete everything from if __name__ == "__main__": to the end of the file, then replace it with this:
if __name__ == "__main__":
    print("=== PDF RAG Pipeline ===\n")

    # Step 1: Extract
    print("Extracting text from PDFs...")
    pdf_texts = extract_all_pdfs(PDF_FOLDER)
    if not pdf_texts:
        print(f"No PDFs found in {PDF_FOLDER}/ - add some PDF files and try again.")
        exit(1)
    print(f"Extracted text from {len(pdf_texts)} PDF(s)\n")

    # Step 2: Chunk, embed, store
    print("Chunking and embedding...")
    collection = create_vector_store(pdf_texts)

What does this code do?

  • The extraction step loads all PDFs and prints character counts.
  • The if not pdf_texts: guard stops the script early if no PDFs are found, giving a clear error message.
  • Calling create_vector_store() deletes the old collection, re-chunks with overlap, and re-embeds everything. You will see the new chunk count printed.
  • Immediately below that code (still inside the if __name__ block), add the interactive Q&A loop:
    # Step 3: Interactive Q&A
    print("\n=== RAG Pipeline Ready ===")
    print("Ask questions about your PDFs (type 'quit' to exit):\n")

    while True:
        question = input("You: ").strip()
        if not question:
            continue
        if question.lower() in ("quit", "exit", "q"):
            print("Goodbye!")
            break

        print("\nSearching...")
        answer = ask_question(collection, question)
        print(f"\nAnswer: {answer}\n")

What does this code do?

  • The while True loop continuously prompts for questions until you type quit.
  • Each question goes through the full RAG pipeline: embed the query, retrieve top chunks, pass them as context to Gemini, and print the answer.
  • Save rag.py.
Run the pipeline and verify the fix

The updated script will delete the old collection, re-chunk everything with 100-character overlap, embed the new chunks, and then drop you into an interactive Q&A session.

  • Run the script by executing this command:
python rag.py

Watch the embedding output. Notice the chunk count is slightly higher than before. With 500-character chunks and 100-character overlap, the advance step is 400 characters instead of 500, producing more chunks from the same text.

Why is the chunk count higher?

Without overlap, a 5000-character document produces 10 chunks (5000 / 500). With 100-character overlap, the same document produces about 13 chunks (5000 / 400, rounded up). The extra chunks are the cost of ensuring boundary coverage.

Once you see the "=== RAG Pipeline Ready ===" message, the interactive loop is waiting for your input.

  • Before you type your question, think: do you expect the boundary-spanning question from Step 4 to succeed or fail this time? Why?
  • Type the same boundary-spanning question that failed in Step 4 and press Enter.

This time, the answer is correct and complete. The information that was previously split across a chunk boundary now appears intact inside at least one overlapping chunk.

  • Type quit to exit the Q&A loop.

Still getting an incomplete answer?

  • Make sure you changed start += chunk_size to start += chunk_size - overlap. Without this change, overlap has no effect.
  • Confirm the default parameter is overlap=CHUNK_OVERLAP, not overlap=0. If the default is still 0, the function produces non-overlapping chunks.
  • Check that the "Stored X chunks" count is higher than before. If it matches the old count, the overlap is not being applied.

Help me debug why my overlapping chunks aren't fixing the boundary problem.

✔️ Awesome, I've got everything!

Great. Double check you have saved rag.py.

ⓧ I'd like to double check the full code

Here is the full rag.py file. Scroll up and confirm each section is present in order:

  • Configuration block with CHUNK_SIZE, CHUNK_OVERLAP, COLLECTION_NAME, and PDF_FOLDER.
  • extract_text_from_pdf() and extract_all_pdfs() functions.
  • Updated chunk_text() with overlap=CHUNK_OVERLAP and start += chunk_size - overlap.
  • embed_texts(), create_vector_store(), retrieve(), and ask_question() functions.
  • The interactive if __name__ == "__main__": block with extraction, embedding, and the Q&A loop.

Secret mission

Add Source Citations with Page Numbers

Your RAG pipeline answers questions, but can you trust the answers? A production system needs to tell you exactly where to check. In this mission, you'll add page-level source citations so every answer points back to the exact PDF page it came from.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally (except for Gemini API calls within the free tier), so there are no ongoing costs.

Resources you used:

  • Your project folder on the Desktop (contains rag.py, pdfs/, chroma_db/, and your Python virtual environment).
  • Gemini API calls made during the project (all within the free tier, no charges incurred).

Keep everything running

No action needed. Choose this if you plan to keep querying your PDFs or want to add more documents to the pipeline.

  • Your chroma_db/ folder stays on disk so you can query your PDFs anytime without re-embedding.
  • To add new PDFs later, drop them into the pdfs/ folder and re-run python rag.py to rebuild the vector store.
  • Your virtual environment stays intact with all dependencies installed.

Pause - I'll come back to this later

Deactivate your virtual environment to free up your terminal. All files stay on disk so you can return later.

  • Deactivate your virtual environment by running:
deactivate
  • When you come back, navigate to your project folder and reactivate with source venv/bin/activate (macOS/Linux) or venv\Scripts\activate (Windows).
  • Your chroma_db/ folder persists automatically. No need to re-embed when you return.

Delete - I don't want to use this again

Remove all project files and start fresh. This deletes your script, PDFs folder, vector database, and virtual environment.

  • Deactivate your virtual environment first (if active) by running:
deactivate
  • Navigate to your Desktop and delete the project folder.

On macOS/Linux:

cd ~/Desktop
rm -rf [[PROJECT_FOLDER="your project folder name"]]

On Windows:

cd ~\Desktop
rmdir /s /q [[PROJECT_FOLDER="your project folder name"]]
  • This removes rag.py, the pdfs/ folder, the chroma_db/ vector database, and the virtual environment.
  • Your Google AI Studio API key remains active in your account. No action needed there since it incurs no cost on the free tier.

That's a Wrap!

That's a Wrap!

Nice work! You just built a complete RAG pipeline from scratch, turning a folder of PDFs into an AI-searchable knowledge base you can query in plain English.

You've learned how to:

  • Build a full retrieve-then-generate pipeline: PDF extraction with PyMuPDF, character-based chunking with overlap, vector embeddings via Gemini, storage in ChromaDB, similarity retrieval, and LLM-powered answer generation.
  • Diagnose and fix retrieval quality: you experienced firsthand how naive chunking splits answers at boundaries, then fixed it by adding overlapping chunks that share context between neighbours.
  • Use task-type-aware embeddings for asymmetric search: you applied RETRIEVAL_DOCUMENT for stored chunks and RETRIEVAL_QUERY for questions, optimising each side of the search independently.
  • Secret Mission: Added page-level source citations so every answer tells the reader exactly which PDF page to verify against.

Ready to quiz yourself?