Chat With Your PDFs Using RAG

Build a Python RAG app that answers questions about any PDF using Claude.

Introduction

30 Second Summary

You have a 50-page PDF report sitting in your Downloads folder. You need one specific answer buried somewhere in those pages. Instead of skimming every page, imagine asking a question in plain English and getting an accurate answer instantly.

In this project, you will build a Python command-line app that lets you chat with any PDF document. Your app extracts the text, stores it in a local vector database, and uses the Claude AI model to answer your questions based on what the document actually says.

What You'll Build

A complete RAG (Retrieval-Augmented Generation) application that connects a local vector database to the Claude API for intelligent question answering over PDF documents.

By the end of this project, you'll have:

  • A PDF text extraction pipeline that pulls content from any PDF and splits it into searchable chunks.
  • A local ChromaDB vector database storing your document's content as embeddings you can query.
  • A RAG query engine that retrieves relevant context and sends it to Claude for accurate, grounded answers.
  • Secret Mission: Add multi-document support so you can load multiple PDFs and ask questions across all of them.

Are there any prerequisites?

None. This guide walks you through installing every tool from scratch, including Python, VS Code, and the required libraries. All you need is a computer and an internet connection.

Set Up Your Development Environment

Before you can build your RAG app, you need a working Python environment and a code editor on your machine.

In this step, you'll install Python, set up VS Code, and create an isolated project folder with its own virtual environment.

In this step, get ready to:
  • Install Python 3.9+ and verify it works.
  • Install VS Code.
  • Create a project folder and virtual environment.
Install Python and verify it works

Your RAG app requires Python 3.9 or higher. Let's check whether you already have it installed.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or Command Prompt (Windows) and press Enter to open it.
  • Check your Python version by running this command:

macOS

python3 --version

Windows

python --version

✔️ I see version 3.9 or higher

Nice, Python is ready to go. Move on to the next substep.

ⓧ I see an older version

You have Python installed, but it's older than 3.9. The libraries this project uses require 3.9 or higher.

On macOS, the system-installed Python is often outdated. Download the latest version from the official Python website:

  • Go to https://www.python.org/downloads and download the latest installer for your operating system.
  • Run the installer. On Windows, check the box that says Add Python to PATH before clicking Install Now.
  • Close and reopen your terminal, then run the version check command again to confirm you see 3.9 or higher.

ⓧ Command not found

Python is not installed on your machine yet. Let's install it from the official website.

  • Go to https://www.python.org/downloads and download the latest installer for your operating system.
  • Run the installer. On Windows, check the box that says Add Python to PATH before clicking Install Now.
  • Close and reopen your terminal after installation completes.
  • Run the version check command again. You should now see a version number of 3.9 or higher.
Install VS Code

VS Code is a free code editor that makes working with Python files straightforward.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Visual Studio Code and press Enter.

✔️ VS Code opened

VS Code is installed and ready. Move on to the next substep.

ⓧ Nothing happened or app not found

VS Code is not installed yet. Let's download it.

  • Go to https://code.visualstudio.com/download in your browser.
  • Click the download button for your operating system (macOS or Windows).
  • Run the downloaded installer and follow the on-screen prompts.
  • Once installed, open VS Code using the search bar method described above.
Create your project folder and virtual environment

A virtual environment is an isolated Python workspace. It keeps this project's libraries separate from anything else on your system.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir rag-pdf-app
cd rag-pdf-app
  • Confirm the folder exists by running:
ls

The output will be empty since the folder is new. That's expected.

  • Create a virtual environment inside your project folder by running:

macOS

python3 -m venv venv

Windows

python -m venv venv

What does this command do?

This tells Python to create a virtual environment in a folder called venv inside your project. All the libraries you install next will live here instead of polluting your system-wide Python installation.

  • Activate the virtual environment by running:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

Make sure you ran the activate command from inside the rag-pdf-app folder. If you're in a different directory, the path to the activate script won't resolve.

On Windows, if you see an error about execution policies, run Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned first, then try the activate command again.

Help me activate my Python virtual environment

Your development environment is ready. Next up, you'll get your Anthropic API key so Claude can power the question-answering side of your RAG app.

Get Your Anthropic API Key

Your development environment is ready to go. Now it's time to get the key ingredient that makes your RAG app intelligent: access to Claude, Anthropic's AI model.

Claude is the LLM that will read your document chunks and answer your questions. To use it, you need an API key from Anthropic's developer console.

In this step, get ready to:
  • Create an Anthropic Console account.
  • Purchase API credits and generate an API key.
  • Store your API key as an environment variable.
Create an Anthropic Console account

The Anthropic Console is where you manage your API keys and billing. You need an account before you can call Claude from your code.

  • Open your browser and go to console.anthropic.com.
  • Click Continue with Google or Continue with email to create your account.
  • Complete the sign-up flow and verify your email if prompted.

What is an API key?

An API key is a secret string that identifies your application to a service. When your Python code calls Claude, it sends this key along with every request so Anthropic knows who is making the call and can bill you accordingly.

Purchase API credits

Anthropic uses a credit system for API access. You need a minimum $5 credit purchase to reach Tier 1, which unlocks the rate limits you need for this project.

  • In the Anthropic Console, click Settings in the left sidebar.
  • Click Billing to open the billing page.
  • Click Buy Credits and purchase at least $5 in credits.
  • Enter your payment details and confirm the purchase.

Why $5 minimum?

Anthropic organizes API access into usage tiers. A $5 credit purchase is required to reach Tier 1, which gives you a $500 monthly spend limit. Your actual usage for this project will be well under $1 since we are using Claude Haiku 4.5, the most cost-effective model at $1 per million input tokens.

Generate your API key and store it as an environment variable

Your Python code needs a way to authenticate with Anthropic's servers. The Anthropic SDK automatically reads a key from an environment variable called ANTHROPIC_API_KEY.

  • In the Anthropic Console, click API Keys in the left sidebar (under Settings).
  • Click Create Key.
  • Give your key a name (e.g. rag-app) and click Create Key.
  • Copy the key immediately. You will not be able to see it again after closing this dialog.

Copy your key now

Anthropic only shows your full API key once, at the moment of creation. If you close the dialog without copying it, you will need to create a new key.

Now set the key as an environment variable in your terminal so the Anthropic SDK can find it.

  • In your terminal (with your virtual environment still active), set the environment variable by running this command:

macOS

export ANTHROPIC_API_KEY='your-api-key-here'

Persist across sessions

This export only lasts for your current terminal session. To make it permanent, add the same line to your shell profile (~/.zshrc or ~/.bashrc).

Windows

$env:ANTHROPIC_API_KEY = "your-api-key-here"

Persist across sessions

This only lasts for your current PowerShell session. To make it permanent, use System Properties > Environment Variables to add ANTHROPIC_API_KEY as a user variable.

  • Replace your-api-key-here with the actual key you just copied from the Anthropic Console.
  • Verify the variable is set by running this command:

macOS

echo $ANTHROPIC_API_KEY

Windows

echo $env:ANTHROPIC_API_KEY

You should see your API key printed in the terminal output.

Seeing a blank output?

Make sure you ran the export command in the same terminal session. If you opened a new terminal window, you need to run the export command again.

On macOS, check that you used export (not just the variable assignment without export). On Windows, make sure you used $env: prefix in PowerShell.

help me set my ANTHROPIC_API_KEY environment variable

Your API key is set and ready. Next up, you will install the Python libraries that power your RAG pipeline.

Install Python Libraries and Create Project Structure

Your API key is stored and ready to use. Now it's time to install the Python libraries that power your RAG app.

You need three libraries working together. One reads PDFs, one stores and searches document chunks, and one talks to Claude. In this step, you'll install all three and set up the main project file.

In this step, get ready to:
  • Install the required Python packages.
  • Create the main project file with imports and configuration.
  • Verify that all libraries load correctly.
Install the required packages

Your RAG app depends on three Python libraries. Each handles a different part of the pipeline.

  • Install all three packages by running this command in your terminal (make sure your virtual environment is still active):
pip install anthropic chromadb pypdf

What do these packages do?

  • anthropic is the official Python SDK for the Claude API. It handles authentication and message formatting.
  • chromadb is a local vector database. It stores your document chunks as embeddings and lets you search them by meaning.
  • pypdf extracts text from PDF files. It replaced the deprecated PyPDF2 library.
  • Wait for the installation to finish. You should see a line that says Successfully installed followed by the package names.

Seeing a build error for hnswlib?

ChromaDB depends on hnswlib, which sometimes fails to compile. Try these fixes:

  • On macOS, update your Xcode command-line tools by running xcode-select --install in your terminal, then retry the install.
  • If you see an error mentioning -march=native, run export HNSWLIB_NO_NATIVE=1 before retrying.
  • Make sure your Python version is 3.9 or higher. Older versions bundle outdated SQLite that ChromaDB cannot use.

Help me fix a pip install error for chromadb

Create the project file

With your libraries installed, you need a Python file to hold your application code. You'll start by declaring all the imports and configuration constants your app will use.

  • In VS Code, click **File** in the top menu bar.
  • Click **New File**.
  • Save the file immediately as rag_app.py inside your rag-pdf-app folder by pressing **Cmd+S** (macOS) or **Ctrl+S** (Windows).
  • Add the following imports and configuration to rag_app.py:
import os
import sys
from pypdf import PdfReader
import chromadb
from anthropic import Anthropic

# --- Configuration ---
CHUNK_SIZE = 500
CHUNK_OVERLAP = 50
COLLECTION_NAME = "pdf_documents"
MODEL = "claude-haiku-4-5"

What does this code do?

  • os and sys are built-in Python modules for file paths and command-line arguments.
  • PdfReader from pypdf is the class that opens and reads PDF files.
  • chromadb gives you the vector database client for storing and querying document chunks.
  • Anthropic is the client class that connects to the Claude API using your ANTHROPIC_API_KEY environment variable.
  • The four constants set your chunking strategy (500 characters with 50 characters of overlap), the database collection name, and which Claude model to use.
  • Save rag_app.py by pressing **Cmd+S** (macOS) or **Ctrl+S** (Windows).
Verify all imports work correctly

Running the file proves that all three libraries installed correctly and Python can find them.

  • Run your file by entering this command in your terminal:
python rag_app.py
  • The script should exit silently with no output. If you see no errors, every import resolved correctly.

Seeing a ModuleNotFoundError?

  • Make sure your virtual environment is active. You should see (venv) at the start of your terminal prompt. If it's missing, activate it again before re-running.
  • Confirm you installed the packages in the same virtual environment. Run pip list to see what's installed.
  • If pypdf is missing, you may have accidentally installed PyPDF2 instead. The correct package name is pypdf (lowercase, no number).

Help me fix an import error in my RAG app

✔️ Awesome, I've got everything!

Great. Double check you've saved your file.

ⓧ I'd like to double check the full code

import os
import sys
from pypdf import PdfReader
import chromadb
from anthropic import Anthropic

# --- Configuration ---
CHUNK_SIZE = 500
CHUNK_OVERLAP = 50
COLLECTION_NAME = "pdf_documents"
MODEL = "claude-haiku-4-5"

Your project is wired up and ready. Next, you'll write the function that extracts text from a PDF and splits it into chunks.

Build the PDF Text Extractor

Your project has all the libraries installed and your API key is ready. Now you need to solve the first real problem: getting text out of a PDF file and into a format your RAG pipeline can work with.

PDFs store text as positioned fragments meant for visual rendering, not as clean readable paragraphs. In this step, you will write two functions that handle this extraction and then split the result into bite-sized chunks that a vector database can search through efficiently.

In this step, get ready to:
  • Write a function that extracts all text from a PDF file.
  • Write a function that splits text into overlapping chunks.
  • Test both functions with a sample PDF.
Write the PDF extraction function

The first piece of your RAG pipeline needs to open a PDF and pull all the text out of every page. The pypdf library provides a PdfReader class that handles the complex PDF parsing for you.

  • In VS Code, open rag_app.py inside your rag-pdf-app folder.
  • Find the last line of the configuration section (the line that says MODEL = "claude-haiku-4-5").
  • Add the PDF extraction function below it by copying this code:
# --- PDF Extraction ---
def extract_text_from_pdf(pdf_path):
    """Extract all text from a PDF file."""
    reader = PdfReader(pdf_path)
    text = ""
    for page in reader.pages:
        page_text = page.extract_text()
        if page_text:
            text += page_text + "\n"
    return text

What does this code do?

PdfReader(pdf_path) opens the PDF file and loads its internal structure into memory.

The for page in reader.pages loop iterates through every page in the document. Each call to page.extract_text() pulls out the text content from that page. The if page_text check skips pages that have no extractable text (like image-only pages).

  • Save rag_app.py.

Seeing a red underline on PdfReader?

Make sure the import at the top of your file reads from pypdf import PdfReader exactly. If you accidentally installed the deprecated PyPDF2 package, the import path is different. Uninstall it and use pypdf instead.

help me fix my pypdf import in rag_app.py

Write the chunking function

A full PDF might contain thousands of words. Sending all of that text to Claude for every question would be expensive and slow. Instead, you split the text into smaller overlapping chunks that your vector database can search through independently.

  • In rag_app.py, add the chunking function directly below the extract_text_from_pdf function you just wrote:
# --- Document Chunking ---
def chunk_text(text, chunk_size=CHUNK_SIZE, overlap=CHUNK_OVERLAP):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        chunks.append(chunk)
        start += chunk_size - overlap
    return chunks

What does this code do?

This function uses a sliding window to cut text into pieces of CHUNK_SIZE characters (500 by default). Each new chunk starts CHUNK_OVERLAP characters (50) before the previous chunk ended.

The overlap ensures that if a sentence gets split across two chunks, the important context still appears in at least one complete chunk. This prevents the vector database from missing relevant information when a query happens to match text near a chunk boundary.

Why overlapping chunks instead of clean splits?

Imagine a sentence that says "The deadline for the Q3 report is September 15th." If a clean split cuts this sentence exactly in half, neither chunk would contain the full answer to "When is the Q3 report due?" Overlapping windows solve this by ensuring context near boundaries appears in both adjacent chunks.

  • Save rag_app.py.

Not sure where to place this function?

The chunk_text function should appear immediately after the return text line at the end of extract_text_from_pdf. Leave one blank line between functions. Check the full code tab below to confirm placement.

help me place the chunk_text function in the right spot in rag_app.py

Test the extractor with a sample PDF

Before building the rest of the pipeline, you should confirm that both functions actually work. You need a PDF to test with.

  • Find any PDF document on your computer and copy it into your rag-pdf-app folder. A short document (1-10 pages) works best for quick testing.

No PDF handy?

You can download a free sample from the internet. Search for "sample PDF" and save any result into your project folder. A simple text-based PDF (like an article or report) will give the clearest results. Avoid scanned image-only PDFs because pypdf cannot extract text from images.

  • Open a new terminal in VS Code by pressing Cmd+` (macOS) or Ctrl+` (Windows).
  • Make sure your virtual environment is still active (you should see (venv) in the terminal prompt).
  • Test both functions by running this command, replacing the filename with your actual PDF:
python -c "from rag_app import extract_text_from_pdf, chunk_text; text = extract_text_from_pdf('[[PDFFILE="your-sample.pdf"]]'); print(f'Extracted {len(text)} characters'); chunks = chunk_text(text); print(f'Split into {len(chunks)} chunks'); print(f'First chunk preview: {chunks[0][:100]}...')"
  • You should see output showing the number of characters extracted, the number of chunks created, and a preview of the first chunk's text.

What should I see?

A successful run prints three lines. The first shows the total character count (typically hundreds to thousands of characters depending on your PDF). The second shows the chunk count. The third shows a preview of the first 100 characters from your document.

For example, a 5-page document might produce something like: Extracted 12847 characters, Split into 28 chunks, and a text preview from the first page.

Seeing 0 characters extracted?

Your PDF might be a scanned image rather than a text-based document. Try a different PDF that was created digitally (like a report, article, or ebook). You can also check by opening the PDF and trying to highlight text with your cursor.

If you see a file not found error, make sure your PDF is in the same folder as rag_app.py and that the filename matches exactly (including the .pdf extension).

help me debug why my PDF extraction returns no text

✔️ Awesome, I've got everything!

Great. Double check you have saved rag_app.py with both functions.

ⓧ I'd like to double check the full code

Here is the complete rag_app.py at this point:

import os
import sys
from pypdf import PdfReader
import chromadb
from anthropic import Anthropic

# --- Configuration ---
CHUNK_SIZE = 500
CHUNK_OVERLAP = 50
COLLECTION_NAME = "pdf_documents"
MODEL = "claude-haiku-4-5"

# --- PDF Extraction ---
def extract_text_from_pdf(pdf_path):
    """Extract all text from a PDF file."""
    reader = PdfReader(pdf_path)
    text = ""
    for page in reader.pages:
        page_text = page.extract_text()
        if page_text:
            text += page_text + "\n"
    return text

# --- Document Chunking ---
def chunk_text(text, chunk_size=CHUNK_SIZE, overlap=CHUNK_OVERLAP):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        chunks.append(chunk)
        start += chunk_size - overlap
    return chunks

Your extraction pipeline is working. Next up, you will store those chunks in a ChromaDB vector database so your app can search through them with natural language questions.

Store Document Chunks in ChromaDB

In the previous step, you built functions to extract text from a PDF and split it into manageable chunks. Those chunks are ready, but they're just sitting in memory with no way to search through them.

To find the right chunks when a user asks a question, you need a vector database. In this step, you'll store your chunks in ChromaDB and verify you can query them back.

In this step, get ready to:
  • Initialize a ChromaDB persistent client that saves data to disk.
  • Write a function to store document chunks in a collection.
  • Verify the chunks are stored and queryable.
Initialize a ChromaDB persistent client

ChromaDB offers two client types: ephemeral (in-memory, lost when the script stops) and persistent (saved to disk). You want persistent storage so your document stays indexed across sessions.

  • Test your ChromaDB connection by running this command in your terminal:
python3 -c "
import chromadb

chroma_client = chromadb.PersistentClient(path='./chroma_db')
collection = chroma_client.get_or_create_collection(name='pdf_documents')
print(collection.name)
"

What does this code do?

chromadb.PersistentClient(path='./chroma_db') creates a database client that stores all data in a folder called chroma_db inside your project directory. Data persists automatically and loads on start if it already exists.

get_or_create_collection either fetches an existing collection by name or creates a new one. This makes the code safe to run multiple times without errors.

  • You should see pdf_documents printed in your terminal.
  • Confirm the database folder was created by running:
ls chroma_db

You should see files and folders that ChromaDB created to store your data.

What about embeddings?

When you add documents to ChromaDB without specifying an embedding function, it uses its built-in default model (all-MiniLM-L6-v2 from Sentence Transformers). This model runs locally on your machine, so you do not need a separate embedding API key.

The first time you store documents, ChromaDB may download this model automatically. This is a one-time download.

Don't see pdf_documents printed?

Make sure your virtual environment is activated. You should see (venv) at the start of your terminal prompt.

If you see an error about SQLite, ChromaDB requires SQLite version greater than 3.35. Run python3 -c "import sqlite3; print(sqlite3.sqlite_version)" to check your version.

Help me fix my ChromaDB connection

Write the storage function

Now you need a function that takes your text chunks and stores them in the ChromaDB collection. Each chunk needs a unique ID so ChromaDB can track it.

  • In rag_app.py, find the end of the chunk_text function (after the return chunks line).
  • Add the following code below it:
# --- ChromaDB Storage ---
def store_chunks(chunks, collection):
    """Store document chunks in ChromaDB."""
    ids = [f"chunk_{i}" for i in range(len(chunks))]
    collection.add(
        documents=chunks,
        ids=ids
    )
    print(f"Stored {len(chunks)} chunks in ChromaDB.")

What does this code do?

ids = [f"chunk_{i}" for i in range(len(chunks))] generates a unique ID for each chunk (chunk_0, chunk_1, etc.). ChromaDB requires every document to have a unique ID.

collection.add(documents=chunks, ids=ids) stores the chunks in ChromaDB. ChromaDB automatically converts each chunk into a vector embedding using its built-in model. You do not need to handle embeddings yourself.

  • Save rag_app.py.

Seeing an indentation error?

Make sure the store_chunks function starts at the same indentation level as the chunk_text function above it. Both should be at the top level (no leading spaces).

Help me fix a Python indentation error in my store_chunks function

✔️ Awesome, I've got everything!

Great. Make sure you've saved rag_app.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete rag_app.py file up to this point:

import os
import sys
from pypdf import PdfReader
import chromadb
from anthropic import Anthropic

# --- Configuration ---
CHUNK_SIZE = 500
CHUNK_OVERLAP = 50
COLLECTION_NAME = "pdf_documents"
MODEL = "claude-haiku-4-5"

# --- PDF Extraction ---
def extract_text_from_pdf(pdf_path):
    """Extract all text from a PDF file."""
    reader = PdfReader(pdf_path)
    text = ""
    for page in reader.pages:
        page_text = page.extract_text()
        if page_text:
            text += page_text + "\n"
    return text

# --- Document Chunking ---
def chunk_text(text, chunk_size=CHUNK_SIZE, overlap=CHUNK_OVERLAP):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        chunks.append(chunk)
        start += chunk_size - overlap
    return chunks

# --- ChromaDB Storage ---
def store_chunks(chunks, collection):
    """Store document chunks in ChromaDB."""
    ids = [f"chunk_{i}" for i in range(len(chunks))]
    collection.add(
        documents=chunks,
        ids=ids
    )
    print(f"Stored {len(chunks)} chunks in ChromaDB.")
Verify the chunks are stored and queryable

The real test is loading a PDF, chunking it, storing the chunks, and then querying them back. This end-to-end test confirms your entire pipeline works.

  • Run the following test in your terminal, replacing the PDF filename with your sample PDF:
python3 -c "
import chromadb
from rag_app import extract_text_from_pdf, chunk_text, store_chunks

# Initialize ChromaDB
chroma_client = chromadb.PersistentClient(path='./chroma_db')

# Delete the old collection to start fresh
chroma_client.delete_collection(name='pdf_documents')
collection = chroma_client.get_or_create_collection(name='pdf_documents')

# Load and chunk the PDF
text = extract_text_from_pdf('[[PDF_FILE="your-sample-pdf.pdf"]]\')
chunks = chunk_text(text)
print(f'Created {len(chunks)} chunks')

# Store chunks
store_chunks(chunks, collection)

# Query to verify
results = collection.query(query_texts=['summary'], n_results=2)
print(f'Query returned {len(results["documents"][0])} results')
print(results['documents'][0][0][:150])
"

What does this test do?

This test runs your full pipeline end-to-end. It extracts text from your PDF, splits it into chunks, stores those chunks in ChromaDB, and then queries the collection with the word "summary" to verify data is retrievable.

The delete_collection call at the start ensures you have a clean slate. Without it, running the test twice would cause duplicate ID errors since the chunk IDs (chunk_0, chunk_1, etc.) are the same each time.

You should see output similar to this (your numbers will vary depending on your PDF):

Created 42 chunks
Stored 42 chunks in ChromaDB.
Query returned 2 results
[first 150 characters of a matching chunk from your PDF]

First run may take a moment

The first time you store documents, ChromaDB downloads its default embedding model. This is a one-time download. Subsequent runs will be much faster.

Seeing an import error or no results?

If you see ModuleNotFoundError: No module named 'rag_app', make sure you're running the command from inside your rag-pdf-app project folder where rag_app.py lives.

If the query returns 0 results, check that your PDF actually contains extractable text. Some scanned PDFs are image-only and won't produce text with pypdf. Try a different PDF that contains selectable text.

Help me troubleshoot my ChromaDB storage test

Your document chunks are now stored in a local vector database, ready to be searched. Next up, you'll build the query engine that retrieves the most relevant chunks and sends them to Claude for answering questions.

Build the RAG Query Engine

Your document chunks are sitting in ChromaDB, ready to be searched. Now you need a way to find the right chunks for a given question and send them to Claude for an intelligent answer.

This is the core of the RAG pattern: retrieve relevant context from your database, then augment Claude's prompt with that context so it can generate a grounded response. The function you build in this step connects the retrieval layer to the generation layer.

In this step, get ready to:
  • Write a function that retrieves relevant chunks from ChromaDB and constructs a prompt for Claude.
  • Call the Claude API with retrieved context and a question.
  • Test the query engine with a sample question against your stored PDF chunks.
Write the query function

The query_documents function is the heart of your RAG app. It takes a question, searches ChromaDB for the most relevant chunks, then sends those chunks to Claude along with the question.

  • In rag_app.py, find the store_chunks function you wrote in the last step.
  • Add the following function below it:
# --- RAG Query ---
def query_documents(question, collection, client):
    """Retrieve relevant chunks and ask Claude."""
    results = collection.query(
        query_texts=[question],
        n_results=3
    )

    retrieved_chunks = results["documents"][0]
    context = "\n\n---\n\n".join(retrieved_chunks)

    message = client.messages.create(
        model=MODEL,
        max_tokens=1024,
        system="You are a helpful assistant that answers questions based on the provided document context. Only use information from the context to answer. If the context does not contain enough information to answer, say so.",
        messages=[
            {
                "role": "user",
                "content": f"Context from the document:\n\n{context}\n\nQuestion: {question}"
            }
        ]
    )

    return message.content[0].text

What does this code do?

This function performs the two halves of the RAG pattern in sequence.

  • collection.query(query_texts=[question], n_results=3) searches ChromaDB for the 3 chunks most similar to your question. ChromaDB converts your question into an embedding and finds the closest matches automatically.
  • results["documents"][0] extracts the actual text content from the query results. The [0] is needed because ChromaDB supports batched queries, so results are nested in a list.
  • The chunks are joined with a --- separator to create a single context string that Claude can read.
  • client.messages.create() sends the context and question to Claude with a system prompt that tells it to only answer from the provided context. This grounding instruction prevents Claude from hallucinating answers.
  • message.content[0].text extracts Claude's text response from the API response object.
  • Save rag_app.py.

Seeing a syntax error?

  • Check that the f prefix is present on the content string inside the messages list. Without it, {context} and {question} won't be interpolated.
  • Make sure the system parameter is a separate argument to messages.create(), not inside the messages list.
  • Verify that the closing parentheses and brackets all match up. The messages parameter contains a list with one dictionary inside it.

help me fix a syntax error in my query_documents function

Test the query engine with your PDF

Your function is written, but you need to confirm it actually works end-to-end. You will query the chunks you stored in the previous step and get a real answer from Claude.

  • Make sure your terminal is in the rag-pdf-app folder with your virtual environment activated.
  • Test the query engine by running this command:
python -c "
import chromadb
from anthropic import Anthropic
from rag_app import query_documents, COLLECTION_NAME

chroma_client = chromadb.PersistentClient(path='./chroma_db')
collection = chroma_client.get_or_create_collection(name=COLLECTION_NAME)
client = Anthropic()

answer = query_documents('What is this document about?', collection, client)
print(answer)
"

What does this test do?

This command imports your query_documents function, connects to the same ChromaDB database you populated earlier, creates an Anthropic client using your ANTHROPIC_API_KEY environment variable, and asks a question about your PDF.

Claude will read the most relevant chunks from your document and produce a grounded answer. The answer you see will be specific to whatever PDF you loaded in the previous step.

  • You should see Claude's answer printed in your terminal. The response will be a paragraph or two summarizing the contents of your PDF based on the retrieved chunks.

Getting an authentication error?

  • Make sure your ANTHROPIC_API_KEY environment variable is still set in this terminal session. If you opened a new terminal window, you need to export it again.
  • Verify your API key starts with sk-ant- and that you have credits available in your Anthropic Console.
  • If you see a connection timeout, check your internet connection. The Claude API requires an active internet connection.

help me fix an authentication error when calling the Anthropic Claude API

✔️ Awesome, I've got everything!

Great. Double check you have saved rag_app.py and confirmed Claude returned an answer about your PDF.

ⓧ I'd like to double check the full code

Here is the full rag_app.py file as it should look at the end of this step:

import os
import sys
from pypdf import PdfReader
import chromadb
from anthropic import Anthropic

# --- Configuration ---
CHUNK_SIZE = 500
CHUNK_OVERLAP = 50
COLLECTION_NAME = "pdf_documents"
MODEL = "claude-haiku-4-5"

# --- PDF Extraction ---
def extract_text_from_pdf(pdf_path):
    """Extract all text from a PDF file."""
    reader = PdfReader(pdf_path)
    text = ""
    for page in reader.pages:
        page_text = page.extract_text()
        if page_text:
            text += page_text + "\n"
    return text

# --- Document Chunking ---
def chunk_text(text, chunk_size=CHUNK_SIZE, overlap=CHUNK_OVERLAP):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        chunks.append(chunk)
        start += chunk_size - overlap
    return chunks

# --- ChromaDB Storage ---
def store_chunks(chunks, collection):
    """Store document chunks in ChromaDB."""
    ids = [f"chunk_{i}" for i in range(len(chunks))]
    collection.add(
        documents=chunks,
        ids=ids
    )
    print(f"Stored {len(chunks)} chunks in ChromaDB.")

# --- RAG Query ---
def query_documents(question, collection, client):
    """Retrieve relevant chunks and ask Claude."""
    results = collection.query(
        query_texts=[question],
        n_results=3
    )

    retrieved_chunks = results["documents"][0]
    context = "\n\n---\n\n".join(retrieved_chunks)

    message = client.messages.create(
        model=MODEL,
        max_tokens=1024,
        system="You are a helpful assistant that answers questions based on the provided document context. Only use information from the context to answer. If the context does not contain enough information to answer, say so.",
        messages=[
            {
                "role": "user",
                "content": f"Context from the document:\n\n{context}\n\nQuestion: {question}"
            }
        ]
    )

    return message.content[0].text

Your RAG query engine is working. Next up, you will wrap everything into an interactive chat loop so you can ask multiple questions in a row without restarting the script.

Create the Interactive Chat Loop

You have all the building blocks ready. Your app can extract text from PDFs, chunk it, store it in ChromaDB, and query Claude with relevant context.

Now it is time to connect all those pieces into a single runnable script. You will write a main() function that orchestrates everything and gives you an interactive chat loop for asking questions about your PDF.

In this step, get ready to:
  • Write a main() function that loads a PDF and initializes all components.
  • Build an interactive question-answer loop with edge case handling.
  • Run the complete RAG application end-to-end with a real PDF.
Write the main function

The main() function is the entry point that ties your four helper functions together. It reads a PDF path from the command line, validates the file exists, processes it through the extraction and chunking pipeline, and sets up both database and API clients.

  • In rag_app.py, scroll to the bottom of the file (after the query_documents function). Add the following code below it:
# --- Main Application ---
def main():
    if len(sys.argv) < 2:
        print("Usage: python rag_app.py <path_to_pdf>")
        sys.exit(1)

    pdf_path = sys.argv[1]

    if not os.path.exists(pdf_path):
        print(f"Error: File not found: {pdf_path}")
        sys.exit(1)

    print(f"Loading PDF: {pdf_path}")
    text = extract_text_from_pdf(pdf_path)
    print(f"Extracted {len(text)} characters of text.")

    chunks = chunk_text(text)
    print(f"Split into {len(chunks)} chunks.")

    chroma_client = chromadb.PersistentClient(path="./chroma_db")
    collection = chroma_client.get_or_create_collection(name=COLLECTION_NAME)

    store_chunks(chunks, collection)

    anthropic_client = Anthropic()

    print("\nRAG App Ready! Ask questions about your PDF.")
    print("Type 'quit' to exit.\n")

What does this code do?

This function orchestrates the full setup pipeline before the chat begins.

  • sys.argv[1] reads the PDF file path from the command line, so you can pass any PDF when running the script.
  • The two validation checks (argument count and file existence) give the user a helpful error message instead of a cryptic traceback.
  • chromadb.PersistentClient(path="./chroma_db") stores your vectors on disk so they survive between sessions.
  • Anthropic() automatically reads the ANTHROPIC_API_KEY environment variable you set earlier.
Add the interactive chat loop

The chat loop continuously reads user input, retrieves relevant document chunks, and sends them to Claude for an answer. It also handles edge cases like empty input and quit commands.

  • Still inside rag_app.py, add the following code directly below the print("Type 'quit' to exit.\n") line, still inside the main() function:
    while True:
        question = input("You: ").strip()
        if not question:
            continue
        if question.lower() in ("quit", "exit", "q"):
            print("Goodbye!")
            break

        answer = query_documents(question, collection, anthropic_client)
        print(f"\nClaude: {answer}\n")

What does this code do?

  • while True keeps the loop running until the user explicitly exits.
  • .strip() removes whitespace from the input. If the user presses Enter without typing anything, continue skips back to the prompt without calling the API.
  • The quit check accepts quit, exit, or q as exit commands.
  • Each valid question gets passed to query_documents() which handles retrieval and Claude's response.
  • Now add the script guard at the very bottom of the file, outside the main() function (no indentation):
if __name__ == "__main__":
    main()

Why add this guard?

The if __name__ == "__main__" pattern ensures main() only runs when you execute the script directly. If you ever import this file from another script, the chat loop will not start automatically.

  • Save rag_app.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before running it.

ⓧ I'd like to double check the full code

import os
import sys
from pypdf import PdfReader
import chromadb
from anthropic import Anthropic

# --- Configuration ---
CHUNK_SIZE = 500
CHUNK_OVERLAP = 50
COLLECTION_NAME = "pdf_documents"
MODEL = "claude-haiku-4-5"

# --- PDF Extraction ---
def extract_text_from_pdf(pdf_path):
    """Extract all text from a PDF file."""
    reader = PdfReader(pdf_path)
    text = ""
    for page in reader.pages:
        page_text = page.extract_text()
        if page_text:
            text += page_text + "\n"
    return text

# --- Document Chunking ---
def chunk_text(text, chunk_size=CHUNK_SIZE, overlap=CHUNK_OVERLAP):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        chunks.append(chunk)
        start += chunk_size - overlap
    return chunks

# --- ChromaDB Storage ---
def store_chunks(chunks, collection):
    """Store document chunks in ChromaDB."""
    ids = [f"chunk_{i}" for i in range(len(chunks))]
    collection.add(
        documents=chunks,
        ids=ids
    )
    print(f"Stored {len(chunks)} chunks in ChromaDB.")

# --- RAG Query ---
def query_documents(question, collection, client):
    """Retrieve relevant chunks and ask Claude."""
    results = collection.query(
        query_texts=[question],
        n_results=3
    )

    retrieved_chunks = results["documents"][0]
    context = "\n\n---\n\n".join(retrieved_chunks)

    message = client.messages.create(
        model=MODEL,
        max_tokens=1024,
        system="You are a helpful assistant that answers questions based on the provided document context. Only use information from the context to answer. If the context does not contain enough information to answer, say so.",
        messages=[
            {
                "role": "user",
                "content": f"Context from the document:\n\n{context}\n\nQuestion: {question}"
            }
        ]
    )

    return message.content[0].text

# --- Main Application ---
def main():
    if len(sys.argv) < 2:
        print("Usage: python rag_app.py <path_to_pdf>")
        sys.exit(1)

    pdf_path = sys.argv[1]

    if not os.path.exists(pdf_path):
        print(f"Error: File not found: {pdf_path}")
        sys.exit(1)

    print(f"Loading PDF: {pdf_path}")
    text = extract_text_from_pdf(pdf_path)
    print(f"Extracted {len(text)} characters of text.")

    chunks = chunk_text(text)
    print(f"Split into {len(chunks)} chunks.")

    chroma_client = chromadb.PersistentClient(path="./chroma_db")
    collection = chroma_client.get_or_create_collection(name=COLLECTION_NAME)

    store_chunks(chunks, collection)

    anthropic_client = Anthropic()

    print("\nRAG App Ready! Ask questions about your PDF.")
    print("Type 'quit' to exit.\n")

    while True:
        question = input("You: ").strip()
        if not question:
            continue
        if question.lower() in ("quit", "exit", "q"):
            print("Goodbye!")
            break

        answer = query_documents(question, collection, anthropic_client)
        print(f"\nClaude: {answer}\n")

if __name__ == "__main__":
    main()
Run the complete application end-to-end

Your RAG app is complete. Time to run it against a real PDF and see the full pipeline in action.

  • First, delete the existing chroma_db folder from your earlier testing. This prevents duplicate ID conflicts when the app stores fresh chunks. Delete it by running:
rm -rf chroma_db

Windows users

If you are on Windows, run this instead:

Remove-Item -Recurse -Force chroma_db
  • Run your RAG app with your sample PDF by running:
python rag_app.py [[PDF_FILENAME="your-sample.pdf"]]

You should see output showing the PDF loading, text extraction, chunking, and storage. Then the app prints a prompt inviting you to ask questions.

  • Type a question about your PDF and press Enter. Claude responds with an answer drawn from the document content.
  • Try 2-3 different questions to see how the retrieval finds relevant chunks.
  • Type quit to exit when you are done exploring.

What should I see?

Your terminal output looks something like this:

Loading PDF: your-sample.pdf
Extracted 12847 characters of text.
Split into 28 chunks.
Stored 28 chunks in ChromaDB.

RAG App Ready! Ask questions about your PDF.
Type 'quit' to exit.

You: What is the main topic of this document?

Claude: Based on the provided context, the main topic is...

The exact numbers and answers depend on your PDF. The important thing is that Claude answers based on your document content, not its general training data.

App not working as expected?

  • If you see Usage: python rag_app.py <path_to_pdf>, make sure you included the PDF filename after the script name in your run command.
  • If you see Error: File not found, check that your PDF file is in the same folder as rag_app.py, or provide the full path to the file.
  • If you get an authentication error from Anthropic, confirm your ANTHROPIC_API_KEY environment variable is still set in this terminal session.

help me debug my RAG app

Your RAG application is fully operational. You have built a complete pipeline that extracts text from a PDF, chunks it, stores vectors in a local database, and lets you chat with the document using Claude. Next, take on the Secret Mission to extend your app with multi-document support.

Secret mission

Add Multi-Document Support

Your RAG app works great with a single PDF. But what happens when you need answers from multiple documents at once? Extend your app to load an entire folder of PDFs and tell you exactly which document each answer came from.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs locally with no idle cloud resources, so there are no ongoing costs while the app is not in use.

Resources you used:

  • Project folder on your Desktop (includes virtual environment, rag_app.py, and ChromaDB data in ./chroma_db).
  • ANTHROPIC_API_KEY environment variable set in your terminal session.
  • Anthropic Console account with purchased API credits.

Keep everything running

No action needed. Choose this if you're still actively building or want to experiment with different PDFs.

  • Your project folder stays on your Desktop, ready to use anytime.
  • Your Anthropic account retains its remaining credits. You only pay when making API calls.
  • The ANTHROPIC_API_KEY environment variable remains available in your terminal.

Pause - I'll come back to this later

Shut down running processes and secure your API key while keeping your project files intact for later.

  • If your virtual environment is active, deactivate it by running:
deactivate
  • Unset your API key from the current terminal session for security. On macOS/Linux:
unset ANTHROPIC_API_KEY
  • On Windows (PowerShell):
Remove-Item Env:ANTHROPIC_API_KEY
  • Your project folder and ChromaDB data remain on your Desktop. When you come back, reactivate the virtual environment and re-export your API key.

Delete - I don't want to use this again

Remove all project resources from your machine and start fresh.

  • Delete the project folder (which includes the virtual environment, code, and ChromaDB data). On macOS/Linux:
cd ~/Desktop
rm -rf pdf-rag-app
  • On Windows (PowerShell):
cd ~/Desktop
Remove-Item -Recurse -Force pdf-rag-app
  • Unset the API key environment variable. On macOS/Linux:
unset ANTHROPIC_API_KEY
  • On Windows (PowerShell):
Remove-Item Env:ANTHROPIC_API_KEY
  • Your Anthropic Console account and remaining credits will persist. There is no ongoing charge. Credits are only consumed when you make API calls.

That's a Wrap!

That's a Wrap!

Nice work! You just built a complete RAG system from scratch. This is the exact architecture behind enterprise AI products that combine private data with LLMs.

You've learned how to:

  • Built a PDF text extraction and chunking pipeline using pypdf with overlapping windows.
  • Stored and queried vector embeddings in a local ChromaDB database.
  • Assembled a complete RAG pipeline powered by the Claude API for intelligent question answering.
  • Added multi-document support with source attribution so your app can search across multiple PDFs.

Ready to quiz yourself?