Build a PDF Q&A System with RAG

Build a RAG pipeline that answers questions from any PDF using Llama 3.3.

Introduction

30 Second Summary

You have PDFs full of valuable information, but finding specific answers means scrolling through dozens of pages. What if you could just ask a question and get an instant, accurate answer pulled straight from the document?

In this project, you will build a command-line tool that loads any PDF, understands its content, and answers your natural language questions using Retrieval-Augmented Generation (RAG). Everything runs on open-source models at zero cost.

What You'll Build

A Python script that implements a complete RAG pipeline using ChromaDB for local vector storage and Llama 3.3 70B on Groq's free API for answer generation.

By the end of this project, you'll have:

  • A working RAG pipeline that loads, chunks, embeds, and stores PDF content in a local vector database.
  • An interactive Q&A loop that retrieves relevant passages and generates grounded answers using LLM inference on Groq.
  • A clear understanding of semantic search and how text embeddings power modern AI document assistants.
  • Secret Mission: Add source citations that show which page and passage each answer came from.

Are there any prerequisites?

You only need basic Python knowledge (variables, functions, and imports). No prior AI or machine learning experience is needed.

Before We Start

Before We Start...

Set Up Your Environment

Before you can build a RAG system, you need a working Python environment with the right packages installed.

In this step, you will create a project folder, install the libraries that power your RAG pipeline, and grab a free API key from Groq so your script can talk to a large language model.

In this step, get ready to:
  • Create a project folder and Python virtual environment.
  • Install all required Python packages.
  • Configure your free Groq API key.
Create a project folder and virtual environment

A virtual environment keeps your project's packages isolated from the rest of your system. This prevents version conflicts between different Python projects on your machine.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.

First, check which version of Python you have installed. This project requires Python 3.10 or higher.

  • Check your Python version by running this command:
python3 --version

✔️ I see version 3.10 or higher

Your Python version is ready. Continue to the next instruction below.

ⓧ I see an older version

Your system has a Python version below 3.10. You need to install a newer version.

  • Visit python.org/downloads and download the latest Python 3 installer for your operating system.
  • Run the installer. On Windows, check the box Add Python to PATH before clicking Install Now.
  • Close and reopen your terminal, then run python3 --version again to confirm.

macOS users with an older default Python

macOS may ship with an older Python. After installing from python.org, your new version lives at /usr/local/bin/python3. Run python3 --version to confirm the updated version is on your path.

ⓧ Command not found

Python is not installed on your machine yet.

  • Visit python.org/downloads and download the latest Python 3 installer for your operating system.
  • Run the installer. On Windows, check the box Add Python to PATH before clicking Install Now.
  • Close and reopen your terminal, then run python3 --version to confirm it prints 3.10 or higher.

Windows users

On Windows, the command might be python --version instead of python3 --version. Use whichever one works after installation.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir pdf-rag
cd pdf-rag
  • Create a virtual environment by running this command:
python3 -m venv venv

What does this command do?

python3 -m venv venv tells Python to create a virtual environment in a folder called venv inside your project directory. All packages you install will live here, isolated from your system Python.

  • Activate the virtual environment by running:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the start of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

Make sure you are inside the pdf-rag folder before activating. Run ls (macOS) or dir (Windows) and confirm you see a venv folder listed.

On Windows, if you see an execution policy error, run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned first, then try activating again.

Help me troubleshoot my virtual environment activation

Install required packages

Your RAG pipeline relies on several Python libraries. LangChain orchestrates the pipeline, ChromaDB stores your vectors locally, and sentence-transformers generates embeddings on your CPU for free.

  • Install all project dependencies by running this command:
pip install langchain==1.3.4 langchain-groq==1.1.2 langchain-huggingface langchain-text-splitters==1.1.2 langchain-community==0.3.30 chromadb==1.5.9 pypdf==6.13.1 sentence-transformers

What are these packages?

  • langchain is the core framework for building LLM-powered applications.
  • langchain-groq connects LangChain to Groq's fast inference API.
  • langchain-huggingface provides local embedding models from HuggingFace.
  • chromadb is a lightweight vector database that runs entirely on your machine.
  • pypdf extracts text from PDF files page by page.
  • sentence-transformers powers the embedding model that converts text into vectors.

The install may take a minute or two as it downloads all dependencies.

  • Verify the installation succeeded by running:
python -c "import langchain; import chromadb; print('Ready!')"

You should see Ready! printed in your terminal with no errors.

Seeing an import error?

Make sure your virtual environment is active. You should see (venv) in your prompt. If not, reactivate it before running pip install again.

If you see a permission error, confirm you are using the pip from inside your virtual environment by running which pip (macOS) or where pip (Windows). The path should include venv.

Help me fix my import error

Configure your Groq API key

Groq provides a free API that runs open-source language models at extreme speed. You will use it to power the answer-generation step of your RAG pipeline.

What is Groq?

Groq is an inference platform that runs open-source models like Llama 3.3 70B on custom LPU hardware. It delivers around 280 tokens per second.

The free tier requires no credit card and you are never charged. If you exceed rate limits, you get a 429 error instead of a bill.

  • Open your browser and go to console.groq.com.
  • Click Sign Up and create an account using your email, Google, or GitHub. No credit card is required.
  • Verify your email if prompted.
  • Once logged in, click API Keys in the left sidebar.
  • Click Create API Key.
  • Give it a name (for example, pdf-rag) and click Submit.
  • Copy the key immediately. Groq only displays it once.

Save your key now

Groq will never show this key again after you close the dialog. If you lose it, you will need to delete it and create a new one. Paste it somewhere safe before continuing.

  • Set your API key as an environment variable by running this command (replace your-api-key-here with the key you just copied):

macOS

export GROQ_API_KEY=your-api-key-here

Windows

set GROQ_API_KEY=your-api-key-here

Why an environment variable?

Storing your key in an environment variable keeps it out of your code. This way you never accidentally commit a secret to version control. The LangChain Groq integration automatically reads GROQ_API_KEY from your environment.

  • Verify the key is set by running:
python -c "import os; print(os.environ.get('GROQ_API_KEY', 'NOT SET')[:8] + '...')"

You should see the first 8 characters of your key followed by ... printed in the terminal.

  • Run one final check to confirm all imports work together:
python -c "import langchain; import chromadb; from langchain_groq import ChatGroq; print('All imports working!')"

You should see All imports working! with no errors.

Seeing NOT SET or an import error?

If you see NOT SET, the environment variable was not saved. Make sure you ran the export (macOS) or set (Windows) command in the same terminal session where you are running Python.

If you see an import error for langchain_groq, re-run the pip install command from the previous substep and make sure (venv) is in your prompt.

Help me fix my API key or import issue

Your environment is ready. Next up, you will load a PDF and split it into chunks that your RAG system can search through.

Load and Chunk Your PDF

Your environment is ready with all packages installed and your Groq API key configured. Now it's time to build the first piece of your RAG pipeline: turning a PDF document into small, searchable pieces of text.

LLMs have a limited context window. They can't read an entire 50-page PDF in one go. The solution is to break the document into smaller "chunks" that can be individually retrieved when a user asks a question.

In this step, get ready to:
  • Place a PDF file in your project folder for the system to read.
  • Write a function that loads the PDF and splits it into overlapping text chunks.
  • Run the script and verify the chunks look correct.
Place a PDF in your project folder

Your RAG system needs a document to work with. Any PDF will do.

  • Find a PDF file on your computer. This could be a research paper, a manual, a report, or any document you'd like to ask questions about.
  • Copy or move the PDF into your pdf-rag folder (the project folder you created in Step 1).
  • Note the exact filename including the .pdf extension. You'll need it in a moment.

Don't have a PDF handy?

Search for any free PDF online. Academic papers from arXiv, government reports, or product manuals all work well. A 5-20 page document is ideal for quick testing.

Create the load and chunk function

Now you'll write the Python function that does two things: extracts text from every page of your PDF, then splits that text into smaller overlapping chunks.

  • Create a new file called rag_pdf.py in your pdf-rag folder by running:
touch rag_pdf.py
  • Confirm the file exists by running:
ls

You should see rag_pdf.py listed alongside your PDF file.

  • Open rag_pdf.py in your code editor, then paste the following code:
import os
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter


def load_and_chunk_pdf(pdf_path):
    """Load a PDF and split it into chunks."""
    # Extract text from every page of the PDF
    loader = PyPDFLoader(pdf_path)
    pages = loader.load()
    print(f"Loaded {len(pages)} pages from {pdf_path}")

    # Split the pages into smaller, overlapping chunks
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000,
        chunk_overlap=200,
    )
    chunks = text_splitter.split_documents(pages)
    print(f"Split into {len(chunks)} chunks")
    return chunks

What does this code do?

  • PyPDFLoader reads the PDF file and extracts the text from each page as a separate document object. Each page retains its metadata (like the page number).
  • RecursiveCharacterTextSplitter breaks those pages into chunks of roughly 1000 characters each. The 200-character overlap means each chunk shares some text with the next one, so context isn't lost at the boundaries.
  • The function returns a list of chunk objects, each containing the text and metadata about which page it came from.

Why overlapping chunks?

Imagine a key sentence that spans two chunks. Without overlap, that sentence gets cut in half and neither chunk captures the full meaning. A 200-character overlap ensures important context near chunk boundaries appears in both chunks.

  • Save rag_pdf.py.

Seeing a ModuleNotFoundError?

  • Make sure your virtual environment is activated. You should see (venv) at the beginning of your terminal prompt.
  • Double check you installed all packages in Step 1. Re-run the pip install command from the previous step if needed.

Help me troubleshoot

Run and inspect the chunks

Time to verify your function works correctly. You'll call it from the terminal and inspect the output.

  • Test your function by running this command (replace the filename with your actual PDF):
python -c "from rag_pdf import load_and_chunk_pdf; chunks = load_and_chunk_pdf('[[PDF_FILE="your-pdf-filename.pdf"]]'); print(f'\nFirst chunk content:\n{chunks[0].page_content[:200]}'); print(f'\nFirst chunk metadata: {chunks[0].metadata}')"

What should I see?

The output shows three things: the number of pages loaded from your PDF, the total number of chunks created (varies by document length), and a preview of the first chunk's text plus its page metadata.

For example, a 10-page document might produce 20-40 chunks depending on how much text is on each page.

  • Verify that the first chunk's text content is readable and comes from the beginning of your PDF.
  • Check that the metadata shows 'page': 0 (pages are zero-indexed).

Seeing an empty output or FileNotFoundError?

  • Make sure the PDF filename in the command exactly matches the file in your pdf-rag folder, including the .pdf extension.
  • Confirm your terminal is in the pdf-rag directory. Run ls to check both rag_pdf.py and your PDF are listed.
  • If your PDF has a space in the filename, wrap the entire name in quotes inside the command.

Help me debug

✔️ Awesome, I've got everything!

Great. Double check you've saved your rag_pdf.py file before moving on.

ⓧ I'd like to double check the full code

Here's the complete rag_pdf.py file so far:

import os
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter


def load_and_chunk_pdf(pdf_path):
    """Load a PDF and split it into chunks."""
    loader = PyPDFLoader(pdf_path)
    pages = loader.load()
    print(f"Loaded {len(pages)} pages from {pdf_path}")

    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000,
        chunk_overlap=200,
    )
    chunks = text_splitter.split_documents(pages)
    print(f"Split into {len(chunks)} chunks")
    return chunks

Your PDF is now loaded and broken into bite-sized chunks. Next up, you'll convert these text chunks into vector embeddings and store them in a database that understands meaning, not just keywords.

Embed and Store in ChromaDB

Your PDF is now loaded and split into bite-sized chunks. But text alone isn't searchable by meaning.

To find relevant passages when you ask a question, you need to convert those chunks into vector embeddings. These are numerical representations that capture the meaning of text. In this step, you'll embed your chunks and store them in a local vector database called ChromaDB.

In this step, get ready to:
  • Initialize a local embedding model that runs on your CPU.
  • Store your embedded chunks in a ChromaDB vector store.
  • Test semantic search by retrieving relevant chunks for a sample question.
Add new imports for embeddings and vector storage

Your script needs two new tools. HuggingFaceEmbeddings converts text into vectors. Chroma stores those vectors in a searchable database on disk.

  • In rag_pdf.py, find the existing import lines at the top of your file:
from langchain_text_splitters import RecursiveCharacterTextSplitter
  • Add these two new import lines directly below it:
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_community.vectorstores import Chroma

What are these imports?

HuggingFaceEmbeddings wraps the sentence-transformers library. It runs a small AI model locally on your machine to convert text into 384-dimensional number arrays (vectors).

Chroma is LangChain's wrapper around ChromaDB. It handles storing, indexing, and searching those vectors for you.

Write the create_vector_store function

This function takes your text chunks, converts each one into a vector embedding, and saves everything into a local ChromaDB database folder.

  • In rag_pdf.py, add this function below the load_and_chunk_pdf function:
def create_vector_store(chunks):
    """Embed chunks and store in ChromaDB."""
    embeddings = HuggingFaceEmbeddings(
        model_name="sentence-transformers/all-MiniLM-L6-v2"
    )
    vector_store = Chroma.from_documents(
        documents=chunks,
        embedding=embeddings,
        persist_directory="./chroma_db",
    )
    print("Vector store created and persisted to ./chroma_db")
    return vector_store

What does this code do?

  • HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2") loads a small (~90MB) embedding model. On first run, it downloads from HuggingFace. After that, it runs entirely on your CPU with no API calls.
  • Chroma.from_documents() takes your chunks, passes each one through the embedding model, and stores the resulting vectors alongside the original text.
  • persist_directory="./chroma_db" tells ChromaDB to save the database to a folder on disk. This means you don't have to re-embed every time you restart the script.
  • Save rag_pdf.py.

Seeing an import error?

  • Make sure your virtual environment is still activated. You should see (venv) at the start of your terminal prompt.
  • Check that from langchain_huggingface import HuggingFaceEmbeddings uses an underscore between langchain and huggingface, not a hyphen.

Still stuck? Help me fix my import error in rag_pdf.py

Test retrieval with a sample query

Time to see semantic search in action. You'll load your PDF, embed it, and then ask a question to see which chunks come back as most relevant.

  • Open the Python interpreter by running this command in your terminal:
python
  • In the Python interpreter, paste the following lines one at a time to test your vector store:
from rag_pdf import load_and_chunk_pdf, create_vector_store
chunks = load_and_chunk_pdf("[[PDF_FILENAME=\"your-pdf-file.pdf\"]]")

What should I see?

The first run downloads the embedding model (~90MB). You'll see a download progress bar. After that, it embeds all your chunks and prints how many were created.

  • Create the vector store and run a similarity search by pasting these lines:
vs = create_vector_store(chunks)
results = vs.similarity_search("what is this document about", k=3)
for r in results:
    print(r.page_content[:100])
    print("---")

What does similarity_search do?

similarity_search("what is this document about", k=3) converts your question into a vector, then finds the 3 chunks whose vectors are closest in meaning. It returns the actual text of those chunks, not just their IDs.

  • You should see three text snippets printed from your PDF that relate to its main topic.
  • Exit the Python interpreter by typing:
exit()
  • Confirm the vector store was persisted to disk by running:
ls chroma_db
  • You should see files and folders inside chroma_db confirming your embeddings are saved locally.

Similarity search returning empty or unrelated results?

  • Make sure you typed your PDF's actual filename correctly (including the .pdf extension).
  • If you see zero chunks, your PDF might be image-based (scanned). Try a different PDF that contains selectable text.
  • If results look unrelated, try a more specific question that matches your document's content.

Need help? Help me debug my ChromaDB similarity search results

✔️ Awesome, I've got everything!

Great. Double check you've saved rag_pdf.py and confirmed the chroma_db folder exists on disk.

ⓧ I'd like to double check the full code

import os
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_community.vectorstores import Chroma


def load_and_chunk_pdf(pdf_path):
    """Load a PDF and split it into chunks."""
    loader = PyPDFLoader(pdf_path)
    pages = loader.load()
    print(f"Loaded {len(pages)} pages from {pdf_path}")

    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000,
        chunk_overlap=200,
    )
    chunks = text_splitter.split_documents(pages)
    print(f"Split into {len(chunks)} chunks")
    return chunks


def create_vector_store(chunks):
    """Embed chunks and store in ChromaDB."""
    embeddings = HuggingFaceEmbeddings(
        model_name="sentence-transformers/all-MiniLM-L6-v2"
    )
    vector_store = Chroma.from_documents(
        documents=chunks,
        embedding=embeddings,
        persist_directory="./chroma_db",
    )
    print("Vector store created and persisted to ./chroma_db")
    return vector_store

Your PDF's knowledge is now searchable by meaning, not just keywords. Next up, you'll connect this retriever to a powerful LLM that generates natural-language answers from the passages it finds.

Build the RAG Question-Answering Chain

Your PDF is now chunked, embedded, and stored in a local ChromaDB vector store. The retrieval engine is ready.

Now you need the generation side of RAG. In this step, you will connect Groq's Llama 3.3 70B model to your vector store and build a question-answering loop. For each question the learner types, the system retrieves the 3 most relevant chunks from the PDF and passes them as context to the LLM.

In this step, get ready to:
  • Set up the ChatGroq LLM with Llama 3.3 70B.
  • Create a RAG prompt and retrieval chain.
  • Build an interactive Q&A loop in the terminal.
Add the new imports

The question-answering function needs two new classes: one to talk to Groq's API and one to build a structured prompt.

  • In rag_pdf.py, find the existing import block at the top of the file.
  • Add these two lines after from langchain_community.vectorstores import Chroma:
from langchain_groq import ChatGroq
from langchain_core.prompts import ChatPromptTemplate

What do these imports do?

  • ChatGroq is a LangChain wrapper that sends requests to Groq's free API. It gives you access to open-source models like Llama 3.3 70B at high speed.
  • ChatPromptTemplate lets you define a reusable message structure (system instructions + user question) that gets filled with variables at runtime.
Create the question-answering function

This function ties together the retrieval and generation steps. It takes a question, fetches the most relevant chunks from your vector store, and asks the LLM to answer using only that context.

  • In rag_pdf.py, add the following function below the create_vector_store function:
def ask_question(vector_store, llm, question):
    """Retrieve relevant chunks and generate an answer."""
    # Turn the vector store into a retriever that fetches the top 3 matches
    retriever = vector_store.as_retriever(search_kwargs={"k": 3})
    relevant_docs = retriever.invoke(question)

    # Combine the retrieved chunks into a single context string
    context = "\n\n".join(doc.page_content for doc in relevant_docs)

    # Define the prompt: system instructions + user question
    prompt = ChatPromptTemplate.from_messages([
        ("system", (
            "You are a helpful assistant that answers questions based on the "
            "provided context. If the answer is not in the context, say "
            "'I don't have enough information to answer that based on the document.' "
            "Keep answers concise (3 sentences max).\n\n"
            "Context:\n{context}"
        )),
        ("human", "{question}"),
    ])

    # Build the chain: prompt -> LLM -> response
    chain = prompt | llm
    response = chain.invoke({"context": context, "question": question})
    return response.content

What does this code do?

  • vector_store.as_retriever(search_kwargs={"k": 3}) converts the vector store into a retriever that returns the 3 most semantically similar chunks to the question.
  • The ChatPromptTemplate gives the LLM strict instructions: answer only from the context, admit when it does not know, and keep answers short.
  • prompt | llm uses LangChain's pipe syntax to create a chain. Data flows left to right: the formatted prompt goes into the LLM, which returns a response.

Why tell the LLM to say it does not know?

Without this instruction, the LLM might hallucinate an answer that sounds confident but is not actually in your document. Explicitly telling it to decline keeps your RAG system trustworthy.

  • Save rag_pdf.py.

You will not see a visible change yet. The next chunk builds the entry point that calls this function.

Write the main function and run the Q&A loop

The main() function orchestrates the entire pipeline. It loads the PDF, builds the vector store, initializes the LLM, and enters an interactive loop where you can keep asking questions.

  • Add the following code at the very bottom of rag_pdf.py, below the ask_question function:
def main():
    # Ask the user which PDF to load
    pdf_path = input("Enter the path to your PDF file: ").strip()

    if not os.path.exists(pdf_path):
        print(f"Error: File '{pdf_path}' not found.")
        return

    print("\n--- Loading and chunking PDF ---")
    chunks = load_and_chunk_pdf(pdf_path)

    print("\n--- Creating vector store (first run downloads embedding model ~90MB) ---")
    vector_store = create_vector_store(chunks)

    print("\n--- Initializing LLM (Llama 3.3 70B on Groq) ---")
    llm = ChatGroq(model="llama-3.3-70b-versatile", temperature=0.1)

    print("\n--- Ready! Ask questions about your PDF ---")
    print("Type 'quit' to exit.\n")

    # Interactive Q&A loop
    while True:
        question = input("Your question: ").strip()
        if question.lower() == "quit":
            print("Goodbye!")
            break
        if not question:
            continue

        answer = ask_question(vector_store, llm, question)
        print(f"\nAnswer: {answer}\n")


if __name__ == "__main__":
    main()

What does this code do?

  • ChatGroq(model="llama-3.3-70b-versatile", temperature=0.1) initializes the Llama 3.3 70B model hosted on Groq. The low temperature keeps answers focused and deterministic.
  • The while True loop continuously accepts questions until you type quit. Each iteration retrieves context and generates a fresh answer.
  • if __name__ == "__main__" ensures the script runs main() only when executed directly, not when imported as a module.
  • Save rag_pdf.py.
  • Run the script by entering this command in your terminal:
python rag_pdf.py
  • When prompted, type the path to your PDF file (e.g. report.pdf if it is in the same folder).
  • Wait for the vector store to build. On first run it downloads a ~90MB embedding model.
  • Once you see Ready! Ask questions about your PDF, type a question about the content of your document.
  • Try asking something that is NOT in the document to confirm the system says it does not have enough information.
  • Type quit to exit.

You should see a concise answer drawn from the relevant passages of your PDF. When you ask something outside the document's scope, the system responds with a refusal instead of hallucinating.

Not getting an answer?

  • If you see an authentication error, confirm your GROQ_API_KEY environment variable is set in the same terminal session where you ran the script.
  • If you see a file-not-found error, make sure the PDF path you entered is correct. If the file is in your pdf-rag folder, just type the filename without any folder prefix.
  • If answers seem unrelated, check that your PDF has selectable text (scanned image PDFs will not work with PyPDFLoader).

Still stuck? Help me debug my RAG script

✔️ Awesome, I've got everything!

Great. Double check you have saved rag_pdf.py with all your changes.

ⓧ I'd like to double check the full code

import os
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_groq import ChatGroq
from langchain_community.vectorstores import Chroma
from langchain_core.prompts import ChatPromptTemplate


def load_and_chunk_pdf(pdf_path):
    """Load a PDF and split it into chunks."""
    loader = PyPDFLoader(pdf_path)
    pages = loader.load()
    print(f"Loaded {len(pages)} pages from {pdf_path}")

    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=1000,
        chunk_overlap=200,
    )
    chunks = text_splitter.split_documents(pages)
    print(f"Split into {len(chunks)} chunks")
    return chunks


def create_vector_store(chunks):
    """Embed chunks and store in ChromaDB."""
    embeddings = HuggingFaceEmbeddings(
        model_name="sentence-transformers/all-MiniLM-L6-v2"
    )
    vector_store = Chroma.from_documents(
        documents=chunks,
        embedding=embeddings,
        persist_directory="./chroma_db",
    )
    print("Vector store created and persisted to ./chroma_db")
    return vector_store


def ask_question(vector_store, llm, question):
    """Retrieve relevant chunks and generate an answer."""
    retriever = vector_store.as_retriever(search_kwargs={"k": 3})
    relevant_docs = retriever.invoke(question)

    context = "\n\n".join(doc.page_content for doc in relevant_docs)

    prompt = ChatPromptTemplate.from_messages([
        ("system", (
            "You are a helpful assistant that answers questions based on the "
            "provided context. If the answer is not in the context, say "
            "'I don't have enough information to answer that based on the document.' "
            "Keep answers concise (3 sentences max).\n\n"
            "Context:\n{context}"
        )),
        ("human", "{question}"),
    ])

    chain = prompt | llm
    response = chain.invoke({"context": context, "question": question})
    return response.content


def main():
    pdf_path = input("Enter the path to your PDF file: ").strip()

    if not os.path.exists(pdf_path):
        print(f"Error: File '{pdf_path}' not found.")
        return

    print("\n--- Loading and chunking PDF ---")
    chunks = load_and_chunk_pdf(pdf_path)

    print("\n--- Creating vector store (first run downloads embedding model ~90MB) ---")
    vector_store = create_vector_store(chunks)

    print("\n--- Initializing LLM (Llama 3.3 70B on Groq) ---")
    llm = ChatGroq(model="llama-3.3-70b-versatile", temperature=0.1)

    print("\n--- Ready! Ask questions about your PDF ---")
    print("Type 'quit' to exit.\n")

    while True:
        question = input("Your question: ").strip()
        if question.lower() == "quit":
            print("Goodbye!")
            break
        if not question:
            continue

        answer = ask_question(vector_store, llm, question)
        print(f"\nAnswer: {answer}\n")


if __name__ == "__main__":
    main()

Secret mission

Add Source Citations

Your RAG system gives great answers, but where did it find that information? Enhance your tool to show exactly which page and passage each answer came from. This turns your Q&A system into something you could actually trust for research.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Your Groq API key uses the free tier and you are never charged.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • pdf-rag project folder on your Desktop (contains rag_pdf.py, venv, chroma_db, and your PDF file).
  • HuggingFace model cache (~90MB of the embedding model stored at ~/.cache/huggingface/hub on macOS/Linux or C:\Users\username\.cache\huggingface\hub on Windows).
  • Groq account and API key (free tier, no charges).

Keep everything running

No action needed. Choose this if you are still actively building or want to come back to this project soon.

  • Your pdf-rag folder stays on your Desktop, ready to use anytime.
  • The HuggingFace embedding model remains cached so future runs skip the download step.
  • Your Groq account stays active on the free tier. You are never charged.

Pause - I'll come back to this later

Free up terminal resources while keeping your files intact. Since everything is local and free, pausing just means closing your terminal session.

  • Close your terminal window. The GROQ_API_KEY environment variable was session-only and disappears automatically.
  • Your pdf-rag folder, virtual environment, and chroma_db data all remain on disk.
  • When you return, reactivate the virtual environment and re-set the GROQ_API_KEY environment variable before running the script again.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each step below to clean up completely.

  • Delete the project folder by running this command (macOS/Linux):
cd ~/Desktop
rm -rf pdf-rag
  • On Windows, run this instead:
cd ~\Desktop
rmdir /s /q pdf-rag
  • Remove the cached HuggingFace embedding model (macOS/Linux):
rm -rf ~/.cache/huggingface/hub/models--sentence-transformers--all-MiniLM-L6-v2
  • On Windows, run this instead:
rmdir /s /q %USERPROFILE%\.cache\huggingface\hub\models--sentence-transformers--all-MiniLM-L6-v2
  • Your Groq API key was set as a session-only environment variable. It disappeared when your terminal closed. No further action needed.
  • Your Groq account remains active on the free tier with no charges. If you want to revoke your API key, visit the API Keys section at console.groq.com and delete the key.

That's a Wrap!

That's a Wrap!

Nice work! You just built a fully functional RAG system that answers questions about any PDF you feed it.

You've learned how to:

  • Built a complete RAG pipeline from scratch, covering PDF loading, text chunking, vector embedding, and semantic retrieval.
  • Used ChromaDB as a local vector store and free sentence-transformers embeddings to persist and search document chunks at zero cost.
  • Connected Groq's open-source Llama 3.3 70B model to generate grounded answers from retrieved context through an interactive terminal Q&A loop.
  • Secret Mission: Added source citations showing which pages and passages each answer came from.

Ready to quiz yourself?