Build a RAG Chatbot with LangChain

Build a document Q&A chatbot using RAG, ChromaDB, and OpenAI.

Introduction

30 Second Summary

Imagine asking an AI chatbot a question about a specific report or manual, and getting a vague answer pulled from its general training data. The information you need is sitting right there in your document, but the AI can't see it.

In this project, you will build a RAG chatbot that reads a PDF document and answers your questions using only the information from that document. You'll see exactly which pages each answer came from.

What You'll Build

A command-line chatbot that ingests a PDF into a local vector database, then uses retrieval-augmented generation to answer questions with source citations.

By the end of this project, you'll have:

  • A Python script that loads, chunks, and embeds a PDF into a local ChromaDB vector database.
  • A retrieval chain built with LangChain that finds relevant context and generates grounded answers using OpenAI's GPT-5.4 mini.
  • An interactive chat loop with source attribution showing which document pages were used for each answer.
  • Secret Mission: Add conversation memory so the chatbot handles follow-up questions like "Tell me more about that."

Are there any prerequisites?

You'll need basic Python knowledge (variables, functions, and running scripts). You'll also need an OpenAI account with an API key and a small amount of credit loaded.

Set Up Your Python Environment

To build a RAG chatbot, you need Python libraries that handle each stage of the pipeline: loading documents, splitting text, creating embeddings, and communicating with an LLM.

Your environment already has all the required packages installed. In this step, you'll open your project in Cursor and create the main Python file that imports everything you need.

In this step, get ready to:
  • Open your project folder in Cursor.
  • Create rag_chatbot.py with all required imports.
Open your project folder in Cursor

Cursor needs to know which folder your project lives in so it can index your files and provide helpful suggestions as you code.

  • In Cursor, click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your rag-chatbot folder on your Desktop, then click Open.

You should see the folder name appear in Cursor's Explorer sidebar on the left.

Don't see the Explorer sidebar?

Press Cmd+Shift+E (macOS) or Ctrl+Shift+E (Windows) to toggle the Explorer panel.

If your folder still doesn't appear, make sure you selected the correct folder in the Open Folder dialog. Help me open my project folder in Cursor.

Create the main chatbot file

Your RAG chatbot will live in a single Python file that orchestrates the entire pipeline. Each import handles a different responsibility: loading PDFs, splitting text, creating embeddings, storing vectors, and generating answers.

  • In Cursor's Explorer sidebar, hover over your project folder name and click the new file icon (a page with a + symbol).
  • Name the file rag_chatbot.py and press Enter.
  • Paste the following code into rag_chatbot.py:
import os
from dotenv import load_dotenv
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_chroma import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

load_dotenv()

What does this code do?

  • load_dotenv() reads your .env file so your API key is available as an environment variable without hardcoding it.
  • PyPDFLoader reads PDF files and converts each page into a document object that LangChain can process.
  • RecursiveCharacterTextSplitter breaks long documents into smaller chunks that fit within an embedding model's token limit.
  • OpenAIEmbeddings and ChatOpenAI connect to OpenAI's embedding and chat models.
  • Chroma provides the local vector database that stores and searches your document embeddings.
  • ChatPromptTemplate, RunnablePassthrough, and StrOutputParser are the building blocks for LangChain's composable chain pattern (LCEL).
  • Save rag_chatbot.py by pressing Cmd+S (macOS) or Ctrl+S (Windows).
  • Open the terminal in Cursor by pressing Ctrl+` (backtick key, both platforms).
  • Verify your imports work by running:
python rag_chatbot.py

You should see no output and no errors. The script imports all libraries and exits cleanly.

Seeing a ModuleNotFoundError?

This means a package is missing from your virtual environment. Make sure your virtual environment is activated (you should see its name in parentheses in your terminal prompt).

If the error names a specific package, reinstall it by running:

pip install langchain langchain-openai langchain-chroma langchain-community pypdf python-dotenv

Help me fix my import errors in rag_chatbot.py.

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

Here's the complete rag_chatbot.py so far:

import os
from dotenv import load_dotenv
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_chroma import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

load_dotenv()

Your project skeleton is ready with all the imports wired up. Next up, you'll configure your OpenAI API key and load your first PDF document into the pipeline.

Configure OpenAI and Load Your Document

Your Python environment is ready with all the dependencies installed. Now it's time to connect your chatbot to OpenAI and give it something to read.

Every RAG system needs two things: access to an LLM for generating answers, and a document to ground those answers in. In this step, you'll configure your API key and load a PDF into your script.

In this step, get ready to:
  • Create a .env file to store your OpenAI API key securely.
  • Place a sample PDF document in your project folder.
  • Load the PDF into your script using LangChain's document loader.
Create your .env file

Your script needs an OpenAI API key to generate embeddings and chat responses. Storing the key in a .env file keeps it out of your source code.

  • In Cursor, create a new file in your project folder called .env.
  • Add the following line to the file:
OPENAI_API_KEY=your-api-key-here
  • Replace your-api-key-here with your actual OpenAI API key from your OpenAI account.
  • Save .env.

What does this file do?

The .env file stores environment variables that python-dotenv loads into your script at runtime. The load_dotenv() call already in your script reads this file automatically.

This keeps your API key separate from your code so you never accidentally share it.

Not sure where to find your API key?

Log in to your OpenAI account and navigate to the API keys page. If you haven't created a key yet, click Create new secret key and copy it immediately. You won't be able to see it again after closing the dialog.

help me find my OpenAI API key

Add a PDF to your project

Your chatbot needs a document to answer questions about. Any PDF will work for this project.

  • Find any PDF document you'd like to ask questions about. This could be a research paper, a report, a textbook chapter, or any document with readable text.
  • Copy or move the PDF into your project folder (the same folder that contains rag_chatbot.py).
  • Rename the file to document.pdf.

Pick something you're curious about

The chatbot works best when you genuinely want to ask questions about the document. A 5-20 page PDF is ideal for testing. Longer documents work too but take slightly longer to process.

  • Confirm the file exists by running:
ls document.pdf

You should see document.pdf listed in the output.

File not found?

Make sure you renamed the file to exactly document.pdf (all lowercase, no spaces). Confirm it's in the same folder as rag_chatbot.py, not in a subfolder.

help me check if my PDF is in the right location

Load the PDF with PyPDFLoader

LangChain provides document loaders that handle different file formats. PyPDFLoader reads each page of a PDF into a separate document object.

  • In rag_chatbot.py, find the line that says load_dotenv(), then add the following code below it:
# Step 1: Load the PDF
loader = PyPDFLoader("document.pdf")
docs = loader.load()
print(f"Loaded {len(docs)} pages from the PDF.")

What does this code do?

  • PyPDFLoader("document.pdf") creates a loader pointed at your PDF file.
  • loader.load() reads every page and returns a list of document objects, one per page.
  • Each document object contains the page text and metadata (like the page number).
  • The print statement confirms how many pages were loaded so you know the ingestion worked.
  • Save rag_chatbot.py.
  • Run your script in the terminal:
python rag_chatbot.py

You should see output like Loaded 12 pages from the PDF. (the number depends on your PDF).

Seeing an authentication error?

If you see a message about an invalid or missing API key, double-check that your .env file is saved in the same folder as rag_chatbot.py and that there are no extra spaces or quotes around your key value.

If you see a FileNotFoundError, make sure your PDF is named exactly document.pdf and is in the project folder.

help me debug why my RAG script isn't loading my PDF

✔️ Awesome, I've got everything!

Great job. Double check you've saved both .env and rag_chatbot.py, then move on to the next step.

ⓧ I'd like to double check the full code

Here is the full .env file:

OPENAI_API_KEY=your-api-key-here

Here is the full rag_chatbot.py so far:

import os
from dotenv import load_dotenv
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_chroma import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

load_dotenv()

# Step 1: Load the PDF
loader = PyPDFLoader("document.pdf")
docs = loader.load()
print(f"Loaded {len(docs)} pages from the PDF.")

Your chatbot can now read a PDF document. Next up, you'll split the document into smaller chunks and store them as embeddings in a vector database.

Chunk and Embed Your Document

Your script can now load a PDF into memory. However, an entire document is too large to send directly to an LLM as context.

In this step, you'll split the document into smaller pieces called chunks, then convert those chunks into embeddings. Embeddings are numerical representations of text that capture meaning. You'll store them in ChromaDB, a local vector database that lets you search by similarity later.

In this step, get ready to:
  • Split your PDF into smaller, overlapping chunks.
  • Create embeddings and store them in a persistent ChromaDB vector store.
  • Verify the chunk count and confirm the database directory exists on disk.
Split the document into chunks

LLMs have a limited context window. Instead of sending the entire document at once, you break it into overlapping pieces so the model can focus on the most relevant sections when answering a question.

  • In rag_chatbot.py, find the line that says print(f"Loaded {len(docs)} pages from the PDF."), then copy the following code below it:
# Step 2: Split into chunks
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200,
)
chunks = text_splitter.split_documents(docs)
print(f"Split into {len(chunks)} chunks.")

What does this code do?

RecursiveCharacterTextSplitter breaks your document into pieces of roughly 1000 characters each. The chunk_overlap=200 setting means each chunk shares 200 characters with the next one.

Overlap prevents important sentences from being cut in half at chunk boundaries. If a key fact straddles two chunks, the overlap ensures it appears fully in at least one of them.

  • Save rag_chatbot.py.

You won't see a change yet. The next chunk stores these pieces in a database so you can search them.

Seeing a red underline on RecursiveCharacterTextSplitter?

Make sure the import from langchain_text_splitters import RecursiveCharacterTextSplitter is at the top of your file. It should already be there from the previous step.

If the error persists, confirm your virtual environment is active and run pip install langchain-text-splitters to reinstall the package.

Help me fix an import error with RecursiveCharacterTextSplitter in my RAG chatbot project.

Create embeddings and store them in ChromaDB

Now you'll convert each chunk into a numerical vector using OpenAI's text-embedding-3-small model. These vectors capture the meaning of each chunk so you can find the most relevant ones later by measuring similarity.

  • In rag_chatbot.py, add the following code below the print(f"Split into {len(chunks)} chunks.") line:
# Step 3: Create embeddings and store in ChromaDB
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vector_store = Chroma(
    collection_name="my_document",
    embedding_function=embeddings,
    persist_directory="./chroma_db",
)
vector_store.add_documents(documents=chunks)
print(f"Stored {len(chunks)} chunks in ChromaDB.")

What does this code do?

  • OpenAIEmbeddings(model="text-embedding-3-small") creates an embedding function that calls OpenAI's API to turn text into 1536-dimensional vectors.
  • Chroma(...) creates a local vector database. The persist_directory="./chroma_db" parameter saves the database to disk so it survives between script runs.
  • vector_store.add_documents(documents=chunks) embeds each chunk and stores both the vector and the original text in the database.

How much does this cost?

The text-embedding-3-small model costs $0.02 per 1 million tokens. A typical 20-page PDF has around 10,000-30,000 tokens. That means embedding your entire document costs a fraction of a cent.

  • Save rag_chatbot.py.
  • Run the script in your terminal by running:
python rag_chatbot.py
  • You should see three print statements showing the number of pages loaded, the number of chunks created, and the number of chunks stored in ChromaDB.
  • Confirm the database was created by running:
ls chroma_db

You should see files inside the chroma_db folder. This is your persisted vector database.

Seeing an authentication error?

This means your OpenAI API key is missing or invalid. Check that your .env file contains OPENAI_API_KEY= followed by your actual key with no spaces or quotes around the value.

Also verify your OpenAI account has credit available. The API requires a positive balance even for very small requests.

Help me debug an OpenAI authentication error when running my RAG chatbot script.

✔️ Awesome, I've got everything!

Great. Double check you've saved rag_chatbot.py and move on to the next step.

ⓧ I'd like to double check the full code

Here is the full rag_chatbot.py file as it should look at the end of this step:

import os
from dotenv import load_dotenv
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_chroma import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

load_dotenv()

# Step 1: Load the PDF
loader = PyPDFLoader("document.pdf")
docs = loader.load()
print(f"Loaded {len(docs)} pages from the PDF.")

# Step 2: Split into chunks
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200,
)
chunks = text_splitter.split_documents(docs)
print(f"Split into {len(chunks)} chunks.")

# Step 3: Create embeddings and store in ChromaDB
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vector_store = Chroma(
    collection_name="my_document",
    embedding_function=embeddings,
    persist_directory="./chroma_db",
)
vector_store.add_documents(documents=chunks)
print(f"Stored {len(chunks)} chunks in ChromaDB.")

Your document is now split into searchable chunks and stored as vectors in a local database. Next up, you'll build the retrieval chain that finds the right chunks and feeds them to GPT-5.4 mini to generate grounded answers.

Build the RAG Question-Answering Chain

Your document chunks are safely stored as embeddings in ChromaDB. Now it is time to build the core intelligence of your chatbot.

When a user asks a question, the system needs to find the most relevant chunks and feed them to an LLM that generates a grounded answer. This is the "retrieval" and "generation" in RAG. In this step, you will wire these pieces together using LangChain's Expression Language (LCEL).

In this step, get ready to:
  • Set up a retriever that finds the 3 most relevant chunks for any question.
  • Create a prompt template that constrains the LLM to answer from context only.
  • Wire the retriever, prompt, and LLM into a single chain using LCEL.
Set up the retriever and prompt template

A retriever is a LangChain component that takes a question and returns the most similar document chunks from your vector store. You also need a prompt template that tells the LLM exactly how to behave.

  • In rag_chatbot.py, find the last line that says print(f"Stored {len(chunks)} chunks in ChromaDB.").
  • Add the following code below that line to create a retriever and define the prompt:
# Step 4: Set up retriever
retriever = vector_store.as_retriever(search_kwargs={"k": 3})

# Step 5: Create the RAG chain
template = """Answer the question based only on the following context. If the context does not contain enough information to answer the question, say "I don't have enough information to answer that."

Context:
{context}

Question: {question}

Answer:"""

prompt = ChatPromptTemplate.from_template(template)
llm = ChatOpenAI(model="gpt-5.4-mini", temperature=0)

What does this code do?

  • vector_store.as_retriever(search_kwargs={"k": 3}) creates a retriever that returns the top 3 most similar chunks for any query. The k parameter controls how many chunks are retrieved.
  • The template string instructs the LLM to answer only from the provided context. If the context lacks relevant information, the LLM admits it does not know rather than making something up.
  • ChatPromptTemplate.from_template() creates a reusable prompt with {context} and {question} placeholders that get filled at runtime.
  • ChatOpenAI(model="gpt-5.4-mini", temperature=0) initialises the LLM. Setting temperature=0 makes responses deterministic and factual.

Why constrain the LLM to context only?

Without this constraint, the LLM might answer from its general training data instead of your document. The prompt template acts as a guardrail, ensuring every answer is grounded in the retrieved chunks.

This is what makes RAG different from a generic chatbot. The LLM becomes a reasoning engine over your specific data rather than a general knowledge assistant.

  • Save rag_chatbot.py.

You will not see a visible change yet. The next chunk wires everything into a working chain.

Wire the RAG chain together with LCEL

LCEL (LangChain Expression Language) lets you compose components using the pipe operator |. Each component passes its output to the next, forming a processing pipeline.

  • Add the following code directly below the llm = ChatOpenAI(...) line you just added:

def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)


rag_chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt
    | llm
    | StrOutputParser()
)

What does this code do?

  • format_docs() takes the list of retrieved document objects and joins their text content into a single string, separated by blank lines. This is what gets inserted into the prompt's {context} placeholder.
  • The first part of the chain is a dictionary that runs two paths in parallel. The "context" key pipes the question through the retriever (which fetches the top 3 chunks), then through format_docs to turn them into text.
  • RunnablePassthrough() passes the raw question string through unchanged for the {question} placeholder.
  • The filled prompt then flows to the LLM, and StrOutputParser() extracts the plain text response from the LLM's output object.
  • Save rag_chatbot.py.

Your RAG chain is now complete. When you call rag_chain.invoke("some question"), the question flows through retrieval, formatting, prompting, and generation in a single call.

Seeing an import error?

If you see an error about RunnablePassthrough or StrOutputParser not being defined, check your import block at the top of the file. Both should already be imported from the prior step.

Confirm you have from langchain_core.runnables import RunnablePassthrough and from langchain_core.output_parsers import StrOutputParser at the top of your file.

help me fix an import error in my LangChain RAG chain

Understand source attribution

A good RAG chatbot does not just give answers. It tells you where those answers came from.

In the next step, you will build the chat loop. Inside that loop, you will call retriever.invoke(question) separately to get the source documents and extract page numbers for attribution. This gives your users confidence that the answer is grounded in real data.

Why retrieve documents separately for sources?

The rag_chain returns a plain string answer. To show which pages were used, you invoke the retriever independently and read each document's metadata (which includes the page number from the PDF loader).

This pattern is common in production RAG systems where transparency and traceability matter.

✔️ Awesome, I've got everything!

Great. Double check you have saved rag_chatbot.py before moving on.

ⓧ I'd like to double check the full code

Here is the full rag_chatbot.py so far. Compare your file against this:

import os
from dotenv import load_dotenv
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_chroma import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

load_dotenv()

# Step 1: Load the PDF
loader = PyPDFLoader("document.pdf")
docs = loader.load()
print(f"Loaded {len(docs)} pages from the PDF.")

# Step 2: Split into chunks
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200,
)
chunks = text_splitter.split_documents(docs)
print(f"Split into {len(chunks)} chunks.")

# Step 3: Create embeddings and store in ChromaDB
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vector_store = Chroma(
    collection_name="my_document",
    embedding_function=embeddings,
    persist_directory="./chroma_db",
)
vector_store.add_documents(documents=chunks)
print(f"Stored {len(chunks)} chunks in ChromaDB.")

# Step 4: Set up retriever
retriever = vector_store.as_retriever(search_kwargs={"k": 3})

# Step 5: Create the RAG chain
template = """Answer the question based only on the following context. If the context does not contain enough information to answer the question, say "I don't have enough information to answer that."

Context:
{context}

Question: {question}

Answer:"""

prompt = ChatPromptTemplate.from_template(template)
llm = ChatOpenAI(model="gpt-5.4-mini", temperature=0)


def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)


rag_chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt
    | llm
    | StrOutputParser()
)

Your RAG chain is assembled and ready to answer questions. Next up, you will wrap this in an interactive chat loop so you can have a real conversation with your document.

Create an Interactive Chat Loop

You've assembled a full RAG chain that can retrieve relevant context and generate grounded answers. The missing piece is a way for you to actually talk to it.

In this step, you'll build a terminal-based chat interface. The loop reads your questions, invokes the RAG chain, and prints answers alongside the source pages that contributed to each response.

In this step, get ready to:
  • Build an interactive loop that reads user questions from the terminal.
  • Invoke the RAG chain and display answers with source page attribution.
  • Test the chatbot with questions about your document.
Add the interactive chat loop

The chat loop ties everything together. It continuously reads input, passes each question through the retrieval and generation pipeline, then shows you which pages the answer came from.

  • In rag_chatbot.py, scroll to the bottom of the file (below the rag_chain definition).
  • Add the following code at the very end of the file:
# Step 6: Interactive chat loop
print("\n--- RAG Chatbot Ready ---")
print("Ask questions about your document. Type 'quit' to exit.\n")

while True:
    question = input("You: ")
    if question.lower() in ("quit", "exit", "q"):
        print("Goodbye!")
        break

    # Get answer
    answer = rag_chain.invoke(question)

    # Get source documents for attribution
    source_docs = retriever.invoke(question)
    sources = set()
    for doc in source_docs:
        page = doc.metadata.get("page", "unknown")
        sources.add(f"Page {page + 1}")

    print(f"\nAssistant: {answer}")
    print(f"Sources: {', '.join(sorted(sources))}\n")

What does this code do?

The loop prints a welcome banner, then waits for your input with input("You: "). If you type quit, exit, or q, it breaks out of the loop.

For each question, it does two things. First, it calls rag_chain.invoke(question) to retrieve relevant chunks and generate an answer. Second, it separately calls retriever.invoke(question) to get the source documents so it can display which pages contributed to the answer.

The page numbers use page + 1 because PyPDFLoader indexes pages starting from 0, but humans count pages starting from 1.

  • Save rag_chatbot.py.

Seeing a syntax error?

Make sure the while True: block is at the top level of the file (not indented inside another function). The entire chat loop should align with the rag_chain definition above it.

Check that all f-strings use matching quotes. The print(f"Sources: {', '.join(sorted(sources))}\n") line uses double quotes on the outside and single quotes inside the join call.

help me fix a syntax error in my RAG chatbot chat loop

✔️ Awesome, I've got everything!

Great. Make sure you've saved rag_chatbot.py before running it in the next substep.

ⓧ I'd like to double check the full code

Here is the complete rag_chatbot.py file. Compare your code against this:

import os
from dotenv import load_dotenv
from langchain_community.document_loaders import PyPDFLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain_chroma import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser

load_dotenv()

# Step 1: Load the PDF
loader = PyPDFLoader("document.pdf")
docs = loader.load()
print(f"Loaded {len(docs)} pages from the PDF.")

# Step 2: Split into chunks
text_splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200,
)
chunks = text_splitter.split_documents(docs)
print(f"Split into {len(chunks)} chunks.")

# Step 3: Create embeddings and store in ChromaDB
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
vector_store = Chroma(
    collection_name="my_document",
    embedding_function=embeddings,
    persist_directory="./chroma_db",
)
vector_store.add_documents(documents=chunks)
print(f"Stored {len(chunks)} chunks in ChromaDB.")

# Step 4: Set up retriever
retriever = vector_store.as_retriever(search_kwargs={"k": 3})

# Step 5: Create the RAG chain
template = """Answer the question based only on the following context. If the context does not contain enough information to answer the question, say "I don't have enough information to answer that."

Context:
{context}

Question: {question}

Answer:"""

prompt = ChatPromptTemplate.from_template(template)
llm = ChatOpenAI(model="gpt-5.4-mini", temperature=0)


def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)


rag_chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt
    | llm
    | StrOutputParser()
)

# Step 6: Interactive chat loop
print("\n--- RAG Chatbot Ready ---")
print("Ask questions about your document. Type 'quit' to exit.\n")

while True:
    question = input("You: ")
    if question.lower() in ("quit", "exit", "q"):
        print("Goodbye!")
        break

    # Get answer
    answer = rag_chain.invoke(question)

    # Get source documents for attribution
    source_docs = retriever.invoke(question)
    sources = set()
    for doc in source_docs:
        page = doc.metadata.get("page", "unknown")
        sources.add(f"Page {page + 1}")

    print(f"\nAssistant: {answer}")
    print(f"Sources: {', '.join(sorted(sources))}\n")
Run and test the chatbot

Time to talk to your document. You'll run the script and ask a few questions to confirm the chatbot returns grounded answers with source citations.

  • Make sure your virtual environment is still active in the terminal.
  • Run the chatbot by entering this command:
python rag_chatbot.py

You should see the loading messages for your PDF, followed by the chatbot banner:

Loaded X pages from the PDF.
Split into Y chunks.
Stored Y chunks in ChromaDB.

--- RAG Chatbot Ready ---
Ask questions about your document. Type 'quit' to exit.

You:
  • Ask a question about your PDF document. Try something specific to the content of the document you loaded.
  • Ask 2-3 more questions to see how the chatbot handles different topics from the document.
  • Try asking something NOT in your document. The chatbot should respond with "I don't have enough information to answer that."
  • Type quit to exit when you're done testing.

What should I see?

Each answer should be relevant to your document's content. Below each answer, you'll see a "Sources" line listing specific page numbers (e.g., "Sources: Page 2, Page 5, Page 7"). These are the pages the retriever found most relevant to your question.

If the chatbot makes up information not in your PDF, or if source pages seem random, double-check that your document.pdf loaded correctly in a previous step.

Getting an API error or timeout?

Check that your .env file contains a valid OPENAI_API_KEY with available credits. The chatbot calls the OpenAI API twice per question (once for the chain, once for the retriever's embedding).

If you see a rate limit error, wait a few seconds and try again. For small documents, each question costs a fraction of a cent.

help me debug an API error in my RAG chatbot

Secret mission

Add Conversation Memory

Your chatbot answers one question at a time, but real conversations have context. What if you ask "Tell me more about that" and the chatbot has no idea what "that" refers to? In this secret mission, add multi-turn memory so follow-up questions just work.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

This project runs entirely locally, so there are no ongoing costs. OpenAI API charges only occur when you run the chatbot script.

Resources you used:

  • Your project folder on the Desktop containing rag_chatbot.py, .env, document.pdf, and the chroma_db/ directory.
  • A Python virtual environment inside your project folder.
  • An OpenAI API key configured for this project.

Keep everything running

No action needed. Choose this if you're still actively building or experimenting with your RAG chatbot.

  • Your project folder stays on your Desktop. You can run the chatbot again any time with python rag_chatbot.py.
  • Your chroma_db/ directory retains your embedded document chunks. Re-running the script will add duplicate chunks unless you clear the database first.
  • Your OpenAI API key remains active. You are only charged when you make API calls.

Pause - I'll come back to this later

Deactivate your virtual environment to free up terminal state. Your files and database stay intact for next time.

  • Deactivate your virtual environment by running:
deactivate
  • No other processes to stop. The chatbot only runs when you execute the script.
  • Your project folder, chroma_db/ database, and .env file all remain on disk, ready for you to return.

Delete - I don't want to use this again

Remove all project files and optionally revoke your API key.

  • Deactivate your virtual environment (if it is still active) by running:
deactivate
  • Delete your entire project folder from the Desktop by running:

macOS

cd ~/Desktop
rm -rf [[PROJECT_FOLDER="your project folder name"]]

Windows

cd ~\Desktop
Remove-Item -Recurse -Force [[PROJECT_FOLDER="your project folder name"]]

This removes rag_chatbot.py, .env, document.pdf, the chroma_db/ directory, and the virtual environment all at once.

  • Optionally, revoke your OpenAI API key to prevent any future charges.
  • Open platform.openai.com/api-keys in your browser.
  • Click the trashcan icon next to the API key you used for this project.

Deleting your key stops all apps using it

If you use this API key in other projects, deleting it will break those projects too. Only delete it if you created it specifically for this project.

Nice Work!

Nice Work!

Nice work! You just built a production-ready RAG pipeline from scratch. Your chatbot can now read a PDF, find the most relevant information, and answer questions grounded in real data.

You've learned how to:

  • Loaded and chunked a PDF into vector embeddings stored in a persistent ChromaDB database.
  • Built a retrieval chain that finds the most relevant document chunks using similarity search.
  • Connected retrieval to GPT-5.4 mini to generate grounded answers with source citations.
  • Added conversation memory so the chatbot handles follow-up questions across multi-turn interactions.

Ready to quiz yourself?