Build a Study Notes Q&A Bot with RAG

Introduction

30 Second Summary

Studying for exams often means flipping through dozens of pages of notes trying to find one specific concept. What if you could just ask your notes a question and get the answer instantly?

In this project, you will build a command-line Q&A bot that reads your university PDF notes and answers questions using AI. Your bot retrieves the most relevant sections from your notes before generating each answer, so responses stay grounded in what you actually studied.

What You'll Build

A Python application that uses Retrieval-Augmented Generation (RAG) to ingest PDF documents, store them as vector embeddings in a persistent database, and answer natural language questions grounded in your actual notes.

By the end of this project, you'll have:

• A document ingestion pipeline that chunks and embeds your PDF notes into a searchable vector database.
• An interactive query script that retrieves relevant chunks and generates grounded answers using a free LLM via OpenRouter.
• A persistent ChromaDB database you can reuse across sessions without re-indexing your notes.
• Secret Mission: Add source citations so every answer shows which PDF and page the information came from.

Are there any prerequisites?

You'll need basic Python scripting experience. During setup, you'll create a free OpenRouter account to access an LLM at no cost.

Set Up Your Environment

To build a Q&A bot that answers questions from your study notes, you need a few tools working together: an AI model to generate answers, an embedding model to understand your documents, and a vector database to store and search through them.

In this step, you'll create a free account on OpenRouter (which gives you access to AI models), install the Python packages that power the RAG pipeline, and organize your PDF notes for ingestion.

In this step, get ready to:

• Create an OpenRouter account and generate an API key.
• Install RAG pipeline packages in a Python virtual environment.
• Add your PDF notes to the project folder.

Create an OpenRouter account and get your API key

OpenRouter is a free API gateway that gives you access to 25+ large language models. You'll use it to send your study questions to an AI model that generates answers grounded in your notes.

• Open your browser and navigate to https://openrouter.ai.
• Click Sign Up and create a free account using your email or a social login.
• Once signed in, navigate to https://openrouter.ai/settings/keys.
• Click Create Key.
• Give your key a name (e.g. study-bot) and click Create.
• Copy the key and save it somewhere safe (a text file or password manager). You will not be able to see this key again after you close the dialog.

What is OpenRouter?

OpenRouter is a unified API that routes your requests to many different AI models. The free tier gives you access to 25+ free models with 50 requests per day. That is more than enough for querying your study notes.

Your API key authenticates your requests. It is like a password for the service, so never share it publicly or commit it to version control.

Don't see a Create Key button?

Make sure you are on the correct page: https://openrouter.ai/settings/keys. If you just created your account, try refreshing the page. Help me find the API key page on OpenRouter.

Create your project and install dependencies

You need a dedicated folder for your project and a virtual environment to keep your packages isolated from other Python projects on your machine.

• Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
• Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.
• Move to your Desktop by running this command:

cd ~/Desktop

• Create your project folder and move into it by running these commands:

mkdir study-notes-rag
cd study-notes-rag

• Create a Python virtual environment by running:

python3 -m venv venv

• Activate the virtual environment by running:

macOS

source venv/bin/activate

You should see (venv) appear at the beginning of your terminal prompt.

Windows

venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt.

What is a virtual environment?

A virtual environment is an isolated Python installation inside your project folder. Packages you install here won't conflict with other Python projects on your system. The (venv) prefix in your prompt confirms it is active.

• Install all the required packages by running:

pip install llama-index-core llama-index-llms-openrouter llama-index-embeddings-huggingface llama-index-readers-file llama-index-vector-stores-chroma chromadb

What are these packages?

• llama-index-core is the core framework that orchestrates the RAG pipeline.
• llama-index-llms-openrouter connects LlamaIndex to the OpenRouter API for text generation.
• llama-index-embeddings-huggingface runs a local embedding model to convert text into vectors.
• llama-index-readers-file handles reading PDF files from disk.
• llama-index-vector-stores-chroma integrates ChromaDB as the vector store for LlamaIndex.
• chromadb is the vector database that stores and searches your document embeddings.

The installation may take a few minutes as it downloads the packages and their dependencies.

• Verify the installation by running this quick import check:

python3 -c "from llama_index.core import VectorStoreIndex; from llama_index.llms.openrouter import OpenRouter; from llama_index.embeddings.huggingface import HuggingFaceEmbedding; import chromadb; print('All packages installed successfully!')"

You should see All packages installed successfully! printed in your terminal.

Seeing an import error?

Make sure your virtual environment is still active. Look for (venv) in your terminal prompt. If it is missing, reactivate it with source venv/bin/activate (macOS) or venv\Scripts\activate (Windows), then re-run the pip install command.

Help me fix a package import error in my study notes RAG project.

Add your PDF notes and open VS Code

Your RAG pipeline needs source documents to work with. You'll create a data folder and add your own university PDF notes to it.

• Create the data folder inside your project by running:

mkdir data

• Open Finder (macOS) or File Explorer (Windows) and navigate to Desktop > study-notes-rag > data.
• Copy 2-3 of your university PDF notes into the data folder. These can be lecture slides, textbook chapters, or any study material you want to query.
• Confirm the files are in place by running:

ls data

You should see your PDF file names listed in the output.

Pick focused notes

For best results, choose PDFs from a single subject area. The RAG pipeline works best when the source material is topically related. Avoid mixing unrelated subjects in the same data folder for now.

• Open VS Code in your project folder by running:

code .

VS Code opens with your study-notes-rag folder visible in the Explorer sidebar. You should see the data folder and the venv folder listed.

VS Code not opening from the terminal?

If the code . command is not recognized, open VS Code manually. Press Cmd+Space (macOS) or the Windows key (Windows), type VS Code, and press Enter. Then click File > Open Folder and select the study-notes-rag folder on your Desktop.

Help me open my project folder in VS Code.

Your environment is ready. Next up, you'll write the ingestion script that loads your PDFs, splits them into chunks, and stores them as vectors in ChromaDB.

Build the Document Ingestion Pipeline

Your environment is ready with all the packages installed and your PDF notes sitting in the data folder.

Now you need to turn those PDFs into something an AI can search through. Raw PDF files are too large for an LLM to process directly, so you will split them into small, overlapping chunks, convert those chunks into numerical vectors (embeddings), and store everything in a database optimized for similarity search.

In this step, get ready to:

• Load your PDF notes and split them into searchable chunks.
• Configure a local embedding model and a free LLM.
• Store your embedded chunks in a persistent ChromaDB vector database.

Load and chunk your PDF documents

Your PDFs contain pages of dense text. To make that text searchable, you need to break it into smaller pieces called chunks.

Chunking matters because an LLM can only process limited context at a time. If chunks are too large, retrieval gets noisy. If they are too small, each chunk loses important surrounding context.

• In VS Code, create a new file called ingest.py in the root of your project folder.
• Add the following code to set up your imports, chunking configuration, and document loading:

import chromadb
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex, StorageContext, Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.openrouter import OpenRouter
from llama_index.vector_stores.chroma import ChromaVectorStore

# Configure chunking
Settings.chunk_size = 512
Settings.chunk_overlap = 50

# Load documents from the data folder
print("Loading documents...")
documents = SimpleDirectoryReader("./data").load_data()
print(f"Loaded {len(documents)} document pages.")

What does this code do?

This code sets up the foundation of your ingestion pipeline.

• The imports bring in everything you need from LlamaIndex and ChromaDB.
• Settings.chunk_size = 512 means each chunk is at most 512 tokens long.
• Settings.chunk_overlap = 50 adds 50 tokens of overlap between chunks so important sentences sitting at chunk boundaries are not lost.
• SimpleDirectoryReader("./data") automatically reads all PDFs from your data folder and converts them into LlamaIndex document objects.

Why 512 tokens with 50 overlap?

A chunk size of 512 tokens is a common starting point for study notes. It is large enough to capture a full paragraph or concept, but small enough to give precise retrieval results.

The 50-token overlap ensures that if a key sentence straddles two chunks, both chunks contain it. You will experiment with these values in a later step.

• Save ingest.py.

You will not run this file yet. The embedding model and LLM configuration come next.

Configure the embedding model and LLM

Your pipeline needs two AI models. The embedding model converts text chunks into numerical vectors for similarity search. The LLM generates natural language answers from the retrieved chunks.

• In ingest.py, find the line that says # Configure chunking.
• Add the following code above that line:

# Configure embedding model (runs locally, no API needed)
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

# Configure LLM via OpenRouter (free model)
Settings.llm = OpenRouter(
    api_key="your-openrouter-api-key-here",
    model="nvidia/nemotron-3-super:free",
    max_tokens=1024,
)

What does this code do?

This configures the two AI models your RAG pipeline depends on.

• HuggingFaceEmbedding with BAAI/bge-small-en-v1.5 runs entirely on your CPU. No API key or internet connection is needed for embeddings. The model is small (~130MB) but delivers strong retrieval quality for English text.
• The OpenRouter LLM connects to nvidia/nemotron-3-super:free, a free model on OpenRouter. Replace your-openrouter-api-key-here with the API key you saved in Step 1.
• max_tokens=1024 limits the response length so answers stay focused.

Why local embeddings instead of an API?

OpenRouter's free tier gives you 50 requests per day. Embedding a set of PDFs can require hundreds of API calls (one per chunk). Running embeddings locally with HuggingFace means your embedding step is unlimited and free, saving your API quota for actual questions.

• Replace your-openrouter-api-key-here with your actual OpenRouter API key from Step 1.
• Save ingest.py.

Seeing a ModuleNotFoundError?

If you see an error about a missing module when you eventually run this file, make sure your virtual environment is activated and that you installed all packages in Step 1. The llama-index-vector-stores-chroma package provides the ChromaVectorStore import. Install it by running:

pip install llama-index-vector-stores-chroma

help me fix a ModuleNotFoundError in my ingest.py file

Create the ChromaDB vector store and index your documents

With your models configured and documents loaded, you need a place to store the embedded chunks. ChromaDB is a vector database that saves embeddings to disk so you can query them later without re-processing your PDFs.

• In ingest.py, scroll to the bottom of the file (below the print(f"Loaded {len(documents)} document pages.") line).
• Add the following code to set up ChromaDB persistent storage:

# Set up ChromaDB persistent storage
chroma_client = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = chroma_client.get_or_create_collection("study_notes")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)

What does this code do?

• PersistentClient(path="./chroma_db") creates a ChromaDB database stored in a chroma_db folder inside your project. Data persists between runs.
• get_or_create_collection("study_notes") creates a named collection (like a table in a traditional database) to hold your embedded chunks.
• ChromaVectorStore wraps the collection so LlamaIndex can read and write to it.
• StorageContext tells LlamaIndex to use ChromaDB as the storage backend instead of in-memory storage.

• Now add the index-building code directly below the storage context:

# Build the index (embeds and stores all chunks)
print("Indexing documents... This may take a minute on first run.")
index = VectorStoreIndex.from_documents(
    documents,
    storage_context=storage_context,
)
print("Indexing complete! Your notes are ready to query.")

What does this code do?

VectorStoreIndex.from_documents() is where the magic happens. It takes your loaded documents, splits them into chunks (using the 512/50 settings you configured), passes each chunk through the BAAI/bge-small-en-v1.5 embedding model, and stores the resulting vectors in your ChromaDB collection.

The first run downloads the embedding model (~130MB) and processes all your PDFs, so it takes a minute or two. Subsequent runs reuse the downloaded model and are much faster.

• Save ingest.py.
• Run the ingestion script by executing this command in your terminal:

python ingest.py

You should see output similar to this:

Loading documents...
Loaded 12 document pages.
Indexing documents... This may take a minute on first run.
Indexing complete! Your notes are ready to query.

The exact number of document pages depends on how many PDFs you added and how long they are.

• Confirm the ChromaDB database was created by running:

ls chroma_db

You should see files and folders inside chroma_db, confirming your embedded chunks are persisted to disk.

Script hangs or takes a very long time?

The first run needs to download the BAAI/bge-small-en-v1.5 model (~130MB). On a slow connection this can take several minutes. Let it finish without interrupting.

If you see an authentication error from OpenRouter, double-check that you replaced your-openrouter-api-key-here with your actual API key in ingest.py.

help me debug my ingest.py script that is failing or hanging

✔️ Awesome, I've got everything!

Great. Double check you have saved ingest.py and that the chroma_db folder exists in your project.

ⓧ I'd like to double check the full code

Here is the complete ingest.py file. Make sure yours matches:

import chromadb
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex, StorageContext, Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.openrouter import OpenRouter
from llama_index.vector_stores.chroma import ChromaVectorStore

# Configure embedding model (runs locally, no API needed)
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

# Configure LLM via OpenRouter (free model)
Settings.llm = OpenRouter(
    api_key="your-openrouter-api-key-here",
    model="nvidia/nemotron-3-super:free",
    max_tokens=1024,
)

# Configure chunking
Settings.chunk_size = 512
Settings.chunk_overlap = 50

# Load documents from the data folder
print("Loading documents...")
documents = SimpleDirectoryReader("./data").load_data()
print(f"Loaded {len(documents)} document pages.")

# Set up ChromaDB persistent storage
chroma_client = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = chroma_client.get_or_create_collection("study_notes")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)

# Build the index (embeds and stores all chunks)
print("Indexing documents... This may take a minute on first run.")
index = VectorStoreIndex.from_documents(
    documents,
    storage_context=storage_context,
)
print("Indexing complete! Your notes are ready to query.")

Your notes are now embedded and stored in a persistent vector database. Next up, you will build a query engine that retrieves relevant chunks and generates answers from them.

Query Your Notes

Your notes are indexed and stored in ChromaDB. The hard work of chunking and embedding is done.

Now you need a way to actually ask questions and get answers from those embedded chunks. In this step, you will build a query script that loads the existing vector database, retrieves the most relevant chunks for any question, and sends them to the LLM to generate a grounded answer.

In this step, get ready to:

• Build a query engine that loads your existing ChromaDB index.
• Ask natural language questions and examine the retrieved source chunks.
• Create an interactive query loop for continuous study sessions.

Build the query engine

The ingestion script embedded your notes and stored them in ChromaDB. Your query script needs to connect to that same database and load the index without re-embedding anything.

• Create a new file called query.py in your project folder by running:

touch query.py

• Confirm the file exists by running:

ls

You should see query.py listed alongside ingest.py and your data folder.

• Open query.py in VS Code and add the imports and model configuration by pasting this code:

import chromadb
from llama_index.core import VectorStoreIndex, Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.openrouter import OpenRouter
from llama_index.vector_stores.chroma import ChromaVectorStore

# Configure embedding model (must match what was used during ingestion)
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

# Configure LLM via OpenRouter (free model)
Settings.llm = OpenRouter(
    api_key="your-openrouter-api-key-here",
    model="nvidia/nemotron-3-super:free",
    max_tokens=1024,
)

What does this code do?

This mirrors the configuration from ingest.py because querying requires the same embedding model that was used during ingestion. If you use a different model, the vector dimensions will not match and retrieval will fail.

The LLM is configured to generate answers from the retrieved chunks. Replace your-openrouter-api-key-here with the same API key you used in ingest.py.

• Below the configuration, add the code that connects to your existing ChromaDB and creates the query engine:

# Load the existing ChromaDB index
chroma_client = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = chroma_client.get_or_create_collection("study_notes")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)

# Build index from existing vector store (no re-embedding)
index = VectorStoreIndex.from_vector_store(vector_store)

# Create query engine
query_engine = index.as_query_engine(similarity_top_k=3)

What does this code do?

PersistentClient reconnects to the same ./chroma_db directory where your embedded chunks are stored. No documents are re-processed.

VectorStoreIndex.from_vector_store builds an index directly from the existing vectors. This is instant because the embeddings already exist in ChromaDB.

as_query_engine(similarity_top_k=3) creates a query engine that retrieves the 3 most semantically similar chunks for each question. The LLM then uses those chunks as context to generate its answer.

• Save query.py.

You will not see a change yet. The next section adds the interactive loop that lets you ask questions.

Seeing an import error?

Make sure your virtual environment is still activated. You should see its name in your terminal prompt. If not, activate it again before running any scripts.

help me fix import errors in my query.py file

Understand what happens during a query

Before adding the interactive loop, it helps to understand the three-stage process that runs every time you ask a question.

How does RAG retrieval work?

When you submit a question, the query engine performs three steps behind the scenes:

• Retrieval: Your question is embedded using the same BAAI/bge-small-en-v1.5 model. ChromaDB then finds the 3 chunks whose embeddings are closest to your question's embedding.
• Context assembly: Those 3 chunks are combined into a prompt alongside your original question.
• Generation: The LLM reads the retrieved chunks and generates an answer grounded in your actual notes, not its general training data.

This is the core of RAG. The LLM never sees your entire document. It only sees the most relevant pieces, which keeps answers focused and accurate.

Create an interactive query loop

A one-shot script would require restarting every time you want to ask another question. An interactive loop lets you have a continuous study session.

• In query.py, add the following interactive loop below the query engine line:

print("Study Notes Q&A Bot ready! Type 'quit' to exit.\n")

while True:
    question = input("Ask a question: ")
    if question.lower() in ("quit", "exit", "q"):
        print("Goodbye!")
        break

    response = query_engine.query(question)
    print(f"\nAnswer: {response}\n")

    # Show retrieved sources
    print("--- Sources ---")
    for i, node in enumerate(response.source_nodes, 1):
        metadata = node.node.metadata
        filename = metadata.get("file_name", "Unknown")
        page = metadata.get("page_label", "N/A")
        score = node.score
        print(f"  [{i}] {filename} (Page {page}) - Relevance: {score:.3f}")
    print()

What does this code do?

The while True loop keeps prompting for questions until you type quit, exit, or q.

response.source_nodes gives you the actual chunks that were retrieved from ChromaDB. Each node carries metadata including the original file_name and page_label, plus a relevance score showing how closely it matched your question.

Displaying sources lets you verify that the answer is actually grounded in your notes. Higher scores mean stronger semantic matches.

• Save query.py.
• Run your Q&A bot by executing:

python query.py

You should see Study Notes Q&A Bot ready! Type 'quit' to exit. printed in your terminal.

• Ask a question about your notes. Try a factual question like "What is..." or a comparison like "What is the difference between...".
• Look at the sources section below the answer. The filenames and page numbers tell you exactly where the information came from.
• Try 2-3 different types of questions (factual, comparison, summary) to see how retrieval quality varies.
• Type quit when you are done testing.

Getting an error about the API key?

Make sure you replaced your-openrouter-api-key-here with your actual OpenRouter API key in query.py. The key must be the same one you used in ingest.py.

help me fix my OpenRouter API key error in query.py

Answer seems unrelated to your notes?

Check the relevance scores in the sources section. If scores are very low (below 0.3), your question might not match the vocabulary in your notes. Try rephrasing using terms that appear in your PDF documents.

help me understand why my RAG bot is returning irrelevant answers

✔️ Awesome, I've got everything!

Great. Double check you have saved query.py and that it runs without errors.

ⓧ I'd like to double check the full code

Here is the complete query.py file:

import chromadb
from llama_index.core import VectorStoreIndex, Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.openrouter import OpenRouter
from llama_index.vector_stores.chroma import ChromaVectorStore

# Configure embedding model (must match what was used during ingestion)
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

# Configure LLM via OpenRouter (free model)
Settings.llm = OpenRouter(
    api_key="your-openrouter-api-key-here",
    model="nvidia/nemotron-3-super:free",
    max_tokens=1024,
)

# Load the existing ChromaDB index
chroma_client = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = chroma_client.get_or_create_collection("study_notes")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)

# Build index from existing vector store (no re-embedding)
index = VectorStoreIndex.from_vector_store(vector_store)

# Create query engine
query_engine = index.as_query_engine(similarity_top_k=3)

print("Study Notes Q&A Bot ready! Type 'quit' to exit.\n")

while True:
    question = input("Ask a question: ")
    if question.lower() in ("quit", "exit", "q"):
        print("Goodbye!")
        break

    response = query_engine.query(question)
    print(f"\nAnswer: {response}\n")

    # Show retrieved sources
    print("--- Sources ---")
    for i, node in enumerate(response.source_nodes, 1):
        metadata = node.node.metadata
        filename = metadata.get("file_name", "Unknown")
        page = metadata.get("page_label", "N/A")
        score = node.score
        print(f"  [{i}] {filename} (Page {page}) - Relevance: {score:.3f}")
    print()

Your study bot is working. You can now ask any question and get answers grounded in your actual notes. Next up, you will experiment with retrieval parameters to see how chunk size and the number of retrieved chunks affect answer quality.

Experiment with Retrieval Quality

Your Q&A bot works, but how well does it work? The quality of every RAG answer depends on two tunable parameters: how large each stored chunk is, and how many chunks the retriever feeds to the LLM.

In this step, you will experiment with both parameters and observe how they change your bot's answers. This hands-on tuning is how practitioners build intuition for real-world RAG systems.

In this step, get ready to:

• Re-index your documents with a smaller chunk size and compare answer quality.
• Adjust the number of retrieved chunks and observe the effect on response quality.

Compare different chunk sizes

Chunk size controls how much text goes into each vector. Your current setup uses 512-token chunks. Smaller chunks mean more precise retrieval, but each chunk carries less surrounding context.

• In ingest.py, find this line:

Settings.chunk_size = 512

• Change 512 to 256, so the line now looks like this:

Settings.chunk_size = 256

Before re-indexing, you need to clear the old chunks from ChromaDB. If you skip this, the old 512-token chunks remain alongside the new 256-token chunks, which ruins the comparison.

• Delete the existing database folder by running this command:

macOS

rm -rf ./chroma_db

Windows

rmdir /s /q chroma_db

• Re-index your documents with the new chunk size by running:

python ingest.py

You should see the same loading and indexing messages as before. Notice the total chunk count may be higher now, because smaller chunks produce more pieces from the same documents.

• Run your query script by running:

python query.py

• Ask the same question you used in the previous step.
• Compare the answer to what you got before with 512-token chunks. Pay attention to whether the answer is more focused or whether it misses broader context.
• Type quit to exit.

What is the chunk size trade-off?

Smaller chunks (like 256 tokens) retrieve more precisely. The retrieved text is tightly focused on the exact concept you asked about.

Larger chunks (like 512 tokens) carry more surrounding context. The LLM gets a fuller picture of the topic, which helps with questions that span multiple related ideas.

There is no single correct value. The best chunk size depends on your documents and the types of questions you ask.

Getting strange or mixed answers?

If your answers seem to combine unrelated topics, you may not have deleted the old database before re-indexing. Delete the chroma_db folder and run ingest.py again.

help me debug why my RAG answers seem to mix content from different chunk sizes

Adjust the number of retrieved chunks

The similarity_top_k parameter controls how many chunks the retriever sends to the LLM. More chunks mean more context, but also more noise if the extra chunks are only loosely relevant.

• In query.py, find this line:

query_engine = index.as_query_engine(similarity_top_k=3)

• Change similarity_top_k=3 to similarity_top_k=5, so the line now looks like this:

query_engine = index.as_query_engine(similarity_top_k=5)

• Save query.py and run it:

python query.py

• Ask the same question again.
• Notice how the answer may be longer or more detailed now, because the LLM has 5 chunks of context instead of 3.
• Check the Sources section. Are the 4th and 5th sources still relevant, or are they only loosely related?
• Type quit to exit.

Now try the opposite extreme.

• In query.py, change similarity_top_k=5 to similarity_top_k=1, so the line now looks like this:

query_engine = index.as_query_engine(similarity_top_k=1)

• Save query.py and run it:

python query.py

• Ask the same question one more time.
• With only 1 chunk of context, the LLM has very little to work with. The answer is likely shorter and may miss important details.
• Type quit to exit.

What is the top_k trade-off?

A higher similarity_top_k gives the LLM more context to draw from. This helps with broad or multi-faceted questions.

A lower similarity_top_k forces the LLM to answer from only the most relevant chunk. This reduces noise but risks missing information that lives in a neighboring chunk.

For study notes, similarity_top_k=3 is a good starting point. It balances precision with enough context for most questions.

Now set your parameters back to the balanced defaults.

• In query.py, change similarity_top_k=1 back to similarity_top_k=3.
• In ingest.py, change Settings.chunk_size = 256 back to Settings.chunk_size = 512.
• Save both files.

Why revert the chunk size in the code?

Your ingest.py file records the intended chunk size for future re-indexing. Setting it back to 512 keeps your code consistent with the recommended default. Your ChromaDB database still holds the 256-token chunks from your experiment. The next time you run ingest.py, it will overwrite them with 512-token chunks.

✔️ Awesome, I've got everything!

Great. Double check you have saved both files.

ⓧ I'd like to double check the full code

Here is ingest.py in full:

import chromadb
from llama_index.core import SimpleDirectoryReader, VectorStoreIndex, StorageContext, Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.openrouter import OpenRouter
from llama_index.vector_stores.chroma import ChromaVectorStore

# Configure embedding model (runs locally, no API needed)
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

# Configure LLM via OpenRouter (free model)
Settings.llm = OpenRouter(
    api_key="your-openrouter-api-key-here",
    model="nvidia/nemotron-3-super:free",
    max_tokens=1024,
)

# Configure chunking
Settings.chunk_size = 512
Settings.chunk_overlap = 50

# Load documents from the data folder
print("Loading documents...")
documents = SimpleDirectoryReader("./data").load_data()
print(f"Loaded {len(documents)} document pages.")

# Set up ChromaDB persistent storage
chroma_client = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = chroma_client.get_or_create_collection("study_notes")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
storage_context = StorageContext.from_defaults(vector_store=vector_store)

# Build the index (embeds and stores all chunks)
print("Indexing documents... This may take a minute on first run.")
index = VectorStoreIndex.from_documents(
    documents,
    storage_context=storage_context,
)
print("Indexing complete! Your notes are ready to query.")

Here is query.py in full:

import chromadb
from llama_index.core import VectorStoreIndex, Settings
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.llms.openrouter import OpenRouter
from llama_index.vector_stores.chroma import ChromaVectorStore

# Configure embedding model (must match what was used during ingestion)
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")

# Configure LLM via OpenRouter (free model)
Settings.llm = OpenRouter(
    api_key="your-openrouter-api-key-here",
    model="nvidia/nemotron-3-super:free",
    max_tokens=1024,
)

# Load the existing ChromaDB index
chroma_client = chromadb.PersistentClient(path="./chroma_db")
chroma_collection = chroma_client.get_or_create_collection("study_notes")
vector_store = ChromaVectorStore(chroma_collection=chroma_collection)

# Build index from existing vector store (no re-embedding)
index = VectorStoreIndex.from_vector_store(vector_store)

# Create query engine
query_engine = index.as_query_engine(similarity_top_k=3)

print("Study Notes Q&A Bot ready! Type 'quit' to exit.\n")

while True:
    question = input("Ask a question: ")
    if question.lower() in ("quit", "exit", "q"):
        print("Goodbye!")
        break

    response = query_engine.query(question)
    print(f"\nAnswer: {response}\n")

    # Show retrieved sources
    print("--- Sources ---")
    for i, node in enumerate(response.source_nodes, 1):
        metadata = node.node.metadata
        filename = metadata.get("file_name", "Unknown")
        page = metadata.get("page_label", "N/A")
        score = node.score
        print(f"  [{i}] {filename} (Page {page}) - Relevance: {score:.3f}")
    print()

You now have hands-on intuition for the two most important retrieval parameters in any RAG system. Next up, a secret mission to make your bot even more useful by adding source citations to every answer.

Secret mission

Add Source Citations

Your Q&A bot gives good answers, but how do you know where the information came from? In this mission, you'll add formatted source citations so every answer traces back to a specific PDF and page number.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources for future study sessions, pause them to come back later, or delete them entirely.

Resources you used:

• Project folder on your Desktop (contains ingest.py, query.py, data/, chroma_db/, and your virtual environment).
• HuggingFace model cache at ~/.cache/huggingface/hub (macOS/Linux) or C:\Users\username\.cache\huggingface\hub (Windows).
• OpenRouter API key (managed at https://openrouter.ai/settings/keys).

Keep everything running

No action needed. Choose this if you plan to add more PDFs or keep querying your notes.

• Your project folder stays on your Desktop, ready for future sessions.
• The HuggingFace embedding model stays cached so subsequent runs start faster.
• Your OpenRouter account remains active on the free tier with no charges.

Pause - I'll come back to this later

Stop any running Python processes to free up memory. All files stay on disk for when you return.

• In your terminal, press Ctrl+C to stop the query loop if it is still running.
• Deactivate your virtual environment by running:

deactivate

• Close VS Code. Your files and ChromaDB data remain on disk.

Delete - I don't want to use this again

Remove all project files, cached models, and revoke your API key. This cannot be undone.

• Deactivate your virtual environment if it is still active:

deactivate

• Delete the project folder (macOS/Linux):

cd ~/Desktop
rm -rf study-notes-rag

• Delete the project folder (Windows):

cd ~\Desktop
rmdir /s /q study-notes-rag

• Remove the cached HuggingFace embedding model (macOS/Linux):

rm -rf ~/.cache/huggingface/hub

• Remove the cached HuggingFace embedding model (Windows):

rmdir /s /q C:\Users\%USERNAME%\.cache\huggingface\hub

• Revoke your OpenRouter API key by going to https://openrouter.ai/settings/keys.
• Find the key you created for this project and click the delete button next to it.

Nice Work!

Nice Work!

You just built a fully functional study assistant powered by Retrieval-Augmented Generation. Your bot retrieves precise information from your own university notes and generates grounded answers using a free LLM.

You've learned how to:

• Build a complete RAG pipeline from scratch, covering document loading, chunking, embedding, vector storage, retrieval, and generation.
• Index your own PDF notes into a persistent ChromaDB vector database and query them with natural language questions via OpenRouter.
• Experiment with chunking sizes and retrieval parameters to understand their impact on answer quality.
• Secret Mission: Added source citations so every answer traces back to a specific PDF and page.

Ready to quiz yourself?