RAG-Powered Study Notes Q&A

Build a local AI assistant that answers questions from your own study notes.

Introduction

30 Second Summary

Every time you ask ChatGPT about an uploaded PDF or use Notion AI to search your workspace, the same architecture is running behind the scenes. The AI retrieves relevant pieces of your documents, then generates an answer grounded in that specific text.

In this project, you will build that exact system from scratch: a command-line study assistant that reads your text notes, stores them in a local vector database, and answers your questions using only the information in those notes.

What You'll Build

You type a question into your terminal, your assistant finds the exact paragraphs from your notes, feeds them to Gemini, and prints a grounded answer citing which file the information came from.

By the end of this project, you'll have:

  • A working RAG pipeline where you ask natural-language questions and get answers sourced entirely from your own documents.
  • Built-in source citations that show you exactly which file and passage each answer came from, plus honest "I don't know" responses when the answer is not in your notes.
  • A reusable local system where you can drop new text files into a folder and immediately query them with no cloud setup or re-training.
  • Secret Mission: Add conversation memory so the system handles follow-up questions like "tell me more about that" without you repeating yourself.

Are there any prerequisites?

You should be comfortable with Python fundamentals: variables, functions, loops, and file I/O. No prior AI or machine learning experience is needed.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

Set Up Your Environment

Before you can ask an AI questions about your personal study notes, you need two things in place. A connection to the Gemini LLM, and a local vector database to store your documents.

In this step, you'll create your project environment, install the two core Python packages, and verify that Gemini responds to a test call.

In this step, get ready to:
  • Create your project folder and install google-genai and chromadb.
  • Get a free Gemini API key from Google AI Studio.
  • Verify your setup with a test script that talks to Gemini.
Create your project folder and install packages

Your project needs its own folder and a Python virtual environment to keep packages isolated from other projects on your machine.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir rag-study-notes
cd rag-study-notes
  • Confirm the folder exists by running:
ls

You should see the rag-study-notes folder listed in the output (or you're already inside it if it shows empty).

Now create a virtual environment and activate it. The steps differ slightly between macOS and Windows.

macOS

  • Create and activate your virtual environment by running these commands:
python3 -m venv venv
source venv/bin/activate

You should see (venv) appear at the beginning of your terminal prompt.

Windows

  • Create and activate your virtual environment by running these commands:
python -m venv venv
venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt.

What is a virtual environment?

A virtual environment is an isolated Python workspace. Packages you install here stay separate from your system Python, so different projects can use different package versions without conflicts.

  • Install the two packages your project needs by running:
pip install google-genai chromadb

You should see download progress bars and a final message confirming both packages were successfully installed.

Installation failed or timed out?

  • Make sure you see (venv) in your terminal prompt. If not, re-run the activation command from the platform tab above.
  • Check your internet connection. Both packages download dependencies from the internet.
  • If you see a permissions error, confirm you are inside the virtual environment (not installing globally).

Help me troubleshoot my pip install failing for google-genai and chromadb.

Get your free Gemini API key

The Gemini API has a free tier that requires no credit card. You just need a Google account to generate a key.

  • Open your browser and go to aistudio.google.com.
  • Sign in with your Google account.
  • If this is your first time, accept the Terms of Service when prompted. Google AI Studio will automatically create a default project for you.
  • Click Get API key in the left panel or from the landing page.
  • Click Create API key to generate a new key.
  • Copy the key that appears. You will only see it once, so keep this tab open.

Why is this free?

Google offers a free tier for the Gemini API where input and output tokens are free of charge. No credit card or billing account is required. New accounts begin on the Free Tier automatically.

Now set your API key as an environment variable so your Python scripts can access it.

macOS

  • Set the environment variable by running this command (replace your-api-key-here with the key you just copied):
export GOOGLE_API_KEY=your-api-key-here

Windows

  • Set the environment variable by running this command (replace your-api-key-here with the key you just copied):
set GOOGLE_API_KEY=your-api-key-here

Can't create an API key?

  • Make sure you are signed in with a personal Google account, not a managed workspace account that may have restrictions.
  • If you see a permissions error, try clearing your browser cache and signing in again.
  • Confirm you are in a supported region for Google AI Studio.

Help me get my Gemini API key from Google AI Studio.

Verify your setup with a test script

Time to confirm that your packages, API key, and Gemini connection all work together.

  • Open Cursor by pressing Cmd+Space (macOS) or the Windows key (Windows), typing Cursor, and pressing Enter.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Select the rag-study-notes folder on your Desktop, then click Open.
  • Create a new file called test_gemini.py by clicking the new file icon in the Explorer sidebar.
  • Add the following code to test_gemini.py:
import os
from google import genai

# Connect to Gemini using the API key from your environment variable
client = genai.Client(api_key=os.environ["GOOGLE_API_KEY"])

# Send a test message to confirm the connection works
response = client.interactions.create(
    model="gemini-3.6-flash",
    input="Say hello and confirm you are working!",
)
print(response.output_text)

What does this code do?

  • `from google import genai` imports the official Google Gen AI SDK.
  • `genai.Client(api_key=...)` creates a client that authenticates with your API key stored in the GOOGLE_API_KEY environment variable.
  • `client.interactions.create()` sends a prompt to the gemini-3.6-flash model and returns the response.
  • `response.output_text` contains the text that Gemini generated.
  • Save test_gemini.py by pressing Cmd+S (macOS) or Ctrl+S (Windows).

Before you run this script, what do you think will happen? Will Gemini respond, or will something go wrong?

  • Switch back to your terminal (make sure you still see (venv) in the prompt) and run:
python test_gemini.py

You should see Gemini respond with a friendly greeting in your terminal, confirming the connection works.

Seeing a KeyError or authentication error?

  • A KeyError: 'GOOGLE_API_KEY' means the environment variable is not set. Re-run the export (macOS) or set (Windows) command from the previous substep in the same terminal session.
  • If you see an authentication or invalid key error, double-check that you copied the full key with no extra spaces.
  • Make sure you are running the command from inside the rag-study-notes folder with the virtual environment activated.

Help me fix my test_gemini.py script that isn't connecting to the Gemini API.

✔️ Awesome, I've got everything!

Great. Double check you've saved test_gemini.py and confirmed a Gemini greeting in your terminal.

ⓧ I'd like to double check the full code

Here is the complete test_gemini.py for comparison:

import os
from google import genai

client = genai.Client(api_key=os.environ["GOOGLE_API_KEY"])

response = client.interactions.create(
    model="gemini-3.6-flash",
    input="Say hello and confirm you are working!",
)
print(response.output_text)

Your environment is ready and Gemini is listening. Next up, you'll build an interactive chat loop so you can have a real conversation with the LLM.

Talk to Gemini

Your connection to Gemini is live. In the last step, you proved that the SDK can reach the model and get a response back.

Now you'll turn that one-shot test into something more useful: an interactive chat loop. This gives you a real-time feedback loop with the LLM that you'll use for the rest of the project.

In this step, get ready to:
  • Create ask_gemini.py with an interactive question-and-answer loop.
  • Test Gemini with a general knowledge question and confirm it responds coherently.
Create the interactive chat script

Your test_gemini.py script sends one hardcoded message and exits. A real development workflow needs something you can keep open and experiment with freely.

  • Create a new file called ask_gemini.py in your project folder (the same folder where test_gemini.py lives).
  • Paste the following code into ask_gemini.py:
import os
from google import genai

# Initialize the Gemini client with your API key
client = genai.Client(api_key=os.environ["GOOGLE_API_KEY"])

# Welcome message with exit instructions
print("Ask Gemini anything! Type 'quit' to exit.\n")

while True:
    # Read the user's question
    question = input("You: ")
    # Exit conditions - quit, exit, or q all work
    if question.lower() in ["quit", "exit", "q"]:
        print("Goodbye!")
        break
    # Skip empty inputs
    if not question.strip():
        continue

    # Send the question to Gemini and print the response
    response = client.interactions.create(
        model="gemini-3.6-flash",
        input=question,
    )
    print(f"\nGemini: {response.output_text}\n")

What does this code do?

  • The while True loop keeps the program running until the user types a quit command. This turns the script into an ongoing conversation instead of a one-and-done call.
  • input("You: ") displays a prompt and waits for you to type something. The script pauses here until you press Enter.
  • client.interactions.create() sends your question to the gemini-3.6-flash model and returns the response. The .output_text property gives you just the text of Gemini's reply.
  • The if not question.strip() check skips blank lines so you don't waste an API call on empty input.
  • Save ask_gemini.py.

✔️ Awesome, I've got everything!

Great. Double check you've saved the file, then move on to the next substep to run it.

ⓧ I'd like to double check the full code

Here is the complete ask_gemini.py file:

import os
from google import genai

# Initialize the Gemini client with your API key
client = genai.Client(api_key=os.environ["GOOGLE_API_KEY"])

# Welcome message with exit instructions
print("Ask Gemini anything! Type 'quit' to exit.\n")

while True:
    # Read the user's question
    question = input("You: ")
    # Exit conditions - quit, exit, or q all work
    if question.lower() in ["quit", "exit", "q"]:
        print("Goodbye!")
        break
    # Skip empty inputs
    if not question.strip():
        continue

    # Send the question to Gemini and print the response
    response = client.interactions.create(
        model="gemini-3.6-flash",
        input=question,
    )
    print(f"\nGemini: {response.output_text}\n")
Run and test with a general knowledge question

Before you run this, what do you expect to see in your terminal? Think about how the loop will behave when you type a question and press Enter.

  • Run your new script by entering this command in your terminal:
python ask_gemini.py

You should see the welcome message appear, then the You: prompt waiting for your input.

  • Type a general knowledge question like What is machine learning? and press Enter.

Gemini responds with a coherent answer, then the You: prompt reappears so you can ask another question.

  • Type quit and press Enter to exit the loop cleanly.

You should see Goodbye! printed, and your terminal returns to the normal command prompt.

Not seeing a response from Gemini?

  • Make sure your virtual environment is still active. You should see the environment name in parentheses at the start of your terminal prompt.
  • Confirm your GOOGLE_API_KEY environment variable is still set. If you opened a new terminal window since Step 1, you need to set it again.
  • Check that the file is saved as ask_gemini.py (not .txt or another extension).

Ask for help: My ask_gemini.py script is not getting a response from Gemini. I have google-genai installed and my API key worked in test_gemini.py but the interactive loop is failing.

You now have a live conversation with an LLM running in your terminal. Next up, you'll discover the limitation that makes this simple chat loop fall short for real study work.

Ask Gemini About Your Notes (It Fails)

In the last step, you proved that Gemini can answer general knowledge questions. It handled "What is machine learning?" with ease.

But what about YOUR specific study notes sitting on your computer? Gemini has never seen those files. This step exposes the core limitation that RAG solves: LLMs cannot access your private data unless you explicitly give it to them.

In this step, get ready to:
  • Create sample study notes with specific facts about machine learning and neural networks.
  • Test whether Gemini can answer questions about your notes.
Create your study notes

You need some private documents that contain specific facts Gemini has never seen. These will be the notes your RAG system searches later.

  • Create a notes/ directory inside your project folder by running this command:
mkdir notes
  • Confirm the directory was created by running:
ls

You should see notes listed alongside your existing Python files.

  • Create a new file at notes/machine_learning_basics.txt and paste this content:
Introduction to Machine Learning

Machine learning is a subset of artificial intelligence that enables systems to learn and improve from experience without being explicitly programmed. There are three main types of machine learning: supervised learning, unsupervised learning, and reinforcement learning.

Supervised Learning:
In supervised learning, the algorithm learns from labeled training data. Each training example has an input and a desired output. The goal is to learn a mapping function from inputs to outputs. Common algorithms include linear regression, decision trees, and support vector machines. Supervised learning is used for tasks like email spam detection, image classification, and price prediction.

Unsupervised Learning:
In unsupervised learning, the algorithm works with unlabeled data and tries to find hidden patterns or structures. The algorithm is not told what the correct output should be. Common techniques include clustering (like K-means), dimensionality reduction (like PCA), and association rules. Unsupervised learning is used for customer segmentation, anomaly detection, and recommendation systems.

Reinforcement Learning:
In reinforcement learning, an agent learns by interacting with an environment. The agent takes actions and receives rewards or penalties based on those actions. The goal is to learn a policy that maximizes cumulative reward over time. Key concepts include the agent, environment, state, action, and reward. Reinforcement learning is used in game playing (like AlphaGo), robotics, and autonomous vehicles.
  • Create a second file at notes/neural_networks.txt and paste this content:
Neural Networks and Deep Learning

A neural network is a computing system inspired by biological neural networks in the human brain. It consists of layers of interconnected nodes (neurons) that process information.

Architecture:
A basic neural network has three types of layers: an input layer that receives data, one or more hidden layers that process the data, and an output layer that produces the final result. Each connection between neurons has a weight that gets adjusted during training. Deep learning refers to neural networks with many hidden layers (deep neural networks).

How Training Works:
Neural networks learn through a process called backpropagation. During forward propagation, input data flows through the network layer by layer, producing an output. The output is compared to the expected result using a loss function, which measures how wrong the prediction is. During backpropagation, the error is propagated backwards through the network, and the weights are adjusted using gradient descent to minimize the loss. This process repeats for many iterations (epochs) until the network achieves acceptable accuracy.

Activation Functions:
Activation functions introduce non-linearity into the network, allowing it to learn complex patterns. Common activation functions include ReLU (Rectified Linear Unit) which outputs zero for negative inputs and the input value for positive inputs, Sigmoid which squashes values between 0 and 1, and Tanh which squashes values between -1 and 1.

Common Applications:
Neural networks power many modern AI applications including image recognition (Convolutional Neural Networks), natural language processing (Transformers and LLMs), speech recognition, and generative AI models like those used in text and image generation.
  • Save both files, then verify they exist by running:
ls notes/

You should see machine_learning_basics.txt and neural_networks.txt listed.

Why these specific notes?

These notes contain very specific claims, like the exact three types of machine learning and how backpropagation works step by step. Gemini knows about these topics generally, but it has never read YOUR exact wording.

That specificity is what makes the next test meaningful. If Gemini gives a generic answer instead of quoting your notes, you know it did not read your files.

Don't see both files?

Make sure you created the files inside the notes/ directory, not in the project root. The path should be notes/machine_learning_basics.txt, not just machine_learning_basics.txt.

Help me figure out why my notes files aren't showing up in the notes directory.

Ask Gemini about your notes

Your notes are sitting right there on your computer. Time to find out if Gemini can use them to answer a question.

  • Run your interactive chat script by running:
python ask_gemini.py

Before you ask your question, think about this: do you expect Gemini to answer correctly about the specific content of your notes? Or will it give a generic response?

  • Type this question at the prompt:
According to my notes, what are the three types of machine learning?

Read Gemini's response carefully. It probably gives a correct-sounding answer about supervised, unsupervised, and reinforcement learning. But look closer. Is it quoting YOUR notes, or is it pulling from its general training data?

  • Now try a question that ONLY your notes can answer correctly. Type:
According to my notes, what specific algorithms are listed under supervised learning?

Your notes specifically list linear regression, decision trees, and support vector machines. Gemini either hallucinates a different list, gives a much broader answer, or admits it cannot access your files. It has no way to read machine_learning_basics.txt from your computer.

  • Type quit to exit the chat.

Why does this happen?

Gemini is a large language model trained on public internet data. It has broad knowledge, but it has zero access to files on your computer. When you ask about "your notes," it has no mechanism to open those files and read them.

This is the exact gap that RAG closes. Instead of hoping the LLM already knows your data, you retrieve the relevant pieces and feed them directly into the prompt. That is what you will build in the next two steps.

You have now seen the fundamental limitation firsthand. Gemini is powerful but blind to your private data. In the next step, you will store your notes in a vector database so they become searchable by meaning, not just keywords.

Store Your Notes in a Vector Database

You proved in the last step that Gemini cannot answer questions about your private study notes. It simply has no access to your files.

To fix this, you need to store your notes in a way that supports semantic search. ChromaDB converts your text into embeddings (numerical representations of meaning) and lets you find the most relevant passages for any question.

In this step, get ready to:
  • Load your study notes and split them into searchable chunks.
  • Store the chunks in a local ChromaDB vector database.
  • Test semantic retrieval by querying for relevant passages.
Load and chunk your documents

Your notes are just text files right now. To search them by meaning, you need to break them into smaller pieces called chunks. Smaller chunks give the search engine more precise results than whole documents.

  • Create a new file called rag.py in your project folder.
  • Add the following code to set up your clients and define the document loading and chunking functions:
import os
import chromadb
from google import genai

client = genai.Client(api_key=os.environ["GOOGLE_API_KEY"])
chroma_client = chromadb.PersistentClient(path="./chroma_db")


def load_documents(directory):
    """Load all .txt files from a directory."""
    documents = []
    for filename in os.listdir(directory):
        if filename.endswith(".txt"):
            filepath = os.path.join(directory, filename)
            with open(filepath, "r", encoding="utf-8") as f:
                content = f.read()
            documents.append({"content": content, "source": filename})
    return documents


def chunk_text(text, chunk_size=500, overlap=50):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks

What does this code do?

This sets up two clients and defines two helper functions:

  • PersistentClient(path="./chroma_db") creates a local ChromaDB database that saves to disk, so you do not need to re-index every time you run the script.
  • load_documents() reads every .txt file from a folder and stores each file's content along with its filename.
  • chunk_text() splits text into 500-character pieces with a 50-character overlap. The overlap ensures sentences at chunk boundaries are not cut off and lost.
  • Save rag.py.
  • Verify that document loading works by running this command:
python -c "from rag import load_documents; print(len(load_documents('./notes')), 'documents loaded')"

You should see 2 documents loaded printed in your terminal.

Seeing a ModuleNotFoundError?

  • Make sure your virtual environment is still activated. You should see its name in parentheses at the start of your terminal prompt.
  • Confirm you are running the command from your project folder (the one containing rag.py and the notes/ directory).

Help me debug a ModuleNotFoundError when importing from rag.py

Store chunks in ChromaDB

Now you need a function that takes your loaded documents, chunks them, and stores everything in ChromaDB. ChromaDB handles embedding automatically using its built-in model, so you do not need to make any extra API calls.

Why ChromaDB over other vector databases?

ChromaDB runs entirely locally with zero cloud accounts required. Alternatives like Pinecone and Weaviate need cloud setup or Docker. For learning RAG concepts, ChromaDB removes all infrastructure friction.

Its built-in embedding model (all-MiniLM-L6-v2) runs locally on your machine, so embeddings are free and instant. You still use Gemini for the generation step.

  • Add the following function below chunk_text() in rag.py:
def build_knowledge_base(documents):
    """Chunk documents and store in ChromaDB."""
    collection = chroma_client.get_or_create_collection(name="study_notes")

    all_chunks = []
    all_ids = []
    all_metadatas = []

    for doc in documents:
        chunks = chunk_text(doc["content"])
        for i, chunk in enumerate(chunks):
            all_chunks.append(chunk)
            all_ids.append(f"{doc['source']}_chunk_{i}")
            all_metadatas.append({"source": doc["source"], "chunk_index": i})

    if all_chunks:
        collection.add(
            documents=all_chunks,
            ids=all_ids,
            metadatas=all_metadatas,
        )

    print(f"Stored {len(all_chunks)} chunks from {len(documents)} documents.")
    return collection

What does this code do?

  • get_or_create_collection(name="study_notes") creates a named collection in ChromaDB, or reuses it if it already exists from a previous run.
  • The loop chunks each document and builds parallel lists of text, IDs, and metadata (source filename). Each chunk gets a unique ID like machine_learning_basics.txt_chunk_0.
  • collection.add() stores the chunks. ChromaDB automatically converts each chunk into a vector embedding behind the scenes.
  • Save rag.py.
  • Verify that your knowledge base builds correctly by running:
python -c "from rag import load_documents, build_knowledge_base; docs = load_documents('./notes'); build_knowledge_base(docs)"

You should see a message like Stored 8 chunks from 2 documents. in your terminal. The exact chunk count depends on your note file lengths.

Seeing a DuplicateIDError?

If you ran the command more than once, ChromaDB may reject duplicate chunk IDs. Delete the chroma_db folder in your project directory and run the command again:

rm -rf chroma_db

Help me fix a DuplicateIDError in ChromaDB

Test semantic retrieval

Your notes are now stored as vectors. The final piece is a retrieval function that finds the most relevant chunks for any question.

  • Add the following code below build_knowledge_base() in rag.py:
def retrieve_context(collection, query, n_results=3):
    """Retrieve relevant chunks for a query."""
    results = collection.query(
        query_texts=[query],
        n_results=n_results,
    )
    return results


if __name__ == "__main__":
    documents = load_documents("./notes")
    collection = build_knowledge_base(documents)
    test_query = "What is backpropagation?"
    results = retrieve_context(collection, test_query)
    print(f"\nTop chunks for: '{test_query}'")
    for i, doc in enumerate(results["documents"][0]):
        source = results["metadatas"][0][i]["source"]
        print(f"\n[{i+1}] Source: {source}\n{doc[:200]}...")

What does this code do?

  • retrieve_context() takes a question, converts it into an embedding, and finds the 3 closest chunks in the vector database. This is semantic search in action.
  • The if __name__ == "__main__" block ties everything together: load documents, store them, then run a test query for "What is backpropagation?" to prove retrieval works.
  • Each result prints its source filename and first 200 characters, so you can verify ChromaDB found the right passage.
  • Save rag.py.
  • Before you run this, what do you think the top result will be? Which of your two note files contains information about backpropagation?
  • Delete the chroma_db folder first (if it exists from the earlier test) so the script starts fresh:
rm -rf chroma_db
  • Run the full script:
python rag.py

You should see two outputs. First, a line confirming storage like Stored 8 chunks from 2 documents. followed by the top 3 chunks for your test query.

The top result should come from neural_networks.txt because that file contains the backpropagation section. ChromaDB found it by meaning, not by keyword matching.

Seeing no output or an empty result?

  • Confirm your notes/ folder contains both .txt files. Run ls notes/ to check.
  • Make sure you deleted the chroma_db folder before running the script. A stale database from a partial earlier run can cause issues.
  • Check that your GOOGLE_API_KEY environment variable is still set. Run echo $GOOGLE_API_KEY to confirm.

Help me debug empty results from ChromaDB retrieval

✔️ Awesome, I've got everything!

Great. Double check you have saved rag.py and that your terminal shows the stored chunks and retrieval results.

ⓧ I'd like to double check the full code

Your rag.py file is over 30 lines, so scroll up and confirm these pieces are present in this order:

  • Imports and client setup (genai.Client and chromadb.PersistentClient).
  • load_documents() function that reads .txt files from a directory.
  • chunk_text() function with chunk_size=500 and overlap=50.
  • build_knowledge_base() function that stores chunks in a collection named study_notes.
  • retrieve_context() function that queries the collection.
  • if __name__ == "__main__" block that loads, stores, and runs a test query for "What is backpropagation?"

Your notes are now stored as searchable vectors. Next up, you will wire this retrieval into Gemini so it can answer questions grounded in your actual study notes.

Build the Complete RAG Pipeline

You've confirmed that ChromaDB can find the most relevant chunks from your notes using semantic search. You also have a working connection to Gemini from Step 2. The missing piece is wiring these two halves together.

In this step, you'll build the function that retrieves relevant context, packages it into a prompt that keeps Gemini grounded in your data, and runs a full interactive Q&A loop. This final connection is the RAG pattern: Retrieve, Augment the prompt, Generate.

In this step, get ready to:
  • Add an ask_question() function that retrieves and formats context for the LLM.
  • Build a grounding prompt that prevents hallucination and requires source citations.
  • Wire everything into an interactive Q&A loop and test with in-scope and out-of-scope questions.
Add the ask_question() function

This function is the bridge between retrieval and generation. It takes a question, pulls relevant chunks from ChromaDB, labels each chunk with its source file, and sends everything to Gemini inside a carefully crafted prompt.

  • In rag.py, find the retrieve_context() function.
  • Add the first part of the ask_question() function directly below it, which handles retrieving and formatting context:
def ask_question(collection, question):
    """Retrieve context and generate an answer."""
    results = retrieve_context(collection, question)

    context_parts = []
    sources = set()
    for i, doc in enumerate(results["documents"][0]):
        source = results["metadatas"][0][i]["source"]
        context_parts.append(f"[Source: {source}]\n{doc}")
        sources.add(source)

    context = "\n\n".join(context_parts)
  • Continue adding the rest of the function directly below the context = line you just wrote. This builds the grounding prompt and calls Gemini:
    prompt = f"""You are a helpful study assistant. Answer the question based ONLY on the provided context from the user's study notes.

If the context does not contain enough information to answer the question, say: "I don't have enough information in your notes to answer this question."

Always cite which source file the information came from.

Context from study notes:
{context}

Question: {question}

Answer:"""

    response = client.interactions.create(
        model="gemini-3.6-flash",
        input=prompt,
    )

    return response.output_text

What does this code do?

The ask_question() function does three things in sequence:

  • It calls retrieve_context() to get the top 3 most relevant chunks for the question. Each chunk is labeled with its source filename using the [Source: filename] prefix so the LLM knows where each piece of information came from.
  • The grounding prompt gives Gemini three strict instructions: answer ONLY from the provided context, always cite the source file, and say "I don't have enough information" when the context is insufficient. This prevents hallucination by constraining the model to your data.
  • The formatted prompt is sent to gemini-3.6-flash via client.interactions.create(), and the response text is returned.
  • Save rag.py.
Wire up the interactive Q&A loop

The current if __name__ block at the bottom of your file just runs a test query and prints raw chunks. You'll replace it with a main() function that loads documents, builds the knowledge base, and starts an interactive loop.

  • In rag.py, scroll to the bottom and find this block:
if __name__ == "__main__":
    documents = load_documents("./notes")
    collection = build_knowledge_base(documents)
    test_query = "What is backpropagation?"
    results = retrieve_context(collection, test_query)
    print(f"\nTop chunks for: '{test_query}'")
    for i, doc in enumerate(results["documents"][0]):
        source = results["metadatas"][0][i]["source"]
        print(f"\n[{i+1}] Source: {source}\n{doc[:200]}...")
  • Delete the entire if __name__ block (all lines from if __name__ to the end of the file).
  • Replace it with the following main() function and updated entry point:
def main():
    print("Loading study notes...")
    documents = load_documents("./notes")

    if not documents:
        print("No .txt files found in ./notes directory!")
        print("Create a 'notes' folder and add .txt files to it.")
        return

    print("Building knowledge base...")
    collection = build_knowledge_base(documents)

    print("\nStudy Notes Q&A Ready! Ask questions about your notes.")
    print("Type 'quit' to exit.\n")

    while True:
        question = input("Your question: ")
        if question.lower() in ["quit", "exit", "q"]:
            print("Goodbye!")
            break
        if not question.strip():
            continue

        answer = ask_question(collection, question)
        print(f"\nAnswer: {answer}\n")


if __name__ == "__main__":
    main()

What does main() do?

The main() function orchestrates the full RAG pipeline in order:

  • It loads all .txt files from the notes/ directory and exits early with a helpful message if none are found.
  • It chunks and stores the documents in ChromaDB (the knowledge base).
  • It starts an interactive loop where the learner types questions and receives answers grounded in their notes.
  • Save rag.py.
Test the complete RAG pipeline

Your previous test run in Step 4 stored data in the ./chroma_db folder. Since build_knowledge_base() uses the same chunk IDs, you need to clear the old data before running the full pipeline.

  • Delete the old ChromaDB data by running this command:
rm -rf ./chroma_db

Before you run the pipeline, think about this: if you ask Gemini "What is backpropagation?" now, do you expect a different result compared to Step 3 when it failed? What changed?

  • Run the complete RAG pipeline:
python rag.py

You should see the loading messages, then the interactive prompt.

  • Type this question and press Enter:
What is backpropagation?

This time, Gemini answers using your actual notes. The response should mention forward propagation, loss functions, gradient descent, and cite neural_networks.txt as the source.

  • Now test the hallucination guardrail by typing an out-of-scope question:
What is the capital of France?

Gemini should respond with something like "I don't have enough information in your notes to answer this question." This proves your grounding prompt is working.

  • Type quit to exit the loop.

Not seeing the expected results?

  • If you see an error about duplicate IDs or existing records, make sure you deleted the ./chroma_db folder before running. Run rm -rf ./chroma_db and try again.
  • If Gemini gives a generic answer without citing your notes, check that your prompt variable includes the {context} placeholder inside the f-string. Without it, Gemini never sees your chunks.
  • If you see "No .txt files found", confirm your notes/ directory is in the same folder as rag.py and contains your two .txt files.

Help me debug my RAG pipeline

✔️ Awesome, I've got everything!

Great. Double check you've saved rag.py and confirmed both in-scope and out-of-scope questions work correctly.

ⓧ I'd like to double check the full code

Your rag.py file contains six sections in this order. Scroll through your file and confirm each one is present:

  • Imports and client setup (os, chromadb, genai imports, both clients initialized).
  • load_documents() function.
  • chunk_text() function.
  • build_knowledge_base() function.
  • retrieve_context() function.
  • ask_question() function (retrieval, formatting, grounding prompt, Gemini call).
  • main() function followed by if __name__ == "__main__": main().

Your RAG pipeline is complete. You've built the exact architecture behind ChatGPT file uploads, Notion AI, and enterprise knowledge bases: retrieve relevant context, augment the prompt, and generate a grounded answer. Next up, you can extend this system with conversation memory in the Secret Mission.

Secret mission

Add Conversation Memory

Your RAG system answers one question at a time, but real conversations have context. If you ask "What is backpropagation?" and then follow up with "Tell me more about that", your system has no idea what "that" refers to. Add conversation memory so follow-up questions work naturally.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally with a free-tier API, so there are no ongoing costs.

Resources you used:

  • Project folder on your Desktop (rag-study-notes or your chosen folder name) containing all Python scripts, study notes, and the ChromaDB database.
  • Python virtual environment inside your project folder.
  • ChromaDB persistent database stored in the chroma_db/ directory.
  • Gemini API key created in Google AI Studio.
  • GOOGLE_API_KEY environment variable set in your terminal.

Keep everything running

No action needed. Choose this if you plan to keep adding study notes and querying them.

  • Your project folder stays on your Desktop with all scripts and notes intact.
  • The ChromaDB database persists in chroma_db/ so you can re-run rag.py and query your notes at any time.
  • Your Gemini API key remains active in Google AI Studio. The free tier has no charges.
  • The GOOGLE_API_KEY environment variable only persists for the current terminal session. You will need to re-set it if you open a new terminal.

Pause - I'll come back to this later

Shut down running processes to free up memory. Your files stay on disk for when you return.

  • Close any terminal windows running your Python scripts.
  • Deactivate your virtual environment by running:
deactivate
  • Your project folder, notes, and ChromaDB data remain on disk. When you come back, reactivate the virtual environment, re-set your GOOGLE_API_KEY environment variable, and run python rag.py to pick up where you left off.

Delete - I don't want to use this again

Remove all project resources and start fresh.

  • Deactivate your virtual environment if it is still active:
deactivate
  • Delete the entire project folder from your Desktop. On macOS/Linux, run:
cd ~/Desktop
rm -rf rag-study-notes
  • On Windows, run:
cd ~\Desktop
rmdir /s /q rag-study-notes
  • This removes all Python scripts, the virtual environment, your study notes, and the ChromaDB database in one step.
  • Unset the environment variable in your current terminal session. On macOS/Linux:
unset GOOGLE_API_KEY
  • On Windows:
set GOOGLE_API_KEY=
  • Delete your Gemini API key from Google AI Studio. Go to aistudio.google.com, click Get API key in the left sidebar, find your key in the list, and click the trash icon to delete it.

Nice Work!

Nice Work!

You just built a production-grade RAG system from scratch. You can now ask natural language questions about your own documents and get grounded, cited answers powered by Gemini.

You've learned how to:

  • Build a complete RAG pipeline covering document loading, text chunking, vector storage, semantic retrieval, and LLM-grounded generation.
  • Use ChromaDB and embeddings to enable semantic search over private documents that an LLM cannot access on its own.
  • Integrate the Google Gemini API with a grounding prompt that enforces source citations and honest "I don't know" responses when context is insufficient.
  • Secret Mission: Added conversation memory so the system handles multi-turn follow-up questions using capped history.

Ready to quiz yourself?