Build a RAG App with Google Gemini

Build a RAG pipeline that answers questions about your own documents.

Introduction

30 Second Summary

Ever wondered how AI chatbots can answer questions about files you upload? Behind the scenes, they break your documents into pieces, find the most relevant ones, and feed them to an AI model so it can give you an accurate answer.

In this project, you will build a RAG (Retrieval-Augmented Generation) application that lets you ask natural language questions about your own documents and get grounded, accurate answers powered by Google Gemini.

What You'll Build

A command-line Python application that ingests text documents, stores their embeddings in a local vector database, and retrieves relevant context to generate accurate answers using Gemini 2.5 Flash.

By the end of this project, you'll have:

  • A document ingestion pipeline that chunks text and generates embeddings with the Gemini Embedding model.
  • A local ChromaDB database storing your document vectors, queryable with cosine similarity.
  • A retrieval and generation pipeline that finds the most relevant context for any question and produces grounded answers with Gemini 2.5 Flash.
  • Secret Mission: metadata filtering so you can scope your questions to specific document sources.

Are there any prerequisites?

You should be comfortable with basic Python concepts like variables, functions, and loops. No prior experience with AI, embeddings, or vector databases is needed.

Before We Start

To build a RAG application, you need Python, a few packages, and access to Google's Gemini API. This step gets all of that in place so you can focus on the interesting parts.

You'll check that Python is installed, create an isolated project environment, and grab a free API key from Google AI Studio.

In this step, get ready to:
  • Install Python and create a virtual environment for the project.
  • Install the required Python packages.
  • Get a free Gemini API key and set it as an environment variable.
Install Python and create a virtual environment

Python is the language you'll use to build the RAG pipeline. A virtual environment keeps this project's packages separate from anything else on your machine.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.
  • Check your Python version by running this command:
python3 --version

✔️ I see version 3.10 or higher

Your Python version is ready. Move on to creating your project folder below.

ⓧ I see an older version

You need Python 3.10 or higher for this project. macOS sometimes ships an older Python that shadows newer installs.

  • Download the latest Python from python.org/downloads.
  • Run the installer and follow the prompts.
  • On macOS, after installing, try python3.14 --version if python3 --version still shows the old version.
  • On Windows, the Python Install Manager from the Microsoft Store is the recommended approach. After installing, run python --version again.

ⓧ Command not found

Python is not installed yet. Follow the steps below for your platform.

  • Download the latest Python from python.org/downloads.
  • On macOS, open the downloaded .pkg file and follow the installer prompts.
  • On Windows, the recommended method is to install the Python Install Manager from the Microsoft Store. Search for "Python" in the Store and click Install.
  • After installation, close and reopen your terminal, then run python3 --version (macOS) or python --version (Windows) to confirm.

Windows users and the python3 command

On Windows, the command is typically python instead of python3. If python3 doesn't work, try python --version instead. Use whichever command shows your version for all remaining steps.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir rag-gemini
cd rag-gemini
  • Create a virtual environment by running this command:

macOS

python3 -m venv venv
  • Activate the virtual environment:
source venv/bin/activate

Windows

python -m venv venv
  • Activate the virtual environment:
.\venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

What is a virtual environment?

A virtual environment is an isolated folder that holds its own copy of Python packages. Anything you install with pip while the environment is active stays inside this folder and won't interfere with other projects on your machine.

Don't see (venv) in your prompt?

Make sure you ran the activate command from inside the rag-gemini folder. If you're on Windows and see a permissions error, run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned in PowerShell, then try activating again.

Still stuck? Help me activate my Python virtual environment.

Install the required packages

Your project needs two packages. The google-genai SDK connects you to Gemini for embeddings and text generation. ChromaDB is a local vector database that stores and searches your document embeddings.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop and select the rag-gemini folder, then click Open.
  • Open a terminal inside VS Code by pressing Ctrl+` (both macOS and Windows).

Why use VS Code's terminal?

VS Code's integrated terminal automatically opens in your project folder. It also detects your virtual environment and activates it for you. If you see (venv) in the prompt, you're all set. If not, re-run the activate command from the previous substep.

  • Install both packages by running this command:
pip install google-genai chromadb

What does this install?

This grabs the google-genai SDK (Google's official Python library for Gemini) and chromadb (a lightweight vector database that runs locally). Together, these are the two main dependencies for your RAG pipeline.

The install takes a minute or two. When it finishes, you should see a success message with no errors.

Seeing a permissions error or install failure?

Make sure (venv) is visible in your terminal prompt. If you install without the virtual environment active, pip may try to write to a system-protected location. Re-activate the venv and try again.

Still stuck? Help me fix my pip install error for google-genai and chromadb.

Get your free Gemini API key

The Gemini API powers both the embedding model and the text generation model in your RAG app. You'll get a free API key from Google AI Studio.

  • Open your browser and go to aistudio.google.com.
  • Sign in with your Google account.
  • For new users, Google AI Studio creates a default project and API key after you accept the Terms of Service. If you already see a key listed, you can use it.
  • If you need to create a new key, click Get API key in the left sidebar, then click Create API key.
  • Copy your API key. You will only see it once, so keep this tab open or paste it somewhere safe temporarily.

Keep your API key secret

Treat your API key like a password. Never commit it to version control or share it publicly. You'll store it in an environment variable so it stays out of your code.

  • Back in your VS Code terminal, set the API key as an environment variable by running this command:

macOS

export GEMINI_API_KEY=your-api-key-here
  • Replace your-api-key-here with the actual key you copied from Google AI Studio.

Windows

$env:GEMINI_API_KEY="your-api-key-here"
  • Replace your-api-key-here with the actual key you copied from Google AI Studio.
  • Verify the variable is set by running:

macOS

echo $GEMINI_API_KEY

Windows

echo $env:GEMINI_API_KEY

You should see your API key printed back in the terminal. This confirms the environment variable is set correctly.

Why an environment variable?

Storing the key in an environment variable keeps it out of your source code. Your Python script reads the key at runtime using os.environ.get("GEMINI_API_KEY"). This way, even if you share your code, the key stays private.

This key only lasts for your current session

Environment variables set with export (macOS) or $env: (Windows) disappear when you close the terminal. If you come back to this project later, you'll need to set GEMINI_API_KEY again.

Having trouble creating an API key?

If you see a permissions error in Google AI Studio, try creating a new Google Cloud project first. You can also create a project that is not associated with an organization to avoid permission restrictions.

Need help? Help me get my Gemini API key working.

Your environment is ready. Next up, you'll create sample documents and build a chunking function that breaks them into searchable pieces.

Set Up Your Environment

You've confirmed what you're building. Now it's time to write the foundation of your RAG application.

In this step, you'll create the main application file and configure it to connect to both Google Gemini (for embeddings and text generation) and ChromaDB (for storing and querying vectors locally).

In this step, get ready to:
  • Confirm your Python environment and packages are ready.
  • Create the rag_app.py application file with configuration constants.
  • Initialize the Gemini and ChromaDB clients.
Confirm your Python environment is ready

Your virtual environment is already active and the required packages are installed. Let's do a quick sanity check to confirm everything is in place.

  • In your terminal (with the virtual environment active), verify your packages by running:
pip list | grep -i -E "google-genai|chromadb"

You should see both google-genai and chromadb listed with their version numbers.

Don't see one or both packages?

Make sure your virtual environment is active. You should see (venv) at the start of your terminal prompt. If a package is missing, reinstall it by running pip install google-genai chromadb.

Help me troubleshoot my virtual environment and missing packages

Create your application file with configuration

Every RAG application needs a central configuration that defines which models to use, where to store data, and how to authenticate. You'll set these up as constants at the top of your file so they're easy to find and change later.

  • In VS Code, create a new file called rag_app.py inside your project folder.
  • Add the following imports and configuration constants:
import os
from google import genai
from google.genai import types
import chromadb

# --- Configuration ---
GEMINI_API_KEY = os.environ.get("GEMINI_API_KEY", "your-api-key-here")
EMBEDDING_MODEL = "gemini-embedding-001"
GENERATION_MODEL = "gemini-2.5-flash"
CHROMA_DB_PATH = "./chroma_db"
COLLECTION_NAME = "my_documents"

What does this code do?

  • os.environ.get("GEMINI_API_KEY", "your-api-key-here") reads your API key from the environment variable you already set. The fallback string is just a safety net for clarity.
  • EMBEDDING_MODEL points to gemini-embedding-001, a text-only embedding model that converts your documents into numerical vectors.
  • GENERATION_MODEL points to gemini-2.5-flash, the model that will generate answers based on retrieved context.
  • CHROMA_DB_PATH tells ChromaDB where to persist your vector database on disk.
  • COLLECTION_NAME is the name of the collection (like a table) inside ChromaDB that will hold your document embeddings.
  • Save rag_app.py.

Seeing a red underline on the imports?

VS Code might not recognize your packages if it's using the wrong Python interpreter. Press Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows), type Python: Select Interpreter, and choose the one inside your venv folder.

Help me fix import errors in VS Code for my Python virtual environment

Initialize the Gemini and ChromaDB clients

With configuration in place, you need to create the client objects that your code will use to talk to the Gemini API and your local ChromaDB database.

  • Below the configuration section in rag_app.py, add the client initialization code:
# --- Initialize Clients ---
gemini_client = genai.Client(api_key=GEMINI_API_KEY)
chroma_client = chromadb.PersistentClient(path=CHROMA_DB_PATH)

What does this code do?

  • genai.Client(api_key=GEMINI_API_KEY) creates a client that authenticates with Google's Gemini API using your key. You'll use this client to generate embeddings and text responses.
  • chromadb.PersistentClient(path=CHROMA_DB_PATH) creates a local database client that saves data to the ./chroma_db folder. This means your embeddings persist between runs.
  • Save rag_app.py.
  • Verify the file runs without errors by running this in your terminal:
python3 rag_app.py

You should see no output and no errors. The script exits cleanly because it only defines configuration and clients so far.

Seeing a ModuleNotFoundError?

This means Python can't find google-genai or chromadb. Confirm your virtual environment is active (look for (venv) in your prompt). If you're on macOS and python3 doesn't match your venv, try python rag_app.py instead.

Help me fix ModuleNotFoundError when running my RAG app

✔️ Awesome, I've got everything!

Great. Double check you've saved rag_app.py and that it runs without errors.

ⓧ I'd like to double check the full code

Here's the complete rag_app.py so far:

import os
from google import genai
from google.genai import types
import chromadb

# --- Configuration ---
GEMINI_API_KEY = os.environ.get("GEMINI_API_KEY", "your-api-key-here")
EMBEDDING_MODEL = "gemini-embedding-001"
GENERATION_MODEL = "gemini-2.5-flash"
CHROMA_DB_PATH = "./chroma_db"
COLLECTION_NAME = "my_documents"

# --- Initialize Clients ---
gemini_client = genai.Client(api_key=GEMINI_API_KEY)
chroma_client = chromadb.PersistentClient(path=CHROMA_DB_PATH)

Your foundation is set. You've got both clients ready to go. Next up, you'll create sample documents and learn how to break them into chunks that your vector database can store.

Create Sample Documents and Chunk Them

Your environment is ready with all packages installed and your API key configured. Now you need data for your RAG application to work with.

A RAG application is only as good as the documents it can search through. But you can't just throw entire documents at an AI model. Large language models have context windows, and stuffing entire articles into a prompt wastes tokens on irrelevant information. The solution is to break documents into small, focused chunks that can be selectively retrieved.

In this step, get ready to:
  • Create sample documents that form your knowledge base.
  • Write a chunking function that splits text into overlapping pieces.
  • Test the chunking function and verify it produces correct output.
Create your sample documents

In a production RAG system, you'd load documents from files, databases, or APIs. For this project, you'll define sample documents directly in Python so you can focus on learning the RAG pipeline without file I/O complexity.

  • In rag_app.py, add the following DOCUMENTS list below the client initialization lines:
# --- Sample Documents ---
DOCUMENTS = [
    {
        "content": "Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation. Python is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented, and functional programming. Python was conceived in the late 1980s by Guido van Rossum at Centrum Wiskunde & Informatica (CWI) in the Netherlands. Python consistently ranks as one of the most popular programming languages.",
        "source": "python_overview.txt",
        "category": "programming",
    },
    {
        "content": "Machine learning is a subset of artificial intelligence that focuses on building systems that learn from data. Instead of being explicitly programmed to perform a task, these systems use algorithms to parse data, learn from it, and then make predictions or decisions. There are three main types of machine learning: supervised learning, unsupervised learning, and reinforcement learning. Supervised learning uses labeled data to train models, while unsupervised learning finds patterns in unlabeled data.",
        "source": "ml_basics.txt",
        "category": "ai",
    },
    {
        "content": "Retrieval-Augmented Generation (RAG) is a technique that combines information retrieval with text generation. In a RAG system, when a user asks a question, the system first retrieves relevant documents from a knowledge base, then passes those documents as context to a large language model to generate an answer. This approach helps reduce hallucinations because the model's response is grounded in actual retrieved data rather than relying solely on its training data.",
        "source": "rag_explained.txt",
        "category": "ai",
    },
    {
        "content": "Vector databases are specialized databases designed to store and query high-dimensional vectors efficiently. They use indexing algorithms like HNSW (Hierarchical Navigable Small World) to enable fast approximate nearest neighbor searches. Popular vector databases include ChromaDB, Pinecone, Weaviate, and Qdrant. Vector databases are essential components in modern AI applications, particularly for semantic search and RAG systems where you need to find documents similar to a query.",
        "source": "vector_databases.txt",
        "category": "databases",
    },
]

What does this code do?

Each document is a dictionary with three keys:

  • content holds the actual text that will be chunked and embedded.
  • source tracks which file the text came from. This becomes metadata you can filter on later.
  • category groups documents by topic. This is useful for scoping searches to specific subjects.

The four documents cover programming, AI, RAG itself, and vector databases. This variety lets you test how well your RAG system retrieves the right context for different questions.

  • Save rag_app.py.
Write the chunking function

When you embed and store documents, smaller chunks give you more precise retrieval. A question about "what is RAG" should pull back just the relevant paragraph, not an entire article. Overlapping chunks ensure no information gets lost at the boundaries between pieces.

Why overlapping chunks?

Imagine a key sentence that falls right at the end of one chunk and the beginning of the next. Without overlap, you'd split that sentence in two and lose its meaning. By overlapping chunks by 50 characters, each piece captures a bit of its neighbor's context.

  • In rag_app.py, add the following chunk_text function below the DOCUMENTS list:
# --- Document Chunking ---
def chunk_text(text, chunk_size=500, overlap=50):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        chunks.append(chunk)
        start += chunk_size - overlap
    return chunks

What does this code do?

  • chunk_size=500 sets each chunk to a maximum of 500 characters. This keeps chunks small enough to be specific but large enough to retain meaning.
  • overlap=50 means each new chunk starts 50 characters before the previous one ended. This prevents losing context at chunk boundaries.
  • The while loop slides a window across the text, grabbing 500-character slices and stepping forward by 450 characters (500 minus the 50-character overlap) each time.
  • Save rag_app.py.

Seeing a syntax error?

Make sure the function is at the top level of your file (no extra indentation). The def keyword should start at column 0, aligned with the DOCUMENTS variable above it.

Also check that there are two blank lines between the end of the DOCUMENTS list and the # --- Document Chunking --- comment.

Help me fix a syntax error in my chunk_text function.

Test the chunking function

Before building the rest of the pipeline, you should verify your chunking function works correctly. A quick test in the terminal confirms that chunks are the right size and overlap is working.

  • Test the chunking function by running this command in your terminal:
python -c "from rag_app import chunk_text, DOCUMENTS; chunks = chunk_text(DOCUMENTS[0]['content']); print(f'Document length: {len(DOCUMENTS[0][chr(99)+chr(111)+chr(110)+chr(116)+chr(101)+chr(110)+chr(116)])} chars'); print(f'Number of chunks: {len(chunks)}'); print(f'First chunk ({len(chunks[0])} chars): {chunks[0][:80]}...')"

You should see output showing the document is about 500+ characters long, split into 2 chunks, with the first chunk being 500 characters.

Getting a ModuleNotFoundError?

Make sure your virtual environment is still active (you should see (venv) in your terminal prompt). If not, reactivate it with source venv/bin/activate on macOS/Linux or .\venv\Scripts\activate on Windows.

Also make sure you saved rag_app.py before running the test command.

Help me debug a ModuleNotFoundError when testing my chunk_text function.

✔️ Awesome, I've got everything!

Great. Double check you've saved rag_app.py with both the DOCUMENTS list and the chunk_text function.

ⓧ I'd like to double check the full code

Here's what your complete rag_app.py should look like at this point:

import os
from google import genai
from google.genai import types
import chromadb

# --- Configuration ---
GEMINI_API_KEY = os.environ.get("GEMINI_API_KEY", "your-api-key-here")
EMBEDDING_MODEL = "gemini-embedding-001"
GENERATION_MODEL = "gemini-2.5-flash"
CHROMA_DB_PATH = "./chroma_db"
COLLECTION_NAME = "my_documents"

# --- Initialize Clients ---
gemini_client = genai.Client(api_key=GEMINI_API_KEY)
chroma_client = chromadb.PersistentClient(path=CHROMA_DB_PATH)

# --- Sample Documents ---
DOCUMENTS = [
    {
        "content": "Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation. Python is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented, and functional programming. Python was conceived in the late 1980s by Guido van Rossum at Centrum Wiskunde & Informatica (CWI) in the Netherlands. Python consistently ranks as one of the most popular programming languages.",
        "source": "python_overview.txt",
        "category": "programming",
    },
    {
        "content": "Machine learning is a subset of artificial intelligence that focuses on building systems that learn from data. Instead of being explicitly programmed to perform a task, these systems use algorithms to parse data, learn from it, and then make predictions or decisions. There are three main types of machine learning: supervised learning, unsupervised learning, and reinforcement learning. Supervised learning uses labeled data to train models, while unsupervised learning finds patterns in unlabeled data.",
        "source": "ml_basics.txt",
        "category": "ai",
    },
    {
        "content": "Retrieval-Augmented Generation (RAG) is a technique that combines information retrieval with text generation. In a RAG system, when a user asks a question, the system first retrieves relevant documents from a knowledge base, then passes those documents as context to a large language model to generate an answer. This approach helps reduce hallucinations because the model's response is grounded in actual retrieved data rather than relying solely on its training data.",
        "source": "rag_explained.txt",
        "category": "ai",
    },
    {
        "content": "Vector databases are specialized databases designed to store and query high-dimensional vectors efficiently. They use indexing algorithms like HNSW (Hierarchical Navigable Small World) to enable fast approximate nearest neighbor searches. Popular vector databases include ChromaDB, Pinecone, Weaviate, and Qdrant. Vector databases are essential components in modern AI applications, particularly for semantic search and RAG systems where you need to find documents similar to a query.",
        "source": "vector_databases.txt",
        "category": "databases",
    },
]


# --- Document Chunking ---
def chunk_text(text, chunk_size=500, overlap=50):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        chunks.append(chunk)
        start += chunk_size - overlap
    return chunks

Your documents are defined and your chunking function is ready. Next up, you'll build the ingestion pipeline that turns these chunks into vector embeddings and stores them in ChromaDB.

Build the Document Ingestion Pipeline

Your documents are chunked and ready to go. But raw text chunks alone aren't searchable by meaning.

To find chunks that are relevant to a question, you need to convert them into vector embeddings. In this step, you'll build an ingestion pipeline that turns each chunk into a numerical vector and stores it in ChromaDB.

In this step, get ready to:
  • Initialize a ChromaDB collection with cosine similarity.
  • Generate vector embeddings for each chunk using the Gemini Embedding model.
  • Store the embeddings, documents, and metadata in your vector database.
Initialize the ChromaDB collection and prepare chunks

Before you can store embeddings, you need a place to put them. A collection in ChromaDB is like a table in a traditional database, but optimized for vector search.

  • In rag_app.py, scroll to the bottom of the file (below the chunk_text function) and add the following ingestion function:
# --- Ingestion Pipeline ---
def ingest_documents(documents):
    """Chunk documents, generate embeddings, and store in ChromaDB."""
    collection = chroma_client.get_or_create_collection(
        name=COLLECTION_NAME,
        metadata={"hnsw:space": "cosine"},
    )

    all_chunks = []
    all_metadatas = []
    all_ids = []

    for doc_index, doc in enumerate(documents):
        chunks = chunk_text(doc["content"])
        for chunk_index, chunk in enumerate(chunks):
            all_chunks.append(chunk)
            all_metadatas.append({
                "source": doc["source"],
                "category": doc["category"],
            })
            all_ids.append(f"doc{doc_index}_chunk{chunk_index}")

What does this code do?

get_or_create_collection creates a new collection called my_documents (or opens it if it already exists). The metadata={"hnsw:space": "cosine"} setting tells ChromaDB to use cosine similarity when comparing vectors.

The loop iterates through each document, chunks its content, and builds three parallel lists: the text chunks themselves, their metadata (source file and category), and a unique ID for each chunk.

  • You won't see a change yet. The next section adds the embedding logic that uses this prepared data.
Generate embeddings with the Gemini Embedding model

Each text chunk needs to become a vector (a list of numbers) that captures its meaning. The Gemini Embedding model converts text into 768-dimensional vectors that you can compare mathematically.

  • Still inside the ingest_documents function, add this code directly after the for loop (at the same indentation level as all_chunks = []):
    # Generate embeddings using Gemini Embedding model
    print(f"Generating embeddings for {len(all_chunks)} chunks...")
    all_embeddings = []
    for chunk in all_chunks:
        response = gemini_client.models.embed_content(
            model=EMBEDDING_MODEL,
            contents=chunk,
            config=types.EmbedContentConfig(output_dimensionality=768),
        )
        all_embeddings.append(response.embeddings[0].values)

What does this code do?

For each chunk, the code calls gemini_client.models.embed_content to generate an embedding. The output_dimensionality=768 parameter tells the model to produce a 768-number vector for each chunk.

The gemini-embedding-001 model supports Matryoshka Representation Learning, which means you can request smaller vectors (768 instead of the default 3072) without significant quality loss. Smaller vectors mean faster search.

  • You won't see a change yet. The next section stores these embeddings in the database.
Store the embeddings in ChromaDB

With embeddings generated, the final piece is writing them into ChromaDB so they persist on disk and are ready for searching.

  • Still inside the ingest_documents function, add this code directly after the embedding loop:
    # Store in ChromaDB
    collection.upsert(
        ids=all_ids,
        documents=all_chunks,
        metadatas=all_metadatas,
        embeddings=all_embeddings,
    )
    print(f"Successfully ingested {len(all_chunks)} chunks into ChromaDB.")
    return collection

What does this code do?

collection.upsert inserts new records or updates existing ones if the IDs already exist. This means you can safely re-run the ingestion without creating duplicate entries.

Each record in ChromaDB now contains four things: a unique ID, the original text chunk, its metadata (source and category), and its embedding vector. ChromaDB persists all of this to the ./chroma_db folder on your disk automatically.

  • Test the ingestion pipeline by temporarily adding this line at the very bottom of rag_app.py (outside any function):
ingest_documents(DOCUMENTS)
  • Save rag_app.py.
  • Run the script by executing this command in your terminal:
python rag_app.py
  • You should see output similar to:
Generating embeddings for 5 chunks...
Successfully ingested 5 chunks into ChromaDB.
  • Confirm the database was created by running:
ls chroma_db
  • You should see files inside the chroma_db folder, confirming your vectors are persisted to disk.
  • Now remove the temporary test line (ingest_documents(DOCUMENTS)) from the bottom of rag_app.py and save the file.

Seeing an API error?

  • Make sure your GEMINI_API_KEY environment variable is set in the terminal where you ran the script. If you opened a new terminal, you need to export it again.
  • Check that your virtual environment is active (you should see (venv) in your terminal prompt).
  • If you see a rate limit error, wait 60 seconds and try again. The free tier has a 10 requests-per-minute limit.

Help me debug my Gemini embedding API error

Why use upsert instead of add?

During development, you'll likely run the ingestion multiple times as you tweak your documents. Using upsert means re-running the script updates existing records instead of throwing a duplicate ID error.

✔️ Awesome, I've got everything!

Great. Double check you've saved rag_app.py and that the temporary ingest_documents(DOCUMENTS) test line has been removed.

ⓧ I'd like to double check the full code

Here is the complete rag_app.py up to this point:

import os
from google import genai
from google.genai import types
import chromadb

# --- Configuration ---
GEMINI_API_KEY = os.environ.get("GEMINI_API_KEY", "your-api-key-here")
EMBEDDING_MODEL = "gemini-embedding-001"
GENERATION_MODEL = "gemini-2.5-flash"
CHROMA_DB_PATH = "./chroma_db"
COLLECTION_NAME = "my_documents"

# --- Initialize Clients ---
gemini_client = genai.Client(api_key=GEMINI_API_KEY)
chroma_client = chromadb.PersistentClient(path=CHROMA_DB_PATH)

# --- Sample Documents ---
DOCUMENTS = [
    {
        "content": "Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation. Python is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented, and functional programming. Python was conceived in the late 1980s by Guido van Rossum at Centrum Wiskunde & Informatica (CWI) in the Netherlands. Python consistently ranks as one of the most popular programming languages.",
        "source": "python_overview.txt",
        "category": "programming",
    },
    {
        "content": "Machine learning is a subset of artificial intelligence that focuses on building systems that learn from data. Instead of being explicitly programmed to perform a task, these systems use algorithms to parse data, learn from it, and then make predictions or decisions. There are three main types of machine learning: supervised learning, unsupervised learning, and reinforcement learning. Supervised learning uses labeled data to train models, while unsupervised learning finds patterns in unlabeled data.",
        "source": "ml_basics.txt",
        "category": "ai",
    },
    {
        "content": "Retrieval-Augmented Generation (RAG) is a technique that combines information retrieval with text generation. In a RAG system, when a user asks a question, the system first retrieves relevant documents from a knowledge base, then passes those documents as context to a large language model to generate an answer. This approach helps reduce hallucinations because the model's response is grounded in actual retrieved data rather than relying solely on its training data.",
        "source": "rag_explained.txt",
        "category": "ai",
    },
    {
        "content": "Vector databases are specialized databases designed to store and query high-dimensional vectors efficiently. They use indexing algorithms like HNSW (Hierarchical Navigable Small World) to enable fast approximate nearest neighbor searches. Popular vector databases include ChromaDB, Pinecone, Weaviate, and Qdrant. Vector databases are essential components in modern AI applications, particularly for semantic search and RAG systems where you need to find documents similar to a query.",
        "source": "vector_databases.txt",
        "category": "databases",
    },
]


# --- Document Chunking ---
def chunk_text(text, chunk_size=500, overlap=50):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        chunks.append(chunk)
        start += chunk_size - overlap
    return chunks


# --- Ingestion Pipeline ---
def ingest_documents(documents):
    """Chunk documents, generate embeddings, and store in ChromaDB."""
    collection = chroma_client.get_or_create_collection(
        name=COLLECTION_NAME,
        metadata={"hnsw:space": "cosine"},
    )

    all_chunks = []
    all_metadatas = []
    all_ids = []

    for doc_index, doc in enumerate(documents):
        chunks = chunk_text(doc["content"])
        for chunk_index, chunk in enumerate(chunks):
            all_chunks.append(chunk)
            all_metadatas.append({
                "source": doc["source"],
                "category": doc["category"],
            })
            all_ids.append(f"doc{doc_index}_chunk{chunk_index}")

    # Generate embeddings using Gemini Embedding model
    print(f"Generating embeddings for {len(all_chunks)} chunks...")
    all_embeddings = []
    for chunk in all_chunks:
        response = gemini_client.models.embed_content(
            model=EMBEDDING_MODEL,
            contents=chunk,
            config=types.EmbedContentConfig(output_dimensionality=768),
        )
        all_embeddings.append(response.embeddings[0].values)

    # Store in ChromaDB
    collection.upsert(
        ids=all_ids,
        documents=all_chunks,
        metadatas=all_metadatas,
        embeddings=all_embeddings,
    )
    print(f"Successfully ingested {len(all_chunks)} chunks into ChromaDB.")
    return collection

Your vector database is loaded and ready for search. Next up, you'll build the retrieval and generation pipeline that finds relevant chunks and uses Gemini to answer questions.

Build the Retrieval and Generation Pipeline

Your documents are chunked, embedded, and stored in ChromaDB. Now you need a way to actually use that knowledge base.

In this step, you'll build the two remaining pieces of the RAG pipeline. First, a retrieval function that finds the most relevant chunks for any question. Second, a generation function that feeds those chunks to Gemini 2.5 Flash with instructions to answer only from the provided context.

In this step, get ready to:
  • Write a retrieval function that embeds a question and finds matching chunks.
  • Build a generation function that constructs a grounded prompt and calls Gemini.
Write the retrieval function

When a user asks a question, your app needs to find the stored chunks that are most relevant to that question. This is semantic search. You embed the question into the same vector space as your documents, then ask ChromaDB to return the nearest neighbors.

  • In rag_app.py, scroll to the bottom of your file (below the ingest_documents function).
  • Add the retrieval function by pasting this code:
# --- Retrieval ---
def retrieve_context(query, n_results=3):
    """Embed the query and retrieve the most similar chunks."""
    collection = chroma_client.get_or_create_collection(
        name=COLLECTION_NAME,
        metadata={"hnsw:space": "cosine"},
    )

    # Embed the query
    query_response = gemini_client.models.embed_content(
        model=EMBEDDING_MODEL,
        contents=query,
        config=types.EmbedContentConfig(output_dimensionality=768),
    )
    query_embedding = query_response.embeddings[0].values

    # Query ChromaDB
    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=n_results,
        include=["documents", "metadatas", "distances"],
    )

    return results

What does this code do?

  • get_or_create_collection reconnects to the same ChromaDB collection you built during ingestion.
  • gemini_client.models.embed_content converts the user's question into a 768-dimension vector, using the same model and dimensionality as your stored chunks.
  • collection.query performs a cosine similarity search and returns the top n_results most relevant chunks along with their metadata and distance scores.

Why use the same embedding model for queries?

The query embedding must live in the same vector space as your document embeddings for cosine similarity to be meaningful. If you used a different model or dimensionality, the distances would be meaningless.

  • Save rag_app.py.
  • You won't see output yet. The next function will use retrieve_context to fetch context before generating an answer.

Seeing a syntax error on save?

  • Make sure the function is at the top level of your file (not indented inside ingest_documents).
  • Check that you have a blank line between the end of ingest_documents and the # --- Retrieval --- comment.
  • Help me debug a syntax error in my retrieve_context function.
Build the answer generation function

Retrieval alone gives you raw text chunks. The generation function takes those chunks, formats them into a structured prompt, and sends everything to Gemini with clear instructions to answer only from the provided context.

What is prompt engineering for RAG?

A RAG prompt has three parts: a system instruction (telling the model to only use provided context), the retrieved context itself, and the user's question. This structure prevents the model from hallucinating answers from its training data.

The instruction to say "I don't have enough information" when context is insufficient is critical. Without it, the model would guess freely, defeating the purpose of RAG.

  • Below your retrieve_context function, add the generation function by pasting this code:
# --- Generation ---
def generate_answer(query, context_results):
    """Generate a grounded answer using retrieved context."""
    # Format the context
    context_parts = []
    for i, doc in enumerate(context_results["documents"][0]):
        source = context_results["metadatas"][0][i]["source"]
        distance = context_results["distances"][0][i]
        context_parts.append(f"[Source: {source}, Relevance: {1 - distance:.2f}]\n{doc}")

    context_text = "\n\n---\n\n".join(context_parts)

    # Build the augmented prompt
    prompt = f"""You are a helpful assistant. Answer the user's question based ONLY on the provided context below. If the context does not contain enough information to answer the question, say "I don't have enough information in my documents to answer that question."

CONTEXT:
{context_text}

QUESTION: {query}

ANSWER:"""

    # Generate response with Gemini
    response = gemini_client.models.generate_content(
        model=GENERATION_MODEL,
        contents=prompt,
        config=types.GenerateContentConfig(
            temperature=0.3,
            max_output_tokens=500,
        ),
    )
    return response.text

What does this code do?

  • The first loop formats each retrieved chunk with its source filename and a relevance score (1 minus the cosine distance). This helps Gemini understand where each piece of context came from.
  • The prompt string uses three sections: a system instruction constraining the model, the formatted context, and the user's question. This is the core RAG pattern.
  • temperature=0.3 keeps the output focused and factual. A higher temperature would make the model more creative, which you don't want when grounding answers in documents.
  • max_output_tokens=500 caps the response length so answers stay concise.
  • Save rag_app.py.
  • Verify your file has no syntax errors by running this command in your terminal:
python rag_app.py

The script should exit cleanly with no output. If you see an error, there's a typo in your code.

Seeing a SyntaxError or IndentationError?

  • Check that the f-string triple quotes f"""...""" are properly closed. The closing """ must appear on its own line.
  • Make sure all lines inside the function body are indented by 4 spaces.
  • Verify you have matching brackets in the GenerateContentConfig call.
  • Help me fix a syntax error in my generate_answer function.

✔️ Awesome, I've got everything!

Great. Double check you've saved your file.

ⓧ I'd like to double check the full code

Here is the complete rag_app.py file up to this point:

import os
from google import genai
from google.genai import types
import chromadb

# --- Configuration ---
GEMINI_API_KEY = os.environ.get("GEMINI_API_KEY", "your-api-key-here")
EMBEDDING_MODEL = "gemini-embedding-001"
GENERATION_MODEL = "gemini-2.5-flash"
CHROMA_DB_PATH = "./chroma_db"
COLLECTION_NAME = "my_documents"

# --- Initialize Clients ---
gemini_client = genai.Client(api_key=GEMINI_API_KEY)
chroma_client = chromadb.PersistentClient(path=CHROMA_DB_PATH)

# --- Sample Documents ---
DOCUMENTS = [
    {
        "content": "Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation. Python is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented, and functional programming. Python was conceived in the late 1980s by Guido van Rossum at Centrum Wiskunde & Informatica (CWI) in the Netherlands. Python consistently ranks as one of the most popular programming languages.",
        "source": "python_overview.txt",
        "category": "programming",
    },
    {
        "content": "Machine learning is a subset of artificial intelligence that focuses on building systems that learn from data. Instead of being explicitly programmed to perform a task, these systems use algorithms to parse data, learn from it, and then make predictions or decisions. There are three main types of machine learning: supervised learning, unsupervised learning, and reinforcement learning. Supervised learning uses labeled data to train models, while unsupervised learning finds patterns in unlabeled data.",
        "source": "ml_basics.txt",
        "category": "ai",
    },
    {
        "content": "Retrieval-Augmented Generation (RAG) is a technique that combines information retrieval with text generation. In a RAG system, when a user asks a question, the system first retrieves relevant documents from a knowledge base, then passes those documents as context to a large language model to generate an answer. This approach helps reduce hallucinations because the model's response is grounded in actual retrieved data rather than relying solely on its training data.",
        "source": "rag_explained.txt",
        "category": "ai",
    },
    {
        "content": "Vector databases are specialized databases designed to store and query high-dimensional vectors efficiently. They use indexing algorithms like HNSW (Hierarchical Navigable Small World) to enable fast approximate nearest neighbor searches. Popular vector databases include ChromaDB, Pinecone, Weaviate, and Qdrant. Vector databases are essential components in modern AI applications, particularly for semantic search and RAG systems where you need to find documents similar to a query.",
        "source": "vector_databases.txt",
        "category": "databases",
    },
]


# --- Document Chunking ---
def chunk_text(text, chunk_size=500, overlap=50):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        chunks.append(chunk)
        start += chunk_size - overlap
    return chunks


# --- Ingestion Pipeline ---
def ingest_documents(documents):
    """Chunk documents, generate embeddings, and store in ChromaDB."""
    collection = chroma_client.get_or_create_collection(
        name=COLLECTION_NAME,
        metadata={"hnsw:space": "cosine"},
    )

    all_chunks = []
    all_metadatas = []
    all_ids = []

    for doc_index, doc in enumerate(documents):
        chunks = chunk_text(doc["content"])
        for chunk_index, chunk in enumerate(chunks):
            all_chunks.append(chunk)
            all_metadatas.append({
                "source": doc["source"],
                "category": doc["category"],
            })
            all_ids.append(f"doc{doc_index}_chunk{chunk_index}")

    # Generate embeddings using Gemini Embedding model
    print(f"Generating embeddings for {len(all_chunks)} chunks...")
    all_embeddings = []
    for chunk in all_chunks:
        response = gemini_client.models.embed_content(
            model=EMBEDDING_MODEL,
            contents=chunk,
            config=types.EmbedContentConfig(output_dimensionality=768),
        )
        all_embeddings.append(response.embeddings[0].values)

    # Store in ChromaDB
    collection.upsert(
        ids=all_ids,
        documents=all_chunks,
        metadatas=all_metadatas,
        embeddings=all_embeddings,
    )
    print(f"Successfully ingested {len(all_chunks)} chunks into ChromaDB.")
    return collection


# --- Retrieval ---
def retrieve_context(query, n_results=3):
    """Embed the query and retrieve the most similar chunks."""
    collection = chroma_client.get_or_create_collection(
        name=COLLECTION_NAME,
        metadata={"hnsw:space": "cosine"},
    )

    # Embed the query
    query_response = gemini_client.models.embed_content(
        model=EMBEDDING_MODEL,
        contents=query,
        config=types.EmbedContentConfig(output_dimensionality=768),
    )
    query_embedding = query_response.embeddings[0].values

    # Query ChromaDB
    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=n_results,
        include=["documents", "metadatas", "distances"],
    )

    return results


# --- Generation ---
def generate_answer(query, context_results):
    """Generate a grounded answer using retrieved context."""
    # Format the context
    context_parts = []
    for i, doc in enumerate(context_results["documents"][0]):
        source = context_results["metadatas"][0][i]["source"]
        distance = context_results["distances"][0][i]
        context_parts.append(f"[Source: {source}, Relevance: {1 - distance:.2f}]\n{doc}")

    context_text = "\n\n---\n\n".join(context_parts)

    # Build the augmented prompt
    prompt = f"""You are a helpful assistant. Answer the user's question based ONLY on the provided context below. If the context does not contain enough information to answer the question, say "I don't have enough information in my documents to answer that question."

CONTEXT:
{context_text}

QUESTION: {query}

ANSWER:"""

    # Generate response with Gemini
    response = gemini_client.models.generate_content(
        model=GENERATION_MODEL,
        contents=prompt,
        config=types.GenerateContentConfig(
            temperature=0.3,
            max_output_tokens=500,
        ),
    )
    return response.text

Your retrieval and generation pipeline is ready. Next up, you'll wire everything together into a working application and test it with real questions.

Test Your RAG Application

You've built both halves of the RAG pipeline: ingestion (chunking and embedding documents) and generation (retrieving context and producing grounded answers). Now it's time to wire them together into a single runnable application.

In this step, you'll add a main() function that orchestrates the entire flow. Then you'll test it with different types of questions to see how grounding changes the quality of AI responses.

In this step, get ready to:
  • Add a main function that runs the complete pipeline end-to-end.
  • Test with answerable and unanswerable questions.
  • Compare RAG-grounded answers against plain Gemini responses.
Run the full pipeline end-to-end

Your script has all the building blocks but no entry point. The main() function will ingest the sample documents, then start an interactive loop where you can ask questions and receive grounded answers.

  • In rag_app.py, scroll to the very bottom of the file (after the generate_answer function) and add the following code:
# --- Main ---
def main():
    print("=" * 60)
    print("RAG Application with Google Gemini")
    print("=" * 60)

    # Step 1: Ingest documents
    print("\n--- Ingesting Documents ---")
    ingest_documents(DOCUMENTS)

    # Step 2: Query loop
    print("\n--- Ready to answer questions! ---")
    print("Type 'quit' to exit.\n")

    while True:
        query = input("Your question: ").strip()
        if query.lower() == "quit":
            break
        if not query:
            continue

        # Retrieve relevant context
        print("\nRetrieving relevant context...")
        results = retrieve_context(query)

        # Show retrieved sources
        print("\nSources found:")
        for i, meta in enumerate(results["metadatas"][0]):
            distance = results["distances"][0][i]
            print(f"  - {meta['source']} (relevance: {1 - distance:.2f})")

        # Generate answer
        print("\nGenerating answer...")
        answer = generate_answer(query, results)
        print(f"\nAnswer: {answer}\n")
        print("-" * 40)


if __name__ == "__main__":
    main()

What does this code do?

  • main() first calls ingest_documents(DOCUMENTS) to chunk, embed, and store all four sample documents in ChromaDB.
  • It then enters a while True loop that prompts you for questions, retrieves the top 3 most relevant chunks, and generates a grounded answer.
  • For each answer, it prints the source filenames and relevance scores so you can see which documents were used.
  • if __name__ == "__main__" ensures the pipeline only runs when you execute the script directly, not when importing it.
  • Save rag_app.py.
  • Run the application by executing this command in your terminal:
python rag_app.py

You should see output showing that embeddings are being generated for your chunks, followed by a prompt saying Your question: waiting for your input.

Seeing an API error?

  • Make sure your GEMINI_API_KEY environment variable is still set in this terminal session. If you opened a new terminal, you need to export it again.
  • Check that your virtual environment is active (you should see (venv) in your terminal prompt).
  • If you see a rate limit error, wait a moment and try again. The free tier has rate limits but they reset quickly for small workloads.

help me debug my Gemini API error

Test with answerable and unanswerable questions

The real test of a RAG system is whether it answers faithfully from its documents and admits when it cannot answer.

  • With your application still running, type an answerable question and press Enter:
What is RAG and how does it help reduce hallucinations?

You should see sources from rag_explained.txt with high relevance scores, followed by a grounded answer that references retrieval and context.

  • Now try a question that is NOT answerable from your documents:
What is the capital of France?

The model should respond with something like "I don't have enough information in my documents to answer that question." This is your RAG system working correctly.

Why does this matter?

A well-built RAG system refuses to hallucinate. The prompt you wrote in the previous step instructs Gemini to only answer from the provided context. When no relevant context exists, the model follows that instruction and declines to guess.

  • Try a few more questions to explore different topics in your knowledge base:
What programming paradigms does Python support?

This should pull from python_overview.txt and mention structured, object-oriented, and functional programming.

  • Type quit and press Enter to exit the application when you're done testing.
Compare a RAG-grounded response vs a plain Gemini response

To truly appreciate what RAG does, compare the grounded answer you just got against what Gemini says without any context.

  • Open a Python interactive session by running:
python
  • Ask Gemini the same question directly (without any retrieved context) by typing these lines:
from google import genai
import os
from google.genai import types

client = genai.Client(api_key=os.environ.get("GEMINI_API_KEY"))
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="What is RAG and how does it help reduce hallucinations?",
)
print(response.text)

What should I see?

Gemini gives a general answer from its training data. It may include information that goes beyond your sample documents, or even contradict them.

Compare this to your RAG answer, which only cited information from rag_explained.txt and stayed within the boundaries of what your documents contain. That is the power of grounding.

  • Exit the Python session by typing:
exit()

The value of grounding

In production, RAG prevents AI from making up answers. Customer support bots, internal knowledge search, and legal research tools all use RAG to ensure responses are traceable back to source documents.

✔️ Awesome, I've got everything!

Great. Double check you've saved rag_app.py and that your application ran successfully.

ⓧ I'd like to double check the full code

Here is the complete rag_app.py file:

import os
from google import genai
from google.genai import types
import chromadb

# --- Configuration ---
GEMINI_API_KEY = os.environ.get("GEMINI_API_KEY", "your-api-key-here")
EMBEDDING_MODEL = "gemini-embedding-001"
GENERATION_MODEL = "gemini-2.5-flash"
CHROMA_DB_PATH = "./chroma_db"
COLLECTION_NAME = "my_documents"

# --- Initialize Clients ---
gemini_client = genai.Client(api_key=GEMINI_API_KEY)
chroma_client = chromadb.PersistentClient(path=CHROMA_DB_PATH)

# --- Sample Documents ---
DOCUMENTS = [
    {
        "content": "Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation. Python is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented, and functional programming. Python was conceived in the late 1980s by Guido van Rossum at Centrum Wiskunde & Informatica (CWI) in the Netherlands. Python consistently ranks as one of the most popular programming languages.",
        "source": "python_overview.txt",
        "category": "programming",
    },
    {
        "content": "Machine learning is a subset of artificial intelligence that focuses on building systems that learn from data. Instead of being explicitly programmed to perform a task, these systems use algorithms to parse data, learn from it, and then make predictions or decisions. There are three main types of machine learning: supervised learning, unsupervised learning, and reinforcement learning. Supervised learning uses labeled data to train models, while unsupervised learning finds patterns in unlabeled data.",
        "source": "ml_basics.txt",
        "category": "ai",
    },
    {
        "content": "Retrieval-Augmented Generation (RAG) is a technique that combines information retrieval with text generation. In a RAG system, when a user asks a question, the system first retrieves relevant documents from a knowledge base, then passes those documents as context to a large language model to generate an answer. This approach helps reduce hallucinations because the model's response is grounded in actual retrieved data rather than relying solely on its training data.",
        "source": "rag_explained.txt",
        "category": "ai",
    },
    {
        "content": "Vector databases are specialized databases designed to store and query high-dimensional vectors efficiently. They use indexing algorithms like HNSW (Hierarchical Navigable Small World) to enable fast approximate nearest neighbor searches. Popular vector databases include ChromaDB, Pinecone, Weaviate, and Qdrant. Vector databases are essential components in modern AI applications, particularly for semantic search and RAG systems where you need to find documents similar to a query.",
        "source": "vector_databases.txt",
        "category": "databases",
    },
]


# --- Document Chunking ---
def chunk_text(text, chunk_size=500, overlap=50):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        chunks.append(chunk)
        start += chunk_size - overlap
    return chunks


# --- Ingestion Pipeline ---
def ingest_documents(documents):
    """Chunk documents, generate embeddings, and store in ChromaDB."""
    collection = chroma_client.get_or_create_collection(
        name=COLLECTION_NAME,
        metadata={"hnsw:space": "cosine"},
    )

    all_chunks = []
    all_metadatas = []
    all_ids = []

    for doc_index, doc in enumerate(documents):
        chunks = chunk_text(doc["content"])
        for chunk_index, chunk in enumerate(chunks):
            all_chunks.append(chunk)
            all_metadatas.append({
                "source": doc["source"],
                "category": doc["category"],
            })
            all_ids.append(f"doc{doc_index}_chunk{chunk_index}")

    # Generate embeddings using Gemini Embedding model
    print(f"Generating embeddings for {len(all_chunks)} chunks...")
    all_embeddings = []
    for chunk in all_chunks:
        response = gemini_client.models.embed_content(
            model=EMBEDDING_MODEL,
            contents=chunk,
            config=types.EmbedContentConfig(output_dimensionality=768),
        )
        all_embeddings.append(response.embeddings[0].values)

    # Store in ChromaDB
    collection.upsert(
        ids=all_ids,
        documents=all_chunks,
        metadatas=all_metadatas,
        embeddings=all_embeddings,
    )
    print(f"Successfully ingested {len(all_chunks)} chunks into ChromaDB.")
    return collection


# --- Retrieval ---
def retrieve_context(query, n_results=3):
    """Embed the query and retrieve the most similar chunks."""
    collection = chroma_client.get_or_create_collection(
        name=COLLECTION_NAME,
        metadata={"hnsw:space": "cosine"},
    )

    # Embed the query
    query_response = gemini_client.models.embed_content(
        model=EMBEDDING_MODEL,
        contents=query,
        config=types.EmbedContentConfig(output_dimensionality=768),
    )
    query_embedding = query_response.embeddings[0].values

    # Query ChromaDB
    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=n_results,
        include=["documents", "metadatas", "distances"],
    )

    return results


# --- Generation ---
def generate_answer(query, context_results):
    """Generate a grounded answer using retrieved context."""
    # Format the context
    context_parts = []
    for i, doc in enumerate(context_results["documents"][0]):
        source = context_results["metadatas"][0][i]["source"]
        distance = context_results["distances"][0][i]
        context_parts.append(f"[Source: {source}, Relevance: {1 - distance:.2f}]\n{doc}")

    context_text = "\n\n---\n\n".join(context_parts)

    # Build the augmented prompt
    prompt = f"""You are a helpful assistant. Answer the user's question based ONLY on the provided context below. If the context does not contain enough information to answer the question, say "I don't have enough information in my documents to answer that question."

CONTEXT:
{context_text}

QUESTION: {query}

ANSWER:"""

    # Generate response with Gemini
    response = gemini_client.models.generate_content(
        model=GENERATION_MODEL,
        contents=prompt,
        config=types.GenerateContentConfig(
            temperature=0.3,
            max_output_tokens=500,
        ),
    )
    return response.text


# --- Main ---
def main():
    print("=" * 60)
    print("RAG Application with Google Gemini")
    print("=" * 60)

    # Step 1: Ingest documents
    print("\n--- Ingesting Documents ---")
    ingest_documents(DOCUMENTS)

    # Step 2: Query loop
    print("\n--- Ready to answer questions! ---")
    print("Type 'quit' to exit.\n")

    while True:
        query = input("Your question: ").strip()
        if query.lower() == "quit":
            break
        if not query:
            continue

        # Retrieve relevant context
        print("\nRetrieving relevant context...")
        results = retrieve_context(query)

        # Show retrieved sources
        print("\nSources found:")
        for i, meta in enumerate(results["metadatas"][0]):
            distance = results["distances"][0][i]
            print(f"  - {meta['source']} (relevance: {1 - distance:.2f})")

        # Generate answer
        print("\nGenerating answer...")
        answer = generate_answer(query, results)
        print(f"\nAnswer: {answer}\n")
        print("-" * 40)


if __name__ == "__main__":
    main()

Secret mission

Add Metadata Filtering

Your RAG app searches all documents at once. But what if you only want answers from a specific category? Add metadata filtering so users can scope their searches to specific sources or topics, just like production RAG systems do.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally with a free-tier API key, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Project folder containing rag_app.py and the chroma_db/ directory.
  • Python virtual environment (venv folder inside your project).
  • GEMINI_API_KEY environment variable (set in your terminal session).
  • Gemini API key created in Google AI Studio.

Keep everything running

No action needed. Choose this if you're still actively building or plan to extend the project further.

  • Your project files, virtual environment, and ChromaDB data all stay on your machine.
  • Your Gemini API key remains active in Google AI Studio but costs nothing while idle on the free tier.
  • The GEMINI_API_KEY environment variable only lasts for the current terminal session. If you close your terminal, you will need to set it again next time.

Pause - I'll come back to this later

Shut down running processes to free up memory. Your files stay on disk so you can pick up where you left off.

  • Deactivate your virtual environment by running:
deactivate
  • Close VS Code (or Cursor) and your terminal window.
  • Your project folder, chroma_db/ data, and virtual environment remain intact on your Desktop.
  • Your Gemini API key stays active in Google AI Studio. No charges accrue while you are on the free tier.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each step below to fully clean up.

  • Deactivate your virtual environment (if still active) by running:
deactivate
  • Delete your project folder (which contains rag_app.py, chroma_db/, and the virtual environment). Run the command for your operating system:

macOS

cd ~/Desktop
rm -rf rag-app

Windows

cd ~\Desktop
rmdir /s /q rag-app
  • Unset the GEMINI_API_KEY environment variable from your current terminal session:

macOS

unset GEMINI_API_KEY

Windows

Remove-Item Env:GEMINI_API_KEY
  • Delete your Gemini API key from Google AI Studio. Open the API Keys page at https://aistudio.google.com/app/apikey, find the key you created for this project, and click the trash icon to delete it.

Nice Work!

Nice Work!

You just built a complete RAG pipeline from scratch. Your application ingests documents, embeds them into vectors, retrieves the most relevant context for any question, and generates grounded answers using Google Gemini.

You've learned how to:

  • Built a document ingestion pipeline that chunks text into overlapping pieces and generates embeddings with the Gemini Embedding model.
  • Stored and queried vectors in a local ChromaDB database using cosine similarity to find semantically relevant document chunks.
  • Generated grounded answers with Gemini 2.5 Flash that stay faithful to your documents by constructing augmented prompts with retrieved context.
  • Added metadata filtering to scope searches to specific document sources or categories, controlling exactly which chunks the model can draw from.

Ready to quiz yourself?