Document Q&A API with RAG and Gemini

Build a RAG pipeline that ingests documents and answers questions with Gemini.

Introduction

30 Second Summary

Every company has documents full of answers that nobody can find quickly. What if you could just ask a question in plain English and get an answer pulled directly from those documents?

In this project, you will build a REST API that ingests text documents, stores them as vector embeddings, and answers natural language questions using Retrieval-Augmented Generation (RAG) with Google Gemini.

What You'll Build

A FastAPI application that chunks documents, embeds them into ChromaDB, and uses semantic search with Gemini to generate grounded answers from your uploaded text.

By the end of this project, you'll have:

  • A FastAPI application with document ingestion and natural language query endpoints.
  • A RAG pipeline that chunks text, generates embeddings, and retrieves relevant context via semantic search.
  • A local ChromaDB vector store persisted to disk that stores and retrieves document embeddings.
  • Secret Mission: Add document listing and metadata filtering so you can query specific documents by source.

Are there any prerequisites?

This project assumes you have basic Python knowledge and some familiarity with FastAPI. You do not need any prior experience with RAG, vector databases, or the Gemini API.

Set Up Your Environment

Before you can build the Q&A API, you need a clean workspace with the right tools installed. Your project depends on five Python packages that handle everything from serving HTTP requests to storing vector embeddings.

In this step, you will create an isolated Python environment, install your dependencies, and verify that your Gemini API key works. This ensures nothing breaks when you start building the actual pipeline.

In this step, get ready to:
  • Create a project folder and Python virtual environment.
  • Install FastAPI, ChromaDB, and the Gemini SDK.
  • Obtain and verify your Gemini API key.
Create a project directory and virtual environment

A virtual environment is a self-contained Python installation that keeps your project's packages separate from your system Python. This prevents version conflicts between projects.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir doc-qa-api
cd doc-qa-api
  • Create a virtual environment by running:
python3 -m venv venv

What does this command do?

python3 -m venv venv tells Python to create a virtual environment in a folder called venv inside your project directory.

All packages you install next will live inside this folder instead of your global Python installation.

  • Activate the virtual environment by running the command for your operating system:

macOS

source venv/bin/activate

Windows

.\venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

Make sure you ran the activation command for your specific operating system. On Windows, you may need to allow script execution first by running Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned in PowerShell.

help me activate my Python virtual environment

Install all dependencies

Your project uses five packages. FastAPI handles HTTP requests, uvicorn runs the server, google-genai connects to Gemini, ChromaDB stores embeddings, and python-dotenv loads environment variables from a file.

  • Install all five packages at once by running:
pip install fastapi uvicorn google-genai chromadb python-dotenv

What does this install?

  • fastapi is a modern Python web framework for building APIs.
  • uvicorn is an ASGI server that runs your FastAPI application.
  • google-genai is Google's official SDK for the Gemini API.
  • chromadb is a vector database that stores and searches embeddings locally.
  • python-dotenv reads key-value pairs from a .env file into environment variables.
  • Verify the packages installed correctly by running:
pip list

You should see fastapi, uvicorn, google-genai, chromadb, and python-dotenv listed in the output along with their version numbers.

Seeing installation errors?

Make sure your virtual environment is active. You should see (venv) in your terminal prompt. If not, re-run the activation command from the previous substep.

If a specific package fails, try installing it individually to isolate the error.

help me fix my pip install errors

Get your Gemini API key and verify it works

The Gemini API requires an API key for authentication. You can get one for free from Google AI Studio with no credit card required.

  • Open your browser and navigate to aistudio.google.com.
  • Sign in with your Google account.
  • Click Get API Key in the left sidebar.
  • Click Create API Key.
  • Copy the generated key. You will only see it once, so keep it available for the next step.

What is an API key?

An API key is a unique string that identifies your application to a service. Gemini uses it to track your usage and enforce rate limits.

Keep your API key private. Never commit it to version control or share it publicly.

Now you will store your key in a .env file so your application can load it securely without hardcoding it.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop and select the doc-qa-api folder, then click Open.
  • Create a new file by clicking the new file icon in the Explorer sidebar, or by pressing Cmd+N (macOS) or Ctrl+N (Windows).
  • Name the file .env.
  • Paste the following into .env, replacing your-api-key-here with the API key you copied from Google AI Studio:
GEMINI_API_KEY=your-api-key-here

Why use a .env file?

A .env file keeps secrets out of your source code. The python-dotenv library reads this file and loads each line as an environment variable that your Python code can access.

  • Save .env.

✔️ Awesome, I've got everything!

Great. Double check you have saved your .env file with your actual API key pasted in.

ⓧ I'd like to double check the full code

GEMINI_API_KEY=your-api-key-here

Your file should have exactly one line with GEMINI_API_KEY= followed by your actual Gemini API key (not the literal text your-api-key-here).

Now verify that your API key works by running a quick test script.

  • Create a new file called test_key.py in your doc-qa-api folder.
  • Paste the following code into test_key.py:
from google import genai
from dotenv import load_dotenv
import os

# Load the API key from .env
load_dotenv()

# Create the Gemini client
client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))

# Send a test request to verify the key works
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Say hello in one sentence."
)

print(response.text)

What does this code do?

  • load_dotenv() reads your .env file and loads GEMINI_API_KEY into the environment.
  • genai.Client(api_key=...) creates a client that authenticates with the Gemini API.
  • client.models.generate_content(...) sends a prompt to the gemini-2.5-flash model and returns a text response.
  • Save test_key.py.
  • Open the integrated terminal in VS Code by pressing Ctrl+` (backtick key, same shortcut on macOS and Windows).
  • Make sure your virtual environment is active. You should see (venv) in the prompt. If not, re-run the activation command from earlier.
  • Run the test script:
python test_key.py

You should see a short greeting printed in the terminal. This confirms your API key is working and the Gemini SDK is correctly installed.

Seeing an authentication error?

Double check that your .env file is in the same folder where you are running the script (doc-qa-api).

Make sure you pasted the full API key with no extra spaces or line breaks.

If you see a ModuleNotFoundError, confirm your virtual environment is active and you ran pip install inside it.

help me fix my Gemini API key test

  • Once you see a successful response, delete test_key.py. You will not need it for the rest of the project.

Your environment is ready. Next up, you will build the module that chunks documents and generates embeddings with Gemini.

Build the Chunking and Embedding Module

Your environment is ready and your API key is verified. Now it's time to build the core intelligence of your Q&A system.

RAG pipelines start by breaking documents into small, searchable pieces and converting them into numerical representations called embeddings. In this step, you'll create the module that handles both of these transformations.

In this step, get ready to:
  • Build a text chunking function with overlap to preserve context at boundaries.
  • Create an embedding function using Gemini's embedding model.
  • Test the module to verify it produces 768-dimensional vectors.
Create the rag.py module with text chunking

Your Q&A API needs two core capabilities: splitting documents into digestible chunks, and converting those chunks into vectors that a database can search. You'll build both in a single Python module called rag.py.

  • In VS Code, create a new file called rag.py in your project folder.
  • Add the imports and client initialization by pasting this code:
from google import genai
from google.genai import types
import os
from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv()

# Initialize the Gemini client with your API key
client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))

What does this code do?

  • from google import genai imports the Gemini SDK you installed in the previous step.
  • from google.genai import types gives you access to configuration objects like embedding settings.
  • load_dotenv() reads your .env file so GEMINI_API_KEY is available as an environment variable.
  • genai.Client(...) creates a reusable client that authenticates every API call you make.
  • Below the client initialization, add the chunk_text function by pasting this code:

def chunk_text(text: str, chunk_size: int = 500, overlap: int = 100) -> list[str]:
    """Split text into overlapping chunks to preserve context at boundaries."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        # Move forward by chunk_size minus overlap
        start += chunk_size - overlap
    return chunks

What does chunk_text do?

Documents can be thousands of characters long, but embedding models work best on smaller passages. This function slides a 500-character window across the text, stepping forward 400 characters at a time (500 minus the 100-character overlap).

The overlap means the end of one chunk repeats as the start of the next. This prevents sentences from being split mid-thought across chunk boundaries.

  • Save rag.py.
  • Verify the file exists by running this in your terminal:
ls rag.py

You should see rag.py in the output.

Don't see rag.py?

  • Make sure you saved the file in your project folder, not a subfolder.
  • Check that your terminal is in the same directory as the file. Run pwd to confirm your current location.

Still stuck? Help me find where my rag.py file was saved.

Add the embedding function

Chunking alone isn't enough for search. You need to convert each text chunk into a numerical vector so the database can measure how similar two pieces of text are.

  • In rag.py, add the embed_texts function below chunk_text by pasting this code:

def embed_texts(texts: list[str]) -> list[list[float]]:
    """Generate 768-dimensional embeddings for a list of text strings."""
    embeddings = []
    for text in texts:
        # Call Gemini's embedding model for each text chunk
        result = client.models.embed_content(
            model="gemini-embedding-2",
            contents=text,
            config=types.EmbedContentConfig(output_dimensionality=768),
        )
        # Extract the embedding vector from the response
        embeddings.append(result.embeddings[0].values)
    return embeddings

What does embed_texts do?

This function loops through a list of text strings and sends each one to the gemini-embedding-2 model. The model returns a 768-dimensional vector for each input.

The output_dimensionality=768 setting explicitly requests 768 dimensions. This is a good balance between embedding quality and storage efficiency for a learning project.

  • Save rag.py.

Seeing a red underline on the imports?

  • Make sure VS Code is using the Python interpreter from your virtual environment. Check the bottom-left status bar for the interpreter path.
  • If VS Code shows the system Python, click the interpreter label in the status bar and select the one inside your project's virtual environment folder.

Still seeing errors? Help me fix my VS Code Python interpreter selection.

✔️ Awesome, I've got everything!

Great. Double check you've saved rag.py with both functions before moving on.

ⓧ I'd like to double check the full code

Here's the complete rag.py file at this point:

from google import genai
from google.genai import types
import os
from dotenv import load_dotenv

load_dotenv()

client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))


def chunk_text(text: str, chunk_size: int = 500, overlap: int = 100) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start += chunk_size - overlap
    return chunks


def embed_texts(texts: list[str]) -> list[list[float]]:
    embeddings = []
    for text in texts:
        result = client.models.embed_content(
            model="gemini-embedding-2",
            contents=text,
            config=types.EmbedContentConfig(output_dimensionality=768),
        )
        embeddings.append(result.embeddings[0].values)
    return embeddings
Test the module standalone

Before building the rest of the pipeline, verify that chunking and embedding both work correctly. You'll run a quick test from the command line to confirm the output dimensions.

  • Test the chunking function by running this command in your terminal:
python -c "from rag import chunk_text; chunks = chunk_text('Python is a versatile language used for web development and AI. ' * 20); print(f'Chunks created: {len(chunks)}'); print(f'First chunk length: {len(chunks[0])} characters')"

You should see output showing multiple chunks, each 500 characters long.

  • Test the embedding function by running this command:
python -c "from rag import embed_texts; result = embed_texts(['Hello world']); print(f'Embedding dimensions: {len(result[0])}')"

You should see Embedding dimensions: 768 printed to the terminal.

Why 768 dimensions?

Each dimension in the vector captures a different aspect of the text's meaning. The gemini-embedding-2 model supports output sizes from 128 to 3072.

768 dimensions give strong semantic accuracy without consuming too much memory in the vector database. For a learning project, this is the sweet spot.

Seeing a ModuleNotFoundError?

  • Make sure your virtual environment is activated. You should see the environment name in parentheses at the start of your terminal prompt.
  • Confirm your terminal's working directory is the project folder where rag.py lives. Run ls to check.
  • If you see an error about google.genai, re-run pip install google-genai with your virtual environment active.

Need more help? Help me debug my ModuleNotFoundError when testing rag.py.

Your chunking and embedding module is working. Next up, you'll connect it to ChromaDB so those embeddings can be stored and searched.

Set Up ChromaDB and the Ingestion Pipeline

You have a working chunking and embedding module. Now you need somewhere to store those embeddings so you can search them later.

In this step, you will set up ChromaDB as your local vector database and build an ingestion pipeline that chunks documents, embeds them, and persists everything to disk.

In this step, get ready to:
  • Initialize ChromaDB with persistent storage.
  • Build a document ingestion function that chunks, embeds, and stores.
  • Test the pipeline with a sample document.
Create the vector store module

ChromaDB is a vector database that stores embeddings on disk and lets you query them by similarity. You will create a dedicated module to manage all interactions with it.

  • Create a new file called vector_store.py in the same folder as rag.py.
  • Add the imports and ChromaDB initialization by pasting this code:
import chromadb
from rag import chunk_text, embed_texts

# Initialize a persistent ChromaDB client that saves data to disk
chroma_client = chromadb.PersistentClient(path="./chroma_data")
# Create (or retrieve) a collection to hold document chunks
collection = chroma_client.get_or_create_collection(name="documents")

What does this code do?

chromadb.PersistentClient(path="./chroma_data") creates a ChromaDB instance that writes its database files to a ./chroma_data folder in your project. Data survives between script runs.

get_or_create_collection(name="documents") fetches the collection if it already exists, or creates a new one. This prevents errors when you restart your app.

  • Save vector_store.py.

You won't see a visible change yet. The next chunk adds the function that actually uses this collection.

Build the ingest_document function

The ingestion function ties your RAG pipeline together. It takes raw document text, splits it into chunks, generates embeddings for each chunk, and stores everything in ChromaDB with metadata so you can trace answers back to their source.

  • In vector_store.py, add the ingest_document function below the collection line:
def ingest_document(text: str, source: str) -> int:
    # Split the document into overlapping chunks
    chunks = chunk_text(text)
    # Generate embeddings for each chunk
    embeddings = embed_texts(chunks)
    # Create unique IDs for each chunk using source name and index
    ids = [f"{source}_{i}" for i in range(len(chunks))]
    # Tag each chunk with its source for later filtering
    metadatas = [{"source": source} for _ in chunks]
    # Store everything in ChromaDB
    collection.add(
        ids=ids,
        documents=chunks,
        embeddings=embeddings,
        metadatas=metadatas,
    )
    return len(chunks)

What does this code do?

chunk_text(text) splits the document into overlapping 500-character pieces using the function you built in the previous step.

embed_texts(chunks) sends each chunk to Gemini's embedding model and gets back 768-dimensional vectors.

collection.add() stores the chunks, their embeddings, IDs, and metadata together. The IDs follow the pattern source_0, source_1, etc. The metadata tags each chunk with its source document name so you can filter by source later.

  • Save vector_store.py.

You won't see a change yet. The next chunk adds a query function, then you will test the full module.

Add the query function

Your ingestion pipeline stores documents, but you also need a way to retrieve relevant chunks when a user asks a question. This query function embeds the question and performs a similarity search against your stored chunks.

  • In vector_store.py, add the query_documents function below ingest_document:
def query_documents(question: str, n_results: int = 3, source: str | None = None):
    # Embed the question using the same model as the documents
    query_embedding = embed_texts([question])[0]
    # Build an optional metadata filter for source-specific queries
    where_filter = {"source": source} if source else None
    # Search ChromaDB for the most similar chunks
    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=n_results,
        where=where_filter,
    )
    return results

What does this code do?

embed_texts([question])[0] converts the user's question into the same 768-dimensional vector space as your stored documents. This lets ChromaDB compare the question against every chunk by distance.

collection.query(query_embeddings=..., n_results=3) finds the 3 chunks closest to the question's embedding. The optional where filter restricts results to a specific source document.

  • Save vector_store.py.

Seeing an import error?

Make sure vector_store.py is in the same directory as rag.py. The from rag import chunk_text, embed_texts line only works when both files live in the same folder.

help me debug an import error in vector_store.py

✔️ Awesome, I've got everything!

Great. Double check you have saved vector_store.py with all three sections: the imports and initialization, the ingest_document function, and the query_documents function.

ⓧ I'd like to double check the full code

Here is the complete vector_store.py file:

import chromadb
from rag import chunk_text, embed_texts

chroma_client = chromadb.PersistentClient(path="./chroma_data")
collection = chroma_client.get_or_create_collection(name="documents")


def ingest_document(text: str, source: str) -> int:
    chunks = chunk_text(text)
    embeddings = embed_texts(chunks)
    ids = [f"{source}_{i}" for i in range(len(chunks))]
    metadatas = [{"source": source} for _ in chunks]
    collection.add(
        ids=ids,
        documents=chunks,
        embeddings=embeddings,
        metadatas=metadatas,
    )
    return len(chunks)


def query_documents(question: str, n_results: int = 3, source: str | None = None):
    query_embedding = embed_texts([question])[0]
    where_filter = {"source": source} if source else None
    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=n_results,
        where=where_filter,
    )
    return results
Test the ingestion pipeline

Now verify the full flow works. You will ingest a sample document and check that ChromaDB stored the chunks correctly.

  • Run this test script in your terminal:
python -c "
from vector_store import ingest_document, collection

sample_text = 'Retrieval-Augmented Generation (RAG) is a technique that combines information retrieval with text generation. It works by first finding relevant documents from a knowledge base, then using those documents as context for a language model to generate accurate answers. RAG helps reduce hallucinations by grounding responses in real data.'

count = ingest_document(sample_text, 'rag_intro')
print(f'Ingested {count} chunks')
print(f'Collection now has {collection.count()} items')
"
  • You should see output showing the number of chunks ingested and the total collection count.

What should I see?

The script prints something like Ingested 1 chunks (the sample text is short enough to fit in one chunk) and Collection now has 1 items. The exact numbers depend on text length. The key confirmation is that the count is greater than 0.

  • Confirm a chroma_data folder appeared in your project directory by running:
ls chroma_data

You should see files inside the folder. This confirms ChromaDB is persisting your data to disk.

Seeing an API error from Gemini?

Check that your .env file has the correct GEMINI_API_KEY value. The embedding call requires a valid key to generate vectors.

Also verify your virtual environment is active. If the terminal prompt does not show the environment name in parentheses, activate it again.

help me debug a Gemini API error when running my vector store ingestion

Your vector store is live and persisting data to disk. Next up, you will build the query and generation pipeline that retrieves relevant chunks and feeds them to Gemini for grounded answer generation.

Build the Query and Generation Pipeline

Your document ingestion pipeline stores chunks and embeddings in ChromaDB. Now you need the other half of the RAG pattern: retrieving relevant chunks and using them to generate grounded answers.

In this step, you will build a test script that exercises the complete pipeline. It ingests a sample document, queries it with a natural language question, and passes the retrieved context to Gemini to generate an answer that cites its sources.

In this step, get ready to:
  • Understand how semantic search retrieves relevant chunks from ChromaDB.
  • Build a generation pipeline that grounds Gemini's answers in retrieved context.
  • Test the full RAG flow end-to-end with a sample document.
Understand the query pipeline

When a user asks a question, the system needs to find the most relevant chunks from all stored documents. This is semantic search: instead of matching keywords, it compares the meaning of the question against the meaning of each chunk.

  • In VS Code, open vector_store.py and find the query_documents function you wrote in the previous step.
def query_documents(question: str, n_results: int = 3, source: str | None = None):
    query_embedding = embed_texts([question])[0]
    where_filter = {"source": source} if source else None
    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=n_results,
        where=where_filter,
    )
    return results

How does semantic search work here?

The function embeds the user's question into the same 768-dimensional vector space as your document chunks. Then ChromaDB compares that question vector against every stored chunk vector using cosine similarity.

The n_results=3 parameter tells ChromaDB to return only the top 3 most similar chunks. The optional where_filter narrows the search to chunks from a specific source document.

Create the end-to-end pipeline test

The generation step takes retrieved chunks, formats them into a prompt, and asks Gemini to answer using only that context. You will build a test script that exercises every stage of the pipeline: ingest, retrieve, and generate.

  • Create a new file called test_pipeline.py in your project folder.
  • Add the imports, client setup, and ingestion logic by pasting this code:
from vector_store import ingest_document, query_documents
from google import genai
from google.genai import types
import os
from dotenv import load_dotenv

load_dotenv()

client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))

# Sample document to ingest for testing
sample_text = """Python is a high-level programming language created by Guido van Rossum.
It was first released in 1991. Python emphasizes code readability and supports
multiple programming paradigms including procedural, object-oriented, and
functional programming. Python is widely used in web development, data science,
artificial intelligence, and automation."""

# Ingest the sample document into ChromaDB
print("Ingesting sample document...")
chunk_count = ingest_document(sample_text, "python_overview")
print(f"Ingested {chunk_count} chunks")

What does this code do?

This first section imports your existing ingest_document and query_documents functions from vector_store.py. It also sets up its own Gemini client for the generation step.

The sample text is a short paragraph about Python. When you run this, it gets chunked and embedded into ChromaDB just like any real document would.

  • Now add the query and generation logic below the ingestion code:
# Query the vector store with a natural language question
question = "Who created Python and when was it first released?"
print(f"\nQuerying: {question}")
results = query_documents(question)

# Build context from retrieved chunks
chunks = results["documents"][0]
context = "\n\n---\n\n".join(
    f"[Source: {results['metadatas'][0][i]['source']}]\n{chunk}"
    for i, chunk in enumerate(chunks)
)

# Construct a prompt that instructs Gemini to answer from context only
prompt = f"""Answer the question using only the context provided below.
If the answer is not in the context, say "I don't have enough information to answer that."
Cite which source the information comes from.

CONTEXT:
{context}

QUESTION: {question}

ANSWER:"""

# Generate the grounded answer using Gemini
print("\nGenerating answer...")
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents=prompt,
    config=types.GenerateContentConfig(
        thinking_config=types.ThinkingConfig(thinking_level="low")
    ),
)
print(f"\nAnswer: {response.text}")

What does the generation logic do?

This is the core of RAG. First, query_documents finds the chunks most semantically similar to your question. Then those chunks get formatted into a structured context block with source labels.

The prompt explicitly tells Gemini to only use the provided context and to cite sources. This prevents hallucination. If the answer is not in the retrieved chunks, Gemini says so instead of making something up.

The thinking_level="low" config keeps responses fast for this Q&A use case.

  • Save test_pipeline.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before running it.

ⓧ I'd like to double check the full code

from vector_store import ingest_document, query_documents
from google import genai
from google.genai import types
import os
from dotenv import load_dotenv

load_dotenv()

client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))

# Sample document to ingest for testing
sample_text = """Python is a high-level programming language created by Guido van Rossum.
It was first released in 1991. Python emphasizes code readability and supports
multiple programming paradigms including procedural, object-oriented, and
functional programming. Python is widely used in web development, data science,
artificial intelligence, and automation."""

# Ingest the sample document into ChromaDB
print("Ingesting sample document...")
chunk_count = ingest_document(sample_text, "python_overview")
print(f"Ingested {chunk_count} chunks")

# Query the vector store with a natural language question
question = "Who created Python and when was it first released?"
print(f"\nQuerying: {question}")
results = query_documents(question)

# Build context from retrieved chunks
chunks = results["documents"][0]
context = "\n\n---\n\n".join(
    f"[Source: {results['metadatas'][0][i]['source']}]\n{chunk}"
    for i, chunk in enumerate(chunks)
)

# Construct a prompt that instructs Gemini to answer from context only
prompt = f"""Answer the question using only the context provided below.
If the answer is not in the context, say "I don't have enough information to answer that."
Cite which source the information comes from.

CONTEXT:
{context}

QUESTION: {question}

ANSWER:"""

# Generate the grounded answer using Gemini
print("\nGenerating answer...")
response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents=prompt,
    config=types.GenerateContentConfig(
        thinking_config=types.ThinkingConfig(thinking_level="low")
    ),
)
print(f"\nAnswer: {response.text}")
Run the end-to-end pipeline test

Time to see every stage of RAG in action. Running this script ingests a document, retrieves relevant chunks, and generates a grounded answer in one go.

  • Run your test script by executing this command:
python test_pipeline.py

You should see output showing the ingestion count, the query being processed, and a generated answer that mentions Guido van Rossum and 1991 with a citation to python_overview as the source.

What should I see?

The output will look something like:

Ingesting sample document...
Ingested 2 chunks

Querying: Who created Python and when was it first released?

Generating answer...

Answer: Python was created by Guido van Rossum and was first released in 1991. (Source: python_overview)

The exact wording of the answer varies each run because Gemini generates it dynamically. The key thing is that it references facts from your document and cites the source.

Seeing an error?

If you see a ModuleNotFoundError, make sure your virtual environment is activated and you installed all dependencies in Step 1.

If you get an API error from Gemini, check that your .env file contains a valid GEMINI_API_KEY and that it is in the same folder as test_pipeline.py.

If you see a duplicate ID warning, that means you ran the script more than once with the same source name. This is safe to ignore for testing purposes.

Help me debug my test_pipeline.py error

Your full RAG pipeline is working. You have proven that documents go in, semantic search finds the right chunks, and Gemini generates grounded answers. Next up, you will wrap this pipeline in a FastAPI application with proper endpoints.

Wire It All Into FastAPI Endpoints

Your RAG modules can chunk text, generate embeddings, store them in ChromaDB, and retrieve relevant results. All that's left is exposing this pipeline to the outside world as a web API.

In this step, you'll wrap your ingestion and query functions in FastAPI endpoints. By the end, anyone (or any frontend) can upload documents and ask questions over HTTP.

In this step, get ready to:
  • Create a FastAPI app with a document upload endpoint.
  • Add a query endpoint that retrieves context and generates grounded answers.
  • Run the server and test the full flow through interactive docs.
Create the FastAPI app and document upload endpoint

The /documents endpoint will accept a text body and a source name, then pass them to the ingestion pipeline you already built. FastAPI's Pydantic integration validates incoming requests automatically.

  • Create a new file called main.py in the same folder as rag.py and vector_store.py.
  • Add the following code to set up the app, define request/response models, and create the document upload endpoint:
from fastapi import FastAPI
from pydantic import BaseModel
from google import genai
from google.genai import types
from dotenv import load_dotenv
import os
from vector_store import ingest_document, query_documents

load_dotenv()

# Initialize the FastAPI app and Gemini client
app = FastAPI(title="Document Q&A API")

client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))


# Define the shape of incoming and outgoing data
class DocumentRequest(BaseModel):
    text: str
    source: str


class QueryRequest(BaseModel):
    question: str
    source: str | None = None


class QueryResponse(BaseModel):
    answer: str
    sources: list[str]


# Endpoint to upload and ingest a document
@app.post("/documents")
def upload_document(request: DocumentRequest):
    chunk_count = ingest_document(request.text, request.source)
    return {"message": f"Document '{request.source}' ingested successfully", "chunks": chunk_count}

What does this code do?

  • The imports bring in FastAPI, Pydantic for data validation, the Gemini SDK, and your own vector_store functions.
  • Three Pydantic models define the expected shape of requests and responses. QueryRequest has an optional source field for filtering results to a specific document.
  • The POST /documents endpoint calls ingest_document from your pipeline and returns a confirmation message with the chunk count.
  • Save main.py.
  • You won't see a change yet. The next substep adds the query endpoint, and then you'll run the server to test everything together.
Add the query endpoint

The query endpoint ties the full RAG loop together in one request: embed the question, retrieve relevant chunks, build a prompt with context, and generate a grounded answer.

  • In main.py, add the following function below the upload_document endpoint:
# Endpoint to query documents and generate a grounded answer
@app.post("/query", response_model=QueryResponse)
def query(request: QueryRequest):
    # Retrieve the most relevant chunks from ChromaDB
    results = query_documents(request.question, source=request.source)
    chunks = results["documents"][0]
    sources = list(set(m["source"] for m in results["metadatas"][0]))

    # Build a context string with source citations for each chunk
    context = "\n\n---\n\n".join(
        f"[Source: {results['metadatas'][0][i]['source']}]\n{chunk}"
        for i, chunk in enumerate(chunks)
    )

    # Construct the prompt that constrains Gemini to answer from context only
    prompt = f"""Answer the question using only the context provided below.
If the answer is not in the context, say "I don't have enough information to answer that."
Cite which source the information comes from.

CONTEXT:
{context}

QUESTION: {request.question}

ANSWER:"""

    # Generate the answer using Gemini
    response = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=prompt,
        config=types.GenerateContentConfig(
            thinking_config=types.ThinkingConfig(thinking_level="low")
        ),
    )
    return QueryResponse(answer=response.text, sources=sources)

What does this code do?

  • query_documents embeds the user's question and retrieves the top 3 matching chunks from ChromaDB.
  • The context string labels each chunk with its source, so Gemini knows which document each piece of information came from.
  • The prompt instructs Gemini to answer only from the provided context and cite sources. This is what makes the answer "grounded" rather than hallucinated.
  • thinking_level="low" keeps response latency low while still allowing Gemini to reason briefly before answering.
  • Save main.py.

Seeing a syntax error?

Make sure the triple-quoted f-string (f"""...""") is not indented inconsistently. The closing """ must sit at the same indentation level as the start of the string content.

help me fix a syntax error in my FastAPI query endpoint

✔️ Awesome, I've got everything!

Great. Double check you've saved main.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete main.py file:

from fastapi import FastAPI
from pydantic import BaseModel
from google import genai
from google.genai import types
from dotenv import load_dotenv
import os
from vector_store import ingest_document, query_documents

load_dotenv()

app = FastAPI(title="Document Q&A API")

client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))


class DocumentRequest(BaseModel):
    text: str
    source: str


class QueryRequest(BaseModel):
    question: str
    source: str | None = None


class QueryResponse(BaseModel):
    answer: str
    sources: list[str]


@app.post("/documents")
def upload_document(request: DocumentRequest):
    chunk_count = ingest_document(request.text, request.source)
    return {"message": f"Document '{request.source}' ingested successfully", "chunks": chunk_count}


@app.post("/query", response_model=QueryResponse)
def query(request: QueryRequest):
    results = query_documents(request.question, source=request.source)
    chunks = results["documents"][0]
    sources = list(set(m["source"] for m in results["metadatas"][0]))
    context = "\n\n---\n\n".join(
        f"[Source: {results['metadatas'][0][i]['source']}]\n{chunk}"
        for i, chunk in enumerate(chunks)
    )
    prompt = f"""Answer the question using only the context provided below.
If the answer is not in the context, say "I don't have enough information to answer that."
Cite which source the information comes from.

CONTEXT:
{context}

QUESTION: {request.question}

ANSWER:"""

    response = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=prompt,
        config=types.GenerateContentConfig(
            thinking_config=types.ThinkingConfig(thinking_level="low")
        ),
    )
    return QueryResponse(answer=response.text, sources=sources)
Run the server and test the endpoints

FastAPI comes with built-in interactive documentation powered by Swagger UI. You can test both endpoints directly in your browser without writing any client code.

  • Start the development server by running this command in your terminal:
uvicorn main:app --reload

What does this command do?

  • uvicorn is the ASGI server that runs your FastAPI app.
  • main:app tells uvicorn to look in main.py for the object named app.
  • --reload makes the server restart automatically whenever you save a file change.
  • You should see output indicating the server is running at http://127.0.0.1:8000.
  • Open your browser and navigate to http://localhost:8000/docs.

You should see the Swagger UI interactive documentation page with your two endpoints listed: POST /documents and POST /query.

Server won't start?

  • Make sure your virtual environment is activated. You should see the environment name in your terminal prompt.
  • Check that you're running the command from the same directory that contains main.py.
  • If you see an import error, verify vector_store.py and rag.py are in the same folder.

help me fix my uvicorn server startup error

Now test the full flow. First, upload a document:

  • Click on the POST /documents endpoint to expand it.
  • Click **Try it out**.
  • Replace the example JSON body with a short test document. For example:
{
  "text": "Python is a high-level programming language created by Guido van Rossum. It was first released in 1991. Python emphasizes code readability and supports multiple programming paradigms including object-oriented and functional programming.",
  "source": "python_intro"
}
  • Click **Execute**.
  • You should see a response with a success message and the number of chunks created.

Next, test the query endpoint:

  • Click on the POST /query endpoint to expand it.
  • Click **Try it out**.
  • Enter a question about the document you just uploaded:
{
  "question": "Who created Python?"
}
  • Click **Execute**.
  • You should see a JSON response with an answer that mentions Guido van Rossum and a sources array containing python_intro.

Why is this grounded?

The answer comes exclusively from the document chunks you uploaded, not from Gemini's general knowledge. If you ask a question about something not in the uploaded documents, the API responds with "I don't have enough information to answer that."

This is the core value of RAG: answers are traceable to specific source documents.

Getting an empty or error response?

  • Check your terminal for error messages. A KeyError usually means ChromaDB returned no results. Try uploading a longer document.
  • If you see a Gemini API error, verify your .env file still has a valid GEMINI_API_KEY.
  • Make sure the ./chroma_data directory is accessible and the server has write permissions.

help me debug my RAG query endpoint response

Secret mission

Add Document Management and Metadata Filtering

Your API can ingest and query documents, but what happens when you have dozens of sources? Right now there's no way to see what's in your vector store or narrow queries to a single document. Time to fix that.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally with a free-tier Gemini API key, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Project folder containing main.py, rag.py, vector_store.py, .env, and the chroma_data directory.
  • Python virtual environment (inside your project folder).
  • Gemini API key from Google AI Studio (free, no ongoing cost).

Keep everything running

No action needed. Choose this if you're still actively building or experimenting with document uploads and queries.

  • Your FastAPI server continues running in the terminal.
  • Your ChromaDB data persists in the chroma_data folder on disk.
  • Your Gemini API key remains active but costs nothing on the free tier.

Pause - I'll come back to this later

Shut down running processes to free up memory. Your files stay on disk so you can resume later.

  • Switch to the terminal where your server is running and press Ctrl+C to stop it.
  • Deactivate your virtual environment by running:
deactivate

Your project files and ChromaDB data remain on disk. To resume, activate the virtual environment again and restart the server.

Delete - I don't want to use this again

Remove all project resources and start fresh. This deletes your code, ChromaDB data, and virtual environment permanently.

  • Switch to the terminal where your server is running and press Ctrl+C to stop it.
  • Deactivate your virtual environment by running:
deactivate
  • Navigate to your Desktop and delete the project folder by running:
cd ~/Desktop
rm -rf [[PROJECT_FOLDER="your project folder name"]]

Windows users

If you're on Windows, use this command instead:

cd ~\Desktop
rmdir /s /q [[PROJECT_FOLDER="your project folder name"]]
  • Optionally, revoke your Gemini API key. Visit aistudio.google.com/api-keys, find your key, click the three-dot menu next to it, and select Delete.

Your API key is free

There is no financial reason to delete your Gemini API key. It stays on the free tier with no charges. Revoke it only if you want to practice good security hygiene.

Nice Work!

Nice Work!

Nice work! You just built a production-style Document Q&A API powered by Retrieval-Augmented Generation.

You've learned how to:

  • Build a complete RAG pipeline from scratch, covering text chunking, vector embedding, semantic retrieval, and grounded answer generation.
  • Create a FastAPI application with document ingestion and natural language query endpoints that form a working Q&A system.
  • Connect Gemini's embedding model (gemini-embedding-2) and generation model (gemini-2.5-flash) with ChromaDB for persistent vector storage and semantic search.
  • Secret Mission: Added a document listing endpoint and metadata filtering so you can query specific documents by source.

Ready to quiz yourself?