Build a RAG-Powered Q&A API

Build a document Q&A API using FastAPI, Gemini, and ChromaDB.

Introduction

30 Second Summary

You've probably searched through a long document looking for one specific answer. What if you could just ask a question and get the exact information you need?

In this project, you will build a document Q&A API that uses Retrieval-Augmented Generation (RAG) to answer natural language questions about uploaded text documents.

What You'll Build

A FastAPI server that ingests documents, chunks and embeds them with Google's Gemini API, stores them in ChromaDB, and answers questions using semantic search and AI-generated responses.

By the end of this project, you'll have:

  • A FastAPI application with POST endpoints for document ingestion and question answering.
  • A RAG pipeline that chunks text, generates vector embeddings with Gemini, and performs semantic search with ChromaDB.
  • Pydantic models validating all request and response data structures.
  • A clean GitHub repository with your complete project pushed as a portfolio piece.
  • Secret Mission: Add a /documents GET endpoint that lists all ingested documents with metadata and chunk counts.

Are there any prerequisites?

None. This guide includes full setup for all required tools, including Python, VS Code, and your Gemini API key. No prior experience with APIs or AI is needed.

Before We Start

You're about to build a RAG-powered Q&A API that can answer questions about any document you upload. Before writing any application code, you need a working development environment with Python, your project files, and an API key for Google's Gemini AI.

In this step, you'll install Python, create your project folder with a virtual environment, and grab a free Gemini API key from Google AI Studio.

In this step, get ready to:
  • Install Python 3.10+ and VS Code.
  • Create the project folder, virtual environment, and file structure.
  • Install dependencies and get your Gemini API key.
Install Python and VS Code

Python is the programming language you'll use throughout this project. You need version 3.10 or higher because the libraries we'll install require it.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or Command Prompt (Windows) and press Enter to open it.
  • Check your Python version by running this command:
python --version

✔️ I see version 3.10 or higher

You should see output like Python 3.10.x or Python 3.14.x. Any version 3.10 or above works perfectly.

Python is ready. Move on to installing VS Code below.

ⓧ I see an older version

If you see Python 3.9 or lower, you need to install a newer version.

  • Visit python.org/downloads and download the latest Python 3 installer for your operating system.
  • Run the installer.
  • On Windows, make sure to check Add python.exe to PATH before clicking Install Now.
  • Close and reopen your terminal, then run python --version again to confirm.

macOS users with an older default Python

macOS sometimes ships with an older Python that the python command points to. After installing a newer version from python.org, try python3 --version instead. Use python3 for all commands in this project if python still shows an older version.

ⓧ Command not found

Python isn't installed yet. Let's fix that.

  • Visit python.org/downloads and download the latest Python 3 installer for your operating system.
  • Run the installer.
  • On Windows, make sure to check Add python.exe to PATH before clicking Install Now.
  • Close and reopen your terminal, then run python --version to confirm.

macOS alternative

On macOS, try python3 --version instead. If that works, use python3 for all commands in this project.

Now let's open VS Code, the code editor you'll use to write the project.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.

VS Code not found?

If VS Code isn't installed, download it from code.visualstudio.com. Run the installer and try searching for it again.

Help me install VS Code

Create your project structure and virtual environment

A virtual environment is an isolated Python workspace. It keeps this project's packages separate from anything else on your computer, so nothing conflicts.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir rag-api
cd rag-api
  • Create a virtual environment by running:
python -m venv venv

macOS users

If python pointed to an older version in the previous step, use python3 -m venv venv here instead.

  • Activate the virtual environment:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the start of your terminal prompt. This confirms the virtual environment is active.

  • Create the project files by running:

macOS

touch main.py rag.py models.py requirements.txt

Windows

New-Item main.py, rag.py, models.py, requirements.txt
  • Confirm the files exist by running:
ls

You should see main.py, rag.py, models.py, requirements.txt, and the venv folder in the output.

  • Open the rag-api folder in VS Code by running:
code .

code command not found?

If the code command doesn't work, open VS Code manually. Click File in the top menu bar, then click Open Folder. Navigate to your Desktop and select the rag-api folder.

Help me open my project in VS Code

Install dependencies and get your Gemini API key

Your project depends on several Python packages for the API server, AI integration, and vector database. You'll list them in a requirements file and install them all at once.

  • In VS Code, click on requirements.txt in the Explorer sidebar to open it.
  • Paste the following package list into the file:
fastapi[standard]==0.136.3
google-genai==2.6.0
chromadb==1.5.9
pydantic==2.13.4
python-dotenv==1.1.0

What are these packages?

fastapi[standard] is the web framework for your API. google-genai is Google's SDK for calling the Gemini AI models.

chromadb is a vector database for storing and searching document embeddings. pydantic validates incoming data. python-dotenv loads your API key from a .env file.

  • Save requirements.txt by pressing Cmd+S (macOS) or Ctrl+S (Windows).
  • Install all the packages by running this command in your terminal:
pip install -r requirements.txt

This takes a minute or two. You should see packages downloading and installing, ending with a success message.

Installation failed?

Make sure your virtual environment is active. You should see (venv) at the start of your terminal prompt. If not, rerun the activation command from the previous substep.

On Windows, if ChromaDB fails to build, you may need Visual C++ Build Tools installed. On macOS, use pip3 if pip gives a command-not-found error.

Help me fix my pip install errors

Now you need a free API key from Google AI Studio to call the Gemini AI models.

  • Open Google AI Studio in your browser.
  • Sign in with your Google account.
  • Click Get API Key in the left sidebar.
  • Click Create API Key.
  • Copy the generated key. You will only see it once, so keep it handy.

Is this free?

Yes. The Gemini API free tier is generous enough for this entire project. You won't be charged for the calls you make during this walkthrough.

  • Back in VS Code, create a new file called .env by clicking File then New File.
  • Save it immediately as .env in your rag-api folder.
  • Paste the following into the file, replacing your-api-key-here with the key you just copied:
GEMINI_API_KEY=your-api-key-here

Why a .env file?

Storing your API key in a .env file keeps it out of your source code. Later, you'll add .env to a .gitignore file so it never gets pushed to GitHub.

  • Save .env by pressing Cmd+S (macOS) or Ctrl+S (Windows).

✔️ Awesome, I've got everything!

Great. Double check you've saved both requirements.txt and .env.

ⓧ I'd like to double check the full code

Here's what requirements.txt should contain:

fastapi[standard]==0.136.3
google-genai==2.6.0
chromadb==1.5.9
pydantic==2.13.4
python-dotenv==1.1.0

And your .env file should look like this (with your actual key in place of the placeholder):

GEMINI_API_KEY=your-api-key-here

Your development environment is ready. Next up, you'll define Pydantic models that validate every piece of data flowing through your API.

Define Pydantic Models

Your environment is set up and your dependencies are installed. Now it's time to define the data structures that control how your API communicates with the outside world.

Pydantic models act as gatekeepers for your API. They define exactly what data comes in, what goes out, and reject anything that doesn't fit your rules. In this step, you'll create models that validate document submissions and questions before they ever reach your RAG pipeline.

In this step, get ready to:
  • Define request models for document ingestion and question input.
  • Add field validators to enforce data quality rules.
  • Create response models for answers and document metadata.
Create the request and response models

Your API will accept two kinds of input: documents to ingest, and questions to answer. Each input type needs its own model that defines required fields and rejects bad data automatically.

  • In VS Code, open models.py from the rag-api folder by clicking it in the Explorer sidebar.
  • Add the imports and the DocumentInput model by pasting this code:
from datetime import datetime
from pydantic import BaseModel, field_validator


# Model for document ingestion requests
class DocumentInput(BaseModel):
    title: str
    text: str

    # Reject empty or whitespace-only document text
    @field_validator("text")
    @classmethod
    def text_must_not_be_empty(cls, v: str) -> str:
        if not v.strip():
            raise ValueError("Document text cannot be empty")
        return v

What does this code do?

The DocumentInput class defines what data the /ingest endpoint will accept. It requires a title string and a text string.

The @field_validator("text") decorator runs a check every time someone submits a document. If the text is empty or only whitespace, Pydantic automatically rejects the request with an error message before your code even runs.

  • You won't see visible output yet. The next chunk adds the question input model below this code.
  • Below the DocumentInput class, add the QuestionInput model by pasting this code:

# Model for question requests
class QuestionInput(BaseModel):
    question: str

    # Reject questions shorter than 10 characters
    @field_validator("question")
    @classmethod
    def question_min_length(cls, v: str) -> str:
        if len(v.strip()) < 10:
            raise ValueError("Question must be at least 10 characters")
        return v

What does this code do?

QuestionInput defines the shape of a question request. The validator checks that the question is at least 10 characters long after stripping whitespace.

This prevents users from submitting meaningless queries like "Hi" or "?" that would waste API calls to Gemini.

  • You won't see visible output yet. The next chunk adds the response model.
  • Below the QuestionInput class, add the AnswerResponse model by pasting this code:

# Model for structured answer responses
class AnswerResponse(BaseModel):
    answer: str
    source_chunks: list[str]
    num_sources: int

What does this code do?

AnswerResponse defines what your API sends back when answering a question. It includes the generated answer text, a list of source chunks that informed the answer, and the number of sources used.

This model guarantees every response has a consistent structure, so anyone consuming your API knows exactly what to expect.

  • Save models.py.
  • Confirm the models import correctly by running this command in your terminal:
python -c "from models import DocumentInput, QuestionInput, AnswerResponse; print('Models loaded successfully')"
  • You should see Models loaded successfully printed in the terminal.

Seeing an import error?

  • Make sure your terminal is in the rag-api folder. Run cd ~/Desktop/rag-api to get there.
  • Make sure your virtual environment is active. You should see (venv) at the start of your terminal prompt.
  • Check that your import line says from pydantic import BaseModel, field_validator with both names separated by a comma.

Help me fix my Pydantic model import error

Test the field validators

Your validators silently protect the API from bad input. Let's trigger them intentionally to see what happens when someone sends invalid data.

  • Test the QuestionInput validator by submitting a question that's too short. Run this command:
python -c "from models import QuestionInput; QuestionInput(question='Hi')"
  • You should see a ValidationError with the message Question must be at least 10 characters. This means your validator caught the invalid input.
  • Now test with a valid question to confirm it passes. Run this command:
python -c "from models import QuestionInput; print(QuestionInput(question='What is Python used for?'))"
  • You should see the model printed with the question field populated. No error means the validator accepted the input.

Why validate at the model level?

By putting validation rules directly in your Pydantic models, FastAPI automatically returns a clear error response to API callers. You never have to write manual if-else checks in your endpoint functions.

The validators run before your code processes the request. Invalid data gets rejected at the gate, keeping your RAG pipeline clean and predictable.

Not seeing a ValidationError?

  • Check that the @field_validator("question") decorator is directly above @classmethod in your QuestionInput class. The order of decorators matters.
  • Make sure the validator function is indented inside the class, not at the module level.

Help me fix my Pydantic field validator

Create the document metadata model

Your API needs one more model to track information about documents that have been ingested. This model uses Python's datetime module to record when each document was processed.

  • Below the AnswerResponse class in models.py, add the DocumentMetadata model by pasting this code:

# Model for tracking ingested document information
class DocumentMetadata(BaseModel):
    title: str
    chunk_count: int
    ingested_at: datetime

What does this code do?

DocumentMetadata stores information about each document your API processes. It tracks the document title, how many chunks it was split into, and the timestamp of when it was ingested.

The ingested_at field uses the datetime type from Python's standard library, which you already imported at the top of the file.

  • Save models.py.
  • Confirm all four models import correctly by running this command:
python -c "from models import DocumentInput, QuestionInput, AnswerResponse, DocumentMetadata; print('All 4 models loaded')"
  • You should see All 4 models loaded printed in the terminal.

Seeing a NameError for datetime?

  • Check that the very first line of models.py says from datetime import datetime. This import is what makes the datetime type available for DocumentMetadata.
  • Make sure you haven't accidentally deleted or modified the import lines at the top of the file.

Help me fix my datetime import issue

✔️ Awesome, I've got everything!

Great. Double check you've saved models.py and move on to the next step.

ⓧ I'd like to double check the full code

Here's the complete models.py file for comparison:

from datetime import datetime
from pydantic import BaseModel, field_validator


class DocumentInput(BaseModel):
    title: str
    text: str

    @field_validator("text")
    @classmethod
    def text_must_not_be_empty(cls, v: str) -> str:
        if not v.strip():
            raise ValueError("Document text cannot be empty")
        return v


class QuestionInput(BaseModel):
    question: str

    @field_validator("question")
    @classmethod
    def question_min_length(cls, v: str) -> str:
        if len(v.strip()) < 10:
            raise ValueError("Question must be at least 10 characters")
        return v


class AnswerResponse(BaseModel):
    answer: str
    source_chunks: list[str]
    num_sources: int


class DocumentMetadata(BaseModel):
    title: str
    chunk_count: int
    ingested_at: datetime

Your data models are locked in. Next up, you'll build the RAG pipeline that chunks documents, embeds them with Gemini, and stores them in ChromaDB for semantic search.

Build the RAG Pipeline

Your Pydantic models are locked in. Now it's time to build the engine that powers the entire Q&A system.

In this step, you'll create the RAG pipeline. This is the sequence of operations that takes raw document text, splits it into searchable pieces, converts those pieces into numerical representations (embeddings), and stores everything in a vector database for fast retrieval.

In this step, get ready to:
  • Write a text chunking function that splits documents into overlapping pieces.
  • Connect to Google's Gemini API to generate vector embeddings.
  • Store chunks and embeddings in a ChromaDB vector database.
Write the text chunking function

Before you can search a document, you need to break it into smaller pieces. Large language models have token limits, and searching smaller chunks gives more precise answers than searching entire documents at once.

  • In VS Code, open rag.py from the Explorer sidebar.
  • Add the imports and initialization code by pasting the following at the top of the file:
import os
from datetime import datetime

import chromadb
from google import genai
from google.genai import types
from dotenv import load_dotenv

from models import DocumentMetadata

# Load environment variables from .env file
load_dotenv()

# Initialize the Gemini client with your API key
client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))

# Create an in-memory ChromaDB client and collection
chroma_client = chromadb.Client()
collection = chroma_client.get_or_create_collection(name="documents")

# Track metadata for all ingested documents
ingested_documents: list[DocumentMetadata] = []

What does this code do?

This sets up the two core services your RAG pipeline depends on.

  • load_dotenv() reads your .env file so GEMINI_API_KEY is available via os.getenv().
  • genai.Client() creates a connection to the Gemini API for both embeddings and text generation.
  • chromadb.Client() starts a ChromaDB server in memory. No external database needed.
  • get_or_create_collection() creates a named collection to hold your document chunks and embeddings.
  • ingested_documents is a Python list that tracks metadata about every document you ingest.

You won't see a visible change yet. The next chunk adds the first function that uses these clients.

  • Now add the chunk_text() function below the ingested_documents line in rag.py:

def chunk_text(text: str, chunk_size: int = 500, overlap: int = 50) -> list[str]:
    # Split text into overlapping chunks for better search coverage
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        # Move forward by (chunk_size - overlap) so chunks share some text
        start = end - overlap
    return chunks

What does this code do?

chunk_text() uses a sliding window to split a document into 500-character pieces. Each piece overlaps the previous one by 50 characters.

The overlap ensures that sentences split across chunk boundaries are still partially captured in both chunks. This improves search accuracy when a user's question matches text near a boundary.

  • Save rag.py.
  • Verify the chunking logic works by running this in your terminal:
python -c "from rag import chunk_text; result = chunk_text('A' * 1000); print(f'{len(result)} chunks created')"

You should see 3 chunks created in the output. A 1000-character string split into 500-character chunks with 50-character overlap produces exactly 3 chunks.

Seeing a ModuleNotFoundError?

  • Make sure your virtual environment is activated. You should see (venv) at the start of your terminal prompt.
  • Check that you saved rag.py before running the command. Unsaved changes won't be picked up.
  • If you see an error about chromadb or google.genai, re-run pip install -r requirements.txt with your venv activated.

Help me fix my ModuleNotFoundError in rag.py

Connect to Gemini for embeddings

Embeddings are numerical representations of text. They let you compare how similar two pieces of text are by measuring the distance between their vectors. Your chunking function produces text. Now you need a function that converts that text into vectors.

  • Add the embed_chunks() function below chunk_text() in rag.py:

def embed_chunks(chunks: list[str]) -> list[list[float]]:
    # Generate a vector embedding for each text chunk using Gemini
    embeddings = []
    for chunk in chunks:
        response = client.models.embed_content(
            model="gemini-embedding-2",
            contents=chunk,
            config=types.EmbedContentConfig(
                task_type="RETRIEVAL_DOCUMENT",
            ),
        )
        # Extract the embedding values from the response
        embeddings.append(response.embeddings[0].values)
    return embeddings

What does this code do?

embed_chunks() loops through each text chunk and calls Gemini's embedding API to convert it into a list of floating-point numbers (a vector).

  • gemini-embedding-2 is Google's latest embedding model. It turns text into high-dimensional vectors optimized for similarity search.
  • RETRIEVAL_DOCUMENT tells Gemini that these embeddings represent stored documents (as opposed to search queries). This task type helps the model optimize the vectors for retrieval use cases.

Why use explicit embeddings instead of ChromaDB's built-in option?

ChromaDB can generate embeddings automatically, but doing it explicitly with Gemini teaches you what's happening under the hood. You control the model, the task type, and the vector output.

This pattern is transferable to any vector database or embedding provider. You're learning the concept, not just the shortcut.

  • Save rag.py.

You won't see a change yet. The embedding function needs actual chunks to process, which happens when the full pipeline runs in the next step.

Store embeddings in ChromaDB

You now have functions to chunk text and generate embeddings. The final piece is storing everything in ChromaDB so you can search it later.

  • Add the store_in_chromadb() function below embed_chunks() in rag.py:

def store_in_chromadb(
    chunks: list[str], embeddings: list[list[float]], title: str
) -> int:
    # Create unique IDs for each chunk based on document title
    ids = [f"{title}_{i}" for i in range(len(chunks))]
    # Attach metadata to each chunk for filtering later
    metadatas = [{"title": title, "chunk_index": i} for i in range(len(chunks))]

    # Add chunks, embeddings, and metadata to the ChromaDB collection
    collection.add(
        ids=ids,
        documents=chunks,
        embeddings=embeddings,
        metadatas=metadatas,
    )

    # Track this document's metadata in our module-level list
    ingested_documents.append(
        DocumentMetadata(
            title=title,
            chunk_count=len(chunks),
            ingested_at=datetime.now(),
        )
    )

    return len(chunks)

What does this code do?

store_in_chromadb() takes the chunks, their embeddings, and the document title, then stores everything in the ChromaDB collection.

  • Each chunk gets a unique ID like "My Document_0", "My Document_1", etc.
  • collection.add() stores the text, its vector embedding, and metadata together. ChromaDB indexes the vectors for fast similarity search.
  • The function also appends a DocumentMetadata entry to ingested_documents so you can track what's been stored.
  • Save rag.py.
  • Verify the full module loads without errors by running:
python -c "import rag; print('RAG module loaded successfully')"

You should see RAG module loaded successfully in the output. This confirms all imports resolve, the Gemini client initializes, and ChromaDB starts its in-memory server.

Seeing an API key error?

  • Check that your .env file contains GEMINI_API_KEY= followed by your actual key (no quotes around the value).
  • Make sure the .env file is in the same folder as rag.py (your rag-api folder).
  • If you see a Failed to build hnswlib error on Windows, install the latest version of Python 3.10+ and try again.

Help me fix my RAG module import error

✔️ Awesome, I've got everything!

Great. Double check you've saved rag.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete rag.py file at the end of this step:

import os
from datetime import datetime

import chromadb
from google import genai
from google.genai import types
from dotenv import load_dotenv

from models import DocumentMetadata

load_dotenv()

client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))
chroma_client = chromadb.Client()
collection = chroma_client.get_or_create_collection(name="documents")

ingested_documents: list[DocumentMetadata] = []


def chunk_text(text: str, chunk_size: int = 500, overlap: int = 50) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks


def embed_chunks(chunks: list[str]) -> list[list[float]]:
    embeddings = []
    for chunk in chunks:
        response = client.models.embed_content(
            model="gemini-embedding-2",
            contents=chunk,
            config=types.EmbedContentConfig(
                task_type="RETRIEVAL_DOCUMENT",
            ),
        )
        embeddings.append(response.embeddings[0].values)
    return embeddings


def store_in_chromadb(
    chunks: list[str], embeddings: list[list[float]], title: str
) -> int:
    ids = [f"{title}_{i}" for i in range(len(chunks))]
    metadatas = [{"title": title, "chunk_index": i} for i in range(len(chunks))]

    collection.add(
        ids=ids,
        documents=chunks,
        embeddings=embeddings,
        metadatas=metadatas,
    )

    ingested_documents.append(
        DocumentMetadata(
            title=title,
            chunk_count=len(chunks),
            ingested_at=datetime.now(),
        )
    )

    return len(chunks)

Your RAG pipeline's foundation is in place. You have chunking, embedding, and storage all wired up. Next up, you'll build the question-answering side that searches these stored chunks and generates answers with Gemini.

Build the Question-Answering Function

Your RAG pipeline can now chunk documents and store their embeddings in ChromaDB. The next piece is the question-answering side.

When a user asks a question, the system needs to find the most relevant chunks and use them to generate an accurate answer. In this step, you'll build the functions that handle search, answer generation, and orchestration.

In this step, get ready to:
  • Implement semantic search to find relevant document chunks.
  • Generate natural language answers using Gemini and retrieved context.
  • Wire everything together into a single orchestrator function.
Implement semantic search

To answer questions accurately, the system needs to find which document chunks are most relevant to the user's question. This is where vector search comes in.

The idea is to embed the question into the same vector space as the stored chunks, then ask ChromaDB to find the nearest neighbors. The closest chunks are the most semantically similar to the question.

  • In rag.py, find the store_in_chromadb() function and add the following new function below it:
def search_relevant_chunks(question: str, n_results: int = 3) -> list[str]:
    query_embedding_response = client.models.embed_content(
        model="gemini-embedding-2",
        contents=question,
        config=types.EmbedContentConfig(
            task_type="RETRIEVAL_QUERY",
        ),
    )
    query_embedding = query_embedding_response.embeddings[0].values

    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=n_results,
    )

    return results["documents"][0] if results["documents"] else []

What does this code do?

This function takes the user's question and converts it into an embedding vector, just like you did with the document chunks earlier.

  • task_type="RETRIEVAL_QUERY" tells Gemini to optimize the embedding for searching (as opposed to RETRIEVAL_DOCUMENT which optimizes for being found).
  • collection.query() finds the 3 chunks whose embeddings are closest to the question embedding.
  • The function returns an empty list if no documents have been ingested yet.
  • Save rag.py.
  • You won't see a change yet. The next function will use these search results to generate an answer.

Seeing a syntax error on save?

  • Make sure the function is at the top level of the file (not indented inside another function).
  • Check that there are two blank lines between the end of store_in_chromadb() and the start of this new function. Python convention uses two blank lines between top-level definitions.
  • Verify the closing parentheses match. The embed_content() call has nested parentheses for EmbedContentConfig.

Why is my search function showing a syntax error?

Generate answers with Gemini

Finding relevant chunks is only half the job. The system also needs to synthesize those chunks into a coherent, natural language answer. This is where the "Generation" in RAG happens.

  • In rag.py, add the following function below search_relevant_chunks():
def generate_answer(question: str, context_chunks: list[str]) -> str:
    context = "\n\n---\n\n".join(context_chunks)
    prompt = f"""Based on the following context, answer the question. 
If the context doesn't contain enough information, say so.

Context:
{context}

Question: {question}

Answer:"""

    response = client.models.generate_content(
        model="gemini-3.5-flash",
        contents=prompt,
    )
    return response.text

What does this code do?

  • context = "\n\n---\n\n".join(context_chunks) joins the retrieved chunks into a single string, separated by divider lines. This gives the model clear boundaries between different pieces of context.
  • The prompt uses an f-string template that places the context and question into a structured format. The instruction tells Gemini to admit when it doesn't have enough information, rather than making things up.
  • client.models.generate_content() calls the gemini-3.5-flash model with the prompt and returns generated text.

Why RETRIEVAL_QUERY vs RETRIEVAL_DOCUMENT?

Gemini's embedding model supports different task types that optimize how the embedding is generated. When you stored document chunks, you used RETRIEVAL_DOCUMENT to tell the model "this is content to be found."

When embedding a question, you use RETRIEVAL_QUERY to tell the model "this is a search query looking for content." The model adjusts its embedding strategy for each role, improving search accuracy.

  • Save rag.py.
  • You won't see a change yet. The next function ties everything together into a single callable pipeline.

Seeing an indentation error in the f-string?

  • Triple-quoted f-strings (f"""...""") are sensitive to indentation. Make sure the closing """ is on the same line as Answer: with no extra spaces.
  • Check that the curly braces {context} and {question} are not escaped. They need to be plain curly braces for the f-string substitution to work.

Help me fix my generate_answer f-string prompt

Wire the pipeline together

Now you have two separate functions: one to search and one to generate. The final function orchestrates them into a single call that handles edge cases and returns a structured response.

  • In rag.py, add the following function below generate_answer():
def ask_question(question: str) -> dict:
    relevant_chunks = search_relevant_chunks(question)

    if not relevant_chunks:
        return {
            "answer": "No relevant documents found. Please ingest documents first.",
            "source_chunks": [],
            "num_sources": 0,
        }

    answer = generate_answer(question, relevant_chunks)

    return {
        "answer": answer,
        "source_chunks": relevant_chunks,
        "num_sources": len(relevant_chunks),
    }

What does this code do?

  • search_relevant_chunks(question) finds the top matching chunks from ChromaDB.
  • The if not relevant_chunks check handles the case where no documents have been ingested. Instead of crashing, it returns a helpful message.
  • The returned dictionary matches the shape of your AnswerResponse Pydantic model. FastAPI will use this model to validate and serialize the response in the next step.
  • Save rag.py.
  • Verify that all your functions load without errors by running this command in your terminal:
python -c "from rag import ask_question; print('All functions loaded successfully')"
  • You should see All functions loaded successfully printed in your terminal. This confirms Python can parse and import the entire file without syntax errors.

Seeing an import error or ModuleNotFoundError?

  • Make sure your virtual environment is still activated. You should see (venv) at the start of your terminal prompt.
  • Check that you're running the command from inside the rag-api folder, not a parent directory.
  • If you see a SyntaxError, the error message will point to a line number. Check that line in rag.py for missing colons, unclosed parentheses, or indentation issues.

Help me debug my rag.py import error

✔️ Awesome, I've got everything!

Great. Make sure you've saved rag.py and the import ran without errors.

ⓧ I'd like to double check the full code

Here's the complete rag.py file with all functions. Compare yours against this:

import os
from datetime import datetime

import chromadb
from google import genai
from google.genai import types
from dotenv import load_dotenv

from models import DocumentMetadata

load_dotenv()

client = genai.Client(api_key=os.getenv("GEMINI_API_KEY"))
chroma_client = chromadb.Client()
collection = chroma_client.get_or_create_collection(name="documents")

ingested_documents: list[DocumentMetadata] = []


def chunk_text(text: str, chunk_size: int = 500, overlap: int = 50) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks


def embed_chunks(chunks: list[str]) -> list[list[float]]:
    embeddings = []
    for chunk in chunks:
        response = client.models.embed_content(
            model="gemini-embedding-2",
            contents=chunk,
            config=types.EmbedContentConfig(
                task_type="RETRIEVAL_DOCUMENT",
            ),
        )
        embeddings.append(response.embeddings[0].values)
    return embeddings


def store_in_chromadb(
    chunks: list[str], embeddings: list[list[float]], title: str
) -> int:
    ids = [f"{title}_{i}" for i in range(len(chunks))]
    metadatas = [{"title": title, "chunk_index": i} for i in range(len(chunks))]

    collection.add(
        ids=ids,
        documents=chunks,
        embeddings=embeddings,
        metadatas=metadatas,
    )

    ingested_documents.append(
        DocumentMetadata(
            title=title,
            chunk_count=len(chunks),
            ingested_at=datetime.now(),
        )
    )

    return len(chunks)


def search_relevant_chunks(question: str, n_results: int = 3) -> list[str]:
    query_embedding_response = client.models.embed_content(
        model="gemini-embedding-2",
        contents=question,
        config=types.EmbedContentConfig(
            task_type="RETRIEVAL_QUERY",
        ),
    )
    query_embedding = query_embedding_response.embeddings[0].values

    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=n_results,
    )

    return results["documents"][0] if results["documents"] else []


def generate_answer(question: str, context_chunks: list[str]) -> str:
    context = "\n\n---\n\n".join(context_chunks)
    prompt = f"""Based on the following context, answer the question. 
If the context doesn't contain enough information, say so.

Context:
{context}

Question: {question}

Answer:"""

    response = client.models.generate_content(
        model="gemini-3.5-flash",
        contents=prompt,
    )
    return response.text


def ask_question(question: str) -> dict:
    relevant_chunks = search_relevant_chunks(question)

    if not relevant_chunks:
        return {
            "answer": "No relevant documents found. Please ingest documents first.",
            "source_chunks": [],
            "num_sources": 0,
        }

    answer = generate_answer(question, relevant_chunks)

    return {
        "answer": answer,
        "source_chunks": relevant_chunks,
        "num_sources": len(relevant_chunks),
    }

Your RAG pipeline is complete. Next up, you'll expose it through FastAPI endpoints so users can interact with it via HTTP requests.

Create the FastAPI Application

Your RAG pipeline is complete. You have functions that chunk text, embed it, store it in ChromaDB, and answer questions using retrieved context.

The missing piece is a way for users to actually interact with this pipeline. Right now, your functions live in a Python file with no entry point. In this step, you will wrap them in a FastAPI web server that exposes two endpoints: one to ingest documents and one to ask questions.

In this step, get ready to:
  • Create a FastAPI application with a document ingestion endpoint.
  • Add a question-answering endpoint that returns validated responses.
  • Run and test both endpoints using the auto-generated Swagger UI.
Set up the FastAPI app with routes

FastAPI uses a central application object that registers all your routes. You also need to import the Pydantic models and RAG functions you built in earlier steps.

  • In VS Code, open main.py and add the imports and app instance by pasting this code:
from fastapi import FastAPI, HTTPException

from models import DocumentInput, QuestionInput, AnswerResponse, DocumentMetadata
from rag import chunk_text, embed_chunks, store_in_chromadb, ask_question, ingested_documents

# Create the FastAPI application with metadata
app = FastAPI(
    title="RAG Q&A API",
    description="A document Q&A API powered by Retrieval-Augmented Generation",
)

What does this code do?

  • The first import brings in FastAPI (the framework) and HTTPException (for returning error responses with status codes).
  • The second and third imports pull in your Pydantic models and RAG pipeline functions from the files you already wrote.
  • The FastAPI() call creates your application instance. The title and description show up in the auto-generated documentation.

Now you need an endpoint that accepts a document, processes it through your RAG pipeline, and confirms success.

  • Below the app = FastAPI(...) block, add the ingestion endpoint by pasting this code:

@app.post("/ingest", status_code=201)
def ingest_document(document: DocumentInput) -> dict:
    # Run the full RAG ingestion pipeline
    chunks = chunk_text(document.text)
    embeddings = embed_chunks(chunks)
    chunk_count = store_in_chromadb(chunks, embeddings, document.title)

    return {
        "message": f"Document '{document.title}' ingested successfully",
        "chunks_created": chunk_count,
    }

What does this code do?

  • The @app.post("/ingest", status_code=201) decorator registers this function as a POST endpoint at /ingest. The 201 status code means "Created" instead of the default 200.
  • FastAPI automatically parses the request body into a DocumentInput object and validates it using the Pydantic model you defined earlier.
  • The function calls your three pipeline functions in sequence: chunk the text, embed the chunks, then store everything in ChromaDB.
  • It returns a dictionary with a success message and the number of chunks created. FastAPI converts this to JSON automatically.
  • Save main.py.

Seeing red underlines in VS Code?

If your editor shows import errors for models or rag, make sure your virtual environment is selected as the Python interpreter. Click the Python version in the bottom-left of VS Code and select the one inside your venv folder.

help me fix import errors in my FastAPI project

Add the question endpoint

The second endpoint accepts a question, runs it through semantic search and answer generation, then returns a structured response validated by your Pydantic model.

  • Below the ingest_document function, add the question-answering endpoint by pasting this code:

@app.post("/ask", response_model=AnswerResponse)
def ask(question_input: QuestionInput) -> AnswerResponse:
    # Call the RAG pipeline and return a validated response
    result = ask_question(question_input.question)
    return AnswerResponse(**result)

What does this code do?

  • The response_model=AnswerResponse parameter tells FastAPI to validate the outgoing response against your Pydantic model. This also generates the response schema in the API documentation.
  • The ask_question() function returns a dictionary with answer, source_chunks, and num_sources keys.
  • The AnswerResponse(**result) unpacks that dictionary into the Pydantic model, which validates the data before returning it as JSON.
  • Save main.py.

Seeing a syntax error?

Check that you have a blank line between the end of the ingest_document function and the @app.post("/ask", ...) decorator. Python requires blank lines between top-level function definitions.

help me fix a syntax error in my FastAPI main.py file

✔️ Awesome, I've got everything!

Great. Double check you have saved main.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete main.py at this point:

from fastapi import FastAPI, HTTPException

from models import DocumentInput, QuestionInput, AnswerResponse, DocumentMetadata
from rag import chunk_text, embed_chunks, store_in_chromadb, ask_question, ingested_documents

app = FastAPI(
    title="RAG Q&A API",
    description="A document Q&A API powered by Retrieval-Augmented Generation",
)


@app.post("/ingest", status_code=201)
def ingest_document(document: DocumentInput) -> dict:
    chunks = chunk_text(document.text)
    embeddings = embed_chunks(chunks)
    chunk_count = store_in_chromadb(chunks, embeddings, document.title)

    return {
        "message": f"Document '{document.title}' ingested successfully",
        "chunks_created": chunk_count,
    }


@app.post("/ask", response_model=AnswerResponse)
def ask(question_input: QuestionInput) -> AnswerResponse:
    result = ask_question(question_input.question)
    return AnswerResponse(**result)
Run and test the API

FastAPI includes a built-in development server and auto-generates interactive documentation called Swagger UI. You can test your endpoints directly in the browser without writing any client code.

  • Start the development server by running this command in your terminal:
fastapi dev main.py

What does this command do?

The fastapi dev command starts a development server with auto-reload enabled. Every time you save a file, the server restarts automatically. This command is available because you installed fastapi[standard] which bundles the FastAPI CLI.

  • You should see output in your terminal showing the server is running at http://127.0.0.1:8000.
  • Open your browser and navigate to http://127.0.0.1:8000/docs to see the Swagger UI.

What is Swagger UI?

Swagger UI is an interactive API documentation page that FastAPI generates automatically from your code. It reads your endpoint decorators, Pydantic models, and type hints to build a complete, testable interface. You can send real requests to your API directly from this page.

You should see your two endpoints listed: POST /ingest and POST /ask.

  • Click on the POST /ingest endpoint to expand it.
  • Click Try it out.
  • Replace the example request body with this sample document:
{
  "title": "Python Basics",
  "text": "Python is a high-level programming language known for its readability and simplicity. It supports multiple programming paradigms including procedural, object-oriented, and functional programming. Python uses indentation to define code blocks instead of curly braces. Variables in Python are dynamically typed, meaning you do not need to declare their type explicitly. Python has a large standard library that provides modules for file I/O, networking, regular expressions, and more."
}
  • Click Execute.
  • You should see a 201 response with a success message and the number of chunks created.

First request takes a moment

The first call to /ingest will take a few seconds because it calls the Gemini API to generate embeddings. Subsequent calls are faster.

  • Now click on the POST /ask endpoint to expand it.
  • Click Try it out.
  • Replace the example request body with this question:
{
  "question": "What programming paradigms does Python support?"
}
  • Click Execute.
  • You should see a 200 response containing an answer field, a source_chunks array, and a num_sources count.

Getting a 500 Internal Server Error?

Check that your .env file has a valid Gemini API key set for GEMINI_API_KEY. Also make sure you ingested a document with /ingest before trying to ask a question with /ask.

help me debug a 500 error from my RAG FastAPI application

Your API is live and responding to requests. Next up, you will push this project to GitHub so it lives in your portfolio as a complete, documented codebase.

Push to GitHub

Your RAG API is fully functional. You tested both endpoints and confirmed the full pipeline works from document ingestion to question answering.

Now it's time to share your work. In this step, you'll push your project to GitHub so it lives as a portfolio piece anyone can discover and run.

In this step, get ready to:
  • Initialize a Git repository with a .gitignore to protect sensitive files.
  • Push your project to GitHub with a descriptive README.
Initialize Git and create .gitignore

Git is a version control system that tracks every change you make to your code. Before you can push to GitHub, you need a local Git repository.

  • Check if Git is installed by running this command in your terminal:
git --version

✔️ I see a version number

Git is ready to go. Move on to the next instruction.

ⓧ Command not found

You need to install Git. Follow the instructions for your operating system.

macOS

  • Install Git via Xcode Command Line Tools by running:
xcode-select --install
  • A dialog will appear asking you to install the tools. Click Install and wait for it to finish.
  • Once complete, run git --version again to confirm.

Windows

  • Download the Git for Windows installer from https://git-scm.com/download/win.
  • Run the installer and accept the default settings.
  • Once complete, close and reopen your terminal, then run git --version again to confirm.
  • If your FastAPI dev server is still running, stop it by pressing Ctrl+C in the terminal.
  • Make sure you are inside the rag-api folder, then initialize a Git repository by running:
git init

What does this command do?

git init creates a hidden .git folder inside your project. This folder is where Git stores the entire history of your code changes.

You should see a message confirming the repository was initialized.

Before committing, you need a .gitignore file. This tells Git which files to exclude from version control.

  • Create a .gitignore file in your rag-api folder and add the following content:
venv/
.env
__pycache__/

Why exclude these files?

venv/ contains thousands of installed package files that anyone can recreate with pip install. Pushing it would bloat your repository.

.env holds your secret API key. You never want credentials in a public repository.

__pycache__/ stores temporary compiled Python files that are auto-generated at runtime.

  • Save .gitignore.
  • Stage all your project files and make the initial commit by running these commands:
git add .
git commit -m "Initial commit"

What do these commands do?

git add . stages every file in the current folder (except those listed in .gitignore) for the next commit.

git commit -m "Initial commit" saves a snapshot of all staged files with a descriptive message.

You should see output listing the files that were committed.

Seeing a message about user identity?

Git may ask you to set your name and email before committing. Run these two commands with your own details:

git config --global user.name "Your Name"
git config --global user.email "your@email.com"

Then retry the git commit command.

help me fix a Git identity error when committing

Create a GitHub repository and push

Your code is committed locally. Now you need a remote repository on GitHub so others can see your work.

  • Create a README.md file in your rag-api folder and add the following content:
# RAG Q&A API

A document Q&A API powered by Retrieval-Augmented Generation using FastAPI, Google Gemini, and ChromaDB.

## Features
- POST /ingest: Upload and embed documents
- POST /ask: Ask questions about ingested documents

## How to Run
1. Clone the repo
2. Create and activate a virtual environment
3. Run `pip install -r requirements.txt`
4. Add your Gemini API key to `.env`
5. Run `fastapi dev main.py`
6. Open http://127.0.0.1:8000/docs

Why write a README?

A README is the first thing anyone sees when they visit your repository. It tells them what your project does and how to run it. This makes your project portfolio-ready.

  • Save README.md.
  • Stage and commit the README by running:
git add README.md
git commit -m "Add README"
  • Open your browser and go to https://github.com.
  • Sign in to your GitHub account. If you do not have one, create a free account.
  • Click the + icon in the top-right corner, then select New repository.
  • Enter rag-api as the repository name.
  • Leave the repository set to Public.
  • Do not check any of the initialization options (no README, no .gitignore, no license). You already have these locally.
  • Click Create repository.

GitHub will show a page with setup instructions. You need the HTTPS URL it displays.

  • Back in your terminal, connect your local repository to GitHub and push your code by running:
git remote add origin [[REPO_URL="https://github.com/your-username/rag-api.git"]]
git branch -M main
git push -u origin main

What do these commands do?

git remote add origin links your local repository to the remote GitHub repository.

git branch -M main renames your default branch to main.

git push -u origin main uploads all your commits to GitHub.

  • Refresh your GitHub repository page in the browser. You should see all your project files listed along with your README displayed below them.

Push failed or asking for credentials?

GitHub no longer accepts passwords for HTTPS pushes. If prompted for credentials, you need a personal access token or SSH key.

  • Go to Settings > Developer settings > Personal access tokens in your GitHub account to generate a token.
  • Use the token as your password when prompted.

help me push my code to GitHub

✔️ Awesome, I've got everything!

Great. Double check you've saved all your files and your GitHub repository shows the project code.

ⓧ I'd like to double check the full code

Here is your .gitignore file:

venv/
.env
__pycache__/

Here is your README.md file:

# RAG Q&A API

A document Q&A API powered by Retrieval-Augmented Generation using FastAPI, Google Gemini, and ChromaDB.

## Features
- POST /ingest: Upload and embed documents
- POST /ask: Ask questions about ingested documents

## How to Run
1. Clone the repo
2. Create and activate a virtual environment
3. Run `pip install -r requirements.txt`
4. Add your Gemini API key to `.env`
5. Run `fastapi dev main.py`
6. Open http://127.0.0.1:8000/docs

Secret mission

Add a /documents GET Endpoint That Lists All Ingested Documents

Your API can ingest documents and answer questions. But what if you forget which documents you've uploaded? Extend your API with a new endpoint that returns metadata about all ingested documents, plus a lookup endpoint for individual documents.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally with the Gemini API free tier, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • rag-api project folder on your Desktop (includes virtual environment, .env file, and all project code).
  • GitHub repository for the project.
  • Gemini API key in Google AI Studio.

Keep everything running

No action needed. Choose this if you're still actively building or want to revisit the project later.

  • Your rag-api folder stays on your Desktop with all code intact.
  • Your Gemini API key remains active in Google AI Studio for future use.
  • Your GitHub repository stays public with your project code.

Pause - I'll come back to this later

Stop any running processes to free up system resources. Your files and accounts stay intact.

  • If the FastAPI dev server is still running, press Ctrl+C in the terminal to stop it.
  • Deactivate your virtual environment by running:
deactivate
  • When you come back, navigate to your rag-api folder, reactivate the virtual environment, and start the server again.

Delete - I don't want to use this again

Remove all project resources and start fresh. This cannot be undone.

  • Stop the FastAPI dev server if it is still running by pressing Ctrl+C in the terminal.
  • Deactivate the virtual environment by running:
deactivate
  • Delete the project folder by running the command for your operating system.

On macOS/Linux:

cd ~/Desktop
rm -rf rag-api

On Windows:

cd ~\Desktop
rmdir /s /q rag-api
  • Delete your GitHub repository by navigating to the repository page on GitHub.
  • Click Settings in the top navigation bar.
  • Scroll to the bottom of the page to the Danger Zone section.
  • Click Delete this repository and follow the confirmation prompts.
  • Delete your Gemini API key by navigating to Google AI Studio.
  • Open the API keys page and delete the key you created for this project.

Nice Work!

Nice Work!

You just built a complete RAG-powered Q&A API from scratch. Your application ingests documents, embeds them with Gemini, stores them in ChromaDB, and answers natural language questions using retrieved context.

Here's what you learned:

  • Built a FastAPI application with POST endpoints for document ingestion and question answering, validated with Pydantic models.
  • Implemented a full RAG pipeline: text chunking, vector embedding with Gemini, semantic search with ChromaDB, and AI-generated answers.
  • Pushed a documented, portfolio-ready project to GitHub with a clean project structure and a README.
  • Added a document listing feature with GET endpoints and proper error handling using HTTP 404 responses.

Ready to quiz yourself?