Build a RAG Backend with LlamaIndex

Ingest course PDFs into ChromaDB and query them with Gemini 2.5 Flash.

Introduction

30 Second Summary

Think about how much time students spend re-reading entire textbooks just to find one specific answer. What if your course materials could answer questions for you, pulling the exact relevant passage instantly?

In this project, you will build a backend API that ingests PDF course materials, stores them as searchable vectors, and answers natural language questions grounded exclusively in your uploaded content.

What You'll Build

A Python FastAPI backend with a Retrieval-Augmented Generation (RAG) pipeline that chunks PDFs, embeds them locally, stores vectors in ChromaDB, and uses Gemini 2.5 Flash to generate answers from retrieved context.

By the end of this project, you'll have:

  • A FastAPI application with /ingest and /query endpoints that accept PDF uploads and answer questions about your course materials.
  • A ChromaDB vector database running in Docker with persistent storage for your embedded course content.
  • A working RAG pipeline using LlamaIndex that connects document retrieval to Gemini 2.5 Flash for grounded, hallucination-free answers.
  • Classroom-based metadata organization so each subject's materials are isolated and queryable independently.
  • Secret Mission: Add subject-aware smart retrieval with semester filtering and a classrooms listing endpoint.

Are there any prerequisites?

You should have basic Python knowledge and be comfortable running terminal commands. Your environment should already have Python, Docker, and VS Code installed.

You will also need a Google AI API key for Gemini. Step 1 walks you through getting one for free.

Set Up Your Environment

Before you can build a RAG pipeline, you need the tools that make it work. Your backend depends on three things: an AI model to generate answers, a vector database to store document embeddings, and a Python framework to tie them together.

In this step, you will get your Gemini API key, create a Python project with all its dependencies, and start a ChromaDB instance in Docker.

In this step, get ready to:
  • Get your Google AI API key for Gemini 2.5 Flash.
  • Create your project folder and install Python dependencies.
  • Start ChromaDB in Docker with persistent storage.
Set up your Google AI API key

Your RAG backend uses Gemini 2.5 Flash to generate answers from retrieved documents. To call the Gemini API, you need an API key from Google AI Studio.

  • Open aistudio.google.com in your browser and sign in with your Google account.
  • Click Get API key in the left sidebar.
  • Click Create API key and copy the key that appears.
  • Save this key somewhere safe. You will use it in a moment when creating your project files.

What is an API key?

An API key is a unique string that identifies your application to Google's servers. It lets you make requests to the Gemini model without needing a full authentication flow.

The free tier includes free input and output tokens for Gemini 2.5 Flash, with rate limits applied. This means the project costs nothing to run during testing.

Create your project and install dependencies

Your project needs FastAPI for the web server, LlamaIndex for the RAG pipeline, and several supporting packages. A virtual environment keeps these dependencies isolated from the rest of your system.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir study-rag-api
cd study-rag-api
  • Create a Python virtual environment by running:
python -m venv venv

What is a virtual environment?

A virtual environment is an isolated Python installation inside your project folder. Packages you install here stay separate from your system Python, preventing version conflicts between projects.

  • Activate the virtual environment:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt.

  • Install all required packages by running this command:
pip install fastapi==0.138.0 uvicorn==0.34.0 python-dotenv==1.1.0 llama-index==0.14.22 llama-index-vector-stores-chroma==0.5.5 llama-index-llms-google-genai==0.9.5 llama-index-embeddings-huggingface==0.7.0 llama-index-readers-file==0.6.0 chromadb==1.5.9 python-multipart==0.0.20

What are these packages?

  • fastapi and uvicorn power your REST API server.
  • llama-index is the core RAG framework that handles document chunking, embedding, and retrieval.
  • llama-index-vector-stores-chroma connects LlamaIndex to ChromaDB.
  • llama-index-llms-google-genai connects LlamaIndex to Gemini for answer generation.
  • llama-index-embeddings-huggingface runs a local embedding model so you pay nothing for vector creation.
  • chromadb is the Python client that talks to your ChromaDB Docker container.

The install may take a few minutes as it downloads the packages and their dependencies.

Seeing permission errors or install failures?

Make sure you see (venv) in your terminal prompt. If you do not, run the activate command again.

If a package fails to build, confirm your Python version is 3.10 or higher by running python --version.

Help me troubleshoot this issue.

  • Create the app directory and its __init__.py file by running these commands:

macOS

mkdir app
touch app/__init__.py

Windows

mkdir app
New-Item app/__init__.py
  • Create your .env file by running:

macOS

echo "GOOGLE_API_KEY=your-api-key-here" > .env

Windows

echo GOOGLE_API_KEY=your-api-key-here > .env
  • Open .env in VS Code and replace your-api-key-here with the API key you copied from Google AI Studio.
  • Save the file.

Why a .env file?

Storing your API key in a .env file keeps it out of your source code. The python-dotenv package loads this file at runtime so your application can access the key without hardcoding it.

  • Confirm your project structure by running:
ls

You should see app, venv, and .env listed in the output.

✔️ Awesome, I've got everything!

Great. Make sure you have saved your .env file with your real API key before moving on.

ⓧ I'd like to double check the full code

Your .env file should contain exactly this (with your real key in place of the placeholder):

GOOGLE_API_KEY=your-api-key-here

Your app/__init__.py file should be empty. It exists only to mark the app folder as a Python package.

Start ChromaDB in Docker

ChromaDB is a vector database that stores your document embeddings for fast similarity search. Running it in Docker means you get a fully configured database with one command.

  • Start the ChromaDB container by running:

macOS

docker run -d -v ./chroma-data:/data -p 8000:8000 chromadb/chroma

Windows

docker run -d -v ${PWD}/chroma-data:/data -p 8000:8000 chromadb/chroma

What does this command do?

  • -d runs the container in the background (detached mode).
  • -v ./chroma-data:/data mounts a local folder so your data survives container restarts.
  • -p 8000:8000 maps the container's port 8000 to your machine's port 8000.
  • chromadb/chroma is the official ChromaDB Docker image.

Docker will pull the image if this is your first time running it. This may take a minute.

  • Verify ChromaDB is running by sending a heartbeat request:
curl http://localhost:8000/api/v2/heartbeat

You should see a JSON response like {"nanosecond heartbeat": 1700000000000000000}. This confirms ChromaDB is running and accepting connections.

Not getting a response from ChromaDB?

Make sure Docker Desktop is running before you execute the docker run command.

If port 8000 is already in use by another service, you will see a bind error. Stop the other service or change the port mapping.

Run docker ps to check if the container is listed. If it shows as exited, run docker logs followed by the container ID to see what went wrong.

Help me debug my ChromaDB Docker container.

Your development environment is fully configured. Next up, you will build the document ingestion pipeline that takes PDF files, splits them into chunks, and stores their embeddings in ChromaDB.

Build the Document Ingestion Pipeline

Your environment is ready with all dependencies installed and ChromaDB running in Docker. Now it's time to build the core of your study assistant: the document ingestion pipeline.

The ingestion pipeline takes a PDF file, splits it into smaller chunks, converts those chunks into vector embeddings, and stores them in ChromaDB. Once stored, those vectors become searchable by meaning, not just keywords.

In this step, get ready to:
  • Create the ingestion module that chunks and embeds PDF documents into ChromaDB.
  • Create the FastAPI server with a POST endpoint for uploading PDFs.
  • Test the full pipeline end-to-end by ingesting a sample PDF.
Create the ingestion module

The ingestion module handles the entire journey from raw PDF to searchable vectors. It uses LlamaIndex to read the PDF, split it into chunks, and build an index stored in ChromaDB.

  • Create a new file called app/ingestion.py inside your app folder.
  • Add the imports and configuration constants by pasting the following code:
import chromadb
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, StorageContext
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core.node_parser import SentenceSplitter
import os


# Connection settings for the ChromaDB Docker container
CHROMA_HOST = "localhost"
CHROMA_PORT = 8000
COLLECTION_NAME = "course_materials"
# Local embedding model - runs on your machine, no API cost
EMBED_MODEL_NAME = "BAAI/bge-small-en-v1.5"

What does this code do?

The imports bring in LlamaIndex components for reading documents, splitting them, and storing vectors. ChromaVectorStore connects LlamaIndex to your Docker-based ChromaDB instance.

The constants define where ChromaDB lives (localhost:8000) and which embedding model to use. BAAI/bge-small-en-v1.5 runs entirely on your machine with zero API cost, producing 384-dimension vectors.

  • Below the constants, add the two helper functions:
def get_chroma_collection():
    # Connect to the ChromaDB Docker container over HTTP
    client = chromadb.HttpClient(host=CHROMA_HOST, port=CHROMA_PORT)
    # Create the collection if it doesn't exist, or get it if it does
    collection = client.get_or_create_collection(name=COLLECTION_NAME)
    return collection


def get_embed_model():
    # Load the HuggingFace embedding model (downloads on first run)
    return HuggingFaceEmbedding(model_name=EMBED_MODEL_NAME)

What do these helpers do?

get_chroma_collection() creates an HTTP connection to your ChromaDB Docker container and returns a collection named course_materials. Collections in ChromaDB are like tables in a database.

get_embed_model() loads the HuggingFace embedding model. The first time you run this, it downloads the model files (about 130MB). After that, it loads from cache.

  • Below the helper functions, add the main ingestion function:
def ingest_pdf(file_path: str, classroom: str) -> int:
    # Read the PDF into LlamaIndex document objects
    documents = SimpleDirectoryReader(input_files=[file_path]).load_data()

    # Tag every document chunk with the classroom name for filtering later
    for doc in documents:
        doc.metadata["classroom"] = classroom

    embed_model = get_embed_model()

    # Set up ChromaDB as the vector store backend
    collection = get_chroma_collection()
    vector_store = ChromaVectorStore(chroma_collection=collection)
    storage_context = StorageContext.from_defaults(vector_store=vector_store)

    # Split documents into chunks of 512 tokens with 50-token overlap
    splitter = SentenceSplitter(chunk_size=512, chunk_overlap=50)
    nodes = splitter.get_nodes_from_documents(documents)

    # Build the vector index - this embeds all chunks and stores them
    index = VectorStoreIndex(
        nodes,
        storage_context=storage_context,
        embed_model=embed_model,
    )

    return len(nodes)

What does ingest_pdf do?

This function is the heart of the pipeline. It reads a PDF, attaches a classroom label to each document (so you can filter by subject later), then splits the content into 512-token chunks with a 50-token overlap between them.

The overlap ensures that sentences split across chunk boundaries still have context in both chunks. Finally, VectorStoreIndex embeds every chunk and writes the vectors to ChromaDB. It returns the total number of chunks created.

  • Save app/ingestion.py.

You won't see a change yet because this module has no way to run on its own. The next substep creates the FastAPI server that calls it.

✔️ Awesome, I've got everything!

Great. Double check you've saved app/ingestion.py before moving on.

ⓧ I'd like to double check the full code

import chromadb
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, StorageContext
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core.node_parser import SentenceSplitter
import os


CHROMA_HOST = "localhost"
CHROMA_PORT = 8000
COLLECTION_NAME = "course_materials"
EMBED_MODEL_NAME = "BAAI/bge-small-en-v1.5"


def get_chroma_collection():
    client = chromadb.HttpClient(host=CHROMA_HOST, port=CHROMA_PORT)
    collection = client.get_or_create_collection(name=COLLECTION_NAME)
    return collection


def get_embed_model():
    return HuggingFaceEmbedding(model_name=EMBED_MODEL_NAME)


def ingest_pdf(file_path: str, classroom: str) -> int:
    documents = SimpleDirectoryReader(input_files=[file_path]).load_data()

    for doc in documents:
        doc.metadata["classroom"] = classroom

    embed_model = get_embed_model()

    collection = get_chroma_collection()
    vector_store = ChromaVectorStore(chroma_collection=collection)
    storage_context = StorageContext.from_defaults(vector_store=vector_store)

    splitter = SentenceSplitter(chunk_size=512, chunk_overlap=50)
    nodes = splitter.get_nodes_from_documents(documents)

    index = VectorStoreIndex(
        nodes,
        storage_context=storage_context,
        embed_model=embed_model,
    )

    return len(nodes)
Create the FastAPI ingest endpoint

Now you need a way to trigger the ingestion pipeline over HTTP. FastAPI gives you a REST API with automatic request validation and interactive documentation.

  • Create a new file called app/main.py inside your app folder.
  • Add the imports, app initialization, and CORS middleware by pasting the following code:
import os
import shutil
import tempfile
from fastapi import FastAPI, UploadFile, File, Form, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from typing import Optional
from dotenv import load_dotenv

from app.ingestion import ingest_pdf

# Load environment variables from .env file
load_dotenv()

# Create the FastAPI app instance
app = FastAPI(title="Study RAG API", version="1.0.0")

# Allow requests from any origin (needed for the mobile app later)
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

What does this setup do?

load_dotenv() reads your .env file so the GOOGLE_API_KEY is available when you add the query endpoint later.

CORSMiddleware allows cross-origin requests from any domain. This is essential because the mobile app (a future project) will call this API from a different origin.

  • Below the middleware setup, add the response model and the two endpoints:
# Response shape for the ingest endpoint
class IngestResponse(BaseModel):
    message: str
    chunks_created: int
    classroom: str


@app.post("/ingest", response_model=IngestResponse)
async def ingest_endpoint(
    file: UploadFile = File(...),
    classroom: str = Form(...),
):
    # Only accept PDF files
    if not file.filename.endswith(".pdf"):
        raise HTTPException(status_code=400, detail="Only PDF files are supported")

    # Save the uploaded file to a temporary directory
    temp_dir = tempfile.mkdtemp()
    temp_path = os.path.join(temp_dir, file.filename)

    try:
        with open(temp_path, "wb") as f:
            shutil.copyfileobj(file.file, f)

        # Run the ingestion pipeline
        num_chunks = ingest_pdf(temp_path, classroom)

        return IngestResponse(
            message=f"Successfully ingested {file.filename}",
            chunks_created=num_chunks,
            classroom=classroom,
        )
    finally:
        # Clean up the temporary file regardless of success or failure
        shutil.rmtree(temp_dir)


@app.get("/health")
async def health_check():
    return {"status": "healthy"}

What does the /ingest endpoint do?

The POST /ingest endpoint accepts two form fields: a PDF file and a classroom name. It saves the uploaded PDF to a temporary directory, passes it to your ingest_pdf function, then cleans up the temp file.

The finally block ensures cleanup happens even if the ingestion fails partway through. The /health endpoint is a simple check that confirms the server is running.

  • Save app/main.py.

Seeing an import error?

If VS Code underlines from app.ingestion import ingest_pdf in red, make sure app/__init__.py exists (it can be empty). This file tells Python that the app folder is a package.

help me fix this import error in my FastAPI project

✔️ Awesome, I've got everything!

Great. Double check you've saved app/main.py before moving on.

ⓧ I'd like to double check the full code

import os
import shutil
import tempfile
from fastapi import FastAPI, UploadFile, File, Form, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from typing import Optional
from dotenv import load_dotenv

from app.ingestion import ingest_pdf

load_dotenv()

app = FastAPI(title="Study RAG API", version="1.0.0")

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)


class IngestResponse(BaseModel):
    message: str
    chunks_created: int
    classroom: str


@app.post("/ingest", response_model=IngestResponse)
async def ingest_endpoint(
    file: UploadFile = File(...),
    classroom: str = Form(...),
):
    if not file.filename.endswith(".pdf"):
        raise HTTPException(status_code=400, detail="Only PDF files are supported")

    temp_dir = tempfile.mkdtemp()
    temp_path = os.path.join(temp_dir, file.filename)

    try:
        with open(temp_path, "wb") as f:
            shutil.copyfileobj(file.file, f)

        num_chunks = ingest_pdf(temp_path, classroom)

        return IngestResponse(
            message=f"Successfully ingested {file.filename}",
            chunks_created=num_chunks,
            classroom=classroom,
        )
    finally:
        shutil.rmtree(temp_dir)


@app.get("/health")
async def health_check():
    return {"status": "healthy"}
Test the ingestion pipeline

Time to see the pipeline in action. You will start the FastAPI server, upload a sample PDF, and verify that chunks land in ChromaDB.

  • Find a sample PDF to test with. Any course handout, lecture notes, or syllabus works. Place it in your project folder (the same folder that contains the app directory).
  • Start the FastAPI server by running this command:
uvicorn app.main:app --reload --port 8001

Why port 8001?

Your ChromaDB Docker container already occupies port 8000. Running FastAPI on port 8001 avoids a conflict between the two services.

  • Open your browser and navigate to http://localhost:8001/docs to see the FastAPI interactive documentation.

You should see the Swagger UI page listing your two endpoints: POST /ingest and GET /health.

  • Click on POST /ingest to expand it.
  • Click Try it out.
  • Click Choose File and select your sample PDF.
  • In the classroom field, type CS101 - Data Structures.
  • Click Execute.

What should I see?

The first request takes longer because the embedding model downloads on first use (about 130MB). After it finishes, you should see a JSON response like:

{
  "message": "Successfully ingested syllabus.pdf",
  "chunks_created": 12,
  "classroom": "CS101 - Data Structures"
}

The exact chunks_created number depends on how long your PDF is. A 2-3 page document typically produces 5-15 chunks.

Server not starting or connection refused?

  • Make sure your virtual environment is activated. You should see (venv) at the start of your terminal prompt.
  • Check that your ChromaDB Docker container is still running. If not, restart it.
  • If you see a ModuleNotFoundError, confirm you ran pip install with all the packages from Step 1.

help me fix my FastAPI server startup issue

Your ingestion pipeline is working. PDFs go in, vector embeddings come out, and ChromaDB stores them for retrieval. Next up, you will build the query endpoint that searches those vectors and generates answers with Gemini.

Build the Query Endpoint with Gemini

Your ingestion pipeline is working. PDFs go in, vector embeddings come out, and ChromaDB stores them for retrieval.

Now you need the other half of the RAG equation: a query engine that takes a student's question, finds the most relevant chunks from ChromaDB, and sends them to Gemini 2.5 Flash for a grounded answer. This is where the magic happens.

In this step, get ready to:
  • Configure the LLM query engine with Gemini and ChromaDB.
  • Create a POST /query endpoint in FastAPI.
  • Test end-to-end retrieval against your uploaded PDF.
Configure the LLM query engine

The query engine connects three pieces: ChromaDB (where your vectors live), a HuggingFace embedding model (to convert the question into a vector), and Gemini 2.5 Flash (to generate a natural language answer from the retrieved chunks).

  • Create a new file called app/query_engine.py and add the following code:
import chromadb
from llama_index.core import VectorStoreIndex
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.llms.google_genai import GoogleGenAI
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core.vector_stores import MetadataFilters, ExactMatchFilter
import os

CHROMA_HOST = "localhost"
CHROMA_PORT = 8000
COLLECTION_NAME = "course_materials"
EMBED_MODEL_NAME = "BAAI/bge-small-en-v1.5"
LLM_MODEL = "gemini-2.5-flash"


def get_query_engine(classroom: str = None):
    # Connect to the ChromaDB collection
    client = chromadb.HttpClient(host=CHROMA_HOST, port=CHROMA_PORT)
    collection = client.get_or_create_collection(name=COLLECTION_NAME)

    vector_store = ChromaVectorStore(chroma_collection=collection)
    embed_model = HuggingFaceEmbedding(model_name=EMBED_MODEL_NAME)

    # Rebuild the index from existing vectors (no re-embedding needed)
    index = VectorStoreIndex.from_vector_store(
        vector_store=vector_store,
        embed_model=embed_model,
    )

    # Initialize Gemini 2.5 Flash as the language model
    llm = GoogleGenAI(model=LLM_MODEL, api_key=os.getenv("GOOGLE_API_KEY"))

    # Apply metadata filter if a classroom is specified
    filters = None
    if classroom:
        filters = MetadataFilters(
            filters=[ExactMatchFilter(key="classroom", value=classroom)]
        )

    # Create the query engine with top-5 similarity search
    query_engine = index.as_query_engine(
        llm=llm,
        similarity_top_k=5,
        filters=filters,
    )

    return query_engine

What does this code do?

This function builds a complete RAG query pipeline each time it is called. It connects to your existing ChromaDB collection and rebuilds the LlamaIndex index from the stored vectors without re-embedding anything.

The GoogleGenAI class connects to Gemini using your API key from the .env file. The similarity_top_k=5 setting means the engine retrieves the 5 most relevant chunks to include as context for Gemini's answer.

The optional classroom filter uses MetadataFilters to restrict retrieval to chunks tagged with a specific classroom. This keeps different subjects isolated.

  • Now add the query_documents function below get_query_engine in the same file:
def query_documents(question: str, classroom: str = None) -> dict:
    # Get a query engine, optionally filtered by classroom
    query_engine = get_query_engine(classroom)
    response = query_engine.query(question)

    # Extract source file names from the retrieved nodes
    sources = []
    for node in response.source_nodes:
        source_info = node.metadata.get("file_name", "Unknown source")
        sources.append(source_info)

    return {
        "answer": str(response),
        "sources": list(set(sources)),
    }

What does this function do?

This is a wrapper that calls the query engine and formats the result into a clean dictionary. It extracts the file_name from each retrieved chunk's metadata so the API response tells the student which source documents the answer came from.

Using list(set(sources)) deduplicates the source list in case multiple chunks came from the same PDF.

  • Save app/query_engine.py.

Seeing an import error?

If your editor shows a red underline on from llama_index.llms.google_genai import GoogleGenAI, make sure your virtual environment is activated. The llama-index-llms-google-genai package was installed in Step 1.

Help me fix this import error.

Create the query endpoint

With the query engine ready, you need a FastAPI endpoint that accepts a question, runs it through the pipeline, and returns the answer with source references.

  • In app/main.py, find this import line:
from app.ingestion import ingest_pdf
  • Add the query_documents import directly below it, so both lines look like this:
from app.ingestion import ingest_pdf
from app.query_engine import query_documents
  • Next, find the class IngestResponse line. Add the following two model classes above it:
class QueryRequest(BaseModel):
    question: str
    classroom: Optional[str] = None


class QueryResponse(BaseModel):
    answer: str
    sources: list[str]

What are these models?

Pydantic models define the shape of data going in and out of your API. QueryRequest expects a question string and an optional classroom filter. QueryResponse guarantees every response includes an answer and a list of sources.

  • Now add the /query endpoint. Find the @app.get("/health") line and paste the following code above it:
@app.post("/query", response_model=QueryResponse)
async def query_endpoint(request: QueryRequest):
    result = query_documents(request.question, request.classroom)

    return QueryResponse(
        answer=result["answer"],
        sources=result["sources"],
    )

What does this endpoint do?

When a POST request hits /query, FastAPI parses the JSON body into a QueryRequest object. It passes the question and optional classroom filter to query_documents, which retrieves relevant chunks from ChromaDB, sends them to Gemini, and returns the grounded answer.

  • Save app/main.py.
  • Your FastAPI server should auto-reload since you started it with --reload in the previous step. Check your terminal for a successful reload message.

Server crashed on reload?

Check that you placed the QueryRequest and QueryResponse classes above IngestResponse in the file. Also confirm the import line from app.query_engine import query_documents matches your file name exactly.

Help me debug why my server crashed.

✔️ Awesome, I've got everything!

Great. Double check you have saved both app/query_engine.py and app/main.py.

ⓧ I'd like to double check the full code

Here is the full app/query_engine.py:

import chromadb
from llama_index.core import VectorStoreIndex
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.llms.google_genai import GoogleGenAI
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core.vector_stores import MetadataFilters, ExactMatchFilter
import os

CHROMA_HOST = "localhost"
CHROMA_PORT = 8000
COLLECTION_NAME = "course_materials"
EMBED_MODEL_NAME = "BAAI/bge-small-en-v1.5"
LLM_MODEL = "gemini-2.5-flash"


def get_query_engine(classroom: str = None):
    client = chromadb.HttpClient(host=CHROMA_HOST, port=CHROMA_PORT)
    collection = client.get_or_create_collection(name=COLLECTION_NAME)

    vector_store = ChromaVectorStore(chroma_collection=collection)
    embed_model = HuggingFaceEmbedding(model_name=EMBED_MODEL_NAME)

    index = VectorStoreIndex.from_vector_store(
        vector_store=vector_store,
        embed_model=embed_model,
    )

    llm = GoogleGenAI(model=LLM_MODEL, api_key=os.getenv("GOOGLE_API_KEY"))

    filters = None
    if classroom:
        filters = MetadataFilters(
            filters=[ExactMatchFilter(key="classroom", value=classroom)]
        )

    query_engine = index.as_query_engine(
        llm=llm,
        similarity_top_k=5,
        filters=filters,
    )

    return query_engine


def query_documents(question: str, classroom: str = None) -> dict:
    query_engine = get_query_engine(classroom)
    response = query_engine.query(question)

    sources = []
    for node in response.source_nodes:
        source_info = node.metadata.get("file_name", "Unknown source")
        sources.append(source_info)

    return {
        "answer": str(response),
        "sources": list(set(sources)),
    }

Here is the full app/main.py:

import os
import shutil
import tempfile
from fastapi import FastAPI, UploadFile, File, Form, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from typing import Optional
from dotenv import load_dotenv

from app.ingestion import ingest_pdf
from app.query_engine import query_documents

load_dotenv()

app = FastAPI(title="Study RAG API", version="1.0.0")

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)


class QueryRequest(BaseModel):
    question: str
    classroom: Optional[str] = None


class QueryResponse(BaseModel):
    answer: str
    sources: list[str]


class IngestResponse(BaseModel):
    message: str
    chunks_created: int
    classroom: str


@app.post("/ingest", response_model=IngestResponse)
async def ingest_endpoint(
    file: UploadFile = File(...),
    classroom: str = Form(...),
):
    if not file.filename.endswith(".pdf"):
        raise HTTPException(status_code=400, detail="Only PDF files are supported")

    temp_dir = tempfile.mkdtemp()
    temp_path = os.path.join(temp_dir, file.filename)

    try:
        with open(temp_path, "wb") as f:
            shutil.copyfileobj(file.file, f)

        num_chunks = ingest_pdf(temp_path, classroom)

        return IngestResponse(
            message=f"Successfully ingested {file.filename}",
            chunks_created=num_chunks,
            classroom=classroom,
        )
    finally:
        shutil.rmtree(temp_dir)


@app.post("/query", response_model=QueryResponse)
async def query_endpoint(request: QueryRequest):
    result = query_documents(request.question, request.classroom)

    return QueryResponse(
        answer=result["answer"],
        sources=result["sources"],
    )


@app.get("/health")
async def health_check():
    return {"status": "healthy"}
Test end-to-end

Time to see if your RAG pipeline actually works. You will send a question about the PDF you ingested in the previous step and check that Gemini's answer is grounded in that document.

  • Open the FastAPI interactive docs by navigating to http://localhost:8001/docs in your browser.
  • Find the POST /query endpoint and click Try it out.
  • In the request body, enter a question about content you know is in your uploaded PDF. For example:
{
  "question": "What are the main topics covered in this course?"
}
  • Click Execute.

You should see a JSON response with an answer field containing information from your PDF, and a sources field listing the filename of your uploaded document.

What should I see?

The response should look something like this (your content will differ based on your PDF):

{
  "answer": "Based on the course materials, the main topics include...",
  "sources": ["your-uploaded-file.pdf"]
}

The key thing to verify is that the answer references real content from your PDF, not generic knowledge. The sources array confirms which document the answer came from.

  • Now test with a question about something NOT in your document. Try asking about an unrelated topic:
{
  "question": "What is the capital of Mars?"
}
  • Click Execute again.

Gemini should indicate that it does not have that information in the provided context. This confirms your RAG pipeline is grounding answers in the documents, not using general knowledge.

Getting a 500 error or timeout?

Make sure your ChromaDB Docker container is still running. You can verify by opening http://localhost:8000/api/v2/heartbeat in your browser. You should see a JSON response with a nanosecond timestamp.

Also check that your GOOGLE_API_KEY in .env is valid. An invalid key returns an authentication error from the Gemini API.

Help me troubleshoot my query endpoint.

Your RAG backend now has both halves working. Documents go in through /ingest and grounded answers come out through /query. Next up, you will add classroom-based organization so different subjects stay separate.

Add Classroom Organization

Your RAG backend can now ingest PDFs and answer questions grounded in their content. But a real study app serves students taking multiple subjects at once.

A student taking both Data Structures and Biology needs their materials kept separate. In this step, you will exercise the classroom organization system your code already supports. You will ingest PDFs into different classrooms and verify that metadata filtering isolates each subject's materials.

In this step, get ready to:
  • Ingest two PDFs with different classroom names.
  • Verify that classroom-filtered queries return only relevant material.
  • Confirm that unfiltered queries search across all materials.
Add metadata to ingested documents

When a student uploads course materials, the system needs to know which class those materials belong to. Your /ingest endpoint already accepts a classroom field, and the ingestion pipeline attaches it as metadata to every chunk stored in ChromaDB.

How does classroom metadata work?

When you upload a PDF with a classroom name, the ingest_pdf function attaches the classroom string to every document's metadata dictionary before chunking.

When LlamaIndex splits the document into chunks and stores them in ChromaDB, that metadata travels with each chunk. Every vector in the database knows which classroom it belongs to.

  • Find a PDF you would like to use as your first course material. Any PDF will work (lecture notes, a syllabus, or a handout).
  • Ingest it into a classroom called CS101 - Data Structures by running this command:
curl -X POST "http://localhost:8001/ingest" -F "file=@[[PDF_PATH_1=\"path/to/your/first-pdf.pdf\"]]" -F "classroom=CS101 - Data Structures"

What does this command do?

The -F flags send form data to the endpoint. The file=@ prefix tells curl to read the file from disk and upload it.

The classroom field gets attached as metadata to every chunk created from this PDF.

You should see a JSON response showing the number of chunks created and the classroom name.

{
  "message": "Successfully ingested your-file.pdf",
  "chunks_created": 12,
  "classroom": "CS101 - Data Structures"
}

Use the interactive docs instead

If you prefer a visual interface, open http://localhost:8001/docs in your browser. Click the POST /ingest endpoint, then click Try it out to upload your file and set the classroom name through a form.

  • Find a second PDF on a different topic.
  • Ingest it into a different classroom by running:
curl -X POST "http://localhost:8001/ingest" -F "file=@[[PDF_PATH_2=\"path/to/your/second-pdf.pdf\"]]" -F "classroom=BIO200 - Biology"
  • Confirm the response shows BIO200 - Biology as the classroom name and a chunk count.

Seeing a connection error?

Make sure your FastAPI server is still running on port 8001. If you restarted your terminal, switch back to your project folder, activate the virtual environment, and run the uvicorn command again.

Also confirm your ChromaDB Docker container is still running on port 8000.

Help me debug my connection error when ingesting a PDF.

Filter queries by classroom

With two classrooms loaded, you can test whether the filtering keeps materials isolated. Your /query endpoint accepts an optional classroom parameter that restricts which chunks the query engine searches.

How does metadata filtering work?

When you include a classroom value in your query request, the get_query_engine function builds an ExactMatchFilter that tells ChromaDB to only return vectors where the classroom metadata matches exactly.

This filtering happens at the vector database level before the LLM ever sees the results. Only chunks from the matching classroom get passed to Gemini as context.

  • Send a query about a topic from your first PDF, filtered to the CS101 - Data Structures classroom:
curl -X POST "http://localhost:8001/query" -H "Content-Type: application/json" -d "{\"question\": \"What topics are covered in this course?\", \"classroom\": \"CS101 - Data Structures\"}"
  • You should see a response with an answer grounded in your first PDF and a sources list referencing that file.
  • Now send the same question filtered to your second classroom:
curl -X POST "http://localhost:8001/query" -H "Content-Type: application/json" -d "{\"question\": \"What topics are covered in this course?\", \"classroom\": \"BIO200 - Biology\"}"
  • The answer should now reference content from your second PDF only. The sources list should show the second file's name.

Getting answers from the wrong PDF?

Double check that you spelled the classroom name exactly as you did during ingestion. The filter uses exact string matching, so CS101 - Data Structures and cs101 - data structures are treated as different classrooms.

Help me figure out why my classroom filter is not working.

Test classroom isolation

The final test confirms that queries without a classroom filter search across all materials. This is the default behavior for students who want to search everything at once.

  • Send a query without any classroom filter by running:
curl -X POST "http://localhost:8001/query" -H "Content-Type: application/json" -d "{\"question\": \"What topics are covered in this course?\"}"
  • The response may now include information from both PDFs, since no filter restricts which chunks are retrieved.
  • Check the sources list in the response. It should show filenames from both classrooms.

Why does this matter for a study app?

Students often take 4-5 courses at once. Without classroom isolation, asking about "recursion" might pull context from a math course and a computer science course, confusing the AI's answer.

Metadata filtering gives students control. They can focus on one subject or search across everything depending on what they need.

✔️ Awesome, I've got everything!

Great. Double check that you have successfully ingested two PDFs into different classrooms and confirmed the filtering works.

ⓧ I'd like to double check the full code

Your code is unchanged from the previous step. Here is the full code for each file for reference.

import chromadb
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, StorageContext
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core.node_parser import SentenceSplitter
import os


CHROMA_HOST = "localhost"
CHROMA_PORT = 8000
COLLECTION_NAME = "course_materials"
EMBED_MODEL_NAME = "BAAI/bge-small-en-v1.5"


def get_chroma_collection():
    client = chromadb.HttpClient(host=CHROMA_HOST, port=CHROMA_PORT)
    collection = client.get_or_create_collection(name=COLLECTION_NAME)
    return collection


def get_embed_model():
    return HuggingFaceEmbedding(model_name=EMBED_MODEL_NAME)


def ingest_pdf(file_path: str, classroom: str) -> int:
    documents = SimpleDirectoryReader(input_files=[file_path]).load_data()

    for doc in documents:
        doc.metadata["classroom"] = classroom

    embed_model = get_embed_model()

    collection = get_chroma_collection()
    vector_store = ChromaVectorStore(chroma_collection=collection)
    storage_context = StorageContext.from_defaults(vector_store=vector_store)

    splitter = SentenceSplitter(chunk_size=512, chunk_overlap=50)
    nodes = splitter.get_nodes_from_documents(documents)

    index = VectorStoreIndex(
        nodes,
        storage_context=storage_context,
        embed_model=embed_model,
    )

    return len(nodes)
import chromadb
from llama_index.core import VectorStoreIndex
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.llms.google_genai import GoogleGenAI
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
from llama_index.core.vector_stores import MetadataFilters, ExactMatchFilter
import os

CHROMA_HOST = "localhost"
CHROMA_PORT = 8000
COLLECTION_NAME = "course_materials"
EMBED_MODEL_NAME = "BAAI/bge-small-en-v1.5"
LLM_MODEL = "gemini-2.5-flash"


def get_query_engine(classroom: str = None):
    client = chromadb.HttpClient(host=CHROMA_HOST, port=CHROMA_PORT)
    collection = client.get_or_create_collection(name=COLLECTION_NAME)

    vector_store = ChromaVectorStore(chroma_collection=collection)
    embed_model = HuggingFaceEmbedding(model_name=EMBED_MODEL_NAME)

    index = VectorStoreIndex.from_vector_store(
        vector_store=vector_store,
        embed_model=embed_model,
    )

    llm = GoogleGenAI(model=LLM_MODEL, api_key=os.getenv("GOOGLE_API_KEY"))

    filters = None
    if classroom:
        filters = MetadataFilters(
            filters=[ExactMatchFilter(key="classroom", value=classroom)]
        )

    query_engine = index.as_query_engine(
        llm=llm,
        similarity_top_k=5,
        filters=filters,
    )

    return query_engine


def query_documents(question: str, classroom: str = None) -> dict:
    query_engine = get_query_engine(classroom)
    response = query_engine.query(question)

    sources = []
    for node in response.source_nodes:
        source_info = node.metadata.get("file_name", "Unknown source")
        sources.append(source_info)

    return {
        "answer": str(response),
        "sources": list(set(sources)),
    }
import os
import shutil
import tempfile
from fastapi import FastAPI, UploadFile, File, Form, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
from typing import Optional
from dotenv import load_dotenv

from app.ingestion import ingest_pdf
from app.query_engine import query_documents

load_dotenv()

app = FastAPI(title="Study RAG API", version="1.0.0")

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)


class QueryRequest(BaseModel):
    question: str
    classroom: Optional[str] = None


class QueryResponse(BaseModel):
    answer: str
    sources: list[str]


class IngestResponse(BaseModel):
    message: str
    chunks_created: int
    classroom: str


@app.post("/ingest", response_model=IngestResponse)
async def ingest_endpoint(
    file: UploadFile = File(...),
    classroom: str = Form(...),
):
    if not file.filename.endswith(".pdf"):
        raise HTTPException(status_code=400, detail="Only PDF files are supported")

    temp_dir = tempfile.mkdtemp()
    temp_path = os.path.join(temp_dir, file.filename)

    try:
        with open(temp_path, "wb") as f:
            shutil.copyfileobj(file.file, f)

        num_chunks = ingest_pdf(temp_path, classroom)

        return IngestResponse(
            message=f"Successfully ingested {file.filename}",
            chunks_created=num_chunks,
            classroom=classroom,
        )
    finally:
        shutil.rmtree(temp_dir)


@app.post("/query", response_model=QueryResponse)
async def query_endpoint(request: QueryRequest):
    result = query_documents(request.question, request.classroom)

    return QueryResponse(
        answer=result["answer"],
        sources=result["sources"],
    )


@app.get("/health")
async def health_check():
    return {"status": "healthy"}

Your RAG backend now organizes materials by classroom and filters retrieval accordingly. Next up, you will take on the Secret Mission and extend this system with even more powerful metadata filtering.

Secret mission

Add Subject-Aware Smart Retrieval

Your RAG backend can filter by classroom, but what about semesters? A real study app needs to handle materials across multiple terms. In this mission, you'll add semester metadata, build a classrooms listing endpoint, and implement combined filtering so students can drill down to exactly the materials they need.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally with a free-tier API. There are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Docker container running ChromaDB on port 8000.
  • chroma-data folder storing your vector database data.
  • Project folder with Python virtual environment and application code.

Keep everything running

No action needed. Choose this if you plan to continue building the mobile app frontend that connects to this backend.

  • Your Docker container stays running and your ingested data persists in the chroma-data folder.
  • Your FastAPI server continues running on port 8001.
  • Your Google AI API key remains in .env and stays within the free tier as long as you are not making high-volume requests.

Pause - I'll come back to this later

Shut down running processes to free up memory and Docker resources. Your files and data stay on disk.

  • Stop the FastAPI server by pressing Ctrl+C in the terminal where it is running.
  • Find your ChromaDB container ID by running:
docker ps
  • Stop the container by running:
docker stop [[CONTAINER_ID="your ChromaDB container ID"]]
  • Your data is preserved in the chroma-data folder. When you are ready to continue, restart with:
docker start [[CONTAINER_ID="your ChromaDB container ID"]]

Delete - I don't want to use this again

Remove all project resources and start fresh. This cannot be undone.

  • Stop the FastAPI server by pressing Ctrl+C in the terminal where it is running.
  • Deactivate your virtual environment by running:
deactivate
  • Find your ChromaDB container ID by running:
docker ps
  • Force-remove the Docker container by running:
docker rm -f [[CONTAINER_ID="your ChromaDB container ID"]]
  • Delete the chroma-data folder and your project folder. On macOS/Linux:
rm -rf chroma-data
  • On Windows (Command Prompt):
rmdir /s /q chroma-data
  • Navigate up one level and delete the project folder. On macOS/Linux:
cd ..
rm -rf study-rag-api
  • On Windows (Command Prompt):
cd ..
rmdir /s /q study-rag-api
  • Your Google AI API key remains in your Google AI Studio account. It costs nothing on the free tier, so no action is needed unless you want to revoke it from the API keys page at aistudio.google.com.

Nice Work!

Nice Work!

You just built a production-ready RAG backend that transforms any course PDF into a queryable knowledge base. Your API ingests documents, stores them as vector embeddings, and generates answers grounded exclusively in your uploaded materials.

You've learned how to:

  • Build a FastAPI backend with /ingest and /query endpoints that accept PDF uploads and return AI-generated answers with source references.
  • Implement a RAG pipeline with LlamaIndex that chunks PDFs, embeds them locally using HuggingFace embeddings, and retrieves relevant context from ChromaDB running in Docker.
  • Add classroom-based metadata filtering so each subject's materials are isolated, with Gemini 2.5 Flash generating answers grounded only in the relevant documents.
  • Secret Mission: Built subject-aware smart retrieval with semester filtering and a classrooms listing endpoint.

Ready to quiz yourself?