Document Q&A API with RAG

Build a REST API that answers questions about uploaded documents using RAG.

Introduction

30 Second Summary

Think about how chatbots answer questions about specific documents. They don't just guess. They search through the actual text and ground their answers in what they find.

In this project, you will build a REST API that accepts document uploads, stores them in a vector database, and answers questions about them using Claude as the language model.

What You'll Build

A Document Q&A API powered by FastAPI, ChromaDB, and Claude that implements a full Retrieval-Augmented Generation pipeline.

By the end of this project, you'll have:

  • A FastAPI application with three endpoints (ingest, query, list) and auto-generated interactive Swagger documentation.
  • A ChromaDB vector store that chunks, embeds, and persists documents to disk with zero configuration.
  • A RAG pipeline that retrieves relevant context and generates grounded answers with source citations via Claude.
  • Secret Mission: Add source filtering to scope queries and a deletion endpoint for document management.

Are there any prerequisites?

You should be comfortable writing Python code (functions, imports, dictionaries). You also need an Anthropic API key, which you can create at console.anthropic.com.

Set Up Your Project

Before you can build an API that answers questions about documents, you need a project environment with the right tools installed. Your API will depend on four libraries working together: a web framework, a vector database, an LLM client, and a config loader.

In this step, you will create a project folder, install all the dependencies, and configure your API key securely. By the end, you will have a clean environment ready for building.

In this step, get ready to:
  • Create a project folder with a Python virtual environment.
  • Install FastAPI, ChromaDB, the Anthropic SDK, and python-dotenv.
  • Configure your API key and protect it with a .gitignore file.
Create the project folder and virtual environment

A virtual environment isolates your project's packages from the rest of your system. This prevents version conflicts between projects.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir rag-api
cd rag-api
  • Confirm the folder exists by running:
ls

You should see the rag-api folder listed in the output.

  • Open the rag-api folder in VS Code.
  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Select the rag-api folder on your Desktop, then click Open.
  • Open a terminal inside VS Code by pressing Ctrl+` (backtick key, same on macOS and Windows).
  • Create a virtual environment by running this command:
python -m venv venv

What does this command do?

python -m venv venv tells Python to create a virtual environment in a folder called venv.

The first venv is the module name. The second venv is the folder where the environment will live.

  • Activate your virtual environment:

macOS

  • Run this command to activate the environment:
source venv/bin/activate

Windows

  • Run this command to activate the environment:
venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt.

Don't see (venv) in your prompt?

On Windows, you may need to enable script execution first. Run Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser in PowerShell, then try the activate command again.

On macOS, double-check that you typed source before the path. Without it, the shell tries to run the file directly and fails silently.

Help me activate my Python virtual environment.

  • Create your requirements.txt file. In the VS Code Explorer sidebar, click the New File icon.
  • Name the file requirements.txt and press Enter.
  • Paste the following pinned dependencies into requirements.txt:
fastapi[standard]==0.136.3
chromadb==1.5.9
anthropic==0.109.1
python-dotenv==1.2.2

What are these packages?

  • fastapi[standard] is the web framework for your API. The [standard] extra includes the development server.
  • chromadb is the vector database that stores and searches your document chunks.
  • anthropic is the official Python SDK for calling Claude.
  • python-dotenv loads environment variables from a .env file so you can keep secrets out of your code.
  • Save requirements.txt by pressing Cmd+S (macOS) or Ctrl+S (Windows).
Install all dependencies
  • Install the packages from your requirements file by running this command:
pip install -r requirements.txt

This will download and install all four packages plus their dependencies. It may take a minute or two.

Seeing permission errors or install failures?

Make sure you see (venv) in your terminal prompt. If you do not, your virtual environment is not active. Go back and run the activate command from the previous substep.

Help me fix pip install errors in my virtual environment.

  • Verify FastAPI installed correctly by running:
python -c "import fastapi; print(fastapi.__version__)"

You should see 0.136.3 printed in the terminal.

  • Verify ChromaDB installed correctly by running:
python -c "import chromadb; print(chromadb.__version__)"

You should see 1.5.9 printed in the terminal.

Configure your API key

Your API will call Claude to generate answers. The Anthropic SDK needs an API key to authenticate your requests.

  • In the VS Code Explorer sidebar, click the New File icon.
  • Name the file .env and press Enter.
  • Paste the following line into your .env file, replacing your-api-key-here with your actual Anthropic API key:
ANTHROPIC_API_KEY=your-api-key-here

Where do I find my Anthropic API key?

Log in to your Anthropic account at console.anthropic.com. Navigate to API Keys in the sidebar and create a new key.

Copy the key immediately. You will not be able to see it again after you leave the page.

  • Save .env by pressing Cmd+S (macOS) or Ctrl+S (Windows).

Now you need to make sure this file never gets committed to version control.

  • Create another new file in the Explorer sidebar. Name it .gitignore.
  • Paste the following contents into .gitignore:
.env
venv/
chroma_db/
__pycache__/

What does .gitignore protect?

  • .env keeps your API key out of version control.
  • venv/ excludes your virtual environment (it can be recreated from requirements.txt).
  • chroma_db/ excludes the vector database storage folder that will be created later.
  • __pycache__/ excludes Python's compiled bytecode files.
  • Save .gitignore.

✔️ Awesome, I've got everything!

Great. Double check you have saved all three files: requirements.txt, .env, and .gitignore.

ⓧ I'd like to double check the full code

Here are all three files you should have at this point.

fastapi[standard]==0.136.3
chromadb==1.5.9
anthropic==0.109.1
python-dotenv==1.2.2
ANTHROPIC_API_KEY=your-api-key-here
.env
venv/
chroma_db/
__pycache__/

Your environment is ready with all four dependencies installed and your API key configured securely. Next up, you will create the FastAPI app and build your first endpoint for document ingestion.

Create the FastAPI App with Document Ingestion

Your environment is set up and ready to go. Now it's time to build the actual API that will power your document Q&A system.

In this step, you will create main.py with all the pieces needed to accept documents and store them for later retrieval. This includes initializing FastAPI, connecting to ChromaDB for vector storage, and building a document ingestion endpoint.

In this step, get ready to:
  • Create the FastAPI app with ChromaDB and Anthropic clients initialized.
  • Define Pydantic request and response models for automatic validation.
  • Build a document ingestion endpoint that chunks text and stores it in the vector database.
Create main.py with the FastAPI app, ChromaDB client, and Anthropic client

Every request to your API flows through a single entry point. You need to set up the web framework, connect to the vector database, and prepare the LLM client before any endpoint can work.

  • In VS Code, create a new file called main.py inside your rag-api folder.
  • Add the following imports and initialization code to main.py:
import os

from dotenv import load_dotenv
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import chromadb
from anthropic import Anthropic

# Load environment variables from .env file
load_dotenv()

# Initialize the FastAPI application
app = FastAPI(title="Document Q&A API", version="1.0.0")

# Set up ChromaDB with persistent storage so data survives restarts
chroma_client = chromadb.PersistentClient(path="./chroma_db")
collection = chroma_client.get_or_create_collection(name="documents")

# Initialize the Anthropic client for Claude API calls
anthropic_client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))

What does this code do?

This block initializes the three services your API depends on.

  • load_dotenv() reads your .env file and loads ANTHROPIC_API_KEY into the environment.
  • FastAPI(title="Document Q&A API") creates your web application. The title appears in the auto-generated documentation.
  • PersistentClient(path="./chroma_db") stores vectors on disk so your ingested documents survive server restarts.
  • get_or_create_collection(name="documents") creates a named collection on first run, and reconnects to it on subsequent runs.

You won't see a change yet. The models and endpoint you add next bring this server to life.

Seeing an import error?

  • Make sure your virtual environment is activated. You should see (venv) at the start of your terminal prompt.
  • If a specific package cannot be found, re-run pip install -r requirements.txt to reinstall.

Still stuck? Help me debug an import error in my main.py file.

Define Pydantic request and response models

Pydantic models define the shape of data your API accepts and returns. FastAPI uses them for automatic request validation and to generate interactive documentation.

  • Add the following model classes below the anthropic_client line in main.py:
# Request model: what the client sends when uploading a document
class DocumentInput(BaseModel):
    title: str
    content: str
    source: str = "manual"


# Response model: what the API returns after ingestion
class DocumentResponse(BaseModel):
    message: str
    document_id: str
    chunks_created: int

What do these models do?

DocumentInput defines what users must send when uploading a document: a title, the full text content, and an optional source label that defaults to "manual".

DocumentResponse defines what the API sends back: a confirmation message, the generated document ID, and how many chunks were created. If a request is missing a required field, FastAPI automatically returns a 422 error with details.

You won't see a change yet. The endpoint in the next substep uses these models.

Build the POST /documents endpoint with text chunking

Before storing a document, you need to split it into smaller chunks. Large language models and vector search both work better with focused, shorter passages rather than entire documents.

  • Add the chunking helper function and the ingestion endpoint below your model classes in main.py:
def chunk_text(text: str, chunk_size: int = 500, overlap: int = 50) -> list[str]:
    # Split text into overlapping chunks for better retrieval
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        # Overlap prevents cutting information at chunk boundaries
        start = end - overlap
    return chunks


@app.post("/documents", response_model=DocumentResponse)
def ingest_document(doc: DocumentInput):
    # Split the document content into manageable chunks
    chunks = chunk_text(doc.content)

    if not chunks:
        raise HTTPException(status_code=400, detail="Document content is empty")

    # Generate a URL-friendly document ID from the title
    doc_id = doc.title.lower().replace(" ", "-")

    # Create unique IDs for each chunk
    ids = [f"{doc_id}-chunk-{i}" for i in range(len(chunks))]
    # Attach metadata so we can trace chunks back to their source
    metadatas = [
        {"title": doc.title, "source": doc.source, "chunk_index": i}
        for i in range(len(chunks))
    ]

    # Store chunks in ChromaDB (embeddings are generated automatically)
    collection.add(
        documents=chunks,
        metadatas=metadatas,
        ids=ids,
    )

    return DocumentResponse(
        message="Document ingested successfully",
        document_id=doc_id,
        chunks_created=len(chunks),
    )

What does this code do?

chunk_text slides a 500-character window across the text with a 50-character overlap. The overlap ensures that sentences split at a boundary still appear in at least one complete chunk.

The ingest_document endpoint receives a document, chunks it, and stores everything in ChromaDB. ChromaDB automatically generates vector embeddings for each chunk using its built-in Sentence Transformers model. The metadata (title, source, chunk_index) travels with each chunk so you can trace answers back to their origin later.

  • Save main.py.
  • Start the development server by running this command in your terminal:
fastapi dev main.py
  • Open your browser and visit http://127.0.0.1:8000/docs.

You should see the Swagger UI with your POST /documents endpoint listed.

What is Swagger UI?

FastAPI automatically generates interactive API documentation at the /docs path. You can test your endpoints directly in the browser without needing a separate tool like Postman. The documentation updates every time you add or modify an endpoint.

Server not starting?

  • Check your terminal for a specific error message. A SyntaxError means a typo in your Python code. An ImportError means a package is missing.
  • Make sure you saved main.py before running the command.
  • If you see a message about the port being in use, stop any other running server with Ctrl+C first.

Need help? Help me fix my FastAPI server startup error.

✔️ Awesome, I've got everything!

Great. Double check you have saved main.py and keep the server running for the next step.

ⓧ I'd like to double check the full code

Here is the complete main.py file so far:

import os

from dotenv import load_dotenv
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import chromadb
from anthropic import Anthropic

load_dotenv()

app = FastAPI(title="Document Q&A API", version="1.0.0")

chroma_client = chromadb.PersistentClient(path="./chroma_db")
collection = chroma_client.get_or_create_collection(name="documents")

anthropic_client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))


class DocumentInput(BaseModel):
    title: str
    content: str
    source: str = "manual"


class DocumentResponse(BaseModel):
    message: str
    document_id: str
    chunks_created: int


def chunk_text(text: str, chunk_size: int = 500, overlap: int = 50) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks


@app.post("/documents", response_model=DocumentResponse)
def ingest_document(doc: DocumentInput):
    chunks = chunk_text(doc.content)

    if not chunks:
        raise HTTPException(status_code=400, detail="Document content is empty")

    doc_id = doc.title.lower().replace(" ", "-")

    ids = [f"{doc_id}-chunk-{i}" for i in range(len(chunks))]
    metadatas = [
        {"title": doc.title, "source": doc.source, "chunk_index": i}
        for i in range(len(chunks))
    ]

    collection.add(
        documents=chunks,
        metadatas=metadatas,
        ids=ids,
    )

    return DocumentResponse(
        message="Document ingested successfully",
        document_id=doc_id,
        chunks_created=len(chunks),
    )

Your API is live with its first endpoint. Next up, you will build the query endpoint that retrieves relevant chunks and sends them to Claude for a grounded answer.

Build the RAG Query Endpoint

Your API can accept documents, chunk them, and store them in a vector database. Now it needs to answer questions about those documents.

In this step, you'll build the query endpoint where Retrieval-Augmented Generation actually happens. ChromaDB retrieves the most relevant chunks for a question, and Claude generates a grounded answer from them.

In this step, get ready to:
  • Define Pydantic models for the query request and response.
  • Build a POST /query endpoint that retrieves relevant chunks and constructs a RAG prompt.
  • Call Claude to generate a grounded answer and return it with source metadata.
Define the query models

Before building the endpoint logic, you need to define the shape of the incoming query request and the outgoing response. Pydantic models give you automatic request validation and auto-generated Swagger documentation for free.

  • In main.py, find the DocumentResponse class.
  • Add the following two classes between DocumentResponse and the chunk_text function:
class QueryInput(BaseModel):
    # The user's natural language question
    question: str
    # How many chunks to retrieve (default 3)
    n_results: int = 3


class QueryResponse(BaseModel):
    # Claude's generated answer
    answer: str
    # Metadata about which chunks were used
    sources: list[dict]

What do these models do?

QueryInput defines what the client sends: a question string and an optional n_results integer controlling how many document chunks to retrieve.

QueryResponse defines what the API returns: Claude's answer plus a list of source dictionaries so the client can see which documents informed the answer.

  • Save main.py.

Seeing a red underline on list[dict]?

The list[dict] syntax requires Python 3.9+. Since this project uses Python 3.12, this should work fine. If your editor still flags it, make sure your VS Code Python interpreter is set to the venv environment.

help me fix a Python type hint error in VS Code

Build the query endpoint with retrieval and prompt construction

This is the core of your RAG pipeline. The endpoint takes a question, retrieves the most semantically similar chunks from ChromaDB, formats them into a context string, and builds a prompt that instructs Claude to answer only from that context.

  • At the bottom of main.py (after the ingest_document function), add the full query endpoint:
@app.post("/query", response_model=QueryResponse)
def query_documents(query: QueryInput):
    # Ask ChromaDB for the most relevant chunks
    results = collection.query(
        query_texts=[query.question],
        n_results=query.n_results,
    )

    # Return 404 if no documents have been ingested yet
    if not results["documents"][0]:
        raise HTTPException(
            status_code=404, detail="No documents found. Ingest documents first."
        )

    # Extract the matching chunks and their metadata
    context_chunks = results["documents"][0]
    metadatas = results["metadatas"][0]

    # Format each chunk with its source title as a label
    context = "\n\n---\n\n".join(
        [
            f"[Source: {m['title']}]\n{chunk}"
            for chunk, m in zip(context_chunks, metadatas)
        ]
    )

    # Build the RAG prompt: instruct Claude to answer from context only
    prompt = f"""Based on the following context, answer the user's question. \
If the answer cannot be found in the context, say "I don't have enough \
information to answer that question."
Always cite which source document(s) your answer comes from.

Context:
{context}

Question: {query.question}"""

    # Send the prompt to Claude
    message = anthropic_client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
    )

    # Extract the answer text from Claude's response
    answer = message.content[0].text

    # Collect source metadata for transparency
    sources = [
        {"title": m["title"], "source": m["source"], "chunk_index": m["chunk_index"]}
        for m in metadatas
    ]

    return QueryResponse(answer=answer, sources=sources)

What does this endpoint do?

collection.query() performs a semantic search. ChromaDB uses its built-in embedding model to convert the question into a vector and find the closest stored chunks.

The context string labels each retrieved chunk with its source title, so Claude knows where each piece of information came from. The prompt explicitly tells Claude to answer only from this context and to cite its sources.

Finally, anthropic_client.messages.create() sends the assembled prompt to claude-haiku-4-5-20251001. The response text becomes the answer, and the chunk metadata becomes the sources list so the client sees exactly which documents informed the answer.

  • Save main.py.

Why instruct Claude to say it does not know?

Without this guardrail, an LLM will confidently answer from its training data even when the retrieved context has nothing relevant. Telling Claude to say it does not have enough information keeps the system honest. Grounded answers with cited sources are what make RAG trustworthy.

Seeing a syntax error in the prompt string?

The prompt uses a triple-quoted f-string with backslash continuations. Make sure each \ at the end of a line has no space after it. Also check that the inner escaped quotes \" around the fallback phrase are present.

help me fix a syntax error in my f-string prompt

Verify the query endpoint in Swagger UI

The endpoint is complete. Time to confirm it appears in your API documentation and handles the empty-database case correctly.

  • Start (or restart) the dev server by running this command in your terminal:
  • If the server is already running, it should auto-reload when you save.
fastapi dev main.py
  • Open your browser and go to http://127.0.0.1:8000/docs.
  • Confirm you see both POST /documents and POST /query listed in the Swagger UI.
  • Expand POST /query, click Try it out, and submit a request with any question.
  • You should see a 404 response with the detail No documents found. Ingest documents first. because you have not ingested any documents yet.

Why is a 404 the correct response here?

When ChromaDB has no documents, the query returns an empty list. Your endpoint detects this and returns a clear 404 error instead of passing empty context to Claude. This prevents Claude from hallucinating an answer with no supporting evidence.

Not seeing POST /query in Swagger UI?

Make sure you saved main.py after adding the endpoint. If the dev server shows an error in the terminal, check for indentation issues or missing imports.

help me debug why my FastAPI endpoint is not showing in Swagger UI

✔️ Awesome, I've got everything!

Great. Make sure you have saved main.py and your Swagger UI shows both endpoints.

ⓧ I'd like to double check the full code

Here is the full main.py at this point:

import os

from dotenv import load_dotenv
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import chromadb
from anthropic import Anthropic

load_dotenv()

app = FastAPI(title="Document Q&A API", version="1.0.0")

chroma_client = chromadb.PersistentClient(path="./chroma_db")
collection = chroma_client.get_or_create_collection(name="documents")

anthropic_client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))


class DocumentInput(BaseModel):
    title: str
    content: str
    source: str = "manual"


class DocumentResponse(BaseModel):
    message: str
    document_id: str
    chunks_created: int


class QueryInput(BaseModel):
    question: str
    n_results: int = 3


class QueryResponse(BaseModel):
    answer: str
    sources: list[dict]


def chunk_text(text: str, chunk_size: int = 500, overlap: int = 50) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks


@app.post("/documents", response_model=DocumentResponse)
def ingest_document(doc: DocumentInput):
    chunks = chunk_text(doc.content)

    if not chunks:
        raise HTTPException(status_code=400, detail="Document content is empty")

    doc_id = doc.title.lower().replace(" ", "-")

    ids = [f"{doc_id}-chunk-{i}" for i in range(len(chunks))]
    metadatas = [
        {"title": doc.title, "source": doc.source, "chunk_index": i}
        for i in range(len(chunks))
    ]

    collection.add(
        documents=chunks,
        metadatas=metadatas,
        ids=ids,
    )

    return DocumentResponse(
        message="Document ingested successfully",
        document_id=doc_id,
        chunks_created=len(chunks),
    )


@app.post("/query", response_model=QueryResponse)
def query_documents(query: QueryInput):
    results = collection.query(
        query_texts=[query.question],
        n_results=query.n_results,
    )

    if not results["documents"][0]:
        raise HTTPException(
            status_code=404, detail="No documents found. Ingest documents first."
        )

    context_chunks = results["documents"][0]
    metadatas = results["metadatas"][0]

    context = "\n\n---\n\n".join(
        [
            f"[Source: {m['title']}]\n{chunk}"
            for chunk, m in zip(context_chunks, metadatas)
        ]
    )

    prompt = f"""Based on the following context, answer the user's question. \
If the answer cannot be found in the context, say "I don't have enough \
information to answer that question."
Always cite which source document(s) your answer comes from.

Context:
{context}

Question: {query.question}"""

    message = anthropic_client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
    )

    answer = message.content[0].text

    sources = [
        {"title": m["title"], "source": m["source"], "chunk_index": m["chunk_index"]}
        for m in metadatas
    ]

    return QueryResponse(answer=answer, sources=sources)

Your RAG query endpoint is wired up and returning a clear error when there is nothing to search. Next, you'll add a list endpoint and test the full workflow by ingesting a real document and asking questions about it.

Add the List Endpoint and Test Your API

Your RAG pipeline can now ingest documents and answer questions about them.

In this step, you will add the final endpoint that lists all stored documents. Then you will test the full workflow end-to-end: ingesting documents, listing them, and asking questions that Claude answers using only the retrieved context.

In this step, get ready to:
  • Build a GET /documents endpoint that shows unique documents and their chunk counts.
  • Test all three endpoints through Swagger UI.
  • Verify that Claude gives grounded answers with correct source citations.
Build the GET /documents endpoint

Right now you can ingest documents and query them, but there is no way to check what has already been stored. A list endpoint solves this by retrieving everything from ChromaDB and deduplicating by title.

  • In main.py, scroll to the very bottom of the file (after the query_documents function) and add the following code:
@app.get("/documents")
def list_documents():
    # Retrieve all items stored in ChromaDB
    all_items = collection.get()

    # Return empty response if no documents exist
    if not all_items["ids"]:
        return {"documents": [], "total_chunks": 0}

    # Deduplicate by title to show unique documents
    unique_docs = {}
    for metadata in all_items["metadatas"]:
        title = metadata["title"]
        if title not in unique_docs:
            unique_docs[title] = {
                "title": title,
                "source": metadata["source"],
                "chunks": 0,
            }
        unique_docs[title]["chunks"] += 1

    return {
        "documents": list(unique_docs.values()),
        "total_chunks": len(all_items["ids"]),
    }

What does this code do?

collection.get() retrieves every item stored in ChromaDB without performing a similarity search. It returns all IDs, documents, and metadata.

The deduplication loop groups chunks by their title metadata field. Since each document gets split into multiple chunks during ingestion, this counts how many chunks belong to each unique document rather than listing every chunk individually.

  • Save main.py.

Seeing a syntax error when you save?

Make sure the new function is at the same indentation level as your other endpoint functions. It should not be nested inside query_documents.

help me fix a syntax error in my FastAPI endpoint code

✔️ Awesome, I've got everything!

Great. Double check you have saved main.py before moving on.

ⓧ I'd like to double check the full code

import os

from dotenv import load_dotenv
from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import chromadb
from anthropic import Anthropic

load_dotenv()

app = FastAPI(title="Document Q&A API", version="1.0.0")

chroma_client = chromadb.PersistentClient(path="./chroma_db")
collection = chroma_client.get_or_create_collection(name="documents")

anthropic_client = Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))


class DocumentInput(BaseModel):
    title: str
    content: str
    source: str = "manual"


class DocumentResponse(BaseModel):
    message: str
    document_id: str
    chunks_created: int


class QueryInput(BaseModel):
    question: str
    n_results: int = 3


class QueryResponse(BaseModel):
    answer: str
    sources: list[dict]


def chunk_text(text: str, chunk_size: int = 500, overlap: int = 50) -> list[str]:
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks


@app.post("/documents", response_model=DocumentResponse)
def ingest_document(doc: DocumentInput):
    chunks = chunk_text(doc.content)

    if not chunks:
        raise HTTPException(status_code=400, detail="Document content is empty")

    doc_id = doc.title.lower().replace(" ", "-")

    ids = [f"{doc_id}-chunk-{i}" for i in range(len(chunks))]
    metadatas = [
        {"title": doc.title, "source": doc.source, "chunk_index": i}
        for i in range(len(chunks))
    ]

    collection.add(
        documents=chunks,
        metadatas=metadatas,
        ids=ids,
    )

    return DocumentResponse(
        message="Document ingested successfully",
        document_id=doc_id,
        chunks_created=len(chunks),
    )


@app.post("/query", response_model=QueryResponse)
def query_documents(query: QueryInput):
    results = collection.query(
        query_texts=[query.question],
        n_results=query.n_results,
    )

    if not results["documents"][0]:
        raise HTTPException(
            status_code=404, detail="No documents found. Ingest documents first."
        )

    context_chunks = results["documents"][0]
    metadatas = results["metadatas"][0]

    context = "\n\n---\n\n".join(
        [
            f"[Source: {m['title']}]\n{chunk}"
            for chunk, m in zip(context_chunks, metadatas)
        ]
    )

    prompt = f"""Based on the following context, answer the user's question. \
If the answer cannot be found in the context, say "I don't have enough \
information to answer that question."
Always cite which source document(s) your answer comes from.

Context:
{context}

Question: {query.question}"""

    message = anthropic_client.messages.create(
        model="claude-haiku-4-5-20251001",
        max_tokens=1024,
        messages=[{"role": "user", "content": prompt}],
    )

    answer = message.content[0].text

    sources = [
        {"title": m["title"], "source": m["source"], "chunk_index": m["chunk_index"]}
        for m in metadatas
    ]

    return QueryResponse(answer=answer, sources=sources)


@app.get("/documents")
def list_documents():
    all_items = collection.get()

    if not all_items["ids"]:
        return {"documents": [], "total_chunks": 0}

    unique_docs = {}
    for metadata in all_items["metadatas"]:
        title = metadata["title"]
        if title not in unique_docs:
            unique_docs[title] = {
                "title": title,
                "source": metadata["source"],
                "chunks": 0,
            }
        unique_docs[title]["chunks"] += 1

    return {
        "documents": list(unique_docs.values()),
        "total_chunks": len(all_items["ids"]),
    }
Test the full workflow via Swagger UI

With all three endpoints in place, you can now test the entire RAG pipeline through FastAPI's built-in interactive documentation.

  • Start the development server by running this command in your terminal:
  • If the server is already running from a previous step, stop it with Ctrl+C first, then restart it.
fastapi dev main.py
  • Open your browser and navigate to http://127.0.0.1:8000/docs.
  • You should see the Swagger UI page with three endpoints listed: POST /documents, POST /query, and GET /documents.

How do I use Swagger UI to test an endpoint?

Click on any endpoint to expand it. Then click the Try it out button in the top right of that endpoint's section. This lets you fill in the request body. After entering your data, click Execute to send the request and see the response below.

  • Click on POST /documents to expand it.
  • Click Try it out.
  • Replace the example request body with a sample document. Paste any text you are familiar with (for example, a paragraph from a Wikipedia article about a topic you know well). Here is an example:
{
  "title": "Python Overview",
  "content": "Python is a high-level, general-purpose programming language. Its design philosophy emphasizes code readability with the use of significant indentation. Python is dynamically typed and garbage-collected. It supports multiple programming paradigms, including structured, object-oriented and functional programming. Python was conceived in the late 1980s by Guido van Rossum at Centrum Wiskunde & Informatica in the Netherlands as a successor to the ABC programming language.",
  "source": "wikipedia"
}
  • Click Execute.
  • You should see a 200 response with a message confirming the document was ingested and the number of chunks created.
  • Now click on GET /documents to expand it.
  • Click Try it out, then Execute.
  • You should see your document listed with its title, source, and chunk count.
  • Click on POST /query to expand it.
  • Click Try it out and enter a question about your document's content:
{
  "question": "Who created Python and when?",
  "n_results": 3
}
  • Click Execute.
  • You should see a response with an answer that references information from your document and a sources array listing which document the answer came from.

Getting a 404 response from the query endpoint?

This means no documents are stored in ChromaDB. Make sure you successfully ingested at least one document using POST /documents before querying.

If you restarted the server and previously ingested documents, they should still be there because ChromaDB uses PersistentClient. Check that the chroma_db/ folder exists in your project directory.

help me troubleshoot my 404 error on the query endpoint

Verify the RAG behavior

A well-built RAG system gives grounded answers when it has context, and honestly says it does not know when it lacks context. Let's confirm both behaviors.

  • In Swagger UI, use POST /query to ask a question that IS directly answered in your document. Confirm Claude gives a grounded answer and cites the correct source title in the sources array.
  • Ask a question that is NOT in your document (for example, a question about an unrelated topic). Confirm Claude responds saying it does not have enough information to answer.
  • Ingest a second document on a different topic using POST /documents. Give it a different title and source value.
  • Query about the second document's content using POST /query. Confirm the sources array in the response references the correct document title.
  • Use GET /documents one more time. You should now see both documents listed with their respective chunk counts.

Why does this matter?

These two behaviors are what separate RAG from plain LLM usage. A plain LLM might hallucinate an answer from its training data. Your RAG system constrains Claude to answer only from the retrieved context, making it verifiable and trustworthy.

The source citations give users a way to trace each answer back to the original document. This transparency is critical for production Q&A systems.

Secret mission

Add Source Filtering and Document Deletion

Your API can answer questions, but it searches everything at once. What if a user only wants answers from a specific source? And what if they want to remove a document entirely? Add source-scoped queries and a deletion endpoint to give users full control over their document store.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on your local machine. There are no ongoing cloud costs or running services to worry about.

The Anthropic API only charged per request during your testing, totaling under $0.10. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • rag-api/ project folder on your Desktop (contains code, virtual environment, and vector database).
  • chroma_db/ folder inside rag-api/ (your persisted ChromaDB vector database).

Keep everything running

No action needed. Choose this if you plan to continue building on this API or ingesting more documents.

  • Your chroma_db/ folder retains all ingested documents between server restarts.
  • Your virtual environment stays ready. Run fastapi dev main.py anytime to start the server again.
  • The Anthropic API only charges when you make requests. There is no idle cost.

Pause - I'll come back to this later

Shut down the running server to free up your terminal. Your files and database stay on disk for next time.

  • Press Ctrl+C in the terminal where the FastAPI dev server is running.
  • Your chroma_db/ folder keeps all your ingested documents. They will still be there when you restart the server.
  • When you come back, reactivate your virtual environment and run fastapi dev main.py to pick up where you left off.

Delete - I don't want to use this again

Remove all project files, the virtual environment, and the vector database. Your ingested documents will be permanently gone.

  • Press Ctrl+C in the terminal to stop the FastAPI dev server if it is still running.
  • Deactivate your virtual environment by running:
deactivate
  • Navigate to your Desktop and delete the entire project folder.

On macOS/Linux:

cd ~/Desktop
rm -rf rag-api

On Windows:

cd ~\Desktop
rmdir /s /q rag-api
  • This removes main.py, requirements.txt, .env, .gitignore, the venv/ virtual environment, and the chroma_db/ vector database.
  • The Anthropic API key in your .env file is deleted with the folder. Your key still exists in your Anthropic account dashboard if you need it again.

Nice Work!

Nice Work!

Nice work! You just built a production-ready Document Q&A API that ingests text, stores it as vectors, retrieves relevant context, and generates grounded answers with source citations.

You've learned how to:

  • Build a REST API with FastAPI including automatic request validation, error handling, and interactive Swagger documentation.
  • Implement a complete RAG pipeline from scratch: text chunking, vector storage in ChromaDB, semantic retrieval, and LLM-grounded generation with Claude.
  • Engineer a prompt that constrains Claude to answer only from retrieved context and cite its sources.
  • Secret Mission: Added metadata filtering to scope queries by source and a deletion endpoint for document management.

Ready to quiz yourself?