Build a Local RAG Document Q&A API

Build a local REST API that answers questions about PDFs using RAG.

Introduction

30 Second Summary

Imagine uploading a 200-page PDF and asking it questions in plain English. Instead of scrolling through pages, you get a precise answer with the exact sources it came from.

In this project, you will build a local REST API that accepts PDF uploads, stores their content as vector embeddings, and answers your questions using a local AI model running on your machine.

What You'll Build

A Retrieval-Augmented Generation (RAG) API powered by FastAPI, Ollama, and ChromaDB that ingests PDFs, embeds their content, and answers questions using semantic search and a local LLM.

By the end of this project, you'll have:

  • A document processing module that extracts text from PDFs and splits it into overlapping chunks for retrieval.
  • A vector store powered by SentenceTransformers and ChromaDB that embeds and searches document chunks using semantic similarity.
  • A FastAPI REST API with four endpoints for uploading documents, asking questions, checking status, and resetting the store.
  • Secret Mission: Document management endpoints that let you list uploaded documents and delete specific documents from the vector store.

Are there any prerequisites?

You should be comfortable with Python basics like functions, classes, and imports. Familiarity with REST APIs is helpful but not required.

Before We Start

Before We Start...

Set Up the Project

Before building the RAG pipeline, you need the right tools installed and configured. This step gets your development environment ready with Python, Ollama, and all the packages the project depends on.

You will verify that Python and Ollama are working, download the Llama 3.2 language model, and create a modular project structure. Then you will install your dependencies inside a virtual environment.

In this step, get ready to:
  • Verify Python and Ollama are installed, and pull the Llama 3.2 model.
  • Create the project directory with a modular file structure and virtual environment.
  • Install all Python dependencies from requirements.txt.
Verify tools and pull the Llama 3.2 model

This project depends on two system-level tools: Python (to run the application code) and Ollama (to run the LLM locally). Let's confirm both are ready before going further.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Confirm Python is installed by running this command:
python3 --version

✔️ I see a version number

You're all set. Any version 3.10 or higher works for this project.

ⓧ Command not found

Python is not installed yet. Follow these steps to install it.

  • Go to python.org/downloads and download the latest Python 3.10+ installer for your operating system.
  • Run the installer and follow the prompts. On Windows, make sure to check the Add Python to PATH checkbox.
  • Close and reopen your terminal, then run python3 --version again.
  • Confirm Ollama is installed by running this command:
ollama --version

✔️ I see a version number

Ollama is ready to go.

ⓧ Command not found

Ollama is not installed yet. Follow these steps to install it.

  • Go to ollama.com/download and download the installer for your operating system.
  • Run the installer and follow the prompts.
  • Close and reopen your terminal, then run ollama --version again.

Now you need to download the language model that will generate answers from your document context. Llama 3.2 3B is a lightweight model that runs well on Apple Silicon and most modern machines.

  • Download the Llama 3.2 3B model by running this command:
ollama pull llama3.2

What does this command do?

This downloads the Llama 3.2 3B model (about 2GB) to your machine. Ollama manages the model files locally so you can run the LLM without any cloud API or internet connection after the initial download.

You should see a progress bar followed by a success message once the download completes.

Download stuck or failing?

Make sure Ollama is running in the background. On macOS, check that the Ollama icon appears in your menu bar. If it is not there, open the Ollama app first, then retry the pull command.

If the download stalls, check your internet connection and try again. Ollama resumes partial downloads automatically.

help me troubleshoot my ollama pull llama3.2 command

  • Test the model by running this command:
ollama run llama3.2 "Say hello"

What should I see?

Ollama generates a short greeting and then drops you into an interactive chat session. You will see a >>> prompt where you can type follow-up messages.

  • Type /bye and press Enter to exit the interactive session and return to your normal terminal.

Model not responding?

If you see a connection error, the Ollama server may not be running. Open the Ollama desktop app and wait a few seconds for it to start, then try the command again.

help me troubleshoot Ollama not responding to my run command

Create the project structure and virtual environment

The project uses four Python modules, each with a single responsibility: document processing, vector storage, AI generation, and API routing. Setting this up now keeps the codebase clean as you build.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir rag-document-api
cd rag-document-api

A virtual environment isolates your project's packages from the rest of your system. This prevents version conflicts between different Python projects on your machine.

  • Create and activate a virtual environment by running these commands:

macOS

python3 -m venv venv
source venv/bin/activate

Windows

python -m venv venv
venv\Scripts\activate

What is a virtual environment?

Think of it as a private toolbox for this project. The venv folder holds its own copy of Python and any packages you install with pip. Nothing you install here affects other projects on your computer.

You should see (venv) at the beginning of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

Make sure you ran the activate command (not just the create command). On macOS, run source venv/bin/activate. On Windows, run venv\Scripts\activate.

If you see a permissions error on Windows, open PowerShell as Administrator and run Set-ExecutionPolicy RemoteSigned, then try activating again.

help me activate my Python virtual environment

  • Create the project files by running these commands:

macOS

touch main.py document_processor.py vector_store.py ai_generator.py
touch requirements.txt .gitignore README.md
mkdir uploads

Windows

New-Item main.py, document_processor.py, vector_store.py, ai_generator.py, requirements.txt, .gitignore, README.md -ItemType File
mkdir uploads
  • Confirm the files exist by running:
ls

You should see main.py, document_processor.py, vector_store.py, ai_generator.py, requirements.txt, .gitignore, README.md, and the uploads/ directory in the output.

Missing files in the output?

Make sure you are inside the rag-document-api folder. Run pwd to check your current directory. If you are somewhere else, run cd ~/Desktop/rag-document-api and try ls again.

Note that .gitignore is a hidden file. On macOS, run ls -a to see hidden files.

help me find my project files in rag-document-api

Install dependencies

The project depends on seven Python packages. You will add them all to requirements.txt with pinned versions, then install them in one command.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code (or your preferred code editor) and press Enter to open it.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Select the rag-document-api folder on your Desktop, then click Open.
  • Open requirements.txt from the Explorer sidebar on the left and add the following package list:
fastapi==0.135.3
uvicorn==0.46.0
python-multipart==0.0.27
chromadb==1.5.8
sentence-transformers==5.4.1
pypdf==6.4.1
ollama==0.6.2

What are these packages?

  • fastapi is the web framework for your API endpoints.
  • uvicorn is the server that runs your FastAPI app.
  • python-multipart handles file uploads in FastAPI.
  • chromadb is the vector database that stores your document embeddings.
  • sentence-transformers converts text into vector embeddings for semantic search.
  • pypdf extracts text from PDF files.
  • ollama is the Python client for communicating with your local Ollama server.
  • Save requirements.txt.
  • Install all the packages by running this command in your terminal:
pip install -r requirements.txt

This takes a few minutes

The sentence-transformers package pulls in PyTorch as a dependency, which is a large download. Expect the install to take 2-5 minutes depending on your internet speed.

You should see a Successfully installed message when everything finishes.

Seeing install errors?

Make sure your virtual environment is active. You should see (venv) at the start of your terminal prompt. If it is missing, re-run the activate command from the previous substep.

If you see a pip: command not found error, try pip3 install -r requirements.txt instead.

help me fix my pip install errors for this RAG project

✔️ Awesome, I've got everything!

Great. Double check that you have saved requirements.txt.

ⓧ I'd like to double check the full code

Here is the complete requirements.txt file. Make sure yours matches exactly.

fastapi==0.135.3
uvicorn==0.46.0
python-multipart==0.0.27
chromadb==1.5.8
sentence-transformers==5.4.1
pypdf==6.4.1
ollama==0.6.2

Your environment is ready. Next up, you will build the document processor that extracts text from PDFs and splits it into chunks.

Build the Document Processor

Your environment is ready with all the Python packages installed and Llama 3.2 standing by. The next step is to teach your API how to read PDF documents.

Before the API can answer questions about a document, it needs to extract the raw text and break it into smaller pieces called chunks. These chunks are what get embedded and stored in the vector database for retrieval later.

In this step, get ready to:
  • Write a function that extracts text from PDF files using pypdf.
  • Write a function that splits text into overlapping chunks for vector storage.
Implement PDF text extraction

When a user uploads a PDF to your API, it arrives as raw bytes. Your first function needs to take those bytes, read every page of the PDF, and return all the text as a single string.

  • Open document_processor.py in VS Code and add the following code:
import io
from pypdf import PdfReader


def extract_text_from_pdf(file_bytes: bytes) -> str:
    """Extract all text from a PDF file provided as raw bytes."""
    # Wrap raw bytes in a file-like buffer for PdfReader
    reader = PdfReader(io.BytesIO(file_bytes))
    text = ""
    for page in reader.pages:
        page_text = page.extract_text()
        # Skip pages with no extractable text (e.g. scanned images)
        if page_text:
            text += page_text + "\n"
    return text.strip()

What does this code do?

  • io.BytesIO wraps the raw bytes into a file-like object that PdfReader can read.
  • The for loop iterates over every page in the PDF and extracts its text content.
  • Pages with no extractable text (like scanned images) are silently skipped by the if page_text: check.
  • text.strip() removes any leading or trailing whitespace from the final result.
  • Save document_processor.py.
  • You won't see a visible change yet. The next function completes this module, and you'll test both together.
Implement text chunking with overlap

Now that you can extract text from a PDF, you need to split it into smaller chunks. LLMs have limited context windows, so sending an entire document at once would dilute the relevance of search results.

  • Add the chunk_text function below extract_text_from_pdf in document_processor.py:
def chunk_text(
    text: str, chunk_size: int = 500, overlap: int = 50
) -> list[str]:
    """Split text into overlapping chunks of approximately chunk_size characters."""
    if not text:
        return []

    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end].strip()
        if chunk:
            chunks.append(chunk)
        # Slide the window forward, overlapping with the previous chunk
        start += chunk_size - overlap

    return chunks

What does this code do?

  • The function takes a text string and splits it into chunks of approximately 500 characters each.
  • start += chunk_size - overlap moves the window forward by 450 characters, creating a 50-character overlap between consecutive chunks.
  • The overlap ensures that sentences split at a chunk boundary still appear in full in at least one chunk, preserving context for retrieval.
  • Empty input returns an empty list immediately, avoiding unnecessary processing.

Why use overlapping chunks?

Imagine a sentence that falls right at the boundary between two chunks. Without overlap, half the sentence would be in chunk A and the other half in chunk B. Neither chunk captures the full meaning.

By overlapping each chunk with the previous one by 50 characters, boundary sentences appear intact in at least one chunk. This makes semantic search more reliable when the API retrieves context for a question.

  • Save document_processor.py.
  • Test the chunk_text function by running this in your terminal:
python3 -c "from document_processor import chunk_text; chunks = chunk_text('a' * 1200); print(f'{len(chunks)} chunks created')"

You should see 3 chunks created in the output. A 1200-character string splits into 3 overlapping chunks of 500 characters each.

Seeing a ModuleNotFoundError?

  • Make sure you're running the command from inside the rag-document-api folder where document_processor.py lives.
  • Check that your virtual environment is activated. You should see (venv) at the start of your terminal prompt.
  • If you see a SyntaxError, double-check the parentheses and colons in the function definition match the code above exactly.
  • Why is my document_processor import failing?

✔️ Awesome, I've got everything!

Double check you've saved document_processor.py and you're ready to move on.

ⓧ I'd like to double check the full code

Here is the complete document_processor.py file. Make sure yours matches exactly.

import io
from pypdf import PdfReader


def extract_text_from_pdf(file_bytes: bytes) -> str:
    """Extract all text from a PDF file provided as raw bytes."""
    reader = PdfReader(io.BytesIO(file_bytes))
    text = ""
    for page in reader.pages:
        page_text = page.extract_text()
        if page_text:
            text += page_text + "\n"
    return text.strip()


def chunk_text(
    text: str, chunk_size: int = 500, overlap: int = 50
) -> list[str]:
    """Split text into overlapping chunks of approximately chunk_size characters."""
    if not text:
        return []

    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end].strip()
        if chunk:
            chunks.append(chunk)
        start += chunk_size - overlap

    return chunks

Your document processor is complete. It can read any PDF and break it into retrieval-friendly chunks. Next up, you'll build the vector store that embeds these chunks and makes them searchable.

Build the Vector Store

Your document processor can extract and chunk PDF text. Now you need somewhere to store those chunks so your API can search through them later.

In this step, you'll build a vector store module that converts text chunks into numerical vectors called vector embeddings. When a user asks a question, the store finds the most relevant chunks by comparing the question's embedding against every stored chunk using semantic search.

In this step, get ready to:
  • Set up ChromaDB and load the SentenceTransformers embedding model.
  • Implement methods to add, search, count, and reset chunks in the vector database.
Initialize ChromaDB and the embedding model

The VectorStore class wraps two tools into one clean interface. ChromaDB handles persistent vector storage (so your data survives server restarts). SentenceTransformers provides the embedding model that converts text into 384-dimensional numerical vectors.

  • Open vector_store.py and add the imports and the VectorStore class with its __init__ method by copying the following code:
import uuid
import chromadb
from sentence_transformers import SentenceTransformer


class VectorStore:
    def __init__(
        self,
        persist_dir: str = "./chroma_db",
        collection_name: str = "documents",
    ):
        # Store vectors locally so they survive server restarts
        self.client = chromadb.PersistentClient(path=persist_dir)
        # Reuse existing collection or create a new one
        self.collection = self.client.get_or_create_collection(
            name=collection_name
        )
        # Load the lightweight embedding model (384-dimensional vectors)
        self.embedding_model = SentenceTransformer("all-MiniLM-L6-v2")

What does this code do?

  • PersistentClient(path="./chroma_db") creates a local ChromaDB database inside a chroma_db/ folder. Unlike an in-memory client, this one writes data to disk so your vectors persist across restarts.
  • get_or_create_collection(name="documents") either opens an existing collection or creates a fresh one. This means the class works whether the server is starting for the first time or the tenth.
  • SentenceTransformer("all-MiniLM-L6-v2") loads a 22M-parameter model that produces 384-dimensional embeddings. It is the standard lightweight choice for semantic search.
  • Save vector_store.py.
  • Test that the class initializes by running this command in your terminal:
python3 -c "from vector_store import VectorStore; store = VectorStore(); print('VectorStore ready')"

You should see VectorStore ready printed in the terminal. If this is your first time loading the model, SentenceTransformers downloads it from Hugging Face (~80MB). This is a one-time download.

Seeing an import error or download failure?

  • If you see ModuleNotFoundError: No module named 'chromadb' or a similar message for sentence_transformers, make sure your virtual environment is activated. You should see (venv) at the start of your terminal prompt.
  • If the model download stalls, check your internet connection and try again. The ~80MB download needs a stable connection.
  • Make sure you are running the command from inside the rag-document-api/ folder, not a parent directory.

Why can't Python find my installed packages?

Implement add, search, count, and reset methods

The VectorStore class needs four methods. These methods form the complete interface that the API endpoints will call.

  • In vector_store.py, add the add_chunks method inside the VectorStore class, below the __init__ method:
    def add_chunks(self, chunks: list[str], document_name: str) -> int:
        """Embed and store text chunks in the vector database."""
        if not chunks:
            return 0

        # Convert text chunks into numerical vectors
        embeddings = self.embedding_model.encode(chunks).tolist()
        # Each chunk needs a unique ID for ChromaDB
        ids = [str(uuid.uuid4()) for _ in chunks]
        # Track which document each chunk came from
        metadatas = [
            {"document": document_name, "chunk_index": i}
            for i in range(len(chunks))
        ]

        self.collection.add(
            ids=ids,
            documents=chunks,
            embeddings=embeddings,
            metadatas=metadatas,
        )
        return len(chunks)

What does this method do?

  • self.embedding_model.encode(chunks).tolist() converts each text chunk into a 384-dimensional numerical vector. The .tolist() call converts the numpy array to a plain Python list that ChromaDB accepts.
  • Each chunk gets a random UUID as its identifier, plus metadata that tracks which document it came from and its position in the original text.
  • collection.add() stores the raw text, its embedding, the ID, and the metadata together in one call.
  • Save vector_store.py.
  • You won't see a change yet. The next method (search) uses the chunks you just stored.
  • Add the search method below add_chunks:
    def search(self, query: str, n_results: int = 5) -> list[dict]:
        """Find the most semantically similar chunks to a query."""
        count = self.collection.count()
        if count == 0:
            return []

        # Don't request more results than exist
        n_results = min(n_results, count)
        # Encode the query with the same model used for indexing
        query_embedding = self.embedding_model.encode([query]).tolist()

        results = self.collection.query(
            query_embeddings=query_embedding,
            n_results=n_results,
        )

        # Reshape ChromaDB's nested list format into flat dicts
        chunks = []
        for i in range(len(results["ids"][0])):
            chunks.append(
                {
                    "text": results["documents"][0][i],
                    "metadata": results["metadatas"][0][i],
                    "distance": results["distances"][0][i],
                }
            )
        return chunks

What does the search method do?

  • The query text is encoded with the same all-MiniLM-L6-v2 model used during indexing. Using the same model for both steps is critical. Different models produce incompatible embeddings.
  • collection.query() compares the query embedding against every stored chunk and returns the closest matches. The distance score tells you how similar each result is (lower is more similar).
  • ChromaDB returns results in a nested list format. The loop at the end reshapes these into a clean list of dictionaries that the rest of the codebase can use easily.
  • Save vector_store.py.
  • You won't see a change yet. The last two helper methods complete the class.
  • Add the get_chunk_count and reset methods below search:
    def get_chunk_count(self) -> int:
        """Return the total number of chunks stored."""
        return self.collection.count()

    def reset(self) -> None:
        """Delete all chunks and recreate an empty collection."""
        self.client.delete_collection(name=self.collection.name)
        self.collection = self.client.get_or_create_collection(
            name="documents"
        )

What do these methods do?

  • get_chunk_count() is a thin wrapper around collection.count(). The API's status endpoint uses this to report how many chunks are stored.
  • reset() deletes the entire collection and immediately recreates an empty one. This gives users a clean slate without restarting the server.
  • Save vector_store.py.
  • Test the full class by running this command:
python3 -c "
from vector_store import VectorStore
store = VectorStore()
store.add_chunks(['The sky is blue', 'Python is great'], 'test.pdf')
print('Chunks stored:', store.get_chunk_count())
results = store.search('color of sky')
print('Top result:', results[0]['text'])
store.reset()
print('After reset:', store.get_chunk_count())
"

You should see three lines of output: Chunks stored: 2, then Top result: The sky is blue, then After reset: 0. The search correctly found the "sky" chunk even though the query used the word "color" instead of "blue." That is semantic search in action.

Test not returning the expected output?

  • If you see IndentationError, make sure all four methods are indented one level inside the VectorStore class. Each def line should start with exactly 4 spaces.
  • If you see AttributeError: 'VectorStore' object has no attribute 'add_chunks', the method is either not inside the class or the file was not saved before running the test.
  • If the search returns the wrong chunk, double check that you pasted the test command exactly as shown. The query color of sky should match the "sky is blue" chunk.

Why is my VectorStore test failing?

✔️ Awesome, I've got everything!

Great. Double check you've saved vector_store.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete vector_store.py file. Make sure your version matches this exactly.

import uuid
import chromadb
from sentence_transformers import SentenceTransformer


class VectorStore:
    def __init__(
        self,
        persist_dir: str = "./chroma_db",
        collection_name: str = "documents",
    ):
        self.client = chromadb.PersistentClient(path=persist_dir)
        self.collection = self.client.get_or_create_collection(
            name=collection_name
        )
        self.embedding_model = SentenceTransformer("all-MiniLM-L6-v2")

    def add_chunks(self, chunks: list[str], document_name: str) -> int:
        """Embed and store text chunks in the vector database."""
        if not chunks:
            return 0

        embeddings = self.embedding_model.encode(chunks).tolist()
        ids = [str(uuid.uuid4()) for _ in chunks]
        metadatas = [
            {"document": document_name, "chunk_index": i}
            for i in range(len(chunks))
        ]

        self.collection.add(
            ids=ids,
            documents=chunks,
            embeddings=embeddings,
            metadatas=metadatas,
        )
        return len(chunks)

    def search(self, query: str, n_results: int = 5) -> list[dict]:
        """Find the most semantically similar chunks to a query."""
        count = self.collection.count()
        if count == 0:
            return []

        n_results = min(n_results, count)
        query_embedding = self.embedding_model.encode([query]).tolist()

        results = self.collection.query(
            query_embeddings=query_embedding,
            n_results=n_results,
        )

        chunks = []
        for i in range(len(results["ids"][0])):
            chunks.append(
                {
                    "text": results["documents"][0][i],
                    "metadata": results["metadatas"][0][i],
                    "distance": results["distances"][0][i],
                }
            )
        return chunks

    def get_chunk_count(self) -> int:
        """Return the total number of chunks stored."""
        return self.collection.count()

    def reset(self) -> None:
        """Delete all chunks and recreate an empty collection."""
        self.client.delete_collection(name=self.collection.name)
        self.collection = self.client.get_or_create_collection(
            name="documents"
        )

Your vector store can embed, store, search, and reset document chunks. Next up, you'll build the AI generator that takes those search results and feeds them to Llama 3.2 to produce grounded answers.

Build the AI Generator

Your vector store can embed document chunks and retrieve the most relevant ones for any query. Now you need the module that takes those retrieved chunks and turns them into a human-readable answer.

In this step, you'll build the AI generator. It packages retrieved chunks into a carefully designed prompt and sends everything to Llama 3.2 running on Ollama.

In this step, get ready to:
  • Design a grounding prompt that prevents LLM hallucinations.
  • Implement the generate_answer function that calls Ollama.
Design the RAG prompt

The most important part of any RAG system is the prompt that talks to the LLM. Without constraints, an LLM will generate plausible-sounding answers from its training data, even when the answer should come from your specific documents.

Your prompt needs two parts. The system prompt sets the rules: answer ONLY from the provided context, admit when the context is insufficient, and never fabricate information. The user prompt combines the actual document chunks with the user's question, separated by --- dividers so the model can distinguish between different source passages.

Why does the grounding prompt matter?

Without a grounding prompt, the LLM draws from its training data instead of your documents. The system prompt constrains it to the provided context only.

This is the core mechanism that makes RAG reliable. It separates a useful document Q&A tool from a hallucination machine.

Implement the generate_answer function
  • Open ai_generator.py and add the full AI generator module by copying the following code:
from ollama import chat


def generate_answer(
    question: str,
    context_chunks: list[dict],
    model: str = "llama3.2",
) -> str:
    """Generate an answer grounded in the provided context chunks."""
    # Join all chunk texts with dividers so the model sees them as separate passages
    context = "\n\n---\n\n".join(
        [chunk["text"] for chunk in context_chunks]
    )

    # System prompt: constrain the model to answer from context only
    system_prompt = (
        "You are a helpful document assistant. Answer the user's question "
        "based ONLY on the provided context. If the context does not contain "
        "enough information to answer the question, say so clearly. Do not "
        "make up information. Reference which parts of the context support "
        "your answer."
    )

    # Combine context and question into the user message
    user_prompt = f"Context:\n{context}\n\nQuestion: {question}"

    # Send both messages to Llama 3.2 via Ollama's local API
    response = chat(
        model=model,
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt},
        ],
    )

    return response.message.content

What does this code do?

  • from ollama import chat imports the chat function from the Ollama Python library. This handles the connection to your locally running Ollama server.
  • The context variable takes all the retrieved chunk texts and joins them with --- dividers. This lets the model distinguish between different source passages.
  • The system_prompt instructs the model to answer ONLY from the provided context and to admit when it doesn't have enough information.
  • The user_prompt combines the context and the question into a single formatted string using an f-string.
  • chat(model=model, messages=[...]) sends the system and user messages to Llama 3.2 via Ollama's local API and returns the generated text.
  • Save ai_generator.py.
  • Verify the module loads without errors by running:
python -c "from ai_generator import generate_answer; print('Module loaded successfully')"

You should see Module loaded successfully in your terminal.

Seeing an import error?

  • Make sure your virtual environment is still activated. You should see (venv) at the start of your terminal prompt.
  • If you see ModuleNotFoundError: No module named 'ollama', run pip install ollama==0.6.2 to reinstall it.
  • Check for typos in the import line. It should be from ollama import chat exactly.

Help me fix an import error in my ai_generator.py file.

✔️ Awesome, I've got everything!

Great. Double check you've saved ai_generator.py and you're ready to move on.

ⓧ I'd like to double check the full code

Here is the complete ai_generator.py file. Compare it with yours line by line.

from ollama import chat


def generate_answer(
    question: str,
    context_chunks: list[dict],
    model: str = "llama3.2",
) -> str:
    """Generate an answer grounded in the provided context chunks."""
    context = "\n\n---\n\n".join(
        [chunk["text"] for chunk in context_chunks]
    )

    system_prompt = (
        "You are a helpful document assistant. Answer the user's question "
        "based ONLY on the provided context. If the context does not contain "
        "enough information to answer the question, say so clearly. Do not "
        "make up information. Reference which parts of the context support "
        "your answer."
    )

    user_prompt = f"Context:\n{context}\n\nQuestion: {question}"

    response = chat(
        model=model,
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt},
        ],
    )

    return response.message.content

Your AI generator is ready. Next up, you'll wire everything together into a FastAPI application with endpoints that the outside world can call.

Build the FastAPI Endpoints

You now have three standalone modules: one that processes PDFs, one that stores and searches vector embeddings, and one that generates answers with Ollama. On their own, they are just Python files sitting on disk.

In this step, you will wire all three modules into a FastAPI application that exposes them as REST API endpoints. Anyone (or any frontend) can then upload PDFs, ask questions, and manage the vector store over HTTP.

In this step, get ready to:
  • Create the FastAPI app with CORS middleware and Pydantic data models.
  • Implement four API endpoints: upload, ask, status, and reset.
  • Add a server entry point so you can launch the API with a single command.
Create the FastAPI app with CORS and models

Every FastAPI application starts with an app instance, middleware configuration, and data models that define the shape of requests and responses.

  • Open main.py and add the imports, app setup, CORS middleware, shared store instance, and Pydantic models by copying this code:
from fastapi import FastAPI, UploadFile, File, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
import uvicorn

# Import the three modules you built
from document_processor import extract_text_from_pdf, chunk_text
from vector_store import VectorStore
from ai_generator import generate_answer

# Create the FastAPI app with metadata for Swagger docs
app = FastAPI(
    title="RAG Document Q&A API",
    description=(
        "Upload PDFs and ask questions answered by a local LLM "
        "using Retrieval-Augmented Generation."
    ),
    version="1.0.0",
)

# Allow any frontend origin to call this API
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# Single shared instance so the embedding model loads once at startup
store = VectorStore()


# Define the shape of question requests
class AskRequest(BaseModel):
    question: str
    n_results: int = 5


# Define the shape of question responses
class AskResponse(BaseModel):
    answer: str
    sources: list[dict]

What does this code do?

  • The FastAPI() call creates the app and sets the title, description, and version that appear in the auto-generated Swagger docs.
  • CORSMiddleware with allow_origins=["*"] lets any frontend domain call your API. Without this, browsers block cross-origin requests.
  • store = VectorStore() creates one shared instance at startup. This means the SentenceTransformers embedding model loads once, not on every request.
  • AskRequest and AskResponse are Pydantic models. FastAPI uses them to validate incoming JSON and to generate the response schema in the docs.

What is CORS?

CORS stands for Cross-Origin Resource Sharing. Browsers enforce a security rule that blocks a web page on one domain from calling an API on a different domain.

Adding CORS middleware tells your API to include special headers in its responses that say "it's OK, let this request through." Without it, any frontend trying to call your API would be blocked.

  • Save main.py.

You won't see a visible change yet. The next chunk adds the endpoints that make this app functional.

Implement the four API endpoints

Your API needs four endpoints: one to upload PDFs, one to ask questions, one to check system health, and one to reset the vector store.

  • In main.py, add the /upload endpoint below the AskResponse class by copying this code:
@app.post("/upload")
async def upload_pdf(file: UploadFile = File(...)):
    """Upload a PDF file, extract text, chunk it, and store embeddings."""
    # Reject non-PDF files immediately
    if not file.filename.endswith(".pdf"):
        raise HTTPException(
            status_code=400, detail="Only PDF files are accepted."
        )

    # Read the uploaded file and extract text
    file_bytes = await file.read()
    text = extract_text_from_pdf(file_bytes)

    if not text:
        raise HTTPException(
            status_code=400,
            detail="Could not extract text from this PDF.",
        )

    # Chunk the text and store embeddings in ChromaDB
    chunks = chunk_text(text)
    num_chunks = store.add_chunks(chunks, document_name=file.filename)

    return {
        "filename": file.filename,
        "chunks_added": num_chunks,
        "total_chunks": store.get_chunk_count(),
    }

What does the upload endpoint do?

This endpoint is the entry point for the entire RAG pipeline. It handles every stage from raw file to stored vectors in one request.

  • UploadFile = File(...) tells FastAPI to expect a multipart file upload. The python-multipart package you installed earlier handles the parsing.
  • The two HTTPException calls act as guardrails. One rejects non-PDF files. The other catches PDFs that contain no extractable text (like scanned images without OCR).
  • The return value gives the caller a summary: the filename, how many chunks were created, and the total chunk count across all uploads.
  • Save main.py.

You won't see a change yet. The next chunk adds the remaining three endpoints.

  • Add the /ask, /status, and /reset endpoints below the upload_pdf function by copying this code:
@app.post("/ask", response_model=AskResponse)
async def ask_question(request: AskRequest):
    """Ask a question and get an AI-generated answer from your documents."""
    if store.get_chunk_count() == 0:
        raise HTTPException(
            status_code=400, detail="No documents uploaded yet."
        )

    # Retrieve relevant chunks and generate a grounded answer
    sources = store.search(request.question, n_results=request.n_results)
    answer = generate_answer(request.question, sources)

    return AskResponse(answer=answer, sources=sources)


@app.get("/status")
async def get_status():
    """Check system health and the number of stored chunks."""
    return {
        "status": "healthy",
        "chunks_stored": store.get_chunk_count(),
    }


@app.delete("/reset")
async def reset_store():
    """Clear all documents and chunks from the vector store."""
    store.reset()
    return {"message": "Vector store cleared successfully."}

What do these three endpoints do?

  • POST /ask is the question-answering endpoint. It checks that at least one document has been uploaded, searches the vector store for relevant chunks, passes them to generate_answer, and returns the AI response alongside the source chunks.
  • GET /status is a health check. It returns whether the system is running and how many chunks are stored. This is useful for monitoring and for frontends that need to know if documents are loaded.
  • DELETE /reset clears the entire vector store. It calls the reset() method you built in vector_store.py, which deletes and recreates the ChromaDB collection.
  • Save main.py.

You won't see a change yet. The next chunk adds the server entry point so you can actually run the API.

Add the server entry point

FastAPI apps need a server to accept incoming HTTP requests. Uvicorn is a lightweight ASGI server that pairs with FastAPI. Adding a __main__ block lets you start the server by running python main.py directly.

  • Add the entry point at the very bottom of main.py, below the reset_store function:
if __name__ == "__main__":
    # reload=True restarts the server when you save code changes
    uvicorn.run("main:app", host="0.0.0.0", port=8000, reload=True)

What does this code do?

  • "main:app" tells Uvicorn to look in the file main.py for the object named app.
  • host="0.0.0.0" makes the server accessible on all network interfaces, not just localhost.
  • reload=True enables hot-reloading during development. Every time you save a file, the server restarts automatically.
  • Save main.py.
  • Start the server by running this command:
python main.py

The first time you run this, the SentenceTransformers embedding model downloads from Hugging Face (about 80MB). This is a one-time download that gets cached locally.

  • Open http://localhost:8000/docs in your browser. You should see the Swagger UI with all four endpoints listed: /upload, /ask, /status, and /reset.

What is Swagger UI?

Swagger UI is interactive API documentation that FastAPI generates automatically from your code. Each endpoint shows its HTTP method, URL, expected request body, and response schema.

You can test endpoints directly from the browser by clicking **Try it out**. The title, description, and version you set in the FastAPI() constructor appear at the top of the page.

Server not starting?

  • If you see ModuleNotFoundError, make sure your virtual environment is activated. You should see (venv) at the start of your terminal prompt.
  • If you see Address already in use, another process is using port 8000. Stop the other process or change the port number in the uvicorn.run() call.
  • If you see a connection error referencing Ollama, make sure the Ollama desktop app is running in the background. The server will still start, but the /ask endpoint will fail without it.

Help me debug my FastAPI server startup error.

  • Press Ctrl+C in the terminal to stop the server when you are done checking.

✔️ Awesome, I've got everything!

Great. Double check you have saved main.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete main.py file. Compare it with yours line by line.

from fastapi import FastAPI, UploadFile, File, HTTPException
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel
import uvicorn

from document_processor import extract_text_from_pdf, chunk_text
from vector_store import VectorStore
from ai_generator import generate_answer

app = FastAPI(
    title="RAG Document Q&A API",
    description=(
        "Upload PDFs and ask questions answered by a local LLM "
        "using Retrieval-Augmented Generation."
    ),
    version="1.0.0",
)

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

store = VectorStore()


class AskRequest(BaseModel):
    question: str
    n_results: int = 5


class AskResponse(BaseModel):
    answer: str
    sources: list[dict]


@app.post("/upload")
async def upload_pdf(file: UploadFile = File(...)):
    """Upload a PDF file, extract text, chunk it, and store embeddings."""
    if not file.filename.endswith(".pdf"):
        raise HTTPException(
            status_code=400, detail="Only PDF files are accepted."
        )

    file_bytes = await file.read()
    text = extract_text_from_pdf(file_bytes)

    if not text:
        raise HTTPException(
            status_code=400,
            detail="Could not extract text from this PDF.",
        )

    chunks = chunk_text(text)
    num_chunks = store.add_chunks(chunks, document_name=file.filename)

    return {
        "filename": file.filename,
        "chunks_added": num_chunks,
        "total_chunks": store.get_chunk_count(),
    }


@app.post("/ask", response_model=AskResponse)
async def ask_question(request: AskRequest):
    """Ask a question and get an AI-generated answer from your documents."""
    if store.get_chunk_count() == 0:
        raise HTTPException(
            status_code=400, detail="No documents uploaded yet."
        )

    sources = store.search(request.question, n_results=request.n_results)
    answer = generate_answer(request.question, sources)

    return AskResponse(answer=answer, sources=sources)


@app.get("/status")
async def get_status():
    """Check system health and the number of stored chunks."""
    return {
        "status": "healthy",
        "chunks_stored": store.get_chunk_count(),
    }


@app.delete("/reset")
async def reset_store():
    """Clear all documents and chunks from the vector store."""
    store.reset()
    return {"message": "Vector store cleared successfully."}


if __name__ == "__main__":
    uvicorn.run("main:app", host="0.0.0.0", port=8000, reload=True)

Your API is live and the Swagger docs confirm all four endpoints are registered. Next up, you will test the full RAG pipeline end-to-end by uploading a real PDF and asking it questions.

Test the Full Pipeline

All four Python modules are complete and wired together through your FastAPI endpoints. You have a full RAG pipeline ready to test.

In this step, you'll start the server and fire real requests at every endpoint. You'll upload a PDF, ask questions about it, and see AI-generated answers come back with source chunks.

In this step, get ready to:
  • Start the FastAPI server and explore the auto-generated Swagger docs.
  • Test the full upload, ask, and reset flow with curl.
Start the server and verify Swagger docs

FastAPI auto-generates interactive API documentation at /docs. You can explore and test every endpoint right from the browser without writing any client code.

  • Make sure your virtual environment is still activated inside your rag-document-api folder. You should see (venv) at the start of your terminal prompt.
  • Start the FastAPI server by running this command:
python main.py

What happens on first start?

The server loads the SentenceTransformers embedding model into memory. If this is the first run, it downloads the model (~80MB) from Hugging Face. This is a one-time download.

Once the server is ready, you'll see uvicorn output ending with Application startup complete.

  • Open your browser and go to http://localhost:8000/docs.

You should see the Swagger UI with all four endpoints listed: POST /upload, POST /ask, GET /status, and DELETE /reset.

  • Click on the GET /status row to expand it.
  • Click Try it out.
  • Click Execute.

The response body should show {"status": "healthy", "chunks_stored": 0}. Zero chunks stored is correct because you haven't uploaded any documents yet.

Server won't start?

  • If you see a ModuleNotFoundError, your virtual environment is not activated. Run source venv/bin/activate from inside your rag-document-api folder.
  • If you see Address already in use, another process is using port 8000. Stop it or change the port number in main.py.
  • If startup seems to hang, the SentenceTransformer model is downloading. Give it a minute to finish.
  • My FastAPI server won't start. What should I check?
Test the upload and ask flow with curl

The Swagger UI is great for exploring your API. For testing the full pipeline with a real PDF, you'll use curl commands in a separate terminal window.

  • Open a new terminal window. Keep the server running in your first terminal.
  • Find any PDF on your machine. A research paper, a textbook, or any document with text content works.
  • Upload it to the API by running this command:
curl -X POST "http://localhost:8000/upload" -F "file=@[[PDF_PATH=\"path/to/your-document.pdf\"]]"

What does this command do?

  • -X POST sends a POST request to the upload endpoint.
  • -F "file=@..." attaches your PDF as a file upload. The @ symbol tells curl to read from a file path on your machine.

You should see a JSON response with the filename, the number of chunks added, and the total chunk count.

  • Make sure Ollama is running in the background. If it is not, open the Ollama desktop app or run ollama serve in another terminal window.
  • Ask a question about your uploaded document by running this command:
curl -X POST "http://localhost:8000/ask" \
  -H "Content-Type: application/json" \
  -d '{"question": "What is this document about?"}'

What should I see?

The response includes two parts. The answer field contains the AI-generated answer from Llama 3.2. The sources field lists the chunks that were retrieved as context.

The answer should directly reference content from your PDF. If the model says it cannot find relevant information, try a more specific question related to your document's content.

First question takes longer

Ollama loads the Llama 3.2 model into memory on the first request. Subsequent questions respond much faster.

  • Test the reset endpoint by running this command:
curl -X DELETE "http://localhost:8000/reset"

You should see {"message": "Vector store cleared successfully."} in the response. This confirms the vector store has been wiped clean.

Getting an error on /ask?

  • If you see a connection error, Ollama is not running. Open the Ollama desktop app or run ollama serve in a separate terminal.
  • If you see No documents uploaded yet, you need to upload a PDF first with the /upload endpoint before asking questions.
  • If the upload returned zero chunks, your PDF may be a scanned image without extractable text. Try a different PDF that contains selectable text.
  • My /ask endpoint is returning an error. What should I check?

Your RAG pipeline is working end-to-end. Next up, you'll finalize the project for GitHub with a .gitignore, a professional README.md, and your first commit.

Finalize for GitHub

You tested every endpoint and confirmed the full RAG pipeline works end to end. Your API is ready for the world to see.

In this step, you'll prepare your project for GitHub. A clean .gitignore keeps generated files out of version control. A professional README.md tells visitors what your project does and how to run it.

In this step, get ready to:
  • Write a .gitignore to exclude generated files from version control.
  • Write a README.md with architecture, setup, and usage documentation.
  • Initialize a Git repository and push to GitHub.
Write the .gitignore

Your project generates files that should never be tracked in Git. The virtual environment alone contains thousands of dependency files. The chroma_db/ directory holds binary vector data that is specific to your machine.

  • Open .gitignore in your text editor and paste the following:
venv/
.venv/
__pycache__/
*.pyc
*.pyo
chroma_db/
uploads/
.env
*.egg-info/
dist/
build/
.DS_Store

What does this file do?

Each line is a pattern that tells Git to ignore matching files and directories:

  • venv/ and .venv/ exclude Python virtual environments with all their installed packages.
  • chroma_db/ excludes the local vector database. Anyone who clones the repo will rebuild it by uploading their own PDFs.
  • .env prevents environment variable files with secrets from being committed.
  • Save .gitignore.

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

venv/
.venv/
__pycache__/
*.pyc
*.pyo
chroma_db/
uploads/
.env
*.egg-info/
dist/
build/
.DS_Store
Write the README

The README.md is the first thing anyone sees when they visit your GitHub repo. It needs to explain what the project does, how to set it up, and how to use it.

  • Open README.md and paste the following:
# RAG Document Q&A API

A fully local, production-style RAG (Retrieval-Augmented Generation) API built with FastAPI, Ollama, and ChromaDB.

## Architecture

PDF Upload → Text Extraction → Chunking → Embedding (SentenceTransformers) → ChromaDB
Question → Embedding → Semantic Search → Context Retrieval → Llama 3.2 (Ollama) → Answer

## Prerequisites

- Python 3.10+
- Ollama installed with `llama3.2` model pulled

## Installation

```bash
git clone https://github.com/YOUR_USERNAME/rag-document-api.git
cd rag-document-api
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```

## Usage

```bash
python main.py
```

Upload a PDF:
```bash
curl -X POST "http://localhost:8000/upload" -F "file=@your-document.pdf"
```

Ask a question:
```bash
curl -X POST "http://localhost:8000/ask" -H "Content-Type: application/json" -d '{"question": "What is this document about?"}'
```

## API Endpoints

- `POST /upload` - Upload a PDF
- `POST /ask` - Ask a question
- `GET /status` - System health
- `DELETE /reset` - Clear all documents

## Project Structure

```
rag-document-api/
├── main.py
├── document_processor.py
├── vector_store.py
├── ai_generator.py
├── requirements.txt
└── README.md
```

What does this README cover?

A strong project README walks a visitor through the full picture in a logical order:

  • Architecture shows the two data flows (upload and query) as a text-based diagram so readers instantly understand the system design.
  • Installation and Usage give anyone the exact commands to clone, set up, and run the project.
  • API Endpoints is a quick reference table so users do not have to read the source code to find the available routes.
  • Replace YOUR_USERNAME in the Installation section with your actual GitHub username.
  • Save README.md.

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

# RAG Document Q&A API

A fully local, production-style RAG (Retrieval-Augmented Generation) API built with FastAPI, Ollama, and ChromaDB.

## Architecture

PDF Upload → Text Extraction → Chunking → Embedding (SentenceTransformers) → ChromaDB
Question → Embedding → Semantic Search → Context Retrieval → Llama 3.2 (Ollama) → Answer

## Prerequisites

- Python 3.10+
- Ollama installed with `llama3.2` model pulled

## Installation

```bash
git clone https://github.com/YOUR_USERNAME/rag-document-api.git
cd rag-document-api
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
```

## Usage

```bash
python main.py
```

Upload a PDF:
```bash
curl -X POST "http://localhost:8000/upload" -F "file=@your-document.pdf"
```

Ask a question:
```bash
curl -X POST "http://localhost:8000/ask" -H "Content-Type: application/json" -d '{"question": "What is this document about?"}'
```

## API Endpoints

- `POST /upload` - Upload a PDF
- `POST /ask` - Ask a question
- `GET /status` - System health
- `DELETE /reset` - Clear all documents

## Project Structure

```
rag-document-api/
├── main.py
├── document_processor.py
├── vector_store.py
├── ai_generator.py
├── requirements.txt
└── README.md
```
Initialize Git and push to GitHub

With your files ready, it is time to create a Git repository and push your code to GitHub. This makes your project accessible to anyone with the link.

  • Initialize your Git repository and make your first commit by running these commands:
git init
git add .
git commit -m "Initial commit: RAG Document Q&A API"

What do these commands do?

  • git init creates a new Git repository in your project folder.
  • git add . stages every file that is not excluded by .gitignore.
  • git commit saves the staged files as your first snapshot with a descriptive message.
  • You should see output listing all the files that were committed. Confirm that venv/ and chroma_db/ do not appear in the list. Your .gitignore is doing its job.

Seeing an error about git?

  • If you see git: command not found, Git is not installed. Install it from git-scm.com and try again.
  • If you see a message about configuring your name and email, run git config --global user.name "Your Name" and git config --global user.email "you@example.com" first.

Help me fix a Git error when committing my project.

Now you need to create a repository on GitHub to push your code to.

  • Open github.com in your browser and sign in.
  • Click the + icon in the top right corner.
  • Click New repository.
  • Enter rag-document-api as the repository name.
  • Leave the repository set to Public.
  • Do not check any of the initialization options (README, .gitignore, or license). You already have these files locally.
  • Click Create repository.
  • Back in your terminal, connect your local repo to GitHub and push by running these commands:
git remote add origin https://github.com/[[YOUR_USERNAME="your GitHub username"]]/rag-document-api.git
git branch -M main
git push -u origin main

What do these commands do?

  • git remote add origin links your local repository to the GitHub repository you just created.
  • git branch -M main renames your default branch to main.
  • git push -u origin main uploads all your committed files to GitHub.
  • Refresh your GitHub repository page in the browser. You should see all your project files listed: main.py, document_processor.py, vector_store.py, ai_generator.py, requirements.txt, .gitignore, and README.md.
  • Your rendered README should appear below the file list with the architecture diagram and usage instructions.

Push failed or authentication error?

  • If you see remote: Permission denied, GitHub no longer accepts password authentication. You need a personal access token or SSH key. Follow GitHub's authentication guide to set one up.
  • If you see fatal: remote origin already exists, you may have already added a remote. Remove it first with git remote remove origin and try again.

Help me fix a GitHub push error.

Secret mission

Add Document Management Endpoints

Your API can upload and query documents, but what about managing them? Add endpoints to list all uploaded documents and delete specific ones from the vector store.

Before you go

Clean Up Your Resources

This project runs entirely on your local machine, so there are no ongoing cloud costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Uvicorn dev server (running process on port 8000).
  • Python virtual environment (venv/ folder inside rag-document-api).
  • ChromaDB persistent data (chroma_db/ folder inside rag-document-api).
  • Ollama Llama 3.2 model (~2GB stored locally by Ollama).
  • SentenceTransformers cached model (~80MB in ~/.cache/huggingface/).
  • Project files on your Desktop (rag-document-api/ folder).
  • GitHub repository.

Keep everything running

No action needed. Choose this if you are still actively building or want to keep experimenting with your RAG API.

  • Your Uvicorn server stays running. You can keep uploading PDFs and asking questions at http://localhost:8000/docs.
  • ChromaDB keeps your embedded document chunks on disk, so they survive server restarts.
  • Ollama continues running in the background, ready to serve Llama 3.2 requests.

Pause - I'll come back to this later

Shut down running processes to free up memory, but keep all your files and data intact so you can resume later.

  • Stop the Uvicorn server by pressing Ctrl+C in the terminal where it is running.
  • Quit the Ollama desktop app. Right-click (or Ctrl-click) the Ollama icon in your menu bar and select Quit Ollama.
  • Deactivate your virtual environment by running deactivate in your terminal.
  • Your project files, ChromaDB data, and downloaded models all stay on disk. To resume, reactivate your virtual environment with source venv/bin/activate, relaunch Ollama, and run python main.py.

Delete - I don't want to use this again

Remove all project resources and start fresh. This frees up several gigabytes of disk space.

  • Stop the Uvicorn server by pressing Ctrl+C in the terminal where it is running.
  • Deactivate the virtual environment by running deactivate.
  • Delete the entire project folder from your Desktop. This removes the virtual environment, ChromaDB data, uploads, and all project files.
  • Remove the Llama 3.2 model from Ollama by running ollama rm llama3.2 in your terminal. This frees up about 2GB.
  • Remove the cached SentenceTransformers model by deleting the ~/.cache/huggingface/ directory. This frees up about 80MB.
  • Optionally, delete the GitHub repository from your GitHub account under Settings at the bottom of the repository page.

Mission accomplished

That's a Wrap!

Nice work! You just built a fully local RAG API from scratch. Everything runs on your machine with zero API costs.

You've learned how to:

  • Implemented the complete RAG pipeline: PDF ingestion, character-based chunking with overlap, SentenceTransformers embedding, ChromaDB vector storage, semantic retrieval, and grounded LLM generation with Ollama.
  • Created a FastAPI REST API with four endpoints and auto-generated Swagger documentation that interviewers can explore instantly.
  • Designed a grounding prompt that constrains LLM answers to the provided document context, preventing hallucinations.
  • Added document management endpoints for listing and deleting individual documents from the vector store.

Ready to quiz yourself?