Build a Local RAG API

Build a private RAG API with FastAPI, Ollama, PostgreSQL, and pgvector.

Introduction

30 Second Summary

Private workplace policies can be hard to search when they live across separate documents. Sending those documents to a hosted AI service can also conflict with privacy requirements.

In this project, you will build a private REST API for Trailhead Labs with FastAPI. It keeps every document on your Mac while retrieval-augmented generation grounds Ollama answers in policy evidence from PostgreSQL.

What You'll Build

Picture a private policy question returning the grounded answer three days per week beside remote-work.md as its visible source.

By the end of this project, you'll have:

  • Interactive API documentation for submitting private-policy questions from your browser.
  • A vector-backed knowledge base that ranks the closest policy evidence for each question.
  • A grounded answer that returns three days per week with remote-work.md listed in its sources.
  • Secret Mission: Add an optional topic filter that limits the policy rows the vector query can retrieve.

Are there any prerequisites?

This project is designed for an Apple Silicon Mac with under 16 GB of memory. No prior setup is required.

The guide covers Python, Visual Studio Code, Ollama, PostgreSQL, the local models, and every required dependency.

Before We Start

This first checkpoint connects every installation and database query to a clear purpose. You are committing to a private Trailhead Labs knowledge API that answers policy questions using local documents only.

Install the Native Local Stack

Your private knowledge API currently has nowhere to run its code. It also has no local model or database to search.

This step gives your Mac a native development stack. Every service stays on your computer.

In this step, get ready to:
  • Install the code editor plus the required Python version.
  • Run two local Ollama models for embeddings plus answer generation.
  • Start PostgreSQL plus create the local knowledge database.

Why Use a Native Stack?

A native stack runs the local model service plus database directly through macOS. This avoids the additional virtual machine used by Docker.

The result uses less memory on an Apple Silicon Mac with limited capacity.

Install Visual Studio Code and Python

Visual Studio Code gives you an editor for the API files you create later. Python runs the API plus its local integration code.

  • Press Cmd+Space to open Spotlight.
  • Type Visual Studio Code.
  • Check whether Visual Studio Code appears in the results.

✔️ Visual Studio Code is installed

  • Select Visual Studio Code from the Spotlight results.
  • Press Cmd+Shift+P to open the Command Palette.
  • Type shell command.
  • Select Shell Command: Install 'code' command in PATH.

ⓧ Visual Studio Code is not installed

  • Download the Apple silicon disk image from the official Visual Studio Code page.
  • Open the downloaded .dmg file in Finder.
  • Drag Visual Studio Code.app into the Applications folder.

Why Move the App?

The Applications folder gives Visual Studio Code a stable location. The terminal command points back to that installed application.

  • Press Cmd+Space to open Spotlight.
  • Type Visual Studio Code.
  • Press Enter to open the app.
  • Press Cmd+Shift+P to open the Command Palette.
  • Type shell command.
  • Select Shell Command: Install 'code' command in PATH.
  • Press Cmd+Space to open Spotlight.
  • Type Terminal.
  • Press Enter to open a fresh Terminal window.
  • Confirm the editor command works by running:
code .

What Does This Command Do?

The code command launches Visual Studio Code from Terminal. The dot asks it to display the current folder.

You should see Visual Studio Code open with the current folder available in its Explorer sidebar.

Python 3.14.8 provides the runtime used by every project script. Checking first tells you which installation path applies.

  • Check your active Python version by running:
python3 --version

What Does This Command Check?

This asks the active python3 executable to print its version. The result confirms which Python your Terminal can access.

✔️ I see Python 3.14.8

Your Python runtime already matches the project. Continue to the final check below.

ⓧ I see an older Python version

The older executable cannot provide the pinned runtime for this project. Install Python 3.14.8 beside the existing system version.

  • Open the official Python 3.14.8 release page.
  • Select macOS installer from the files table.
  • Open the downloaded .pkg file.
  • Complete the macOS installer with its default options.
  • Close the current Terminal window.

ⓧ Command not found

Terminal cannot find a Python installation yet. The official installer adds the required runtime.

  • Open the official Python 3.14.8 release page.
  • Select macOS installer from the files table.
  • Open the downloaded .pkg file.
  • Complete the macOS installer with its default options.
  • Close the current Terminal window.
  • Press Cmd+Space to open Spotlight.
  • Type Terminal.
  • Press Enter to open a new Terminal window.
  • Verify the selected Python runtime by running:
python3 --version

Why Check Again?

A fresh Terminal reloads its executable paths. This check proves that python3 now resolves to the project version.

You should see Python 3.14.8 in the output.

Still Seeing the Wrong Python Version?

Close every Terminal window after the installer completes. Open a fresh window before checking again.

Confirm that you downloaded the installer from the Python 3.14.8 release page.

Help me fix my Python version.

Install Ollama and its models

Ollama runs model inference on your Mac. EmbeddingGemma converts text into vectors.

Gemma 3 1B generates the final answers. Using a small model keeps the project practical on a Mac with under 16 GB of memory.

  • Check whether Ollama is installed by running:
ollama --version

What Does This Command Check?

This asks the local Ollama service to report its installed version. The result determines whether the required application is available.

✔️ I see Ollama 0.40.3

Your Ollama application matches the project. Keep it running from the macOS menu bar.

ⓧ I see an older Ollama version

  • Quit the running Ollama application from its macOS menu bar item.
  • Download Ollama 0.40.3 from the official macOS page.
  • Open the downloaded ollama.dmg file.
  • Drag Ollama.app into the Applications folder.
  • Press Cmd+Space to open Spotlight.
  • Type Ollama.
  • Press Enter to start Ollama.
  • Allow Ollama to make its command available in Terminal if macOS requests permission.

ⓧ Command not found

  • Download Ollama 0.40.3 from the official macOS page.
  • Open the downloaded ollama.dmg file.
  • Drag Ollama.app into the Applications folder.
  • Press Cmd+Space to open Spotlight.
  • Type Ollama.
  • Press Enter to start Ollama.
  • Allow Ollama to make its command available in Terminal if macOS requests permission.

The first model run can spend several minutes downloading its files. Quiet progress during this download does not mean Ollama has stalled.

  • Run a test embedding with EmbeddingGemma by entering:
ollama run embeddinggemma "Hello world"

What Does This Test Prove?

The command downloads embeddinggemma when needed. It then converts the test text into numeric coordinates.

Those coordinates become the searchable representation of each policy chunk later.

You should see a JSON array containing decimal numbers. That array proves local embedding generation works.

The chat model also downloads on its first run. Expect this command to take several minutes before the prompt becomes interactive.

  • Start the Gemma 3 1B session by running:
ollama run gemma3:1b

What Does This Command Start?

The command loads gemma3:1b into the local Ollama service. The interactive session lets you confirm that the model can generate text.

  • Type Reply with ready at the interactive prompt.
  • Press Enter to submit the prompt.

You should see a short response that includes the requested word. The response proves local answer generation works.

  • Press Ctrl+D to exit the model session.
  • Confirm both downloaded models by running:
ollama ls

What Does This List Show?

The ollama ls command lists models stored on your Mac. This confirms that later scripts can call each model without downloading it again.

You should see entries for embeddinggemma plus gemma3:1b.

Model Download Failed?

Keep Ollama running from the macOS menu bar. A closed app leaves the Terminal command without a local service.

Check your internet connection before retrying the model command. The first run must download the model files.

Help me fix my Ollama model download.

Install PostgreSQL and create the database

Postgres.app packages a local PostgreSQL server as a macOS application. The server gives your API durable storage for policy text plus vectors.

  • Check whether PostgreSQL command-line tools are available by running:
psql --version

What Does This Command Check?

The psql client prints the version available through your Terminal path. The result confirms whether the required database tools are ready.

✔️ I see PostgreSQL 18.6

Your PostgreSQL command-line tools match the project. Keep Postgres.app running while you continue.

ⓧ I see an older PostgreSQL version

  • Download the 123 MB Postgres.app 2.9.6 package from the official downloads page.
  • Drag Postgres.app into the Applications folder.
  • Press Cmd+Space to open Spotlight.
  • Type Postgres.
  • Press Enter to open Postgres.app.
  • Click Initialize to create the PostgreSQL server.

The next command adds Postgres.app's tools to the macOS search path. macOS may ask for your local login password.

  • Configure the PostgreSQL command-line path by running:
sudo mkdir -p /etc/paths.d && echo /Applications/Postgres.app/Contents/Versions/latest/bin | sudo tee /etc/paths.d/postgresapp

What Does This Command Do?

The first part ensures that macOS has a directory for system path entries. The second part records Postgres.app's binary folder there.

A fresh Terminal can then locate tools such as psql plus createdb.

ⓧ Command not found

  • Download the 123 MB Postgres.app 2.9.6 package from the official downloads page.
  • Drag Postgres.app into the Applications folder.
  • Press Cmd+Space to open Spotlight.
  • Type Postgres.
  • Press Enter to open Postgres.app.
  • Click Initialize to create the PostgreSQL server.

The next command adds Postgres.app's tools to the macOS search path. macOS may ask for your local login password.

  • Configure the PostgreSQL command-line path by running:
sudo mkdir -p /etc/paths.d && echo /Applications/Postgres.app/Contents/Versions/latest/bin | sudo tee /etc/paths.d/postgresapp

What Does This Command Do?

The first part ensures that macOS has a directory for system path entries. The second part records Postgres.app's binary folder there.

A fresh Terminal can then locate tools such as psql plus createdb.

  • Close the current Terminal window.
  • Press Cmd+Space to open Spotlight.
  • Type Terminal.
  • Press Enter to open a new Terminal window.
  • Create the local knowledge database by running:
createdb rag_api

What Does This Command Create?

The createdb tool asks the running PostgreSQL server to create rag_api. This database stores the policy table you build later.

  • Confirm the database accepts a SQL query by running:
psql -d rag_api -c "SELECT version();"

What Does This Query Prove?

The -d rag_api option connects to your new database. The SELECT version(); query asks its server to identify itself.

A returned version row proves that the database exists plus the server is responding.

You should see a version row containing PostgreSQL 18.6.

Database Command Failed?

Keep Postgres.app open while you run database commands. Its server must remain active.

Open a fresh Terminal after configuring the path. An existing Terminal does not reload the new path entry.

Help me fix my local PostgreSQL setup.

Before you run the final check, what versions plus model names do you expect the new Terminal to report?

  • Verify the complete native stack by running these checks:
python3 --version
ollama ls
psql --version

What Does the Final Check Cover?

  • The first command confirms the API runtime.
  • The second command confirms both local models remain available.
  • The third command confirms PostgreSQL tools are available in Terminal.

You should see Python 3.14.8 from the first check. The model list should include embeddinggemma plus gemma3:1b.

The last check should report PostgreSQL 18.6. That confirms the native stack is ready for your local API.

Your Mac can now run Python code, generate local model responses, plus store private policy data. Next, you expose the model's knowledge gap through your first API endpoint.

Expose the Model Knowledge Gap

Your native local stack is ready. The next question is whether Gemma 3 1B can answer a private company-policy question on its own.

You'll create a small FastAPI endpoint that forwards each question directly to Ollama. The result gives you a baseline with no supplied policy evidence.

In this step, get ready to:
  • Create an isolated project with pinned dependencies.
  • Build a typed endpoint that forwards questions to the local model.
  • Test the endpoint against a private-policy question.
Set up the Python project

A Python virtual environment keeps this project's packages separate from the rest of your Mac. The requirements file pins every dependency to a known version.

  • Switch back to Terminal from the previous step.
  • Move to your Desktop by running this command:
cd ~/Desktop

Why start on the Desktop?

The Desktop gives your project a predictable location. You can find the folder quickly from Terminal or Finder.

  • Create the project folder by running these commands:
mkdir local-rag-api
cd local-rag-api
code .

What do these commands do?

  • The first command creates the local-rag-api folder on your Desktop.
  • The second command moves Terminal into that folder.
  • The final command opens the current folder in Visual Studio Code.

You'll see the empty local-rag-api folder in the Explorer sidebar.

Did the project folder fail to open?

Confirm that Terminal is inside local-rag-api. Check that the folder appears on your Desktop.

If the editor stays closed, confirm that the code command remains available in this Terminal session.

Help me open the project folder

  • Create requirements.txt inside local-rag-api using the Explorer sidebar.
  • Paste the pinned package list below into requirements.txt:
fastapi==0.143.0
httpx==0.28.1
psycopg[binary]==3.3.6
pydantic==2.14.1
uvicorn==0.54.0

What are these packages for?

  • FastAPI defines the web application and its endpoint.
  • HTTPX sends asynchronous HTTP requests to the local model.
  • Psycopg connects Python to PostgreSQL in later steps.
  • Pydantic validates the request and response data.
  • Uvicorn serves the FastAPI application locally.
  • Save requirements.txt.

You'll see requirements.txt in the Explorer sidebar with all five package pins.

Is requirements.txt missing?

Confirm that the file sits directly inside local-rag-api. Check the spelling for an accidental extra file extension.

Help me check requirements.txt

✔️ Awesome, I've got everything!

Your dependency pins match the project.

ⓧ I'd like to double check the full code

fastapi==0.143.0
httpx==0.28.1
psycopg[binary]==3.3.6
pydantic==2.14.1
uvicorn==0.54.0
  • Switch back to Terminal from earlier.
  • Create the virtual environment by running this command:
python3 -m venv .venv

What does this command do?

The command creates an isolated environment inside .venv. Packages installed there stay scoped to this project.

  • Return to the Explorer sidebar in Visual Studio Code.

You'll see a new .venv folder inside local-rag-api.

Is the virtual environment missing?

Confirm that Terminal is inside local-rag-api. Rerun the environment command from that folder if needed.

Help me create the virtual environment

  • Switch back to Terminal from earlier.
  • Activate the environment and install the pinned dependencies by running these commands:
source .venv/bin/activate
python -m pip install -r requirements.txt

What happens during installation?

  • The first command activates the environment for this Terminal session.
  • The second command asks pip to read every package pin from requirements.txt.
  • The environment keeps the installed packages inside .venv.

The command prints download and installation progress before returning to a prompt. Your Terminal prompt also shows that .venv is active.

Did a dependency fail to install?

Confirm that .venv appears in your Terminal prompt. Check your internet connection before retrying the installation.

Help me fix the dependency installation

Your isolated project environment is ready. Every package needed by the first API version now lives inside .venv.

Build the direct model endpoint

The first API version needs a request model and a response model. It also needs the local Ollama address plus the chat model name.

  • Create app.py inside local-rag-api using the Explorer sidebar.
  • Paste the application setup below into app.py:
import httpx
from fastapi import FastAPI
from pydantic import BaseModel

OLLAMA_BASE_URL = "http://localhost:11434"
CHAT_MODEL = "gemma3:1b"

app = FastAPI(title="Trailhead Labs Local RAG API")


class AskRequest(BaseModel):
    question: str


class AskResponse(BaseModel):
    answer: str

What does this setup define?

  • OLLAMA_BASE_URL stores the address of the local Ollama service.
  • CHAT_MODEL selects the local chat model.
  • app creates the FastAPI application shown in the interactive docs.
  • AskRequest validates the incoming question.
  • AskResponse guarantees that the endpoint returns an answer string.
  • Save app.py.
  • Switch back to the activated Terminal from earlier.
  • Start the baseline application by running this command:
uvicorn app:app --reload

What does this command do?

Uvicorn loads the app object from app.py. Reload mode watches the file so saved changes restart the local server automatically.

You'll see Uvicorn start the application on http://127.0.0.1:8000. This Terminal remains occupied while the server is running.

Did the server fail to start?

Confirm that .venv is active in the Terminal prompt. Check that Terminal is inside local-rag-api.

Help me start Uvicorn

  • Return to app.py in Visual Studio Code.
  • Add the endpoint below after the AskResponse class:
@app.post("/ask", response_model=AskResponse)
async def ask(request: AskRequest) -> AskResponse:
    async with httpx.AsyncClient(timeout=120.0) as client:
        response = await client.post(
            f"{OLLAMA_BASE_URL}/api/chat",
            json={
                "model": CHAT_MODEL,
                "messages": [{"role": "user", "content": request.question}],
                "stream": False,
            },
        )
        response.raise_for_status()
        return AskResponse(answer=response.json()["message"]["content"])

How does the endpoint work?

  • The ask() function receives the validated request body.
  • AsyncClient sends the question to Ollama without blocking the API process.
  • messages forwards the question as the only user message.
  • stream uses one complete JSON response.
  • AskResponse extracts the model's text into the API response.
  • Save app.py.

You'll see Uvicorn reload the application in the running Terminal. The new /ask route is now available.

Did Uvicorn reject the updated file?

Compare the indentation inside ask() with the code block above. Check every closing bracket in the request body.

Help me fix app.py

✔️ Awesome, I've got everything!

Your direct endpoint matches the first working API version.

ⓧ I'd like to double check the full code

import httpx
from fastapi import FastAPI
from pydantic import BaseModel

OLLAMA_BASE_URL = "http://localhost:11434"
CHAT_MODEL = "gemma3:1b"

app = FastAPI(title="Trailhead Labs Local RAG API")


class AskRequest(BaseModel):
    question: str


class AskResponse(BaseModel):
    answer: str


@app.post("/ask", response_model=AskResponse)
async def ask(request: AskRequest) -> AskResponse:
    async with httpx.AsyncClient(timeout=120.0) as client:
        response = await client.post(
            f"{OLLAMA_BASE_URL}/api/chat",
            json={
                "model": CHAT_MODEL,
                "messages": [{"role": "user", "content": request.question}],
                "stream": False,
            },
        )
        response.raise_for_status()
        return AskResponse(answer=response.json()["message"]["content"])
Test the private-policy question

FastAPI generates interactive documentation from your request model and route definition. You can use that page to send a real request without building a separate client.

  • Visit http://127.0.0.1:8000/docs in your web browser.

You'll see the Trailhead Labs API documentation with a row for POST /ask.

  • Select the POST /ask row.
  • Click Try it out.
  • Replace the request body with the following JSON:
{
  "question": "How many days per week can Trailhead Labs employees work remotely?"
}

What is this request testing?

The question field matches AskRequest. The endpoint forwards this private-policy question directly to the local model.

Before you send the request, do you think the model can provide a trustworthy answer with a policy citation?

  • Click Execute to send the question.

You'll see an answer field in the response. The response has no sources field or remote-work.md citation.

The model may guess a policy, refuse the question, or admit that it lacks the information. Even a correct-sounding answer has no evidence behind it.

Did the request fail completely?

  • Confirm that the Uvicorn Terminal remains running.
  • Confirm that Ollama remains active from the previous step.
  • Compare the request body with the JSON block above.

Help me test the direct endpoint

That knowledge gap is now visible in a working API. Next, you'll give PostgreSQL a vector-backed place to store the missing policy evidence.

Give PostgreSQL a Vector Memory

Your local FastAPI endpoint can reach Ollama. Its answer exposed that the model has no record of Trailhead Labs policy.

The failed direct answer needs durable evidence that supports similarity search. PostgreSQL will hold that evidence in a pgvector-backed table.

In this step, get ready to:
  • Write the SQL schema for the vector-backed knowledge base.
  • Apply the schema to the rag_api database.
  • Inspect the documents table to verify its columns.
Create the vector schema

An embedding is a fixed-length list of numbers that represents meaning. The vector(128) type gives every policy chunk the same shape.

  • Select the new-file icon in the file sidebar for local-rag-api.
  • Name the file schema.sql.

You should see an empty schema.sql editor tab inside the open project.

  • Define the database structure by pasting this code into schema.sql:
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE IF NOT EXISTS documents (
    id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    source text NOT NULL,
    topic text NOT NULL,
    content text NOT NULL,
    embedding vector(128) NOT NULL
);

What Does This Schema Do?

  • CREATE EXTENSION makes pgvector's vector type available to this database.
  • id gives each document row an automatically generated primary key.
  • The three text columns keep each policy chunk tied to its source metadata.
  • embedding vector(128) reserves space for a required 128-dimensional embedding.
  • Save schema.sql by pressing Cmd+S.
  • Confirm schema.sql appears in the file sidebar beside app.py and requirements.txt.

Schema File Missing?

  • Check that the file name ends with .sql.
  • Confirm the file sits directly inside local-rag-api.
  • Save the file again after correcting its name or location.

Help me find my schema file.

✔️ Awesome, I've got everything!

Your saved schema.sql contains the complete vector table definition.

ⓧ I'd like to double check the full code

  • Compare your saved schema.sql with this complete file.
CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE IF NOT EXISTS documents (
    id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
    source text NOT NULL,
    topic text NOT NULL,
    content text NOT NULL,
    embedding vector(128) NOT NULL
);

What Should Match?

These two statements are the complete contents of schema.sql. The first enables vector support.

The second creates the empty documents table with its required columns.

Apply the schema to PostgreSQL

The saved file describes the target database structure. PostgreSQL creates that structure when it executes the statements.

Uvicorn is still using your original terminal session. A second session keeps the API available while you update the database.

  • Create a second terminal session from the terminal area in Visual Studio Code.
  • Confirm its prompt path ends with local-rag-api.
  • Apply the saved schema to the database by running this command:
psql -d rag_api -f schema.sql

What Does This Command Do?

  • -d rag_api selects the local project database.
  • -f schema.sql tells psql to execute the saved schema file.

You should see confirmation lines for both SQL statements. Good progress. The rag_api database now owns an empty vector-ready table.

Schema Command Failed?

  • Confirm the PostgreSQL server from earlier is still running.
  • Confirm schema.sql is listed in the open local-rag-api folder.
  • Retry the command from the second terminal session.

Help me apply my PostgreSQL schema.

Inspect the documents table

A table definition is the fastest proof that the database created the intended structure. This check confirms the metadata fields plus the embedding dimensions before any rows are inserted.

Before you inspect it, which columns do you expect the database to report?

  • Inspect the table definition by running this command:
psql -d rag_api -c "\d documents"

What Does This Command Do?

\d documents asks psql to describe the table. The result shows each column's type plus its constraints.

You should see id, source, topic, content, and embedding in the table definition. The embedding row should use vector(128).

That's the storage layer ready. PostgreSQL can now preserve policy text beside its future vector representation.

Table Definition Missing?

  • Confirm the schema command completed before this inspection.
  • Confirm both commands target the rag_api database.
  • Compare your saved schema with the full-file reference above.

Help me inspect my documents table.

Your PostgreSQL knowledge base now has the right shape. Next, you'll fill it with policy chunks and prove that cosine search finds the relevant evidence.

Seed and Search the Knowledge Base

Your PostgreSQL table is ready, but it is empty. An empty knowledge base leaves the model with the same private-policy gap you exposed in the previous step.

You will use Ollama to create 128-dimensional embeddings for three policy chunks. Psycopg will store them before a raw cosine query proves that vector search can find the remote-work evidence.

In this step, get ready to:
  • Define three Trailhead Labs policy chunks.
  • Embed the policy chunks before inserting them into PostgreSQL.
  • Run a cosine query that ranks the remote-work policy first.
Define the policy chunks

Each database row needs policy text plus metadata that identifies its source. The embedding turns that text into numbers that PostgreSQL can compare by meaning.

  • In the VS Code Explorer sidebar from earlier, click the New File icon.
  • Enter seed.py in the file name field.
  • Press Enter.
  • Add the script setup to seed.py by pasting this code:
import asyncio
import json

import httpx
import psycopg

OLLAMA_BASE_URL = "http://localhost:11434"
EMBED_MODEL = "embeddinggemma"
DATABASE_DSN = "dbname=rag_api"
EMBEDDING_DIMENSIONS = 128

What Does This Setup Provide?

  • The imports provide asynchronous requests, JSON serialization, HTTP calls, and database access.
  • OLLAMA_BASE_URL points the script to the local Ollama service.
  • EMBEDDING_DIMENSIONS keeps every generated vector aligned with the database column's vector(128) type.
  • Save seed.py.
  • Confirm seed.py appears beside app.py in the VS Code Explorer sidebar.

Missing seed.py?

Check that the file sits directly inside local-rag-api. A nested file will run from the wrong location.

Help me locate the new seed file.

  • Add the policy data below EMBEDDING_DIMENSIONS = 128 by pasting this code:
DOCUMENTS = [
    {
        "source": "remote-work.md",
        "topic": "benefits",
        "content": (
            "Trailhead Labs employees may work remotely up to three days per week. "
            "Teams coordinate their shared office day with their team lead."
        ),
    },
    {
        "source": "expenses.md",
        "topic": "finance",
        "content": (
            "Expense reports must be submitted within 30 calendar days of purchase. "
            "A receipt is required for any single expense above 25 dollars."
        ),
    },
    {
        "source": "security.md",
        "topic": "security",
        "content": (
            "Suspected phishing messages must be reported to the security team within one hour. "
            "Do not forward the suspicious message to coworkers."
        ),
    },
]

How Are the Policies Structured?

  • source records the document name that the final API can show as evidence.
  • topic adds a category that can constrain retrieval later.
  • content contains the policy text that becomes an embedding.
  • Save seed.py.
  • Confirm the DOCUMENTS list contains remote-work.md, expenses.md, and security.md.

Seeing Python syntax warnings?

Check that every policy entry has matching braces. Confirm the DOCUMENTS list ends with a closing square bracket.

Help me check the policy list syntax.

Embed and store the policies

An embedding represents the meaning of a text chunk as a vector. The same embedding model must process the stored documents plus the search question so their vectors are comparable.

  • Add the embedding helper below the DOCUMENTS list by pasting this code:
def vector_literal(values: list[float]) -> str:
    return json.dumps(values, separators=(",", ":"))


async def main() -> None:
    async with httpx.AsyncClient(timeout=120.0) as client:
        document_response = await client.post(
            f"{OLLAMA_BASE_URL}/api/embed",
            json={
                "model": EMBED_MODEL,
                "input": [document["content"] for document in DOCUMENTS],
                "dimensions": EMBEDDING_DIMENSIONS,
            },
        )
        document_response.raise_for_status()
        document_embeddings = document_response.json()["embeddings"]

        question = "How many days per week can Trailhead Labs employees work remotely?"
        question_response = await client.post(
            f"{OLLAMA_BASE_URL}/api/embed",
            json={
                "model": EMBED_MODEL,
                "input": question,
                "dimensions": EMBEDDING_DIMENSIONS,
            },
        )
        question_response.raise_for_status()
        question_embedding = question_response.json()["embeddings"][0]

How Does Embedding Work Here?

  • vector_literal() serializes each Python list into a vector literal that PostgreSQL can accept.
  • /api/embed creates one 128-dimensional embedding for each policy chunk.
  • The second request embeds the remote-work question with the same model.
  • Save seed.py.
  • Confirm vector_literal() aligns with the left edge of the file.
  • Confirm the request code remains indented inside main().

Seeing indentation warnings?

Keep the HTTP request lines nested inside async with. Python uses indentation to define both the client session and the main() function.

Help me fix the embedding block indentation.

  • Add the row builder below question_embedding by pasting this code:
    rows = [
        (
            document["source"],
            document["topic"],
            document["content"],
            vector_literal(embedding),
        )
        for document, embedding in zip(DOCUMENTS, document_embeddings, strict=True)
    ]

    with psycopg.connect(DATABASE_DSN) as connection:
        with connection.cursor() as cursor:
            cursor.execute("DELETE FROM documents")
            cursor.executemany(
                """
                INSERT INTO documents (source, topic, content, embedding)
                VALUES (%s, %s, %s, %s::vector)
                """,
                rows,
            )

How Does the Insert Stay Safe?

  • zip() pairs each policy dictionary with the embedding created from its content.
  • DELETE FROM documents gives every run the same final set of three rows.
  • executemany() inserts all rows through parameterized SQL placeholders.
  • Save seed.py.
  • Confirm the row builder remains indented inside main().
  • Confirm cursor.executemany() receives rows after the SQL string.

Does the insert block look misaligned?

Align rows = with the earlier async with statement. Nest each cursor operation one level inside the cursor block.

Help me align the database insert block.

Run the cosine search

Cosine distance compares the direction of two vectors. A smaller distance means the question sits closer to a policy chunk in the embedding space.

  • Add the search query below the insertion block by pasting this code:
    query_vector = vector_literal(question_embedding)
    with psycopg.connect(DATABASE_DSN) as connection:
        result = connection.execute(
            """
            SELECT source, 1 - (embedding <=> %s::vector) AS similarity
            FROM documents
            ORDER BY embedding <=> %s::vector
            LIMIT 1
            """,
            (query_vector, query_vector),
        ).fetchone()

    print(f"Seeded {len(rows)} document chunks.")
    if result is not None:
        print(f"Top match: {result[0]} (similarity: {float(result[1]):.3f})")


if __name__ == "__main__":
    asyncio.run(main())

How Does the Search Rank Evidence?

  • <=> calculates cosine distance between each stored embedding and the question vector.
  • 1 - cosine distance converts the distance into a similarity score.
  • LIMIT 1 returns only the strongest match.
  • asyncio.run(main()) starts the complete seeding workflow when you run the file.
  • Save seed.py.

✔️ Awesome, I've got everything!

Your complete seed.py now defines the policies, generates their embeddings, inserts the rows, and searches for the closest source.

ⓧ I'd like to double check the full code

Compare your complete seed.py with this reference before running it.

import asyncio
import json

import httpx
import psycopg

OLLAMA_BASE_URL = "http://localhost:11434"
EMBED_MODEL = "embeddinggemma"
DATABASE_DSN = "dbname=rag_api"
EMBEDDING_DIMENSIONS = 128

DOCUMENTS = [
    {
        "source": "remote-work.md",
        "topic": "benefits",
        "content": (
            "Trailhead Labs employees may work remotely up to three days per week. "
            "Teams coordinate their shared office day with their team lead."
        ),
    },
    {
        "source": "expenses.md",
        "topic": "finance",
        "content": (
            "Expense reports must be submitted within 30 calendar days of purchase. "
            "A receipt is required for any single expense above 25 dollars."
        ),
    },
    {
        "source": "security.md",
        "topic": "security",
        "content": (
            "Suspected phishing messages must be reported to the security team within one hour. "
            "Do not forward the suspicious message to coworkers."
        ),
    },
]


def vector_literal(values: list[float]) -> str:
    return json.dumps(values, separators=(",", ":"))


async def main() -> None:
    async with httpx.AsyncClient(timeout=120.0) as client:
        document_response = await client.post(
            f"{OLLAMA_BASE_URL}/api/embed",
            json={
                "model": EMBED_MODEL,
                "input": [document["content"] for document in DOCUMENTS],
                "dimensions": EMBEDDING_DIMENSIONS,
            },
        )
        document_response.raise_for_status()
        document_embeddings = document_response.json()["embeddings"]

        question = "How many days per week can Trailhead Labs employees work remotely?"
        question_response = await client.post(
            f"{OLLAMA_BASE_URL}/api/embed",
            json={
                "model": EMBED_MODEL,
                "input": question,
                "dimensions": EMBEDDING_DIMENSIONS,
            },
        )
        question_response.raise_for_status()
        question_embedding = question_response.json()["embeddings"][0]

    rows = [
        (
            document["source"],
            document["topic"],
            document["content"],
            vector_literal(embedding),
        )
        for document, embedding in zip(DOCUMENTS, document_embeddings, strict=True)
    ]

    with psycopg.connect(DATABASE_DSN) as connection:
        with connection.cursor() as cursor:
            cursor.execute("DELETE FROM documents")
            cursor.executemany(
                """
                INSERT INTO documents (source, topic, content, embedding)
                VALUES (%s, %s, %s, %s::vector)
                """,
                rows,
            )

    query_vector = vector_literal(question_embedding)
    with psycopg.connect(DATABASE_DSN) as connection:
        result = connection.execute(
            """
            SELECT source, 1 - (embedding <=> %s::vector) AS similarity
            FROM documents
            ORDER BY embedding <=> %s::vector
            LIMIT 1
            """,
            (query_vector, query_vector),
        ).fetchone()

    print(f"Seeded {len(rows)} document chunks.")
    if result is not None:
        print(f"Top match: {result[0]} (similarity: {float(result[1]):.3f})")


if __name__ == "__main__":
    asyncio.run(main())

Before you run the seed script, predict which policy chunk cosine search will rank first for the remote-work question.

  • Return to the terminal from earlier.
  • Seed the database before running the cosine search with this command:
python seed.py

What Does This Run Do?

The script embeds all three policy chunks before replacing the table's existing rows. It then embeds the test question before ranking the stored vectors by cosine distance.

You should see Seeded 3 document chunks. followed by a line beginning with Top match: remote-work.md. The similarity value appears at the end of the second line.

You have now proved that semantic retrieval can find the private policy that the direct model call could not reliably know.

Script failed or returned another source?

Confirm Ollama remains available before checking that the embeddinggemma model is present. Confirm PostgreSQL remains available before checking that rag_api contains the documents table.

Help me debug the seed and search result.

Your knowledge base now stores searchable policy evidence. Next, you will connect this retrieval result to the API so every answer can include grounded source metadata.

Turn Retrieval into Grounded Answers

Your PostgreSQL cosine search now identifies the right policy chunk. However, the API still sends each question directly to the model without that evidence.

In this step, you'll complete the retrieval-augmented generation workflow. FastAPI retrieves local evidence before Ollama generates a grounded answer.

In this step, get ready to:
  • Embed each incoming question with the same local model used for the policy chunks.
  • Retrieve the three closest documents with parameterized cosine SQL.
  • Return a grounded answer with visible source metadata.
Embed incoming questions

Each question needs a 128-dimensional embedding before it can be compared with the stored policy vectors. The API must use the same model plus the same dimensions as seed.py.

  • In app.py, replace the import section plus the constants through CHAT_MODEL with this expanded configuration:
import asyncio
import json

import httpx
import psycopg
from fastapi import FastAPI
from pydantic import BaseModel

OLLAMA_BASE_URL = "http://localhost:11434"
EMBED_MODEL = "embeddinggemma"
CHAT_MODEL = "gemma3:1b"
DATABASE_DSN = "dbname=rag_api"
EMBEDDING_DIMENSIONS = 128

What does this configuration add?

  • The asyncio import lets the API move synchronous database work into a worker thread.
  • The json import converts Python lists into vector literals for PostgreSQL.
  • Psycopg connects the API to the local rag_api database.
  • EMBED_MODEL plus EMBEDDING_DIMENSIONS keep question vectors compatible with the stored document vectors.
  • Save app.py.
  • Switch back to the terminal with the activated virtual environment.
  • Start the development server by running this command:
uvicorn app:app --reload

What does this command do?

Uvicorn imports the app object from app.py. Reload mode watches the file for saved changes during the rest of this step.

You should see the server start without an import error. Keep this terminal running so each saved edit reloads the API.

Server stopped during import?

Confirm the terminal still shows (.venv) before its prompt. Check that psycopg[binary]==3.3.6 remains in requirements.txt.

Compare the import names with the code above if the terminal identifies a missing module or name.

Help me diagnose the import failure.

  • In app.py, add these helpers directly below the existing AskResponse class:
def vector_literal(values: list[float]) -> str:
    return json.dumps(values, separators=(",", ":"))


async def embed_text(text: str) -> list[float]:
    async with httpx.AsyncClient(timeout=120.0) as client:
        response = await client.post(
            f"{OLLAMA_BASE_URL}/api/embed",
            json={"model": EMBED_MODEL, "input": text, "dimensions": EMBEDDING_DIMENSIONS},
        )
        response.raise_for_status()
        return response.json()["embeddings"][0]

How does question embedding work?

  • vector_literal() serializes an embedding into the format accepted by the vector query.
  • HTTPX sends the question to Ollama's local embedding endpoint.
  • embed_text() returns the first 128-dimensional vector from the response.
  • Save app.py.
  • Return to the Uvicorn terminal to confirm the server reloads without a Python error.

The successful reload proves the API can import its new embedding helper. The direct question route remains available while you add retrieval.

Reload failing after the helper?

Check that vector_literal() plus embed_text() sit outside every class. Their def keywords must begin at the left edge of app.py.

Help me fix the embedding helper.

Retrieve local evidence

The question vector becomes useful when pgvector compares it with the three stored document vectors. A parameterized SQL query keeps the vector plus the result limit separate from the SQL text.

  • Add the retrieval function directly below embed_text() in app.py:
def retrieve_sources(query_embedding: list[float], limit: int = 3) -> list[dict]:
    query_vector = vector_literal(query_embedding)
    with psycopg.connect(DATABASE_DSN) as connection:
        rows = connection.execute(
            """SELECT source, topic, content, 1 - (embedding <=> %s::vector) AS similarity FROM documents ORDER BY embedding <=> %s::vector LIMIT %s""",
            (query_vector, query_vector, limit),
        ).fetchall()
    return [{"source": row[0], "topic": row[1], "content": row[2], "similarity": float(row[3])} for row in rows]

How does retrieval rank the sources?

  • retrieve_sources() converts the question embedding into a PostgreSQL vector literal.
  • The <=> operator orders rows by cosine distance.
  • The expression beginning with 1 - converts each distance into a similarity value for the response.
  • The parameter tuple binds both vector uses plus the limit value without formatting them into the SQL string.
  • Save app.py.
  • Return to the Uvicorn terminal to confirm the retrieval function loads without an error.

The API now has a database function that returns source names plus topics. Each result also carries its policy text plus cosine similarity.

Database function blocking the reload?

Check that the SQL string has three %s placeholders. Confirm the parameter tuple contains query_vector twice plus limit once.

Help me inspect the retrieval function.

  • Add this health route immediately above the existing @app.post("/ask", response_model=AskResponse) decorator:
@app.get("/health")
async def health() -> dict[str, str]:
    return {"status": "ok"}

Why add a health route?

The /health route gives you a quick check that Uvicorn has loaded the edited application. It isolates server health from embedding plus database work.

  • Save app.py.
  • Enter http://127.0.0.1:8000/health in your browser's address bar.

You should see a JSON response with status set to ok. This confirms the updated API remains available.

Health check not loading?

Confirm the Uvicorn terminal is still running. Check that the health route sits above the /ask route without extra indentation.

Help me restore the health route.

Generate and verify grounded answers

Retrieval now supplies the evidence that the direct route lacked. The final route labels that evidence by source before asking the chat model to answer only from the supplied context.

  • Replace the existing AskResponse class with these response models:
class Source(BaseModel):
    source: str
    topic: str
    similarity: float


class AskResponse(BaseModel):
    answer: str
    sources: list[Source]

What changes in the response?

Source validates the metadata returned for each retrieved row. AskResponse now exposes both the generated answer plus the evidence behind it.

  • Replace the existing ask() route with this grounded version:
@app.post("/ask", response_model=AskResponse)
async def ask(request: AskRequest) -> AskResponse:
    query_embedding = await embed_text(request.question)
    sources = await asyncio.to_thread(retrieve_sources, query_embedding)
    if not sources:
        return AskResponse(answer="I do not have enough local context to answer that question.", sources=[])
    context = "\n\n".join(f"[{item['source']}]\n{item['content']}" for item in sources)
    async with httpx.AsyncClient(timeout=120.0) as client:
        response = await client.post(
            f"{OLLAMA_BASE_URL}/api/chat",
            json={"model": CHAT_MODEL, "messages": [{"role": "system", "content": "Answer only from the supplied context. If the context does not contain the answer, say so. Cite source labels in square brackets."}, {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {request.question}"}], "stream": False, "options": {"temperature": 0}},
        )
        response.raise_for_status()
        answer = response.json()["message"]["content"]
    return AskResponse(answer=answer, sources=[Source(source=item["source"], topic=item["topic"], similarity=item["similarity"]) for item in sources])

How does the grounded route work?

  • embed_text() converts the incoming question into a vector.
  • asyncio.to_thread() runs the synchronous database query outside the asynchronous request loop.
  • context labels each retrieved policy chunk with its source filename.
  • The system message restricts generation to the supplied evidence. The final response converts every retrieved row into validated Source metadata.
  • Save app.py.
  • Return to the Uvicorn terminal to confirm the complete route reloads without an error.

The successful reload means the request model plus retrieval path can be imported together. Your next API request tests the complete workflow.

Route failing during reload?

Check that the old ask() function was fully replaced. Confirm there is only one @app.post("/ask", response_model=AskResponse) decorator in app.py.

Help me fix the grounded route.

✔️ Awesome, I've got everything!

Great. Double check that app.py is saved before you test the grounded response.

ⓧ I'd like to double check the full code

Compare your complete app.py with this reference:

import asyncio
import json

import httpx
import psycopg
from fastapi import FastAPI
from pydantic import BaseModel

OLLAMA_BASE_URL = "http://localhost:11434"
EMBED_MODEL = "embeddinggemma"
CHAT_MODEL = "gemma3:1b"
DATABASE_DSN = "dbname=rag_api"
EMBEDDING_DIMENSIONS = 128

app = FastAPI(title="Trailhead Labs Local RAG API")


class AskRequest(BaseModel):
    question: str


class Source(BaseModel):
    source: str
    topic: str
    similarity: float


class AskResponse(BaseModel):
    answer: str
    sources: list[Source]


def vector_literal(values: list[float]) -> str:
    return json.dumps(values, separators=(",", ":"))


async def embed_text(text: str) -> list[float]:
    async with httpx.AsyncClient(timeout=120.0) as client:
        response = await client.post(
            f"{OLLAMA_BASE_URL}/api/embed",
            json={"model": EMBED_MODEL, "input": text, "dimensions": EMBEDDING_DIMENSIONS},
        )
        response.raise_for_status()
        return response.json()["embeddings"][0]


def retrieve_sources(query_embedding: list[float], limit: int = 3) -> list[dict]:
    query_vector = vector_literal(query_embedding)
    with psycopg.connect(DATABASE_DSN) as connection:
        rows = connection.execute(
            """SELECT source, topic, content, 1 - (embedding <=> %s::vector) AS similarity FROM documents ORDER BY embedding <=> %s::vector LIMIT %s""",
            (query_vector, query_vector, limit),
        ).fetchall()
    return [{"source": row[0], "topic": row[1], "content": row[2], "similarity": float(row[3])} for row in rows]


@app.get("/health")
async def health() -> dict[str, str]:
    return {"status": "ok"}


@app.post("/ask", response_model=AskResponse)
async def ask(request: AskRequest) -> AskResponse:
    query_embedding = await embed_text(request.question)
    sources = await asyncio.to_thread(retrieve_sources, query_embedding)
    if not sources:
        return AskResponse(answer="I do not have enough local context to answer that question.", sources=[])
    context = "\n\n".join(f"[{item['source']}]\n{item['content']}" for item in sources)
    async with httpx.AsyncClient(timeout=120.0) as client:
        response = await client.post(
            f"{OLLAMA_BASE_URL}/api/chat",
            json={"model": CHAT_MODEL, "messages": [{"role": "system", "content": "Answer only from the supplied context. If the context does not contain the answer, say so. Cite source labels in square brackets."}, {"role": "user", "content": f"Context:\n{context}\n\nQuestion: {request.question}"}], "stream": False, "options": {"temperature": 0}},
        )
        response.raise_for_status()
        answer = response.json()["message"]["content"]
    return AskResponse(answer=answer, sources=[Source(source=item["source"], topic=item["topic"], similarity=item["similarity"]) for item in sources])

How to compare the file

Check the file from top to bottom. Pay close attention to the response models plus the arguments passed into asyncio.to_thread().

  • Enter http://127.0.0.1:8000/docs in your browser's address bar.
  • Expand POST /ask in the interactive documentation.
  • Click Try it out.
  • Replace the request body's question value with How many days per week can Trailhead Labs employees work remotely?.

Before you execute the request, which policy source do you expect the vector query to rank most highly?

  • Click Execute.

You should see an answer that says employees may work remotely three days per week. The sources list should include remote-work.md with the benefits topic.

You closed the grounding loop. The answer now exposes the local evidence that supports it.

  • Replace the request body's question value with How much annual leave do Trailhead Labs employees receive?.
  • Click Execute.

The answer should avoid inventing an annual-leave policy because the retrieved context does not contain one. The response still shows which local sources were considered.

Grounded response missing or incorrect?

Confirm Ollama plus PostgreSQL are still running locally. Run seed.py again if the documents table no longer contains the three policy rows.

Check that the known question matches the remote-work wording above. Confirm the response includes a sources array.

Help me debug the grounded response.

Secret mission

Add Topic-Filtered Retrieval

Vector similarity can surface evidence from the wrong policy area. Add an optional topic filter that narrows the candidate rows in parameterized SQL before pgvector ranks them.

Clean Up Your Resources

Clean Up Your Resources

Your local setup has no ongoing cloud costs. Choose a path below for keeping, pausing, or deleting it.

Resources you used:

  • Local services running through Ollama, PostgreSQL, FastAPI, and Uvicorn.
  • Two Ollama models stored locally as embeddinggemma and gemma3:1b.
  • The rag_api database containing the three Trailhead Labs rows in documents with pgvector embeddings.
  • The local-rag-api folder containing .venv, app.py, requirements.txt, schema.sql, and seed.py.

Keep everything running

No action needed. Choose this if you want to keep demonstrating your local RAG API.

  • Your /ask endpoint remains available through the interactive documentation.
  • The rag_api database keeps all three embedded policy documents.
  • The two Ollama models remain ready for local embedding and generation requests.
  • The running services continue using some memory on your Mac.

Pause - I'll come back to this later

Shut down the running services to free up memory. Your database files and downloaded models remain on disk.

  • Stop Uvicorn by pressing Control-C in its Terminal window.
  • Stop the PostgreSQL server in Postgres.app.
  • Quit Ollama from its macOS menu bar icon.

Your Mac now has the local services stopped. The stored documents and models are ready for your next session.

  • Restart the PostgreSQL server in Postgres.app when you return.
  • Press Cmd+Space to open macOS search.
  • Enter Ollama in the search field.
  • Press Return to launch Ollama.
  • Return to the activated Terminal window in local-rag-api.
  • Restart the API by running this command:
uvicorn app:app --reload

What Does This Command Do?

Uvicorn serves the saved FastAPI application again. The reload option watches app.py for changes during local development.

You can return to http://127.0.0.1:8000/docs once the startup output appears.

Delete - I don't want to use this again

Deleting these resources is permanent. The commands below remove only the project database, downloaded models, virtual environment, and project folder.

  • Stop Uvicorn by pressing Control-C in its Terminal window.
  • Remove the local project data by running these commands from the activated Terminal window in local-rag-api:
dropdb rag_api
ollama rm embeddinggemma
ollama rm gemma3:1b
deactivate
cd ..
rm -rf local-rag-api

What Gets Removed?

  • The first command removes the rag_api database with its three document rows.
  • The next two commands remove both downloaded Ollama models.
  • The remaining commands leave the virtual environment before deleting the entire local-rag-api folder.
  • Stop the PostgreSQL server in Postgres.app.
  • Quit Ollama from its macOS menu bar icon.

Cleanup complete. Python, Visual Studio Code, Postgres.app, and Ollama stay installed for future local projects.

Nice Work!

Nice Work!

You did it! Your local RAG API built with FastAPI now retrieves private Trailhead Labs evidence from PostgreSQL before Ollama writes a source-backed answer.

You've learned how to:

  • Build a typed FastAPI REST endpoint with Pydantic request and response models. Use HTTPX to send asynchronous requests to local Ollama models.
  • Store 128-dimensional embeddings in PostgreSQL with pgvector. Rank the three policy chunks using parameterized SQL and cosine similarity.
  • Retrieve evidence before generation to produce grounded private-policy answers. Return source metadata that exposes the source name, topic, and similarity score.
  • Secret Mission: Add topic-filtered retrieval through a parameterized predicate. Confirm that a topic with no rows returns the empty-source fallback.

Ready to quiz yourself?