Build a Grounded RAG API

Build a source-backed RAG API with MongoDB, Qdrant, and FastAPI.

Introduction

30 Second Summary

A chatbot can answer general questions in seconds. Your company's private rules remain outside its view.

In this project, you will build a grounded company-policy assistant with FastAPI and turn it into a stateful agent with Google ADK. The first phase proves retrieval, while the second adds persistent sessions and shared caching for a production-shaped local stack.

What You'll Build

You will ask one policy question across multiple turns, restart the application, and watch the agent resume the same conversation before Redis serves a repeated request from cache.

By the end of this project, you'll have:

  • Grounded policy answers that return inspectable evidence from Qdrant instead of guessing about private rules.
  • A tool-using agent that decides when to retrieve company policy through the existing RAG path.
  • Durable and faster conversations that survive restarts in PostgreSQL and reuse repeated work through Redis.
  • Secret Mission: Add a department filter that restricts retrieval to People, Finance, or Engineering evidence.

Are there any prerequisites?

You need Docker Desktop, Visual Studio Code, an email address, and a payment method for the minimum $5 OpenAI prepaid credit purchase.

Python runs inside the project container, which removes the need for a host installation.

Before We Start

Before any hands-on work begins, lock in the policy assistant you are building. You are committing to answers grounded only in evidence copied from a read-only MongoDB source.

Set Up the Local RAG Stack

Your policy assistant depends on four services sharing one reproducible network. Docker Desktop prevents Windows setup differences from changing how those services run.

In this step, you will prepare credentials for the model providers. You will configure MongoDB, Qdrant, LiteLLM, and FastAPI with Docker Compose.

In this step, get ready to:
  • Create the OpenAI and Gemini provider credentials.
  • Build the configuration files for the local RAG stack.
  • Start the four containers and verify the local endpoints.
Create provider credentials and protect them

The application sends embedding requests to OpenAI. It sends answer-generation requests to Google AI Studio through the same gateway.

This part includes a minimum $5 OpenAI prepaid credit purchase. Auto-reload stays off so the account cannot automatically add more credit.

  • Sign in to the OpenAI API platform in your browser.
  • Open your OpenAI project settings.
  • Open API Keys.
  • Select + Create new secret key.
  • Copy the secret key into a temporary secure location.
  • Add the minimum $5 prepaid credit to the account.
  • Turn off Use auto-reload.

Keep the key private

The OpenAI secret is displayed once. Keep it outside screenshots because anyone with the value can use your prepaid credit.

  • Sign in to Google AI Studio in your browser.
  • Open API Keys.
  • Select Create API key.
  • Copy the authorization key into a temporary secure location.

Why use one gateway?

LiteLLM gives the application one OpenAI-compatible interface for both providers. The application uses local model aliases instead of provider-specific model names.

This keeps provider credentials outside the Python source. It also gives later model calls one consistent route.

Create the stack configuration and starter app

The project files define every container before the first model request. Visual Studio Code keeps those files together while PowerShell runs the setup commands from the opened folder.

  • Press the Windows key to open Windows search.
  • Type Visual Studio Code.
  • Press Enter to start Visual Studio Code.
  • Select File from the menu bar.
  • Select Open Folder.
  • Choose the empty folder prepared for this project.
  • Open the integrated terminal with Ctrl+`.
  • Choose PowerShell from the terminal profile dropdown.

You should see the empty project folder in the Explorer sidebar. The PowerShell prompt starts inside that folder.

The environment template documents the three required secret names without containing real credentials.

  • Select New File in the Explorer sidebar.
  • Enter .env.example as the file name.
  • Fill .env.example by copying this configuration:
OPENAI_API_KEY=your-openai-api-key-here
GEMINI_API_KEY=your-gemini-api-key-here
LITELLM_MASTER_KEY=sk-your-long-random-litellm-key-here

What does this configuration do?

  • OPENAI_API_KEY identifies the credential used for embedding requests.
  • GEMINI_API_KEY identifies the credential used for answer-generation requests.
  • LITELLM_MASTER_KEY protects access to the local gateway.
  • Save .env.example with Ctrl+S.

The file now appears in the Explorer sidebar with three placeholder values.

File name looks different?

Check that the name begins with a dot. Remove any hidden .txt extension added to the file.

Ask for help if the name still changes: help me create a .env.example file in Visual Studio Code on Windows

  • Create the private environment file by running this command:
Copy-Item .env.example .env

What does this command do?

PowerShell copies the safe template to a local .env file. Docker Compose reads the real values from this private copy.

You should see .env appear beside .env.example in the Explorer sidebar.

Copy command failed?

Confirm that the PowerShell prompt is inside the folder showing .env.example in Explorer.

Ask for help with the current folder: help me find why PowerShell cannot copy my .env.example file

  • Select .env in the Explorer sidebar.
  • Replace your-openai-api-key-here with your OpenAI secret key.
  • Replace your-gemini-api-key-here with your Gemini authorization key.
  • Replace sk-your-long-random-litellm-key-here with a long random LiteLLM master key.
  • Save .env with Ctrl+S.

Your real credentials now live in the local .env file. Keep that file out of screenshots and source control.

  • Create .gitignore with the Explorer New File control.
  • Fill .gitignore by copying these exclusions:
.env
__pycache__/
*.pyc

What do these exclusions protect?

The first rule keeps the credential file outside source control. The remaining rules exclude generated Python cache files.

  • Save .gitignore with Ctrl+S.

The Explorer sidebar now lists both environment files. Only .env.example contains safe placeholders.

The dependency list pins the Python packages installed inside the application image.

  • Create requirements.txt with the Explorer New File control.
  • Fill requirements.txt by copying these package pins:
fastapi==0.143.0
uvicorn==0.54.0
pymongo==4.18.3
qdrant-client==1.19.1
openai==3.30.0
pydantic==2.14.1

What do these packages provide?

FastAPI and Uvicorn serve the API. PyMongo reads the source collection.

The Qdrant client handles vector storage. The OpenAI client reaches both provider aliases through LiteLLM.

  • Save requirements.txt with Ctrl+S.

You should see six pinned dependencies in requirements.txt.

Missing a dependency line?

Compare the file from top to bottom. Each package must remain on its own line with the exact version pin.

Ask for a comparison if needed: help me compare my requirements.txt with the six required package pins

The Dockerfile builds the Python application environment inside a Python container.

  • Create Dockerfile with the Explorer New File control.
  • Fill Dockerfile by copying this image definition:
FROM python:3.12.15-slim

WORKDIR /workspace

COPY requirements.txt .
RUN python -m pip install --no-cache-dir -r requirements.txt

COPY app ./app
COPY scripts ./scripts

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

What does this image contain?

The image starts with Python 3.12.15 and installs the pinned dependencies. It copies the application packages into /workspace.

The final command serves app.main:app on port 8000.

  • Save Dockerfile with Ctrl+S.

The saved Dockerfile now defines the application image from base image to server command.

Dockerfile has no syntax colours?

Confirm that the file is named exactly Dockerfile without a file extension.

Ask for help if VS Code treats it as plain text: help me fix Dockerfile language detection in Visual Studio Code

The Compose file places all four services on one local network. It also preserves database data in named volumes.

  • Create compose.yaml with the Explorer New File control.
  • Add the database services to compose.yaml by copying this first section:
services:
  mongo:
    image: mongo:9.0.2
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: local-root-password
      MONGO_INITDB_DATABASE: knowledge
    volumes:
      - mongo_data:/data/db
      - ./mongo-init.js:/docker-entrypoint-initdb.d/01-init.js:ro

  qdrant:
    image: qdrant/qdrant:v1.19.2
    ports:
      - "6333:6333"
    volumes:
      - qdrant_data:/qdrant/storage

What does this section configure?

MongoDB receives an initial database name. Its initialization script is mounted read-only.

Qdrant publishes port 6333 for its API and dashboard. Named volumes preserve both services across container restarts.

  • Save compose.yaml with Ctrl+S.

The file now shows separate mongo and qdrant service sections.

YAML indentation looks uneven?

Use spaces for every indentation level. Keep mongo and qdrant aligned beneath services.

Ask for help with the structure: help me check the indentation in the database section of compose.yaml

  • Place the cursor after the final Qdrant volume line.
  • Insert one blank line.
  • Add the gateway service and application service by copying this section:
  litellm:
    image: ghcr.io/berriai/litellm:v1.105.0
    command: ["--config", "/app/config.yaml"]
    environment:
      OPENAI_API_KEY: ${OPENAI_API_KEY}
      GEMINI_API_KEY: ${GEMINI_API_KEY}
      LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}
    ports:
      - "4000:4000"
    volumes:
      - ./litellm-config.yaml:/app/config.yaml:ro

  app:
    build: .
    command: ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]
    environment:
      OPENAI_API_KEY: ${LITELLM_MASTER_KEY}
      OPENAI_BASE_URL: http://litellm:4000/v1
    ports:
      - "8000:8000"
    volumes:
      - ./app:/workspace/app
      - ./scripts:/workspace/scripts
    depends_on:
      - mongo
      - qdrant
      - litellm

volumes:
  mongo_data:
  qdrant_data:

How do the services connect?

LiteLLM receives the provider keys from .env and exposes port 4000. The application reaches that gateway through the internal http://litellm:4000/v1 address.

The application publishes port 8000 and mounts the Python folders for reloads. Its dependencies start on the same Compose network.

  • Save the completed compose.yaml with Ctrl+S.

You should now see four service names in the file. They are mongo, qdrant, litellm, and app.

Service nested in the wrong place?

Keep all four service names aligned two spaces beneath services. Keep the final volumes key aligned with services.

Ask for help with the complete structure: help me check whether my Compose services and volumes are aligned correctly

The LiteLLM configuration maps short local aliases to the embedding and chat models.

  • Create litellm-config.yaml with the Explorer New File control.
  • Fill litellm-config.yaml by copying this gateway configuration:
model_list:
  - model_name: embed-small
    litellm_params:
      model: openai/text-embedding-3-small
      api_key: os.environ/OPENAI_API_KEY
    model_info:
      mode: embedding

  - model_name: rag-chat
    litellm_params:
      model: gemini/gemini-3.8-flash
      api_key: os.environ/GEMINI_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

What do the aliases control?

The embed-small alias sends embedding requests to openai/text-embedding-3-small. The rag-chat alias sends chat requests to gemini/gemini-3.8-flash.

The master key controls client access to the gateway. Provider keys remain in environment variables.

  • Save litellm-config.yaml with Ctrl+S.

The saved file now contains one embedding alias and one chat alias.

Alias names do not match?

Check that the aliases are exactly embed-small and rag-chat. These names must match the Python constants.

Ask for help with the mapping: help me compare my LiteLLM model aliases with the project configuration

The MongoDB initialization script supplies fictional evidence. It also creates the database-scoped read-only account used by ingestion.

  • Create mongo-init.js with the Explorer New File control.
  • Fill mongo-init.js by copying this initialization script:
db.documents.insertMany([
  {
    title: "Remote Work Policy",
    department: "People",
    content: "Employees may work remotely up to three days per week. Team leads approve the recurring schedule. Employees must remain available during their team's published collaboration hours."
  },
  {
    title: "Expense Policy",
    department: "Finance",
    content: "Meals during approved business travel are reimbursed up to $60 per day. Itemized receipts are required for any single expense above $25. Expense reports must be submitted within 10 business days."
  },
  {
    title: "Incident Response Policy",
    department: "Engineering",
    content: "A Priority 1 incident requires an incident commander within 15 minutes. The commander opens the incident channel, assigns communications ownership, and schedules a review within two business days."
  }
]);

db.createUser({
  user: "rag_reader",
  pwd: "rag-reader-local-password",
  roles: [{ role: "read", db: "knowledge" }]
});

What does the initialization script create?

The script inserts one policy for People. It also inserts one policy for Finance and one for Engineering.

The rag_reader account receives only the read role for the knowledge database. The later ingestion job can retrieve evidence without changing the source.

  • Save mongo-init.js with Ctrl+S.

You should see three document objects followed by the rag_reader user definition.

Initialization script shows red underlines?

Check every quoted policy string for matching double quotes. Keep the apostrophes inside the remote-work text unchanged.

Ask for help with the syntax: help me find a syntax problem in my MongoDB initialization script

The package folders make the Python modules available when Docker builds the application image.

  • Select New Folder in the Explorer sidebar.
  • Enter app as the folder name.
  • Select New Folder again.
  • Enter scripts as the folder name.

You should now see app and scripts in the Explorer sidebar.

  • Select the app folder in Explorer.
  • Create __init__.py with the New File control.
  • Fill app/__init__.py with this package marker:
# Package marker.

Why add a package marker?

The marker gives the app directory a concrete Python file before Docker copies it. It also records the directory in the project.

  • Save app/__init__.py with Ctrl+S.

The app folder now contains its package marker.

  • Select the scripts folder in Explorer.
  • Create __init__.py with the New File control.
  • Fill scripts/__init__.py with this package marker:
# Package marker.

What does this marker preserve?

The marker ensures the scripts directory exists when the application image is built. A later step adds the ingestion job beside it.

  • Save scripts/__init__.py with Ctrl+S.

The scripts folder now contains its package marker.

The starter application creates the gateway clients during startup. Its health route gives you a model-free check that the API container is reachable.

  • Select the app folder in Explorer.
  • Create main.py with the New File control.
  • Add the imports and shared clients to app/main.py by copying this code:
import os

from fastapi import FastAPI
from openai import OpenAI
from qdrant_client import QdrantClient

QDRANT_URL = "http://qdrant:6333"
COLLECTION_NAME = "policy_chunks"
EMBEDDING_MODEL = "embed-small"
CHAT_MODEL = "rag-chat"

app = FastAPI(title="Grounded Policy Assistant")
llm = OpenAI(
    base_url=os.environ["OPENAI_BASE_URL"],
    api_key=os.environ["OPENAI_API_KEY"],
)
qdrant = QdrantClient(url=QDRANT_URL)

What does this code prepare?

The constants name the Qdrant collection and the two LiteLLM aliases. The FastAPI object identifies the local policy assistant.

The OpenAI client reads its gateway URL and master key from the container environment. The Qdrant client uses the internal service address.

  • Save app/main.py with Ctrl+S.

The file now contains the shared application clients and model aliases.

Import lines show editor warnings?

Host Python packages are not required for this project. Docker installs the dependencies inside the application image during the build.

Ask for help if the file itself differs: help me compare the imports and client setup in app/main.py

  • Place the cursor after the qdrant client line.
  • Insert two blank lines.
  • Add the health route by copying this code:
@app.get("/health")
def health() -> dict[str, str]:
    return {"status": "ok"}

What does the health route prove?

A request to /health returns a small JSON object. That response proves Uvicorn imported the application and FastAPI handled the request.

  • Save the completed app/main.py with Ctrl+S.

You should see a health() function returning status set to ok.

Health route is indented incorrectly?

Keep the route decorator and function definition aligned at the left edge. Indent only the return statement by four spaces.

Ask for help with the function: help me check the indentation of my FastAPI health route

Your project files are now assembled. Use the comparison below to catch a missing line before Docker builds the stack.

✔️ Awesome, I've got everything!

Great. Save every open file before starting the containers.

ⓧ I'd like to double check the full code

OPENAI_API_KEY=your-openai-api-key-here
GEMINI_API_KEY=your-gemini-api-key-here
LITELLM_MASTER_KEY=sk-your-long-random-litellm-key-here

Environment template

This reference file contains placeholders only. Keep real credentials in .env.

.env
__pycache__/
*.pyc

Source control exclusions

These rules exclude the private environment file and generated Python cache files.

FROM python:3.12.15-slim

WORKDIR /workspace

COPY requirements.txt .
RUN python -m pip install --no-cache-dir -r requirements.txt

COPY app ./app
COPY scripts ./scripts

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

Application image

This file installs the dependencies and starts the FastAPI server inside the container.

# Package marker.

Application package marker

This file preserves the app package in the image.

import os

from fastapi import FastAPI
from openai import OpenAI
from qdrant_client import QdrantClient

QDRANT_URL = "http://qdrant:6333"
COLLECTION_NAME = "policy_chunks"
EMBEDDING_MODEL = "embed-small"
CHAT_MODEL = "rag-chat"

app = FastAPI(title="Grounded Policy Assistant")
llm = OpenAI(
    base_url=os.environ["OPENAI_BASE_URL"],
    api_key=os.environ["OPENAI_API_KEY"],
)
qdrant = QdrantClient(url=QDRANT_URL)


@app.get("/health")
def health() -> dict[str, str]:
    return {"status": "ok"}

Starter API

This file creates the gateway clients and exposes the health endpoint.

services:
  mongo:
    image: mongo:9.0.2
    environment:
      MONGO_INITDB_ROOT_USERNAME: root
      MONGO_INITDB_ROOT_PASSWORD: local-root-password
      MONGO_INITDB_DATABASE: knowledge
    volumes:
      - mongo_data:/data/db
      - ./mongo-init.js:/docker-entrypoint-initdb.d/01-init.js:ro

  qdrant:
    image: qdrant/qdrant:v1.19.2
    ports:
      - "6333:6333"
    volumes:
      - qdrant_data:/qdrant/storage

  litellm:
    image: ghcr.io/berriai/litellm:v1.105.0
    command: ["--config", "/app/config.yaml"]
    environment:
      OPENAI_API_KEY: ${OPENAI_API_KEY}
      GEMINI_API_KEY: ${GEMINI_API_KEY}
      LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}
    ports:
      - "4000:4000"
    volumes:
      - ./litellm-config.yaml:/app/config.yaml:ro

  app:
    build: .
    command: ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000", "--reload"]
    environment:
      OPENAI_API_KEY: ${LITELLM_MASTER_KEY}
      OPENAI_BASE_URL: http://litellm:4000/v1
    ports:
      - "8000:8000"
    volumes:
      - ./app:/workspace/app
      - ./scripts:/workspace/scripts
    depends_on:
      - mongo
      - qdrant
      - litellm

volumes:
  mongo_data:
  qdrant_data:

Compose stack

This file defines the four services and the two named data volumes.

model_list:
  - model_name: embed-small
    litellm_params:
      model: openai/text-embedding-3-small
      api_key: os.environ/OPENAI_API_KEY
    model_info:
      mode: embedding

  - model_name: rag-chat
    litellm_params:
      model: gemini/gemini-3.8-flash
      api_key: os.environ/GEMINI_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY

Model gateway aliases

This file maps the local embedding and chat aliases to their provider models.

db.documents.insertMany([
  {
    title: "Remote Work Policy",
    department: "People",
    content: "Employees may work remotely up to three days per week. Team leads approve the recurring schedule. Employees must remain available during their team's published collaboration hours."
  },
  {
    title: "Expense Policy",
    department: "Finance",
    content: "Meals during approved business travel are reimbursed up to $60 per day. Itemized receipts are required for any single expense above $25. Expense reports must be submitted within 10 business days."
  },
  {
    title: "Incident Response Policy",
    department: "Engineering",
    content: "A Priority 1 incident requires an incident commander within 15 minutes. The commander opens the incident channel, assigns communications ownership, and schedules a review within two business days."
  }
]);

db.createUser({
  user: "rag_reader",
  pwd: "rag-reader-local-password",
  roles: [{ role: "read", db: "knowledge" }]
});

MongoDB seed data

This script inserts the three fictional policies and creates the database-scoped reader.

fastapi==0.143.0
uvicorn==0.54.0
pymongo==4.18.3
qdrant-client==1.19.1
openai==3.30.0
pydantic==2.14.1

Python dependencies

This file pins every Python package installed in the application image.

# Package marker.

Scripts package marker

This file preserves the scripts package for the indexing job added later.

Start the stack and verify every service

Docker now has everything it needs to build the application and connect the services. The health endpoint checks the API without spending provider credit.

  • Press the Windows key to open Windows search.
  • Type Docker Desktop.
  • Press Enter to start Docker Desktop.
  • Wait for Docker Desktop to finish starting.

Docker Desktop is ready when it can accept Compose commands. That is the foundation for all four local services.

The first build can take a few minutes because Docker downloads the pinned images and Python packages. A quiet terminal during part of that download is expected.

  • Build and start the stack in the background by running this command:
docker compose up --build -d

What does this command do?

The --build option creates the application image from the Dockerfile. The -d option leaves the services running in the background.

MongoDB runs its initialization script against the fresh named volume. That creates the three policy documents and the read-only rag_reader account.

The command returns to the PowerShell prompt after the containers start.

Compose build failed?

Confirm that Docker Desktop is running. Check the first error line for a file name or YAML indentation problem.

Ask for help with the build output: help me diagnose my Docker Compose build failure

LiteLLM can finish initializing after the other containers. Its recent logs show whether the gateway loaded the configuration.

  • Inspect the latest LiteLLM activity by running this command:
docker compose logs --tail 20 litellm

What does this command show?

The command prints the most recent twenty log lines from the litellm service. It keeps the gateway running while you inspect its startup activity.

You should see gateway startup activity without a repeating configuration failure.

Gateway keeps restarting?

Check the log for a referenced environment variable or configuration line. Confirm that all three values in .env replaced their placeholders.

Ask for help without sharing keys: help me diagnose repeating LiteLLM startup failures from redacted logs

Before you run the final checks, what do you expect the health response and service list to prove?

  • Call the FastAPI health endpoint by running this command:
Invoke-RestMethod http://localhost:8000/health

What does this request check?

PowerShell sends a GET request to the local health endpoint. It converts the JSON response into an object you can read in the terminal.

You should see status set to ok. That result proves the application container imported the FastAPI app and served a request.

Health request cannot connect?

Give the application a little more startup time. Check that port 8000 is not already used by another local program.

Ask for help with the container state: help me diagnose why the FastAPI health endpoint is unreachable from PowerShell

  • Check the four service states by running this command:
docker compose ps

What does the service list confirm?

Compose reports the containers belonging to this project. The list lets you confirm that every required service remains running.

You should see mongo, qdrant, litellm, and app in the output.

A service is missing or stopped?

Use that service name to focus on its logs. Fix the referenced configuration file before running the Compose startup command again.

Ask for help with the service list: help me diagnose a missing or stopped Docker Compose service

You should see the Qdrant dashboard load. This confirms that the vector database is reachable before the first collection is created.

Your local RAG foundation is running. Next, you will ask the plain model about a fictional company rule and watch it expose the missing knowledge.

Expose the Knowledge Gap

The four-service stack now answers health checks. The next question is whether Gemini can handle a company-policy request with what it already knows.

The fictional policy exists only in MongoDB. In this step, you'll add a plain FastAPI route that reaches Gemini through LiteLLM without supplying those records.

In this step, get ready to:
  • Define a validated request body for policy questions.
  • Add an ungrounded POST /ask-plain endpoint through the existing model client.
  • Submit the remote-work question to capture the ungrounded response.
Add the validation imports

Pydantic checks incoming request data before your endpoint uses it. The request model needs two validation classes from the package.

  • In app/main.py, replace the import section at the top with this code:
import os

from fastapi import FastAPI
from openai import OpenAI
from pydantic import BaseModel, Field
from qdrant_client import QdrantClient

What Does This Import Add?

BaseModel provides the foundation for a typed request body. Field lets the model enforce limits on the submitted question.

  • Save app/main.py.
  • Switch back to the browser from earlier.
  • Visit http://localhost:8000/health to confirm the application reloaded successfully.

The response still shows status set to ok. This proves the new import loaded without breaking the existing route.

Health Check Stopped Working?

Check that the import section in app/main.py matches the code above. Save the file again after correcting any spelling differences.

Help me find why my FastAPI health route stopped loading after I added the Pydantic import.

Add the plain question endpoint

The request model limits each question to between 3 and 500 characters. The new route sends that question through the existing model gateway with a clear boundary for private policy requests.

  • In app/main.py, find the existing @app.get("/health") route.
  • Replace that route through the end of the file with this expanded section:
class Question(BaseModel):
    question: str = Field(min_length=3, max_length=500)


@app.get("/health")
def health() -> dict[str, str]:
    return {"status": "ok"}


@app.post("/ask-plain")
def ask_plain(request: Question) -> dict[str, str]:
    completion = llm.chat.completions.create(
        model=CHAT_MODEL,
        messages=[
            {
                "role": "system",
                "content": (
                    "You are a careful assistant. If a question depends on private "
                    "company policy that was not supplied, say that you do not have "
                    "the internal information needed to answer."
                ),
            },
            {"role": "user", "content": request.question},
        ],
        reasoning_effort="low",
    )
    return {"answer": completion.choices[0].message.content or ""}

What Does This Code Do?

  • Question defines the shape of every request sent to the plain endpoint.
  • Field(min_length=3, max_length=500) rejects questions outside the accepted length range.
  • ask_plain() sends the submitted question through the existing llm client.
  • CHAT_MODEL selects the model alias already configured in LiteLLM.
  • The system message sets the boundary for questions that depend on unavailable private information.
  • Save app/main.py.
  • Switch back to the browser from earlier.
  • Visit http://localhost:8000/docs to check the updated API surface.

Solid checkpoint: the docs now list POST /ask-plain alongside the health endpoint.

Plain Endpoint Missing?

Confirm that app/main.py is saved. Check that the @app.post("/ask-plain") decorator sits directly above ask_plain().

Help me diagnose why POST /ask-plain is missing from my FastAPI docs.

✔️ Awesome, I've got everything!

Your file now matches the target for this step.

ⓧ I'd like to double check the full code

import os

from fastapi import FastAPI
from openai import OpenAI
from pydantic import BaseModel, Field
from qdrant_client import QdrantClient

QDRANT_URL = "http://qdrant:6333"
COLLECTION_NAME = "policy_chunks"
EMBEDDING_MODEL = "embed-small"
CHAT_MODEL = "rag-chat"

app = FastAPI(title="Grounded Policy Assistant")
llm = OpenAI(
    base_url=os.environ["OPENAI_BASE_URL"],
    api_key=os.environ["OPENAI_API_KEY"],
)
qdrant = QdrantClient(url=QDRANT_URL)


class Question(BaseModel):
    question: str = Field(min_length=3, max_length=500)


@app.get("/health")
def health() -> dict[str, str]:
    return {"status": "ok"}


@app.post("/ask-plain")
def ask_plain(request: Question) -> dict[str, str]:
    completion = llm.chat.completions.create(
        model=CHAT_MODEL,
        messages=[
            {
                "role": "system",
                "content": (
                    "You are a careful assistant. If a question depends on private "
                    "company policy that was not supplied, say that you do not have "
                    "the internal information needed to answer."
                ),
            },
            {"role": "user", "content": request.question},
        ],
        reasoning_effort="low",
    )
    return {"answer": completion.choices[0].message.content or ""}
Test the model without evidence

The plain route gives you a baseline for the model's general knowledge. Testing the fictional remote-work rule reveals what the request can support without company evidence.

  • In the API docs from earlier, expand POST /ask-plain.
  • Select Try it out.
  • Replace the example question with How many days per week can employees work remotely?.

Before you submit, do you think general model knowledge is enough to answer a fictional internal rule?

  • Submit the request using the control beneath the request editor.

You'll see an answer that says the internal policy information is unavailable. That shortfall is intentional because no company evidence reached the model.

  • Capture the response area in a screenshot for the later comparison.

Seeing a Confident Policy Answer?

Compare the system message inside ask_plain() with the full-file reference above. Submit the same remote-work question again after saving any correction.

Help me troubleshoot why my ungrounded endpoint invented a company-policy answer.

You now have a clean baseline: the plain model reports its missing company context.

Next, you'll turn the read-only MongoDB documents into searchable evidence inside Qdrant.

Index the Read-Only MongoDB Source

Your plain endpoint now proves that Gemini cannot answer a fictional internal policy question without company evidence.

This step builds the searchable evidence layer for your RAG pipeline.

The indexing job connects to MongoDB through the read-only account. This protects the source from accidental writes.

In this step, get ready to:
  • Build normalized chunks from the permitted MongoDB fields.
  • Generate 1536-dimensional embeddings through LiteLLM.
  • Store three searchable points in Qdrant.
Read and chunk the policy source

An inclusion projection limits the copied fields to the evidence this project needs. The chunk_text() function normalizes whitespace before slicing each document.

  • Switch back to Visual Studio Code.
  • Create scripts/index_data.py with the VS Code file sidebar.
  • Add the imports, index settings, and chunking function by copying this code:
import os

from openai import OpenAI
from pymongo import MongoClient
from qdrant_client import QdrantClient
from qdrant_client.models import Distance, PointStruct, VectorParams

MONGODB_URI = (
    "mongodb://rag_reader:rag-reader-local-password@mongo:27017/"
    "knowledge?authSource=knowledge"
)
QDRANT_URL = "http://qdrant:6333"
COLLECTION_NAME = "policy_chunks"
EMBEDDING_MODEL = "embed-small"
VECTOR_SIZE = 1536
CHUNK_SIZE = 500
CHUNK_OVERLAP = 80


def chunk_text(text: str) -> list[str]:
    normalized = " ".join(text.split())
    step = CHUNK_SIZE - CHUNK_OVERLAP
    return [
        normalized[start : start + CHUNK_SIZE]
        for start in range(0, len(normalized), step)
        if normalized[start : start + CHUNK_SIZE]
    ]

What does this code do?

  • The imports provide clients for the source database, model gateway, and vector store.
  • The constants keep the source address, collection name, model alias, and vector dimensions consistent.
  • The 500-character chunk size keeps each policy excerpt focused.
  • The 80-character overlap preserves context across longer chunk boundaries.
  • Save scripts/index_data.py.
  • Confirm the VS Code file sidebar lists index_data.py beneath the scripts folder.

Cannot find the new script?

Check that index_data.py sits inside the existing scripts folder. A file beside that folder uses the wrong container path.

Ask for help with the file location.

The first half of main() reads the source records through rag_reader. Its projection copies only _id, title, department, and content.

  • Place your cursor below the chunk_text() function.
  • Add the source extraction and chunk-building logic by copying this code:
def main() -> None:
    mongo = MongoClient(MONGODB_URI)
    collection = mongo["knowledge"]["documents"]
    documents = list(
        collection.find(
            {},
            {"_id": 1, "title": 1, "department": 1, "content": 1},
        )
    )
    if not documents:
        raise RuntimeError("No MongoDB source documents were found.")

    chunks = []
    for document in documents:
        for chunk_number, text in enumerate(chunk_text(document["content"]), start=1):
            chunks.append(
                {
                    "document_id": str(document["_id"]),
                    "title": document["title"],
                    "department": document["department"],
                    "chunk_number": chunk_number,
                    "text": text,
                }
            )

How is the source protected?

  • The connection uses the database-scoped rag_reader account.
  • The projection restricts extraction to four permitted source fields.
  • Each chunk carries its source title, department, document ID, and sequence number.
  • The missing-document check stops the job before it sends an empty embedding request.
  • Save scripts/index_data.py.
  • Confirm the file now contains the documents query followed by the chunks loop.
Create and store the embeddings

A dense embedding turns each policy chunk into a vector of numbers. Similar meanings produce nearby vectors during retrieval.

The job sends all three chunk texts through the OpenAI-compatible LiteLLM client in one request. The returned item indexes preserve the connection between each vector and its source metadata.

  • Place your cursor below the completed chunks loop inside main().
  • Add the embedding request and ordered vector extraction by copying this code:
    llm = OpenAI(
        base_url=os.environ["OPENAI_BASE_URL"],
        api_key=os.environ["OPENAI_API_KEY"],
    )
    embedding_response = llm.embeddings.create(
        model=EMBEDDING_MODEL,
        input=[chunk["text"] for chunk in chunks],
        encoding_format="float",
    )
    vectors = [
        item.embedding
        for item in sorted(embedding_response.data, key=lambda item: item.index)
    ]

How are the vectors created?

  • The client uses the LiteLLM gateway address and master key already supplied to the app container.
  • The embed-small alias requests one embedding for each chunk text.
  • The encoding_format="float" setting returns numeric vectors that Qdrant can store.
  • Sorting by response index keeps each vector aligned with its original chunk.
  • Save scripts/index_data.py.
  • Confirm the script reads provider credentials only from the existing environment variables.

Do the vector assignments look misaligned?

Keep the list of chunk texts in the same order as the chunks list. Keep the response sort keyed to each item's index.

Ask for help checking the vector order.

A Qdrant collection defines the size and comparison method for every stored vector. This collection uses cosine similarity across vectors with 1536 dimensions.

  • Place your cursor below the vectors assignment inside main().
  • Add the collection reset, point upload, and script entry point by copying this code:
    qdrant = QdrantClient(url=QDRANT_URL)
    if qdrant.collection_exists(COLLECTION_NAME):
        qdrant.delete_collection(collection_name=COLLECTION_NAME)

    qdrant.create_collection(
        collection_name=COLLECTION_NAME,
        vectors_config=VectorParams(size=VECTOR_SIZE, distance=Distance.COSINE),
    )
    qdrant.upsert(
        collection_name=COLLECTION_NAME,
        wait=True,
        points=[
            PointStruct(id=index, vector=vector, payload=chunk)
            for index, (vector, chunk) in enumerate(zip(vectors, chunks), start=1)
        ],
    )

    mongo.close()
    print(f"Indexed {len(chunks)} chunks from {len(documents)} documents.")


if __name__ == "__main__":
    main()

How does storage stay repeatable?

  • The collection check removes an older copy of policy_chunks before a fresh run.
  • The new collection requires 1536-dimensional vectors with cosine distance.
  • Each point stores its vector beside the matching chunk payload.
  • The final message reports the document and chunk totals.
  • Save scripts/index_data.py.
  • Compare your finished script with the complete reference below.

✔️ Awesome, I've got everything!

Your scripts/index_data.py file now contains the complete repeatable indexing job.

ⓧ I'd like to double check the full code

import os

from openai import OpenAI
from pymongo import MongoClient
from qdrant_client import QdrantClient
from qdrant_client.models import Distance, PointStruct, VectorParams

MONGODB_URI = (
    "mongodb://rag_reader:rag-reader-local-password@mongo:27017/"
    "knowledge?authSource=knowledge"
)
QDRANT_URL = "http://qdrant:6333"
COLLECTION_NAME = "policy_chunks"
EMBEDDING_MODEL = "embed-small"
VECTOR_SIZE = 1536
CHUNK_SIZE = 500
CHUNK_OVERLAP = 80


def chunk_text(text: str) -> list[str]:
    normalized = " ".join(text.split())
    step = CHUNK_SIZE - CHUNK_OVERLAP
    return [
        normalized[start : start + CHUNK_SIZE]
        for start in range(0, len(normalized), step)
        if normalized[start : start + CHUNK_SIZE]
    ]


def main() -> None:
    mongo = MongoClient(MONGODB_URI)
    collection = mongo["knowledge"]["documents"]
    documents = list(
        collection.find(
            {},
            {"_id": 1, "title": 1, "department": 1, "content": 1},
        )
    )
    if not documents:
        raise RuntimeError("No MongoDB source documents were found.")

    chunks = []
    for document in documents:
        for chunk_number, text in enumerate(chunk_text(document["content"]), start=1):
            chunks.append(
                {
                    "document_id": str(document["_id"]),
                    "title": document["title"],
                    "department": document["department"],
                    "chunk_number": chunk_number,
                    "text": text,
                }
            )

    llm = OpenAI(
        base_url=os.environ["OPENAI_BASE_URL"],
        api_key=os.environ["OPENAI_API_KEY"],
    )
    embedding_response = llm.embeddings.create(
        model=EMBEDDING_MODEL,
        input=[chunk["text"] for chunk in chunks],
        encoding_format="float",
    )
    vectors = [
        item.embedding
        for item in sorted(embedding_response.data, key=lambda item: item.index)
    ]

    qdrant = QdrantClient(url=QDRANT_URL)
    if qdrant.collection_exists(COLLECTION_NAME):
        qdrant.delete_collection(collection_name=COLLECTION_NAME)

    qdrant.create_collection(
        collection_name=COLLECTION_NAME,
        vectors_config=VectorParams(size=VECTOR_SIZE, distance=Distance.COSINE),
    )
    qdrant.upsert(
        collection_name=COLLECTION_NAME,
        wait=True,
        points=[
            PointStruct(id=index, vector=vector, payload=chunk)
            for index, (vector, chunk) in enumerate(zip(vectors, chunks), start=1)
        ],
    )

    mongo.close()
    print(f"Indexed {len(chunks)} chunks from {len(documents)} documents.")


if __name__ == "__main__":
    main()

How to use this reference

This reference is the exact finished scripts/index_data.py file for this step. It gives you one place to spot missing lines or indentation.

Run and inspect the index

The finished job now has everything it needs to turn the seeded policy documents into searchable points. Running it also proves that the read-only source account can complete ingestion.

Before you run the job, consider whether the three short documents produce three chunks or more.

  • Return to PowerShell.
  • Run the indexing job inside the app container with this command:
docker compose exec app python scripts/index_data.py

What does this command do?

Docker Compose runs scripts/index_data.py inside the existing app container. That container can reach MongoDB, LiteLLM, and Qdrant through the Compose network.

You will see Indexed 3 chunks from 3 documents.

You have crossed the hard part because Qdrant now holds the searchable evidence your grounded endpoint needs.

Did the indexing job fail?

  • Confirm Docker Desktop still shows all four project containers running.
  • Check that scripts/index_data.py matches the full reference above.
  • Confirm your existing .env file still contains all three credentials.

Ask for help with the indexing output.

The dashboard gives you a direct view of the collection created by the script. Inspecting a point confirms that its vector remains attached to traceable source metadata.

  • Open http://localhost:6333/dashboard in your browser.
  • Choose policy_chunks from the collection list.
  • Inspect one stored point in the collection view.
  • Confirm its payload includes document_id, title, department, chunk_number, and text.

The collection reports three points. Each point uses a 1536-dimensional vector.

Cannot see the three points?

  • Refresh the Qdrant dashboard after the indexing command finishes.
  • Confirm PowerShell printed Indexed 3 chunks from 3 documents.
  • Select policy_chunks again after the dashboard refreshes.

Ask for help checking the collection.

Your evidence index is ready while /ask-plain remains an ungrounded baseline. Next, you will retrieve these Qdrant points before Gemini writes its answer.

Ground the API with Qdrant Retrieval

Your indexing job turned three policy documents into searchable points in Qdrant. The FastAPI service still needs a path that retrieves those points before answering a question.

This step closes that gap by retrieving semantically relevant chunks before Gemini writes the answer. The response returns the evidence separately for inspection.

In this step, get ready to:
  • Connect the API to the populated Qdrant collection.
  • Retrieve matching policy chunks with their source metadata.
  • Generate grounded answers with inspectable evidence.
Wire Qdrant into the API

The app needs one shared Qdrant client for every request. It also needs an exception that can explain when the policy index is unavailable.

  • Select app/main.py from the VS Code Explorer sidebar.
  • Replace the opening configuration through the closing parenthesis of llm by pasting this block:
import os

from fastapi import FastAPI, HTTPException
from openai import OpenAI
from pydantic import BaseModel, Field
from qdrant_client import QdrantClient

QDRANT_URL = "http://qdrant:6333"
COLLECTION_NAME = "policy_chunks"
EMBEDDING_MODEL = "embed-small"
CHAT_MODEL = "rag-chat"

app = FastAPI(title="Grounded Policy Assistant")
llm = OpenAI(
    base_url=os.environ["OPENAI_BASE_URL"],
    api_key=os.environ["OPENAI_API_KEY"],
)
qdrant = QdrantClient(url=QDRANT_URL)

What does this code do?

  • The HTTPException import lets the endpoint return a clear service error when the collection is missing.
  • The Qdrant constants identify the internal service URL and the existing policy_chunks collection.
  • The EMBEDDING_MODEL alias sends query text through LiteLLM using the same embedding route as the indexing job.
  • The shared qdrant client gives the endpoint access to the vector collection.
  • Save app/main.py.
  • Refresh http://localhost:8000/health in your browser.

You should see {"status":"ok"}. Good progress: your API reloaded with its Qdrant client ready.

Health Check Stopped Loading?

Compare the opening of app/main.py with the block above. A missing import or closing parenthesis prevents the server from reloading.

Check that qdrant = QdrantClient(url=QDRANT_URL) appears after the closing parenthesis of llm.

Ask for help with the reload failure.

Build the retrieval pipeline

The grounded route turns each question into an embedding. Qdrant compares that vector with the three indexed policy vectors.

  • Place your cursor after the existing ask_plain() function in app/main.py.
  • Start the grounded endpoint by pasting this block:
@app.post("/ask")
def ask_grounded(request: Question) -> dict[str, object]:
    if not qdrant.collection_exists(COLLECTION_NAME):
        raise HTTPException(
            status_code=503,
            detail="The policy index is missing. Run scripts/index_data.py first.",
        )

    embedding_response = llm.embeddings.create(
        model=EMBEDDING_MODEL,
        input=request.question,
        encoding_format="float",
    )
    query_vector = embedding_response.data[0].embedding

    hits = qdrant.query_points(
        collection_name=COLLECTION_NAME,
        query=query_vector,
        with_payload=True,
        limit=3,
    ).points

How does retrieval work?

  • The collection check stops the request with status code 503 when the evidence index is unavailable.
  • The embedding request converts the learner's question into a vector through the embed-small alias.
  • The query_points() call ranks the indexed vectors by similarity.
  • The with_payload=True setting returns the source metadata beside each matching vector.
  • Continue the ask_grounded() function by pasting this block directly below the retrieval code:
    sources = []
    for hit in hits:
        payload = hit.payload or {}
        sources.append(
            {
                "title": payload.get("title", "Unknown source"),
                "department": payload.get("department", "Unknown department"),
                "text": payload.get("text", ""),
                "score": round(hit.score, 4),
            }
        )

    context = "\n\n".join(
        f"[{index}] {source['title']} ({source['department']})\n{source['text']}"
        for index, source in enumerate(sources, start=1)
    )

How is the evidence structured?

  • The sources list keeps each title and department available for inspection.
  • The text field preserves the exact policy excerpt retrieved from Qdrant.
  • The rounded score shows the vector similarity for each result.
  • The context string numbers each excerpt so Gemini can cite it with brackets.
  • Save app/main.py.
  • Open http://localhost:8000/docs in your browser.

You should now see POST /ask beside the existing endpoints. That is the retrieval path in place: the app can now turn a question into inspectable Qdrant evidence.

Missing the New Endpoint?

Confirm that @app.post("/ask") starts at the left edge of the file. The complete function body beneath it must remain indented by four spaces.

Check that the new function appears after the closing line of ask_plain().

Ask for help finding why the route did not load.

Generate grounded answers

Retrieved text becomes useful when the model receives a strict grounding instruction. The response keeps the generated answer separate from its supporting sources.

  • Finish the ask_grounded() function by pasting this block below the context builder:
    completion = llm.chat.completions.create(
        model=CHAT_MODEL,
        messages=[
            {
                "role": "system",
                "content": (
                    "Answer only from the supplied policy excerpts. Cite supporting "
                    "sources with bracketed numbers such as [1]. If the excerpts do "
                    "not contain the answer, say so explicitly."
                ),
            },
            {
                "role": "user",
                "content": f"Question: {request.question}\n\nPolicy excerpts:\n{context}",
            },
        ],
        reasoning_effort="low",
    )

    return {
        "answer": completion.choices[0].message.content or "",
        "sources": sources,
    }

How does grounding constrain the answer?

  • The system instruction limits the answer to the supplied policy excerpts.
  • The user message places the question beside the numbered evidence blocks.
  • The citation instruction asks Gemini to connect claims to bracketed source numbers.
  • The returned object exposes both the answer and the source records.
  • Save app/main.py.

✔️ Awesome, I've got everything!

Your grounded endpoint is complete. Confirm that app/main.py is saved before testing it.

ⓧ I'd like to double check the full code

import os

from fastapi import FastAPI, HTTPException
from openai import OpenAI
from pydantic import BaseModel, Field
from qdrant_client import QdrantClient

QDRANT_URL = "http://qdrant:6333"
COLLECTION_NAME = "policy_chunks"
EMBEDDING_MODEL = "embed-small"
CHAT_MODEL = "rag-chat"

app = FastAPI(title="Grounded Policy Assistant")
llm = OpenAI(
    base_url=os.environ["OPENAI_BASE_URL"],
    api_key=os.environ["OPENAI_API_KEY"],
)
qdrant = QdrantClient(url=QDRANT_URL)


class Question(BaseModel):
    question: str = Field(min_length=3, max_length=500)


@app.get("/health")
def health() -> dict[str, str]:
    return {"status": "ok"}


@app.post("/ask-plain")
def ask_plain(request: Question) -> dict[str, str]:
    completion = llm.chat.completions.create(
        model=CHAT_MODEL,
        messages=[
            {
                "role": "system",
                "content": (
                    "You are a careful assistant. If a question depends on private "
                    "company policy that was not supplied, say that you do not have "
                    "the internal information needed to answer."
                ),
            },
            {"role": "user", "content": request.question},
        ],
        reasoning_effort="low",
    )
    return {"answer": completion.choices[0].message.content or ""}


@app.post("/ask")
def ask_grounded(request: Question) -> dict[str, object]:
    if not qdrant.collection_exists(COLLECTION_NAME):
        raise HTTPException(
            status_code=503,
            detail="The policy index is missing. Run scripts/index_data.py first.",
        )

    embedding_response = llm.embeddings.create(
        model=EMBEDDING_MODEL,
        input=request.question,
        encoding_format="float",
    )
    query_vector = embedding_response.data[0].embedding

    hits = qdrant.query_points(
        collection_name=COLLECTION_NAME,
        query=query_vector,
        with_payload=True,
        limit=3,
    ).points

    sources = []
    for hit in hits:
        payload = hit.payload or {}
        sources.append(
            {
                "title": payload.get("title", "Unknown source"),
                "department": payload.get("department", "Unknown department"),
                "text": payload.get("text", ""),
                "score": round(hit.score, 4),
            }
        )

    context = "\n\n".join(
        f"[{index}] {source['title']} ({source['department']})\n{source['text']}"
        for index, source in enumerate(sources, start=1)
    )

    completion = llm.chat.completions.create(
        model=CHAT_MODEL,
        messages=[
            {
                "role": "system",
                "content": (
                    "Answer only from the supplied policy excerpts. Cite supporting "
                    "sources with bracketed numbers such as [1]. If the excerpts do "
                    "not contain the answer, say so explicitly."
                ),
            },
            {
                "role": "user",
                "content": f"Question: {request.question}\n\nPolicy excerpts:\n{context}",
            },
        ],
        reasoning_effort="low",
    )

    return {
        "answer": completion.choices[0].message.content or "",
        "sources": sources,
    }

The remote-work question now tests whether retrieval closes the knowledge gap. Set up the request before predicting which source Qdrant will rank highest.

  • Return to http://localhost:8000/docs in your browser.
  • Expand POST /ask.
  • Click Try it out.
  • Replace the question value with How many days per week can employees work remotely?.

Before you send the request, which indexed policy do you expect Qdrant to rank highest?

  • Click Execute.

You should see an answer stating up to three days per week. The sources should include Remote Work Policy, People, and a numeric similarity score.

  • Expand POST /ask-plain.
  • Click Try it out.
  • Enter How many days per week can employees work remotely? as the question value.
  • Click Execute.
  • Compare the two response bodies.

The plain endpoint should report that it lacks the internal information. The grounded endpoint should answer from the retrieved policy evidence.

  • Return to the expanded POST /ask endpoint.
  • Replace the question value with What is the daily meal reimbursement limit while traveling?.

Before you send the second question, which department should appear in the strongest source?

  • Click Execute.

You should see $60 per day in the answer. The sources should include Expense Policy with its similarity score.

Sources Empty or Irrelevant?

Confirm that the policy_chunks collection still contains three points in the Qdrant dashboard. Rerun the indexing job from the previous step if the collection is missing.

Check that with_payload=True remains in query_points(). The source builder needs those payload fields.

Ask for help diagnosing the retrieval result.

You have closed the knowledge gap: the model now answers from retrieved evidence with inspectable sources. Next, you will give that retrieval path to an agent that can choose when to use it.

Secret mission

Filter Retrieval by Department

Add a department constraint before similarity ranking. Then prove Finance retrieves the expense evidence. Prove People leaves the meal question unanswered.

Clean Up Your Resources

Clean Up Your Resources

Your local containers and named volumes are free to keep. Choose whether to keep the stack running, pause it for later, or delete its runtime data and credentials.

Cost warning

Embedding calls to the OpenAI API consume prepaid credit whenever you test the grounded endpoint. Deleting the local stack does not refund purchased OpenAI credits.

Resources you used:

  • The local Docker Compose stack with mongo for MongoDB, qdrant for Qdrant, litellm for LiteLLM, and app for FastAPI.
  • The named Docker volumes mongo_data and qdrant_data.
  • The local .env file containing provider credentials.
  • The OpenAI API key and the Gemini API key.

Keep everything running

No action needed. Choose this if you are still testing the grounded API or plan to extend it soon.

  • Leave the four Compose services running while you continue testing.
  • Keep .env outside source control.

Pause - I'll come back to this later

Shut down the four services to free up local memory. Your source files and named-volume data remain ready for your return.

  • Switch back to the PowerShell terminal from earlier.
  • Confirm the current project folder contains compose.yaml.
  • Stop the Compose services while preserving their data by running this command:
docker compose stop

What Does This Command Do?

This command stops the four containers. It preserves mongo_data and qdrant_data for your next session.

  • Return to Docker Desktop from earlier.
  • Confirm that mongo, qdrant, litellm, and app are stopped.

Services Still Running?

Make sure you ran the command from the same project folder that contains compose.yaml.

Help me stop the Compose services.

When you return, the named volumes restore the MongoDB seed data and the populated Qdrant collection.

  • Resume the preserved Compose services by running this command:
docker compose start

What Does This Command Do?

This command starts the existing containers without rebuilding them. The services reuse the data stored in their named volumes.

  • Return to Docker Desktop from earlier.
  • Confirm that all four Compose services are running again.

Services Not Starting?

Make sure Docker Desktop from earlier is running.

Run the command from the project folder that contains compose.yaml.

Help me restart the Compose services.

Delete - I don't want to use this again

Deleting the stack is permanent for its containers and stored data. Your source files remain available while the local credentials are removed or revoked.

  • Switch back to the PowerShell terminal from earlier.
  • Confirm the current project folder contains compose.yaml.
  • Remove the Compose containers and named volumes by running this command:
docker compose down -v

What Does This Command Do?

This removes the project containers and Compose network. The -v flag also removes mongo_data and qdrant_data.

  • Return to Docker Desktop from earlier.
  • Confirm the Compose project no longer shows its four containers.
  • Confirm that mongo_data and qdrant_data no longer appear among the local volumes.

Resources Still Listed?

Make sure the command finished before checking Docker Desktop. Confirm that PowerShell was using the project folder containing compose.yaml.

Help me remove the remaining Compose resources.

The remaining cleanup removes the local secret file and disables both provider keys.

  • Switch back to VS Code from earlier.
  • Right-click .env in the file sidebar.
  • Choose the delete option from the context menu.
  • Confirm that .env no longer appears in the project files.

Removing the local file prevents the credentials from remaining on your machine. The provider keys still need to be revoked at their source.

  • Return to the OpenAI project settings from earlier.
  • Select API Keys.
  • Revoke the project key using its action menu.
  • Confirm that the key no longer has active access.

The Gemini key is managed separately in Google AI Studio, so revoke it there as the final credential cleanup.

  • Return to Google AI Studio from earlier.
  • Select API Keys.
  • Revoke the Gemini API key using its action menu.
  • Confirm that the key no longer has active access.

Nice Work!

Nice Work!

Congratulations. You built a containerized FastAPI service that turns private policy records into grounded answers through RAG.

You've learned how to:

  • Expose an ungrounded knowledge gap by comparing a plain model response with a source-backed policy answer.
  • Protect MongoDB with a read-only account. You also sent chunk embeddings through LiteLLM for storage in Qdrant.
  • Return inspectable Gemini answers with source attribution. Each response includes titles. It also includes departments. It includes similarity scores too.
  • Secret Mission: Restrict retrieval with a department filter. Your tests proved that filtered evidence can produce a matching answer or expose a deliberate knowledge gap.

Ready to quiz yourself?