Containerize a Knowledge Assistant API
Build a RAG-style FastAPI service and containerize it with Docker.
Introduction
30 Second Summary
Every organization has critical knowledge buried in documents that nobody can find quickly. The answers exist somewhere in folders of capability statements, proposal processes, and compliance guidelines, but digging them out takes forever.
In this project, you will build a containerized knowledge assistant API that answers natural language questions by searching your internal documents using a RAG retrieval pattern. You'll create a FastAPI service, containerize it with Docker, and add Prometheus metrics for production observability.
What You'll Build
Picture running a single command, then asking your API "What contract vehicles do we support?" and watching it return the exact relevant passages from your documentation in milliseconds.
By the end of this project, you'll have:
- A live API where you POST a question about your organization and get back the most relevant passages from your internal knowledge documents instantly.
- A TF-IDF similarity search engine you can test directly from the command line before it ever touches a container.
- A production-ready container you can start with one command and demo to anyone, complete with /health and /metrics endpoints for observability.
- Secret Mission: Add confidence scoring and source attribution so the API tells users exactly how reliable each answer is and which document it came from.
Are there any prerequisites?
You'll need basic Python knowledge and comfort with a terminal. Docker Desktop should already be installed on your Windows machine.
Before We Start
Before diving in, take a moment to commit to what you're building and why. You're about to create a containerized knowledge assistant API that answers questions from internal documents, and you'll be able to demonstrate production-readiness decisions like health checks, metrics, and structured logging in your interview.
Set Up Your Project
Before writing any application code, you need the right Python version installed and a clean project structure in place.
In this step, you will install Python 3.12, create an isolated virtual environment for your dependencies, and scaffold the directories and files the knowledge assistant needs.
In this step, get ready to:
- Install Python 3.12 and confirm it is available on your PATH.
- Create a virtual environment and install project dependencies.
- Scaffold the project file structure.
Install Python 3.12
This project requires Python 3.12 or higher. Let's check whether you already have it.
- Press the Windows key to open your search bar.
- Type PowerShell and press Enter to open it.
- Check your Python version by running this command:
python --version✔️ I see version 3.12 or higher
You should see output like Python 3.12.10 or similar. Python is ready to go.
ⓧ I see an older version
If you see a version lower than 3.12 (for example Python 3.10.x), you need to install Python 3.12 alongside it.
- Open https://www.python.org/downloads/release/python-31210/ in your browser.
- Scroll down to Files and click Windows installer (64-bit) to download.
- Run the installer.
- On the first screen, check the box labelled Add python.exe to PATH.
- Click Install Now and wait for the installation to complete.
- Close your current PowerShell window and open a new one so the updated PATH takes effect.
- Run python --version again. You should now see Python 3.12.10.
Older version still showing up?
If you still see the old version after opening a new PowerShell, the old Python is earlier on your PATH. Open Settings, search for Environment Variables, and move the Python 3.12 entry above the older one in your Path variable.
ⓧ Command not found
Python is not installed yet. Let's install it now.
- Open https://www.python.org/downloads/release/python-31210/ in your browser.
- Scroll down to Files and click Windows installer (64-bit) to download.
- Run the installer.
- On the first screen, check the box labelled Add python.exe to PATH.
- Click Install Now and wait for the installation to complete.
- Close your current PowerShell window and open a new one so the PATH update takes effect.
- Confirm the install by running:
python --versionYou should see Python 3.12.10 in the output.
Still not seeing Python 3.12?
Make sure you checked Add python.exe to PATH during installation. If you missed it, re-run the installer, choose Modify, and ensure the PATH checkbox is selected.
Help me fix my Python PATH on Windows
Create the project directory and virtual environment
A virtual environment gives your project its own isolated set of packages. This prevents version conflicts between different projects on your machine.
- Move to your Desktop by running this command:
cd ~/Desktop- Create your project folder and move into it by running these commands:
mkdir knowledge-assistant
cd knowledge-assistant- Create and activate the virtual environment by running these commands:
python -m venv .venv
.\.venv\Scripts\activateYour PowerShell prompt now shows (.venv) at the beginning. This confirms you are working inside the isolated environment.
What is a virtual environment?
A virtual environment is a self-contained folder that holds its own Python interpreter and packages. Anything you install with pip while the environment is active goes into .venv instead of your system-wide Python.
This keeps your project reproducible. When you share the project or deploy it in a Docker container later, you will know exactly which packages are needed.
- Install the project dependencies by running:
pip install "fastapi[standard]" scikit-learn prometheus-clientYou will see pip downloading and installing several packages. This takes a minute or two depending on your connection.
What are these packages?
- fastapi[standard] is the web framework for building the API endpoints. The [standard] extra includes uvicorn (the production server) and other useful defaults.
- scikit-learn provides the TF-IDF vectorizer and cosine similarity functions for finding relevant document chunks.
- prometheus-client lets you expose request metrics in the format that Prometheus monitoring systems expect.
- Confirm the packages installed correctly by running:
pip show fastapiYou should see package metadata including Version: 0.139.2 (or close to it). This confirms FastAPI is installed in your virtual environment.
Seeing a permissions error during install?
Make sure your prompt starts with (.venv). If it does not, the virtual environment is not active. Run .\.venv\Scripts\activate again, then retry the pip install command.
Help me fix this permissions error
Create the file structure
The knowledge assistant needs a few directories and empty files that you will fill in over the next steps.
- Create all the project directories and files by running these commands:
mkdir app
mkdir docs
New-Item app/__init__.py
New-Item app/main.py
New-Item app/rag.py
New-Item Dockerfile
New-Item requirements.txtWhat does each file do?
- app/__init__.py marks the app folder as a Python package so its modules can be imported.
- app/main.py will hold the FastAPI application and endpoints.
- app/rag.py will hold the retrieval engine that searches knowledge documents.
- docs/ will store the text documents the assistant searches through.
- Dockerfile will define how to package the app into a container.
- requirements.txt will list the pinned package versions for reproducible installs.
Before you check the result, what files and folders do you expect to see listed?
- Verify the full structure exists by running:
lsYou should see app, docs, Dockerfile, and requirements.txt listed in the output (along with the .venv folder).
- Confirm the files inside the app/ directory by running:
ls appYou should see __init__.py, main.py, and rag.py listed.
Missing files in the listing?
If any files are missing, re-run the New-Item command for the missing file. Make sure you are inside the knowledge-assistant folder when you run the commands (your prompt should show this path).
Help me find my missing project files
Your project skeleton is ready and your dependencies are installed. Next up, you will build the RAG retrieval engine that searches through documents and finds relevant answers.
Build the RAG Retrieval Engine
In Step 1, you set up your Python environment and project structure. Now it is time to give your knowledge assistant something to be smart about.
The core value of this service is answering questions from internal documents. Without a retrieval engine, there is nothing to containerize or monitor in later steps. You will create sample knowledge documents and build the engine that searches them using TF-IDF similarity.
In this step, get ready to:
- Create sample knowledge documents that the assistant will search.
- Build a retrieval engine that finds relevant document chunks for any question.
- Test the engine standalone to confirm it returns accurate results.
Create the knowledge documents
Your knowledge assistant needs documents to search. You will create five text files covering different organizational topics: capabilities, proposals, teaming, contract vehicles, and security.
- In VS Code, expand the docs folder in the Explorer sidebar.
- Create a new file called capability_statement.txt inside docs/ and paste the following content:
Company Capability Statement
Core Competencies:
- Cloud infrastructure modernization (AWS, Azure, GCP)
- DevSecOps pipeline implementation and CI/CD automation
- AI/ML platform operations and model lifecycle management
- Kubernetes container orchestration and microservices architecture
- Zero Trust security implementation and NIST 800-171 compliance
Past Performance:
- Migrated 200+ applications to cloud for federal agency (2023)
- Implemented ATO-compliant CI/CD pipeline reducing deployment time by 80%
- Built ML inference platform serving 10M+ predictions daily
NAICS Codes: 541512, 541513, 541519, 518210
Certifications: ISO 27001, SOC 2 Type II, FedRAMP Ready- Create another new file called proposal_process.txt inside docs/ and paste this content:
Proposal Response Process
Step 1: Opportunity Identification
- Monitor SAM.gov, GovWin, and agency forecasts daily
- Score opportunities using PWIN criteria (past performance, relationships, solution fit)
- Gate: Bid/No-Bid decision within 48 hours of RFP release
Step 2: Solution Development
- Assign capture lead and technical architect
- Map requirements to capabilities and identify gaps
- Develop staffing plan and identify key personnel
Step 3: Writing and Review
- Color team reviews: Pink (compliance), Red (scoring), Gold (executive)
- All proposal volumes due to review leads 72 hours before submission
- Final quality check: compliance matrix, page counts, formatting
Step 4: Submission
- Submit via designated portal (SAM.gov, eBuy, agency-specific)
- Confirm receipt and save confirmation number
- Post-submission debrief within 5 business days
Checklist before submission:
- All sections addressed per RFP instructions
- Past performance references contacted and confirmed
- Pricing validated against rate card and approved by CFO
- Security clearance requirements verified for all key personnel- Save both files.
- Confirm the files exist by running this in your PowerShell terminal:
ls docs/You should see capability_statement.txt and proposal_process.txt listed in the output.
- Create teaming_strategy.txt inside docs/ with this content:
Teaming and Partnership Strategy
Partner Selection Criteria:
- Complementary technical capabilities (not overlapping)
- Active security clearances at required levels
- Past performance on similar contract vehicles
- Cultural fit and communication responsiveness
- Financial stability (Dun and Bradstreet score above 80)
Teaming Agreement Template:
- Work share split defined by CLIN
- IP ownership follows development effort
- Non-compete clause limited to specific opportunity
- Communication protocols and POC designated within 5 days
- Termination clause with 30-day notice
Subcontracting Approach:
- Small business subcontracting plan (23% target)
- Mentor-protege relationships for capability development
- Joint ventures for large IDIQ vehicles
- Pass-through rates capped at 10% for transparency- Create contract_vehicles.txt inside docs/ with this content:
Contract Vehicles and Access Points
GSA Multiple Award Schedule (MAS):
- IT Professional Services (SIN 54151S)
- Cloud and Cloud-Related IT Professional Services (SIN 518210C)
- Ordering period: 5-year base + three 5-year options
- Ceiling rates updated annually
SEWP V (NASA):
- Category B: different different different server hardware and peripherals
- Category D: cloud-based services, outsourcing, managed services
- Direct ordering for federal agencies
- Average order processing: 3-5 business days
CIO-SP3 Small Business:
- Task areas: IT services, cybersecurity, health IT, cloud
- Ceiling: $20B across all awardees
- Best-in-class vehicle for IT services
Agency-Specific BPAs:
- DHS EAGLE II follow-on
- VA T4NG for health IT services
- DOD JEDI replacement (JWCC) for cloud services
All vehicles require current SAM.gov registration and active CAGE code.- Save both files.
- Confirm the directory now has four files by running:
ls docs/You should see all four files listed.
- Create the final document, security_guidelines.txt, inside docs/:
Security and Compliance Guidelines
Data Handling:
- All proposal data classified as CUI (Controlled Unclassified Information)
- CUI markings required on all document headers and footers
- Encryption at rest (AES-256) and in transit (TLS 1.2+)
- Access limited to personnel with need-to-know
NIST 800-171 Controls (Key Areas):
- Access Control: Multi-factor authentication required
- Audit and Accountability: All system access logged and retained 90 days
- Configuration Management: Baseline configs documented and enforced
- Incident Response: 72-hour reporting window to contracting officer
- System and Communications Protection: Network segmentation required
Clearance Requirements:
- Public Trust: minimum for most civilian agency work
- Secret: required for DOD and intelligence community support
- Top Secret/SCI: required for classified program support
Compliance Reminders:
- Annual security awareness training for all staff
- Quarterly vulnerability scans on all systems
- POAM (Plan of Action and Milestones) updated monthly
- System Security Plan reviewed and updated annually- Save the file.
- Verify all five documents are present by running:
ls docs/You should see five .txt files listed: capability_statement.txt, contract_vehicles.txt, proposal_process.txt, security_guidelines.txt, and teaming_strategy.txt.
Why sample documents?
These documents simulate the kind of internal knowledge a real organization would feed into an assistant. Each covers a different domain so you can test whether the retrieval engine finds the right document for a given question.
In production, you would point this at actual organizational files. The RAG pattern works the same regardless of content.
Don't see all five files?
Make sure each file was saved inside the docs/ folder, not in the project root. Check the Explorer sidebar in VS Code to confirm the files appear nested under docs.
help me check my docs/ folder structure
Build the KnowledgeBase class
The retrieval engine lives in app/rag.py. It loads your text documents, splits them into chunks by paragraph, and builds a TF-IDF index so it can find the most relevant chunks for any question.
Before writing the engine code, record the exact package versions your project uses. This file will be critical when you containerize the application later.
- Open requirements.txt in the project root (it is currently empty) and paste the following:
fastapi[standard]==0.139.2
scikit-learn==1.9.0
prometheus-client==0.25.0
uvicorn[standard]==0.34.0What does this file do?
Pinning exact versions (with ==) ensures that every environment installs the same packages. Without pinned versions, a future install could pull a breaking update and silently break your application.
- Save requirements.txt.
- Now open app/rag.py (currently empty) and paste the following code to set up the class and its document loading logic:
import os
import logging
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
logger = logging.getLogger(__name__)
class KnowledgeBase:
def __init__(self, docs_dir: str):
self.docs_dir = docs_dir
self.chunks: list[dict] = []
self.vectorizer = TfidfVectorizer(stop_words="english")
self._load_documents()
self._build_index()
def _load_documents(self):
for filename in os.listdir(self.docs_dir):
if not filename.endswith(".txt"):
continue
filepath = os.path.join(self.docs_dir, filename)
with open(filepath, "r", encoding="utf-8") as f:
content = f.read()
paragraphs = [p.strip() for p in content.split("\n\n") if p.strip()]
for paragraph in paragraphs:
self.chunks.append({
"text": paragraph,
"source": filename,
})
logger.info(f"Loaded {len(self.chunks)} chunks from {self.docs_dir}")What does this code do?
The __init__ method runs automatically when you create a KnowledgeBase instance. It triggers document loading and index building in sequence.
The _load_documents method reads every .txt file in the docs directory, splits each file on double newlines (paragraph boundaries), and stores each paragraph as a separate searchable chunk with its source filename.
- Directly below the _load_documents method, paste the _build_index method:
def _build_index(self):
if not self.chunks:
logger.warning("No chunks loaded, index will be empty")
return
texts = [chunk["text"] for chunk in self.chunks]
self.tfidf_matrix = self.vectorizer.fit_transform(texts)
logger.info("TF-IDF index built successfully")What does this code do?
The TfidfVectorizer converts each text chunk into a numerical vector based on word importance. Words that appear frequently in one chunk but rarely across all chunks get higher weights.
The resulting tfidf_matrix is a sparse matrix where each row represents one chunk. This is what you will compare against when a question comes in.
- Save app/rag.py.
- Test that the class loads documents correctly by running this command:
python -c "from app.rag import KnowledgeBase; kb = KnowledgeBase('docs'); print(f'{kb.document_count} documents loaded')"You should see output ending with 5 documents loaded. The logging lines above it confirm that chunks were loaded and the index was built.
Seeing a ModuleNotFoundError?
Make sure your virtual environment is active. You should see (.venv) at the start of your PowerShell prompt. If not, activate it by running:
.\.venv\Scripts\activatehelp me fix a ModuleNotFoundError when importing KnowledgeBase
- Now add the query method and document_count property below _build_index. Paste this code at the bottom of the KnowledgeBase class:
def query(self, question: str, top_k: int = 3) -> list[dict]:
if not self.chunks:
return []
question_vector = self.vectorizer.transform([question])
similarities = cosine_similarity(question_vector, self.tfidf_matrix).flatten()
top_indices = similarities.argsort()[-top_k:][::-1]
results = []
for idx in top_indices:
results.append({
"text": self.chunks[idx]["text"],
"source": self.chunks[idx]["source"],
"score": float(similarities[idx]),
})
return results
@property
def document_count(self) -> int:
return len(set(chunk["source"] for chunk in self.chunks))What does the query method do?
When a question arrives, query() converts it into the same TF-IDF vector space as your document chunks. It then computes cosine similarity between the question vector and every chunk vector.
The chunks with the highest similarity scores are the most relevant to the question. The method returns the top 3 by default, each with its text, source filename, and similarity score.
- Save app/rag.py.
✔️ Awesome, I've got everything!
Great. Double check you have saved app/rag.py and move on to the final test below.
ⓧ I'd like to double check the full code
Your app/rag.py should contain these sections in order. Scroll up and confirm each is present:
- Imports (os, logging, TfidfVectorizer, cosine_similarity) and the logger definition.
- The KnowledgeBase class with __init__ (initializes chunks list, vectorizer, and calls _load_documents and _build_index).
- The _load_documents method (reads .txt files, splits on double newlines, appends to self.chunks).
- The _build_index method (fits the TF-IDF vectorizer on all chunk texts).
- The query method (transforms the question, computes cosine similarity, returns top results).
- The document_count property (counts unique source filenames).
Test the retrieval engine
The retrieval engine is complete. Before you move on, you need to prove that it actually finds relevant passages for a given question.
Before you run the next command, predict: if you ask about "contract vehicles", which document should the engine pull results from?
- Test the full retrieval pipeline by running:
python -c "from app.rag import KnowledgeBase; kb = KnowledgeBase('docs'); print(kb.query('contract vehicles'))"You should see a list of dictionaries, each containing a text field with a relevant passage, a source field showing contract_vehicles.txt, and a score between 0 and 1.
What is the RAG pattern?
RAG stands for Retrieval-Augmented Generation. Instead of relying on a language model's training data alone, you retrieve relevant context from your own documents and include it alongside the question.
In this project, you are building the retrieval half. In production, you would pass these retrieved chunks to an LLM as context. The pattern works the same whether you use TF-IDF (like here) or vector embeddings with a dedicated database.
Seeing an empty list or unexpected results?
If you see [], check that your docs/ folder contains all five .txt files. The KnowledgeBase only loads files ending in .txt, so a different extension will be silently skipped.
If you see an AttributeError about tfidf_matrix, make sure the _build_index method is present and properly indented inside the class.
help me debug my KnowledgeBase query results
Your retrieval engine is working. It can load documents, build an index, and find the most relevant chunks for any question. Next up, you will wire this engine to HTTP endpoints so users and systems can actually reach it.
Wire Up the FastAPI Endpoints
Your retrieval engine can find relevant document chunks, but it has no way for users or other systems to reach it. Without HTTP endpoints, the engine is locked inside a Python script that only you can run locally.
In this step, you will wrap the KnowledgeBase in a FastAPI web service with three endpoints: a health check, a question-answering route, and a Prometheus-compatible metrics feed.
In this step, get ready to:
- Add structured JSON logging and Prometheus metrics instrumentation.
- Build a middleware layer that tracks every request.
- Create /health, /ask, and /metrics endpoints and verify them live.
Set up structured logging and Prometheus metrics
Production services need two things from day one: logs you can search and metrics you can graph. You will set up both before writing a single endpoint.
- Open app/main.py in VS Code (currently empty) and paste the following imports and logging configuration:
import time
import logging
import json
from datetime import datetime, timezone
from fastapi import FastAPI, Request
from fastapi.responses import JSONResponse
from pydantic import BaseModel
from prometheus_client import Counter, Histogram, make_asgi_app
from app.rag import KnowledgeBase
# Structured JSON logging
class JSONFormatter(logging.Formatter):
def format(self, record):
log_entry = {
"timestamp": datetime.now(timezone.utc).isoformat(),
"level": record.levelname,
"message": record.getMessage(),
"module": record.module,
}
return json.dumps(log_entry)
handler = logging.StreamHandler()
handler.setFormatter(JSONFormatter())
logging.basicConfig(level=logging.INFO, handlers=[handler])
logger = logging.getLogger(__name__)What does this code do?
The JSONFormatter class overrides Python's default log format so every log line is a machine-parseable JSON object with a timestamp, level, message, and module.
This matters because in production, log aggregation tools (like Loki or CloudWatch) index JSON fields automatically. Plain-text logs require custom parsing rules.
- Directly below the logger setup, add the Prometheus metric definitions and application startup:
# Prometheus metrics
REQUEST_COUNT = Counter(
"app_requests_total",
"Total requests",
["method", "endpoint", "status_code"],
)
REQUEST_LATENCY = Histogram(
"app_request_latency_seconds",
"Request latency in seconds",
["method", "endpoint"],
)
# Application setup
app = FastAPI(title="Knowledge Assistant", version="1.0.0")
kb = KnowledgeBase(docs_dir="docs")
# Mount Prometheus metrics endpoint
metrics_app = make_asgi_app()
app.mount("/metrics", metrics_app)What does this code do?
REQUEST_COUNT is a Prometheus Counter that increments once per request, labeled by HTTP method, endpoint path, and status code. REQUEST_LATENCY is a Histogram that records how long each request takes.
make_asgi_app() creates a tiny ASGI application that serves Prometheus metrics in the standard text exposition format. Mounting it at /metrics means any Prometheus scraper can collect data from your service.
- Save app/main.py.
- Start the development server by running this command in your PowerShell terminal:
fastapi dev app/main.py- Open your browser and navigate to http://127.0.0.1:8000/metrics.
You should see a wall of plain text starting with lines like # HELP and # TYPE. That is Prometheus exposition format.
Server won't start?
- Make sure your virtual environment is active. Your prompt should show (.venv) at the beginning.
- Confirm you are in the knowledge-assistant directory, not inside app/.
- If you see ModuleNotFoundError, re-run pip install "fastapi[standard]" scikit-learn prometheus-client in your active venv.
Help me debug why my FastAPI dev server won't start.
Add the request middleware and health endpoint
The metrics are defined, but nothing increments them yet. A middleware function will intercept every request, measure its latency, and emit a structured log line.
- In app/main.py, add the following middleware below the app.mount("/metrics", metrics_app) line:
# Middleware for metrics and logging
@app.middleware("http")
async def metrics_middleware(request: Request, call_next):
start_time = time.time()
response = await call_next(request)
latency = time.time() - start_time
if request.url.path != "/metrics":
REQUEST_COUNT.labels(
method=request.method,
endpoint=request.url.path,
status_code=response.status_code,
).inc()
REQUEST_LATENCY.labels(
method=request.method,
endpoint=request.url.path,
).observe(latency)
logger.info(json.dumps({
"method": request.method,
"path": request.url.path,
"status_code": response.status_code,
"latency_ms": round(latency * 1000, 2),
}))
return responseWhat does this middleware do?
Every HTTP request passes through this function before reaching its endpoint. It starts a timer, lets the request proceed, then records how long it took.
The if request.url.path != "/metrics" guard prevents the metrics endpoint from tracking its own scrape requests, which would create noisy self-referential data.
- Directly below the middleware, add the Pydantic request and response models, then the health endpoint:
# Request/Response models
class AskRequest(BaseModel):
question: str
class ChunkResult(BaseModel):
text: str
source: str
score: float
class AskResponse(BaseModel):
question: str
results: list[ChunkResult]
retrieved_at: str
# Endpoints
START_TIME = time.time()
@app.get("/health")
def health_check():
return {
"status": "healthy",
"documents_loaded": kb.document_count,
"uptime_seconds": round(time.time() - START_TIME, 1),
}What does this code do?
The three Pydantic models define the shape of data flowing through the API. FastAPI uses them to validate incoming requests and generate OpenAPI documentation automatically.
The /health endpoint returns the service status, how many documents are loaded, and how long the server has been running. Orchestration systems like Kubernetes use this to decide if the container is alive.
- Save app/main.py. The dev server auto-reloads on save.
- Open your browser and navigate to http://127.0.0.1:8000/health.
You should see a JSON response: {"status": "healthy", "documents_loaded": 5, "uptime_seconds": ...}.
Seeing a 404 Not Found?
- Check your terminal. If it shows a reload error, look for a syntax mistake in the middleware or models.
- Confirm the @app.get("/health") decorator is directly above def health_check(): with no blank lines between them.
Why am I getting a 404 when I visit /health on my FastAPI dev server?
Add the /ask endpoint
The health check confirms the service is alive. Now you will add the endpoint that actually answers questions by calling your KnowledgeBase retrieval engine.
- In app/main.py, add the following /ask endpoint below the health_check function:
@app.post("/ask", response_model=AskResponse)
def ask_question(request: AskRequest):
results = kb.query(request.question, top_k=3)
return AskResponse(
question=request.question,
results=[ChunkResult(**r) for r in results],
retrieved_at=datetime.now(timezone.utc).isoformat(),
)What does this endpoint do?
When a client sends a POST request with a JSON body like {"question": "What contract vehicles do we support?"}, this function passes the question to kb.query(), gets back the top 3 most relevant document chunks, and wraps them in a typed response.
The response_model=AskResponse parameter tells FastAPI to validate outgoing data and generate accurate API docs.
- Save app/main.py. The server auto-reloads.
Before you test, what do you expect the response to contain when you ask about teaming strategy?
- Open a second PowerShell window and test the /ask endpoint by running:
Invoke-RestMethod -Uri "http://127.0.0.1:8000/ask" -Method POST -Body '{"question": "What is our teaming strategy?"}' -ContentType "application/json"You should see a response containing document chunks from teaming_strategy.txt with relevance scores. The source field shows which file each passage came from.
Getting a connection error?
- Make sure your first PowerShell window still has the FastAPI server running. If you closed it, re-activate the venv and re-run fastapi dev app/main.py.
- If you see a JSON parsing error, check that the single quotes around the body string are straight quotes, not curly.
Help me debug my PowerShell request to the /ask endpoint.
Why check /metrics after /ask?
Switch back to your browser and reload http://127.0.0.1:8000/metrics. Search for app_requests_total in the output. You will see counters that prove your middleware is tracking every request automatically.
✔️ Awesome, I've got everything!
Great. Double check you have saved app/main.py and the dev server is still running without errors.
ⓧ I'd like to double check the full code
Scroll up and confirm your app/main.py contains these sections in order:
- Imports (time, logging, json, datetime, FastAPI, pydantic, prometheus_client, KnowledgeBase).
- JSONFormatter class and logger setup.
- REQUEST_COUNT Counter and REQUEST_LATENCY Histogram.
- app = FastAPI(...) and kb = KnowledgeBase(...).
- metrics_app mount at /metrics.
- metrics_middleware function.
- AskRequest, ChunkResult, and AskResponse model classes.
- START_TIME, /health endpoint, and /ask endpoint.
Your knowledge assistant now has a live HTTP interface. Anyone on the network can ask questions and get relevant answers from your internal documents. Next up, you will package this entire service into a Docker container so it runs identically on any machine.
Containerize with Docker
Your knowledge assistant runs perfectly on your machine. But "works on my machine" is not a deployment strategy. Docker packages your application into a container image that runs identically on any machine, any cloud, any Kubernetes cluster.
In this step, you will write a Dockerfile, build a container image, and run your FastAPI service inside it. The container includes everything the app needs. No local Python installation required on the target machine.
In this step, get ready to:
- Write a Dockerfile and .dockerignore to package the project.
- Build a Docker image called knowledge-assistant.
- Run the container and verify all three endpoints respond from the host.
Write the Dockerfile and .dockerignore
Before creating the Docker files, stop the FastAPI dev server that is still running from the previous step.
- Switch back to the PowerShell window where fastapi dev is running.
- Press Ctrl+C to stop the server.
A .dockerignore file tells Docker which files to skip when copying your project into the image. This keeps the image small and avoids leaking secrets.
- Create a file called .dockerignore in the root of your knowledge-assistant folder, and paste the following content:
.venv
__pycache__
*.pyc
.git
.envWhat does this file do?
- .venv excludes your local virtual environment. The container installs its own dependencies from scratch.
- __pycache__ and *.pyc exclude compiled Python bytecode files that are platform-specific.
- .env prevents any local secret files from being baked into the image.
Now for the main event. The Dockerfile defines how Docker builds your image, layer by layer.
- Create a file called Dockerfile (no file extension) in the root of your knowledge-assistant folder, and paste the following content:
FROM python:3.12-slim
WORKDIR /code
COPY ./requirements.txt /code/requirements.txt
RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
COPY ./app /code/app
COPY ./docs /code/docs
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]What does this Dockerfile do?
- FROM python:3.12-slim starts from a minimal Python image, keeping the final container small.
- COPY ./requirements.txt is copied BEFORE the application code. Docker caches this layer, so if you only change app code later, Docker skips reinstalling packages on rebuild.
- --no-cache-dir prevents pip from storing download caches inside the image, saving space.
- CMD runs uvicorn directly in exec form. This gives you explicit control over the server process in production.
This pattern comes from FastAPI's official Docker deployment guide.
Why a single-stage build instead of multi-stage?
Multi-stage Docker builds compile dependencies in a builder stage, then copy only the runtime artifacts into a slimmer final image. For Python apps with pure-Python dependencies, the single-stage python:3.12-slim approach is simpler and achieves nearly the same image size.
In an interview, you can mention multi-stage as a "what I'd add next" optimization for projects with compiled C extensions.
- Save both files.
- Confirm both files exist by running:
ls Dockerfile, .dockerignoreYou should see both Dockerfile and .dockerignore listed in the output.
Don't see one of the files?
- Make sure Dockerfile has no file extension. Some editors add .txt automatically. Rename it if needed.
- The .dockerignore file starts with a dot. If your editor hides dotfiles, check the Explorer sidebar in VS Code where hidden files still appear.
- Make sure both files are in the knowledge-assistant root folder, not inside app/ or docs/.
help me find my Dockerfile or .dockerignore
✔️ Awesome, I've got everything!
Great. Make sure both files are saved before moving on to the build step.
ⓧ I'd like to double check the full code
Here is the complete .dockerignore:
.venv
__pycache__
*.pyc
.git
.envAnd the complete Dockerfile:
FROM python:3.12-slim
WORKDIR /code
COPY ./requirements.txt /code/requirements.txt
RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
COPY ./app /code/app
COPY ./docs /code/docs
EXPOSE 8000
CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]Build the Docker image
Docker Desktop must be running for the build to work. If it is not already open, start it now.
- Make sure Docker Desktop is running by checking for its icon in your system tray.
- Build the Docker image by running this command from the knowledge-assistant folder:
docker build -t knowledge-assistant .This takes 30-60 seconds. You will see Docker pulling the python:3.12-slim base image, installing your pip packages, and copying your files.
What does this command do?
- docker build reads your Dockerfile and executes each instruction as a layer.
- -t knowledge-assistant tags (names) the resulting image so you can reference it later.
- . tells Docker the build context is the current directory. Docker sends these files to the build engine.
- Wait for the build to complete. The final line of output will confirm the image was tagged successfully.
Build failed with an error?
- If you see a message about not connecting to the Docker daemon, Docker Desktop is not running. Open it, wait for it to finish starting, then retry the build command.
- If pip install fails inside the build, check that requirements.txt is in the project root (not inside app/) and that the package versions match what you installed earlier.
- If Docker says the Dockerfile is empty or not found, make sure the file is named exactly Dockerfile with a capital D and no extension.
help me fix my Docker build error
Run the container and verify all endpoints
The image is built. Now you will run it as a container and confirm that all three endpoints respond from your host machine.
- Start the container by running:
docker run -p 8000:8000 knowledge-assistantWhat does this command do?
- docker run creates and starts a new container from the knowledge-assistant image.
- -p 8000:8000 maps port 8000 on your host to port 8000 inside the container. Without this flag, the container's port is unreachable from your machine.
You should see uvicorn startup output in the terminal, including a line showing the application is running on 0.0.0.0:8000.
- This container occupies your current terminal. Open a second PowerShell window to test the endpoints.
- Press the Windows key, type PowerShell, and press Enter to open a new window.
First, verify the health endpoint. Before you run this, what do you expect the response to contain?
- Test the /health endpoint by running:
Invoke-RestMethod http://localhost:8000/healthYou should see a JSON response with status: healthy, documents_loaded: 5, and an uptime_seconds value.
Next, test the knowledge retrieval endpoint. Do you think it will find information about teaming from inside the container?
- Test the /ask endpoint by running:
$body = '{"question": "What is our teaming strategy?"}'
Invoke-RestMethod -Method Post -Uri http://localhost:8000/ask -ContentType "application/json" -Body $bodyYou should see a JSON response containing results with relevant text chunks from teaming_strategy.txt. Each result includes a source and score field.
- Finally, test the /metrics endpoint by running:
Invoke-RestMethod http://localhost:8000/metricsYou should see Prometheus-formatted text with counters like app_requests_total showing the requests you just made.
Getting connection refused or empty responses?
- Switch back to the first PowerShell window where docker run is running. If you see error output or the container exited, re-check your Dockerfile and requirements.txt.
- Make sure you used -p 8000:8000 in the docker run command. Without the port mapping, the container's port is not reachable from your host.
- If you stopped the old dev server but another process still holds port 8000, the container fails to bind. Run docker ps to check if another container is using that port.
help me fix my Docker container connection issue
Your knowledge assistant is now running inside a Docker container, completely independent of your local Python environment. Next up, the Secret Mission will level up your RAG responses with confidence scoring and source attribution.
Secret mission
Add Confidence Scoring and Source Attribution
Your knowledge assistant returns answers, but it never tells you how confident it is or which document the answer came from. In a real RAG system, users need both signals to decide whether they can trust the response. Add confidence scoring and source attribution to make your /ask endpoint production-grade.
Clean Up Your Resources
Clean Up Your Resources
This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.
Resources you used:
- Docker container running the knowledge-assistant image.
- Docker image knowledge-assistant stored locally.
- knowledge-assistant project folder (includes virtual environment, application code, and sample documents).
- Python 3.12 installation.
Keep everything running
No action needed. Choose this if you plan to continue with Project 2, where you deploy this container to Kubernetes.
- Your Docker container keeps serving the API on port 8000.
- Your project folder and virtual environment stay on disk, ready for future changes.
- Python 3.12 remains available for the rest of this series.
Pause - I'll come back to this later
Stop the running container to free up memory. Your project files and Docker image stay on disk so you can resume later.
- Stop the container by running this command in PowerShell:
docker stop $(docker ps -q --filter ancestor=knowledge-assistant)- Deactivate your virtual environment by running:
deactivateWhen you come back, activate the virtual environment again with ..\.venv\Scripts\activate and restart the container with docker run -p 8000:8000 knowledge-assistant.
Delete - I don't want to use this again
Remove all project resources and start fresh. Follow each step below.
- Stop the running container:
docker stop $(docker ps -q --filter ancestor=knowledge-assistant)- Remove the Docker image:
docker rmi knowledge-assistant- Navigate out of the project folder and delete it:
cd ..
Remove-Item -Recurse -Force .\knowledge-assistant- Confirm the folder is gone by running:
lsYou should no longer see knowledge-assistant in the directory listing.
- To uninstall Python 3.12, press the Windows key and open Settings.
- Select Apps, then Installed apps.
- Search for Python 3.12, click the three dots, and select Uninstall.
- Confirm the uninstall when prompted.
Nice Work!
Nice Work!
Nice work! You just built and containerized a fully functional internal knowledge assistant API from scratch.
You can now run docker run -p 8000:8000 knowledge-assistant and demo a service that retrieves answers from internal documents, exposes production-grade observability endpoints, and runs identically on any machine with Docker installed.
You've learned how to:
- Build a RAG-style knowledge retrieval engine using TF-IDF vectorization and cosine similarity to answer natural language questions from internal documents.
- Create a production-ready FastAPI service with Prometheus metrics instrumentation, structured JSON logging, and health check endpoints for full observability.
- Containerize the application with a Docker image following best practices for layer caching and lean production deployments.
- (Secret Mission) Added confidence scoring and source attribution to the /ask endpoint, automatically filtering out low-relevance results below a similarity threshold.
Ready to quiz yourself?