Deploy a RAG Agent API to Kubernetes

Build a FastAPI RAG agent, containerize it, and deploy it to Kubernetes.

Introduction

30 Second Summary

When teams build AI-powered features, the real challenge is not writing the code. It is getting that code running reliably where users can actually reach it. That is where Kubernetes comes in.

In this project, you will build a document Q&A agent as a REST API, containerize it with Docker, and deploy it to a local Kubernetes cluster. You will feed it documents and query it with natural language questions, just like a production AI service.

What You'll Build

A RAG-powered documentation Q&A agent exposed as a FastAPI REST API, containerized with Docker, and deployed to Minikube.

By the end of this project, you'll have:

A FastAPI application with async endpoints, Pydantic validation, and a health check.
A RAG agent that ingests documents into ChromaDB and answers questions using LangChain and Ollama.
A complete Kubernetes deployment with Deployments, Services, ConfigMaps, and health probes accessible from your browser.
Secret Mission: Add a Horizontal Pod Autoscaler and load-test your API to watch Kubernetes scale pods automatically.

Are there any prerequisites?

You should be familiar with Python, Docker basics, and the concept of RAG (retrieval-augmented generation). No prior Kubernetes experience is needed. This project will guide you through every cluster command from scratch.

Before We Start

Before We Start...

Install Minikube and kubectl

To deploy your RAG agent to Kubernetes, you need a local cluster running on your machine. Minikube creates a single-node Kubernetes cluster inside Docker Desktop, giving you a real Kubernetes environment without needing cloud infrastructure.

You also need kubectl, the command-line tool that talks to any Kubernetes cluster. Together, Minikube and kubectl give you everything production teams use to deploy containers.

In this step, get ready to:

Install Minikube and kubectl on Windows.
Start your first Kubernetes cluster.
Explore the Kubernetes dashboard.

Install Minikube on Windows using winget

Minikube runs a Kubernetes cluster inside a Docker container on your machine. Before installing, you need to confirm Docker Desktop is set to use Linux containers.

Verify Docker Desktop is using Linux containers by running this command:

docker info --format '{{.OSType}}'

✔️ I see linux

Docker Desktop is correctly configured. You are ready to install Minikube.

ⓧ I see windows

You need to switch Docker Desktop to Linux containers before continuing.

Right-click the Docker Desktop icon in your system tray.
Click Switch to Linux containers...
Wait for Docker Desktop to restart, then run the command above again to confirm it now says linux.

Install Minikube by running this command:

winget install Kubernetes.minikube

What does this command do?

The Windows Package Manager (winget) downloads and installs Minikube from the official Kubernetes repository. This gives you the minikube command in your terminal.

Close and reopen your terminal so the new command is available on your PATH.
Verify the installation by running:

minikube version

✔️ I see a version number

Minikube is installed and ready to use.

ⓧ Command not found

Your terminal cannot find the minikube command. Try these fixes:

Make sure you closed and reopened your terminal after installation.
Run winget install Kubernetes.minikube again and check for errors in the output.

Why can't my terminal find the minikube command after installing it with winget?

Install kubectl and verify cluster connectivity

The kubectl tool is the standard command-line interface for all Kubernetes clusters. You will use it throughout this project to deploy, inspect, and manage your application.

Install kubectl by running this command:

winget install -e --id Kubernetes.kubectl

What does this command do?

The -e flag ensures an exact match on the package ID. --id Kubernetes.kubectl specifies the official Kubernetes kubectl package, avoiding accidentally installing a similarly-named package.

Close and reopen your terminal again so the new command is available.
Verify kubectl is installed by running:

kubectl version --client

✔️ I see a version number

kubectl is installed and ready. You will connect it to your cluster in the next substep.

ⓧ Command not found

Your terminal cannot find the kubectl command.

Make sure you closed and reopened your terminal after installation.
Run winget install -e --id Kubernetes.kubectl again and check for errors.

Why can't my terminal find kubectl after installing it with winget?

Start your first Minikube cluster and explore the dashboard

With both tools installed, you can now create your local Kubernetes cluster. Minikube uses Docker Desktop as its driver, spinning up a container that acts as your cluster node.

Start your Minikube cluster by running this command:

minikube start --driver=docker

What does this command do?

The --driver=docker flag tells Minikube to use Docker Desktop as its container runtime. Minikube creates a Docker container that runs a full Kubernetes control plane inside it.

This process downloads Kubernetes images and configures networking. It can take 2-5 minutes on the first run.

Once the command finishes, verify the cluster is running:

minikube status

You should see output showing host: Running, kubelet: Running, and apiserver: Running.

Confirm kubectl is connected to your cluster by running:

kubectl get nodes

You should see a single node named minikube with a status of Ready.

Cluster not starting?

Make sure Docker Desktop is running before you start Minikube.
Confirm Docker Desktop is set to Linux containers (not Windows containers).
Minikube requires at least 2 CPUs, 2GB RAM, and 20GB disk space. Check that Docker Desktop has enough resources allocated in its Settings > Resources panel.

Why won't my Minikube cluster start with Docker Desktop on Windows?

Open the Kubernetes dashboard by running:

minikube dashboard

What is the Kubernetes dashboard?

The dashboard is a web-based UI for your cluster. It opens automatically in your browser and shows you all the resources running inside Kubernetes.

Press Ctrl+C in your terminal to stop the dashboard proxy when you are done exploring.

Your local Kubernetes cluster is up and running. Next up, you will build the FastAPI application that your cluster will host.

Build the FastAPI Application

With your Minikube cluster running, it's time to build the application that will live inside it.

In this step, you'll create a FastAPI REST API with typed request and response models, dependency injection, and a health check endpoint. These patterns mirror how production teams structure real AI services.

In this step, get ready to:

Create the project structure with Pydantic request and response models.
Build async endpoints with dependency injection.
Add a health check endpoint for Kubernetes probes.

Create the project structure with Pydantic models

Every API needs a home. You'll create a project folder on your Desktop, define the Python dependencies, and build the Pydantic models that describe the shape of every request and response your API handles.

Move to your Desktop by running this command:

cd ~/Desktop

Create your project folder and move into it by running these commands:

mkdir rag-agent-api
cd rag-agent-api

Create the app package directory by running:

mkdir app

Create the empty __init__.py file inside app by running:

New-Item app/__init__.py

What is init.py?

An empty __init__.py file tells Python that the app directory is a package. Without it, Python won't be able to resolve imports like from app.models import ....

Confirm your directory structure by running:

ls app

You should see __init__.py listed in the output.

Don't see init.py?

Make sure you ran mkdir app before creating the file. The file cannot be created inside a directory that doesn't exist yet.

If you're using a terminal that doesn't recognize New-Item, open the app folder in your text editor and create a new empty file named __init__.py.

Help me troubleshoot my project directory

Now you'll pin the Python packages your project depends on.

Create a new file called requirements.txt in the root of your rag-agent-api folder. Open your text editor and paste this content:

fastapi[standard]==0.136.3
langchain-ollama==1.1.0
langchain-chroma==1.1.0
langchain-text-splitters==0.3.8
pydantic-settings==2.9.1
chromadb==1.5.9

What does this file do?

The requirements.txt file pins every Python package your project needs to an exact version. This ensures your local environment, your Docker container, and your Kubernetes pods all run the same dependencies.

fastapi[standard] is the web framework that powers your API endpoints.
langchain-ollama and langchain-chroma connect your RAG agent to Ollama and ChromaDB.
pydantic-settings lets you manage configuration through environment variables.

Save requirements.txt.

Next, you'll define the data models. Every request and response flowing through your API will be validated against these schemas automatically.

Create a new file called models.py inside the app folder with this content:

from pydantic import BaseModel, Field


class IngestRequest(BaseModel):
    content: str = Field(..., min_length=1, description="Document text to ingest")
    metadata: dict = Field(default_factory=dict, description="Optional metadata")


class IngestResponse(BaseModel):
    status: str
    document_id: str
    chunks: int


class QueryRequest(BaseModel):
    question: str = Field(..., min_length=1, description="Question to ask the agent")


class QueryResponse(BaseModel):
    answer: str
    sources: list[str] = Field(default_factory=list)


class HealthResponse(BaseModel):
    status: str
    ollama_connected: bool
    documents_indexed: int

What does this code do?

Each class defines the exact shape of data your API accepts or returns. FastAPI uses these models for automatic validation and documentation generation.

IngestRequest expects a content string (at least 1 character) and optional metadata.
QueryRequest expects a question string. If a caller sends an empty string, FastAPI rejects it automatically.
HealthResponse reports whether Ollama is reachable and how many documents are indexed.

Save app/models.py.

Now you'll create a centralized settings file so your app knows where to find Ollama and which models to use.

Create a new file called config.py inside the app folder with this content:

from pydantic_settings import BaseSettings


class Settings(BaseSettings):
    ollama_base_url: str = "http://host.minikube.internal:11434"
    chat_model: str = "llama3.2:1b"
    embedding_model: str = "nomic-embed-text"
    chroma_persist_dir: str = "/app/chroma_data"

    class Config:
        env_prefix = "RAG_"


settings = Settings()

What does this code do?

The Settings class reads configuration from environment variables prefixed with RAG_. If no environment variable is set, it falls back to the default values you see here.

The ollama_base_url points to host.minikube.internal so pods running inside your Minikube cluster can reach Ollama on your host machine.

Why host.minikube.internal?

Ollama runs on your Windows machine, but your API will run inside Kubernetes pods. Pods can't use localhost to reach the host because each pod has its own network namespace.

Minikube provides host.minikube.internal as a special hostname that resolves to your host machine from inside the cluster.

Save app/config.py.

Unsure where to save these files?

Your project folder should look like this: rag-agent-api/requirements.txt at the top level, and rag-agent-api/app/__init__.py, rag-agent-api/app/models.py, and rag-agent-api/app/config.py inside the app subdirectory.

Help me verify my project structure

Build async endpoints with dependency injection

Your models define what data looks like. Now you'll build the actual API that receives requests, routes them to the RAG service, and returns responses.

Create a new file called main.py inside the app folder. Start with the imports, app setup, and dependency injection by pasting this code:

from contextlib import asynccontextmanager

from fastapi import FastAPI, HTTPException

from app.models import (
    IngestRequest, IngestResponse,
    QueryRequest, QueryResponse,
    HealthResponse,
)
from app.rag_service import RAGService


rag_service: RAGService | None = None


@asynccontextmanager
async def lifespan(app: FastAPI):
    global rag_service
    rag_service = RAGService()
    yield
    rag_service = None


app = FastAPI(
    title="RAG Agent API",
    version="1.0.0",
    lifespan=lifespan,
)


def get_rag_service() -> RAGService:
    if rag_service is None:
        raise HTTPException(status_code=503, detail="Service not ready")
    return rag_service

What does this code do?

The lifespan context manager creates the RAGService when the app starts and cleans it up on shutdown. This is FastAPI's modern replacement for the older on_event decorators.

The get_rag_service() function is a dependency injector. Every endpoint calls it to get the service instance. If the service hasn't started yet, it returns a 503 error instead of crashing.

Now add the two POST endpoints that power document ingestion and querying.

Add the /ingest and /query endpoints at the bottom of app/main.py by pasting this code below get_rag_service:

@app.post("/ingest", response_model=IngestResponse)
async def ingest(request: IngestRequest):
    service = get_rag_service()
    doc_id, chunks = await service.ingest(request.content, request.metadata)
    return IngestResponse(status="ingested", document_id=doc_id, chunks=chunks)


@app.post("/query", response_model=QueryResponse)
async def query(request: QueryRequest):
    service = get_rag_service()
    answer, sources = await service.query(request.question)
    return QueryResponse(answer=answer, sources=sources)

What does this code do?

/ingest accepts a document, passes it to the RAG service for chunking and embedding, then returns a document ID and chunk count.
/query accepts a natural language question, retrieves relevant context from the vector store, generates an answer with the LLM, and returns it with source references.
Both endpoints use async def so FastAPI can handle multiple requests concurrently without blocking.

Save app/main.py.

Seeing a red underline on the RAGService import?

That's expected. The file app/rag_service.py doesn't exist yet. You'll create it in the next step. Your editor may show an import error, but the file will still save correctly.

Help me understand this import error

Add a /health endpoint for Kubernetes probes

Kubernetes uses health probes to decide whether a pod is ready to receive traffic. Your /health endpoint gives Kubernetes a way to check that your API is alive and Ollama is reachable.

In app/main.py, find the get_rag_service function. Add the /health endpoint directly below it (above the /ingest endpoint you just added):

@app.get("/health", response_model=HealthResponse)
async def health():
    service = get_rag_service()
    ollama_ok, doc_count = service.health_check()
    return HealthResponse(
        status="healthy" if ollama_ok else "degraded",
        ollama_connected=ollama_ok,
        documents_indexed=doc_count,
    )

What does this code do?

The /health endpoint calls health_check() on the RAG service to verify Ollama connectivity and report document count.

If Ollama is unreachable, it returns a degraded status instead of crashing. Kubernetes will use this response to decide whether to route traffic to this pod or restart it.

Save app/main.py.

Not sure where /health goes in the file?

The /health endpoint should sit between the get_rag_service() function and the @app.post("/ingest", ...) decorator. Check the full file below if you're unsure about placement.

Help me with endpoint placement

✔️ Awesome, I've got everything!

Great. Double check you've saved all your files, then move on to the next step.

ⓧ I'd like to double check the full code

Here is the complete app/main.py:

from contextlib import asynccontextmanager

from fastapi import FastAPI, HTTPException

from app.models import (
    IngestRequest, IngestResponse,
    QueryRequest, QueryResponse,
    HealthResponse,
)
from app.rag_service import RAGService


rag_service: RAGService | None = None


@asynccontextmanager
async def lifespan(app: FastAPI):
    global rag_service
    rag_service = RAGService()
    yield
    rag_service = None


app = FastAPI(
    title="RAG Agent API",
    version="1.0.0",
    lifespan=lifespan,
)


def get_rag_service() -> RAGService:
    if rag_service is None:
        raise HTTPException(status_code=503, detail="Service not ready")
    return rag_service


@app.get("/health", response_model=HealthResponse)
async def health():
    service = get_rag_service()
    ollama_ok, doc_count = service.health_check()
    return HealthResponse(
        status="healthy" if ollama_ok else "degraded",
        ollama_connected=ollama_ok,
        documents_indexed=doc_count,
    )


@app.post("/ingest", response_model=IngestResponse)
async def ingest(request: IngestRequest):
    service = get_rag_service()
    doc_id, chunks = await service.ingest(request.content, request.metadata)
    return IngestResponse(status="ingested", document_id=doc_id, chunks=chunks)


@app.post("/query", response_model=QueryResponse)
async def query(request: QueryRequest):
    service = get_rag_service()
    answer, sources = await service.query(request.question)
    return QueryResponse(answer=answer, sources=sources)

Here is app/models.py:

from pydantic import BaseModel, Field


class IngestRequest(BaseModel):
    content: str = Field(..., min_length=1, description="Document text to ingest")
    metadata: dict = Field(default_factory=dict, description="Optional metadata")


class IngestResponse(BaseModel):
    status: str
    document_id: str
    chunks: int


class QueryRequest(BaseModel):
    question: str = Field(..., min_length=1, description="Question to ask the agent")


class QueryResponse(BaseModel):
    answer: str
    sources: list[str] = Field(default_factory=list)


class HealthResponse(BaseModel):
    status: str
    ollama_connected: bool
    documents_indexed: int

Here is app/config.py:

from pydantic_settings import BaseSettings


class Settings(BaseSettings):
    ollama_base_url: str = "http://host.minikube.internal:11434"
    chat_model: str = "llama3.2:1b"
    embedding_model: str = "nomic-embed-text"
    chroma_persist_dir: str = "/app/chroma_data"

    class Config:
        env_prefix = "RAG_"


settings = Settings()

Here is requirements.txt:

fastapi[standard]==0.136.3
langchain-ollama==1.1.0
langchain-chroma==1.1.0
langchain-text-splitters==0.3.8
pydantic-settings==2.9.1
chromadb==1.5.9

Your API skeleton is complete. You have typed models, a centralized config, dependency injection, and three endpoints ready to serve traffic.

Next up, you'll wire the RAG agent logic that powers the /ingest and /query endpoints.

Wire Up the RAG Agent

Your FastAPI endpoints are ready, but they reference a RAGService class that does not exist yet. Starting the server right now would crash because main.py tries to import from app/rag_service.py.

In this step, you will build the RAG agent that powers your API. It connects LangChain to your local Ollama instance for language generation and embeddings, then uses ChromaDB as a vector store for document retrieval.

In this step, get ready to:

Connect LangChain with Ollama and ChromaDB for document ingestion and retrieval.
Build the agent logic that retrieves context and generates answers.
Test the complete API locally with fastapi dev.

Connect LangChain with Ollama and ChromaDB

The RAGService class is the core of your application. It wires together an embedding model, a chat model, a vector store, and a text splitter into a single service that your endpoints call.

Create a new file called rag_service.py inside your app folder.
Add the imports and the __init__ method by copying the following code into the file:

import uuid

from langchain_ollama import ChatOllama, OllamaEmbeddings
from langchain_chroma import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_text_splitters import RecursiveCharacterTextSplitter

from app.config import settings


class RAGService:
    def __init__(self):
        # Set up the embedding model for converting text to vectors
        self.embeddings = OllamaEmbeddings(
            model=settings.embedding_model,
            base_url=settings.ollama_base_url,
        )
        # Set up the chat model for generating answers
        self.llm = ChatOllama(
            model=settings.chat_model,
            base_url=settings.ollama_base_url,
            temperature=0.2,
        )
        # Set up the vector store for document retrieval
        self.vectorstore = Chroma(
            collection_name="documents",
            embedding_function=self.embeddings,
            persist_directory=settings.chroma_persist_dir,
        )
        # Set up the text splitter for chunking documents
        self.text_splitter = RecursiveCharacterTextSplitter(
            chunk_size=500,
            chunk_overlap=50,
        )
        # Build the prompt template and chain
        self.prompt = ChatPromptTemplate.from_messages([
            ("system", "You are a helpful assistant. Answer the question based only on the provided context. If the context does not contain the answer, say so.\n\nContext:\n{context}"),
            ("human", "{question}"),
        ])
        self.chain = self.prompt | self.llm | StrOutputParser()

What does this code do?

The __init__ method sets up all the components your RAG agent needs. OllamaEmbeddings converts text into numerical vectors so documents can be searched by meaning. ChatOllama handles generating natural language answers.

The Chroma vector store holds your document embeddings. RecursiveCharacterTextSplitter breaks long documents into smaller pieces that fit into the model's context window.

Finally, the chain uses LangChain's pipe syntax to connect the prompt template, the LLM, and an output parser into a single callable pipeline.

Save app/rag_service.py.

You will not see a visible change yet. The next substep adds the query and health-check methods that complete the service.

Build the agent logic that retrieves context and generates answers

The query method is where the RAG pattern comes together. It searches for relevant document chunks, feeds them as context to the LLM, and returns the generated answer.

Add the ingest method directly below the __init__ method, still inside the RAGService class:

    async def ingest(self, content: str, metadata: dict) -> tuple[str, int]:
        # Generate a unique ID for this document
        doc_id = str(uuid.uuid4())
        # Split the content into smaller chunks
        chunks = self.text_splitter.create_documents(
            texts=[content],
            metadatas=[{**metadata, "doc_id": doc_id}],
        )
        # Store the chunks in the vector store
        self.vectorstore.add_documents(chunks)
        return doc_id, len(chunks)

What does the ingest method do?

When a user sends a document to the /ingest endpoint, this method splits it into chunks, attaches a unique doc_id, and stores the chunks in ChromaDB. The vector store converts each chunk into an embedding for similarity search later.

Add the query method below the ingest method in app/rag_service.py:

    async def query(self, question: str) -> tuple[str, list[str]]:
        # Search for the 3 most relevant document chunks
        results = self.vectorstore.similarity_search(question, k=3)
        # Combine the chunks into a single context string
        context = "\n\n".join(doc.page_content for doc in results)
        # Track which documents contributed to the answer
        sources = list(set(
            doc.metadata.get("doc_id", "unknown") for doc in results
        ))
        # Generate an answer using the context and question
        answer = await self.chain.ainvoke({
            "context": context,
            "question": question,
        })
        return answer, sources

What does the query method do?

This method performs the three steps of RAG: retrieve, augment, and generate. First, similarity_search finds the 3 most relevant chunks from the vector store. Then those chunks become the context variable injected into the prompt.

Finally, ainvoke sends the prompt to Ollama and returns the generated answer. The sources list tracks which ingested documents contributed, so callers know where the information came from.

Add the health_check method below the query method:

    def health_check(self) -> tuple[bool, int]:
        # Check if Ollama is reachable
        try:
            self.llm.invoke("ping")
            ollama_ok = True
        except Exception:
            ollama_ok = False
        # Check how many documents are indexed
        try:
            count = self.vectorstore._collection.count()
        except Exception:
            count = 0
        return ollama_ok, count

What does health_check do?

Kubernetes will call your /health endpoint to determine if your pod is ready to serve traffic. This method verifies two things: that Ollama responds to a simple prompt, and how many document chunks are indexed in ChromaDB.

If Ollama is unreachable, the health endpoint returns "degraded" instead of "healthy". Kubernetes uses this signal to route traffic away from unhealthy pods.

Save app/rag_service.py.

Seeing a red squiggly line under an import?

If your editor shows import errors, make sure you installed all dependencies from requirements.txt in the previous step. Re-run pip install -r requirements.txt from your project folder if needed.

help me fix import errors in my RAG service file

✔️ Awesome, I've got everything!

Great. Double check you have saved app/rag_service.py before moving on.

ⓧ I'd like to double check the full code

import uuid

from langchain_ollama import ChatOllama, OllamaEmbeddings
from langchain_chroma import Chroma
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_text_splitters import RecursiveCharacterTextSplitter

from app.config import settings


class RAGService:
    def __init__(self):
        self.embeddings = OllamaEmbeddings(
            model=settings.embedding_model,
            base_url=settings.ollama_base_url,
        )
        self.llm = ChatOllama(
            model=settings.chat_model,
            base_url=settings.ollama_base_url,
            temperature=0.2,
        )
        self.vectorstore = Chroma(
            collection_name="documents",
            embedding_function=self.embeddings,
            persist_directory=settings.chroma_persist_dir,
        )
        self.text_splitter = RecursiveCharacterTextSplitter(
            chunk_size=500,
            chunk_overlap=50,
        )
        self.prompt = ChatPromptTemplate.from_messages([
            ("system", "You are a helpful assistant. Answer the question based only on the provided context. If the context does not contain the answer, say so.\n\nContext:\n{context}"),
            ("human", "{question}"),
        ])
        self.chain = self.prompt | self.llm | StrOutputParser()

    async def ingest(self, content: str, metadata: dict) -> tuple[str, int]:
        doc_id = str(uuid.uuid4())
        chunks = self.text_splitter.create_documents(
            texts=[content],
            metadatas=[{**metadata, "doc_id": doc_id}],
        )
        self.vectorstore.add_documents(chunks)
        return doc_id, len(chunks)

    async def query(self, question: str) -> tuple[str, list[str]]:
        results = self.vectorstore.similarity_search(question, k=3)
        context = "\n\n".join(doc.page_content for doc in results)
        sources = list(set(
            doc.metadata.get("doc_id", "unknown") for doc in results
        ))
        answer = await self.chain.ainvoke({
            "context": context,
            "question": question,
        })
        return answer, sources

    def health_check(self) -> tuple[bool, int]:
        try:
            self.llm.invoke("ping")
            ollama_ok = True
        except Exception:
            ollama_ok = False
        try:
            count = self.vectorstore._collection.count()
        except Exception:
            count = 0
        return ollama_ok, count

Test the complete API locally

Your application code is complete. Before containerizing it, you should verify everything works on your local machine. The API needs two Ollama models to function: one for chat and one for embeddings.

Pull the chat model by running this command:

ollama pull llama3.2:1b

Pull the embedding model by running:

ollama pull nomic-embed-text

Why two models?

RAG pipelines need two different models. The embedding model (nomic-embed-text) converts text into vectors for similarity search. The chat model (llama3.2:1b) reads the retrieved context and generates a natural language answer.

Your app's config defaults to http://host.minikube.internal:11434 as the Ollama URL. That hostname only resolves from inside a Kubernetes pod. For local testing, you need to override it to point at localhost.

Set the environment variable and start the development server by running:

macOS

RAG_OLLAMA_BASE_URL=http://localhost:11434 fastapi dev app/main.py

Windows

$env:RAG_OLLAMA_BASE_URL="http://localhost:11434"
fastapi dev app/main.py

You should see output confirming the server started at http://127.0.0.1:8000.

Server failed to start?

If you see an ImportError, double check that your virtual environment is active and all packages from requirements.txt are installed.

help me fix a startup error with my FastAPI RAG agent

Open your browser and navigate to http://127.0.0.1:8000/health.

You should see a JSON response showing "status": "healthy" and "ollama_connected": true. This confirms your RAGService successfully connected to Ollama.

Now test the full RAG flow. Open http://127.0.0.1:8000/docs in your browser to access the interactive Swagger UI.
Click the POST /ingest endpoint, then click Try it out.
Replace the request body with this sample document:

{
  "content": "Kubernetes is a container orchestration platform. It automates deployment, scaling, and management of containerized applications. Pods are the smallest deployable units in Kubernetes.",
  "metadata": {"source": "test"}
}

Click Execute. You should see a response with "status": "ingested" and a document_id.
Now click the POST /query endpoint, then click Try it out.
Replace the request body with this question:

{
  "question": "What are pods in Kubernetes?"
}

Click Execute. You should see a JSON response with an answer field that mentions pods being the smallest deployable units.

Getting a 503 or connection error?

Make sure Ollama is running in the background. You can verify by visiting http://localhost:11434 in your browser. If you see a response, the server is up.

If the health endpoint shows "ollama_connected": false, confirm you set the RAG_OLLAMA_BASE_URL environment variable before starting the server.

help me debug my RAG agent connection to Ollama

Press Ctrl+C in your terminal to stop the development server.

Your RAG agent is working end-to-end. Next up, you will package this application into a Docker container so it can run inside your Kubernetes cluster.

Containerize with Docker

Your RAG agent works locally, but Kubernetes doesn't run Python scripts directly. It runs containers.

In this step, you'll package your FastAPI application into a Docker image. Then you'll load that image into Minikube so your cluster can use it without pulling from a remote registry.

In this step, get ready to:

Write a Dockerfile for your FastAPI application.
Build and test the container locally.
Load the image into Minikube's container runtime.

Write the Dockerfile

A Dockerfile is a recipe that tells Docker how to build your application into a portable image. It specifies the base environment, installs dependencies, copies your code, and defines how to start the app.

In your code editor, create a new file called Dockerfile in the root of your project folder (the same level as requirements.txt).
Paste the following code into your Dockerfile:

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app/ ./app/

EXPOSE 8000

CMD ["fastapi", "run", "app/main.py", "--host", "0.0.0.0", "--port", "8000"]

What does this Dockerfile do?

Each instruction builds a layer in your image:

FROM python:3.12-slim uses a minimal Python image as the foundation. The slim variant keeps the image small by excluding compilers and docs.
COPY requirements.txt is placed before COPY app/ so that Docker only reinstalls dependencies when requirements.txt changes. This is called layer caching and speeds up rebuilds.
CMD uses fastapi run (production mode) bound to 0.0.0.0 so the app accepts connections from outside the container.

Save Dockerfile.

Why fastapi run instead of fastapi dev?

The fastapi dev command enables hot-reloading and debug mode, which is great for local development. Inside a container destined for Kubernetes, you want fastapi run instead. It disables debug mode and runs with production-optimized settings.

Build and test the container locally

Before loading the image into Minikube, confirm it builds successfully and the server starts up.

Build the Docker image by running this command in your terminal (make sure you're in the project folder where your Dockerfile lives):

docker build -t rag-agent:latest .

What does this command do?

docker build reads your Dockerfile and builds an image layer by layer.
-t rag-agent:latest tags the image with the name rag-agent and version latest. You'll reference this tag in your Kubernetes manifests later.
. tells Docker to use the current directory as the build context.

Wait for the build to finish. You should see each step complete and a final line confirming the image was successfully built.

Build failing?

Make sure Docker Desktop is running. You can check by looking for the Docker whale icon in your system tray.
Confirm your terminal is in the same folder as your Dockerfile. Run ls and verify you see Dockerfile, requirements.txt, and the app/ folder listed.
If you see a Python dependency error, double-check that requirements.txt matches the version pins from the previous step.

Help me debug my Docker build error

Now test that the container starts correctly. You will not be able to reach Ollama from inside the container yet, but you can verify the server boots up.

Run the container locally by running this command:

docker run --rm -p 8000:8000 rag-agent:latest

What does this command do?

--rm automatically removes the container after it stops, keeping your system clean.
-p 8000:8000 maps port 8000 on your machine to port 8000 inside the container.

Watch the terminal output. You should see FastAPI start up and display a message showing it is listening on 0.0.0.0:8000.
Press Ctrl+C to stop the container once you have confirmed it starts.

Ollama connection errors are expected here

The health check will likely show ollama_connected: false because the container cannot reach your host's Ollama using host.minikube.internal outside of Minikube. That hostname only resolves inside the Minikube cluster. The important thing is that FastAPI starts without crashing.

Load the image into Minikube

Minikube runs its own container runtime, separate from your local Docker. To make your image available to the cluster, you need to explicitly load it into Minikube's runtime.

Load the image into your Minikube cluster by running this command:

minikube image load rag-agent:latest

What does this command do?

This copies the rag-agent:latest image from your local Docker daemon into Minikube's in-cluster container runtime. Without this step, your Kubernetes pods would fail to pull the image because it only exists on your machine.

This is why the Deployment manifest will use imagePullPolicy: Never in the next step. It tells Kubernetes not to look for the image in a remote registry and only use what is already loaded locally.

Wait for the command to complete. This may take a minute as it transfers the image.
Verify the image is loaded by running:

minikube image ls

Look for docker.io/library/rag-agent:latest in the output. This confirms Minikube has your image ready for deployments.

Image not showing in the list?

Make sure your Minikube cluster is running. Run minikube status to check.
Confirm the image exists locally by running docker images rag-agent. If it is not listed, the build step did not complete successfully.

Help me troubleshoot loading my Docker image into Minikube

✔️ Awesome, I've got everything!

Great. Double check you have saved your Dockerfile and that the image appears in minikube image ls.

ⓧ I'd like to double check the full code

Here is the complete Dockerfile for this step:

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app/ ./app/

EXPOSE 8000

CMD ["fastapi", "run", "app/main.py", "--host", "0.0.0.0", "--port", "8000"]

Your application is now containerized and ready for Kubernetes. Next up, you'll write the manifests that tell Kubernetes how to run, configure, and expose your RAG agent.

Deploy to Kubernetes

Your RAG agent is containerized and sitting inside Minikube's container runtime. A single container on its own has no way to restart itself if it crashes, scale to handle more traffic, or expose a stable network address.

Kubernetes solves all three problems through declarative manifests. You describe what you want (two replicas, a health check, a network port) and Kubernetes makes it happen. In this step, you'll write three manifests that configure, deploy, and expose your API.

In this step, get ready to:

Create a ConfigMap with your application's environment variables.
Write a Deployment manifest with liveness and readiness probes.
Expose the API through a Service and access it from your browser.

Create a ConfigMap for environment configuration

Your app reads settings like the Ollama URL and model names from environment variables prefixed with RAG_. A ConfigMap stores these values centrally so every pod receives identical configuration without baking values into the container image.

Create a k8s directory to hold your Kubernetes manifests by running this command:

mkdir k8s

Confirm the directory exists by running:

ls

You should see k8s listed alongside your app folder and Dockerfile.

Create a file called k8s/configmap.yaml and paste the following content:

apiVersion: v1
kind: ConfigMap
metadata:
  name: rag-agent-config
data:
  RAG_OLLAMA_BASE_URL: "http://host.minikube.internal:11434"
  RAG_CHAT_MODEL: "llama3.2:1b"
  RAG_EMBEDDING_MODEL: "nomic-embed-text"
  RAG_CHROMA_PERSIST_DIR: "/app/chroma_data"

What does this manifest do?

A ConfigMap is a Kubernetes object that holds key-value pairs. The data section maps environment variable names to their values.

When a pod references this ConfigMap, Kubernetes injects all four variables into the container's environment. Your Settings class in app/config.py reads them automatically because it uses the RAG_ prefix.

Save k8s/configmap.yaml.
Apply the ConfigMap to your cluster by running:

kubectl apply -f k8s/configmap.yaml

You should see configmap/rag-agent-config created in the output.

Verify the ConfigMap is stored in the cluster by running:

kubectl get configmap rag-agent-config

You should see a row with rag-agent-config and a DATA column showing 4 (one entry per key).

Seeing a validation error?

Check that your YAML indentation uses spaces, not tabs. YAML is whitespace-sensitive and tabs cause parse errors.

Make sure the data keys are indented exactly two spaces under data:.

Help me debug my ConfigMap YAML

✔️ Awesome, I've got everything!

Great. Double check you've saved k8s/configmap.yaml and the ConfigMap is applied.

ⓧ I'd like to double check the full code

apiVersion: v1
kind: ConfigMap
metadata:
  name: rag-agent-config
data:
  RAG_OLLAMA_BASE_URL: "http://host.minikube.internal:11434"
  RAG_CHAT_MODEL: "llama3.2:1b"
  RAG_EMBEDDING_MODEL: "nomic-embed-text"
  RAG_CHROMA_PERSIST_DIR: "/app/chroma_data"

Write the Deployment manifest with health probes

A Deployment tells Kubernetes how many copies of your container to run and how to monitor their health. You'll configure liveness and readiness probes that hit your /health endpoint so Kubernetes can automatically restart unhealthy pods.

Create a file called k8s/deployment.yaml and paste the following content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: rag-agent
  labels:
    app: rag-agent
spec:
  replicas: 2
  selector:
    matchLabels:
      app: rag-agent
  template:
    metadata:
      labels:
        app: rag-agent
    spec:
      containers:
        - name: rag-agent
          image: rag-agent:latest
          imagePullPolicy: Never
          ports:
            - containerPort: 8000
          envFrom:
            - configMapRef:
                name: rag-agent-config
          livenessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 30
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 10
            periodSeconds: 5
          resources:
            requests:
              memory: "256Mi"
              cpu: "250m"
            limits:
              memory: "512Mi"
              cpu: "500m"

What does this manifest do?

replicas: 2 runs two identical pods for redundancy.
imagePullPolicy: Never tells Kubernetes to use the image already loaded in Minikube's runtime instead of trying to pull from a remote registry.
envFrom injects all keys from your ConfigMap as environment variables.
livenessProbe hits /health every 10 seconds after a 30-second startup delay. If it fails, Kubernetes restarts the pod.
readinessProbe checks /health every 5 seconds. A pod only receives traffic once this probe passes.
resources sets minimum (requests) and maximum (limits) CPU and memory for each pod.

Why two different probes?

The liveness probe answers: is this pod alive? If not, kill it and start a fresh one. The readiness probe answers: is this pod ready for traffic? If not, stop sending it requests until it recovers.

Separating the two prevents Kubernetes from routing requests to pods that are still loading models, while also catching pods that have hung permanently.

Save k8s/deployment.yaml.
Apply the Deployment to your cluster by running:

kubectl apply -f k8s/deployment.yaml

You should see deployment.apps/rag-agent created in the output.

Watch the pods come up by running:

kubectl get pods

You should see two pods with names starting with rag-agent-. The READY column will show 1/1 and STATUS will show Running once both pods pass their readiness probes.

Pods still showing 0/1?

The readiness probe has a 10-second initial delay. Wait about 30 seconds and run kubectl get pods again. If they remain at 0/1 after a minute, check logs with kubectl logs followed by the pod name.

Pods stuck in CrashLoopBackOff?

This usually means the container fails to start. Common causes:

Ollama is not running on your host, or is not bound to 0.0.0.0. Make sure the OLLAMA_HOST environment variable is set to 0.0.0.0 and Ollama is restarted.
The image name doesn't match. Confirm you loaded the image as rag-agent:latest into Minikube.

Help me debug my pods in CrashLoopBackOff

✔️ Awesome, I've got everything!

Great. Double check you've saved k8s/deployment.yaml and both pods show 1/1 READY.

ⓧ I'd like to double check the full code

apiVersion: apps/v1
kind: Deployment
metadata:
  name: rag-agent
  labels:
    app: rag-agent
spec:
  replicas: 2
  selector:
    matchLabels:
      app: rag-agent
  template:
    metadata:
      labels:
        app: rag-agent
    spec:
      containers:
        - name: rag-agent
          image: rag-agent:latest
          imagePullPolicy: Never
          ports:
            - containerPort: 8000
          envFrom:
            - configMapRef:
                name: rag-agent-config
          livenessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 30
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 10
            periodSeconds: 5
          resources:
            requests:
              memory: "256Mi"
              cpu: "250m"
            limits:
              memory: "512Mi"
              cpu: "500m"

Create a Service and expose the API with minikube tunnel

Your pods are running, but they have no stable network address. A Service gives your pods a single, consistent endpoint and load-balances traffic between them.

Create a file called k8s/service.yaml and paste the following content:

apiVersion: v1
kind: Service
metadata:
  name: rag-agent-service
spec:
  selector:
    app: rag-agent
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8000
  type: LoadBalancer

What does this manifest do?

selector: app: rag-agent routes traffic only to pods with that label (your Deployment's pods).
port: 80 is the port clients connect to. targetPort: 8000 is the port inside your container where FastAPI listens.
type: LoadBalancer requests an external IP so you can access the Service from outside the cluster.

Save k8s/service.yaml.
Apply the Service to your cluster by running:

kubectl apply -f k8s/service.yaml

You should see service/rag-agent-service created in the output.

Verify the Service is registered by running:

kubectl get svc rag-agent-service

You should see a row with TYPE LoadBalancer and PORT(S) showing 80. The EXTERNAL-IP will show <pending> until you start the tunnel.

Why is the external IP pending?

Minikube doesn't have a real cloud load balancer. The minikube tunnel command simulates one by creating a network route from your host to the cluster. Once the tunnel runs, the EXTERNAL-IP becomes 127.0.0.1.

Open a second terminal window. You need to keep the tunnel running while you access the API.
In the new terminal, start the tunnel by running:

minikube tunnel

What does minikube tunnel do?

This command creates a network route so your host machine can reach LoadBalancer services inside the cluster. It must stay running in this terminal for the duration of your session. If you close it, the external IP disappears.

On Windows, the tunnel may prompt for administrator permissions. Allow it to proceed.

Switch back to your original terminal.
Confirm the external IP is assigned by running:

kubectl get svc rag-agent-service

The EXTERNAL-IP should now show 127.0.0.1 instead of <pending>.

Test your deployed API by opening your browser and navigating to:

http://127.0.0.1/health

You should see a JSON response showing your health check output with "status": "healthy", "ollama_connected": true, and "documents_indexed": 0.

Browser showing connection refused?

Make sure minikube tunnel is still running in your second terminal. If you closed it, restart it.
Check that the EXTERNAL-IP shows 127.0.0.1 in kubectl get svc rag-agent-service. If it still shows <pending>, the tunnel is not active.
Confirm your pods are ready with kubectl get pods. Both pods should show 1/1 in the READY column.

Help me access my Kubernetes service from the browser

✔️ Awesome, I've got everything!

Great. Double check you've saved k8s/service.yaml and the health endpoint responds in your browser.

ⓧ I'd like to double check the full code

apiVersion: v1
kind: Service
metadata:
  name: rag-agent-service
spec:
  selector:
    app: rag-agent
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8000
  type: LoadBalancer

Your RAG agent is now running on Kubernetes with two replicas, automatic health monitoring, and a stable network endpoint. Next up, you'll learn how to operate and debug this deployment using kubectl commands.

Operate and Debug Your Deployment

Your RAG agent is deployed and serving traffic. In a real production environment, the deploy is just the beginning.

Operating a Kubernetes deployment means knowing how to inspect pod health, read container logs, scale capacity on demand, and roll out updates without downtime. In this step, you'll practice all of these skills on your running cluster.

In this step, get ready to:

Inspect pods and read container logs with kubectl.
Scale your deployment up and down manually.
Perform a rolling update and learn to roll back.

Use kubectl to inspect pods and view logs

When something goes wrong in production, the first thing you do is check whether your pods are running and read their logs. These two commands are the foundation of Kubernetes debugging.

List all pods in your deployment by running this command:

kubectl get pods

What does this show?

This lists every pod managed by your cluster. You should see two pods with names starting with rag-agent- followed by a random suffix. The STATUS column shows Running if everything is healthy.

The READY column shows how many containers in the pod are passing their readiness probe. You should see 1/1 for each pod.

Pick one of your pod names from the output above, then get detailed information about it by running:

kubectl describe pod rag-agent-[[POD_SUFFIX="your pod's random suffix, e.g. 7d4b9c6f5-x2k9z"]]

What does kubectl describe show?

The describe command shows the full lifecycle of a pod. You can see which node it landed on, what image it pulled, the environment variables injected from your ConfigMap, and the event history (scheduling, pulling, starting).

Look at the Conditions section near the bottom. Both Ready and ContainersReady should show True.

View the application logs from one of your pods by running:

kubectl logs rag-agent-[[POD_SUFFIX="your pod's random suffix, e.g. 7d4b9c6f5-x2k9z"]]

You should see FastAPI startup logs showing the server listening on port 8000.

Follow logs in real time

Add the -f flag to stream logs continuously, just like tail -f on a regular file. Press Ctrl+C to stop streaming.

Pod showing CrashLoopBackOff?

This usually means the container starts and immediately exits. Check that Ollama is running on your host and bound to 0.0.0.0. The pods reach Ollama via host.minikube.internal:11434, so if Ollama is only listening on localhost the health check fails and the pod restarts.

help me debug my pod that is stuck in CrashLoopBackOff

Scale the deployment up and down

Scaling is how you handle traffic spikes. Kubernetes lets you increase or decrease the number of pod replicas with a single command, and it handles all the networking and load balancing automatically.

Scale your deployment up to 4 replicas by running:

kubectl scale deployment rag-agent --replicas=4

Watch the new pods come online by running:

kubectl get pods

You should now see 4 pods listed. The two new pods will briefly show a ContainerCreating status before transitioning to Running.

What is happening behind the scenes?

When you scale up, Kubernetes creates new pods from the same template in your Deployment manifest. Each new pod pulls the same image, injects the same ConfigMap environment variables, and is added to the Service's load balancer pool once its readiness probe passes.

Scale back down to 2 replicas by running:

kubectl scale deployment rag-agent --replicas=2

Confirm the extra pods are terminating by running:

kubectl get pods

You should see the two extra pods in Terminating status. After a few seconds, only your original 2 pods remain.

Still seeing 4 pods?

Pods take a few seconds to terminate gracefully. Run kubectl get pods again after 10-15 seconds. If they persist, check for stuck finalizers with kubectl describe pod on one of the terminating pods.

help me figure out why my pods are not terminating

Update the deployment with a rolling update

In production, you deploy new versions without taking the service offline. Kubernetes achieves this with rolling updates, which gradually replace old pods with new ones.

First, rebuild your Docker image with a version tag so Kubernetes recognizes it as a new image:

minikube image build -t rag-agent:v2 .

Why a new image tag?

Kubernetes triggers a rolling update when the pod template changes. Since your deployment uses imagePullPolicy: Never, changing the image tag from rag-agent:latest to rag-agent:v2 tells Kubernetes the pods need replacing.

Trigger the rolling update by telling Kubernetes to use the new image tag:

kubectl set image deployment/rag-agent rag-agent=rag-agent:v2

What does this command do?

The syntax is kubectl set image deployment/<name> <container>=<image>. Here, deployment/rag-agent is your Deployment name, the first rag-agent is the container name from the manifest, and rag-agent:v2 is the new image.

Kubernetes now starts new pods with the v2 image and terminates old pods one at a time, keeping the service available throughout.

Watch the rollout progress by running:

kubectl rollout status deployment/rag-agent

You should see messages about new replicas being updated and old replicas scaling down. Once complete, you'll see a confirmation that the rollout finished successfully.

Now practice rolling back. Undo the update you just performed by running:

kubectl rollout undo deployment/rag-agent

What does rollout undo do?

This reverts your deployment to the previous revision. Kubernetes stores revision history in its ReplicaSets. The undo command swaps back to the last known good configuration, which in this case is the original rag-agent:latest image.

Confirm the rollback completed by running:

kubectl rollout status deployment/rag-agent

You should see confirmation that the deployment has successfully rolled out.

Verify your pods are back on the original image by running:

kubectl get pods

You should see 2 pods in Running status with fresh names (the rollback created new pods).

Rollout stuck or pods not starting?

If the rollout hangs, the new pods may be failing their readiness probe. Run kubectl get pods to check their status, then kubectl describe pod on a failing pod to see the events. The most common cause is Ollama not being reachable from the new pods.

help me debug a stuck rolling update

Secret mission

Autoscale Under Load

Your deployment handles traffic well with two replicas. But what happens when a wave of requests hits and CPU spikes? In production, teams don't scale pods manually. They configure Kubernetes to do it automatically. In this secret mission, you'll add a Horizontal Pod Autoscaler and load-test your API to watch Kubernetes spin up new pods on its own.

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

Minikube cluster (includes all Kubernetes Deployments, Services, ConfigMaps, and the HPA).
Docker image rag-agent:latest (stored inside the Minikube container runtime).
Project folder on your Desktop containing app/, k8s/, Dockerfile, and requirements.txt.
Ollama models pulled during setup (llama3.2:1b and nomic-embed-text).
Minikube and kubectl installed via winget.

Keep everything running

No action needed. Choose this if you're still actively experimenting with Kubernetes or planning to add more endpoints to your API.

Your Minikube cluster stays running and uses CPU and memory on your machine.
The minikube tunnel process must stay open in its terminal for the LoadBalancer to remain accessible.
Ollama models remain on disk and are ready for instant use in other projects.

Pause - I'll come back to this later

Pause your cluster to free up CPU and memory without losing any state. Your Deployments, Services, and ConfigMaps remain intact.

Stop the minikube tunnel process by pressing Ctrl+C in the terminal where it is running.
Pause the cluster by running:

minikube pause

When you're ready to resume, run:

minikube unpause

Then restart the tunnel in a separate terminal:

minikube tunnel

Delete - I don't want to use this again

Remove all project resources and start fresh. This deletes the cluster, all Kubernetes objects inside it, and the Docker images stored in its runtime.

Stop the minikube tunnel process by pressing Ctrl+C in the terminal where it is running.
Delete the Minikube cluster and all its data by running:

minikube delete

Remove the Ollama models you pulled for this project by running:

ollama rm llama3.2:1b
ollama rm nomic-embed-text

Delete the project folder from your Desktop by running:

cd ~\Desktop
Remove-Item -Recurse -Force rag-agent-api

Uninstall Minikube and kubectl by running:

winget uninstall --id Kubernetes.minikube
winget uninstall --id Kubernetes.kubectl

Nice Work!

You just deployed an AI-powered RAG agent to Kubernetes! You built a complete production-style AI service from scratch, containerized it, and orchestrated it on a real cluster.

You've learned how to:

Build a FastAPI REST API with async endpoints, Pydantic validation, and dependency injection.
Create a RAG agent that ingests documents and answers questions using LangChain, ChromaDB, and Ollama.
Deploy to Kubernetes with Deployments, Services, ConfigMaps, and health probes on a local Minikube cluster.
Added a Horizontal Pod Autoscaler and watched Kubernetes scale your pods automatically in response to load.

Ready to quiz yourself?

Deploy a RAG Agent API to Kubernetes

Introduction

30 Second Summary

What You'll Build

Are there any prerequisites?

Before We Start

Before We Start...

Install Minikube and kubectl

In this step, get ready to:

Install Minikube on Windows using winget

✔️ I see linux

ⓧ I see windows

What does this command do?

✔️ I see a version number

ⓧ Command not found

Install kubectl and verify cluster connectivity

What does this command do?

✔️ I see a version number

ⓧ Command not found

Start your first Minikube cluster and explore the dashboard

What does this command do?

Cluster not starting?

What is the Kubernetes dashboard?

Build the FastAPI Application

In this step, get ready to:

Create the project structure with Pydantic models

What is __init__.py?

Don't see __init__.py?

What does this file do?

What does this code do?

What does this code do?

Why host.minikube.internal?

Unsure where to save these files?

Build async endpoints with dependency injection

What does this code do?

What does this code do?

Seeing a red underline on the RAGService import?

Add a /health endpoint for Kubernetes probes

What does this code do?

Not sure where /health goes in the file?

✔️ Awesome, I've got everything!

ⓧ I'd like to double check the full code

Wire Up the RAG Agent

In this step, get ready to:

Connect LangChain with Ollama and ChromaDB

What does this code do?

Build the agent logic that retrieves context and generates answers

What does the ingest method do?

What does the query method do?

What does health_check do?

Seeing a red squiggly line under an import?

✔️ Awesome, I've got everything!

ⓧ I'd like to double check the full code

Test the complete API locally

Why two models?

macOS

Windows

Server failed to start?

Getting a 503 or connection error?

Containerize with Docker

In this step, get ready to:

Write the Dockerfile

What does this Dockerfile do?

Why fastapi run instead of fastapi dev?

Build and test the container locally

What does this command do?

Build failing?

What does this command do?

Ollama connection errors are expected here

Load the image into Minikube

What does this command do?

Image not showing in the list?

✔️ Awesome, I've got everything!

ⓧ I'd like to double check the full code

Deploy to Kubernetes

In this step, get ready to:

Create a ConfigMap for environment configuration

What does this manifest do?

Seeing a validation error?

✔️ Awesome, I've got everything!

What is init.py?

Don't see init.py?