Containerize a RAG API with Docker

Learn how professional DevOps engineers package applications for consistent deployment!

Introduction

This project has been archived

We're no longer updating this project - we recommend checking out our active projects instead!

If you've already completed this project, don't worry! Your documentation is still available for you to check out and share.

If you have any questions, leave a question in our Discord and we'll get back to you ASAP!

⚡️ 30 second Summary

How do big companies like Netflix, Uber, and Spotify make sure their apps always work well and the same way, whether they're running on a laptop, a company server, or in the cloud?

They use modern DevOps tools. This includes using containers to bundle up their code and dependencies so applications run identically anywhere - your laptop, a teammate's computer, or a production server.

What you'll build

Welcome to Project 2 of a 4-part DevOps × AI series! In this project, you'll containerize a RAG API with Docker.

This project builds on Project #1 of this series: Build a RAG API with FastAPI, where you built a RAG API using FastAPI, Chroma, and Ollama. After this project, you can continue with Project #3: Deploy to Kubernetes and Project #4: Automate Testing with GitHub Actions.

I haven't done Project #1 yet

No worries! You can either complete Project #1 first (which we highly recommend!), or we'll guide you through a quick setup at the start of this project to catch up. Either way works.

By the end, you'll have:

🐳 A Docker image of your RAG API
💎 Your image shared to the world through Docker Hub

This is the same workflow many professional DevOps engineers use to ship AI workloads into production at scale 🛳️ The best part? Docker runs locally on your computer with no cloud costs or fees.

You might be wondering...

Why Docker?

You could build the perfect AI API that works flawlessly on your laptop, but when you deploy it to a server, it could still break. Docker solves this by packaging your AI API and everything it needs into a container that runs identically anywhere - your laptop, a teammate's computer, or a production server.

🙋‍♀️ I don't have experience in Docker!

You don't need any prior experience in DevOps or Docker to do this project. We'll explain all the concepts along the way, and you can always ask questions as they come up.

Want a complete demo of how to do this project, from start to finish? Check out our 🎬 walkthrough with Natasha 🎬

If you're up for a bit of a challenge, quiz yourself on the key concepts up ahead in this project.

This project is part of a series:

Set Up Your API

Let's set up your RAG API before we containerize it in Docker!

If you've done Project #1: Build a RAG API with FastAPI of this series, this will be super quick. If not, no worries - we'll set up each component as we go.

What did we do in Project #1 again?

In Project #1: Build a RAG API with FastAPI, you learned how to create a RAG (Retrieval-Augmented Generation) API. This means you built an API that can use AI to understand your documents and answer questions about them. You wrote code to load your knowledge base into Chroma, then created a FastAPI app that uses Ollama to give answers using your own data.

💡 What is an API?

An API (Application Programming Interface) is a way for different programs to communicate with each other.

In this project, your RAG API receives questions from users, searches through your documents to find relevant information, uses AI to generate answers, and sends those answers back. Without an API, users would have no way to interact with your RAG system - it's the interface that makes your system accessible!

In this step, you're going to:

Set up your RAG API's code, database and dependencies
Set up your virtual environment
Run Ollama

Open Cursor

Open Cursor in your computer.

I don't have Cursor!

We'll use Cursor for this step-by-step guide, but you can pick another IDE like VS Code if you prefer.

If you don't have another IDE to use, check out Cursor's download page if you'd like to have a go at doing this on your own, or try Step #1 of our Build a Mini Spotify with Cursor project for step-by-step guidance!

Select Open project from the welcome screen.

Check your project folder

Select Desktop.

Can you find the project folder called nextwork-rag-api?

Where can I find the folder?

Your API code should be in a project folder on your computer. Common locations are your Desktop or Documents folder.

✅ Yes, I have the folder

Perfect!

Select your project folder called nextwork-rag-api
Select Open.

❌ No, I can't find the folder

Let's create it:

From Cursor's folder selection window, click into Desktop.
Select New folder.
Name your folder nextwork-rag-api

Select this folder as your repository.

Check your required files

Do you have all the required API files in your nextwork-rag-api folder?

You should have:

app.py - your FastAPI application
embed.py - the script that creates embeddings
k8s.txt - your knowledge base document

Why do we need these files?

Your RAG API needs three files to work:

app.py has your API code. This handles requests from users and connects everything together.
embed.py - Turns your text into embeddings (i.e. numbers that represent the meaning of a word) that the computer can search through. This lets Chroma find the right information when someone asks a question.
k8s.txt - The file with your information. This is what your API reads to answer questions.

Without these files, your API won't work.

✅ Yes, I have all the files

Nice! Let's check if you've got all your Python packages next.

❌ No, I'm missing some files

No worries! Let's create the files that make up your API.

Make sure you're in your nextwork-rag-api folder in Cursor.

In Cursor's file explorer (left sidebar), select New File.

Name the file k8s.txt
Press Enter.
Click inside the newly created k8s.txt file in the editor.
Copy and paste in the following content:

Kubernetes is a container orchestration platform used to manage containers at scale.

Press Cmd + S (macOS) or Ctrl + S (Windows/Linux) to save the file.

You should see a white circle disappear next to the file name once it's saved.

In Cursor's file explorer (left sidebar), select New File again.
Name the file embed.py
Press Enter.

Click inside the newly created embed.py file in the editor.
Copy and paste in the following content:

import chromadb

client = chromadb.PersistentClient(path="./db")
collection = client.get_or_create_collection("docs")

with open("k8s.txt", "r") as f:
    text = f.read()

collection.add(documents=[text], ids=["k8s"])

print("Embedding stored in Chroma")

Press Cmd + S (macOS) or Ctrl + S (Windows/Linux) to save the file.
In Cursor, select New File in your project's root.
Name your new Python file app.py
Press Enter.

Open app.py in your text editor and copy and paste in the following content:

from fastapi import FastAPI
import chromadb
import ollama

app = FastAPI()
chroma = chromadb.PersistentClient(path="./db")
collection = chroma.get_or_create_collection("docs")

@app.post("/query")
def query(q: str):
    results = collection.query(query_texts=[q], n_results=1)
    context = results["documents"][0][0] if results["documents"] else ""

    answer = ollama.generate(
        model="tinyllama",
        prompt=f"Context:\n{context}\n\nQuestion: {q}\n\nAnswer clearly and concisely:"
    )

    return {"answer": answer["response"]}

Press Cmd + S (macOS) or Ctrl + S (Windows/Linux) to save the file.

Having trouble creating the files?

Try troubleshooting your error here, for my RAG API project, can you please suggest solutions that match my operating system?) or share your error message with the NextWork community!

Set up your virtual environment

Why do we need a virtual environment?

A virtual environment keeps your project's Python packages separate from other projects on your computer - without a virtual environment, installing packages for one project could break another!

When we put your API in a Docker container, Docker will create its own separate environment. But running a virtual environment now makes sure your computer setup matches what will run in the container.

Do you have a venv folder in your project directory?

✅ Yes, I have the venv folder

Great! Now let's activate it.

Select Terminal -> New Terminal in Cursor's menu bar.

Your terminal should look like this:

Run the command for activating your virtual environment:

🍎 macOS

source venv/bin/activate

You should see (venv) appear at the beginning of your terminal prompt.

🖼️ Windows

venv\Scripts\activate

You should see (venv) appear at the beginning of your PowerShell prompt.

Perfect! Your virtual environment is now activated.

❌ No, I don't have the venv folder

No worries!

Select Terminal -> New Terminal in Cursor's menu bar.

First, let's check which Python version you have:

🍎 macOS

python3 --version

Check what you see:

✅ I see Python 3.13.x

Perfect! Create your virtual environment:

python3 -m venv venv

⚠️ I see Python 3.14.x

Python 3.14 is too new for some packages we need. Let's use Python 3.13 explicitly:

Why Python 3.13?

We're using Python 3.13 for this project because it has full compatibility with all the packages we need (FastAPI, ChromaDB, Ollama, and Uvicorn).

Python 3.14 is very new and some packages haven't been updated to support it yet, which can cause compatibility issues. If you see Python 3.14 when checking your version, you'll need to use Python 3.13 explicitly when creating your virtual environment.

python3.13 -m venv venv

Don't have Python 3.13?

If you see command not found, you'll need to install Python 3.13 first. Visit the Python 3.13.11 release page and download the installer for macOS.

This creates a venv/ folder in your project directory.

Now activate your virtual environment:

source venv/bin/activate

You should see (venv) appear at the beginning of your terminal prompt.

🖼️ Windows

python --version

Check what you see:

✅ I see Python 3.13.x

Perfect! Create your virtual environment:

python -m venv venv

⚠️ I see Python 3.14.x

Python 3.14 is too new for some packages we need. Let's use Python 3.13 explicitly:

py -3.13 -m venv venv

Why Python 3.13?

We're using Python 3.13 for this project because it has full compatibility with all the packages we need (FastAPI, ChromaDB, Ollama, and Uvicorn). Python 3.14 is very new and some packages haven't been updated to support it yet, which can cause compatibility issues. If you see Python 3.14 when checking your version, you'll need to use Python 3.13 explicitly when creating your virtual environment.

💡 Don't have Python 3.13?

If you see command not found, you'll need to install Python 3.13 first. Visit the Python 3.13.11 release page and download the Windows installer. Make sure to check "Add Python to PATH" during installation.

This creates a venv/ folder in your project directory.

Now activate your virtual environment:

venv\Scripts\activate

You should see (venv) appear at the beginning of your PowerShell prompt.

Great! Your virtual environment is now activated.

Still having trouble setting up your virtual environment?

Try troubleshooting your error here, or share your error message with the NextWork community!

Check your Python dependencies

Let's check if you've got the Python packages you need to run your API!

🍎 macOS

Check the required packages are installed:

pip list | grep -E "fastapi|uvicorn|chromadb|ollama"

You should see all four packages listed in the terminal response.

I'm missing packages!

That's okay! Let's run the command for installing all the required packages:

pip install fastapi uvicorn chromadb ollama

Once the command has finished running, check that the required packages are installed:

pip list | grep -E "fastapi|uvicorn|chromadb|ollama"

💡 Why do we need these dependencies again?

Each Python package does something important for your RAG API to work:

FastAPI - Creates your API and handles web requests. This is what makes your app accessible via HTTP requests.
Uvicorn - Runs your FastAPI app. Think of it as the server that actually starts your API and listens for requests.
ChromaDB - Stores your document embeddings and searches through them. When someone asks a question, ChromaDB finds the most relevant information.
Ollama - The Python package that allows your API code to talk to the Ollama service running on your computer (i.e. this package is not the Ollama service itself, which you'll check for later). Your API uses this package to send questions to the AI model and receive answers.

When we put your API in a Docker container, all these packages need to be included. Checking they're installed now makes sure everything will work when we containerize it!

🖼️ Windows

Verify the required packages are installed:

pip list | Select-String -Pattern "fastapi|uvicorn|chromadb|ollama"

You should see all four packages listed in your terminal's response.

Why do we need these dependencies again?

Each Python package does something important for your RAG API to work:

FastAPI - Creates your API and handles web requests. This is what makes your app accessible via HTTP requests.
Uvicorn - Runs your FastAPI app. Think of it as the server that actually starts your API and listens for requests.
ChromaDB - Stores your document embeddings and searches through them. When someone asks a question, ChromaDB finds the most relevant information.
Ollama - Lets your API talk to the Ollama service on your computer. This is how your API sends questions to the AI model and gets answers back.

When we put your API in a Docker container, all these packages need to be included. Checking they're installed now makes sure everything will work when we containerize it!

🙋‍♀️ I'm missing packages!

Nooooo worries - let's run the command for installing all the required packages:

pip install fastapi uvicorn chromadb ollama

Once the command has finished running, check that the required packages are installed:

pip list | Select-String -Pattern "fastapi|uvicorn|chromadb|ollama"

Open PowerShell and navigate to your project folder.
Create a virtual environment (if you don't have one):

python -m venv venv

Activate your virtual environment:

venv\Scripts\activate

Install the required packages:

pip install fastapi uvicorn chromadb ollama

Verify your database is set up

Why do we need a database?

Your Chroma database stores vector embeddings of your documents. When a question is asked, your API searches these embeddings for meaning, and passes the search results (raw embeddings and text) to an LLM to generate a response.

Your database must exist before you containerize your API, so your Docker container can access the text you've stored in your database and answer questions.

In your project directory, check that the db/ folder exists.
Expand the db/ folder.

You should see Chroma database files inside.

✅ Yes, I have the database folder

Perfect!

❌ No, my database folder is missing or empty

Let's create it! We'll need to run your Python script embed.py, which converts your k8s.txt file into embeddings and stores them in your Chroma database.

In your terminal, run the embeddings script:

python embed.py

You should see Embedding stored in Chroma in your terminal, and a db/ folder in your project directory!

Still having trouble creating your database?

Try troubleshooting your error here, or share your error message with the NextWork community!

Run Ollama

Why do we need Ollama?

Ollama acts as a bridge or gateway to AI language models (LLMs) running locally on your compuyter.

When your API gets a question, it first finds relevant context in your Chroma database, then sends both the question and context to Ollama. Ollama then connects to the actual language model and asks it to generate a human-like response. This is the "Generation" part of RAG! Without Ollama as the gateway, your API could find information, but it couldn’t turn that information into a natural language answer.

Let's check if Ollama is running:

Run this command in your terminal:

curl http://localhost:11434

What does this command do?

This sends a quick HTTP request to Ollama to confirm the service is running; a response means Ollama is up, while a connection error means it isn't started yet.

What do you see?

✅ I see 'Ollama is running'

Awesome!

❌ I see 'Failed to connect'

No worries! Ollama is installed but not running. Let's start it:

🍎 macOS

Run this command:

open -a "Ollama"

What does this command do?

open -a "Ollama" tells macOS to launch the Ollama application. The -a flag specifies the application name, and macOS will find it in your Applications folder.

You should see the Ollama icon appear in your menu bar.

It says Ollama app not found!

No worries - this just means Ollama isn’t fully installed on your Mac yet. Here’s how you can complete your setup:

Paste this command in your terminal and press Enter:

  brew install --cask ollama

Wait for the installation to finish.

You should see a confirmation message when it’s done.

Run the command to open Ollama. bash open -a "Ollama"

🖼️ Windows

Open PowerShell and run:

ollama serve

What does this command do?

ollama serve starts the Ollama server on your computer. It listens on port 11434 for requests from your API. Unlike macOS, on Windows you need to keep the terminal window open for Ollama to keep running.

You should see:

Listening on 127.0.0.1:11434

It says Ollama app not found!

No worries - let's get it installed:

Open a browser and go to https://ollama.com/download/windows.
Select Download for Windows and run the installer (OllamaSetup.exe).
Accept the defaults so Ollama sets up its Windows service and adds itself to your PATH.
Re-open PowerShell and run ollama serve again. You should now see Listening on 127.0.0.1:11434.

If you still get "not recognized," restart PowerShell so the updated PATH loads, then retry ollama serve.

Keep this window open - Ollama must stay running!

Still having trouble installing or starting Ollama on Windows?

Try troubleshooting your error here, or share your error message with the NextWork community!

Wait 5-10 seconds for Ollama to start, then test it again:

curl http://localhost:11434

Still having trouble starting Ollama?

Try troubleshooting your error here, or share your error message with the NextWork community!

This tells us Ollama is ready 👌

Now let's verify the tinyllama model is available.

Run the tinyllama model

Why do we need tinyllama?

Tinyllama is the actual AI language model that generates responses to questions. When your API sends a question and context to Ollama, Ollama uses the tinyllama model to process that information and generate a coherent answer.

Tinyllama is also a smaller, faster model perfect for learning and local development - it's about 600MB compared to larger models that can be several gigabytes. Your containerized API will use this same model, so it needs to be downloaded and available on your host machine before you containerize your application.

Now let's make sure the tinyllama model is downloaded and ready:

Run this command to list all available models:

ollama list

What do you see?

✅ I see tinyllama in the list

Legendddd 👏

❌ I don't see tinyllama

No worries! Let's download the tinyllama model. This will take a few minutes (the model is about 600MB):

Run this command:

ollama pull tinyllama

You should see progress updates showing the download:

What just downloaded?

You just downloaded the tinyllama AI model - a file (about 600MB) that contains everything the AI needs to understand and generate text. When your containerized API needs to answer a question, it will send the question to Ollama, which uses this model to generate the response.

Wait for success to appear at the end. It can take up to 5 minutes to finish - no worries, good things take time 🌞
Verify the model is now available:

ollama list

Still having trouble downloading tinyllama?

Try troubleshooting your error here, or share your error message with the NextWork community!

You should see tinyllama in the output:

This means the tinyllama model is ready to use! Awesome, your API will use it to generate AI responses.

Test your API

Before we containerize your API, we want to make sure it actually works. This saves time later - if something's missing now, we can fix it before we start building Docker images!

In your terminal, start your API:

uvicorn app:app --host 127.0.0.1 --port 8000

What does this command do?

This command starts your FastAPI server.

uvicorn is the server that runs your API
app:app tells it to look for your FastAPI app in the app.py file
--host 127.0.0.1 --port 8000 makes your API available at http://127.0.0.1:8000 on your computer.

🙋‍♀️ It says uvicorn command not found!

This usually means your virtual environment isn't active or uvicorn isn't installed yet - make sure you to check you've got both ready.

Keep this terminal window open - your API needs to keep running.

Let's open a new terminal in Cursor to send a question to our very own API.

Select the New Terminal button at the top of your terminal window.

A new terminal will open up beneath your existing terminal.

Tip

You can always change terminals by switching tabs on the right hand side of your terminal window!

In the new terminal, run this command to test the API:

🍎 macOS

curl -X POST "http://127.0.0.1:8000/query" -G --data-urlencode "q=What is Kubernetes?"

What does this command do?

This command sends a test question to your API.

curl is a tool for making web requests
-X POST means we're sending data (not just reading)
-G --data-urlencode "q=What is Kubernetes?" sends your question as a parameter.

Your API should search its database, find information about Kubernetes, and return an AI-generated answer!

🖼️ Windows

In a new PowerShell window, run this command to test the API:

Invoke-WebRequest -Uri "http://127.0.0.1:8000/query?q=What%20is%20Kubernetes%3F" -Method POST

What does this command do?

This PowerShell command sends a test question to your API. Invoke-WebRequest is PowerShell's way of making web requests, the -Uri parameter is your API's address with the question (URL-encoded), and -Method POST means we're sending data. Your API should search its database, find information about Kubernetes, and return an AI-generated answer!

What do you see?

✅ I got a JSON response

Perfect! You should see a JSON response like:

{"answer":"Kubernetes is a container orchestration platform that helps manage containers at scale..."}

What does this prove?

This proves that your API is working correctly! It means:

Your FastAPI server is running properly
Chroma can access your knowledge base
Ollama is generating AI responses
All your dependencies are installed correctly

Great! - your API is working properly!

❌ I got an error or no response

No worries! Let's troubleshoot:

Connection refused

If you see Connection refused or Could not connect:

Make sure your API is still running in the first terminal window.
If the API stopped, check the error message in that terminal.
Start the API again:

uvicorn app:app --host 127.0.0.1 --port 8000

Wait a few seconds for it to start, then try your curl command again.

Ollama connection error

If you see errors about connecting to Ollama:

Make sure Ollama is running on your computer:

curl http://localhost:11434

If Ollama isn't running, start it (see Step #1 for instructions).
Your API needs Ollama to generate AI responses, so it must be running before you test your API.

Request timeout

If the request times out or takes very long:

The first request might be slow as Chroma loads the database and Ollama generates the response.
Wait 30-60 seconds and try again.
Check the terminal where your API is running for any error messages.

Other errors

If you see a different error:

Check the terminal where your API is running - the error message will usually tell you what's wrong.
Common issues:
- Missing files: Make sure app.py, embed.py, and k8s.txt are in your project folder.
- Missing packages: Make sure you activated your virtual environment and installed all dependencies.
- Port already in use: Another application might be using port 8000. Try a different port or stop the other application.

Still having trouble testing your API?

Try troubleshooting your error here, or share your error message with the NextWork community!

Great! All checks have passed and your API is ready.

Now let's install Docker so we can containerize your API 😙

Install and Set Up Docker

Now that your API is set up and working, let's install Docker.

Docker will help us package your RAG API into a container that runs identically anywhere - your laptop, a teammate's computer, or a production server.

Why containerize my RAG API?

Your RAG API needs specific versions of Python, FastAPI, ChromaDB, and the Ollama client, plus your pre-computed embeddings database. Without Docker, sharing your API means asking teammates to:

Install the exact Python version you're using
Set up a virtual environment
Install all dependencies
Run embed.py to build the database
Make sure everything matches your setup

Containerization packages all of this into one container. Your teammate (or a production server) just runs the container - no setup needed. The container includes your code, dependencies, and even your embeddings database, so it works the same everywhere.

In this step, you're going to:

Install Docker Desktop

Install Docker

First, let's check if you already have Docker installed.

Run the following command:

docker --version

❌ Command not found

That tells us Docker isn't installed yet - let's get it set up 🐳

Visit the Docker Desktop download page.

What's Docker Desktop? Is that the same thing as Docker?

Docker Desktop is a program that makes it easy to work with Docker, a tool for creating and managing containers. Think of Docker Desktop as the user-friendly app you install on your computer that includes Docker itself, along with a nice interface and helpful tools.

Engineers use Docker Desktop to build, test, and deploy applications right from their computers. It handles all the complex setup behind the scenes, so you can focus on using Docker in a user-friendly way.

Now download the correct installer for your system:

🍎 macOS

Select the correct download for your Mac

If you have Apple Silicon (M1, M2, or M3), select Download for Mac - Apple Chip.
If you have Intel, select Download for Mac - Intel Chip.

I don't know what version I should download for my Mac!

First, let's check what type of Mac you have.

Click the Apple logo in the top right of your display

Select About This Mac.

If you see Apple M1, Apple M2, or Apple M3 under Chip, you have Apple Silicon → Download for Mac - Apple Chip
If you see Intel Core, you have an Intel Mac → Download for Mac - Intel Chip

Install Docker Desktop

What's Docker Desktop? Is that the same thing as Docker?

Open the Docker.dmg file from your Downloads folder.

Drag the Docker icon to your Applications folder.

Open Docker from your Applications folder.

Why open Docker Desktop?

Opening Docker Desktop starts the Docker service in the background. This service needs to be running for any Docker commands to work. You'll see a whale icon in your menu bar when Docker is running.

Click Open at the security warning.

You may see a Docker Subscription Service Agreement screen.

Click Accept.

You'll see a configuration screen to finish setting up Docker Desktop.

Keep Use recommended settings selected (the default option).
Click Finish.

What are recommended settings?

Recommended settings usually include turning on the right virtualization backend for your OS, setting up file sharing so containers can read and write files from key folders on your machine, and giving Docker reasonable CPU and memory limits so it doesn't overwhelm your laptop. For most developers, these defaults are safe and work well out of the box, and you can always fine-tune them later in Settings if you notice Docker is using too many resources or you want to share different folders.

Docker may prompt for your password to apply these configurations.

Enter your password and click OK.

You'll see a Welcome to Docker screen.

Click Skip in the top right.

You should now see the Docker Desktop dashboard!

Now let's verify Docker is working!

Open Terminal on your Mac
Run this command:

docker --version

What should I see?

You should see something like Docker version 24.0.7, build afdd53b. The version and build ID on your computer will probably be different – that's totally expected and your Docker will work just the same. Why are version numbers different?

Perfect! Docker is installed and working on your Mac.

🖼️ Windows

Click Download for Windows 10/11 (64-bit)

While we wait for the download, did you know Docker Desktop on Windows requires WSL 2? Let's check if you have it installed.

Open PowerShell (right-click Start menu → Windows PowerShell).
Run the following command to check if WSL 2 is installed:

wsl --version

✅ I see WSL version info

Nice! WSL 2 is already installed.

⚠️ Command not found or error

If you see command not found or an error, you need to install WSL 2.

Open Windows Terminal as Administrator (right-click Start → Windows Terminal (Admin)).
Run the following command to install WSL 2:

wsl --install

What does wsl --install do?

This command installs WSL 2 and the default Ubuntu Linux distribution. WSL 2 is a lightweight Linux environment that runs inside Windows, which Docker Desktop uses to run Linux containers.

Wait for the installation to complete (this may take 5-10 minutes).
Restart your computer when prompted or when the installation completes.
After restarting, verify WSL 2 is installed by running wsl --version in PowerShell.

Still having WSL 2 issues?

If you continue to have problems installing WSL 2, you may need to:

Enable virtualization in your BIOS settings
Make sure Windows is up to date (Settings → Windows Update)
Try troubleshooting your error here, or share your error message with the NextWork community!

Install Docker Desktop

Let's install Docker Desktop!

What's Docker Desktop? Is that the same thing as Docker?

Open your Downloads folder.
Find the Docker Desktop Installer.exe.
Double-click it to start the installation.

You'll see a security warning: Open File - Security Warning.

Click Run.

You might be prompted: Do you want this app to make changes to your device?

Click Yes.

On the configuration screen, you'll see a checkbox for Add shortcut to desktop (optional).

Click OK.

Pro Tip

The desktop shortcut makes it easy to open Docker Desktop later, but you can also find it in your Start menu.

Docker will now begin Unpacking files.

What's Docker unpacking?

Docker Desktop is installing its core components: the Docker Engine (runs containers), Docker CLI (command-line tools), and desktop interface. It's also setting up the virtualization layer that lets containers run on Windows using WSL 2.

When unpacking is complete, you'll see Installation succeeded with a message that Docker Desktop is installed.

Click Close.

If you see a Restart required popup, restart your computer when you're ready.

Click Restart to complete the installation.

Why restart?

Docker Desktop needs to install system-level components and drivers. The restart ensures these components are properly loaded before you start using Docker.

After restarting, you'll need to open Docker Desktop.

Open Docker Desktop from your Start menu.

You'll see a Welcome to Docker screen asking you to sign in.

Click Skip in the top right corner.

You should now see the Docker Desktop dashboard!

Why open Docker Desktop?

Opening Docker Desktop starts the Docker service in the background. This service needs to be running for any Docker commands to work. You'll see a whale icon in your system tray when Docker is running.

Now let's verify Docker is working!

Open PowerShell on your Windows computer
Run this command:

docker --version

What should I see?

You should see something like Docker version 24.0.7, build afdd53b. The version and build ID will likely be different on your computer - this is normal and doesn't affect functionality. Why are version numbers different?

Perfect! Docker is installed and working on your Windows machine.

Wait! ✋ Let's double-check Docker Desktop is running

Quick check before we move on - Docker Desktop needs to be running for the next steps to work:

Look for the whale icon 🐳 in your menu bar (Mac) or system tray (Windows)
Don't see it? Open Docker Desktop from your Applications folder (Mac) or Start menu (Windows)

Why check this?

Docker commands won't work unless Docker Desktop is running in the background. The whale icon is your visual confirmation that everything is ready to go!

✅ I see a version number

Perfect! That should mean Docker Desktop is already installed. Now let's make sure it's running.

Open Docker Desktop from your Applications folder (Mac) or Start menu (Windows).

Hmm, I can't find Docker Desktop

Aha! That means we will still need to download Docker Desktop after all. It's possible for your terminal to show a docker version, but not have Docker Desktop if it was installed, and later deleted, in the past.

Make sure to switch over to the ❌ Command not found tab and check out the steps for installing Docker Desktop.

Wait for Docker to fully start (the whale icon should be steady, not animated).

You should see the Docker Desktop dashboard:

Verify Docker is working

Let's test that Docker is working correctly. This is an important step - it confirms everything is set up properly!

Run this command in your terminal:

docker run hello-world

Wait 10-20 seconds for your terminal to run the command.

What do you see?

✅ I see 'Hello from Docker!'

Perfect! You should see output like:

Hello from Docker!
This message shows that your installation appears to be working correctly.

What just happened?

Docker downloaded a small test image called "hello-world" and ran it in a container. This proves that:

Docker can download images from the internet
Docker can create and run containers
Your installation is working correctly

This is the same process we'll use to run your RAG API in a container!

❌ I see an error

No worries! Let's troubleshoot common Docker errors:

Cannot connect to Docker daemon

If you see Cannot connect to Docker daemon or Is the docker daemon running?:

This means Docker Desktop isn't running or hasn't fully started yet.

Check if Docker Desktop is running (look for the whale icon).
If the icon is animated, wait for it to become steady.
If Docker Desktop isn't running, start it from your Applications folder (Mac) or Start menu (Windows).
Wait 30-60 seconds for Docker to fully initialize.
Try the command again.

Permission denied

If you see permission denied errors:

🍎 macOS

Make sure your user account is in the docker group (usually handled automatically).
Try running the command with sudo (not recommended for regular use): bash sudo docker run hello-world - If that works, Docker Desktop might need to be restarted to apply permissions. > 🙋‍♀️ Still having permission issues on macOS? > Try troubleshooting your error here, or share your error message with the NextWork community!

🖼️ Windows

Make sure you're running your terminal as Administrator.
Right-click on PowerShell or Command Prompt and select Run as Administrator.
Try the command again.

Still having permission issues on Windows?

Try troubleshooting your error here, or share your error message with the NextWork community!

Network or connection errors

If you see network errors when downloading the hello-world image:

Check your internet connection.
Some corporate networks block Docker Hub. Try again later or use a different network.
Docker might be downloading the image in the background - wait a minute and check Docker Desktop's activity.

Still having network issues?

Try troubleshooting your error here, or share your error message with the NextWork community!

Other error

If you see a different error:

Check Docker Desktop's status: Open Docker Desktop and look for any error messages.
Try restarting Docker Desktop.
Check Docker Desktop logs: Troubleshoot > View logs.
Visit Docker's troubleshooting guide for more help.

Still having trouble verifying Docker?

Try troubleshooting your error here, or share your error message with the NextWork community!

Great work! You've successfully installed and verified Docker.

Now you're ready to package your RAG API in a Docker container! 🐳

Containerize Your API

Let's package your RAG API in a Docker container so it can run identically anywhere - your laptop, a teammate's computer, or a production server.

In this step, you're going to:

Create a Dockerfile for your RAG API
Build a Docker image of your RAG API
Test your containerized API

Create Dockerfile

Let's create your Dockerfile step by step.

What is a Dockerfile?

A Dockerfile is a text file with instructions for building a Docker image. Every Dockerfile follows a template where each line is a command or instruction that Docker executes in sequence:

FROM specifies the base image (like Python 3.11)
WORKDIR sets the working directory inside the container
COPY brings your code files into the image
RUN executes commands (like installing packages)
EXPOSE documents which port the app uses
CMD defines what command runs when the container starts

When you build a Docker image from this Dockerfile, Docker creates a layered, read-only filesystem snapshot that includes your application and everything it needs to run. This Docker image can then be distributed and run anywhere Docker is installed!

💡 Extra for Experts: How do Docker layers work?

Each command in a Dockerfile creates a new "layer" - a filesystem snapshot. Docker caches these layers, so if you rebuild your image and only change one file, Docker can reuse the cached layers from previous builds. This makes rebuilds much faster!

The layers are stacked on top of each other, with the base image at the bottom and your application code at the top.

Right-click in your project folder in Cursor's file explorer.
Select New File.

Name it Dockerfile (no extension)

Note

Check Make sure:

You've created Dockerfile in your project's root directory (where you store app.py, embed.py, and k8s.txt).
Your Dockerfile is named exactly Dockerfile with no extension (not Dockerfile.txt or dockerfile).

Open Dockerfile in your IDE.
Paste this content:

FROM python:3.11-slim
WORKDIR /app
RUN apt-get update && apt-get install -y curl && rm -rf /var/lib/apt/lists/*
COPY app.py embed.py k8s.txt ./
RUN pip install fastapi uvicorn chromadb ollama
RUN python embed.py
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

What is a Dockerfile?

A Dockerfile is a set of instructions that tells Docker how to build your container image. Think of it as a recipe - each line adds a layer to your image.

The key instructions in your Dockerfile are:

FROM python:3.11-slim - Tells Docker to start with a pre-built image that has Python 3.11 installed. This is the foundation everything else builds on.
WORKDIR /app - Sets /app as the container's working directory (like cd /app).
COPY app.py embed.py k8s.txt ./ - Copies your application files from your computer into the container. These files become part of the image.
RUN pip install... and RUN python embed.py - Execute commands during build time. RUN installs packages and pre-computes embeddings, so they're baked into the image.
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"] - Defines what command runs when the container starts. This starts your FastAPI server.

Save the file press Cmd+S on Mac or Ctrl+S on Windows.

The white circle next to your filename should disappear once the file is saved!

Build Docker Image

Now let's build your Docker image.

What does building a Docker image mean?

A Docker image is a portable, read-only package of your app code, dependencies, and settings so it can run the same way everywhere.

To build a Docker image of you RAG API, Docker reads your Dockerfile top to bottom, runs the listed instructions, and snapshots the results into a reusable image.

In your Cursor terminal, run:

docker build -t rag-app .

What does this command do?

docker build - Tells Docker to build an image from a Dockerfile
-t rag-app - Tags (names) the image as "rag-app". The -t stands for "tag". You can think of this as giving your image a name so you can reference it later.
. - Uses the current directory (where the Dockerfile is) as the build context. The . means "current directory" - Docker will look for the Dockerfile here and use files from this directory.

This will take 3-5 minutes the first time. You'll see many steps being executed:

While we wait...

What do all these steps mean?

Each line in the build output corresponds to a command in your Dockerfile:

[1/6] FROM ... - Pulls the base Python image from Docker Hub (this might take a minute if it's your first time)
[2/6] WORKDIR ... - Sets the working directory inside the container
[3/6] RUN ... - Installs system dependencies (like curl)
[4/6] COPY ... - Brings your code files into the image
[5/6] RUN pip install ... - Installs Python packages (this is usually the longest step)
[6/6] RUN python embed.py - Pre-computes embeddings for your knowledge base

Each step creates a new "layer" in your image. If you rebuild later and only change one file, Docker will use cached layers for unchanged steps, making rebuilds much faster!

Once your terminal is done running the command, what do you see?

✅ Build completed successfully

Perfect - your Docker image is ready!

Let's check that your image was created.

Run the command for listing all your Docker images:

🍎 macOS

docker images | grep rag-app

🖼️ Windows

docker images | Select-String rag-app

What does this do?

Select-String filters the output to show only lines containing "rag-app".

You should see rag-app with the tag latest.

Great! Your image is built and ready to run.

❌ I see a build error

No worries! Let's troubleshoot common Docker build errors:

File not found errors

If you see errors like COPY failed: file not found or No such file or directory:

This means Docker can't find the files you're trying to copy.

Make sure you're in your project's root directory (the one with app.py, embed.py, and k8s.txt).

Check the files exist:

ls -la

You should see app.py, embed.py, and k8s.txt in the list.

Check that your Dockerfile uses the correct file names (case-sensitive on Linux/Mac).
Make sure you're running docker build from the directory containing the Dockerfile.

pip install errors

If you see errors during pip install:

Check your internet connection - pip needs to download packages.
Some packages might have installation issues. Try building again - sometimes it's a temporary network issue.
If chromadb fails to install, make sure you have enough disk space (it needs several GB).

embed.py execution error

If you see errors when running python embed.py:

Make sure k8s.txt exists and has content.
Check that embed.py is correct (it should match the code from Project 1).
The error message will tell you what's wrong - read it carefully!

No space left on device

If you see no space left on device:

Docker images can take up a lot of space. Check your disk space:

df -h

Clean up unused Docker images:

docker system prune -a

Warning

This removes all unused images, not just rag-app. Only do this if you're sure you don't need other images.

Free up more disk space on your computer, then try building again.

Other build errors

If you see a different error:

Read the error message carefully - it usually tells you what went wrong.
Check Docker Desktop is running (the whale icon should be steady).
Try building again - sometimes it's a temporary issue.
Check Docker Desktop logs: Troubleshoot > View logs.

Still having trouble building your Docker image?

Try troubleshooting your error here, or share your error message with the NextWork community!

Run Your Container

Now let's run your API in a Docker container! This is exciting - you're about to see your API running in an isolated environment.

Why are we running our container?

The Docker image is just a blueprint - it's not running yet. Running the container would actually start your API and makes it accessible at http://localhost:8000.

💡 What's the difference between an image and a container?

An image is a read-only template with your application and all its dependencies. It's like a snapshot or blueprint.
A container is a running instance of an image. You can run multiple containers from the same image, and each one is isolated from the others.

Run your container:

docker run -p 8000:8000 rag-app

What does this command do?

docker run - Creates and starts a new container from an image
-p 8000:8000 - Maps port 8000 on your host (your computer) to port 8000 in the container. This is called port mapping - it makes the container's port 8000 accessible from your computer.
rag-app - The name of the image to run (the one we built earlier)

What do you see?

✅ Container is running

Awesome! Your containerized API is running. Keep this terminal window open - the container must stay running.

❌ I see an error

No worries! Let's troubleshoot common container run errors:

Port already in use

If you see port is already allocated or address already in use:

This means something else is already using port 8000 on your computer.

Why does this happen?

A port is just a numbered slot where your computer delivers network requests. Only one running app can “listen” on a specific port at a time. If another app is already listening on port 8000, Docker can’t map your container to that port until it’s free.

Check what's using port 8000:

🍎 macOS

lsof -i :8000

Note the PID (process ID) and stop it:

kill -9 <PID>

🖼️ Windows

netstat -ano | findstr :8000

Note the PID and stop it:

taskkill /PID <PID> /F

Or use a different port:

docker run -p 8001:8000 rag-app

Then access your API at http://localhost:8001 instead.

Image not found

If you see Unable to find image 'rag-app':

Make sure you built the image first (run docker build -t rag-app .).
Check the image exists:

🍎 macOS

docker images | grep rag-app

🖼️ Windows

docker images | Select-String rag-app

If it's not there, try building your Docker image again. Click here to jump back to the steps for building a Docker image!

Cannot connect to Ollama

If your API starts but can't connect to Ollama:

Make sure Ollama is running on your host (check with curl http://localhost:11434).
The container needs to access Ollama on your host machine. We'll configure this properly in the Kubernetes step, but for now, make sure Ollama is running.

Still having trouble connecting to Ollama from the container?

Try troubleshooting your error here, or share your error message with the NextWork community!

Other errors

If you see a different error:

Check Docker Desktop is running.
Try stopping and starting Docker Desktop, then run the container again.
Try deleting your existing Docker image, rebuilding it, then run the container agian.
Try troubleshooting your error here, or share your error message with the NextWork community! if you're still stuck!

Test Your Containerized API

Let's test that your API works inside the container! This is the moment of truth - we'll see if everything we packaged works correctly.

Select the New Terminal button at the top of your Cursor terminal window.

Run this command to test your API:

🍎 macOS / 🐧 Linux

curl -X POST "http://127.0.0.1:8000/query" -G --data-urlencode "q=What is Kubernetes?"

What does this command do?

This command sends a test question to your API.

curl is a tool for making web requests
-X POST means we're sending data (not just reading)
-G --data-urlencode "q=What is Kubernetes?" sends your question as a parameter.

Your API should search its database, find information about Kubernetes, and return an AI-generated answer!

🖼️ Windows

In PowerShell, run this command:

Invoke-WebRequest -Uri "http://127.0.0.1:8000/query?q=What%20is%20Kubernetes%3F" -Method POST

What does this command do?

💡 Why can't I use curl in PowerShell?

PowerShell aliases curl to Invoke-WebRequest, but they have different syntax. The -X flag doesn't work with PowerShell's Invoke-WebRequest, so we use -Method POST instead. The URL also needs to be properly encoded (spaces become %20, question marks become %3F).

What do you see?

✅ I got a JSON response

What does this prove?

This proves that:

Your API code works inside a Docker container
All dependencies are correctly packaged (FastAPI, Chroma, etc.)
The container can access Ollama running on your host (for AI generation)
Your API behaves the same way in a container as it did locally

This is huge! You've successfully containerized your RAG API. It now runs in an isolated environment that works identically on any machine with Docker.

Awesome! Your containerized API is working perfectly!

❌ I got an error or no response

No worries! Let's troubleshoot:

Connection refused

If you see Connection refused:

Make sure the container is still running (check the first terminal window).
If the container stopped, check the error message in that terminal.
Try running the container again:

docker run -p 8000:8000 rag-app

Internal Server Error

If you see errors that say Internal Server Error:

Switch terminals back to the one running your API. The terminal should be called docker
Check the error message in your terminal.

You'll most likely see a ConnectionError because your terminal has Failed to connect to Ollama!

I have a different error!

Interesting! Try troubleshooting your error here, or share your error message with the NextWork community!

Let's make sure Ollama is running on your computer.

Start a new terminal.
Run the command that checks for a connection to Ollama:

curl http://localhost:11434

❌ Ollama isn't running

If Ollama isn't running, start it.

⚠️ Ollama is running (but they query still returns Internal Server Error)

If Ollama is running but you still see ConnectionError: Failed to connect to Ollama, your container can't reach Ollama on your computer. Let's fix this!

Why did this error happen?

Your RAG API's source code i.e. app.py uses the default way to set up a connection to Ollama, which tries to connect to http://localhost:11434. This works when Ollama is running in your local computer, but inside a container, localhost means "inside the container itself".

This means your app is now trying to connect with Ollama in your container, not your actual computer where Ollama is running! Because of this, Ollama can't be found, and we've triggered a ConnectionError.

💡 Extra for Experts: Why not run Ollama in the container?

Running Ollama in a separate container is actually a common production pattern! It allows you to:

Scale Ollama independently from your API
Update the model without rebuilding your API container
Use specialized hardware (like GPUs) for the model

For this project, we keep it simple by running Ollama on the host, but you could containerize it too!

Let's update your code to point to the right place.

Update your app.py file

Why are we updating app.py?

Right now, your containerized API can't find Ollama because it's looking for Ollama inside the container, but Ollama is actually running on your computer (outside the container). We need to tell your API where to find Ollama - on your host machine!

Stop your container by pressing Ctrl + C in the docker terminal where it's running.

Why stop the container?

We need to update the code, rebuild the image, and restart the container. Stopping it now lets us make changes without conflicts.

Open app.py in Cursor.
Find where you create the collection. Right after this line:

collection = chroma.get_or_create_collection("docs")

Add this line to create an Ollama client that knows where to find Ollama:

ollama_client = ollama.Client(host="http://host.docker.internal:11434")

Why create a Client instance?

Think of a client instance as a connection between your container and local computer - it knows exactly where to reach Ollama and how to talk to it. The ollama library needs you to create a Client instance so it knows where Ollama is running. Without this, it assumes Ollama is inside the container (which it's not - Ollama is on your computer!).

💡 What is host.docker.internal?

host.docker.internal is a special address from inside a container that points to your actual computer (the "host"). This lets your containerized API reach Ollama running on your computer!

🙋‍♀️ I'm using Linux - does host.docker.internal work?

On Linux, host.docker.internal doesn't work by default. Instead, find your computer's IP address:

Run: ip addr show | grep "inet " | grep -v 127.0.0.1
In app.py, replace host.docker.internal:11434 with your IP address (e.g., http://172.17.0.1:11434)

Now find the ollama.generate() call. Change it from:

answer = ollama.generate(...)

To use your client instance instead:

answer = ollama_client.generate(
    model="tinyllama",
    prompt=f"Context:\n{context}\n\nQuestion: {q}\n\nAnswer clearly and concisely:"
)

Why change from ollama.generate() to ollama_client.generate()?

Now that we've created a client with the correct host address, we need to use that client to make API calls. The client knows where to find Ollama on your computer!

Save the file (press Cmd+S on Mac or Ctrl+S on Windows).
✋ Double check - have you saved your changes?

Rebuild your Docker image

Now we need to rebuild your image with the updated code:

In your terminal, run this command:

docker build -t rag-app .

Wait for the build to complete. It should be faster this time since Docker caches most layers from the first time you built the image!

When the build is done, let's run your container again.

Start your container:

docker run -p 8000:8000 rag-app

In a new terminal, run the command to test your API again:
- MacOS: curl -X POST "http://127.0.0.1:8000/query" -G --data-urlencode "q=What is Kubernetes?"
- Windows: Invoke-WebRequest -Uri "http://127.0.0.1:8000/query?q=What%20is%20Kubernetes%3F" -Method POST

You should now get a successful JSON response! 🎉

Request timeout

If the request times out or takes very long:

The first request might be slow as Chroma loads the database.
Wait a bit and try again.
Check the container logs in the first terminal window for any errors.

Still experiencing timeouts?

Try troubleshooting your error here, or share your error message with the NextWork community!

Other errors

If you see a different error:

Check the container logs in the first terminal window.
The error message will usually tell you what's wrong.
Make sure all your files (app.py, embed.py, k8s.txt) were copied correctly into the image.

Still having trouble testing your containerized API?

Try troubleshooting your error here, or share your error message with the NextWork community!

What's the difference between running locally and in Docker?

Running locally:

Uses your system Python and packages
Depends on your specific environment
Might break if dependencies change

Running in Docker:

Uses a consistent Python environment inside the container
Includes all dependencies in the image
Works identically on any machine with Docker
Isolated from your system

Stop the Container

When you're done testing:

Go back to the docker terminal where the container is running.
Press Ctrl + C to stop the container.

Secret mission

Psst... want to share your containerized API with the world? 🤔

Your secret mission, should you choose to accept it, is to push your Docker image to Docker Hub and learn how to distribute containers publicly. This is how professional teams share containerized applications - whether for collaboration, CI/CD pipelines, or deployment to cloud platforms!

What is Docker Hub?

Docker Hub is a cloud-based registry where you can store and share Docker images. Think of it like GitHub, but for containers! You can:

Push your images to make them publicly available
Pull images created by others
Use images in CI/CD pipelines
Deploy images to cloud platforms like AWS, Google Cloud, or Azure

Share your Docker Image

Delete your resources

Now that we've built and tested our containerized API, you might want to clean up some resources. The good news is that everything we created is free to keep - Docker images on your computer just take up disk space, and Docker Hub images are free for public repositories!

Do I need to delete anything?

No! All the resources we created are free. Docker images on your computer just use disk space, and your Docker Hub image is free to keep publicly available. You only need to delete things if you want to free up disk space or remove your image from Docker Hub.

✅ Keep everything (recommended)

Great choice! There's no cost to keeping your Docker images or your Docker Hub repository. You can always delete them later if you need the disk space.

Your Docker Hub image

Your image on Docker Hub will remain there and is free to keep. It's publicly available for others to use, and it doesn't cost anything. If you want to remove it later, visit https://hub.docker.com/r/your-username/rag-app and delete the repository.

🗑️ Delete local Docker images

If you want to free up disk space on your computer, you can remove the Docker images:

List your images:

docker images

Remove the rag-app images:

docker rmi rag-app [[YOUR_USERNAME="your-username"]]/rag-app

Remove unused images (optional):

docker image prune -a

Tip

This removes all unused images, not just rag-app. Only do this if you're sure you don't need other images!

🗑️ Delete Docker Hub image

If you want to remove your image from Docker Hub:

Visit https://hub.docker.com/r/your-username/rag-app in your browser.
Sign in to Docker Hub.
Select Settings (or the gear icon).
Scroll down and select Delete Repository.
Type the repository name to confirm and select Delete.

Note

This only removes the image from Docker Hub. Your local images will still be on your computer unless you delete them separately.

🛑 Stop Docker Desktop

If you're done using Docker for now, you can stop Docker Desktop:

macOS: Right-click the Docker icon in your menu bar and select Quit Docker Desktop.
Windows: Right-click the Docker icon in your system tray and select Quit Docker Desktop.

Tip

You can always start Docker Desktop again later when you need it. Stopping it doesn't delete any images or containers.

Well done!

Nice work! 🐳 You've just containerized your RAG API with Docker and shared it on Docker Hub.

You've learned how to:

Package your API in a Docker container that runs identically anywhere
Build Docker images from Dockerfiles
Push images to Docker Hub for distribution
Pull and run images from Docker Hub

Ready to quiz yourself? 💪

What's Next in the DevOps × AI Series

You've built the foundation - a RAG API that's containerized and ready to share! Next up...

☸️ Project 3: Deploy to Kubernetes How do you orchestrate containers at scale? Project 3 teaches you to deploy your containerized API to Kubernetes, learning how production systems manage containers automatically.
🔄 Project 4: Automate Deployments with GitHub Actions When your knowledge base changes, how do you rebuild and redeploy automatically? Project 4 adds CI/CD pipelines to automate the entire build → test → deploy workflow.

p.s. Does it say "Still tasks to complete!" at the bottom of the screen?

This means you still have screenshots left to upload, or questions left to answer!

Press Ctrl+F (Windows) or Command+F (Mac) on your keyboard.
Search for the text Return to later.
Jump straight to your incomplete tasks!
🙋‍♀️ Still stuck? Ask the community!