Containerize Your RAG API with Docker

Package a FastAPI RAG app into Docker containers and publish to GHCR.

Introduction

30 Second Summary

You build an app on your machine and it works perfectly. Share it with someone else and suddenly nothing runs because they have the wrong Python version, missing packages, or a different operating system.

In this project, you will package your RAG API into Docker containers, orchestrate multiple services with Docker Compose, and publish your container image to GitHub Container Registry so anyone can pull and run it.

What You'll Build

A fully containerized version of your RAG API where FastAPI runs in one container and Ollama runs in another, communicating over a Docker network with persistent data stored in named volumes.

By the end of this project, you'll have:

  • A Dockerfile that packages your RAG API into a portable container image with layer caching for efficient builds.
  • A Docker Compose setup that orchestrates your API and Ollama as separate services with persistent ChromaDB data.
  • A published container image on GitHub Container Registry that anyone can pull and run without installing Python or dependencies.
  • Secret Mission: Production-harden your image with a multi-stage build, non-root user, and health check.

Are there any prerequisites?

You should have completed the "Build a RAG API with FastAPI" NextWork project or have an equivalent FastAPI + ChromaDB + Ollama RAG API running locally. You also need a GitHub account.

Install Docker Desktop and Set Up Project Files

Your RAG API works on your machine, but deploying it elsewhere means installing Python, ChromaDB, Ollama, and matching exact versions. Docker solves this by packaging your application and all its dependencies into a portable container that runs identically anywhere.

In this step, you'll install Docker Desktop, confirm it works by running a test container, and create the project files you'll containerize in the next steps.

In this step, get ready to:
  • Install Docker Desktop and verify it's running.
  • Run a test container to confirm Docker works end-to-end.
  • Create your project directory with all application files.
Download and install Docker Desktop

Docker Desktop is the application that runs the Docker engine on your Mac. It also bundles Docker Compose, which you'll use later to orchestrate multiple containers.

What is Docker Desktop?

Docker Desktop is a GUI application that runs a lightweight Linux virtual machine on your Mac. Inside that VM, Docker builds and runs containers. You interact with it through terminal commands like docker and docker compose.

  • Open your browser and navigate to https://www.docker.com/products/docker-desktop/.
  • Click the download button for your Mac (choose Mac with Apple chip or Mac with Intel chip).
  • Double-click the downloaded Docker.dmg file to open the installer.
  • Drag the Docker icon into the Applications folder.
  • Press Cmd+Space to open Spotlight search.
  • Type Docker and press Enter to open Docker Desktop.
  • Accept the Docker Subscription Service Agreement when prompted.
  • Wait for Docker Desktop to finish starting. You'll see the Docker whale icon appear in your menu bar.

Why does Docker need a service agreement?

Docker Desktop is free for personal use, education, and small businesses. The service agreement confirms which license tier applies to you.

Docker Desktop may present an installation window asking about default or advanced settings. Use the default settings.

  • Open your terminal by pressing Cmd+Space, typing Terminal, and pressing Enter.
  • Verify Docker is installed by running this command:
docker --version
  • Also verify Docker Compose is available by running:
docker compose version

✔️ I see version numbers for both commands

You should see output similar to Docker version 28.x.x, build ... and Docker Compose version v2.x.x. The exact numbers depend on when you installed. Docker is ready.

ⓧ Command not found

This means the Docker CLI binaries are not in your PATH yet. Try these fixes:

  • Make sure Docker Desktop is fully running (check for the whale icon in your menu bar).
  • Close and reopen your terminal. Docker Desktop creates symlinks on first launch, but your terminal session needs to reload PATH.
  • If it still doesn't work, open Docker Desktop, go to Settings > Advanced, and confirm the CLI tools path is set to /usr/local/bin.

What about the macOS version requirement?

Docker Desktop requires macOS Sonoma (version 14) or later. If you're on an older macOS version, you'll need to update your operating system before Docker Desktop will install.

Run a test container to confirm Docker works

The best way to confirm Docker is fully operational is to run a real container. Docker provides a small welcome image for exactly this purpose.

  • Run the Docker welcome container by entering this command:
docker run -d -p 8080:80 docker/welcome-to-docker

What does this command do?

  • docker run creates and starts a new container.
  • -d runs it in detached mode (in the background).
  • -p 8080:80 maps port 8080 on your machine to port 80 inside the container.
  • docker/welcome-to-docker is the container image Docker provides for testing.

You should see a Docker welcome page confirming that your container is running successfully.

  • Stop and remove the test container by running:
docker rm -f $(docker ps -q)

What does this cleanup command do?

docker ps -q lists the IDs of all running containers. docker rm -f force-removes them. This is a quick way to clean up without remembering the container name.

Don't see the welcome page?

  • Make sure Docker Desktop is fully started (whale icon is steady, not animating).
  • Check that nothing else is using port 8080. Run lsof -i :8080 to check.
  • If you see a "permission denied" error, make sure Docker Desktop finished its initial setup.

Help me troubleshoot why Docker isn't running my test container on localhost:8080.

Create the project directory and application files

Now you'll create a fresh project directory for your containerized RAG API. The code is based on the API you built in the previous project, but modified to accept an OLLAMA_HOST environment variable so it can connect to Ollama running in a separate container.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir rag-api
cd rag-api
  • Confirm the folder was created by running:
ls

You should see the empty rag-api folder listed on your Desktop (if you ran ls before cd), and you're now inside it.

  • Open your rag-api folder in your preferred code editor. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type the name of your code editor (e.g. VS Code) and press Enter to open it.
  • Open the rag-api folder from your Desktop in the editor.

You'll create four files in this directory. Each file plays a specific role in the containerized application.

  • Create a new file called main.py in your rag-api folder and add the following code:
import os
from fastapi import FastAPI
import ollama
import chromadb
from chromadb.utils.embedding_functions.ollama_embedding_function import (
    OllamaEmbeddingFunction,
)

app = FastAPI()

# Read Ollama host from environment variable, default to localhost
OLLAMA_HOST = os.getenv("OLLAMA_HOST", "http://localhost:11434")

# Set up ChromaDB for persistent vector storage
client = chromadb.PersistentClient(path="./chroma_db")

# Configure the embedding function to use Ollama
ef = OllamaEmbeddingFunction(
    model_name="nomic-embed-text",
    url=OLLAMA_HOST,
)

# Get or create the collection for profile data
collection = client.get_or_create_collection(
    name="personal_profile",
    embedding_function=ef,
)


@app.get("/ask")
def ask(question: str):
    # Query the vector store for relevant context
    results = collection.query(
        query_texts=[question],
        n_results=2,
    )
    context = "\n\n".join(results["documents"][0])

    # Build the augmented prompt with retrieved context
    augmented_prompt = f"""Use the following context to answer the question.
If the context doesn't contain relevant information, say so.

Context:
{context}

Question: {question}"""

    # Send the augmented prompt to the LLM via Ollama
    ollama_client = ollama.Client(host=OLLAMA_HOST)
    response = ollama_client.chat(
        model="qwen2.5:0.5b",
        messages=[{"role": "user", "content": augmented_prompt}],
    )

    return {
        "question": question,
        "answer": response["message"]["content"],
        "context_used": results["documents"][0],
    }

What does this code do?

  • OLLAMA_HOST reads from an environment variable so the app can connect to Ollama running in a separate container (not just localhost).
  • The /ask endpoint queries ChromaDB for relevant context, builds an augmented prompt, and sends it to the Ollama LLM.
  • ollama.Client(host=OLLAMA_HOST) creates an explicit client pointing at the configurable host, instead of the module-level default.
  • Save main.py.
  • Create a new file called build_knowledge_base.py and add the following code:
import os
import chromadb
from chromadb.utils.embedding_functions.ollama_embedding_function import (
    OllamaEmbeddingFunction,
)

# Read Ollama host from environment variable
OLLAMA_HOST = os.getenv("OLLAMA_HOST", "http://localhost:11434")

# Read the profile text file
with open("profile.txt", "r") as f:
    text = f.read()

# Split into chunks by double newlines
chunks = [chunk.strip() for chunk in text.split("\n\n") if chunk.strip()]

print(f"Loaded {len(chunks)} chunks from profile.txt")

# Set up ChromaDB with persistent storage
client = chromadb.PersistentClient(path="./chroma_db")

# Configure embedding function pointing at Ollama
ef = OllamaEmbeddingFunction(
    model_name="nomic-embed-text",
    url=OLLAMA_HOST,
)

# Create the collection for profile data
collection = client.get_or_create_collection(
    name="personal_profile",
    embedding_function=ef,
)

# Add all chunks to the vector store
collection.add(
    ids=[f"chunk{i}" for i in range(len(chunks))],
    documents=chunks,
    metadatas=[{"source": "profile", "chunk_index": i} for i in range(len(chunks))],
)

print(f"Added {len(chunks)} chunks to the 'personal_profile' collection.")
print("Knowledge base built successfully!")

What does this script do?

This script reads profile.txt, splits it into chunks, and stores them as vector embeddings in ChromaDB. It runs once to populate the knowledge base that the API queries.

Like main.py, it uses the OLLAMA_HOST environment variable so it can reach Ollama whether running locally or in a container.

  • Save build_knowledge_base.py.
  • Create a new file called profile.txt and add the following content:
My name is Alex.
I'm currently learning about cloud computing, AI, and DevOps.

My career goal is to become a cloud engineer.
I'm especially interested in containerization and AI deployment.

I have experience with Python, FastAPI, and building RAG pipelines.
I'm currently building projects on NextWork to grow my hands-on skills.

For fun, I enjoy hiking and reading about new technologies.
A fun fact about me is that I containerized my first AI app today.

Personalize your profile

Feel free to replace the content in profile.txt with your own information. This is the data your RAG API will use to answer questions about you.

  • Save profile.txt.
  • Create a new file called requirements.txt and add the following pinned dependencies:
fastapi==0.137.0
uvicorn==0.34.0
chromadb==1.5.9
ollama==0.4.8

Why pin exact versions?

Pinning versions (e.g. fastapi==0.137.0) ensures your container builds are reproducible. Without pins, a new release of any package could break your application inside the container.

  • Save requirements.txt.
  • Confirm all four files exist by running this in your terminal:
ls

You should see main.py, build_knowledge_base.py, profile.txt, and requirements.txt listed in the output.

Missing a file?

  • Make sure you saved each file in the rag-api folder on your Desktop, not in a subfolder or your home directory.
  • Check that your terminal is in the right directory. Run pwd to confirm you're in ~/Desktop/rag-api.

Help me find where my project files went. I created them but ls doesn't show them.

✔️ Awesome, I've got everything!

Great. Double check you've saved all four files in your rag-api folder.

ⓧ I'd like to double check the full code

Here's what each file should contain:

main.py:

import os
from fastapi import FastAPI
import ollama
import chromadb
from chromadb.utils.embedding_functions.ollama_embedding_function import (
    OllamaEmbeddingFunction,
)

app = FastAPI()

OLLAMA_HOST = os.getenv("OLLAMA_HOST", "http://localhost:11434")

client = chromadb.PersistentClient(path="./chroma_db")

ef = OllamaEmbeddingFunction(
    model_name="nomic-embed-text",
    url=OLLAMA_HOST,
)

collection = client.get_or_create_collection(
    name="personal_profile",
    embedding_function=ef,
)


@app.get("/ask")
def ask(question: str):
    results = collection.query(
        query_texts=[question],
        n_results=2,
    )
    context = "\n\n".join(results["documents"][0])

    augmented_prompt = f"""Use the following context to answer the question.
If the context doesn't contain relevant information, say so.

Context:
{context}

Question: {question}"""

    ollama_client = ollama.Client(host=OLLAMA_HOST)
    response = ollama_client.chat(
        model="qwen2.5:0.5b",
        messages=[{"role": "user", "content": augmented_prompt}],
    )

    return {
        "question": question,
        "answer": response["message"]["content"],
        "context_used": results["documents"][0],
    }

build_knowledge_base.py:

import os
import chromadb
from chromadb.utils.embedding_functions.ollama_embedding_function import (
    OllamaEmbeddingFunction,
)

OLLAMA_HOST = os.getenv("OLLAMA_HOST", "http://localhost:11434")

with open("profile.txt", "r") as f:
    text = f.read()

chunks = [chunk.strip() for chunk in text.split("\n\n") if chunk.strip()]

print(f"Loaded {len(chunks)} chunks from profile.txt")

client = chromadb.PersistentClient(path="./chroma_db")

ef = OllamaEmbeddingFunction(
    model_name="nomic-embed-text",
    url=OLLAMA_HOST,
)

collection = client.get_or_create_collection(
    name="personal_profile",
    embedding_function=ef,
)

collection.add(
    ids=[f"chunk{i}" for i in range(len(chunks))],
    documents=chunks,
    metadatas=[{"source": "profile", "chunk_index": i} for i in range(len(chunks))],
)

print(f"Added {len(chunks)} chunks to the 'personal_profile' collection.")
print("Knowledge base built successfully!")

profile.txt:

My name is Alex.
I'm currently learning about cloud computing, AI, and DevOps.

My career goal is to become a cloud engineer.
I'm especially interested in containerization and AI deployment.

I have experience with Python, FastAPI, and building RAG pipelines.
I'm currently building projects on NextWork to grow my hands-on skills.

For fun, I enjoy hiking and reading about new technologies.
A fun fact about me is that I containerized my first AI app today.

requirements.txt:

fastapi==0.137.0
uvicorn==0.34.0
chromadb==1.5.9
ollama==0.4.8

Your project is set up and Docker is running. Next up, you'll write a Dockerfile that packages this application into a container image.

Write the Dockerfile

In the previous step, you installed Docker Desktop and set up your application files. Now it's time to write the instructions that tell Docker how to package your app into a portable image.

A Dockerfile is a recipe that Docker reads line by line to build an image. Each line adds a layer, and the order matters for build speed. You'll also create a .dockerignore file to keep unnecessary files out of your image.

In this step, get ready to:
  • Create a .dockerignore to exclude unnecessary files from your image.
  • Write a Dockerfile that packages your RAG API.
  • Build and verify your first Docker image.
Create a .dockerignore file

When Docker builds an image, it sends everything in your project folder to the Docker engine. A .dockerignore file tells Docker which files to skip, keeping your image small and secure.

  • Create a new file called .dockerignore in your project folder, then paste in the following:
venv/
__pycache__/
*.pyc
.git/
.env
chroma_db/
.DS_Store

What does this file do?

  • venv/ excludes your local Python virtual environment. The container installs its own dependencies from requirements.txt.
  • __pycache__/ and *.pyc exclude compiled Python files that are machine-specific.
  • chroma_db/ excludes local database files. You'll recreate the knowledge base inside the container later.
  • .env prevents secrets from being baked into the image.
  • Save .dockerignore.
  • Confirm the file exists by running:
ls -a .dockerignore

You should see .dockerignore printed in the output.

Don't see the file?

Make sure you're in the correct project directory. Run pwd to check your current location.

Also check that the filename starts with a dot. Files starting with . are hidden by default in most file browsers.

Why can't I see my .dockerignore file?

Write the Dockerfile

The Dockerfile is where you define exactly how your application gets packaged. Each instruction creates a layer in the final image, and Docker caches unchanged layers to speed up future builds.

  • Create a new file called Dockerfile (no file extension) in your project folder, then paste in the following:
FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8000

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

What does each instruction do?

  • FROM python:3.12-slim sets your starting point. This is a minimal Python 3.12 image without extra OS packages, keeping things lightweight.
  • WORKDIR /app creates and moves into a /app directory inside the container. All following commands run from here.
  • COPY requirements.txt . copies only the dependency list first. This is a layer caching trick. Docker reuses this layer on future builds unless requirements.txt changes.
  • RUN pip install --no-cache-dir -r requirements.txt installs your Python packages without saving pip's download cache, keeping the image smaller.
  • COPY . . copies the rest of your application code into the image.
  • CMD sets the default command that runs when the container starts. It launches your FastAPI app with uvicorn on port 8000.

Why copy requirements.txt separately?

Docker builds images in layers, and it caches layers that haven't changed. If you copy all your code first and then install dependencies, Docker has to reinstall every package whenever any file changes.

By copying requirements.txt first, Docker only re-runs pip install when your dependencies actually change. This saves minutes on every rebuild.

  • Save Dockerfile.

✔️ Awesome, I've got everything!

Great. Double check you've saved your file with no file extension. The filename should be exactly Dockerfile with a capital D.

ⓧ I'd like to double check the full code

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8000

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

Seeing a file extension like Dockerfile.txt?

Some text editors automatically add an extension. Make sure the file is named exactly Dockerfile with no .txt or other extension at the end.

How do I rename my Dockerfile to remove the extension?

Build and verify the image

With both files in place, you can now ask Docker to read your Dockerfile and build a container image from it.

  • Build your Docker image by running this command:
docker build -t rag-api .

What does this command do?

  • docker build tells Docker to read the Dockerfile and create an image.
  • -t rag-api tags (names) the image rag-api so you can reference it easily later.
  • . tells Docker to use the current directory as the build context (the files it can access).

The build takes a minute or two as Docker downloads the base image and installs your Python packages. You'll see each step printed in your terminal as Docker processes each layer.

  • Once the build finishes, verify your image exists by running:
docker images rag-api

You should see a table with columns for REPOSITORY, TAG, IMAGE ID, CREATED, and SIZE. The rag-api image should appear with the tag latest.

Build failed with an error?

If you see a syntax error, check that your Dockerfile has no typos and uses the exact instructions from above. Every instruction keyword (FROM, COPY, RUN, etc.) must be uppercase.

If the error mentions a missing file, make sure you're running the command from inside your project folder where Dockerfile, main.py, and requirements.txt all live.

Help me debug my Docker build error

Your RAG API is now packaged as a portable Docker image. Next up, you'll orchestrate it alongsideOllama using Docker Compose so both services can communicate over a shared network.

Orchestrate with Docker Compose

Your Docker image is built and ready to go. But your RAG API depends on Ollama running as a separate service. Starting each container manually and wiring them together would be tedious and error-prone.

Docker Compose solves this by letting you define multiple services in a single YAML file. One command starts everything, creates a shared network, and connects the services together.

In this step, get ready to:
  • Create a docker-compose.yml file defining two services (API and Ollama).
  • Start both services and pull the required AI models.
  • Test the full stack by querying the /ask endpoint.
Create docker-compose.yml with two services

Your RAG API needs two containers: one for the FastAPI application and one for Ollama (the LLM backend). A Compose file declares both services, their ports, volumes, and how they connect.

  • Create a file called docker-compose.yml in your rag-api folder and paste the following code:
# Define the services that make up the application
services:
  # Ollama service provides the LLM and embedding models
  ollama:
    image: ollama/ollama
    ports:
      - "11434:11434"
    volumes:
      # Named volume keeps downloaded models between restarts
      - ollama_data:/root/.ollama
    restart: unless-stopped

  # API service runs the FastAPI RAG application
  api:
    build: .
    ports:
      - "8000:8000"
    environment:
      # Tell the API to reach Ollama via the Docker network hostname
      - OLLAMA_HOST=http://ollama:11434
    volumes:
      # Named volume keeps ChromaDB data between restarts
      - chroma_data:/app/chroma_db
    depends_on:
      - ollama
    restart: unless-stopped

# Declare named volumes for persistent storage
volumes:
  ollama_data:
  chroma_data:

What does this file do?

  • ollama service: pulls the official Ollama image, exposes port 11434, and stores downloaded models in a named volume so they persist across restarts.
  • api service: builds your Dockerfile, exposes port 8000, and passes OLLAMA_HOST=http://ollama:11434 as an environment variable so your Python code knows how to reach Ollama over the Docker network.
  • depends_on ensures Compose starts the Ollama container before the API container.
  • The volumes: section at the bottom declares two named volumes. Docker manages these on disk so your ChromaDB data and Ollama models survive container restarts.
  • Save docker-compose.yml.
  • Confirm the file exists by running:
ls docker-compose.yml

You should see docker-compose.yml in the output.

File not showing up?

Make sure you saved the file in the same folder as your Dockerfile and main.py. Run ls to check what files are in your current directory.

help me find where my docker-compose.yml should go

✔️ Awesome, I've got everything!

Great. Make sure you have saved docker-compose.yml before moving on.

ⓧ I'd like to double check the full code

Here is the complete docker-compose.yml for comparison:

services:
  ollama:
    image: ollama/ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama_data:/root/.ollama
    restart: unless-stopped

  api:
    build: .
    ports:
      - "8000:8000"
    environment:
      - OLLAMA_HOST=http://ollama:11434
    volumes:
      - chroma_data:/app/chroma_db
    depends_on:
      - ollama
    restart: unless-stopped

volumes:
  ollama_data:
  chroma_data:
Start the services and pull models

With the Compose file in place, a single command builds your API image and starts both containers in the background.

  • Start both services by running:
docker compose up -d

What does this command do?

docker compose up builds images (if needed), creates containers, a shared network, and named volumes. The -d flag runs everything in detached mode so your terminal stays free for the next commands.

  • Wait for both containers to show as running. Confirm by running:
docker ps

You should see two containers listed: one for ollama/ollama and one for your rag-api-api image.

Only one container running?

If the API container exited, it may have failed to connect to Ollama on startup. This is expected because Ollama needs a few seconds to initialize. Run docker compose up -d again to restart the stopped container.

help me debug why my API container keeps exiting

The Ollama container starts empty. You need to download the LLM and embedding models into it before your API can respond to questions.

  • Pull the LLM model into the Ollama container by running:
docker exec rag-api-ollama-1 ollama pull qwen2.5:0.5b
  • Pull the embedding model by running:
docker exec rag-api-ollama-1 ollama pull nomic-embed-text

Why pull models separately?

Ollama downloads models on demand. The docker exec command runs a command inside an already-running container. Here you are telling the Ollama process inside the container to download qwen2.5:0.5b (the LLM) and nomic-embed-text (the embedding model).

These models are stored in the ollama_data volume, so you only need to pull them once. They persist across container restarts.

Each model download may take a minute or two depending on your internet speed. You will see a progress bar in the terminal.

  • Build the knowledge base inside the API container by running:
docker exec rag-api-api-1 python build_knowledge_base.py

You should see output confirming chunks were loaded and added to the collection.

Seeing a connection error?

If build_knowledge_base.py fails with a connection error to Ollama, the Ollama container may still be starting up. Wait 10 seconds and try the command again.

If you see an error about the container name not being found, run docker ps to check the exact container names Docker assigned. They follow the pattern <folder>-<service>-<number>.

help me fix a connection error when building the knowledge base in Docker

Test the full stack

Both services are running and the knowledge base is loaded. Time to verify the entire pipeline works end-to-end.

  • Open your browser and navigate to http://localhost:8000/docs.
  • You should see the Swagger UI page with the /ask endpoint listed.
  • Click the /ask endpoint to expand it, then click Try it out.
  • Enter a question like What is Alex's career goal? in the question field.
  • Click Execute.

You should see a JSON response containing question, answer, and context_used fields. The answer should reference information from your profile.txt file.

  • Check the container logs to see what happened behind the scenes by running:
docker compose logs api

You should see HTTP request logs showing the GET /ask request you just made.

Getting an empty or error response?

If the response says the context doesn't contain relevant information, the knowledge base may not have built correctly. Run docker exec rag-api-api-1 python build_knowledge_base.py again and retry.

If you see a 500 error, check docker compose logs api for Python tracebacks. The most common cause is Ollama still pulling a model.

help me debug why my /ask endpoint is returning an error

Your multi-container RAG stack is running. Next up, you will publish this image to GitHub Container Registry so anyone can pull and run it without setting up Python or dependencies.

Publish to GitHub Container Registry

Your RAG API is running smoothly in its multi-container setup. Now it is time to share your work with the world.

A container registry is like a library for Docker images. By publishing your image to GitHub Container Registry, anyone can pull and run your API without installing Python, ChromaDB, or any dependencies manually.

In this step, get ready to:
  • Create a GitHub Personal Access Token for registry authentication.
  • Authenticate Docker with GitHub Container Registry.
  • Tag and push your image to ghcr.io.
Create a GitHub Personal Access Token

GitHub Packages only supports authentication using a personal access token (classic). You need one with the write:packages scope to push container images to ghcr.io.

  • Open this URL in your browser to go directly to the token creation page with the correct scope pre-selected:
https://github.com/settings/tokens/new?scopes=write:packages

Why this URL instead of navigating manually?

When you select write:packages in the GitHub UI manually, it also auto-selects the repo scope, which grants broad access to all your repositories. Using this URL pre-selects only write:packages, keeping your token's permissions minimal.

  • In the Note field, enter docker-ghcr as a descriptive name for your token.
  • Select an expiration from the Expiration dropdown. Choose 30 days or Custom if you want a different timeframe.
  • Confirm that the write:packages checkbox is already selected in the scopes list.
  • Click Generate token at the bottom of the page.
  • Copy the generated token immediately and save it somewhere safe (you will not be able to see it again after leaving this page).

Do not share or screenshot your token

Your personal access token is a secret credential. Treat it like a password. Do not paste it into any file you might commit to version control, and do not take a screenshot of the token creation page.

Authenticate Docker with ghcr.io

With your token copied, you can now log Docker into GitHub Container Registry. This lets Docker push images to your account.

  • Save your token as an environment variable by running this command (replace your-token-here with the token you just copied):
export CR_PAT=your-token-here
  • Log Docker into ghcr.io by running this command:
echo $CR_PAT | docker login ghcr.io -u [[GITHUB_USERNAME="your-github-username"]] --password-stdin

What does this command do?

  • echo $CR_PAT prints your token to the terminal output.
  • The pipe (|) sends that output directly into the next command.
  • docker login ghcr.io -u USERNAME --password-stdin reads the password from the piped input instead of prompting you to type it. This is more secure than passing the token directly on the command line.
  • You should see Login Succeeded in your terminal output.

Seeing an authentication error?

  • Make sure you replaced your-token-here with the actual token you copied from GitHub. The export command should not contain the literal placeholder text.
  • Double-check that write:packages was selected when you created the token. If not, delete the token and create a new one with the correct scope.
  • Confirm your GitHub username is spelled correctly (it is case-sensitive).

Why can't I log into ghcr.io?

Tag and push the image

Docker images need a registry-specific tag before they can be pushed. The tag tells Docker where to upload the image.

  • Tag your image with a version number by running this command:
docker tag rag-api ghcr.io/[[GITHUB_USERNAME="your-github-username"]]/rag-api:1.0.0
  • Also tag the same image as latest by running:
docker tag rag-api ghcr.io/[[GITHUB_USERNAME="your-github-username"]]/rag-api:latest

Why two tags?

The 1.0.0 tag pins a specific release so you can always pull that exact version later. The latest tag is a convenience pointer that always refers to the most recent version you pushed.

  • Push both tags to GitHub Container Registry by running these commands:
docker push ghcr.io/[[GITHUB_USERNAME="your-github-username"]]/rag-api:1.0.0
docker push ghcr.io/[[GITHUB_USERNAME="your-github-username"]]/rag-api:latest

Each push uploads the image layers to ghcr.io. This may take a minute depending on your internet connection.

  • Verify your published package by visiting your GitHub packages page in your browser:
https://github.com/[[GITHUB_USERNAME="your-github-username"]]?tab=packages
  • You should see rag-api listed as a package with the tags you just pushed.

Don't see your package listed?

  • Make sure both docker push commands completed without errors. Look for a progress bar followed by a digest hash in the output.
  • Confirm your GitHub username in the tag matches your actual username exactly (lowercase).
  • Try refreshing the packages page. It may take a moment for the package to appear.

Why isn't my image showing up on GitHub Packages?

Secret mission

Production-Harden Your Image

Your container image works, but it runs as root, ships pip and build tools, and has no health monitoring. In production, that is a security risk and an operational blind spot. Can you refactor the Dockerfile with a multi-stage build, a non-root user, and a health check?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally and publishes to a free public registry, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Docker Desktop (installed locally, no ongoing cost).
  • Docker images stored locally (can consume disk space).
  • Container image on ghcr.io (free for public packages).
  • GitHub Personal Access Token (can be revoked anytime).
  • Project files in the rag-api folder on your Desktop.

Keep everything running

No action needed. Choose this if you are still actively building or plan to extend your containerized RAG API.

  • Your containers, images, and GHCR package remain available.
  • Docker Desktop continues running in the background (uses some RAM and CPU).
  • Your GitHub Personal Access Token stays active until you revoke it or it expires.

Pause - I'll come back to this later

Shut down running containers to free up memory. Your images and files stay on disk so you can restart anytime.

  • Stop all project containers by running this in your rag-api folder:
docker compose down
  • When you are ready to resume, restart everything with:
docker compose up -d
  • You can also quit Docker Desktop from the whale icon in the menu bar. It restarts when you reopen the app.

Delete - I don't want to use this again

Remove all project containers, images, volumes, and files. This frees up disk space and revokes access tokens.

  • Remove containers, images, and volumes by running this in your rag-api folder:
docker compose down --rmi all --volumes
  • Delete the project folder from your Desktop:
cd ~/Desktop
rm -rf rag-api
  • Revoke your GitHub Personal Access Token by visiting https://github.com/settings/tokens and clicking Delete next to the token you created.
  • Delete the GHCR package by visiting your GitHub packages page at https://github.com/YOUR_GITHUB_USERNAME?tab=packages. Click the rag-api package, then go to Package settings and click Delete this package.
  • If you no longer need Docker on your machine, you can remove Docker Desktop by dragging it from your Applications folder to the Trash.

Nice Work!

Nice Work!

Nice work! You just containerized a full RAG API and published it to a container registry for anyone to pull and run.

Your AI application now runs identically on any machine with Docker installed. No Python version conflicts, no manual dependency installs, no "it works on my machine" headaches.

You've learned how to:

  • Write a Dockerfile with layer caching to package a FastAPI application into a portable container image.
  • Orchestrate a multi-container stack with Docker Compose, using named volumes for persistent ChromaDB and Ollama model data.
  • Publish a container image to GitHub Container Registry so others can pull and run your application with a single command.
  • Secret Mission: Production-hardened your image with a multi-stage build, a non-root user, and a HEALTHCHECK instruction.

Ready to quiz yourself?