Local LLM Dev Environment

Deploy a private AI chat platform locally using Docker, Ollama, and Open WebUI.

Introduction

30 Second Summary

Every time you use an AI chatbot online, your prompts travel to someone else's servers. What if you could run a powerful AI assistant entirely on your own machine, with no data leaving your computer?

In this project, you will deploy a self-hosted AI chat platform on your local machine using Docker containers. You'll have a private ChatGPT-like experience with no API keys, no cloud costs, and no data leaving your machine.

What You'll Build

A containerized AI chat platform running Ollama (LLM inference engine) and Open WebUI (chat interface) orchestrated with Docker Compose.

By the end of this project, you'll have:

  • A working WSL 2 Ubuntu environment configured for Docker development.
  • A Docker Compose configuration orchestrating two containerized services.
  • A running Llama 3.2 model accessible through both a REST API and a browser-based chat interface.
  • Secret Mission: Pull a second model and configure multi-model serving in the same chat interface.

Are there any prerequisites?

You need a Windows 10 or 11 machine (64-bit) with at least 8GB of RAM (16GB recommended) and 20GB of free disk space. No prior Linux or Docker experience is required.

Install WSL 2 and Ubuntu

Docker containers run on Linux. To use Docker on Windows, you need a real Linux environment running alongside your Windows desktop.

WSL 2 (Windows Subsystem for Linux) gives you exactly that. It runs a full Linux kernel inside a lightweight virtual machine, letting you use Linux tools natively on Windows. In this step, you will install WSL 2 with Ubuntu and prepare your Linux environment for the rest of the project.

In this step, get ready to:
  • Install WSL 2 with the Ubuntu distribution on your Windows machine.
  • Create your Linux user account with a username and password.
  • Verify WSL 2 is running correctly with the right version.
Install WSL 2 from PowerShell

The wsl --install command enables the necessary Windows features, downloads the latest Linux kernel, sets WSL 2 as the default, and installs Ubuntu in a single step.

  • Press the Windows key to open your search bar.
  • Type PowerShell in the search bar.
  • Right-click Windows PowerShell and select Run as administrator.
  • Install WSL 2 with Ubuntu by running this command:
wsl --install

What does this command do?

The wsl --install command performs four actions automatically:

  • Enables the Windows Subsystem for Linux optional component.
  • Enables the Virtual Machine Platform component.
  • Downloads and installs the latest Linux kernel.
  • Downloads and installs the Ubuntu distribution.
  • When the command finishes, restart your computer. The install process requires a reboot to finalize.

Seeing help text instead of an installation?

If wsl --install prints help/usage text instead of starting an installation, WSL may already be partially installed on your system. Try running this instead:

wsl --install -d Ubuntu

If the install hangs at 0%, try downloading directly from the web:

wsl --install --web-download -d Ubuntu

Also confirm that virtualization is enabled in your BIOS/UEFI settings. Help me troubleshoot my WSL installation

Set up your Ubuntu user account

After your computer restarts, Ubuntu launches automatically to finish its setup. If it does not open on its own, you can find it in your Start menu.

  • Press the Windows key to open your search bar.
  • Type Ubuntu and press Enter to open it.
  • Ubuntu will prompt you to create a UNIX username. Type a short, lowercase username (like your first name) and press Enter.
  • Type a password when prompted and press Enter. Type it again to confirm.

Why can't I see my password as I type?

Nothing appears on screen while you type your password. This is called blind typing and is a standard Linux security feature. Your keystrokes are being registered even though you cannot see them.

  • Update your Ubuntu system packages by running these commands:
sudo apt update && sudo apt upgrade -y

What does this command do?

sudo apt update refreshes the list of available packages from Ubuntu's repositories. sudo apt upgrade -y installs any newer versions of packages already on your system.

The -y flag automatically confirms all prompts so the upgrade runs without pausing for input.

  • Wait for the update to complete. You will see package download progress in the terminal. When it finishes, you will be returned to the command prompt.

Getting a permission denied error?

Make sure you typed sudo at the start of the command. Without sudo, you do not have permission to install system packages. Enter your password when prompted.

Help me fix permission errors in my Ubuntu terminal

Verify your WSL 2 installation

With Ubuntu running and updated, you need to confirm that WSL is using version 2 (not version 1) and that it meets the minimum version required for Docker Desktop.

  • Open a new PowerShell window (you do not need Administrator mode this time).
  • Check your WSL version by running this command:
wsl --version

This prints the WSL version number and its components. Docker Desktop requires WSL version 2.1.5 or higher.

✔️ I see version 2.1.5 or higher

Your WSL installation is ready for Docker Desktop. Move on to the next check below.

ⓧ I see an older version

Your WSL is installed but needs an update. Run this command in PowerShell to get the latest version:

wsl --update

After the update completes, run wsl --version again to confirm the version is now 2.1.5 or higher.

ⓧ Command not found

The wsl command is not recognized. This usually means the installation did not complete or the restart did not happen. Try these steps:

  • Restart your computer if you have not done so since running wsl --install.
  • After restarting, open PowerShell as Administrator and run wsl --install again.
  • If the problem persists, open Settings > Windows Update and install all available updates, then try again.

Help me get the wsl command working on my machine

  • Now switch back to your Ubuntu terminal and confirm the distribution by running:
lsb_release -a
  • You should see output that includes the distributor ID as Ubuntu and a version number. This confirms your Linux environment is ready.

lsb_release command not found?

This can happen if the base system packages were not fully installed. Run the following command, then try lsb_release -a again:

sudo apt install lsb-release -y

Help me verify my Ubuntu installation in WSL

Your Linux environment is ready. Next up, you will install Docker Desktop and connect it to your WSL 2 Ubuntu distribution.

Install Docker Desktop and Configure WSL Integration

Your Ubuntu environment is ready. Now you need a way to run your AI services in isolated, reproducible packages called containers.

Docker Desktop is the Windows application that manages containers through your WSL 2 backend. Once configured, you can run Docker commands directly from your Ubuntu terminal as if you were on a native Linux machine.

In this step, get ready to:
  • Install Docker Desktop on Windows with the WSL 2 backend.
  • Enable WSL integration with your Ubuntu distribution.
  • Verify Docker runs correctly inside Ubuntu.
Download and install Docker Desktop

Docker Desktop bundles the Docker engine, CLI tools, and a graphical dashboard into one installer. It uses your WSL 2 backend to run Linux containers natively on Windows.

  • Open this link in your browser to reach the Docker Desktop download page: https://docs.docker.com/desktop/setup/install/windows-install/
  • Click the download button for Windows to download Docker Desktop Installer.exe.
  • Run the installer once the download completes.
  • When the installer shows configuration options, make sure Use WSL 2 instead of Hyper-V is selected.
  • Complete the installation and restart your computer if prompted.

Why WSL 2 instead of Hyper-V?

WSL 2 uses a lightweight virtual machine that shares resources with Windows more efficiently than a full Hyper-V VM. It also gives you direct filesystem access between Windows and Linux, which is critical for the development workflow you will use throughout this project.

Docker Desktop is free for learning

Docker Desktop is free for personal use, education, and non-commercial open source. You do not need a paid subscription for this project.

Installer won't run?

Make sure hardware virtualization is enabled in your BIOS settings. This is required for WSL 2 and Docker to function. Search your computer manufacturer's name plus "enable virtualization in BIOS" for specific instructions.

help me enable virtualization for Docker Desktop

Enable WSL integration for Ubuntu

Docker Desktop needs to know which WSL distributions should have access to Docker. By default, it may not connect to your Ubuntu installation until you toggle it on.

  • Open Docker Desktop from your Windows taskbar or Start menu.
  • Wait for Docker Desktop to finish starting up. You will see a green status indicator when it is ready.
  • Click the gear icon to open Settings.
  • In the left sidebar, click Resources.
  • Click WSL Integration.
  • Ensure the toggle next to your Ubuntu distribution is switched on.
  • Click Apply & restart.

What does WSL integration do?

Toggling on your Ubuntu distribution tells Docker Desktop to inject the Docker CLI and Docker Compose binaries into that distribution. Without this, running docker inside Ubuntu would return "command not found" even though Docker Desktop is installed on Windows.

Don't see Ubuntu in the list?

Close Docker Desktop completely, open PowerShell, and run wsl --list to confirm Ubuntu appears. If it does, restart Docker Desktop. The WSL Integration page refreshes its list on startup.

help me get Ubuntu to show up in Docker Desktop WSL integration settings

Verify Docker works in Ubuntu

With WSL integration enabled, Docker commands should now work directly in your Ubuntu terminal. You will run three quick checks to confirm everything is connected.

  • Return to your Ubuntu terminal.
  • Check that the Docker CLI is accessible by running:
docker --version
  • Check that Docker Compose is available by running:
docker compose version

Both commands should print version numbers. The exact numbers will depend on your Docker Desktop release.

  • Confirm the full Docker pipeline works by running:
docker run hello-world

Docker will download a small test image and run it. You should see a message starting with "Hello from Docker!" followed by a confirmation that your installation is working correctly.

✔️ I see Hello from Docker!

Docker is fully working inside your Ubuntu terminal. The Docker daemon on Windows, the CLI in WSL, and the container runtime are all connected.

ⓧ Command not found

This means WSL integration did not take effect yet. Try these fixes in order:

  • Close your Ubuntu terminal completely and reopen it. WSL integration changes require a fresh shell session.
  • Confirm Docker Desktop is running. Look for the whale icon in your Windows system tray.
  • Go back to Docker Desktop Settings > Resources > WSL Integration and verify the Ubuntu toggle is on. Click Apply & restart again.
  • If it still fails, restart Docker Desktop entirely by right-clicking the whale icon and selecting Restart.

help me fix Docker command not found in my WSL Ubuntu terminal

ⓧ Permission denied

If you see a "permission denied" error when running docker commands, try closing and reopening your Ubuntu terminal. Docker Desktop's WSL integration should handle permissions automatically without needing to add your user to a Docker group.

If the error persists after reopening the terminal, restart Docker Desktop from the system tray.

help me fix Docker permission denied in WSL Ubuntu

Docker is running inside your Linux environment. Next up, you will create your project structure and prepare it for the AI platform files.

Set Up Project Structure and Git

Your Docker environment is confirmed working inside WSL 2. Now it's time to create a proper home for your project files and set up version control.

Every professional infrastructure project lives in a Git repository. In this step, you'll install Git, create your project directory, and connect everything to VS Code for editing.

In this step, get ready to:
  • Install and configure Git in your Ubuntu environment.
  • Create the project directory and initialize a Git repository.
  • Open the project in VS Code with WSL integration.
Install and configure Git

Git tracks every change you make to your project files. It lets you roll back mistakes and keep a clean history of your infrastructure code.

  • Install Git inside your Ubuntu terminal by running this command:
sudo apt install git -y

What does this command do?

The sudo prefix runs the command with administrator privileges. apt install git downloads Git from Ubuntu's package manager. The -y flag auto-confirms the installation so you don't have to type "yes" manually.

  • When prompted, enter the UNIX password you created during Ubuntu setup. Characters won't appear as you type.
  • Confirm the installation succeeded by running:
git --version

You should see a version number printed, such as git version 2.43.0.

Permission denied or command not found?

If you see "permission denied", make sure you included sudo at the start of the command. If you see "Unable to locate package", run sudo apt update first to refresh the package list, then retry the install.

Help me troubleshoot my Git installation in WSL

  • Configure Git with your name and email by running these commands:
git config --global user.name "[[YOUR_GIT_NAME="Your Name"]]"
git config --global user.email "[[YOUR_GIT_EMAIL="your@email.com"]]"

Why does Git need my name and email?

Every Git commit records who made the change. These values are attached to your commits as metadata. They don't create an account anywhere.

  • Verify your configuration by running:
git config --global --list

You should see your name and email printed in the output.

  • Create your project directory and move into it by running these commands:
mkdir -p ~/ai-platform/phase-1 && cd ~/ai-platform/phase-1

What does this command do?

The mkdir -p command creates nested directories in one shot. The ~ refers to your Linux home folder inside WSL. This keeps project files on the Linux filesystem for best performance with Docker.

Linux filesystem for Docker performance

Always store project files inside the Linux filesystem (~/) rather than the Windows mount at /mnt/c/. Docker bind mounts perform significantly better on the native Linux filesystem.

  • Confirm you're in the right directory by running:
pwd

You should see /home/<your-username>/ai-platform/phase-1 printed.

Initialize your Git repository
  • Initialize a new Git repository by running:
git init

You should see a message saying Initialized empty Git repository followed by the path.

  • Create your .gitignore file by running this command:
cat > .gitignore << 'EOF'
# Environment files with secrets
.env

# Docker volumes (managed by Docker, not in repo)
data/

# OS files
.DS_Store
Thumbs.db

# IDE
.vscode/
*.swp
EOF

What is a .gitignore file?

A .gitignore file tells Git which files and folders to exclude from version control. This keeps sensitive files (like .env) and generated files (like Docker volume data) out of your repository.

The cat > .gitignore << 'EOF' syntax is a heredoc. It writes everything between the two EOF markers into the file.

  • Verify the file was created correctly by running:
cat .gitignore

You should see the comment lines and file patterns printed in your terminal.

File is empty or missing?

If cat .gitignore shows nothing or says "No such file", the heredoc command may not have completed properly. Make sure you pasted the entire block including the final EOF on its own line with no spaces before it.

Help me fix my .gitignore file creation in the terminal

✔️ Awesome, I've got everything!

Great. Double check you see the .gitignore file in your directory by running ls -a (the -a flag shows hidden files that start with a dot).

ⓧ I'd like to double check the full code

Here is the complete .gitignore file. Compare it with your output from cat .gitignore.

# Environment files with secrets
.env

# Docker volumes (managed by Docker, not in repo)
data/

# OS files
.DS_Store
Thumbs.db

# IDE
.vscode/
*.swp
Open the project in VS Code

VS Code is a code editor that connects directly to your WSL environment. You'll use it to edit all project files from here on.

  • Check whether you already have VS Code installed on Windows.

✔️ I already have VS Code installed

You're all set. Make sure the WSL extension is installed in VS Code. You can check by opening VS Code, pressing Ctrl+Shift+X to open the Extensions panel, and searching for "WSL". If it's not installed, click Install on the extension by Microsoft.

ⓧ I need to install VS Code

  • Open your browser on Windows and go to code.visualstudio.com/Download.
  • Click the Windows download button and run the installer.
  • During installation, check the Add to PATH option when prompted. This allows the code command to work from your terminal.
  • Complete the installation and open VS Code.
  • Press Ctrl+Shift+X to open the Extensions panel.
  • Search for "WSL" and install the extension by Microsoft.

Why install the WSL extension?

The WSL extension lets VS Code connect directly into your Ubuntu environment. Without it, the code . command from your WSL terminal won't be able to open your project.

  • Return to your Ubuntu terminal (make sure you're still in the ~/ai-platform/phase-1 directory).
  • Open the project in VS Code by running:
code .

What happens when you run this?

The first time you run code . from WSL, VS Code downloads and installs the VS Code Server component inside your Linux environment. This takes a few seconds and only happens once.

After a moment, a VS Code window opens on Windows with your project folder loaded. You'll see a green "WSL: Ubuntu" indicator in the bottom-left corner confirming the connection.

VS Code should open with your phase-1 folder visible in the Explorer sidebar. You should see your .gitignore file listed.

VS Code doesn't open or shows an error?

If the code command is not found, close and reopen your Ubuntu terminal. The PATH update from the VS Code installation may require a fresh session.

If VS Code opens but doesn't show the WSL indicator, make sure the WSL extension is installed. Press Ctrl+Shift+X and search for "WSL".

Help me connect VS Code to my WSL Ubuntu environment

Your project structure is ready. Next up, you'll write the Docker Compose configuration that defines your entire AI platform as code.

Create Docker Compose Configuration

Your project directory is set up with Git tracking changes. Now you need to define how your two AI services will run together and communicate.

Docker Compose lets you define multi-container applications in a single YAML file. You will create a configuration that orchestrates Ollama (the LLM inference engine) and Open WebUI (the chat interface) so they start together and talk to each other automatically.

In this step, get ready to:
  • Create a docker-compose.yml file defining Ollama and Open WebUI services.
  • Create a .env file and README.md to externalize configuration and document your project.
  • Validate the compose configuration before launching.
Create the Docker Compose file

Docker Compose uses a YAML file to define which containers run, how they connect, and where they store data. Your compose file will define two services that work together as a complete AI chat platform.

What is Docker Compose?

Docker Compose is a tool for defining and running multi-container applications. Instead of starting each container manually with long docker run commands, you describe your entire application stack in one file.

One command (docker compose up) starts everything together with networking already configured between services.

  • In VS Code, create a new file called docker-compose.yml in the phase-1 folder by clicking the new file icon in the Explorer sidebar.
  • Add the following code to define the Ollama service:
services:
  ollama:
    image: ollama/ollama:${OLLAMA_DOCKER_TAG:-latest}
    container_name: ollama
    volumes:
      - ollama:/root/.ollama
    ports:
      - "${OLLAMA_PORT:-11434}:11434"
    restart: unless-stopped

What does this code do?

  • services: is the top-level key where you define each container.
  • image: ollama/ollama tells Docker which image to pull from Docker Hub. The ${OLLAMA_DOCKER_TAG:-latest} syntax reads the tag from your .env file, defaulting to latest if not set.
  • volumes: mounts a named volume at /root/.ollama so downloaded models persist even if the container is recreated.
  • ports: maps the container's internal port 11434 to the same port on your machine.
  • restart: unless-stopped means the container restarts automatically unless you explicitly stop it.
  • Below the ollama service (after restart: unless-stopped), add the Open WebUI service and the volumes section:

  open-webui:
    image: ghcr.io/open-webui/open-webui:${WEBUI_DOCKER_TAG:-main}
    container_name: open-webui
    volumes:
      - open-webui:/app/backend/data
    depends_on:
      - ollama
    ports:
      - "${OPEN_WEBUI_PORT:-3000}:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - WEBUI_SECRET_KEY=${WEBUI_SECRET_KEY:-}
    extra_hosts:
      - host.docker.internal:host-gateway
    restart: unless-stopped

volumes:
  ollama: {}
  open-webui: {}

What does this code do?

  • depends_on: ensures Ollama starts before Open WebUI.
  • ports: "${OPEN_WEBUI_PORT:-3000}:8080" maps the container's internal port 8080 to port 3000 on your machine. You will access the chat UI at http://localhost:3000.
  • OLLAMA_BASE_URL=http://ollama:11434 tells Open WebUI where to find the Ollama service. Docker Compose creates a shared network so containers can reach each other by service name.
  • extra_hosts: adds a DNS entry for host.docker.internal so the container can reach your host machine if needed.
  • The volumes: section at the bottom declares the named volumes. Docker manages these automatically and keeps your data safe across container restarts.
  • Save docker-compose.yml.

Seeing red squiggly lines in VS Code?

YAML is sensitive to indentation. Make sure you are using spaces (not tabs) and that each level is indented exactly 2 spaces from its parent. Check that open-webui: is at the same indentation level as ollama: (2 spaces under services:).

help me fix YAML indentation errors in my docker-compose.yml

✔️ Awesome, I've got everything!

Great. Double check you have saved your docker-compose.yml file.

ⓧ I'd like to double check the full code

services:
  ollama:
    image: ollama/ollama:${OLLAMA_DOCKER_TAG:-latest}
    container_name: ollama
    volumes:
      - ollama:/root/.ollama
    ports:
      - "${OLLAMA_PORT:-11434}:11434"
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:${WEBUI_DOCKER_TAG:-main}
    container_name: open-webui
    volumes:
      - open-webui:/app/backend/data
    depends_on:
      - ollama
    ports:
      - "${OPEN_WEBUI_PORT:-3000}:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
      - WEBUI_SECRET_KEY=${WEBUI_SECRET_KEY:-}
    extra_hosts:
      - host.docker.internal:host-gateway
    restart: unless-stopped

volumes:
  ollama: {}
  open-webui: {}

Why separate containers instead of a bundled image?

Open WebUI offers a bundled image that includes Ollama inside it. Using separate containers teaches you Docker Compose networking, service dependencies, and container orchestration.

These are transferable skills you will use in every multi-service deployment. Keeping services separate also lets you update, scale, or replace each one independently.

Create the environment and documentation files

The docker-compose.yml references variables like ${OLLAMA_PORT} that need values. A .env file provides those values in one place so you can change ports or image tags without editing the compose file directly.

  • Create a new file called .env in the phase-1 folder and add the following configuration:
# Docker image tags
OLLAMA_DOCKER_TAG=latest
WEBUI_DOCKER_TAG=main

# Port configuration
OLLAMA_PORT=11434
OPEN_WEBUI_PORT=3000

# Security - generate with: openssl rand -hex 32
WEBUI_SECRET_KEY=

What does this file do?

  • OLLAMA_DOCKER_TAG and WEBUI_DOCKER_TAG control which image versions Docker pulls. Changing these lets you pin to a specific release without touching the compose file.
  • OLLAMA_PORT and OPEN_WEBUI_PORT let you remap ports if another service on your machine already uses 11434 or 3000.
  • WEBUI_SECRET_KEY is left empty for now. You can generate one later with openssl rand -hex 32 to prevent sessions from resetting when the container restarts.
  • Save .env.
  • Confirm the file is hidden from Git by checking that your .gitignore (created in the previous step) already includes .env on its first line.

Why keep .env out of Git?

The .env file will eventually hold secrets like WEBUI_SECRET_KEY. Excluding it from version control prevents accidental exposure if you push to a remote repository.

  • Create a new file called README.md in the phase-1 folder and add the following documentation:
# AI Platform - Phase 1: Local LLM Development Environment

A self-hosted AI chat platform running Ollama + Open WebUI via Docker Compose.

## Quick Start

```bash
# Start services
docker compose up -d

# Pull a model
docker exec -it ollama ollama pull llama3.2

# Open the chat UI
# Visit http://localhost:3000
```

## Services

| Service | Port | Description |
|---------|------|-------------|
| Ollama | 11434 | LLM inference engine |
| Open WebUI | 3000 | Chat interface |

## Commands

```bash
# Start
docker compose up -d

# Stop
docker compose down

# View logs
docker compose logs -f

# Check status
docker compose ps

# Pull a new model
docker exec -it ollama ollama pull <model-name>

# List models
docker exec -it ollama ollama list
```

## Models

- `llama3.2` - General purpose (3B, ~2GB)
- `qwen2.5-coder:7b` - Code generation (7B, ~4.7GB)

What does this file do?

A README.md is the first thing anyone sees when they open your project folder or visit your repository. It documents how to start the platform, which services run, and what commands are available.

This is professional practice. Every production project includes a README so teammates (or future-you) can get up and running quickly.

  • Save README.md.
  • In the VS Code Explorer sidebar, confirm you now see three files: docker-compose.yml, .env, and README.md alongside your existing .gitignore.

Don't see .env in the sidebar?

Files starting with a dot are hidden by default on some systems. In VS Code, the Explorer sidebar shows all files regardless. If you still cannot see it, make sure you saved the file in the phase-1 folder and not a parent directory.

help me find my .env file in VS Code on WSL

Validate the compose configuration

Before launching containers, you can ask Docker Compose to parse your YAML and report any errors. This catches typos and indentation mistakes before they become confusing runtime failures.

  • Switch back to your Ubuntu terminal and validate the compose file by running this command:
docker compose config

✔️ I see a YAML output of my services

Docker Compose parsed your file successfully. The output is the fully resolved configuration with all variable substitutions applied. You are ready to launch.

ⓧ I see an error message

The error message will point to the line or key causing the problem. Common causes:

  • Incorrect indentation. YAML uses spaces only (not tabs). Each nested level must be exactly 2 spaces.
  • Missing colon after a key name (e.g. services instead of services:).
  • Quotes around port mappings must use straight double quotes, not curly quotes.

Fix the issue in VS Code, save the file, and run docker compose config again.

help me fix my docker-compose.yml syntax error

Your Docker Compose configuration is locked in. Next up, you will launch both containers and pull your first AI model.

Launch the Platform and Pull Your First Model

Your Docker Compose configuration is validated and ready to go.

Now it is time to bring the platform to life. You will start both containers, download your first open-source LLM, and verify the entire stack works by talking to the model directly through its API.

In this step, get ready to:
  • Start the Ollama and Open WebUI containers with Docker Compose.
  • Pull the Llama 3.2 model into the Ollama container.
  • Test the model by sending it a message via the REST API.
Start the services

Docker Compose reads your docker-compose.yml file and creates both containers in the correct order. The open-webui service waits for ollama to start first because of the depends_on rule you defined.

  • Launch both services in the background by running this command:
docker compose up -d

What does this command do?

The -d flag means "detached mode." Your containers run in the background so you keep your terminal free for other commands.

  • Watch the startup logs to confirm both containers initialize successfully:
docker compose logs -f

You should see Ollama report that it is listening on 0.0.0.0:11434. Open WebUI will log something like Uvicorn running on http://0.0.0.0:8080 when it is ready.

  • Press Ctrl+C to exit the log stream once both services report they are listening.
  • Verify both containers are running by checking their status:
docker compose ps

You should see both ollama and open-webui listed with a status of Up.

Containers not starting?

If you see "port already in use," another process is using port 11434 or 3000. Close any other Ollama or web server instances, then run docker compose up -d again.

If a container shows Exited status, check its logs with docker compose logs ollama or docker compose logs open-webui to see the error.

Need more help? Why won't my Docker Compose containers start?

Pull your first LLM

The Ollama container is running, but it does not have any models yet. You need to download a model before you can chat with it.

Llama 3.2 is a 3B parameter model from Meta. At roughly 2GB, it is small enough to run on CPU while still producing meaningful responses.

  • Pull the Llama 3.2 model into your Ollama container by running this command:
docker exec -it ollama ollama pull llama3.2

What does this command do?

The docker exec -it ollama part runs a command inside the running ollama container. The ollama pull llama3.2 part tells Ollama to download the Llama 3.2 model from its model registry.

You will see a progress bar as the model downloads. This takes a few minutes depending on your internet speed.

  • Once the download completes, verify the model is available by running:
docker exec -it ollama ollama list

You should see llama3.2 listed with its size information.

Download stuck or failing?

If the pull seems frozen, check your internet connection. The download is roughly 2GB so it may take several minutes on slower connections.

If you see a disk space error, free up at least 5GB on the drive where Docker stores volumes. You can check Docker's disk usage with docker system df.

Still stuck? Help me troubleshoot my Ollama model download

Test the model via the Ollama API

Ollama exposes a REST API on port 11434. You can send it messages directly from your terminal without needing the web UI. This is how other applications (including Open WebUI) communicate with the model behind the scenes.

  • Send a test message to the model by running this command:
curl http://localhost:11434/api/chat -d '{"model": "llama3.2", "messages": [{"role": "user", "content": "Hello, what can you do?"}], "stream": false}'

What does this command do?

This sends a POST request to Ollama's chat API. The JSON body specifies which model to use (llama3.2), a messages array with your prompt, and stream: false so it returns the complete response in one JSON object instead of streaming it word by word.

After a few seconds, you should see a JSON response in your terminal containing the model's reply. The response includes a message field with the model's generated text.

Getting "connection refused"?

Make sure the Ollama container is still running. Run docker compose ps to check. If it shows Exited, restart with docker compose up -d.

If you see a model-not-found error, the pull may not have completed. Run docker exec -it ollama ollama list to confirm llama3.2 appears.

Need help? Why is my curl request to the Ollama API failing?

Your local LLM platform is live. Next up, you will open the Open WebUI chat interface in your browser and have a real conversation with your model.

Configure Open WebUI and Chat With Your Model

Your local LLM is running and responding to API calls. That proves the inference engine works. But chatting through curl commands is not practical for everyday use.

Open WebUI gives you a browser-based chat interface that connects directly to your Ollama server. In this step, you will create an admin account, have your first conversation with the model, and commit your project to Git.

In this step, get ready to:
  • Create your admin account in Open WebUI.
  • Chat with your local LLM through the browser interface.
  • Commit your project to Git and verify data persists across container restarts.
Create your admin account

Open WebUI requires an account to use the chat interface. The very first account you create automatically receives Administrator privileges.

  • Open your browser and navigate to http://localhost:3000.
  • Click Sign Up on the login page.
  • Enter your name in the Name field.
  • Enter an email address in the Email field.
  • Enter a password in the Password field.
  • Click Create Account.

Why does the first account get admin privileges?

Open WebUI treats the first registered user as the instance owner. This account controls user management, system settings, and model configuration.

Anyone who signs up after this point gets a regular user role.

After signing in, you should see the Open WebUI chat interface with a message input field at the bottom of the screen.

Seeing a blank page or connection error?

Make sure the Open WebUI container is still running. Switch back to your Ubuntu terminal and run docker compose ps to check. If the container shows as exited, run docker compose up -d to restart it.

Help me troubleshoot my Open WebUI connection

Chat with your local model

With your admin account ready, you can now have your first conversation with the Llama 3.2 model through the browser.

  • Click the model selector dropdown at the top of the chat interface.
  • Select llama3.2 from the list of available models.
  • Type a message in the input field at the bottom. Try something like Hello, what can you do?.
  • Press Enter or click the send button.

You should see the model generate a response in the chat window. The text streams in token by token since the model is running on your CPU.

What if the response is slow?

Llama 3.2 runs on your CPU in this phase, so responses may take 10-30 seconds depending on your hardware. This is expected. In later phases, GPU acceleration will dramatically speed up inference.

  • Click your profile icon in the bottom-left corner of the sidebar, then click Settings to explore the available configuration options.

Settings worth exploring

The General section lets you change the theme and language. The Models section shows which models are available from your Ollama server. You do not need to change anything right now.

Model not appearing in the dropdown?

If llama3.2 does not show up in the model selector, the Open WebUI container may not be able to reach Ollama. Switch to your Ubuntu terminal and run docker compose logs open-webui to check for connection errors.

Help me fix my model not showing in Open WebUI

Commit your work and verify persistence

Your project files are ready and your platform is working. The last thing to do is save a snapshot of your work in Git and confirm that your data survives container restarts.

  • Commit all project files to Git by running this command in your Ubuntu terminal:
git add -A && git commit -m "Phase 1: Local LLM platform with Ollama and Open WebUI"

What does this command do?

git add -A stages all files in the project directory (except those listed in .gitignore). The git commit -m part saves a snapshot with a descriptive message.

Your .env file is excluded because .gitignore blocks it from being tracked.

You should see output listing the files that were committed, including docker-compose.yml, .gitignore, and README.md.

Now test that your chat history and model data persist when containers are recreated. Docker named volumes store data independently from the containers themselves.

  • Stop all services by running:
docker compose down
  • Restart the services by running:
docker compose up -d
  • Open http://localhost:3000 in your browser again.
  • Sign in with the account you created earlier.
  • Confirm that your previous chat conversation is still visible in the sidebar.

Your chat history and admin account survived the restart. This is because the named volumes (ollama and open-webui) persist data on disk even when containers are destroyed and recreated.

Why do named volumes matter?

Without named volumes, all data inside a container disappears when the container is removed. Named volumes live outside the container lifecycle.

This means your downloaded models (stored in the ollama volume) and your chat history (stored in the open-webui volume) are safe across restarts, upgrades, and rebuilds.

Chat history missing after restart?

If your previous conversations are gone, you may have accidentally used docker compose down -v which deletes volumes along with containers. The -v flag removes named volumes. Without that flag, docker compose down only removes containers and networks.

Help me recover my missing chat history

Secret mission

Multi-Model Setup

Your platform currently serves a single general-purpose model. Real AI infrastructure teams deploy multiple specialized models and route tasks to the right one. In this mission, you'll pull a code-focused model and experience multi-model serving firsthand.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on your local machine, so there are no ongoing cloud costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Docker containers (ollama and open-webui).
  • Docker volumes storing LLM models and WebUI data (~7GB+).
  • Docker images (ollama/ollama:latest and ghcr.io/open-webui/open-webui:main).
  • Project folder (~/ai-platform/phase-1).
  • Docker Desktop (installed on Windows).
  • WSL 2 Ubuntu distribution.

Keep everything running

No action needed. Choose this if you plan to continue using your local LLM platform or move on to the next phase.

  • Your containers are set to restart: unless-stopped, so they restart automatically when Docker Desktop launches.
  • Docker Desktop starts with Windows by default. Your LLM chat interface will be available at http://localhost:3000 whenever your computer is on.
  • Your downloaded models (llama3.2 and qwen2.5-coder:7b) persist in the ollama Docker volume.

Pause - I'll come back to this later

Shut down running containers to free up CPU and RAM while keeping all your data intact. You can restart everything later with a single command.

  • Open your Ubuntu terminal and navigate to your project folder by running:
cd ~/ai-platform/phase-1
  • Stop the containers by running:
docker compose down
  • This stops and removes the containers but preserves your volumes (models and chat history).
  • To restart later, run docker compose up -d from the same folder.
  • Optionally, close Docker Desktop to free additional memory. Your data stays safe in the Docker volumes.

Delete - I don't want to use this again

Remove all project resources and start fresh. This permanently deletes your downloaded models, chat history, and project files.

  • Open your Ubuntu terminal and navigate to your project folder by running:
cd ~/ai-platform/phase-1
  • Stop containers and remove Docker volumes by running:
docker compose down -v
  • Remove the Docker images by running:
docker rmi ollama/ollama:latest ghcr.io/open-webui/open-webui:main
  • Delete the project folder by running:
cd ~
rm -rf ~/ai-platform
  • (Optional) Uninstall Docker Desktop from Windows:
  • Open Windows Settings.
  • Select Apps, then Apps & features.
  • Find Docker Desktop in the list and click Uninstall.
  • (Optional) Remove the WSL Ubuntu distribution by running this in PowerShell as Administrator:
wsl --unregister Ubuntu

This is permanent

Unregistering Ubuntu deletes all data, settings, and software inside that distribution. Only do this if you will not need WSL Ubuntu for other projects.

Nice Work!

Nice Work!

You just built a fully private, self-hosted AI chat platform running entirely on your local machine. No API keys, no cloud costs, no data leaving your computer.

You've learned how to:

  • Deploy a Docker Compose configuration orchestrating Ollama and Open WebUI as containerized services inside WSL 2 on Windows.
  • Pull and serve an open-source LLM (Llama 3.2) locally and test it via the REST API with curl.
  • Manage the full container lifecycle with Docker Compose commands to start, stop, monitor logs, and confirm data persists across restarts.
  • Configured multi-model serving by pulling a code-focused model (qwen2.5-coder:7b) and switching between models in the same chat interface.

Ready to quiz yourself?