Build a Custom AI Chatbot with Ollama

Run a local LLM, create a custom AI personality, and build a Python chatbot.

Introduction

30 Second Summary

AI chatbots like ChatGPT run on massive cloud servers, but you can run your own AI right on your computer. No internet connection, no API keys, no monthly fees.

In this project, you will install Ollama to run a large language model locally, create a custom AI personality, and build a Python chatbot that streams responses in real time.

What You'll Build

A local AI chatbot with a custom personality that runs entirely on your machine, powered by Ollama and Python.

By the end of this project, you'll have:

  • Ollama installed and serving language models locally on your machine.
  • A custom AI personality built from an Ollama Modelfile with its own system prompt and tuned parameters.
  • A Python chatbot script with conversation memory and real-time streaming output.
  • Secret Mission: Add structured JSON output so your chatbot can return parseable data for practical use cases.

Are there any prerequisites?

None. This project includes full setup instructions for every tool you need. You just need a Windows computer with at least 8 GB of free disk space for the language model.

Install Ollama and Python

Before you can chat with an AI model on your own machine, you need the tools that make it possible.

In this step, you will install two things: Ollama (the tool that runs AI models locally) and Python (the programming language you will use to build your chatbot script).

In this step, get ready to:
  • Install Ollama on your Windows machine.
  • Verify Ollama is running locally.
  • Install and verify Python 3.
Install Ollama on Windows

Ollama is a lightweight tool that lets you download and run large language models on your own computer. It runs as a background service and provides a local API that your Python script will talk to later.

  • Press the Windows key to open your search bar.
  • Type PowerShell and press Enter to open it.
  • Check if Ollama is already installed by running this command:
ollama ls

✔️ I see a list (or empty table)

Ollama is already installed on your machine. You can skip ahead to the verification substep below.

ⓧ The term 'ollama' is not recognized

Ollama is not installed yet. Install it by running this command in PowerShell:

irm https://ollama.com/install.ps1 | iex

What does this command do?

This downloads the official Ollama installer script from ollama.com and runs it immediately. The install does not require Administrator privileges and places files in your home directory.

  • Wait for the installation to complete. This may take a minute or two depending on your internet speed.

Install command failing?

  • Make sure you are using PowerShell, not Command Prompt. The irm command is a PowerShell-specific alias.
  • Check that your internet connection is stable. The script downloads Ollama binaries from the web.
  • If you see a security warning about running scripts, type Y and press Enter to allow it.

Help me troubleshoot my Ollama installation on Windows

Verify Ollama is running

After installation, Ollama automatically starts a local server in the background. You can confirm it is running by visiting its API address in your browser.

  • Open your web browser (Edge, Chrome, or any browser you prefer).
  • Type http://localhost:11434 into the address bar and press Enter.

You should see the text Ollama is running displayed on the page.

What is localhost:11434?

When Ollama installs, it starts a small web server on your machine. localhost means "this computer" and 11434 is the port number Ollama listens on.

Your Python chatbot will send requests to this address later to get responses from the AI model.

Not seeing Ollama is running?

  • Check your Windows system tray (bottom-right corner near the clock) for the Ollama icon. If it is not there, Ollama may not have started automatically.
  • Try opening a new PowerShell window and running ollama ls to see if the command is recognized. If it works, the server should be running.
  • If Ollama still does not respond, restart your computer and try visiting the URL again.

Help me get Ollama running on my Windows machine

Install Python 3

Python is the programming language you will use to write your chatbot script. Let's check if it is already on your machine.

  • In your PowerShell window, check for Python by running this command:
python --version

✔️ I see a version number

Python is already installed. You should see something like Python 3.x.x in the output. You are ready to move on.

ⓧ Python was not found or not recognized

Python is not installed yet. Follow these steps to install it:

  • Open your browser and go to https://www.python.org/downloads.
  • Click the large Download Python button at the top of the page.
  • Run the downloaded installer.
  • On the first screen of the installer, check the box that says Add python.exe to PATH. This is critical.
  • Click Install Now.
  • Wait for the installation to complete, then close the installer.
  • Close your current PowerShell window and open a new one (this picks up the updated PATH).
  • Verify Python is installed by running:
python --version

You should now see a version number like Python 3.x.x in the output.

Still not seeing a version number?

  • The most common cause is forgetting to check Add python.exe to PATH during installation. Uninstall Python from Settings > Apps and reinstall with the checkbox enabled.
  • Make sure you opened a brand new PowerShell window after installing. Old windows do not pick up PATH changes.

Help me get Python working on my Windows machine

Why Add python.exe to PATH?

PATH is a list of folders your computer searches when you type a command. Without Python on this list, PowerShell cannot find the python command. Checking that box during installation saves you from configuring it manually later.

Both tools are installed and ready. Next up, you will pull your first AI model and start chatting with it directly in the terminal.

Pull and Run Your First Model

In the previous step, you installed Ollama and Python on your machine. Now it's time to give Ollama a model to work with.

Ollama manages AI models like a package manager manages software. You download (or "pull") a model once, and then you can chat with it directly in your terminal whenever you want.

In this step, get ready to:
  • Pull the Gemma 3 model using the Ollama CLI.
  • Start an interactive chat session with the model.
  • Learn how to use multi-line input and exit the session.
Pull the Gemma 3 model

Before you can chat with an AI model, you need to download it to your machine. Ollama calls this "pulling" a model.

  • Pull the Gemma 3 model by running this command in PowerShell:
ollama pull gemma3

What does this command do?

ollama pull gemma3 downloads the Gemma 3 model files to your machine. This is a one-time download. Once pulled, the model stays on your computer and you can use it anytime without re-downloading.

Gemma 3 is a capable open-source model from Google that runs well on most hardware. The download is approximately 3-5 GB, so it may take a few minutes depending on your internet speed.

  • Wait for the download to complete. You'll see a progress bar in your terminal.
  • Confirm the model is available by running:
ollama ls

You should see gemma3 listed in the output with its size and a timestamp.

Download seems stuck or failed?

Make sure you have a stable internet connection. If the download fails partway through, run ollama pull gemma3 again. Ollama resumes from where it left off.

If you see a connection error, confirm Ollama is still running by visiting http://localhost:11434 in your browser. You should see a response from the Ollama server.

help me troubleshoot why my Ollama model pull is failing

Start an interactive chat session

Now that Gemma 3 is on your machine, you can talk to it directly in your terminal. No API keys, no internet connection needed.

  • Start an interactive chat session with Gemma 3 by running:
ollama run gemma3

What does this command do?

ollama run gemma3 starts an interactive session where you can type messages and get responses from the model. It loads the model into memory and gives you a prompt to start chatting.

If the model hasn't been pulled yet, ollama run will pull it automatically first. Since you already pulled it, this step will be instant.

  • You'll see a >>> prompt appear. Type a message like:
What are three interesting facts about Tokyo?

Press Enter and watch as the model generates a response directly in your terminal. The text appears word-by-word as the model produces it.

Seeing a long pause before any response?

The first message after starting a session takes longer because the model is loading into memory. Subsequent messages will be faster.

If your machine has limited RAM (less than 8 GB), responses may be slow. This is normal for running AI models locally.

my Ollama model is responding very slowly, how can I improve performance?

Explore multi-line input and exiting

The interactive session has a few useful features beyond simple one-line messages.

  • To send a multi-line message, type """ on a line by itself, then type your multi-line text, and close with another """ on its own line. Try it now:
"""
I'm planning a 5-day trip to Japan.
I like food, history, and nature.
What itinerary would you suggest?
"""

Why use multi-line input?

Multi-line input is useful when you want to give the model more context in a single message. Instead of cramming everything into one line, you can write naturally across multiple lines.

  • When you're done chatting, exit the session by typing:
/bye

This returns you to your normal PowerShell prompt.

Other ways to exit

You can also press Ctrl+C to exit the session at any time.

You've got a working AI model running locally on your machine. Next up, you'll create a custom personality for it using a Modelfile.

Create a Custom Model with a Modelfile

In the last step, you pulled Gemma 3 and had your first conversation with a local LLM. The responses were capable, but the model behaved like a generic assistant with no distinct personality.

Ollama lets you create custom models with unique personalities using a Modelfile. In this step, you'll define a friendly travel guide named Atlas and register it as its own model you can chat with.

In this step, get ready to:
  • Write a Modelfile that gives your model a custom personality.
  • Register the custom model with Ollama.
  • Test the custom model's personality in a live chat.
Write a Modelfile that defines your custom personality

A Modelfile is a configuration file that tells Ollama how to build a custom model. It specifies which base model to use, what personality to give it, and how creative its responses should be.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create a project folder and move into it by running these commands:
mkdir ai-chatbot
cd ai-chatbot

Your terminal prompt should now show ai-chatbot in the path, confirming you're inside the new folder.

Folder not created?

Make sure you ran cd ~/Desktop first. If PowerShell says the folder already exists, just run cd ai-chatbot to move into it.

help me navigate to my project folder in PowerShell

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop, select the ai-chatbot folder, then click Select Folder.
  • In the Explorer sidebar on the left, click the New File icon (a page with a small plus sign).
  • Type Modelfile as the file name and press Enter.

Why no file extension?

Ollama expects the file to be named exactly Modelfile with no extension like .txt or .yaml. This is similar to how Docker uses a file named Dockerfile.

  • Paste the following content into your Modelfile:
FROM gemma3
PARAMETER temperature 0.8
PARAMETER num_ctx 4096
SYSTEM You are a friendly and knowledgeable travel guide named Atlas. You help people plan trips, suggest destinations, and share interesting facts about places around the world. You speak in an enthusiastic but concise way, and you always suggest one unexpected hidden gem when recommending places.

What does this Modelfile do?

  • FROM gemma3 tells Ollama to use Gemma 3 as the base model for your custom creation.
  • PARAMETER temperature 0.8 controls how creative the responses are. Higher values (closer to 1) make the model more varied and expressive.
  • PARAMETER num_ctx 4096 sets the context window size. This determines how much of the conversation the model remembers at once.
  • SYSTEM defines the personality and behaviour rules. This is the instruction that shapes every response Atlas gives.
  • Save Modelfile by pressing Ctrl+S (Windows) or Cmd+S (macOS).
  • Confirm the file appears in the Explorer sidebar as Modelfile (no extension showing).

File saved with an extension?

If your file was saved as Modelfile.txt, right-click it in the Explorer sidebar and select Rename. Remove the .txt part so the name is just Modelfile.

help me rename my Modelfile to remove the file extension

✔️ Awesome, I've got everything!

Great. Double check you've saved your file.

ⓧ I'd like to double check the full code

FROM gemma3
PARAMETER temperature 0.8
PARAMETER num_ctx 4096
SYSTEM You are a friendly and knowledgeable travel guide named Atlas. You help people plan trips, suggest destinations, and share interesting facts about places around the world. You speak in an enthusiastic but concise way, and you always suggest one unexpected hidden gem when recommending places.
Register the custom model with Ollama

With the Modelfile written, you can now tell Ollama to build a custom model from it. This registers a new model named atlas that you can run anytime.

  • In your PowerShell terminal, make sure you're inside the ai-chatbot folder on your Desktop.
  • Create your custom model by running this command:
ollama create atlas -f Modelfile

What does this command do?

  • ollama create tells Ollama to build a new model.
  • atlas is the name you're giving your custom model.
  • -f Modelfile points to the configuration file you just wrote.
  • You should see output that ends with the word success.
  • Verify your model was registered by running:
ollama ls

You should see both gemma3 and atlas listed in the NAME column.

Seeing an error about the Modelfile?

Make sure your terminal is inside the ai-chatbot folder where you saved the Modelfile. Run ls to confirm you see Modelfile listed.

If the file isn't there, navigate to the correct folder with cd ~/Desktop/ai-chatbot and try again.

help me fix my ollama create command

Test the custom model interactively

Your custom model is now registered. Time to chat with Atlas and see its travel guide personality in action.

  • Start a chat session with your custom model by running:
ollama run atlas
  • Try asking Atlas a travel question, like:
What are the best places to visit in Japan?

Notice how Atlas responds with enthusiasm, gives concise recommendations, and suggests an unexpected hidden gem. This personality comes directly from the SYSTEM instruction in your Modelfile.

How is this different from base Gemma 3?

When you ran ollama run gemma3 in the previous step, the model responded in a neutral, encyclopedic way. Atlas has a defined personality: enthusiastic, concise, and always sharing hidden gems.

This is the power of a system prompt. You didn't retrain the model or change its weights. You gave it a role to play, and it follows those instructions in every response.

  • When you're done chatting, type /bye and press Enter to exit the session.

Your custom AI personality is alive and responding. Next up, you'll build a Python chatbot that talks to Atlas programmatically, with conversation memory and streaming responses.

Build a Python Chatbot with Conversation History

Your custom Atlas model has its own personality and speaks like a travel guide. Now it is time to go beyond the interactive terminal and talk to Atlas from a Python script.

In this step, you will install the Ollama Python library and build a chatbot that remembers what you said earlier in the conversation. You will also add streaming so responses print word-by-word, just like a real chat app.

In this step, get ready to:
  • Install the Ollama Python library.
  • Write a chatbot script that maintains conversation history.
  • Add streaming and run a multi-turn conversation with Atlas.
Install the Ollama Python library and write your chatbot

The Ollama Python library gives your scripts a direct line to any model running on your machine. It handles the connection to the local Ollama server so you can focus on the conversation logic.

  • Install the Ollama Python library by running this command:
pip install ollama

Seeing 'pip is not recognized'?

If your terminal does not recognize pip, try running python -m pip install ollama instead. This uses Python directly to find the pip module.

help me fix pip not being recognized in my terminal

Now you will create the chatbot script. The most important concept here is conversation history. Without it, the model forgets everything the moment you send a new message. Your script will store every exchange in a list and pass the full list each time, so Atlas remembers context from earlier in the chat.

  • Create a new file called chatbot.py in the same folder as your Modelfile.
  • Copy the following code into chatbot.py:
from ollama import chat


def main():
    print("=" * 50)
    print("  Welcome to Atlas - Your AI Travel Guide!")
    print("  Type 'quit' to exit")
    print("=" * 50)
    print()

    # Store every message so the model remembers context
    messages = []

    while True:
        user_input = input("You: ")

        # Let the user leave the chat gracefully
        if user_input.lower() in ("quit", "exit"):
            print("\nAtlas: Safe travels! See you next time.")
            break

        # Skip empty messages
        if not user_input.strip():
            continue

        # Add the user's message to the conversation history
        messages.append({"role": "user", "content": user_input})

What does this code do?

  • from ollama import chat loads the chat function from the Ollama Python library. This function talks to the Ollama server running on your machine.
  • messages = [] creates an empty list that will hold every message in the conversation. Each time you or Atlas speaks, the message gets appended here.
  • The while True loop keeps the conversation going until you type quit or exit.
  • messages.append({"role": "user", "content": user_input}) saves your message into the history list. The model receives this full list on every turn, so it knows what was said before.

You will not see a change yet. The next section adds the streaming response code that completes the function.

Add streaming support so responses appear word-by-word

Without streaming, you would wait in silence until the entire response finishes generating. Streaming prints each word the moment it is ready, giving you real-time feedback just like ChatGPT or any modern chat interface.

  • In chatbot.py, find the line that says messages.append({"role": "user", "content": user_input}). Copy the following code directly below it, inside the while True loop:
        # Print the name label without a newline so the response flows after it
        print("Atlas: ", end="", flush=True)

        # Call the model with full conversation history, streaming enabled
        stream = chat(
            model="atlas",
            messages=messages,
            stream=True,
        )

        # Print each chunk as it arrives
        response_text = ""
        for chunk in stream:
            content = chunk["message"]["content"]
            print(content, end="", flush=True)
            response_text += content

        print("\n")

        # Save the assistant's full response to conversation history
        messages.append({"role": "assistant", "content": response_text})


if __name__ == "__main__":
    main()

What does the streaming code do?

  • stream=True tells the Ollama server to send the response in small pieces instead of one big block. This is what makes text appear word-by-word.
  • The for chunk in stream loop grabs each piece as it arrives. flush=True forces the terminal to display each piece immediately instead of waiting for a newline.
  • response_text collects the full response so you can append it to messages at the end. This keeps the assistant's reply in the conversation history for future turns.
  • if __name__ == "__main__" makes the script run the main() function when you execute the file directly.
  • Save chatbot.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before moving on.

ⓧ I'd like to double check the full code

Here is the complete chatbot.py file. Make sure yours matches exactly:

from ollama import chat


def main():
    print("=" * 50)
    print("  Welcome to Atlas - Your AI Travel Guide!")
    print("  Type 'quit' to exit")
    print("=" * 50)
    print()

    messages = []

    while True:
        user_input = input("You: ")

        if user_input.lower() in ("quit", "exit"):
            print("\nAtlas: Safe travels! See you next time.")
            break

        if not user_input.strip():
            continue

        messages.append({"role": "user", "content": user_input})

        print("Atlas: ", end="", flush=True)

        stream = chat(
            model="atlas",
            messages=messages,
            stream=True,
        )

        response_text = ""
        for chunk in stream:
            content = chunk["message"]["content"]
            print(content, end="", flush=True)
            response_text += content

        print("\n")

        messages.append({"role": "assistant", "content": response_text})


if __name__ == "__main__":
    main()
Run your chatbot and have a multi-turn conversation

Your script is ready. Time to talk to Atlas and see conversation history in action.

  • Run the chatbot by entering this command:
python chatbot.py

You should see the welcome banner appear in your terminal:

==================================================
  Welcome to Atlas - Your AI Travel Guide!
  Type 'quit' to exit
==================================================
  • Type a question like What's a good destination in Japan for someone who loves food? and press Enter.
  • Watch Atlas respond word-by-word in real time.
  • Ask a follow-up question that references the previous answer, like What's the best time of year to visit there?. Atlas should remember the place it suggested because your script passes the full conversation history.
  • Type quit when you are done exploring.

How do I know conversation history is working?

If Atlas correctly answers a follow-up question without you repeating context, the conversation history is working. For example, if you ask about Japan and then say "how do I get there from London?", Atlas should know "there" means Japan.

Chatbot not responding or showing an error?

Make sure the Ollama server is still running. Open a separate terminal window and run ollama ls to confirm the atlas model is listed. If the server stopped, restart it by running ollama serve in that window.

Also check that you are running the script from the same folder where chatbot.py lives.

help me debug why my Python chatbot can't connect to Ollama

Secret mission

Add Structured JSON Output

Your chatbot currently responds with free-form text. What if you could make it return structured data that your code can parse and display in a custom format? In this mission, you will add JSON output support and build a recipe generator that returns ingredients, steps, and cooking times as parseable data.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on your local machine, so there are no ongoing cloud costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Ollama (local LLM server running in the background).
  • Gemma 3 model (approximately 3-5 GB of disk space).
  • Custom atlas model (built from your Modelfile).
  • Project files (Modelfile and chatbot.py).
  • Ollama Python library (installed via pip).

Keep everything running

No action needed. Choose this if you want to keep experimenting with your chatbot or try new models.

  • Ollama continues running in the background and uses minimal resources when idle.
  • Your models stay downloaded and ready to use at any time.
  • Your project files remain on your Desktop.

Pause - I'll come back to this later

Stop running processes to free up memory, but keep all files and models on disk so you can resume later.

  • Right-click the Ollama icon in your system tray (bottom-right of your taskbar) and select Quit Ollama.
  • Your models and project files stay on disk. To resume later, open Ollama again from your Start menu.

Delete - I don't want to use this again

Remove all project resources and start fresh. This frees up several gigabytes of disk space.

  • Remove the custom atlas model and the base Gemma 3 model by running these commands in PowerShell:
ollama rm atlas
ollama rm gemma3
  • Uninstall Ollama through Windows Settings. Press the Windows key, type Add or remove programs, and press Enter.
  • Find Ollama in the list, click it, and select Uninstall.
  • Delete any remaining model data by removing the .ollama folder. Open PowerShell and run:
Remove-Item -Recurse -Force "$env:USERPROFILE\.ollama"
  • Uninstall the Ollama Python library by running:
pip uninstall ollama -y
  • Delete your project files by running:
Remove-Item -Recurse -Force "$env:USERPROFILE\Desktop\ollama-chatbot"

What about Python?

If you installed Python specifically for this project and do not plan to use it for anything else, you can uninstall it through the same Add or remove programs menu. Search for Python in the list.

If you had Python before starting this project, leave it installed.

Nice Work!

Nice Work!

You just built a fully local AI chatbot running entirely on your own machine. No API keys, no cloud costs, no internet required.

Here's what you learned:

  • Installed Ollama and pulled an open-source language model to run locally on your Windows machine.
  • Created a custom AI personality using a Modelfile with a system prompt and tuned parameters.
  • Built a Python chatbot with conversation memory and real-time streaming responses.
  • Secret Mission: Added structured JSON output so your chatbot returns parseable, formatted data for practical use cases.

Ready to quiz yourself?