Build a Local AI Chatbot with Ollama

Build a Python chatbot that runs a local AI model with streaming responses.

Introduction

30 Second Summary

Every time you use ChatGPT, your messages travel to a remote server, get processed, and come back. What if you could run that same kind of AI right on your own computer, with no internet required?

In this project, you will build a personal AI chatbot that runs entirely on your local machine using Ollama and Python. You will type messages, watch the AI respond in real time, and customize its personality with a system prompt.

What You'll Build

A command-line AI chatbot powered by a local Gemma 3 model through Ollama, with streaming responses and multi-turn conversation memory.

By the end of this project, you'll have:

A Python script that connects to a local AI model and sends chat messages.
Streaming responses that print word by word in real time, just like ChatGPT.
Conversation memory so the AI remembers what you said earlier in the chat.
A custom system prompt that gives your chatbot its own unique personality.
Secret Mission: Turn your chatbot into a web app with a real chat interface using Streamlit.

Are there any prerequisites?

This project assumes you already have three tools installed: Ollama (running as a background service), Python 3.13.1 (installed via Anaconda), and VS Code. No coding experience is required.

Before We Start

Before We Start...

Set Up Your Python Environment

You have Ollama and Python installed on your machine. Right now though, your Python code has no way to communicate with Ollama.

In this step, you'll install a Python library that bridges the gap. You'll also download the AI model your chatbot will use and create the project file where all your code will live.

In this step, get ready to:

Install the ollama Python library.
Pull the Gemma 3 AI model.
Create your project folder and chatbot.py file.

Install the Ollama Python library

The ollama Python library lets your Python code talk directly to Ollama. It provides simple functions like chat() that send messages to any model running on your machine.

Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
Type Visual Studio Code and press Enter to open it.
Open a new terminal by clicking Terminal in the top menu bar, then clicking New Terminal. You can also press Ctrl+` as a shortcut.

What is a terminal?

A terminal is a text-based interface where you type commands for your computer to execute. VS Code has one built in, so you can run commands without switching between windows.

Install the ollama Python library by running this command:

pip install ollama==0.6.2

Confirm it installed correctly by running:

python -c "import ollama; print('ollama library ready!')"

You should see ollama library ready! printed in your terminal.

Seeing an error after the install?

If you see ModuleNotFoundError, the install may not have completed successfully. Run pip install ollama==0.6.2 again and look for any red error messages in the output.
If you see pip: command not found, your terminal may not be using the Anaconda Python. Close and reopen VS Code, then try again.
Help me fix my ollama library installation error.

Pull the Gemma 3 model

Ollama can run many different AI models. For this project, you'll use Gemma 3, a model created by Google that runs well on most computers without needing a powerful graphics card.

Pull the Gemma 3 model by running this command (this downloads about 3 GB and may take a few minutes):

ollama pull gemma3

Once the download finishes, verify the model is available by running:

ollama list

You should see gemma3 listed in the output.

Seeing a connection error?

Ollama needs to be running in the background for this command to work. Look for the Ollama icon in your system tray (bottom-right of your screen on Windows) or menu bar (macOS). If it is not there, open Ollama from your Start menu or Applications folder.
If you see a network timeout, check your internet connection and try running the command again.
Help me fix my Ollama connection error.

What does pulling a model mean?

Pulling a model downloads it from Ollama's model library to your local machine. Once downloaded, the model runs entirely on your computer with no internet required.

The Gemma 3 model you just pulled has 4 billion parameters. It is small enough to run comfortably on most machines with 8 GB or more of RAM.

Create your project folder and chatbot file

Now you need a dedicated folder for your chatbot project and a file to write your code in. You'll create both and open them in VS Code.

Move to your Desktop by running this command:

cd ~/Desktop

Create your project folder by running:

mkdir ollama-chatbot

Confirm the folder was created by running:

ls

You should see ollama-chatbot in the output.

In VS Code, click File in the top menu bar.
Click Open Folder.
Navigate to your Desktop, select the ollama-chatbot folder, then click Select Folder (Windows) or Open (macOS).
If VS Code asks whether you trust the authors of this folder, click Yes, I trust the authors.

VS Code reloads with the ollama-chatbot folder open. You should see the folder name in the Explorer sidebar on the left.

In the Explorer sidebar, hover over the OLLAMA-CHATBOT heading to reveal the action icons.
Click the New File icon (a page with a plus sign).
Type chatbot.py and press Enter.

You should see chatbot.py open in the editor as an empty file. This is the only file you need for the entire project.

Don't see the New File icon?

Make sure you opened the ollama-chatbot folder (not a file) using File > Open Folder. The Explorer sidebar only shows action icons when a folder is open.
If the Explorer sidebar is not visible, click View in the top menu bar, then click Explorer.
Help me create a new file in VS Code.

Your environment is all set. Next up, you'll write your first lines of Python and get the AI to respond.

Send Your First Message to the AI

Your Python environment is set up and the Gemma 3 model is ready on your machine. Now it is time to write your first lines of Python code.

You will use the chat function from the Ollama library to send a message to the AI and print its reply. This is the same pattern that every chatbot uses under the hood.

In this step, get ready to:

Write a Python script that sends a message to the Gemma 3 model.
Run the script and see the AI's response in your terminal.

Write a script that sends a message to the AI

To talk to an AI model through code, you send it a list of messages. Each message is a Python dictionary with two keys: role (who is speaking) and content (what they said).

In chatbot.py, add the following code:

from ollama import chat

# Send a message to the Gemma 3 model
response = chat(
    model="gemma3",
    messages=[{"role": "user", "content": "Hello! What is Python?"}],
)

# Print the AI's response text
print(response["message"]["content"])

What does this code do?

from ollama import chat imports the chat function from the Ollama Python library you installed in Step 1.
chat() sends your message to the Gemma 3 model running locally on your machine through Ollama.
messages is a list containing one dictionary. The role of "user" means you are the one speaking.
response["message"]["content"] digs into the response to extract just the text of the AI's reply.

What is a message role?

AI chat models use roles to keep track of who said what in a conversation. The "user" role is you (the human). The "assistant" role is the AI. There is also a "system" role that you will use later to control the AI's personality.

Save chatbot.py by pressing Cmd+S (macOS) or Ctrl+S (Windows).

You will not see anything happen yet. You will run this script in the next substep to see the AI's response.

Run your script and see the AI respond

Your script is saved and ready to go. Time to run it and watch the AI respond to your message.

Run your chatbot script by entering this command in your VS Code terminal:

python chatbot.py

After a few seconds, you should see the AI's response printed in your terminal. It will be a paragraph or two explaining what Python is.

Why does it take a few seconds?

The Gemma 3 model is running entirely on your computer's hardware. It needs a moment to process your message and generate a response. The speed depends on your machine's RAM and CPU.

Getting a connection error?

Make sure Ollama is running on your machine. On Windows, check your system tray (the small icons near the clock in the bottom-right corner of your screen) for the Ollama icon. If it is not there, search for Ollama in your Start menu and open it.
Make sure you saved chatbot.py before running the command. Unsaved changes will not be picked up.
Verify the Gemma 3 model is downloaded by running ollama list in your terminal. You should see gemma3 in the output.
Still stuck? Help me fix a connection error when running my chatbot.py script with Ollama.

✔️ Awesome, I've got everything!

Great. Double check you have saved chatbot.py and move on to the next step.

ⓧ I'd like to double check the full code

Here is the complete chatbot.py file so far. Make sure your code matches this exactly.

from ollama import chat

response = chat(
    model="gemma3",
    messages=[{"role": "user", "content": "Hello! What is Python?"}],
)

print(response["message"]["content"])

Your Python code is officially talking to a local AI model. Next up, you will make this chatbot feel much more like ChatGPT by adding streaming responses and a conversation loop.

Add Streaming and Build a Conversation Loop

In the last step, you sent your first message to the AI and got a response back. Your Python code is officially talking to the Gemma 3 model running on your machine.

Right now, the chatbot prints the entire response at once and only handles a single hardcoded message. In this step, you'll make responses stream word by word and build a conversation loop that remembers what you've discussed.

In this step, get ready to:

Enable streaming responses so the AI's words appear one at a time.
Build a conversation loop that remembers what you've discussed.
Test a multi-turn conversation with the AI.

Enable streaming so responses appear word by word

Your current chatbot waits for the AI to finish its entire response before printing anything. That's like reading a text message that only appears after the other person finishes typing the whole thing.

With streaming, the AI sends words one at a time as it generates them. This is how ChatGPT shows text appearing gradually.

Here's what the streaming pattern looks like in the Ollama Python library:

# Send the conversation to the AI with streaming enabled
stream = chat(
    model="gemma3",
    messages=messages,
    stream=True,
)

# Print each token as it arrives and accumulate the full response
full_response = ""
for chunk in stream:
    token = chunk["message"]["content"]
    print(token, end="", flush=True)
    full_response += token

What does this code do?

Adding stream=True to the chat() function changes its return value from a single response to an iterator. Each item in the iterator is a small chunk containing one token (a word or part of a word).

The flush=True parameter in print() forces each token to display immediately instead of being buffered. Without it, tokens might pile up invisibly before appearing all at once.

Wrap your code in an interactive loop

Right now your script sends one hardcoded message and exits. You need a loop that keeps asking for input and a list that grows with each exchange so the AI can see everything you've discussed.

Select all the code in chatbot.py by pressing Cmd+A (macOS) or Ctrl+A (Windows), then replace it with this code:

from ollama import chat


def main():
    # Store conversation history as a list of message dictionaries
    messages = []

    while True:
        # Get input from the user
        user_input = input("You: ").strip()

        # Skip empty inputs
        if not user_input:
            continue

        # Exit the loop if the user types quit or exit
        if user_input.lower() in ["quit", "exit"]:
            print("Goodbye!")
            break

        # Add the user's message to conversation history
        messages.append({"role": "user", "content": user_input})

        print("AI: ", end="", flush=True)

        # Send the full conversation to the AI with streaming enabled
        stream = chat(
            model="gemma3",
            messages=messages,
            stream=True,
        )

        # Print each token as it arrives and accumulate the full response
        full_response = ""
        for chunk in stream:
            token = chunk["message"]["content"]
            print(token, end="", flush=True)
            full_response += token

        print("\n")

        # Add the AI's complete response to conversation history
        messages.append({"role": "assistant", "content": full_response})


if __name__ == "__main__":
    main()

What does this code do?

messages = [] creates an empty list that will hold every message in the conversation. Each message is a dictionary with a role and content key.
while True creates an infinite loop. The chatbot keeps asking for input until you type quit or exit.
Every user message gets appended to messages before sending. After the AI responds, its complete reply is also appended. This growing list is how the AI "remembers" the conversation.
full_response collects every streamed token into one complete string. This is necessary because streaming gives you one small piece at a time, but you need the full text to store in messages.
if __name__ == "__main__" is a Python pattern that runs main() only when you execute the file directly. It keeps the code organized inside a function instead of running loose at the top level.

Save chatbot.py by pressing Cmd+S (macOS) or Ctrl+S (Windows).

Seeing a syntax error?

Check that the indentation uses spaces, not tabs. Every line inside def main(): should be indented by exactly 4 spaces.
Make sure every opening bracket, parenthesis, and quote has a matching closing one. A missing closing ) or " will cause an error.
Double check you replaced ALL of the old code. If any of the old lines remain mixed in, Python will throw an error.

Help me fix a syntax error in my chatbot.py file

✔️ Awesome, I've got everything!

Great. Double check you've saved chatbot.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete chatbot.py file. Make sure yours matches exactly.

from ollama import chat


def main():
    messages = []

    while True:
        user_input = input("You: ").strip()

        if not user_input:
            continue

        if user_input.lower() in ["quit", "exit"]:
            print("Goodbye!")
            break

        messages.append({"role": "user", "content": user_input})

        print("AI: ", end="", flush=True)

        stream = chat(
            model="gemma3",
            messages=messages,
            stream=True,
        )

        full_response = ""
        for chunk in stream:
            token = chunk["message"]["content"]
            print(token, end="", flush=True)
            full_response += token

        print("\n")

        messages.append({"role": "assistant", "content": full_response})


if __name__ == "__main__":
    main()

Test a multi-turn conversation

The real test of conversation memory is whether the AI can answer a follow-up question that only makes sense if it remembers what you said earlier.

Run your chatbot by entering this command in the VS Code terminal:

python chatbot.py

You should see a You: prompt waiting for your input.
Type a question and press Enter. For example, type What are three popular programming languages?
Watch the AI's response stream in word by word. Each token appears immediately instead of waiting for the whole reply.
Ask a follow-up question that references the previous answer, such as Which of those is best for beginners?
Ask at least one more follow-up to continue the thread.
Type quit to exit the chatbot when you're done.

What should I see?

The AI's response should appear token by token, similar to how ChatGPT displays text. When you ask a follow-up like "Which of those is best for beginners?", the AI should reference the specific languages it mentioned in its first answer.

If the AI seems confused by your follow-up or gives a generic answer that ignores the earlier messages, check that both messages.append() calls are present in your code. One adds the user message and the other adds the AI's response.

Seeing a connection error?

Ollama must be running in the background. Check your system tray (bottom-right corner on Windows) for the Ollama icon. If it's not there, search for Ollama in the Start menu and open it.
Make sure you pulled the Gemma 3 model in Step 1. Run ollama list in your terminal to confirm gemma3 appears in the list.

Help me fix a connection error with Ollama

Your chatbot can now hold a full conversation with streaming responses and conversation memory. Next up, you'll give it a unique personality with a custom system prompt.

Give Your Chatbot a Personality

Your chatbot can hold multi-turn conversations and stream responses in real time. Now it is time to give it a unique personality.

A system prompt tells the AI how to behave, like giving it a character to play. In this step, you will add one to your chatbot along with a welcome banner and a history-clearing command.

In this step, get ready to:

Add a system prompt that defines the AI's personality.
Add a welcome banner and a /clear command.
Test the chatbot with different personalities.

Add a system prompt to shape the AI's behavior

A system prompt is a special message with role set to system. It is always the first message in the list, and it tells the AI how to behave for the entire conversation.

In chatbot.py, find the line from ollama import chat. Add the following code below it:

SYSTEM_PROMPT = """You are a friendly coding tutor. You explain concepts simply
using analogies. You always encourage the learner. Keep responses concise."""

What is a system prompt?

SYSTEM_PROMPT is a multi-line string (wrapped in triple quotes """) that describes the AI's personality. The AI reads this before every response and shapes its tone, style, and behavior to match.

You can write anything here. A friendly tutor, a pirate, a Shakespearean poet. The AI will stay in character for the whole conversation.

Now you need to feed this system prompt to the AI as the first message in every conversation.

Inside def main(), find this line:

    messages = []

Replace [] with a list containing the system prompt message, so the line looks like this:

    messages = [{"role": "system", "content": SYSTEM_PROMPT}]

Why does the system prompt go in the messages list?

The Ollama chat() function receives the entire conversation as a list of messages. By making the system prompt the very first item, the AI sees it before any user messages. Every response it generates will follow the personality you defined.

You will not see a change yet. The next substep adds a welcome banner that confirms the chatbot is running.

Add a welcome banner and a /clear command

A welcome banner gives the user a clear signal that the chatbot has started. It also shows the available commands upfront so the user does not have to guess.

In chatbot.py, find the messages = [{"role": "system", "content": SYSTEM_PROMPT}] line you just updated. Add the following code above it (still inside def main()):

    print("=" * 50)
    print("  Local AI Chatbot (Powered by Ollama)")
    print("  Type 'quit' to exit, '/clear' to reset")
    print("=" * 50)
    print()

What does this code do?

print("=" * 50) prints a line of 50 equal signs. This creates a visual border for the banner.
The middle two lines display the chatbot name and the available commands.
print() adds a blank line after the banner so the first prompt is not cramped against it.

You will not see the banner until you run the script. Next, add the /clear command so users can reset the conversation without restarting the script.

Scroll down inside the while True loop. Find the line that says messages.append({"role": "user", "content": user_input}). Add the following code above it:

        if user_input == "/clear":
            messages = [{"role": "system", "content": SYSTEM_PROMPT}]
            print("Conversation cleared.\n")
            continue

What does this code do?

When the user types /clear, the code resets the messages list back to just the system prompt. This wipes the conversation history while keeping the AI's personality intact.
continue skips the rest of the loop so the cleared message is not sent to the AI as a chat message.

Save chatbot.py.
Run your chatbot by running this command in the VS Code terminal:

python chatbot.py

You should see the welcome banner printed in your terminal, followed by a You: prompt waiting for your input.

Don't see the welcome banner?

Make sure the five print() lines are inside def main() and above the messages = [...] line. They should be indented with four spaces.
Check that Ollama is running in the background. If you see a connection error, look for the Ollama icon in your system tray and make sure it is active.
Why isn't my chatbot welcome banner showing up?

✔️ Awesome, I've got everything!

Double check you have saved chatbot.py.

ⓧ I'd like to double check the full code

Here is the complete chatbot.py file. Compare it against yours and make sure everything matches.

from ollama import chat

SYSTEM_PROMPT = """You are a friendly coding tutor. You explain concepts simply
using analogies. You always encourage the learner. Keep responses concise."""


def main():
    print("=" * 50)
    print("  Local AI Chatbot (Powered by Ollama)")
    print("  Type 'quit' to exit, '/clear' to reset")
    print("=" * 50)
    print()

    messages = [{"role": "system", "content": SYSTEM_PROMPT}]

    while True:
        user_input = input("You: ").strip()

        if not user_input:
            continue

        if user_input.lower() in ["quit", "exit"]:
            print("Goodbye!")
            break

        if user_input == "/clear":
            messages = [{"role": "system", "content": SYSTEM_PROMPT}]
            print("Conversation cleared.\n")
            continue

        messages.append({"role": "user", "content": user_input})

        print("AI: ", end="", flush=True)

        stream = chat(
            model="gemma3",
            messages=messages,
            stream=True,
        )

        full_response = ""
        for chunk in stream:
            token = chunk["message"]["content"]
            print(token, end="", flush=True)
            full_response += token

        print("\n")

        messages.append({"role": "assistant", "content": full_response})


if __name__ == "__main__":
    main()

Test your custom AI personality

Your chatbot now has a personality, a welcome banner, and a clear command. Time to put it all to the test.

If the chatbot is not already running, start it by running this command:

python chatbot.py

Type a coding question like What is a variable? and press Enter. The AI should respond in a friendly, encouraging tone with analogies, matching the system prompt you defined.
Ask a follow-up question like Can you give me another example? to confirm the AI still remembers the context of your conversation.
Type /clear and press Enter. You should see Conversation cleared. printed in the terminal.
Ask the same follow-up question again. This time the AI should not remember your earlier conversation, because the history was cleared.
Type quit to exit the chatbot.

Try a different personality

Open chatbot.py and change the SYSTEM_PROMPT text to something completely different. Try a pirate, a Shakespearean poet, or a fitness coach.

Save the file, run the chatbot again, and see how the AI's responses change. When you are done experimenting, change the system prompt back to the original coding tutor personality.

AI not following the personality?

Make sure the system prompt message is the first item in the messages list. If it is not at index 0, the AI may not read it before other messages.
Check that the role is set to system (not user or assistant).
Why is my Ollama chatbot not following the system prompt personality?

Secret mission

Turn Your Chatbot into a Web App

Your chatbot works great in the terminal. But what if it had a real web interface with chat bubbles and streaming responses, just like ChatGPT? In this Secret Mission, you will use Streamlit to build exactly that.

Before you go

Clean Up Your Resources

This project runs entirely on your local machine, so there are no ongoing cloud costs. Decide whether to keep your resources as they are, pause Ollama to free up memory, or delete everything entirely.

Resources you used:

Ollama running as a background service.
Gemma 3 model downloaded locally (about 3 GB).
The ollama Python library installed via pip.
Your ollama-chatbot project folder containing chatbot.py.

Keep everything running

No action needed. Choose this if you plan to keep experimenting with your chatbot or building more projects with Ollama.

Ollama continues running in the background and is ready for your next project.
The Gemma 3 model stays downloaded so you can use it instantly.
Your chatbot.py file stays in your project folder, ready to run or modify.

Pause - I'll come back to this later

Shut down Ollama to free up system memory while keeping everything installed for later.

Find the Ollama icon in your system tray (bottom-right corner of your taskbar).
Right-click the Ollama icon and select Quit Ollama.
Your Gemma 3 model and project files stay on disk. When you are ready to come back, just reopen Ollama and run your script again.

Delete - I don't want to use this again

Remove all project resources and free up disk space.

Delete the Gemma 3 model by opening a terminal and running:

ollama rm gemma3

Uninstall the ollama Python library by running:

pip uninstall ollama

Delete your ollama-chatbot folder from your Desktop.
To fully remove Ollama itself, open Settings, select Apps, then Apps & features. Find Ollama in the list and click Uninstall.

Mission accomplished

That's a Wrap!

Nice work! You just built a fully functional AI chatbot that runs entirely on your local machine, powered by Ollama and Python.

You've learned how to:

Built a local AI chatbot that sends messages to a Gemma 3 model through Ollama and receives responses, with no cloud APIs or API keys needed.
Implemented real-time streaming so responses appear word by word, and built conversation memory so the AI remembers context across multiple messages.
Customized your chatbot's personality with a system prompt and added chat commands like /clear to manage conversations.
Secret Mission: Turned your terminal chatbot into a web app with a real chat interface using Streamlit.

Ready to quiz yourself?