Build an Advanced AI Toolkit with Ollama

Master local LLM control with vision, tool calling, and embeddings.

Introduction

30 Second Summary

Running AI models locally means you control your data, avoid API costs, and can experiment without limits. The Ollama Python library gives you programmatic access to every capability of these local models.

In this project, you will build a multi-file Python toolkit that demonstrates every major feature of the Ollama Python library. You will go hands-on with parameter tuning, vision analysis, structured outputs, tool calling, embeddings, and thinking mode.

What You'll Build

A collection of Python scripts that each demonstrate a different advanced capability of the Ollama Python library, from fine-grained generation control to autonomous tool-calling agents.

By the end of this project, you'll have:

A parameter playground that demonstrates how temperature, seed, stop sequences, and streaming affect model output.
A vision analyzer and structured data extractor that return validated JSON using Pydantic schemas.
A tool-calling agent that invokes Python functions automatically, and a semantic search engine using embeddings and cosine similarity.
Secret Mission: Build an autonomous agent loop that combines streaming, thinking mode, and multi-tool calling to solve problems without manual intervention.

Are there any prerequisites?

This project assumes basic Python knowledge and familiarity with Ollama basics (installing models and making simple chat calls). If you have never used Ollama before, complete an introductory project first.

Set Up Your Environment and Models

To build your advanced Ollama toolkit, you need three things in place: Python to write the scripts, Ollama to run models locally, and the right models downloaded for each capability you will explore.

In this step, you will install everything and create the project folder structure that houses your toolkit scripts.

In this step, get ready to:

Install Python dependencies and verify Ollama is running.
Pull the required models for vision, text generation, and embeddings.
Create the project folder structure with all six toolkit scripts.

Install Python dependencies and verify Ollama is running

The Ollama Python library gives you programmatic access to all of Ollama's capabilities. Pydantic handles structured output validation later in the project.

First, confirm you have Python 3.10+ installed. Check your version by running this command:

python3 --version

✔️ I see version 3.10 or higher

Python is ready. Move on to the Ollama check below.

ⓧ I see a version lower than 3.10

You need Python 3.10+ for this project. Download and install the latest version from python.org/downloads.

On macOS, the system may ship an older Python by default. After installing the newer version, use python3.10 (or python3.11, python3.12, etc.) directly, or use a version manager like pyenv to set your default.

After installing, run python3 --version again to confirm it shows 3.10 or higher.

ⓧ Command not found

Python is not installed on your machine. Download and install it from python.org/downloads.

Choose the latest stable release (3.12 or 3.13).
Run the installer and follow the prompts.
After installing, close and reopen your terminal, then run python3 --version to confirm.

Next, confirm Ollama is installed and running. Check by running:

ollama --version

✔️ I see a version number

Ollama is installed. Make sure the Ollama app is running (you should see the llama icon in your menu bar or system tray).

Verify Ollama's server is responding by opening http://localhost:11434 in your browser.
You should see a message confirming Ollama is running.

ⓧ Command not found

Ollama is not installed yet. Install it using the official command for your platform:

macOS

Install Ollama by running:

curl -fsSL https://ollama.com/install.sh | sh

This downloads and installs the Ollama app to /Applications and adds the ollama CLI to your PATH. It requires macOS 14 Sonoma or later.

Windows

Open PowerShell and install Ollama by running:

irm https://ollama.com/install.ps1 | iex

This requires Windows 10 or later.

Linux

Install Ollama by running:

curl -fsSL https://ollama.com/install.sh | sh

The script detects your architecture and installs the appropriate binary.

After installation, run ollama --version again to confirm it shows a version number.
Open http://localhost:11434 in your browser to verify the server is running.

Install the Python packages for this project by running:

pip install ollama pydantic

What does this install?

The ollama package is the official Python SDK for communicating with your local Ollama server. The pydantic package provides data validation that you will use for structured output extraction in later steps.

Confirm the library installed correctly by running:

python3 -c "import ollama; print(ollama.__version__)"

You should see a version number printed (e.g. 0.6.2 or later).

Seeing a ModuleNotFoundError?

Make sure pip and python3 point to the same Python installation. Try running pip3 install ollama pydantic instead, or use python3 -m pip install ollama pydantic to be explicit.

Help me troubleshoot this

Pull required models for the project

Your toolkit scripts use three models, each suited to a different task. You will pull them all now so they are ready when you need them.

Pull the general-purpose and vision-capable model by running:

ollama pull gemma3

Why gemma3 for both text and vision?

Gemma 3 supports both text generation and image understanding natively. This means you only need one model for text tasks, parameter experiments, tool calling, and vision analysis.

Pull the embedding model by running:

ollama pull all-minilm

What is all-minilm?

This is a lightweight embedding model that converts text into numerical vectors. You will use it to build a semantic search engine in a later step.

Verify both models are available by running:

ollama list

You should see both gemma3 and all-minilm listed in the output.

Model pull seems stuck or fails?

Model downloads can be large (several GB for gemma3). Make sure you have a stable internet connection and enough disk space. If the download fails, run the same ollama pull command again. It will resume where it left off.

Help me troubleshoot the download

Create the project folder structure

Your toolkit is a collection of independent Python scripts, each demonstrating a different capability. You will create all six files now so the folder structure is ready.

Move to your Desktop by running this command:

cd ~/Desktop

Create your project folder and move into it by running these commands:

mkdir ollama-toolkit
cd ollama-toolkit

Create the six empty script files by running:

touch params_playground.py vision_analyzer.py structured_extractor.py tool_agent.py embedding_search.py thinking_mode.py

Confirm the files exist by running:

ls

You should see all six files listed: params_playground.py, vision_analyzer.py, structured_extractor.py, tool_agent.py, embedding_search.py, and thinking_mode.py.

Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
Type VS Code and press Enter to open it.
In VS Code, click File in the top menu bar.
Click Open Folder.
Navigate to your Desktop and select the ollama-toolkit folder, then click Open.

You should see all six .py files in the VS Code Explorer sidebar on the left.

Don't see the files in VS Code?

Make sure you opened the ollama-toolkit folder itself, not the Desktop folder. The Explorer sidebar should show the six .py files at the top level.

Help me find my files in VS Code

Your environment is ready. Next up, you will build a parameter playground that shows how temperature, top_k, top_p, and other options shape model output.

Master Generation Parameters

Your environment is ready with Ollama running and your models pulled. Now it's time to start building.

Every LLM call accepts parameters that control how the model generates text. Temperature, seed, top_k, and stop sequences all influence whether output is creative, deterministic, or precisely bounded. In this step, you'll build a parameter playground script that lets you observe these effects side by side.

In this step, get ready to:

Compare output at different temperature values (0.1, 0.7, 1.2) and use a seed for reproducible results.
Use stop sequences and context window controls for precise generation.
Stream tokens in real-time and measure generation speed.

Build a parameter comparison script

The temperature parameter controls randomness in token selection. Low values produce focused, predictable output. High values produce diverse, creative output.

Open params_playground.py in VS Code by clicking it in the Explorer sidebar.
Add the imports, header, and temperature comparison loop by pasting this code:

import ollama
from ollama import Options

print("=" * 60)
print("OLLAMA PARAMETER PLAYGROUND")
print("=" * 60)

prompt = "Write a creative one-sentence story about a robot learning to paint."

# Temperature comparison
print("\n--- Temperature Comparison ---")
for temp in [0.1, 0.7, 1.2]:
    response = ollama.generate(
        model='gemma3',
        prompt=prompt,
        options={'temperature': temp, 'num_predict': 60},
        stream=False,
    )
    print(f"\nTemperature {temp}:")
    print(f"  {response['response']}")

What does this code do?

The script imports the ollama library and its Options class, then runs the same creative prompt three times with different temperature values.

Temperature 0.1 produces nearly identical output each time (very focused).
Temperature 0.7 gives balanced creativity (the default for most models).
Temperature 1.2 pushes the model toward more surprising word choices.
The num_predict option caps the response at 60 tokens so output stays short and comparable.

Save params_playground.py.

You won't run the script yet. You'll add more sections first and run the complete playground at the end.

Next, you'll add a deterministic output section that uses a fixed seed to prove the model can produce identical results.

Add the deterministic output section below your temperature comparison code:

# Deterministic output with seed
print("\n--- Deterministic Output (seed=42, temperature=0) ---")
for run in range(2):
    response = ollama.generate(
        model='gemma3',
        prompt="Name three colors.",
        options={'temperature': 0, 'seed': 42, 'num_predict': 30},
        stream=False,
    )
    print(f"  Run {run + 1}: {response['response']}")

What does this code do?

This runs the same prompt twice with identical settings. When temperature is 0 and seed is fixed at 42, the model eliminates all randomness. Both runs should produce identical output, proving reproducibility.

Save params_playground.py.

Seeing a connection error?

Make sure Ollama is still running. Check that http://localhost:11434 is responding in your browser.

If gemma3 isn't loaded, the first call may take a few seconds while the model loads into memory.

Need more help? Why is my Ollama Python script failing to connect?

Explore advanced options: stop sequences and context window

Beyond temperature and seed, you can control exactly where the model stops generating and how much context it can see. Stop sequences are strings that halt generation the moment they appear.

Add the stop sequences section below your deterministic output code:

# Stop sequences
print("\n--- Stop Sequences ---")
response = ollama.generate(
    model='gemma3',
    prompt="List 5 programming languages:\n1.",
    options={'stop': ['\n4.'], 'num_predict': 100},
    stream=False,
)
print(f"  Stopped output: 1.{response['response']}")

What does this code do?

The stop option accepts a list of strings. Generation halts as soon as any of them appear in the output. Here, '\n4.' stops the model before it reaches item 4. The output only contains items 1 through 3.

This is useful when you want to limit list output, stop before a certain section marker, or prevent the model from generating unwanted continuation text.

Save params_playground.py.

The context window determines how many tokens of conversation history the model can see at once. The default is 4096 tokens, but you can override it.

Add the context window section below your stop sequences code:

# Context window control
print("\n--- Context Window (num_ctx) ---")
response = ollama.generate(
    model='gemma3',
    prompt="Summarize the concept of machine learning in one paragraph.",
    options=Options(num_ctx=2048, num_predict=100, temperature=0.5),
    stream=False,
)
print(f"  Response: {response['response'][:200]}...")

What does this code do?

This section uses the Options class instead of a plain dictionary. Both approaches work identically. The Options class gives you IDE autocomplete on parameter names.

The num_ctx=2048 setting limits the context window to 2048 tokens. For short prompts this makes no visible difference, but for long conversations it determines how much history the model retains.

Save params_playground.py.

Add streaming output with real-time token display

By default, the Ollama library waits for the model to finish generating before returning the full response. Streaming lets you print tokens as they arrive, which gives the user immediate feedback.

Add the streaming section below your context window code:

# Streaming with token counting
print("\n--- Streaming Output ---")
print("  ", end="")
stream = ollama.generate(
    model='gemma3',
    prompt="Explain gravity in two sentences.",
    options={'num_predict': 60},
    stream=True,
)
for chunk in stream:
    print(chunk['response'], end='', flush=True)
    if chunk.get('done'):
        duration_s = chunk.get('total_duration', 0) / 1e9
        eval_count = chunk.get('eval_count', 0)
        if duration_s > 0:
            print(f"\n  [{eval_count} tokens in {duration_s:.1f}s = {eval_count/duration_s:.1f} tok/s]")

What does this code do?

Setting stream=True turns the response into an iterator. Each chunk contains a partial response in chunk['response'].

The flush=True argument forces Python to print each token immediately instead of buffering. When the final chunk arrives (done is True), it includes metadata like total_duration and eval_count so you can calculate tokens per second.

Save params_playground.py.

The last feature to add is keep_alive, which controls how long the model stays loaded in memory after a request.

Add the keep_alive section and the closing banner below your streaming code:

# Keep alive demonstration
print("\n--- Keep Alive ---")
response = ollama.generate(
    model='gemma3',
    prompt="Hello",
    keep_alive='10m',
    options={'num_predict': 5},
    stream=False,
)
print(f"  Model will stay loaded for 10 minutes.")
print(f"  Response: {response['response']}")

print("\n" + "=" * 60)
print("PLAYGROUND COMPLETE")
print("=" * 60)

What does keep_alive do?

By default, Ollama keeps a model loaded in memory for 5 minutes after the last request. The keep_alive parameter lets you override this. Set it to '10m' for 10 minutes, or 0 to unload the model immediately after the response.

Save params_playground.py.
Run the full script in your terminal by running this command:

python params_playground.py

You should see output for each section. The temperature comparison shows three different responses to the same prompt. The deterministic section produces two identical outputs. The streaming section prints tokens one by one, followed by a tokens-per-second measurement.

Script runs but output looks wrong?

If the deterministic outputs don't match, confirm both runs use temperature: 0 and seed: 42. A temperature above 0 reintroduces randomness even with a fixed seed.

If the streaming section shows no tokens-per-second, the model may have returned 0 for total_duration. This can happen with very short prompts. Try increasing num_predict to 100.

Need more help? Help me debug my params_playground.py output

✔️ Awesome, I've got everything!

Great. Double check you've saved your file.

ⓧ I'd like to double check the full code

import ollama
from ollama import Options

print("=" * 60)
print("OLLAMA PARAMETER PLAYGROUND")
print("=" * 60)

prompt = "Write a creative one-sentence story about a robot learning to paint."

# Temperature comparison
print("\n--- Temperature Comparison ---")
for temp in [0.1, 0.7, 1.2]:
    response = ollama.generate(
        model='gemma3',
        prompt=prompt,
        options={'temperature': temp, 'num_predict': 60},
        stream=False,
    )
    print(f"\nTemperature {temp}:")
    print(f"  {response['response']}")

# Deterministic output with seed
print("\n--- Deterministic Output (seed=42, temperature=0) ---")
for run in range(2):
    response = ollama.generate(
        model='gemma3',
        prompt="Name three colors.",
        options={'temperature': 0, 'seed': 42, 'num_predict': 30},
        stream=False,
    )
    print(f"  Run {run + 1}: {response['response']}")

# Stop sequences
print("\n--- Stop Sequences ---")
response = ollama.generate(
    model='gemma3',
    prompt="List 5 programming languages:\n1.",
    options={'stop': ['\n4.'], 'num_predict': 100},
    stream=False,
)
print(f"  Stopped output: 1.{response['response']}")

# Context window control
print("\n--- Context Window (num_ctx) ---")
response = ollama.generate(
    model='gemma3',
    prompt="Summarize the concept of machine learning in one paragraph.",
    options=Options(num_ctx=2048, num_predict=100, temperature=0.5),
    stream=False,
)
print(f"  Response: {response['response'][:200]}...")

# Streaming with token counting
print("\n--- Streaming Output ---")
print("  ", end="")
stream = ollama.generate(
    model='gemma3',
    prompt="Explain gravity in two sentences.",
    options={'num_predict': 60},
    stream=True,
)
for chunk in stream:
    print(chunk['response'], end='', flush=True)
    if chunk.get('done'):
        duration_s = chunk.get('total_duration', 0) / 1e9
        eval_count = chunk.get('eval_count', 0)
        if duration_s > 0:
            print(f"\n  [{eval_count} tokens in {duration_s:.1f}s = {eval_count/duration_s:.1f} tok/s]")

# Keep alive demonstration
print("\n--- Keep Alive ---")
response = ollama.generate(
    model='gemma3',
    prompt="Hello",
    keep_alive='10m',
    options={'num_predict': 5},
    stream=False,
)
print(f"  Model will stay loaded for 10 minutes.")
print(f"  Response: {response['response']}")

print("\n" + "=" * 60)
print("PLAYGROUND COMPLETE")
print("=" * 60)

You now have a working parameter playground that demonstrates the core generation controls. Next up, you'll use a vision model to analyze images and return structured data.

Analyze Images with Vision Models

In the previous step, you explored how generation parameters shape model output. You saw how temperature, seeds, and stop sequences give you fine-grained control over text generation.

Now you will make models see. Some models like gemma3 support multimodal input, meaning you can pass images alongside text prompts. In this step, you will build a vision analyzer that describes images and extracts structured data from them using Pydantic schemas.

In this step, get ready to:

Send an image to a vision model and get a natural-language description.
Combine vision with structured outputs to extract validated JSON from images.

Send an image to a vision model

To analyze an image, you pass a file path in the images field of a chat message. The SDK handles encoding the image data for you.

Open vision_analyzer.py in VS Code and add the imports and image download section by pasting this code:

import ollama
from pydantic import BaseModel
from typing import Literal, Optional
import urllib.request
import os

print("=" * 60)
print("OLLAMA VISION ANALYZER")
print("=" * 60)

# Download a sample image if not present
IMAGE_PATH = "sample_image.jpg"
if not os.path.exists(IMAGE_PATH):
    print("\nDownloading sample image...")
    url = "https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/PNG_transparency_demonstration_1.png/280px-PNG_transparency_demonstration_1.png"
    urllib.request.urlretrieve(url, IMAGE_PATH)
    print(f"  Saved to {IMAGE_PATH}")

What does this code do?

The script starts by defining a path for a sample image. If the image does not already exist on disk, it downloads a PNG transparency demo from Wikipedia using Python's built-in urllib.request module.

This approach means you can re-run the script without re-downloading the file every time.

Below the download section, add the basic vision call that sends the image to gemma3 and prints its description:

# Basic vision: describe an image
print("\n--- Basic Image Description ---")
response = ollama.chat(
    model='gemma3',
    messages=[{
        'role': 'user',
        'content': 'Describe this image in detail. What do you see?',
        'images': [IMAGE_PATH],
    }],
)
print(f"  {response.message.content}")

What does this code do?

The 'images': [IMAGE_PATH] field inside the message dict tells Ollama to include the image alongside the text prompt. The SDK reads the file from disk and encodes it automatically.

The model returns a free-form text description of whatever it sees in the image. You access it via response.message.content just like a normal chat response.

Save vision_analyzer.py.
Run the script so far to confirm the image downloads and the model produces a description:

python vision_analyzer.py

You should see a "Downloading sample image..." message followed by a text description of the image under "Basic Image Description".

Not seeing a description?

Make sure Ollama is still running. Check http://localhost:11434 in your browser to confirm.
If you see a network error during the download, check your internet connection. The script needs to reach Wikipedia to fetch the sample image.
Confirm you pulled the gemma3 model in Step 1. Vision requires a multimodal model.

Ask for help: My vision_analyzer.py isn't producing a description. How do I debug Ollama vision calls?

Combine vision with structured outputs

A free-form description is useful, but structured data is more powerful. By passing a Pydantic schema to the format parameter, you can force the model to return validated JSON that matches an exact shape you define.

Below the basic vision call, add the Pydantic model definitions that describe the structure you want the model to return:

# Structured vision output with Pydantic
print("\n--- Structured Vision Output ---")


class Object(BaseModel):
    name: str
    confidence: float
    attributes: str


class ImageDescription(BaseModel):
    summary: str
    objects: list[Object]
    scene: str
    colors: list[str]
    time_of_day: Literal['Morning', 'Afternoon', 'Evening', 'Night', 'Unknown']
    setting: Literal['Indoor', 'Outdoor', 'Unknown']
    text_content: Optional[str] = None

What does this code do?

You define two Pydantic models. Object represents a single detected item with a name, confidence score, and attributes. ImageDescription is the top-level schema with fields for summary, detected objects, scene description, colors, time of day, and setting.

The Literal type constrains the model to only return values from a fixed set. The Optional type allows a field to be None if the model cannot determine a value.

Below the class definitions, add the structured vision call that uses the schema and validates the response:

response = ollama.chat(
    model='gemma3',
    messages=[{
        'role': 'user',
        'content': 'Analyze this image. Describe the objects, scene, colors, and setting.',
        'images': [IMAGE_PATH],
    }],
    format=ImageDescription.model_json_schema(),
    options={'temperature': 0},
)

image_desc = ImageDescription.model_validate_json(response.message.content)
print(f"  Summary: {image_desc.summary}")
print(f"  Scene: {image_desc.scene}")
print(f"  Setting: {image_desc.setting}")
print(f"  Colors: {', '.join(image_desc.colors)}")
print(f"  Objects detected:")
for obj in image_desc.objects:
    print(f"    - {obj.name} (confidence: {obj.confidence}, {obj.attributes})")

print("\n" + "=" * 60)
print("VISION ANALYSIS COMPLETE")
print("=" * 60)

What does this code do?

The format=ImageDescription.model_json_schema() parameter tells Ollama to constrain the model's output to match your schema exactly. The model returns raw JSON instead of free-form text.

Setting temperature to 0 makes the output deterministic, which is important for structured extraction where you want consistent results.

Finally, ImageDescription.model_validate_json() parses the raw JSON string into a fully validated Python object. If the model returns invalid JSON, Pydantic raises an error instead of silently passing bad data through.

Save vision_analyzer.py and run it again:

python vision_analyzer.py

You should see both the free-form description and a structured output section showing the summary, scene, setting, colors, and detected objects with confidence scores.

Seeing a validation error?

If Pydantic raises a ValidationError, the model may have returned malformed JSON. Try running the script again. Local models occasionally produce slightly off-schema output on the first attempt.
Make sure options={'temperature': 0} is set. Higher temperatures increase the chance of JSON formatting errors.
Check that the format parameter uses model_json_schema() (a method call with parentheses), not model_json_schema (a reference without calling it).

Ask for help: My structured vision output is failing with a validation error. How do I fix it?

✔️ Awesome, I've got everything!

Great. Double check you have saved vision_analyzer.py and that both the basic description and structured output sections ran successfully.

ⓧ I'd like to double check the full code

import ollama
from pydantic import BaseModel
from typing import Literal, Optional
import urllib.request
import os

print("=" * 60)
print("OLLAMA VISION ANALYZER")
print("=" * 60)

# Download a sample image if not present
IMAGE_PATH = "sample_image.jpg"
if not os.path.exists(IMAGE_PATH):
    print("\nDownloading sample image...")
    url = "https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/PNG_transparency_demonstration_1.png/280px-PNG_transparency_demonstration_1.png"
    urllib.request.urlretrieve(url, IMAGE_PATH)
    print(f"  Saved to {IMAGE_PATH}")

# Basic vision: describe an image
print("\n--- Basic Image Description ---")
response = ollama.chat(
    model='gemma3',
    messages=[{
        'role': 'user',
        'content': 'Describe this image in detail. What do you see?',
        'images': [IMAGE_PATH],
    }],
)
print(f"  {response.message.content}")

# Structured vision output with Pydantic
print("\n--- Structured Vision Output ---")


class Object(BaseModel):
    name: str
    confidence: float
    attributes: str


class ImageDescription(BaseModel):
    summary: str
    objects: list[Object]
    scene: str
    colors: list[str]
    time_of_day: Literal['Morning', 'Afternoon', 'Evening', 'Night', 'Unknown']
    setting: Literal['Indoor', 'Outdoor', 'Unknown']
    text_content: Optional[str] = None


response = ollama.chat(
    model='gemma3',
    messages=[{
        'role': 'user',
        'content': 'Analyze this image. Describe the objects, scene, colors, and setting.',
        'images': [IMAGE_PATH],
    }],
    format=ImageDescription.model_json_schema(),
    options={'temperature': 0},
)

image_desc = ImageDescription.model_validate_json(response.message.content)
print(f"  Summary: {image_desc.summary}")
print(f"  Scene: {image_desc.scene}")
print(f"  Setting: {image_desc.setting}")
print(f"  Colors: {', '.join(image_desc.colors)}")
print(f"  Objects detected:")
for obj in image_desc.objects:
    print(f"    - {obj.name} (confidence: {obj.confidence}, {obj.attributes})")

print("\n" + "=" * 60)
print("VISION ANALYSIS COMPLETE")
print("=" * 60)

Your vision analyzer can now both describe images in natural language and extract structured, validated data from them. Next up, you will build a full structured extraction pipeline that pulls typed data out of unstructured text.

Extract Structured Data with Pydantic

In the previous step, you used Pydantic schemas to get structured vision output from an image. That same technique works for any text input, turning messy paragraphs into clean, validated data objects.

In this step, you will build a full data extraction pipeline. You will define schemas for different types of data (people, articles, pets), feed unstructured text to the model, and get back validated Python objects every time.

In this step, get ready to:

Define Pydantic schemas for structured data extraction.
Extract single entities and lists of entities from unstructured text.
Handle edge cases with error handling when the model returns unexpected data.

Build a data extraction pipeline using Pydantic schemas

Structured outputs force the model to return JSON that matches a schema you define. Instead of parsing free-text responses yourself, you get validated Python objects directly.

Open structured_extractor.py in VS Code.
Add the imports, header, and all four Pydantic model definitions by pasting this code:

import ollama
from pydantic import BaseModel
from typing import Optional

print("=" * 60)
print("OLLAMA STRUCTURED DATA EXTRACTOR")
print("=" * 60)


# Define extraction schemas
class Person(BaseModel):
    name: str
    age: Optional[int] = None
    occupation: str
    skills: list[str]


class Article(BaseModel):
    title: str
    author: str
    topic: str
    key_points: list[str]
    sentiment: str


class Pet(BaseModel):
    name: str
    animal: str
    age: Optional[int] = None
    breed: Optional[str] = None
    favorite_activity: str


class PetList(BaseModel):
    pets: list[Pet]

What does this code do?

Each class defines the exact shape of data you want the model to return. Pydantic validates the JSON against these schemas automatically.

Person extracts a single individual with name, age, occupation, and skills.
Article extracts metadata like title, author, topic, key points, and sentiment.
Pet and PetList demonstrate nested structures. A list of Pet objects inside a wrapper model.
Optional[int] = None means the field can be missing from the source text without causing an error.

Now add the Person extraction logic below the schema definitions:

# Extract a person from text
print("\n--- Person Extraction ---")
person_text = """
John Smith is a 34-year-old software architect based in Seattle.
He specializes in distributed systems, cloud infrastructure, and Go programming.
Before his current role, he spent 5 years at a major tech company working on microservices.
"""

response = ollama.chat(
    model='gemma3',
    messages=[{
        'role': 'user',
        'content': f'Extract person information from this text:\n\n{person_text}',
    }],
    format=Person.model_json_schema(),
    options={'temperature': 0},
)

person = Person.model_validate_json(response.message.content)
print(f"  Name: {person.name}")
print(f"  Age: {person.age}")
print(f"  Occupation: {person.occupation}")
print(f"  Skills: {', '.join(person.skills)}")

What does this code do?

format=Person.model_json_schema() tells the model to return JSON matching the Person schema exactly.
options={'temperature': 0} ensures deterministic output for consistent structured extraction.
Person.model_validate_json() parses the JSON string and validates it against the schema, returning a typed Python object.

Save structured_extractor.py.

Seeing a syntax error?

Make sure the triple-quoted string for person_text uses three double-quotes on each side. Check that the f-string in the content field has matching quotes.

Help me fix a syntax error in my structured_extractor.py file

Handle lists and nested structures

Extracting a single entity is useful, but real-world text often mentions multiple items. The PetList wrapper model lets you extract a list of Pet objects from a single paragraph.

Add the multiple entity extraction below the Person extraction code:

# Extract multiple pets from text
print("\n--- Multiple Entity Extraction (Pet List) ---")
pets_text = """
I have three pets at home. Luna is a 3-year-old tabby cat who loves chasing laser
pointers. Max is a golden retriever, about 5 years old, and his favorite thing is
playing fetch at the park. Finally there is Bubbles, my 2-year-old betta fish who
enjoys swimming through the castle decoration in her tank.
"""

response = ollama.chat(
    model='gemma3',
    messages=[{
        'role': 'user',
        'content': f'Extract all pets from this text:\n\n{pets_text}',
    }],
    format=PetList.model_json_schema(),
    options={'temperature': 0},
)

pet_list = PetList.model_validate_json(response.message.content)
print(f"  Found {len(pet_list.pets)} pets:")
for pet in pet_list.pets:
    print(f"    - {pet.name}: {pet.animal}, age {pet.age}, breed: {pet.breed or 'N/A'}")
    print(f"      Favorite activity: {pet.favorite_activity}")

What does this code do?

The paragraph describes three pets (Luna, Max, Bubbles) in natural language. By passing format=PetList.model_json_schema(), the model returns a JSON array wrapped in a pets field, with each pet validated against the Pet schema.

Optional fields like breed gracefully handle cases where the text does not mention that detail. The model fills in what it can find and leaves the rest as None.

Add the Article extraction below the PetList code:

# Article extraction
print("\n--- Article Extraction ---")
article_text = """
'The Future of Renewable Energy' by Dr. Sarah Chen explores how solar and wind
technologies are becoming cheaper than fossil fuels. The article highlights three
key developments: improved battery storage, new perovskite solar cells, and
offshore wind farms reaching record efficiency. Chen argues optimistically that
most countries could achieve 80% renewable energy by 2040.
"""

response = ollama.chat(
    model='gemma3',
    messages=[{
        'role': 'user',
        'content': f'Extract article metadata from this text:\n\n{article_text}',
    }],
    format=Article.model_json_schema(),
    options={'temperature': 0},
)

article = Article.model_validate_json(response.message.content)
print(f"  Title: {article.title}")
print(f"  Author: {article.author}")
print(f"  Topic: {article.topic}")
print(f"  Sentiment: {article.sentiment}")
print(f"  Key Points:")
for point in article.key_points:
    print(f"    - {point}")

What does this code do?

The Article schema includes a key_points: list[str] field. The model identifies individual key points from the paragraph and returns them as a proper list. It also infers the sentiment from the author's tone.

Finally, add the error handling demonstration and closing print at the bottom:

# Error handling demonstration
print("\n--- Error Handling ---")
try:
    bad_response = ollama.chat(
        model='gemma3',
        messages=[{'role': 'user', 'content': 'Say hello'}],
        format=Person.model_json_schema(),
        options={'temperature': 0},
    )
    person = Person.model_validate_json(bad_response.message.content)
    print(f"  Extracted (may have defaults): {person.name}, {person.occupation}")
except Exception as e:
    print(f"  Handled error: {type(e).__name__}: {e}")

print("\n" + "=" * 60)
print("EXTRACTION COMPLETE")
print("=" * 60)

Why wrap extraction in try/except?

When the source text does not contain relevant information for the schema, the model may invent placeholder values or return JSON that fails validation. The try/except block catches these failures gracefully so your application does not crash.

In this demo, asking the model to "Say hello" while forcing a Person schema may produce made-up values or a validation error. Either outcome is handled safely.

Save structured_extractor.py.
Run the script by executing this command in your terminal:

python structured_extractor.py

You should see structured output for each extraction task: a Person with name, age, occupation, and skills. Then a list of 3 pets (Luna, Max, Bubbles) with their details. Then article metadata with key points. Finally, the error handling section.

Seeing a validation error or unexpected output?

Make sure Ollama is still running. Check by visiting http://localhost:11434 in your browser.
If you see an import error for pydantic, make sure you installed it in Step 1 with pip install pydantic.
If the model returns unexpected field values, confirm you set options={'temperature': 0} in each ollama.chat() call.

Help me debug my structured_extractor.py output

✔️ Awesome, I've got everything!

Great. Double check you have saved structured_extractor.py and that all four extraction demos ran successfully.

ⓧ I'd like to double check the full code

import ollama
from pydantic import BaseModel
from typing import Optional

print("=" * 60)
print("OLLAMA STRUCTURED DATA EXTRACTOR")
print("=" * 60)


# Define extraction schemas
class Person(BaseModel):
    name: str
    age: Optional[int] = None
    occupation: str
    skills: list[str]


class Article(BaseModel):
    title: str
    author: str
    topic: str
    key_points: list[str]
    sentiment: str


class Pet(BaseModel):
    name: str
    animal: str
    age: Optional[int] = None
    breed: Optional[str] = None
    favorite_activity: str


class PetList(BaseModel):
    pets: list[Pet]


# Extract a person from text
print("\n--- Person Extraction ---")
person_text = """
John Smith is a 34-year-old software architect based in Seattle.
He specializes in distributed systems, cloud infrastructure, and Go programming.
Before his current role, he spent 5 years at a major tech company working on microservices.
"""

response = ollama.chat(
    model='gemma3',
    messages=[{
        'role': 'user',
        'content': f'Extract person information from this text:\n\n{person_text}',
    }],
    format=Person.model_json_schema(),
    options={'temperature': 0},
)

person = Person.model_validate_json(response.message.content)
print(f"  Name: {person.name}")
print(f"  Age: {person.age}")
print(f"  Occupation: {person.occupation}")
print(f"  Skills: {', '.join(person.skills)}")

# Extract multiple pets from text
print("\n--- Multiple Entity Extraction (Pet List) ---")
pets_text = """
I have three pets at home. Luna is a 3-year-old tabby cat who loves chasing laser
pointers. Max is a golden retriever, about 5 years old, and his favorite thing is
playing fetch at the park. Finally there is Bubbles, my 2-year-old betta fish who
enjoys swimming through the castle decoration in her tank.
"""

response = ollama.chat(
    model='gemma3',
    messages=[{
        'role': 'user',
        'content': f'Extract all pets from this text:\n\n{pets_text}',
    }],
    format=PetList.model_json_schema(),
    options={'temperature': 0},
)

pet_list = PetList.model_validate_json(response.message.content)
print(f"  Found {len(pet_list.pets)} pets:")
for pet in pet_list.pets:
    print(f"    - {pet.name}: {pet.animal}, age {pet.age}, breed: {pet.breed or 'N/A'}")
    print(f"      Favorite activity: {pet.favorite_activity}")

# Article extraction
print("\n--- Article Extraction ---")
article_text = """
'The Future of Renewable Energy' by Dr. Sarah Chen explores how solar and wind
technologies are becoming cheaper than fossil fuels. The article highlights three
key developments: improved battery storage, new perovskite solar cells, and
offshore wind farms reaching record efficiency. Chen argues optimistically that
most countries could achieve 80% renewable energy by 2040.
"""

response = ollama.chat(
    model='gemma3',
    messages=[{
        'role': 'user',
        'content': f'Extract article metadata from this text:\n\n{article_text}',
    }],
    format=Article.model_json_schema(),
    options={'temperature': 0},
)

article = Article.model_validate_json(response.message.content)
print(f"  Title: {article.title}")
print(f"  Author: {article.author}")
print(f"  Topic: {article.topic}")
print(f"  Sentiment: {article.sentiment}")
print(f"  Key Points:")
for point in article.key_points:
    print(f"    - {point}")

# Error handling demonstration
print("\n--- Error Handling ---")
try:
    bad_response = ollama.chat(
        model='gemma3',
        messages=[{'role': 'user', 'content': 'Say hello'}],
        format=Person.model_json_schema(),
        options={'temperature': 0},
    )
    person = Person.model_validate_json(bad_response.message.content)
    print(f"  Extracted (may have defaults): {person.name}, {person.occupation}")
except Exception as e:
    print(f"  Handled error: {type(e).__name__}: {e}")

print("\n" + "=" * 60)
print("EXTRACTION COMPLETE")
print("=" * 60)

You now have a reusable pattern for extracting structured data from any text. Next up, you will build a tool-calling agent that lets the model invoke Python functions automatically.

Build a Tool-Calling Agent

Your structured extractor can pull validated data from text, but the model is still passive. It answers questions without being able to do anything in the real world.

Tool calling changes that. You define regular Python functions, pass them to the model, and let it decide when to invoke them. The Ollama Python SDK auto-generates JSON schemas from your function signatures and docstrings, so the model knows exactly what each tool does and what arguments it needs.

In this step, get ready to:

Define typed Python functions with Google-style docstrings that the SDK auto-converts to tool schemas.
Build a multi-tool agent loop that executes functions and feeds results back to the model.
Test the agent with single-tool, multi-tool, and edge-case queries.

Define tool functions with auto-generated schemas

The Ollama Python SDK reads type annotations and Google-style docstrings from your functions to auto-generate the JSON schema the model needs. Each function you define becomes a tool the model can invoke by name.

Open tool_agent.py in VS Code and add the imports, header, and four tool functions by pasting the following code:

import ollama
from datetime import datetime

print("=" * 60)
print("OLLAMA TOOL-CALLING AGENT")
print("=" * 60)


# Define tools as typed Python functions with docstrings
def get_current_time() -> str:
    """Get the current date and time.

    Returns:
        str: The current date and time in a readable format.
    """
    return datetime.now().strftime("%Y-%m-%d %H:%M:%S")


def calculate(expression: str) -> str:
    """Evaluate a mathematical expression safely.

    Args:
        expression: A mathematical expression to evaluate (e.g., '2 + 2', 'sqrt(16)')

    Returns:
        str: The result of the calculation.
    """
    import math
    allowed_names = {
        'sqrt': math.sqrt, 'abs': abs, 'round': round,
        'pow': pow, 'min': min, 'max': max,
        'pi': math.pi, 'e': math.e,
    }
    try:
        result = eval(expression, {"__builtins__": {}}, allowed_names)
        return str(result)
    except Exception as e:
        return f"Error: {e}"


def search_notes(query: str) -> str:
    """Search through saved notes for relevant information.

    Args:
        query: The search term to look for in notes.

    Returns:
        str: Matching notes or a message if nothing found.
    """
    notes = {
        "meeting": "Team meeting at 3pm on Friday to discuss Q2 roadmap.",
        "shopping": "Buy milk, eggs, bread, and coffee beans.",
        "project": "Project deadline is June 30. Need to finish API integration.",
        "birthday": "Mom's birthday is March 15. Dad's is October 3.",
    }
    results = []
    for key, note in notes.items():
        if query.lower() in key or query.lower() in note.lower():
            results.append(note)
    return "\n".join(results) if results else f"No notes found matching '{query}'."


def get_weather(city: str) -> str:
    """Get the current weather for a city.

    Args:
        city: The name of the city.

    Returns:
        str: The current weather conditions.
    """
    weather_data = {
        "new york": "72F, Partly Cloudy",
        "london": "58F, Rainy",
        "tokyo": "68F, Clear",
        "paris": "64F, Overcast",
    }
    return weather_data.get(city.lower(), f"Weather data not available for {city}.")

What does this code do?

Each function follows the same pattern: type-annotated parameters, a return type, and a Google-style docstring with Args and Returns sections. The SDK reads these to auto-generate the JSON schema the model uses to decide which tool to call and what arguments to pass.

The calculate function uses a restricted eval with an allowed_names dict so only safe math operations are available. The other functions use hardcoded data to simulate real-world APIs.

Why typed functions instead of manual schemas?

Before library version 0.4, you had to write JSON schemas manually for each tool. Now you pass Python functions directly and the SDK generates the schema from your type annotations and docstrings. Less boilerplate, fewer bugs.

You won't see output yet. The next chunk builds the agent loop that actually invokes these tools.

Build a multi-tool agent with a conversation loop

The agent loop is the core pattern: send a message, check if the model wants to call tools, execute each tool, feed results back, and let the model produce a final answer.

Below the get_weather function, add the function registry and agent loop by pasting this code:

# Map function names to callables
available_functions = {
    'get_current_time': get_current_time,
    'calculate': calculate,
    'search_notes': search_notes,
    'get_weather': get_weather,
}

all_tools = [get_current_time, calculate, search_notes, get_weather]


def run_agent(user_message: str):
    """Run the agent with a user message, handling tool calls."""
    print(f"\nUser: {user_message}")
    print("-" * 40)

    messages = [{'role': 'user', 'content': user_message}]

    # First call to the model
    response = ollama.chat(
        model='gemma3',
        messages=messages,
        tools=all_tools,
    )

    # Check if the model wants to call tools
    if not response.message.tool_calls:
        print(f"Assistant: {response.message.content}")
        return

    # Process tool calls
    messages.append(response.message)

    for tool_call in response.message.tool_calls:
        func_name = tool_call.function.name
        func_args = tool_call.function.arguments

        print(f"  [Tool Call] {func_name}({func_args})")

        if func_name in available_functions:
            if func_args:
                result = available_functions[func_name](**func_args)
            else:
                result = available_functions[func_name]()
            print(f"  [Result] {result}")
        else:
            result = f"Error: Function '{func_name}' not found."
            print(f"  [Error] {result}")

        messages.append({
            'role': 'tool',
            'content': str(result),
            'tool_name': func_name,
        })

    # Get final response with tool results
    final_response = ollama.chat(
        model='gemma3',
        messages=messages,
        tools=all_tools,
    )
    print(f"\nAssistant: {final_response.message.content}")

What does this code do?

The available_functions dict maps function name strings to callables. When the model returns a tool call, you look up the function by name and invoke it.

The all_tools list passes the raw function references to ollama.chat(). The SDK reads each function's signature and docstring to generate the tool schema automatically.

Inside run_agent, the key pattern is: call the model with tools=all_tools, check response.message.tool_calls, execute each tool, append results as role: 'tool' messages with a tool_name field, then call the model again to get the final answer.

Now add the example calls at the bottom of the file to test the agent with different query types:

# Run several examples
print("\n--- Single Tool Call ---")
run_agent("What time is it right now?")

print("\n--- Calculation ---")
run_agent("What is the square root of 144 plus 10?")

print("\n--- Note Search ---")
run_agent("Do I have any notes about a project deadline?")

print("\n--- Weather Query ---")
run_agent("What's the weather like in Tokyo?")

print("\n--- Multi-step Question ---")
run_agent("What's the weather in London and what time is it?")

print("\n" + "=" * 60)
print("AGENT COMPLETE")
print("=" * 60)

What do these examples test?

The first four queries each invoke a single tool. The last query ("What's the weather in London and what time is it?") tests whether the model can return multiple tool calls in a single response. Your agent loop handles both cases because it iterates over all tool_calls in the response.

Save tool_agent.py.
Run the script by executing this command in your terminal:

python tool_agent.py

You should see the model making tool calls (printed as [Tool Call] lines), the results from each function (printed as [Result] lines), and the model's final response summarizing the information.

Not seeing any tool calls?

If the model responds directly without invoking tools, it may not have recognized the functions as relevant. Make sure your docstrings clearly describe what each function does. The model uses those descriptions to decide when a tool is appropriate.

If you see a connection error, confirm Ollama is still running at http://localhost:11434 and that the gemma3 model is pulled.

Help me debug my tool-calling agent

✔️ Awesome, I've got everything!

Great. Double check you have saved your file.

ⓧ I'd like to double check the full code

import ollama
from datetime import datetime

print("=" * 60)
print("OLLAMA TOOL-CALLING AGENT")
print("=" * 60)


# Define tools as typed Python functions with docstrings
def get_current_time() -> str:
    """Get the current date and time.

    Returns:
        str: The current date and time in a readable format.
    """
    return datetime.now().strftime("%Y-%m-%d %H:%M:%S")


def calculate(expression: str) -> str:
    """Evaluate a mathematical expression safely.

    Args:
        expression: A mathematical expression to evaluate (e.g., '2 + 2', 'sqrt(16)')

    Returns:
        str: The result of the calculation.
    """
    import math
    allowed_names = {
        'sqrt': math.sqrt, 'abs': abs, 'round': round,
        'pow': pow, 'min': min, 'max': max,
        'pi': math.pi, 'e': math.e,
    }
    try:
        result = eval(expression, {"__builtins__": {}}, allowed_names)
        return str(result)
    except Exception as e:
        return f"Error: {e}"


def search_notes(query: str) -> str:
    """Search through saved notes for relevant information.

    Args:
        query: The search term to look for in notes.

    Returns:
        str: Matching notes or a message if nothing found.
    """
    notes = {
        "meeting": "Team meeting at 3pm on Friday to discuss Q2 roadmap.",
        "shopping": "Buy milk, eggs, bread, and coffee beans.",
        "project": "Project deadline is June 30. Need to finish API integration.",
        "birthday": "Mom's birthday is March 15. Dad's is October 3.",
    }
    results = []
    for key, note in notes.items():
        if query.lower() in key or query.lower() in note.lower():
            results.append(note)
    return "\n".join(results) if results else f"No notes found matching '{query}'."


def get_weather(city: str) -> str:
    """Get the current weather for a city.

    Args:
        city: The name of the city.

    Returns:
        str: The current weather conditions.
    """
    weather_data = {
        "new york": "72F, Partly Cloudy",
        "london": "58F, Rainy",
        "tokyo": "68F, Clear",
        "paris": "64F, Overcast",
    }
    return weather_data.get(city.lower(), f"Weather data not available for {city}.")


# Map function names to callables
available_functions = {
    'get_current_time': get_current_time,
    'calculate': calculate,
    'search_notes': search_notes,
    'get_weather': get_weather,
}

all_tools = [get_current_time, calculate, search_notes, get_weather]


def run_agent(user_message: str):
    """Run the agent with a user message, handling tool calls."""
    print(f"\nUser: {user_message}")
    print("-" * 40)

    messages = [{'role': 'user', 'content': user_message}]

    # First call to the model
    response = ollama.chat(
        model='gemma3',
        messages=messages,
        tools=all_tools,
    )

    # Check if the model wants to call tools
    if not response.message.tool_calls:
        print(f"Assistant: {response.message.content}")
        return

    # Process tool calls
    messages.append(response.message)

    for tool_call in response.message.tool_calls:
        func_name = tool_call.function.name
        func_args = tool_call.function.arguments

        print(f"  [Tool Call] {func_name}({func_args})")

        if func_name in available_functions:
            if func_args:
                result = available_functions[func_name](**func_args)
            else:
                result = available_functions[func_name]()
            print(f"  [Result] {result}")
        else:
            result = f"Error: Function '{func_name}' not found."
            print(f"  [Error] {result}")

        messages.append({
            'role': 'tool',
            'content': str(result),
            'tool_name': func_name,
        })

    # Get final response with tool results
    final_response = ollama.chat(
        model='gemma3',
        messages=messages,
        tools=all_tools,
    )
    print(f"\nAssistant: {final_response.message.content}")


# Run several examples
print("\n--- Single Tool Call ---")
run_agent("What time is it right now?")

print("\n--- Calculation ---")
run_agent("What is the square root of 144 plus 10?")

print("\n--- Note Search ---")
run_agent("Do I have any notes about a project deadline?")

print("\n--- Weather Query ---")
run_agent("What's the weather like in Tokyo?")

print("\n--- Multi-step Question ---")
run_agent("What's the weather in London and what time is it?")

print("\n" + "=" * 60)
print("AGENT COMPLETE")
print("=" * 60)

Your agent can now invoke Python functions on demand and incorporate the results into its responses. Next up, you will build a semantic search engine using embeddings and cosine similarity.

Build a Semantic Search Engine with Embeddings

Your tool-calling agent can invoke functions based on what the user asks. But what if the model needs to find relevant information from a collection of documents before answering? This is where embeddings come in.

Embeddings convert text into numerical vectors that capture meaning. Similar texts end up with similar vectors, making it possible to search by concept rather than exact keywords. In this step, you will build a semantic search engine from scratch and use it to implement a basic RAG pattern.

In this step, get ready to:

Generate embeddings for a collection of documents using batch embedding.
Implement cosine similarity search to find the most relevant documents for any query.
Build a basic RAG pipeline that uses search results as context for generation.

Generate embeddings and build a simple semantic search system

Before you can search by meaning, you need a collection of documents and a way to measure how similar two pieces of text are. You will start by defining your document corpus and a mathematical similarity function.

Open embedding_search.py in VS Code.
Add the imports, embedding model constant, and document collection by pasting this code:

import ollama
import math

print("=" * 60)
print("OLLAMA EMBEDDING SEARCH ENGINE")
print("=" * 60)

EMBED_MODEL = 'all-minilm'

# Documents to index
documents = [
    "Python is a high-level programming language known for its simplicity and readability.",
    "Machine learning is a subset of artificial intelligence that learns from data.",
    "Docker containers package applications with their dependencies for consistent deployment.",
    "PostgreSQL is a powerful open-source relational database management system.",
    "React is a JavaScript library for building user interfaces with reusable components.",
    "Kubernetes orchestrates containerized applications across clusters of machines.",
    "Natural language processing enables computers to understand human language.",
    "Git is a distributed version control system for tracking changes in source code.",
    "REST APIs use HTTP methods to create, read, update, and delete resources.",
    "Neural networks are computing systems inspired by biological neural networks.",
]

What does this code do?

EMBED_MODEL = 'all-minilm' sets the embedding model you pulled in Step 1. This lightweight model converts text into 384-dimensional vectors quickly.

The documents list contains 10 short descriptions covering different tech topics. These will form your searchable knowledge base.

Next, add the cosine similarity function directly below the documents list:


def cosine_similarity(vec_a: list[float], vec_b: list[float]) -> float:
    """Calculate cosine similarity between two vectors."""
    dot_product = sum(a * b for a, b in zip(vec_a, vec_b))
    magnitude_a = math.sqrt(sum(a * a for a in vec_a))
    magnitude_b = math.sqrt(sum(b * b for b in vec_b))
    if magnitude_a == 0 or magnitude_b == 0:
        return 0.0
    return dot_product / (magnitude_a * magnitude_b)

What is cosine similarity?

Cosine similarity measures the angle between two vectors, returning a value between -1 and 1. A score of 1 means the vectors point in the same direction (highly similar). A score of 0 means they are unrelated.

The function computes the dot product of the two vectors and divides by the product of their magnitudes. The zero-check prevents division errors when a vector has no magnitude.

Now add the batch embedding and index-building code below the cosine_similarity function:


# Index documents with batch embedding
print("\n--- Indexing Documents ---")
print(f"  Embedding {len(documents)} documents with '{EMBED_MODEL}'...")

batch_response = ollama.embed(model=EMBED_MODEL, input=documents)
embeddings = batch_response['embeddings']

print(f"  Done! Each embedding has {len(embeddings[0])} dimensions.")

# Build the index (list of dicts)
index = []
for i, (doc, emb) in enumerate(zip(documents, embeddings)):
    index.append({'id': i, 'text': doc, 'embedding': emb})

What does this code do?

ollama.embed(model=EMBED_MODEL, input=documents) sends all 10 documents in one batch call instead of embedding them one at a time. This is much faster for multiple inputs.

The response contains an embeddings list, one vector per document. You then pair each document with its embedding in the index list so you can look up the original text after a similarity search.

Why batch embedding instead of single calls?

The ollama.embed() function accepts either a single string or a list of strings for the input parameter. Passing a list embeds everything in one round-trip to the model, avoiding the overhead of 10 separate API calls.

Save embedding_search.py.

Seeing a model not found error?

Make sure you pulled the all-minilm model in Step 1. You can verify by running ollama list in your terminal and checking that all-minilm appears.

help me fix an ollama model not found error for all-minilm

Implement cosine similarity search from scratch

With your documents embedded and indexed, you can now build a search function that finds the most relevant documents for any natural-language query.

Add the search function below the index-building code:


def search(query: str, top_k: int = 3) -> list[dict]:
    """Search the index for documents similar to the query."""
    query_response = ollama.embed(model=EMBED_MODEL, input=query)
    query_embedding = query_response['embeddings'][0]

    results = []
    for item in index:
        score = cosine_similarity(query_embedding, item['embedding'])
        results.append({'text': item['text'], 'score': score})

    results.sort(key=lambda x: x['score'], reverse=True)
    return results[:top_k]

What does this code do?

The search function takes a query string and returns the top matching documents. It embeds the query using the same model, computes cosine similarity against every document in the index, sorts by score, and returns the top top_k results.

Because both queries and documents are embedded with the same model, their vectors live in the same space. This means a query about "managing containers" will score highly against the Docker and Kubernetes documents, even without sharing exact keywords.

Add the demo queries section below the search function:


# Run search queries
print("\n--- Search Queries ---")
queries = [
    "How do I manage containers?",
    "What language is good for beginners?",
    "How do computers understand text?",
]

for query in queries:
    print(f"\n  Query: '{query}'")
    results = search(query, top_k=3)
    for i, r in enumerate(results, 1):
        print(f"    {i}. [{r['score']:.4f}] {r['text']}")

What should I see?

Each query prints its top 3 results with a similarity score in square brackets. For example, "How do I manage containers?" should rank the Docker and Kubernetes documents highest, even though the query never uses those exact words.

Finally, add the basic RAG pattern at the bottom of the file:


# Basic RAG pattern: use top result as context
print("\n--- Basic RAG Pattern ---")
rag_query = "What is the best tool for version control?"
print(f"  Query: '{rag_query}'")

search_results = search(rag_query, top_k=2)
context = "\n".join([r['text'] for r in search_results])

response = ollama.generate(
    model='gemma3',
    prompt=f"Based on the following context, answer the question.\n\nContext:\n{context}\n\nQuestion: {rag_query}\n\nAnswer:",
    options={'temperature': 0.3, 'num_predict': 100},
    stream=False,
)
print(f"  Context used: {search_results[0]['text']}")
print(f"  Answer: {response['response']}")

print("\n" + "=" * 60)
print("SEARCH ENGINE COMPLETE")
print("=" * 60)

What is the RAG pattern?

RAG (Retrieval-Augmented Generation) is a technique where you search for relevant documents first, then pass them as context to the LLM. The model generates an answer grounded in your specific data rather than relying solely on its training.

Here you search for the top 2 documents matching the query, join them into a context string, and include them in the prompt to ollama.generate(). The low temperature of 0.3 keeps the answer focused on the provided context.

Save embedding_search.py.
Run the script by executing this command in your terminal:

python embedding_search.py

You should see the indexing confirmation, three search queries with ranked results and similarity scores, and a RAG-generated answer about version control.

Script runs but similarity scores all look the same?

Make sure your cosine_similarity function uses math.sqrt for the magnitude calculation. A missing square root will produce incorrect scores.

Also confirm you are using all-minilm as the embedding model. Using a chat model like gemma3 for embeddings will not produce meaningful similarity results.

help me debug my embedding search similarity scores

✔️ Awesome, I've got everything!

Great. Double check you have saved embedding_search.py and that the script executed without errors.

ⓧ I'd like to double check the full code

import ollama
import math

print("=" * 60)
print("OLLAMA EMBEDDING SEARCH ENGINE")
print("=" * 60)

EMBED_MODEL = 'all-minilm'

# Documents to index
documents = [
    "Python is a high-level programming language known for its simplicity and readability.",
    "Machine learning is a subset of artificial intelligence that learns from data.",
    "Docker containers package applications with their dependencies for consistent deployment.",
    "PostgreSQL is a powerful open-source relational database management system.",
    "React is a JavaScript library for building user interfaces with reusable components.",
    "Kubernetes orchestrates containerized applications across clusters of machines.",
    "Natural language processing enables computers to understand human language.",
    "Git is a distributed version control system for tracking changes in source code.",
    "REST APIs use HTTP methods to create, read, update, and delete resources.",
    "Neural networks are computing systems inspired by biological neural networks.",
]


def cosine_similarity(vec_a: list[float], vec_b: list[float]) -> float:
    """Calculate cosine similarity between two vectors."""
    dot_product = sum(a * b for a, b in zip(vec_a, vec_b))
    magnitude_a = math.sqrt(sum(a * a for a in vec_a))
    magnitude_b = math.sqrt(sum(b * b for b in vec_b))
    if magnitude_a == 0 or magnitude_b == 0:
        return 0.0
    return dot_product / (magnitude_a * magnitude_b)


# Index documents with batch embedding
print("\n--- Indexing Documents ---")
print(f"  Embedding {len(documents)} documents with '{EMBED_MODEL}'...")

batch_response = ollama.embed(model=EMBED_MODEL, input=documents)
embeddings = batch_response['embeddings']

print(f"  Done! Each embedding has {len(embeddings[0])} dimensions.")

# Build the index (list of dicts)
index = []
for i, (doc, emb) in enumerate(zip(documents, embeddings)):
    index.append({'id': i, 'text': doc, 'embedding': emb})


def search(query: str, top_k: int = 3) -> list[dict]:
    """Search the index for documents similar to the query."""
    query_response = ollama.embed(model=EMBED_MODEL, input=query)
    query_embedding = query_response['embeddings'][0]

    results = []
    for item in index:
        score = cosine_similarity(query_embedding, item['embedding'])
        results.append({'text': item['text'], 'score': score})

    results.sort(key=lambda x: x['score'], reverse=True)
    return results[:top_k]


# Run search queries
print("\n--- Search Queries ---")
queries = [
    "How do I manage containers?",
    "What language is good for beginners?",
    "How do computers understand text?",
]

for query in queries:
    print(f"\n  Query: '{query}'")
    results = search(query, top_k=3)
    for i, r in enumerate(results, 1):
        print(f"    {i}. [{r['score']:.4f}] {r['text']}")

# Basic RAG pattern: use top result as context
print("\n--- Basic RAG Pattern ---")
rag_query = "What is the best tool for version control?"
print(f"  Query: '{rag_query}'")

search_results = search(rag_query, top_k=2)
context = "\n".join([r['text'] for r in search_results])

response = ollama.generate(
    model='gemma3',
    prompt=f"Based on the following context, answer the question.\n\nContext:\n{context}\n\nQuestion: {rag_query}\n\nAnswer:",
    options={'temperature': 0.3, 'num_predict': 100},
    stream=False,
)
print(f"  Context used: {search_results[0]['text']}")
print(f"  Answer: {response['response']}")

print("\n" + "=" * 60)
print("SEARCH ENGINE COMPLETE")
print("=" * 60)

Your semantic search engine finds relevant documents by meaning, not just keywords. Next up, you will explore thinking mode to expose the model's internal reasoning process before it generates an answer.

Unlock Thinking Mode and Chain of Thought

Your embedding search engine can find semantically similar documents and feed them as context to a model. That covers the retrieval side of working with LLMs. Now you will explore what happens inside the model's reasoning process itself.

Most LLM calls return a polished final answer. Thinking mode changes that by exposing the model's internal chain of thought before it settles on a response. This is invaluable for debugging incorrect answers, understanding how a model approaches complex logic, and building trust in automated reasoning.

In this step, get ready to:

Compare model responses with and without thinking mode enabled.
Use thinking mode with both ollama.chat() and ollama.generate().
Stream thinking tokens in real time, separating reasoning from the final answer.

Enable thinking mode to expose the model's reasoning

The think parameter tells the model to show its work before delivering a final answer. You will test this on a classic trick question to see how thinking changes the model's behavior.

Open thinking_mode.py in VS Code.
Add the import, header, and basic thinking comparison by pasting this code:

import ollama

print("=" * 60)
print("OLLAMA THINKING MODE")
print("=" * 60)

# Basic thinking mode comparison
print("\n--- Thinking vs Non-Thinking Comparison ---")
problem = "A farmer has 17 sheep. All but 9 run away. How many sheep does the farmer have left?"

print(f"  Problem: {problem}")

# Without thinking
print("\n  Without thinking mode:")
response = ollama.chat(
    model='gemma3',
    messages=[{'role': 'user', 'content': problem}],
)
print(f"    Answer: {response.message.content}")

# With thinking
print("\n  With thinking mode:")
response = ollama.chat(
    model='gemma3',
    messages=[{'role': 'user', 'content': problem}],
    think=True,
)
if response.message.thinking:
    print(f"    Thinking: {response.message.thinking[:300]}...")
print(f"    Answer: {response.message.content}")

What does this code do?

This sends the same tricky question to the model twice. The sheep problem is designed to catch hasty reasoning. "All but 9 run away" means 9 remain, but models sometimes subtract 9 from 17.

The first call uses no think parameter, so you only see the final answer.
The second call passes think=True, which exposes the model's internal reasoning in response.message.thinking.
You print the first 300 characters of the thinking to keep the output manageable.

Next, add a more complex reasoning problem below the basic comparison. This tests whether thinking mode helps with multi-step logic puzzles.

Add the complex reasoning section below the basic comparison code:

# Complex reasoning with thinking
print("\n--- Complex Reasoning ---")
complex_problem = """
If it takes 5 machines 5 minutes to make 5 widgets, how long would it take
100 machines to make 100 widgets? Explain your reasoning step by step.
"""

print(f"  Problem: {complex_problem.strip()}")
response = ollama.chat(
    model='gemma3',
    messages=[{'role': 'user', 'content': complex_problem}],
    think=True,
)
if response.message.thinking:
    print(f"\n  Model's internal reasoning:")
    print(f"  {'=' * 40}")
    # Print first 500 chars of thinking
    thinking_text = response.message.thinking
    if len(thinking_text) > 500:
        print(f"  {thinking_text[:500]}...")
    else:
        print(f"  {thinking_text}")
    print(f"  {'=' * 40}")
print(f"\n  Final answer:\n  {response.message.content}")

What does this code do?

The widget problem is another classic reasoning trap. The answer is 5 minutes (each machine makes one widget in 5 minutes), but intuition suggests 100 minutes.

The code prints up to 500 characters of the model's internal reasoning so you can see how it works through the problem.
The response.message.thinking field contains the full chain-of-thought reasoning the model performed before settling on its answer.

Save thinking_mode.py.

Not seeing any thinking output?

Make sure you are using a model that supports thinking mode. The gemma3 model supports think=True.
Check that response.message.thinking is not None. Some shorter problems may not trigger extended thinking.
Verify Ollama is still running at http://localhost:11434.

Why is my thinking mode not returning any reasoning content?

Stream thinking responses and use generate with thinking

Thinking mode also works with ollama.generate(). The access pattern is slightly different because generate() returns a dictionary instead of an object.

Add the thinking-with-generate section below the complex reasoning code:

# Thinking with generate
print("\n--- Thinking with generate() ---")
response = ollama.generate(
    model='gemma3',
    prompt="What is 23 * 47? Think through this carefully.",
    think=True,
    stream=False,
)
if response.get('thinking'):
    print(f"  Thinking: {response['thinking'][:200]}...")
print(f"  Response: {response['response']}")

What is different about generate with thinking?

With ollama.generate(), you access thinking via response.get('thinking') (dictionary key) instead of response.message.thinking (object attribute). The final output lives in response['response'] instead of response.message.content.

Now for the most powerful combination. Streaming with thinking mode lets you watch the model reason in real time, then see the final answer appear token by token.

Add the streaming-with-thinking section at the bottom of the file:

# Streaming with thinking mode
print("\n--- Streaming with Thinking ---")
print("  Problem: What day comes two days after the day before yesterday if today is Wednesday?")
print()
print("  [Thinking] ", end="")

thinking_content = ""
answer_content = ""
in_thinking = True

stream = ollama.chat(
    model='gemma3',
    messages=[{
        'role': 'user',
        'content': 'What day comes two days after the day before yesterday if today is Wednesday?'
    }],
    think=True,
    stream=True,
)

for chunk in stream:
    if chunk.message.thinking:
        thinking_content += chunk.message.thinking
        print(chunk.message.thinking, end='', flush=True)
    if chunk.message.content:
        if in_thinking:
            print(f"\n\n  [Answer] ", end="")
            in_thinking = False
        answer_content += chunk.message.content
        print(chunk.message.content, end='', flush=True)

print("\n")
print("\n" + "=" * 60)
print("THINKING MODE COMPLETE")
print("=" * 60)

What does the streaming loop do?

Each chunk from the stream may contain thinking tokens, content tokens, or both.
The in_thinking boolean tracks whether the model is still in its reasoning phase. When the first content token arrives, it switches to printing the answer.
The thinking_content and answer_content strings accumulate the full text, which you could use for logging or further processing.

What about thinking levels?

The think parameter also accepts string values like 'low', 'medium', and 'high' for fine-grained control over reasoning intensity. These levels are supported by specific models. With gemma3, the boolean True enables the default thinking depth.

Save thinking_mode.py.
Run the script by executing this command in your terminal:

python thinking_mode.py

You should see the sheep problem answered with and without thinking, the widget problem's chain-of-thought reasoning, a generate() thinking result, and finally the streaming section showing thinking tokens followed by the answer.

Script taking a long time?

Thinking mode makes multiple calls to the model, and each one includes additional reasoning tokens. The script may take 2-5 minutes depending on your hardware.
If a single call hangs for more than a few minutes, check that Ollama is still responsive by visiting http://localhost:11434 in your browser.
If you see a ConnectionError, restart Ollama and try again.

My thinking mode script is hanging or erroring out. How do I debug this?

✔️ Awesome, I've got everything!

Great. Double check you have saved thinking_mode.py and that the script ran successfully.

ⓧ I'd like to double check the full code

import ollama

print("=" * 60)
print("OLLAMA THINKING MODE")
print("=" * 60)

# Basic thinking mode comparison
print("\n--- Thinking vs Non-Thinking Comparison ---")
problem = "A farmer has 17 sheep. All but 9 run away. How many sheep does the farmer have left?"

print(f"  Problem: {problem}")

# Without thinking
print("\n  Without thinking mode:")
response = ollama.chat(
    model='gemma3',
    messages=[{'role': 'user', 'content': problem}],
)
print(f"    Answer: {response.message.content}")

# With thinking
print("\n  With thinking mode:")
response = ollama.chat(
    model='gemma3',
    messages=[{'role': 'user', 'content': problem}],
    think=True,
)
if response.message.thinking:
    print(f"    Thinking: {response.message.thinking[:300]}...")
print(f"    Answer: {response.message.content}")

# Complex reasoning with thinking
print("\n--- Complex Reasoning ---")
complex_problem = """
If it takes 5 machines 5 minutes to make 5 widgets, how long would it take
100 machines to make 100 widgets? Explain your reasoning step by step.
"""

print(f"  Problem: {complex_problem.strip()}")
response = ollama.chat(
    model='gemma3',
    messages=[{'role': 'user', 'content': complex_problem}],
    think=True,
)
if response.message.thinking:
    print(f"\n  Model's internal reasoning:")
    print(f"  {'=' * 40}")
    # Print first 500 chars of thinking
    thinking_text = response.message.thinking
    if len(thinking_text) > 500:
        print(f"  {thinking_text[:500]}...")
    else:
        print(f"  {thinking_text}")
    print(f"  {'=' * 40}")
print(f"\n  Final answer:\n  {response.message.content}")

# Thinking with generate
print("\n--- Thinking with generate() ---")
response = ollama.generate(
    model='gemma3',
    prompt="What is 23 * 47? Think through this carefully.",
    think=True,
    stream=False,
)
if response.get('thinking'):
    print(f"  Thinking: {response['thinking'][:200]}...")
print(f"  Response: {response['response']}")

# Streaming with thinking mode
print("\n--- Streaming with Thinking ---")
print("  Problem: What day comes two days after the day before yesterday if today is Wednesday?")
print()
print("  [Thinking] ", end="")

thinking_content = ""
answer_content = ""
in_thinking = True

stream = ollama.chat(
    model='gemma3',
    messages=[{
        'role': 'user',
        'content': 'What day comes two days after the day before yesterday if today is Wednesday?'
    }],
    think=True,
    stream=True,
)

for chunk in stream:
    if chunk.message.thinking:
        thinking_content += chunk.message.thinking
        print(chunk.message.thinking, end='', flush=True)
    if chunk.message.content:
        if in_thinking:
            print(f"\n\n  [Answer] ", end="")
            in_thinking = False
        answer_content += chunk.message.content
        print(chunk.message.content, end='', flush=True)

print("\n")
print("\n" + "=" * 60)
print("THINKING MODE COMPLETE")
print("=" * 60)

Secret mission

Build an Autonomous Agent Loop

Ready for the real deal? Combine streaming, thinking mode, and multi-tool calling into a fully autonomous agent that solves multi-step problems without manual intervention. Watch it think, decide, and act in real-time.

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally, so there are no ongoing costs.

Resources you used:

The ollama-toolkit project folder on your Desktop.
Pulled Ollama models (gemma3 and all-minilm).
Python packages (ollama and pydantic).

Keep everything running

No action needed. Choose this if you are still actively building or plan to extend the toolkit with the Secret Mission.

Your project files remain in the ollama-toolkit folder on your Desktop.
Ollama models stay cached on disk and load quickly on next use.
The ollama and pydantic Python packages remain installed.

Pause - I'll come back to this later

Free up memory by stopping the Ollama background process. Your files and models stay on disk.

Stop the Ollama server if it is running in the background. Close the Ollama application or quit it from the system tray.
Your project files and pulled models remain on disk. When you return, start Ollama again and everything will work as before.

Delete - I don't want to use this again

Remove all project resources completely. Follow each step below.

Delete the project folder by running this command (macOS/Linux):

rm -rf ~/Desktop/ollama-toolkit

On Windows, run this instead:

rmdir /s /q "$HOME\Desktop\ollama-toolkit"

Remove the pulled Ollama models to free disk space:

ollama rm gemma3
ollama rm all-minilm

Uninstall the Python packages:

pip uninstall ollama pydantic -y

What about Ollama itself?

If you plan to use Ollama for other projects, keep it installed. The models you removed were the main disk-space consumers.

That's a Wrap!

Nice work! You have built a comprehensive Ollama Python toolkit that demonstrates every major capability of local LLM interaction.

You've learned how to:

Controlled generation parameters (temperature, top_k, top_p, seed, stop) and built a vision analyzer that extracts structured data from images using Pydantic schemas.
Built a tool-calling agent that auto-invokes Python functions based on model decisions, with automatic schema generation from type annotations and docstrings.
Created a semantic search engine using embeddings and cosine similarity to implement a basic RAG pattern, and exposed model reasoning with thinking mode.
Secret Mission: Built an autonomous multi-tool agent loop that combines streaming, thinking mode, and multi-step tool calling to solve problems without manual intervention.

Ready to quiz yourself?