Build a Local AI Triage Pipeline

Build a local AI pipeline that validates and evaluates support ticket triage.

Introduction

30 Second Summary

The same support ticket can seem routine to one person and urgent to another. Those inconsistent decisions make reliable automation difficult.

In this project, you will build a command-line support ticket triage pipeline powered by Ollama that returns validated JSON. A labelled evaluation set measures the local model's decisions.

What You'll Build

After you paste a support ticket into the terminal, a validated triage record appears with a category, priority, summary, plus an escalation decision.

By the end of this project, you'll have:

  • A local triage workflow that turns any support ticket into four validated fields in the terminal.
  • A clear baseline comparison showing how unconstrained prose differs from schema-constrained JSON.
  • A repeatable evaluation report that marks every case as PASS, FAIL, or ERROR. Its field-level score lets you compare prompt revisions using the same tickets.
  • Secret Mission: Extend the output contract with a dedicated security category. Add a regression case that proves the original evaluation still runs.

Are there any prerequisites?

You need basic Python familiarity plus a compatible Windows computer with Python and Visual Studio Code installed.

The Ollama setup requires at least 4 GB of free storage for the application plus about 815 MB for the local model.

Before We Start

A triage pipeline only becomes useful when its purpose is clear. This is your moment to define how consistent records help support staff route tickets reliably.

Set Up Ollama and the Local Model

The triage pipeline needs a real model before it can classify a ticket. Running that model through local inference keeps ticket text on your computer.

Ollama gives Python a local HTTP API. This service avoids API keys and usage charges.

You will prepare the Windows environment first. You will finish by proving that the gemma3:1b model can respond from your computer.

In this step, get ready to:
  • Confirm the computer meets the project requirements.
  • Install Ollama as the local inference service.
  • Open the project workspace with a working gemma3:1b model.
Check Python and system readiness

PowerShell can report the installed Python versions. It can also show the Windows release details and available storage.

  • Press the Windows key to open the search bar.
  • Type PowerShell into the search bar.
  • Press Enter to open PowerShell.
  • Check Python and system readiness by running these commands:
py --list
Get-ComputerInfo -Property "*version"
Get-PSDrive -PSProvider FileSystem

What Do These Checks Show?

  • The py --list command displays the Python installations available through the Windows launcher.
  • The Get-ComputerInfo -Property "*version" command displays version information for Windows.
  • The Get-PSDrive -PSProvider FileSystem command lists file system drives with used and free space.

The Windows version output is dense at first glance.

  • Confirm the Python list contains at least one Python 3 installation.
  • Confirm the Windows information represents Windows 10 22H2 or newer.
  • Confirm the installation drive has at least 4 GB free for Ollama.
  • Confirm another 815 MB is available for the model.

Your computer has the runtime and storage needed for this local pipeline. That clears the first environment check.

Missing Python or Storage?

A missing Python 3 entry can mean that PowerShell was open before Python finished installing. Close PowerShell before opening a fresh window.

If the installation drive lacks enough free space, clear space before continuing. The Ollama installation and model download both need room.

Help me understand my readiness check.

Install Ollama and open the workspace

Ollama runs as a native Windows application in the background. Its local service makes the model available at http://localhost:11434.

Why Local Inference?

Local inference runs the model on your computer. Support ticket text stays inside the local pipeline.

Running models on your own hardware has no usage charges. The local API still gives your Python application a real model service to call.

The installer can spend a few minutes downloading files, so a quiet PowerShell window does not mean it is stuck. Windows may also ask for permission to continue.

  • Install Ollama from PowerShell by running this command:
irm https://ollama.com/install.ps1 | iex

What Does This Command Do?

  • The irm portion downloads Ollama's official Windows installation script.
  • The iex portion runs that script in the current PowerShell session.
  • The installer adds the Ollama command and starts its background service.
  • Check the installed Ollama version by running:
ollama -v

What Does This Check Prove?

The version command confirms that PowerShell can reach the installed Ollama command. It also gives you the release number needed for the model compatibility check.

  • Match your version result to the tab below.

✔️ I see version 0.35.1 or higher

Your local service is on the expected release. The gemma3:1b model requires Ollama 0.6 or later, so this version supports it.

ⓧ I see an older version

An older Ollama release can lack support required by the model. Update it before downloading gemma3:1b.

  • Update Ollama by rerunning the official Windows installer:
irm https://ollama.com/install.ps1 | iex

What Does This Command Do?

The command downloads the current Windows installer script. It applies the current Ollama release to your existing installation.

  • Check the updated version by running:
ollama -v

What Should You See?

The command should now report Ollama v0.35.1 or a later release.

ⓧ Command not found

A newly installed command may need a fresh PowerShell session. Reopening PowerShell also gives the installer another clean attempt.

  • Close the current PowerShell window.
  • Press the Windows key to open the search bar.
  • Type PowerShell into the search bar.
  • Press Enter to open a fresh PowerShell window.
  • Install Ollama again by running:
irm https://ollama.com/install.ps1 | iex

What Does This Command Do?

The command reruns the official Windows installer. A completed installation makes the Ollama command available to fresh terminal sessions.

  • Confirm PowerShell can reach Ollama by running:
ollama -v

What Should You See?

You should see an Ollama version number. The expected release is v0.35.1 or later.

A dedicated folder gives every script and evaluation file one predictable home. You will create that folder on your Desktop before opening it with Visual Studio Code.

  • Create the ai-ticket-triage workspace on your Desktop by running these commands:
Set-Location -Path $HOME\Desktop
New-Item -ItemType Directory -Path ai-ticket-triage
Set-Location -Path ai-ticket-triage -PassThru
code .

What Do These Commands Do?

  • The first Set-Location command moves PowerShell to your Desktop.
  • The New-Item command creates the ai-ticket-triage folder.
  • The second Set-Location command enters that folder and prints its path.
  • The code . command opens the current folder as a VS Code workspace.

PowerShell should print a path ending in Desktop\ai-ticket-triage.

VS Code should show AI-TICKET-TRIAGE as the open workspace in the left sidebar.

VS Code Did Not Open?

The code command may be unavailable if VS Code was installed without its command-line path option. You can open the folder through the application instead.

  • Press the Windows key to open the search bar.
  • Type Visual Studio Code into the search bar.
  • Press Enter to open VS Code.
  • Click File in the top menu.
  • Select Open Folder....
  • Choose the ai-ticket-triage folder from your Desktop.

Help me open my workspace.

Download and test the local model

The Ollama service is running, but the project still needs a model. The gemma3:1b model is small enough for this local classification pipeline.

The first launch downloads an 815 MB model, so expect this command to take several minutes. Download progress in PowerShell confirms that the process is still moving.

  • Download and start the local model by running this command in PowerShell:
ollama run gemma3:1b

What Does This Command Do?

  • The ollama run command downloads a missing model before starting it.
  • The gemma3:1b value selects the exact local model used throughout this project.
  • The running model opens an interactive prompt in the same PowerShell window.

When the download completes, you should see an interactive model prompt in PowerShell.

Before you send the test prompt, what kind of reply do you expect from a direct readiness instruction?

  • Send the readiness test by entering:
Reply with READY

What Does This Test Prove?

This prompt asks for a simple response with no project logic involved. Any model reply proves that the download completed and local inference is working.

You should see a model response below your prompt. Its exact wording can vary, but it should respond to the readiness instruction.

  • Press Ctrl+C to return to PowerShell.

You should see the PowerShell prompt return inside the ai-ticket-triage folder.

That is the local inference stack working from end to end. Ollama can now serve model requests through http://localhost:11434.

Your local model is installed and responding. Next, you will connect a Python script to the Ollama API and see the first ticket classification.

Build a Baseline Triage Script

Your local Ollama service has already answered a direct prompt. The next job is to make your Python project send that prompt through code.

A first working call gives your pipeline a real result to inspect. Its unconstrained prose exposes why model text is not yet a dependable software interface.

In this step, get ready to:
  • Create triage.py with the local model settings.
  • Send a support ticket to Ollama through its local HTTP API.
  • Run the baseline to inspect its unvalidated prose response.
Create the Ollama client

The client turns a Python dictionary into an HTTP request for the local model. It also extracts the model's text from the response.

  • In the Visual Studio Code Explorer sidebar, click the new-file icon beside the AI-TICKET-TRIAGE workspace name.
  • Name the file by typing triage.py.

You should see a blank triage.py editor tab. The file should also appear beneath the workspace name.

  • Add the imports plus the local model settings by pasting this code into triage.py:
import argparse
import json
import urllib.request

API_URL = "http://localhost:11434/api/chat"
MODEL = "gemma3:1b"

What Does This Code Set Up?

  • The argparse module prepares the script to receive a ticket from PowerShell.
  • The json module converts data to and from JSON.
  • The urllib.request module sends the request without an extra package.
  • The API_URL value points to Ollama's local chat endpoint.
  • The MODEL value selects gemma3:1b.
  • Save triage.py.
  • Confirm triage.py remains visible beneath the workspace name.

Cannot Find triage.py?

Check that the file sits directly beneath the AI-TICKET-TRIAGE workspace name. Confirm its name ends with .py.

Ask for help with the file location.

  • Add the request function below the MODEL line by pasting this code:
def call_ollama(messages):
    payload = {
        "model": MODEL,
        "messages": messages,
        "stream": False
    }

    request = urllib.request.Request(
        API_URL,
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST"
    )

    try:
        with urllib.request.urlopen(request, timeout=120) as response:
            body = json.loads(response.read().decode("utf-8"))
    except OSError as error:
        raise RuntimeError(
            "Could not reach Ollama at {}: {}".format(API_URL, error)
        ) from error

    return body["message"]["content"]

How Does the Request Work?

  • The payload carries the model name plus the chat messages.
  • The stream value asks Ollama to return one complete response.
  • The Request converts the payload into a POST request.
  • The try block waits up to 120 seconds for local inference.
  • The final line returns the assistant text stored in message.content.
  • Save triage.py.
  • Inspect the call_ollama block for red underlines.

You should see no red error markers around the function. Its nested lines should remain indented beneath call_ollama.

Seeing Red Underlines?

Check the indentation inside payload plus try. Match every opening bracket with its closing bracket.

Ask for help checking the function.

Add the broad triage prompt

The first prompt gives the model a broad classification task. Its only formatting guidance asks for two explanatory sentences.

  • Add triage_baseline below call_ollama by pasting this code:
def triage_baseline(ticket):
    prompt = (
        "Classify this support ticket and explain your decision in two sentences.\n\n"
        "Ticket: " + ticket
    )
    return call_ollama([{"role": "user", "content": prompt}])

What Does the Baseline Do?

  • The ticket parameter holds the support request entered in PowerShell.
  • The prompt asks for a classification plus a short explanation.
  • The final line sends one user message through call_ollama.
  • Save triage.py.
  • Inspect the triage_baseline block for red underlines.

You should see the prompt enclosed in parentheses. The function should finish with a call to call_ollama.

Is the Prompt Underlined?

Check that both string lines remain inside the same pair of parentheses. Confirm the second string begins with Ticket: .

Ask for help checking the prompt.

Run the baseline from PowerShell

The script still needs a command-line entry point. This final section reads the ticket argument before printing a labelled response.

  • Add main below triage_baseline by pasting this code:
def main():
    parser = argparse.ArgumentParser(
        description="Triage support tickets with a local Ollama model."
    )
    parser.add_argument("ticket", help="support ticket text")
    parser.add_argument(
        "--baseline",
        action="store_true",
        help="show the unconstrained baseline response"
    )
    args = parser.parse_args()

    try:
        print("BASELINE OUTPUT (unvalidated):")
        print(triage_baseline(args.ticket))
    except RuntimeError as error:
        parser.exit(1, "Error: {}\n".format(error))


if __name__ == "__main__":
    main()

How Does the Command-Line Entry Point Work?

  • The parser collects the support ticket from the command.
  • The --baseline flag identifies this run as the unconstrained comparison.
  • The first print statement labels the output as unvalidated.
  • The second print statement displays the model response.
  • The final condition runs main when PowerShell executes the file.
  • Save triage.py.
  • Compare your complete file with the reference below.

✔️ Awesome, I've got everything!

Your baseline script is complete. Confirm triage.py is saved before running it.

ⓧ I'd like to double check the full code

Compare your saved triage.py file with this complete reference.

import argparse
import json
import urllib.request

API_URL = "http://localhost:11434/api/chat"
MODEL = "gemma3:1b"


def call_ollama(messages):
    payload = {
        "model": MODEL,
        "messages": messages,
        "stream": False
    }

    request = urllib.request.Request(
        API_URL,
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST"
    )

    try:
        with urllib.request.urlopen(request, timeout=120) as response:
            body = json.loads(response.read().decode("utf-8"))
    except OSError as error:
        raise RuntimeError(
            "Could not reach Ollama at {}: {}".format(API_URL, error)
        ) from error

    return body["message"]["content"]


def triage_baseline(ticket):
    prompt = (
        "Classify this support ticket and explain your decision in two sentences.\n\n"
        "Ticket: " + ticket
    )
    return call_ollama([{"role": "user", "content": prompt}])


def main():
    parser = argparse.ArgumentParser(
        description="Triage support tickets with a local Ollama model."
    )
    parser.add_argument("ticket", help="support ticket text")
    parser.add_argument(
        "--baseline",
        action="store_true",
        help="show the unconstrained baseline response"
    )
    args = parser.parse_args()

    try:
        print("BASELINE OUTPUT (unvalidated):")
        print(triage_baseline(args.ticket))
    except RuntimeError as error:
        parser.exit(1, "Error: {}\n".format(error))


if __name__ == "__main__":
    main()

How to Compare the File

Check the order of the imports plus constants first. Then compare each function name with its indentation.

Before you run the script, what kind of response do you expect from such a broad classification prompt?

  • Switch back to PowerShell in the project folder.
  • Run the baseline with the sample billing ticket by entering this command:
python triage.py "I was charged twice for the same subscription renewal." --baseline

What Does This Command Test?

  • The command executes triage.py with one support ticket.
  • The --baseline flag marks this as the unconstrained comparison run.
  • The script sends the ticket to the local model before printing its response.

You should see BASELINE OUTPUT (unvalidated): followed by a prose response. The exact wording can vary between runs.

The model answered in free-form prose. Another program has no fixed fields to check.

Did the Baseline Fail to Respond?

Confirm Ollama is still running in the background. Check that your PowerShell prompt is inside the ai-ticket-triage folder.

Compare the saved API_URL plus MODEL values with the full-file reference.

Ask for help diagnosing the failed request.

That is your first end-to-end AI pipeline result. Your Python script now reaches the local model from PowerShell.

Your baseline is working. Next, you'll replace free-form prose with a validated JSON contract.

Enforce a Structured Output Contract

Your baseline call to Ollama proved that the local model can triage a ticket. Its prose can change shape between requests.

A dependable software interface needs a fixed contract. In this step, you will constrain the response with a JSON Schema and reject any result that breaks the contract.

In this step, get ready to:
  • Define the four fields in the triage contract.
  • Send the contract through the local model request.
  • Parse each response through Python validation.
Define the output contract

A schema describes the exact shape that a model response must follow. Your contract requires four fields for every ticket.

  • In triage.py, place your cursor directly below MODEL = "gemma3:1b".
  • Define the allowed values plus the schema by pasting this code:
CATEGORIES = {"billing", "account", "technical", "general"}
PRIORITIES = {"low", "medium", "high", "critical"}

TRIAGE_SCHEMA = {
    "type": "object",
    "properties": {
        "category": {"type": "string"},
        "priority": {"type": "string"},
        "summary": {"type": "string"},
        "escalate": {"type": "boolean"}
    },
    "required": ["category", "priority", "summary", "escalate"]
}

SYSTEM_PROMPT = """You are a support ticket triage engine.
Return values that follow the supplied JSON schema.
Use exactly one category and one priority value from the supplied rules.
Write summary as one short sentence.
"""

What does this contract define?

  • The CATEGORIES set lists the category values that Python accepts.
  • The PRIORITIES set lists the accepted urgency levels.
  • The TRIAGE_SCHEMA object requires category, priority, summary, plus escalate.
  • The SYSTEM_PROMPT tells the model to follow the supplied schema.
  • Save triage.py.
  • Confirm the script still runs by executing the baseline command in PowerShell:
python triage.py "I was charged twice for the same subscription renewal." --baseline

What does this check prove?

The command still exercises the existing baseline path. You should see the labelled unvalidated prose response from earlier.

That response proves the new contract definitions did not break the script.

Seeing a Python syntax error?

  • Check that CATEGORIES begins directly below the MODEL line with no indentation.
  • Check that the closing braces in TRIAGE_SCHEMA match the code above.
  • Confirm that SYSTEM_PROMPT opens with three double quotes.

Help me find the syntax problem in my contract definitions.

The contract now exists inside Python. The request helper must send that contract through Ollama's format field.

  • In triage.py, select the complete call_ollama function.
  • Replace the selected function by pasting this code:
def call_ollama(messages, output_format=None):
    payload = {
        "model": MODEL,
        "messages": messages,
        "stream": False,
        "options": {"temperature": 0}
    }
    if output_format is not None:
        payload["format"] = output_format

    request = urllib.request.Request(
        API_URL,
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST"
    )

    try:
        with urllib.request.urlopen(request, timeout=120) as response:
            body = json.loads(response.read().decode("utf-8"))
    except OSError as error:
        raise RuntimeError(
            "Could not reach Ollama at {}: {}".format(API_URL, error)
        ) from error

    return body["message"]["content"]

How does the request change?

  • The optional output_format parameter lets each caller decide whether it needs structured output.
  • The format field carries the schema when a caller supplies one.
  • The temperature value of 0 makes local completions more deterministic.
  • The existing baseline caller supplies no schema. Its unconstrained comparison remains available.
  • Save triage.py.
  • Verify that the optional schema support preserves the baseline by running:
python triage.py "I was charged twice for the same subscription renewal." --baseline

What should remain unchanged?

You should still see a labelled prose response. The baseline does not supply TRIAGE_SCHEMA to call_ollama.

Baseline request stopped working?

  • Confirm that output_format=None remains in the function definition.
  • Check that the format field sits inside the conditional block.
  • Confirm that Ollama is still running in the background.

Help me debug the updated Ollama request helper.

Parse and validate the model response

Schema-constrained output gives the model a target shape. Python validation forms the second boundary by checking every returned value before another application can use it.

  • In triage.py, place your cursor below the triage_baseline function.
  • Add the validation function by pasting this code:
def validate_result(result):
    required_fields = {"category", "priority", "summary", "escalate"}

    if not isinstance(result, dict):
        raise ValueError("The model result is not a JSON object.")
    if set(result.keys()) != required_fields:
        raise ValueError("The result must contain exactly: {}".format(
            ", ".join(sorted(required_fields))
        ))
    if result["category"] not in CATEGORIES:
        raise ValueError("Unknown category: {}".format(result["category"]))
    if result["priority"] not in PRIORITIES:
        raise ValueError("Unknown priority: {}".format(result["priority"]))
    if not isinstance(result["summary"], str) or not result["summary"].strip():
        raise ValueError("Summary must be a non-empty string.")
    if type(result["escalate"]) is not bool:
        raise ValueError("Escalate must be true or false.")

What does validation protect?

  • The first check requires a Python dictionary.
  • The field check rejects missing keys plus unexpected keys.
  • The value checks reject unknown categories plus unknown priorities.
  • The final checks require a non-empty summary plus a Boolean escalation decision.
  • Save triage.py.
  • Check the new validation function for syntax problems by running the baseline command:
python triage.py "I was charged twice for the same subscription renewal." --baseline

What does this run confirm?

You should see the labelled baseline response again. This confirms that Python can load the new validate_result function without a syntax error.

Did the script stop loading?

  • Confirm that validate_result starts at the left edge of the file.
  • Check the indentation beneath every if statement.
  • Compare each closing parenthesis with the function above.

Help me fix the validate_result function.

The validator needs a structured triage function to feed it. That function combines the system instruction with the ticket before parsing the returned content.

  • Place your cursor directly below the validate_result function.
  • Add the structured triage function by pasting this code:
def triage_ticket(ticket):
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "Ticket:\n" + ticket}
    ]
    content = call_ollama(messages, TRIAGE_SCHEMA)
    result = json.loads(content)
    validate_result(result)
    return result

How does structured triage work?

  • The messages list separates the reusable system instruction from the ticket.
  • The call passes TRIAGE_SCHEMA as the requested output format.
  • The json.loads call converts the model text into Python data.
  • The result reaches the caller only after validate_result accepts it.
  • Save triage.py.
  • Confirm that the file remains runnable by executing the baseline command:
python triage.py "I was charged twice for the same subscription renewal." --baseline

What does this checkpoint show?

You should see the familiar baseline output. Python has now loaded the complete structured path without changing the existing comparison.

Did this checkpoint fail?

  • Confirm that triage_ticket begins below the complete validate_result function.
  • Check that "Ticket:\n" contains the backslash character before n.
  • Confirm that TRIAGE_SCHEMA is the second argument passed to call_ollama.

Help me debug the triage_ticket function.

Route normal tickets through the contract

The command-line entry point currently sends every ticket through triage_baseline. You will preserve that route behind --baseline while making structured triage the normal path.

  • In triage.py, locate the current main function.
  • Use this reference to confirm that you found the complete function:
def main():
    parser = argparse.ArgumentParser(
        description="Triage support tickets with a local Ollama model."
    )
    parser.add_argument("ticket", help="support ticket text")
    parser.add_argument(
        "--baseline",
        action="store_true",
        help="show the unconstrained baseline response"
    )
    args = parser.parse_args()

    try:
        print("BASELINE OUTPUT (unvalidated):")
        print(triage_baseline(args.ticket))
    except RuntimeError as error:
        parser.exit(1, "Error: {}\n".format(error))

What does the current function do?

The current function always prints the baseline label. It also catches connection failures from the local model request.

  • Replace the complete main function with this updated version:
def main():
    parser = argparse.ArgumentParser(
        description="Triage support tickets with a local Ollama model."
    )
    parser.add_argument("ticket", help="support ticket text")
    parser.add_argument(
        "--baseline",
        action="store_true",
        help="show the unconstrained baseline response"
    )
    args = parser.parse_args()

    try:
        if args.baseline:
            print("BASELINE OUTPUT (unvalidated):")
            print(triage_baseline(args.ticket))
        else:
            print(json.dumps(triage_ticket(args.ticket), indent=2))
    except (RuntimeError, ValueError) as error:
        parser.exit(1, "Error: {}\n".format(error))

How does the command choose a path?

  • The args.baseline branch preserves the unconstrained comparison.
  • The normal branch sends the ticket through triage_ticket.
  • The json.dumps call prints the validated result with readable indentation.
  • The expanded exception handler reports validation failures through the command-line parser.
  • Save triage.py.

Before you run the final check, which output shape do you expect now?

  • Run the same billing ticket without the baseline flag:
python triage.py "I was charged twice for the same subscription renewal."

What should you see?

You should see a JSON object with exactly four fields. Those fields are category, priority, summary, plus escalate.

The values can vary with local inference. The output shape must remain fixed.

Structured ticket did not print?

  • Confirm that Ollama is still running in the background.
  • Check that the request passes TRIAGE_SCHEMA into call_ollama.
  • Compare any validation message with the accepted values in CATEGORIES plus PRIORITIES.

Help me diagnose my structured triage result.

✔️ Awesome, I've got everything!

Your saved script now preserves the baseline while making validated structured output the normal path.

ⓧ I'd like to double check the full code

  • Compare your saved triage.py with this complete version:
import argparse
import json
import urllib.request

API_URL = "http://localhost:11434/api/chat"
MODEL = "gemma3:1b"
CATEGORIES = {"billing", "account", "technical", "general"}
PRIORITIES = {"low", "medium", "high", "critical"}

TRIAGE_SCHEMA = {
    "type": "object",
    "properties": {
        "category": {"type": "string"},
        "priority": {"type": "string"},
        "summary": {"type": "string"},
        "escalate": {"type": "boolean"}
    },
    "required": ["category", "priority", "summary", "escalate"]
}

SYSTEM_PROMPT = """You are a support ticket triage engine.
Return values that follow the supplied JSON schema.
Use exactly one category and one priority value from the supplied rules.
Write summary as one short sentence.
"""


def call_ollama(messages, output_format=None):
    payload = {
        "model": MODEL,
        "messages": messages,
        "stream": False,
        "options": {"temperature": 0}
    }
    if output_format is not None:
        payload["format"] = output_format

    request = urllib.request.Request(
        API_URL,
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST"
    )

    try:
        with urllib.request.urlopen(request, timeout=120) as response:
            body = json.loads(response.read().decode("utf-8"))
    except OSError as error:
        raise RuntimeError(
            "Could not reach Ollama at {}: {}".format(API_URL, error)
        ) from error

    return body["message"]["content"]


def triage_baseline(ticket):
    prompt = (
        "Classify this support ticket and explain your decision in two sentences.\n\n"
        "Ticket: " + ticket
    )
    return call_ollama([{"role": "user", "content": prompt}])


def validate_result(result):
    required_fields = {"category", "priority", "summary", "escalate"}

    if not isinstance(result, dict):
        raise ValueError("The model result is not a JSON object.")
    if set(result.keys()) != required_fields:
        raise ValueError("The result must contain exactly: {}".format(
            ", ".join(sorted(required_fields))
        ))
    if result["category"] not in CATEGORIES:
        raise ValueError("Unknown category: {}".format(result["category"]))
    if result["priority"] not in PRIORITIES:
        raise ValueError("Unknown priority: {}".format(result["priority"]))
    if not isinstance(result["summary"], str) or not result["summary"].strip():
        raise ValueError("Summary must be a non-empty string.")
    if type(result["escalate"]) is not bool:
        raise ValueError("Escalate must be true or false.")


def triage_ticket(ticket):
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "Ticket:\n" + ticket}
    ]
    content = call_ollama(messages, TRIAGE_SCHEMA)
    result = json.loads(content)
    validate_result(result)
    return result


def main():
    parser = argparse.ArgumentParser(
        description="Triage support tickets with a local Ollama model."
    )
    parser.add_argument("ticket", help="support ticket text")
    parser.add_argument(
        "--baseline",
        action="store_true",
        help="show the unconstrained baseline response"
    )
    args = parser.parse_args()

    try:
        if args.baseline:
            print("BASELINE OUTPUT (unvalidated):")
            print(triage_baseline(args.ticket))
        else:
            print(json.dumps(triage_ticket(args.ticket), indent=2))
    except (RuntimeError, ValueError) as error:
        parser.exit(1, "Error: {}\n".format(error))


if __name__ == "__main__":
    main()

What should match?

Check the order of the contract definitions plus the four functions. Confirm that main contains both the baseline branch and the structured branch.

That is the contract in action. Next, you will run several labelled tickets through the same pipeline to measure where its decisions succeed or fail.

Measure the Pipeline with an Evaluation Set

Your pipeline now turns one support ticket into validated JSON. That proves the contract works for one example.

A single convincing result can hide inconsistent model behavior. A labelled evaluation set gives each prompt version the same six cases.

The scoring loop measures category, priority, and escalation. Every mismatch stays visible for investigation.

In this step, get ready to:
  • Create six labelled support ticket cases.
  • Add a repeatable field-level evaluation loop.
  • Run the suite to reveal passes, failures, and errors.
Create the labelled evaluation set

Each case pairs a support ticket with its expected triage values. Reusing these labels makes future prompt changes comparable.

Why exclude the summary?

The suite scores only fields with one objective answer. Several concise summaries could describe the same ticket correctly.

Category, priority, and escalation produce a fair field-level score.

  • In the Visual Studio Code file sidebar from earlier, select the ai-ticket-triage folder.
  • Use the new-file icon to create tickets.json inside ai-ticket-triage.

You should see tickets.json beside triage.py in the file sidebar.

  • Use the full-file reference in the second tab below to fill tickets.json with the six labelled cases.

✔️ I have the six cases

Your dataset is ready once tickets.json contains six ticket objects. Each object should include one expected category, priority, and escalation value.

ⓧ I'd like to double check the full code

Compare your tickets.json file with this complete reference:

[
  {
    "ticket": "I was charged twice for the same subscription renewal.",
    "expected": {
      "category": "billing",
      "priority": "medium",
      "escalate": false
    }
  },
  {
    "ticket": "I cannot sign in after resetting my password and I need access today.",
    "expected": {
      "category": "account",
      "priority": "high",
      "escalate": false
    }
  },
  {
    "ticket": "Every user in our office sees Service Unavailable and nobody can work.",
    "expected": {
      "category": "technical",
      "priority": "critical",
      "escalate": true
    }
  },
  {
    "ticket": "Where can I download last month's invoice?",
    "expected": {
      "category": "billing",
      "priority": "low",
      "escalate": false
    }
  },
  {
    "ticket": "A failed import deleted customer records and there is no backup.",
    "expected": {
      "category": "technical",
      "priority": "critical",
      "escalate": true
    }
  },
  {
    "ticket": "Can you change the contact email on our profile?",
    "expected": {
      "category": "account",
      "priority": "low",
      "escalate": false
    }
  }
]

How is each case structured?

  • The ticket value contains the text sent to the local model.
  • The expected object holds the three values used for scoring.
  • The six cases cover billing, account, and technical requests at different priority levels.
  • Save tickets.json by pressing Ctrl+S.
  • Switch back to PowerShell from earlier.
  • Validate the dataset by running this command:
python -m json.tool tickets.json

What should I see?

Python reads the file as JSON and prints the formatted dataset back to PowerShell. The command returns to the prompt without a parsing error when the structure is valid.

You should see all six ticket objects in the formatted output.

Seeing a JSON parsing error?

  • Compare the punctuation around the case named in the error.
  • Check that every property name uses double quotes.
  • Check that commas separate adjacent ticket objects.

I need help fixing my evaluation dataset.

Add the evaluation loop

The evaluator needs to read the dataset from disk before it can score each case. Python's Path class provides that file access.

  • Switch back to triage.py in the workspace from earlier.
  • Find these imports at the top of the file:
import argparse
import json
import urllib.request

What is missing here?

The existing imports support command-line arguments, JSON processing, and HTTP requests. The evaluation dataset also needs a path that Python can read.

  • Replace those imports with this updated group:
import argparse
import json
import urllib.request
from pathlib import Path

What does Path add?

The Path class represents tickets.json as a file path. Its read_text() method supplies the dataset text to the JSON parser.

  • Save triage.py by pressing Ctrl+S.
  • Check the updated file syntax by running this command in PowerShell:
python -c "compile(open('triage.py').read(), 'triage.py', 'exec')"

What should I see?

PowerShell returns to the prompt without printing a syntax error. This confirms the updated import is valid Python syntax.

The scoring loop processes every case independently. A failed case remains visible without preventing later cases from running.

  • In triage.py, find the blank line after the triage_ticket() function.
  • Add this complete run_evaluation() function before main() by pasting the code below:
def run_evaluation(path):
    cases = json.loads(path.read_text(encoding="utf-8"))
    fields = ("category", "priority", "escalate")
    correct = 0
    total = len(cases) * len(fields)

    for number, case in enumerate(cases, start=1):
        try:
            actual = triage_ticket(case["ticket"])
            expected = case["expected"]
            matched = [field for field in fields if actual[field] == expected[field]]
            correct += len(matched)
            status = "PASS" if len(matched) == len(fields) else "FAIL"
            print("{} {}: {}".format(status, number, case["ticket"]))
            if status == "FAIL":
                print("  expected: {}".format(expected))
                print("  actual:   {}".format(
                    {field: actual[field] for field in fields}
                ))
        except (KeyError, RuntimeError, ValueError) as error:
            print("ERROR {}: {}".format(number, error))

    score = correct / total if total else 0
    print("\nField accuracy: {}/{} ({:.0%})".format(correct, total, score))

How does the evaluator work?

  • The cases variable holds the six labelled objects loaded from tickets.json.
  • The fields tuple limits scoring to category, priority, and escalation.
  • Each loop calls triage_ticket() with one ticket.
  • The exception handler prints ERROR for one case while the remaining cases continue.
  • The final calculation reports correct field comparisons out of the total possible comparisons.
  • Save triage.py by pressing Ctrl+S.
  • Check the evaluator syntax by running the same command again:
python -c "compile(open('triage.py').read(), 'triage.py', 'exec')"

What does this check prove?

PowerShell returns to the prompt without displaying a syntax error when the new function is valid. The next substep connects that function to a command-line flag.

Seeing a syntax error?

  • Check that run_evaluation() begins at the left edge of the file.
  • Check that the lines inside each for, try, if, and except block use consistent indentation.
  • Check that the function sits above main().

Help me fix the syntax in my evaluation function.

Connect the evaluation mode

The evaluator now exists inside the script. The command-line parser needs an optional mode that runs the dataset without requiring a single ticket argument.

  • In triage.py, locate the existing main() function near the bottom of the file.
  • Select the complete function from def main(): through its parser.exit() line.
  • Replace the selection with this updated function:
def main():
    parser = argparse.ArgumentParser(
        description="Triage support tickets with a local Ollama model."
    )
    parser.add_argument("ticket", nargs="?", help="support ticket text")
    parser.add_argument(
        "--baseline",
        action="store_true",
        help="show the unconstrained baseline response"
    )
    parser.add_argument(
        "--eval",
        action="store_true",
        help="run the labelled evaluation set"
    )
    args = parser.parse_args()

    try:
        if args.eval:
            run_evaluation(Path("tickets.json"))
        elif args.ticket:
            if args.baseline:
                print("BASELINE OUTPUT (unvalidated):")
                print(triage_baseline(args.ticket))
            else:
                print(json.dumps(triage_ticket(args.ticket), indent=2))
        else:
            parser.error("provide a ticket or use --eval")
    except (KeyError, RuntimeError, ValueError) as error:
        parser.exit(1, "Error: {}\n".format(error))

How does the new mode work?

  • The optional ticket argument allows the script to run without ticket text when evaluation mode is selected.
  • The --eval flag sends tickets.json to run_evaluation().
  • The elif args.ticket branch preserves the baseline and structured single-ticket modes.
  • The expanded exception list reports malformed dataset entries without exposing a Python traceback.
  • Save triage.py by pressing Ctrl+S.
  • Use the second tab below to compare the complete script before running the suite.

✔️ Awesome, I've got everything!

Your script now has the dataset loader, evaluation loop, and --eval command-line mode. Double check that both project files are saved.

ⓧ I'd like to double check the full code

Compare your complete triage.py file with this reference:

import argparse
import json
import urllib.request
from pathlib import Path

API_URL = "http://localhost:11434/api/chat"
MODEL = "gemma3:1b"
CATEGORIES = {"billing", "account", "technical", "general"}
PRIORITIES = {"low", "medium", "high", "critical"}

TRIAGE_SCHEMA = {
    "type": "object",
    "properties": {
        "category": {"type": "string"},
        "priority": {"type": "string"},
        "summary": {"type": "string"},
        "escalate": {"type": "boolean"}
    },
    "required": ["category", "priority", "summary", "escalate"]
}

SYSTEM_PROMPT = """You are a support ticket triage engine.
Return values that follow the supplied JSON schema.
Use exactly one category and one priority value from the supplied rules.
Write summary as one short sentence.
"""


def call_ollama(messages, output_format=None):
    payload = {
        "model": MODEL,
        "messages": messages,
        "stream": False,
        "options": {"temperature": 0}
    }
    if output_format is not None:
        payload["format"] = output_format

    request = urllib.request.Request(
        API_URL,
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST"
    )

    try:
        with urllib.request.urlopen(request, timeout=120) as response:
            body = json.loads(response.read().decode("utf-8"))
    except OSError as error:
        raise RuntimeError(
            "Could not reach Ollama at {}: {}".format(API_URL, error)
        ) from error

    return body["message"]["content"]


def triage_baseline(ticket):
    prompt = (
        "Classify this support ticket and explain your decision in two sentences.\n\n"
        "Ticket: " + ticket
    )
    return call_ollama([{"role": "user", "content": prompt}])


def validate_result(result):
    required_fields = {"category", "priority", "summary", "escalate"}

    if not isinstance(result, dict):
        raise ValueError("The model result is not a JSON object.")
    if set(result.keys()) != required_fields:
        raise ValueError("The result must contain exactly: {}".format(
            ", ".join(sorted(required_fields))
        ))
    if result["category"] not in CATEGORIES:
        raise ValueError("Unknown category: {}".format(result["category"]))
    if result["priority"] not in PRIORITIES:
        raise ValueError("Unknown priority: {}".format(result["priority"]))
    if not isinstance(result["summary"], str) or not result["summary"].strip():
        raise ValueError("Summary must be a non-empty string.")
    if type(result["escalate"]) is not bool:
        raise ValueError("Escalate must be true or false.")


def triage_ticket(ticket):
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "Ticket:\n" + ticket}
    ]
    content = call_ollama(messages, TRIAGE_SCHEMA)
    result = json.loads(content)
    validate_result(result)
    return result


def run_evaluation(path):
    cases = json.loads(path.read_text(encoding="utf-8"))
    fields = ("category", "priority", "escalate")
    correct = 0
    total = len(cases) * len(fields)

    for number, case in enumerate(cases, start=1):
        try:
            actual = triage_ticket(case["ticket"])
            expected = case["expected"]
            matched = [field for field in fields if actual[field] == expected[field]]
            correct += len(matched)
            status = "PASS" if len(matched) == len(fields) else "FAIL"
            print("{} {}: {}".format(status, number, case["ticket"]))
            if status == "FAIL":
                print("  expected: {}".format(expected))
                print("  actual:   {}".format(
                    {field: actual[field] for field in fields}
                ))
        except (KeyError, RuntimeError, ValueError) as error:
            print("ERROR {}: {}".format(number, error))

    score = correct / total if total else 0
    print("\nField accuracy: {}/{} ({:.0%})".format(correct, total, score))


def main():
    parser = argparse.ArgumentParser(
        description="Triage support tickets with a local Ollama model."
    )
    parser.add_argument("ticket", nargs="?", help="support ticket text")
    parser.add_argument(
        "--baseline",
        action="store_true",
        help="show the unconstrained baseline response"
    )
    parser.add_argument(
        "--eval",
        action="store_true",
        help="run the labelled evaluation set"
    )
    args = parser.parse_args()

    try:
        if args.eval:
            run_evaluation(Path("tickets.json"))
        elif args.ticket:
            if args.baseline:
                print("BASELINE OUTPUT (unvalidated):")
                print(triage_baseline(args.ticket))
            else:
                print(json.dumps(triage_ticket(args.ticket), indent=2))
        else:
            parser.error("provide a ticket or use --eval")
    except (KeyError, RuntimeError, ValueError) as error:
        parser.exit(1, "Error: {}\n".format(error))


if __name__ == "__main__":
    main()

What should match?

The complete file keeps every baseline and structured-output feature from earlier. It adds only the file-path import, evaluation function, and evaluation command mode required in this step.

The suite calls your local model six times. Expect a pause between case results on slower computers.

Before you run the suite, which fields do you expect the current broad prompt to classify consistently?

  • Run the complete evaluation suite in PowerShell with this command:
python triage.py --eval

What should I see?

You will see one PASS, FAIL, or ERROR line for each of the six cases. A failed case also prints its expected values and actual values.

The final line begins with Field accuracy and reports the correct comparisons out of 18. Your exact score can vary because the local model generates the classifications.

Does the suite stop or report errors?

  • Confirm the Ollama application from earlier is running in the background.
  • Confirm tickets.json sits beside triage.py in ai-ticket-triage.
  • Compare the reported case with its expected object when the suite prints ERROR.

Help me debug my evaluation run.

You now have a repeatable report that exposes model behavior across six labelled tickets. Every future prompt change can face the same test.

Your first score gives you evidence instead of guesswork. Next, you will turn the visible mismatches into explicit triage rules and rerun the unchanged suite.

Improve the Triage Rules

Your labelled evaluation set now runs the same six tickets and reports field-level accuracy. That repeatable baseline gives you evidence for the next improvement.

The first report exposed cases where the model had to guess because the prompt only described the output shape. In this step, you will replace that ambiguity with explicit category rules, priority rules, and a narrow escalation policy.

In this step, get ready to:
  • Define explicit rules for the four ticket categories.
  • Add priority rules and a narrow escalation policy.
  • Rerun the unchanged evaluation set for a fair comparison.
Replace guesses with explicit rules

A JSON Schema defines the output shape. The SYSTEM_PROMPT defines how the model chooses each value.

  • Locate SYSTEM_PROMPT directly below TRIAGE_SCHEMA in triage.py.
  • Select the complete SYSTEM_PROMPT assignment through its closing triple quotes.
  • Replace the selected assignment with this decision policy:
SYSTEM_PROMPT = """You are a support ticket triage engine.
Return values that follow the supplied JSON schema.

Category rules:
- billing: charges, payments, subscriptions, refunds, or invoices
- account: sign-in, password, profile, or account settings
- technical: errors, outages, broken features, data imports, or data loss
- general: questions or requests that do not fit the other categories

Priority rules:
- critical: widespread outage, confirmed data loss, or an active security incident
- high: one user is blocked, failures repeat, or action is needed within one day
- medium: billing dispute, single-user error with a workaround, or degraded service
- low: informational question or routine request with no current impact

Set escalate to true only for critical tickets or explicit legal or security threats.
Use exactly one category and one priority value from the rules.
Write summary as one short sentence.
"""

What does this policy change?

  • The category section maps ticket topics to one allowed label.
  • The priority section maps customer impact to one severity level.
  • The escalation sentence reserves true for critical tickets or explicit threats.
  • The final instructions keep every result inside the validated contract.
  • Save triage.py in Visual Studio Code.

✔️ Awesome, I've got everything!

Your prompt now contains category rules, priority rules, and the narrow escalation policy.

ⓧ I'd like to double check the full code

Compare triage.py with this completed version:

import argparse
import json
import urllib.request
from pathlib import Path

API_URL = "http://localhost:11434/api/chat"
MODEL = "gemma3:1b"
CATEGORIES = {"billing", "account", "technical", "general"}
PRIORITIES = {"low", "medium", "high", "critical"}

TRIAGE_SCHEMA = {
    "type": "object",
    "properties": {
        "category": {"type": "string"},
        "priority": {"type": "string"},
        "summary": {"type": "string"},
        "escalate": {"type": "boolean"}
    },
    "required": ["category", "priority", "summary", "escalate"]
}

SYSTEM_PROMPT = """You are a support ticket triage engine.
Return values that follow the supplied JSON schema.

Category rules:
- billing: charges, payments, subscriptions, refunds, or invoices
- account: sign-in, password, profile, or account settings
- technical: errors, outages, broken features, data imports, or data loss
- general: questions or requests that do not fit the other categories

Priority rules:
- critical: widespread outage, confirmed data loss, or an active security incident
- high: one user is blocked, failures repeat, or action is needed within one day
- medium: billing dispute, single-user error with a workaround, or degraded service
- low: informational question or routine request with no current impact

Set escalate to true only for critical tickets or explicit legal or security threats.
Use exactly one category and one priority value from the rules.
Write summary as one short sentence.
"""


def call_ollama(messages, output_format=None):
    payload = {
        "model": MODEL,
        "messages": messages,
        "stream": False,
        "options": {"temperature": 0}
    }
    if output_format is not None:
        payload["format"] = output_format

    request = urllib.request.Request(
        API_URL,
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST"
    )

    try:
        with urllib.request.urlopen(request, timeout=120) as response:
            body = json.loads(response.read().decode("utf-8"))
    except OSError as error:
        raise RuntimeError(
            "Could not reach Ollama at {}: {}".format(API_URL, error)
        ) from error

    return body["message"]["content"]


def triage_baseline(ticket):
    prompt = (
        "Classify this support ticket and explain your decision in two sentences.\n\n"
        "Ticket: " + ticket
    )
    return call_ollama([{"role": "user", "content": prompt}])


def validate_result(result):
    required_fields = {"category", "priority", "summary", "escalate"}

    if not isinstance(result, dict):
        raise ValueError("The model result is not a JSON object.")
    if set(result.keys()) != required_fields:
        raise ValueError("The result must contain exactly: {}".format(
            ", ".join(sorted(required_fields))
        ))
    if result["category"] not in CATEGORIES:
        raise ValueError("Unknown category: {}".format(result["category"]))
    if result["priority"] not in PRIORITIES:
        raise ValueError("Unknown priority: {}".format(result["priority"]))
    if not isinstance(result["summary"], str) or not result["summary"].strip():
        raise ValueError("Summary must be a non-empty string.")
    if type(result["escalate"]) is not bool:
        raise ValueError("Escalate must be true or false.")


def triage_ticket(ticket):
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": "Ticket:\n" + ticket}
    ]
    content = call_ollama(messages, TRIAGE_SCHEMA)
    result = json.loads(content)
    validate_result(result)
    return result


def run_evaluation(path):
    cases = json.loads(path.read_text(encoding="utf-8"))
    fields = ("category", "priority", "escalate")
    correct = 0
    total = len(cases) * len(fields)

    for number, case in enumerate(cases, start=1):
        try:
            actual = triage_ticket(case["ticket"])
            expected = case["expected"]
            matched = [field for field in fields if actual[field] == expected[field]]
            correct += len(matched)
            status = "PASS" if len(matched) == len(fields) else "FAIL"
            print("{} {}: {}".format(status, number, case["ticket"]))
            if status == "FAIL":
                print("  expected: {}".format(expected))
                print("  actual:   {}".format(
                    {field: actual[field] for field in fields}
                ))
        except (KeyError, RuntimeError, ValueError) as error:
            print("ERROR {}: {}".format(number, error))

    score = correct / total if total else 0
    print("\nField accuracy: {}/{} ({:.0%})".format(correct, total, score))


def main():
    parser = argparse.ArgumentParser(
        description="Triage support tickets with a local Ollama model."
    )
    parser.add_argument("ticket", nargs="?", help="support ticket text")
    parser.add_argument(
        "--baseline",
        action="store_true",
        help="show the unconstrained baseline response"
    )
    parser.add_argument(
        "--eval",
        action="store_true",
        help="run the labelled evaluation set"
    )
    args = parser.parse_args()

    try:
        if args.eval:
            run_evaluation(Path("tickets.json"))
        elif args.ticket:
            if args.baseline:
                print("BASELINE OUTPUT (unvalidated):")
                print(triage_baseline(args.ticket))
            else:
                print(json.dumps(triage_ticket(args.ticket), indent=2))
        else:
            parser.error("provide a ticket or use --eval")
    except (KeyError, RuntimeError, ValueError) as error:
        parser.exit(1, "Error: {}\n".format(error))


if __name__ == "__main__":
    main()
Rerun the unchanged evaluation

Regression testing compares behavior against fixed examples. Keeping tickets.json unchanged isolates the effect of your new decision policy.

Before you run the suite, which ticket types do you expect the explicit rules to help most?

  • Rerun all six labelled cases from PowerShell with this command:
python triage.py --eval

You will see six case lines marked PASS, FAIL, or ERROR. The final line reports Field accuracy across the three objective fields.

Suite not completing?

  • Confirm the Ollama application from earlier is still running if every case reports a connection problem.
  • Save triage.py if the output still reflects the earlier prompt.
  • Compare the opening triple quotes with the closing triple quotes if Python reports a syntax problem.

Help me diagnose my evaluation run.

  • Compare the new Field accuracy line with the first score you captured.
  • Review each FAIL block if one appears.
  • Confirm the report contains six numbered case lines.
  • Find the final field-level accuracy total.

Why keep the evaluation set unchanged?

Keeping tickets.json unchanged makes the prompt policy the only project change in this comparison. The mismatch details show which decisions still need clearer rules.

You have closed the engineering loop: the unchanged cases now show the effect of your explicit decision policy.

Secret mission

Add a Security Category

Can your validated triage contract recognize unauthorized access without losing its existing behavior? Extend the policy with a security category and prove the change against all seven cases.

Clean Up Your Resources

Clean Up Your Resources

Everything in this project runs on your Windows computer through local inference with Ollama at no usage charge. Choose whether to keep the setup, pause Ollama for later, or remove every local resource.

Resources you used:

  • Local application: The Ollama installation that runs the inference service on your computer.
  • Local model data: The gemma3:1b model plus Ollama configuration stored under %HOMEPATH%\.ollama.
  • Local project files: The ai-ticket-triage folder containing triage.py plus tickets.json.

Keep everything running

No action needed. Choose this if you are still improving the triage rules or rerunning the evaluation.

Ollama remains available for local requests. Your project can run again without API keys or usage charges.

Pause - I'll come back to this later

Shut down Ollama to stop the local inference service while keeping your code plus downloaded model on disk.

  • Locate the Ollama icon in the Windows system tray.
  • Open the Ollama icon menu.
  • Choose Quit.
  • Confirm the Ollama icon disappears from the system tray.

Your project files plus model data remain on your computer. Relaunch Ollama from the Windows Start menu when you want to continue.

Delete - I don't want to use this again

Remove all project resources. Choose this when you want to start fresh.

Deletion is permanent. The Keep and Pause options protect your files if you may return.

Start with the Ollama application that provides the local inference service.

  • Press the Windows key to open search.
  • Type Add or remove programs.
  • Press Enter.
  • Find Ollama in the installed apps list.
  • Select Ollama.
  • Choose Uninstall.
  • Confirm Ollama no longer appears in the installed apps list.

The Ollama application is now removed. Its downloaded model plus configuration remain in %HOMEPATH%\.ollama until you delete that folder.

The ai-ticket-triage folder contains your pipeline plus seven-case regression dataset.

  • Switch back to Visual Studio Code.
  • Click the X in the top-right corner to close Visual Studio Code.

Visual Studio Code no longer holds the project folder open.

  • Press the Windows key to open search.
  • Type File Explorer.
  • Press Enter to open File Explorer.
  • Select Desktop in the left sidebar.
  • Select the ai-ticket-triage folder.
  • Press Shift+Delete.
  • Confirm the permanent deletion prompt.
  • Verify the ai-ticket-triage folder no longer appears on your Desktop.

The remaining %HOMEPATH%\.ollama folder holds the downloaded model plus Ollama configuration.

  • Click the File Explorer address bar.
  • Enter %HOMEPATH%.
  • Press Enter.
  • Select the .ollama folder.
  • Press Shift+Delete.
  • Confirm the permanent deletion prompt.
  • Verify the .ollama folder no longer appears under %HOMEPATH%.

That completes the cleanup. Your project code plus local model data are now removed from the computer.

Nice Work!

Nice Work!

You did it! Your Python pipeline now turns support tickets into validated triage records through local inference.

You've learned how to:

  • Built a Python client for Ollama's local HTTP API to turn support tickets into triage records.
  • Enforced a JSON Schema output contract that rejects invalid fields before another application consumes them.
  • Improved triage rules through a labelled evaluation set that reports field-level accuracy.
  • Completed the Secret Mission by extending the contract with a security category backed by a regression case.

Ready to quiz yourself?