Build a Local AI Bug Triage Tool

Build a local Python tool that turns bug reports into validated AI triage.

Introduction

30 Second Summary

Messy bug reports force software teams to guess what broke. Sensitive reports also create risk when they must leave the machine for analysis.

In this project, you will build a Python terminal assistant that turns an unclear bug report into a validated triage record using a local Ollama model. You will also prove its deterministic checks with automated tests.

What You'll Build

Your terminal turns a messy checkout report into a validated JSON triage record that identifies incorrect billing without sending the report to a cloud API.

By the end of this project, you'll have:

  • A rule-based baseline you can run to watch an ambiguous billing report receive the weak low severity and other category.
  • A local AI triage flow that returns eight validated fields for the same report through a REST API without an API key.
  • Four automated tests you can run to prove valid records pass while incomplete records and unsupported severity values are rejected without calling the model.
  • Secret Mission: Add a deterministic human-review gate that flags critical results or records with at least two missing-information items.

Before We Start

Before any setup begins, this is your moment to commit to what you are building. You will define how a local triage assistant turns unclear bug reports into structured decisions that a software team can review.

Set Up Your Local AI Workspace

Your triage assistant needs a dependable local workspace before it can process a report.

Verifying Python with Visual Studio Code from Windows PowerShell proves that each tool is reachable. Ollama then provides local model inference without a cloud account or API key.

In this step, get ready to:
  • Verify the existing developer tools from Windows PowerShell.
  • Prepare the ai-bug-triage workspace in Visual Studio Code.
  • Set up Ollama with the selected local model.
Verify your tools and create the workspace

Command-line version checks test whether PowerShell can find each installed tool. A dedicated folder keeps every later project file under one name.

  • Press the Windows key to open Windows Search.
  • Type PowerShell into the search field.
  • Press Enter to open Windows PowerShell.
  • Verify Python and Visual Studio Code by running these commands:
python --version
code --version

What do these checks confirm?

  • The python --version command prints the Python version available to PowerShell.
  • The code --version command prints the Visual Studio Code version details.

You should see a Python version line first. You should then see the Visual Studio Code version details.

Is one of the tools missing?

  • Confirm that Python appears in Windows Search.
  • Confirm that Visual Studio Code appears in Windows Search.
  • Restart PowerShell so it can load any recent installation changes.

If PowerShell still cannot reach a tool, help me diagnose why PowerShell cannot find Python or Visual Studio Code.

Good progress. PowerShell can now reach both tools that support the rest of the project.

The path shown at the left of your PowerShell prompt is the parent location for the new project folder.

  • Create the ai-bug-triage folder in that location by running this command:
New-Item -ItemType Directory -Path ".\ai-bug-triage"

What does this command do?

The command creates a directory named ai-bug-triage beneath the path shown in your prompt. PowerShell prints the new directory entry as confirmation.

You should see a directory entry named ai-bug-triage in the PowerShell output.

Was the folder creation rejected?

  • Check whether a folder named ai-bug-triage already exists in the displayed parent location.
  • Choose a Windows user location where you can create folders.

If PowerShell still rejects the command, help me troubleshoot the PowerShell folder creation.

  • Move into the project folder and open it in Visual Studio Code by running these commands:
Set-Location -Path ".\ai-bug-triage"
code .

How does this open the workspace?

  • The first command changes PowerShell's location to ai-bug-triage.
  • The second command opens the current folder in Visual Studio Code.

You should see ai-bug-triage as the open folder in the Visual Studio Code Explorer sidebar. Your PowerShell prompt should also end with the same folder name.

Did the folder stay closed?

  • Check that the PowerShell prompt ends with ai-bug-triage.
  • Restart Visual Studio Code if a previous window remains in front.

If the command finishes without opening the folder, help me troubleshoot the Visual Studio Code folder command.

Install and verify Ollama

Ollama runs as a local Windows application that serves model requests from your computer. This project uses version 0.34.4 so your setup matches the tested workflow.

  • Check whether Ollama is already available by running this command in PowerShell:
ollama --version

What does this check confirm?

The command asks the installed Ollama application for its version information. The result determines whether you can continue or need the Windows installer.

✔️ I see version 0.34.4

That is the required version. Your Ollama installation is ready for the model download.

ⓧ I see an older version

Use the official Windows installer to bring Ollama to version 0.34.4.

  • Visit the official Windows download page in your browser.
  • Download OllamaSetup.exe from the page.
  • Run OllamaSetup.exe from your browser's downloads list.
  • Wait for the installation to finish.
  • Press the Windows key to open Windows Search.
  • Type PowerShell into the search field.
  • Press Enter to open a new PowerShell window.
  • Verify the installed version by running this command:
ollama --version

What should the new check show?

The fresh PowerShell window loads the installed Ollama command. Its output should include 0.34.4.

ⓧ Command not found

The Windows installer adds Ollama to your user account without requiring administrator rights.

  • Visit the official Windows download page in your browser.
  • Download OllamaSetup.exe from the page.
  • Run OllamaSetup.exe from your browser's downloads list.
  • Wait for the installation to finish.
  • Press the Windows key to open Windows Search.
  • Type PowerShell into the search field.
  • Press Enter to open a new PowerShell window.
  • Verify the installed version by running this command:
ollama --version

What should the version check show?

The new PowerShell window should recognize the Ollama command. Its output should include 0.34.4.

That completes the application setup. Ollama is now available from PowerShell under your Windows user account.

Ollama serves its local API at http://localhost:11434. Requests to this local address do not need authentication.

Is the installation still missing?

  • Restart Windows PowerShell after the installer finishes.
  • Run the installer again if it closed before completing.
  • Confirm that Ollama appears in Windows Search.

If the version check still fails, help me diagnose the Ollama installation.

Download and test the local model

The selected 1B model is the engine that your later Python code calls. Its first run stores 832 MB of model data locally before answering.

Download time depends on your internet connection. An active progress display means the first run is still working.

Before you run the final check, do you expect the response to appear before or after the model download finishes?

  • Download the selected model and test its response by running this command:
ollama run llama3.2:1b-text-q4_1 "Reply with READY"

What does this command do?

  • The ollama run command starts the selected model locally.
  • The model identifier llama3.2:1b-text-q4_1 selects the exact model artifact used by this project.
  • The quoted instruction asks the model to return a short readiness response.

The first run shows download progress. Once the download finishes, you should see a model response containing READY.

Did the model response fail?

  • Confirm that the Ollama tray application is running.
  • Check your internet connection if the model download stops.
  • Run the model check again after the connection is stable.

If the download finishes without a response, help me troubleshoot the local Ollama model.

✔️ Awesome, I've got everything!

Your tools, project folder, Ollama service, and local model are ready for the application code.

ⓧ I'd like to double check the full setup

  • Ollama 0.34.4: Installed and available from Windows PowerShell.
  • ai-bug-triage: Local project folder created and open in Visual Studio Code.
  • llama3.2:1b-text-q4_1: Downloaded locally and verified with an Ollama READY response.

Your local AI workspace is ready to build in. Next, you will create a keyword baseline that gives the AI workflow a clear comparison point.

Build the Rule-Based Baseline

Your local Ollama model is ready on your machine. You now need a simple result that gives its future decisions something to beat.

A rule-based baseline classifies reports through literal keyword matches. The sample report avoids those trigger words so its misclassification becomes visible before AI enters the workflow.

In this step, get ready to:
  • Store the ambiguous checkout report in triage.py.
  • Build a classifier that reacts only to literal keywords.
  • Run the baseline to expose its incorrect severity and category.
Create the sample report

The sample describes a duplicate charge plus a missing order. Its wording leaves out the exact phrases that the baseline uses as evidence.

  • Select the file-creation icon at the top of the Visual Studio Code file sidebar.
  • Enter triage.py as the file name.
  • Add the local model settings and shared sample report by pasting this code:
import json

MODEL = "llama3.2:1b-text-q4_1"
OLLAMA_URL = "http://localhost:11434/api/chat"

SAMPLE_REPORT = (
    "After checkout, the page says 'Payment received', but the customer was "
    "charged twice and no order appears in order history. The page does not "
    "display an error message."
)

What does this code set up?

  • The json module formats the eventual result as readable JSON.
  • The MODEL value identifies the local model already available through Ollama.
  • The OLLAMA_URL value records the local chat endpoint for the AI workflow added later.
  • The SAMPLE_REPORT text describes serious business impact without using the baseline's trigger phrases.
  • Save triage.py.
  • Select triage.py in the Visual Studio Code file sidebar.

You'll see the checkout report in the editor. It mentions the duplicate charge plus the missing order.

Can't see the sample report?

Confirm that triage.py sits directly inside the ai-bug-triage folder. Check that the file name ends with .py.

Compare the parentheses and quotation marks with the code above. Help me check why my Python sample report is not displaying correctly.

Add the keyword rules

The classifier lowers the report text before checking a small set of phrases. Every report without a matching severity phrase falls into low.

  • Place your cursor below the closing parenthesis of SAMPLE_REPORT.
  • Add the literal keyword classifier by pasting this code:
def rule_based_triage(report):
    text = report.lower()
    if "security breach" in text:
        severity = "critical"
    elif any(keyword in text for keyword in ("crash", "data loss", "corruption")):
        severity = "high"
    else:
        severity = "low"

    if "crash" in text:
        category = "crash"
    elif "security" in text:
        category = "security"
    elif "slow" in text or "performance" in text:
        category = "performance"
    elif "button" in text or "layout" in text:
        category = "ui"
    else:
        category = "other"
    return {"severity": severity, "category": category}

How does the baseline decide?

  • The lowercase copy lets each phrase match regardless of capitalization.
  • The first rule group assigns critical or high only when an exact trigger phrase appears.
  • The second rule group maps literal words to a fixed category.
  • The fallback values produce low severity plus the other category.
  • Save triage.py.
  • Return to the PowerShell terminal from earlier.
  • Check that Python can load the current definitions by running this command:
python triage.py

What does this check prove?

Python reads the constants and defines rule_based_triage(). A clean return to the PowerShell prompt proves that the current file has valid syntax.

Seeing a Python traceback?

Use the reported line number to find the affected rule. Compare its indentation and quotation marks with the function above.

Confirm that every if statement has an indented body. Help me fix the syntax in my rule-based classifier.

Run the baseline and expose the gap

A terminal menu gives you a controlled entry point for the classifier. Option 1 sends the shared report through the keyword rules and formats the returned dictionary.

  • Place your cursor below the return line in rule_based_triage().
  • Add the terminal menu by pasting this code:
def main():
    print("AI Bug Report Triage Assistant")
    print("1. Run the rule-based baseline")
    choice = input("Choose 1: ").strip()
    if choice == "1":
        print("\nRule-based result:\n")
        print(json.dumps(rule_based_triage(SAMPLE_REPORT), indent=2))


if __name__ == "__main__":
    main()

What does the menu do?

  • The main() function prints the app title and its first menu option.
  • The input prompt captures the learner's selected option as text.
  • Option 1 passes SAMPLE_REPORT to rule_based_triage().
  • The indent=2 argument makes the returned fields easier to read in the terminal.
  • Save triage.py.

Before you run it, do you expect literal keyword matching to recognize the billing impact?

  • Launch the baseline from PowerShell by running this command:
python triage.py

What happens when it starts?

The command starts triage.py and displays its terminal menu. The program waits at the Choose 1: prompt.

  • Enter 1 at the Choose 1: prompt.

You'll see severity set to low. You'll see category set to other.

That failure is your win for this step. The result proves that literal keywords miss the duplicate billing impact and the broken order record.

Baseline result looks different?

Confirm that SAMPLE_REPORT matches the provided checkout report. Extra trigger words can change the classification.

Check that the fallback branches assign low and other. Help me find why my baseline does not return the expected fields.

✔️ Awesome, I've got everything!

Great. Double-check that you saved triage.py after seeing the deliberate misclassification.

ⓧ I'd like to double check the full code

import json

MODEL = "llama3.2:1b-text-q4_1"
OLLAMA_URL = "http://localhost:11434/api/chat"

SAMPLE_REPORT = (
    "After checkout, the page says 'Payment received', but the customer was "
    "charged twice and no order appears in order history. The page does not "
    "display an error message."
)


def rule_based_triage(report):
    text = report.lower()
    if "security breach" in text:
        severity = "critical"
    elif any(keyword in text for keyword in ("crash", "data loss", "corruption")):
        severity = "high"
    else:
        severity = "low"

    if "crash" in text:
        category = "crash"
    elif "security" in text:
        category = "security"
    elif "slow" in text or "performance" in text:
        category = "performance"
    elif "button" in text or "layout" in text:
        category = "ui"
    else:
        category = "other"
    return {"severity": severity, "category": category}


def main():
    print("AI Bug Report Triage Assistant")
    print("1. Run the rule-based baseline")
    choice = input("Choose 1: ").strip()
    if choice == "1":
        print("\nRule-based result:\n")
        print(json.dumps(rule_based_triage(SAMPLE_REPORT), indent=2))


if __name__ == "__main__":
    main()

Your baseline now fails in a visible and repeatable way. Next, you'll use the report's context to produce a richer triage record.

Add Context-Aware AI Triage

Your baseline proved that duplicate billing can hide behind ordinary words. Its low and other result gives the engineering team too little context.

This step sends the same report to the local Ollama REST API. A JSON Schema plus Python validation keeps the model response inside eight trusted fields.

In this step, get ready to:
  • Define the required structure for every AI triage record.
  • Send the report to the local model through Ollama.
  • Run AI sample mode plus custom-report mode.

Why direct REST?

Python's standard library can send the request without another package. This exposes the API boundary that your validation protects.

The first JSON document is Ollama's response envelope. The second JSON document is the triage record inside message.content.

Define the triage contract

A schema tells the model which fields to return. Application validation still checks the decoded result before any other code uses it.

  • Switch back to triage.py in VS Code.
  • Find the current import line:
import json

What is this line?

This is the existing import at the top of triage.py. The next edit expands this import group with Python's HTTP request tools.

  • Replace the current import line with this import group:
import json
from urllib.error import URLError
from urllib.request import Request, urlopen

What do these imports provide?

  • The json module encodes the request body. It also decodes both response layers.
  • The Request class describes the HTTP request.
  • The urlopen function sends the request. The URLError exception identifies connection failures.
  • Find the closing parenthesis beneath SAMPLE_REPORT.
  • Add the schema plus its allowed-value sets below that parenthesis by pasting this code:
TRIAGE_SCHEMA = {
    "type": "object",
    "properties": {
        "title": {"type": "string"},
        "severity": {"type": "string", "enum": ["critical", "high", "medium", "low"]},
        "category": {"type": "string", "enum": ["crash", "security", "data-integrity", "performance", "ui", "other"]},
        "affected_component": {"type": "string"},
        "summary": {"type": "string"},
        "reproduction_steps": {"type": "array", "items": {"type": "string"}},
        "missing_information": {"type": "array", "items": {"type": "string"}},
        "recommended_action": {"type": "string"},
    },
    "required": ["title", "severity", "category", "affected_component", "summary", "reproduction_steps", "missing_information", "recommended_action"],
}

SEVERITIES = set(TRIAGE_SCHEMA["properties"]["severity"]["enum"])
CATEGORIES = set(TRIAGE_SCHEMA["properties"]["category"]["enum"])

What does the schema protect?

  • The properties mapping defines the type of each triage field.
  • The required list makes all eight fields mandatory.
  • The enum values limit severity plus category to choices your application understands.
  • The SEVERITIES and CATEGORIES sets give Python fast membership checks.
  • Save triage.py.
  • Confirm that the existing baseline still runs by entering this command in PowerShell:
python triage.py

What does this check prove?

The command loads the new imports plus schema before starting the existing menu. Reaching the menu proves that Python can parse the additions.

  • Enter 1 at the menu prompt.

You should still see low severity plus the other category. The original baseline remains intact.

  • Find the end of rule_based_triage(report).
  • Add the validation function below its final return line by pasting this code:
def validate_triage(result):
    if not isinstance(result, dict):
        raise ValueError("AI response must be a JSON object.")
    required = set(TRIAGE_SCHEMA["required"])
    received = set(result)
    missing = required - received
    unexpected = received - required
    if missing:
        raise ValueError("Missing fields: " + ", ".join(sorted(missing)))
    if unexpected:
        raise ValueError("Unexpected fields: " + ", ".join(sorted(unexpected)))
    if result["severity"] not in SEVERITIES:
        raise ValueError("Unsupported severity: " + str(result["severity"]))
    if result["category"] not in CATEGORIES:
        raise ValueError("Unsupported category: " + str(result["category"]))

    for field in ("title", "affected_component", "summary", "recommended_action"):
        if not isinstance(result[field], str) or not result[field].strip():
            raise ValueError(field + " must be a non-empty string.")
    for field in ("reproduction_steps", "missing_information"):
        if not isinstance(result[field], list):
            raise ValueError(field + " must be a list.")
        if not all(isinstance(item, str) for item in result[field]):
            raise ValueError(field + " must contain only strings.")
    return result

What does validation check?

  • The first check requires a Python dictionary.
  • The set comparisons reject missing fields. They also reject unexpected fields.
  • The membership checks reject unsupported severity or category values.
  • The loops require meaningful strings plus lists containing only strings.
  • Save triage.py.
  • Check that the validator has not broken the existing menu by running this command:
python triage.py

What should load?

You should reach the one-option baseline menu without a syntax error. The validator is now available for the AI path.

  • Enter 1 to complete the check.

Seeing a syntax error?

Check that TRIAGE_SCHEMA appears before validate_triage(result). Confirm that each closing brace matches the reference block.

Check that validate_triage(result) starts at the left edge of the file. Help me fix this validator syntax error.

Connect Python to Ollama

The request combines a severity rubric with the report. It also asks Ollama to constrain its response using TRIAGE_SCHEMA.

  • Find the end of validate_triage(result).
  • Add the first part of triage_with_ai(report) below the validator by pasting this code:
def triage_with_ai(report):
    system_prompt = (
        "You are a software bug triage assistant. Use only evidence in the report. "
        "Severity rubric: critical means an active security breach, system-wide outage, or irreversible widespread loss; "
        "high means incorrect billing, data loss or corruption, or a blocked critical workflow; "
        "medium means a feature behaves incorrectly but work can continue; low means cosmetic or minor inconvenience. "
        "Infer impact from context, not isolated keywords. Use data-integrity for incorrect persisted state or transaction records. "
        "List absent evidence under missing_information. Return only JSON that matches the supplied schema."
    )
    payload = {
        "model": MODEL,
        "messages": [
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": report},
        ],
        "stream": False,
        "format": TRIAGE_SCHEMA,
        "options": {"temperature": 0},
    }
    request = Request(
        OLLAMA_URL,
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST",
    )

How is the request built?

  • The system prompt defines the severity rubric. It tells the model to report missing evidence.
  • The messages list separates the system instructions from the user's report.
  • The stream value requests one complete response.
  • The format value supplies the schema. The temperature value reduces randomness.
  • Place the response-handling block directly below the closing parenthesis of Request.
    try:
        with urlopen(request, timeout=120) as response:
            api_response = json.loads(response.read().decode("utf-8"))
        model_content = api_response["message"]["content"]
        return validate_triage(json.loads(model_content))
    except (URLError, TimeoutError) as exc:
        detail = getattr(exc, "reason", exc)
        raise RuntimeError("Could not reach Ollama. Confirm that the Ollama tray app is running. " + str(detail)) from exc
    except (KeyError, TypeError, ValueError, json.JSONDecodeError) as exc:
        raise RuntimeError("Ollama returned an invalid triage response: " + str(exc)) from exc

How is the response made safe?

  • The first decode turns Ollama's outer response into a Python dictionary.
  • The message.content lookup retrieves the model's JSON text.
  • The second decode turns that text into the triage dictionary.
  • The exception handlers convert connection failures plus malformed responses into clear application errors.
  • Save triage.py.
  • Confirm that Python can load the completed function by running this command:
python triage.py

What does this load check prove?

Reaching the menu proves that the complete request function has valid Python syntax. It also proves that the new imports resolve correctly.

  • Enter 1 to finish the load check.

Can't load the request function?

Confirm that the try block remains indented inside triage_with_ai(report). Its indentation should align with system_prompt.

Check that both except blocks align with try. Help me fix this Ollama request function.

Add AI and custom-report modes

The final helpers collect multiline input plus print validated JSON. The expanded menu connects those helpers to the local AI request.

  • Find the end of triage_with_ai(report).
  • Add the input plus output helpers below that function by pasting this code:
def read_report():
    print("Paste the bug report. Press Enter on an empty line to finish:")
    lines = []
    while True:
        line = input()
        if line == "":
            break
        lines.append(line)
    report = "\n".join(lines).strip()
    if len(report) < 20:
        raise ValueError("The bug report must contain at least 20 characters.")
    return report


def print_result(result):
    print("\nValidated triage JSON:\n")
    print(json.dumps(result, indent=2))

What do these helpers do?

  • The read_report() loop collects each line until the learner submits an empty line.
  • The minimum length check rejects reports with too little detail.
  • The print_result(result) helper formats the validated dictionary as readable JSON.
  • Save triage.py.
  • Check that the helper functions load by running this command:
python triage.py

What should still work?

You should reach the existing one-option menu. This confirms that Python loaded both helper functions.

  • Enter 1 to close the load check.
  • Find the current main() function:
def main():
    print("AI Bug Report Triage Assistant")
    print("1. Run the rule-based baseline")
    choice = input("Choose 1: ").strip()
    if choice == "1":
        print("\nRule-based result:\n")
        print(json.dumps(rule_based_triage(SAMPLE_REPORT), indent=2))


if __name__ == "__main__":
    main()

What is the current menu boundary?

The current main() function only exposes the baseline. The replacement keeps option 1 while adding both AI paths.

  • Replace the current main() function plus its final launch block with this expanded version:
def main():
    print("AI Bug Report Triage Assistant")
    print("1. Run the rule-based baseline")
    print("2. Triage the built-in sample with local AI")
    print("3. Triage your own report with local AI")
    choice = input("Choose 1, 2, or 3: ").strip()
    if choice == "1":
        print("\nRule-based result:\n")
        print(json.dumps(rule_based_triage(SAMPLE_REPORT), indent=2))
        return
    try:
        if choice == "2":
            report = SAMPLE_REPORT
        elif choice == "3":
            report = read_report()
        else:
            raise ValueError("Choose 1, 2, or 3.")
        print_result(triage_with_ai(report))
    except (RuntimeError, ValueError) as exc:
        print("\nTriage failed: " + str(exc))


if __name__ == "__main__":
    main()

How does the expanded menu work?

  • Option 1 preserves the deliberate baseline failure.
  • Option 2 sends SAMPLE_REPORT to the local model.
  • Option 3 collects a multiline report before sending it.
  • The final exception handler prints a readable failure instead of a Python traceback.
  • Save triage.py.

Before you run it, predict whether the local model will preserve the baseline pair or classify the billing impact from context.

  • Start the completed application by running this command:
python triage.py

What starts now?

The command starts the three-option menu. Option 2 performs the end-to-end local AI check.

  • Enter 2 at the menu prompt.

You should see a validated JSON object containing title, severity, category, affected component, summary, reproduction steps, missing information, and recommended action.

The result should classify the duplicate billing impact as high severity plus data-integrity. That richer result closes the gap exposed by the baseline.

  • Start the application again by running this command:
python triage.py

What are you checking next?

This run checks the multiline input path. The same schema plus validator protects the custom result.

  • Enter 3 at the menu prompt.
  • Paste a bug report containing at least 20 characters.
  • Press Enter on an empty line to submit the report.

You should see another validated eight-field JSON record based on your report. The report remains inside your local Ollama workflow.

AI triage not completing?

Confirm that the Ollama tray application is running. The local request depends on its background service.

Check that OLLAMA_URL still matches the value near the top of triage.py. Help me diagnose this local AI triage failure.

✔️ Awesome, I've got everything!

Great. Save triage.py after confirming both AI modes return validated JSON.

ⓧ I'd like to double check the full code

import json
from urllib.error import URLError
from urllib.request import Request, urlopen

MODEL = "llama3.2:1b-text-q4_1"
OLLAMA_URL = "http://localhost:11434/api/chat"

SAMPLE_REPORT = (
    "After checkout, the page says 'Payment received', but the customer was "
    "charged twice and no order appears in order history. The page does not "
    "display an error message."
)

TRIAGE_SCHEMA = {
    "type": "object",
    "properties": {
        "title": {"type": "string"},
        "severity": {"type": "string", "enum": ["critical", "high", "medium", "low"]},
        "category": {"type": "string", "enum": ["crash", "security", "data-integrity", "performance", "ui", "other"]},
        "affected_component": {"type": "string"},
        "summary": {"type": "string"},
        "reproduction_steps": {"type": "array", "items": {"type": "string"}},
        "missing_information": {"type": "array", "items": {"type": "string"}},
        "recommended_action": {"type": "string"},
    },
    "required": ["title", "severity", "category", "affected_component", "summary", "reproduction_steps", "missing_information", "recommended_action"],
}

SEVERITIES = set(TRIAGE_SCHEMA["properties"]["severity"]["enum"])
CATEGORIES = set(TRIAGE_SCHEMA["properties"]["category"]["enum"])


def rule_based_triage(report):
    text = report.lower()
    if "security breach" in text:
        severity = "critical"
    elif any(keyword in text for keyword in ("crash", "data loss", "corruption")):
        severity = "high"
    else:
        severity = "low"

    if "crash" in text:
        category = "crash"
    elif "security" in text:
        category = "security"
    elif "slow" in text or "performance" in text:
        category = "performance"
    elif "button" in text or "layout" in text:
        category = "ui"
    else:
        category = "other"
    return {"severity": severity, "category": category}


def validate_triage(result):
    if not isinstance(result, dict):
        raise ValueError("AI response must be a JSON object.")
    required = set(TRIAGE_SCHEMA["required"])
    received = set(result)
    missing = required - received
    unexpected = received - required
    if missing:
        raise ValueError("Missing fields: " + ", ".join(sorted(missing)))
    if unexpected:
        raise ValueError("Unexpected fields: " + ", ".join(sorted(unexpected)))
    if result["severity"] not in SEVERITIES:
        raise ValueError("Unsupported severity: " + str(result["severity"]))
    if result["category"] not in CATEGORIES:
        raise ValueError("Unsupported category: " + str(result["category"]))

    for field in ("title", "affected_component", "summary", "recommended_action"):
        if not isinstance(result[field], str) or not result[field].strip():
            raise ValueError(field + " must be a non-empty string.")
    for field in ("reproduction_steps", "missing_information"):
        if not isinstance(result[field], list):
            raise ValueError(field + " must be a list.")
        if not all(isinstance(item, str) for item in result[field]):
            raise ValueError(field + " must contain only strings.")
    return result


def triage_with_ai(report):
    system_prompt = (
        "You are a software bug triage assistant. Use only evidence in the report. "
        "Severity rubric: critical means an active security breach, system-wide outage, or irreversible widespread loss; "
        "high means incorrect billing, data loss or corruption, or a blocked critical workflow; "
        "medium means a feature behaves incorrectly but work can continue; low means cosmetic or minor inconvenience. "
        "Infer impact from context, not isolated keywords. Use data-integrity for incorrect persisted state or transaction records. "
        "List absent evidence under missing_information. Return only JSON that matches the supplied schema."
    )
    payload = {
        "model": MODEL,
        "messages": [
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": report},
        ],
        "stream": False,
        "format": TRIAGE_SCHEMA,
        "options": {"temperature": 0},
    }
    request = Request(
        OLLAMA_URL,
        data=json.dumps(payload).encode("utf-8"),
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    try:
        with urlopen(request, timeout=120) as response:
            api_response = json.loads(response.read().decode("utf-8"))
        model_content = api_response["message"]["content"]
        return validate_triage(json.loads(model_content))
    except (URLError, TimeoutError) as exc:
        detail = getattr(exc, "reason", exc)
        raise RuntimeError("Could not reach Ollama. Confirm that the Ollama tray app is running. " + str(detail)) from exc
    except (KeyError, TypeError, ValueError, json.JSONDecodeError) as exc:
        raise RuntimeError("Ollama returned an invalid triage response: " + str(exc)) from exc


def read_report():
    print("Paste the bug report. Press Enter on an empty line to finish:")
    lines = []
    while True:
        line = input()
        if line == "":
            break
        lines.append(line)
    report = "\n".join(lines).strip()
    if len(report) < 20:
        raise ValueError("The bug report must contain at least 20 characters.")
    return report


def print_result(result):
    print("\nValidated triage JSON:\n")
    print(json.dumps(result, indent=2))


def main():
    print("AI Bug Report Triage Assistant")
    print("1. Run the rule-based baseline")
    print("2. Triage the built-in sample with local AI")
    print("3. Triage your own report with local AI")
    choice = input("Choose 1, 2, or 3: ").strip()
    if choice == "1":
        print("\nRule-based result:\n")
        print(json.dumps(rule_based_triage(SAMPLE_REPORT), indent=2))
        return
    try:
        if choice == "2":
            report = SAMPLE_REPORT
        elif choice == "3":
            report = read_report()
        else:
            raise ValueError("Choose 1, 2, or 3.")
        print_result(triage_with_ai(report))
    except (RuntimeError, ValueError) as exc:
        print("\nTriage failed: " + str(exc))


if __name__ == "__main__":
    main()

How should you use this reference?

Compare this file with your saved triage.py. Every import, identifier, indentation level, plus literal value should match.

Your application now turns an unclear report into a validated eight-field triage record. Next, you will test the deterministic rules without depending on a model call.

Test the Core Logic

Your local AI path now turns an unstructured report into a validated triage record. The remaining risk is that deterministic validation could regress while model output still looks convincing.

Unit tests isolate the keyword baseline from Ollama. They also exercise the validation rules without making a live inference request.

In this step, get ready to:
  • Create a reusable valid triage fixture.
  • Prove the known keyword-baseline limitation.
  • Exercise the validator's acceptance boundaries with automated tests.
Create the shared test fixture

A shared valid result gives every validation test the same starting point. Each test can copy that object before changing one specific condition.

  • In the VS Code Explorer sidebar from earlier, select the ai-bug-triage folder.
  • Click the New File icon in the Explorer toolbar.

You'll see an active field for the new file name.

  • Enter test_triage.py as the file name.
  • Press Enter to create the file.

The empty test_triage.py file is now open beside triage.py.

  • Populate test_triage.py with the shared test fixture by pasting this code:
import unittest
from triage import SAMPLE_REPORT, rule_based_triage, validate_triage

VALID_TRIAGE = {
    "title": "Duplicate charge with missing order",
    "severity": "high",
    "category": "data-integrity",
    "affected_component": "checkout",
    "summary": "Payment was duplicated and the order record was not created.",
    "reproduction_steps": ["Complete checkout", "Open order history"],
    "missing_information": [],
    "recommended_action": "Inspect payment idempotency and order persistence logs.",
}

What Does This Fixture Do?

  • The unittest import provides Python's test framework.
  • The three imports from triage.py target the deterministic application logic.
  • The triage_with_ai function is absent from the imports. This test file cannot trigger a model request through that function.
  • The VALID_TRIAGE dictionary contains the eight fields accepted by validate_triage.
  • Save test_triage.py.
  • Confirm the fixture loads successfully by running this command in the PowerShell terminal from earlier:
python -m unittest -v

What Does This Command Check?

  • The unittest runner discovers test files in the current folder.
  • The -v flag prints detailed results for each discovered test.
  • At this point, the successful run proves that the fixture imports cleanly.

You'll see the test runner complete successfully with no test cases discovered. That output confirms the fixture has valid Python syntax.

Fixture Not Loading?

Confirm that PowerShell remains inside the ai-bug-triage folder. Both Python files must appear in that folder.

Compare every quote and comma in VALID_TRIAGE with the snippet above. Help me diagnose why my test fixture does not load.

Add four deterministic tests

The suite preserves the baseline's known failure as an expected result. It also checks one accepted triage record plus two rejected records.

  • In test_triage.py, place your cursor below the closing brace of VALID_TRIAGE.
  • Add the test class by pasting this code:
class TriageTests(unittest.TestCase):
    def test_rule_based_baseline_misses_context(self):
        result = rule_based_triage(SAMPLE_REPORT)
        self.assertEqual(result["severity"], "low")
        self.assertEqual(result["category"], "other")

    def test_valid_result_passes(self):
        self.assertEqual(validate_triage(VALID_TRIAGE), VALID_TRIAGE)

    def test_missing_field_is_rejected(self):
        invalid = VALID_TRIAGE.copy()
        del invalid["recommended_action"]
        with self.assertRaises(ValueError):
            validate_triage(invalid)

    def test_invalid_severity_is_rejected(self):
        invalid = VALID_TRIAGE.copy()
        invalid["severity"] = "urgent"
        with self.assertRaises(ValueError):
            validate_triage(invalid)


if __name__ == "__main__":
    unittest.main()

What Do These Tests Prove?

  • The baseline test preserves the deliberate low severity result. It also preserves the deliberate other category result.
  • The valid-result test confirms that all eight accepted fields pass through unchanged.
  • The missing-field test proves that validate_triage rejects an incomplete record.
  • The invalid-severity test proves that an unsupported value raises ValueError.
  • The final guard allows the test file to run directly when needed.
  • Save test_triage.py.

Before you run the suite, do you expect any test to contact the local model?

  • Run all four tests in verbose mode by using this command:
python -m unittest -v

Why Does the Suite Stay Local?

  • Test discovery imports test_triage.py.
  • Every method beginning with test_ runs as a separate test case.
  • The suite never calls triage_with_ai. Its result does not depend on model output.

You'll see each of the four test names pass. The summary ends with Ran 4 tests followed by OK.

That is the core logic locked down. The baseline expectation now has repeatable proof.

Tests Not Passing?

Confirm that each test method remains indented inside TriageTests. Python uses indentation to determine which methods belong to the class.

Check that VALID_TRIAGE contains every required field before the tests copy it. Help me debug my four unittest failures.

✔️ Awesome, I've got everything!

Your four tests pass. Double-check that you saved test_triage.py.

ⓧ I'd like to double check the full code

import unittest
from triage import SAMPLE_REPORT, rule_based_triage, validate_triage

VALID_TRIAGE = {
    "title": "Duplicate charge with missing order",
    "severity": "high",
    "category": "data-integrity",
    "affected_component": "checkout",
    "summary": "Payment was duplicated and the order record was not created.",
    "reproduction_steps": ["Complete checkout", "Open order history"],
    "missing_information": [],
    "recommended_action": "Inspect payment idempotency and order persistence logs.",
}


class TriageTests(unittest.TestCase):
    def test_rule_based_baseline_misses_context(self):
        result = rule_based_triage(SAMPLE_REPORT)
        self.assertEqual(result["severity"], "low")
        self.assertEqual(result["category"], "other")

    def test_valid_result_passes(self):
        self.assertEqual(validate_triage(VALID_TRIAGE), VALID_TRIAGE)

    def test_missing_field_is_rejected(self):
        invalid = VALID_TRIAGE.copy()
        del invalid["recommended_action"]
        with self.assertRaises(ValueError):
            validate_triage(invalid)

    def test_invalid_severity_is_rejected(self):
        invalid = VALID_TRIAGE.copy()
        invalid["severity"] = "urgent"
        with self.assertRaises(ValueError):
            validate_triage(invalid)


if __name__ == "__main__":
    unittest.main()

How to Use This Reference

This reference combines the shared fixture with all four deterministic tests. It matches the completed test_triage.py file.

Run a final custom report

The tests prove the deterministic boundaries. A final custom demo confirms that the same validation layer still works inside the complete local AI flow.

Before you run the final demo, do you expect your custom report to pass through the same eight-field validation path as the built-in sample?

  • Start the existing terminal application by running this command:
python triage.py

What Does This Command Start?

  • The command runs the completed application in triage.py.
  • The existing menu offers the baseline mode plus two local AI modes.
  • Option 3 routes your own report through the AI request and validation path.
  • Enter 3 at the menu prompt.
  • Paste a custom bug report containing at least 20 characters.
  • Press Enter on an empty line to submit the report.

You'll see Validated triage JSON: followed by all eight validated fields for your report.

Custom Report Not Completing?

Confirm that the Ollama tray application remains running. The custom mode needs the local API before validation can begin.

Use a report longer than 20 characters. Finish the input with a separate empty line.

If the response is rejected, read the message printed after Triage failed:. Help me diagnose my custom triage demo.

Your application now has repeatable proof for its deterministic core. Its custom mode still reaches the local AI workflow.

Secret mission

Add a Human Review Gate

Add a deterministic gate that stops critical or incomplete triage results for human review. You will display the decision after every validated result and prove the rules with three more tests.

Clean Up Your Resources

Clean Up Your Resources

Choose whether to keep your local resources ready, pause the running service, or remove everything from your computer. This project runs entirely on your computer, so there are no paid or metered cloud resources.

Resources you used:

  • A local Python project in the ai-bug-triage folder containing triage.py and test_triage.py.
  • An Ollama installation at version 0.34.4.
  • A downloaded local AI model stored under the identifier llama3.2:1b-text-q4_1.

Keep everything running

No action is needed while you are actively building or demonstrating the triage assistant. Keeping these resources preserves the quickest path back to your working application.

  • Leave Ollama installed so the local model stays ready for another demonstration.
  • Keep the ai-bug-triage folder with both Python files.

Pause - I'll come back to this later

Pausing stops local inference while preserving the application files and downloaded model. You can return without downloading the model again.

  • Quit the Ollama tray application from the Windows notification area.

The Ollama local service stops accepting requests. The model and the ai-bug-triage folder remain on your computer.

Delete - I don't want to use this again

Complete removal frees the local storage used by the model, application, and project files. A separate copy of the folder protects your code if you want to revisit it.

Remove the downloaded model:

  • Remove the selected model from local storage by running this command in PowerShell:
ollama rm llama3.2:1b-text-q4_1

What does this command remove?

This command deletes only the selected model files from Ollama's local storage. Ollama remains installed until you remove the application separately.

  • Confirm PowerShell returns to its prompt without reporting a failure.

Model Removal Failed?

  • Confirm the Ollama tray application is running.
  • Retry the model removal.

Get help diagnosing the model-removal failure.

Uninstall Ollama:

  • Open Add or remove programs from Windows search.
  • Find Ollama in the installed applications list.
  • Select the uninstall option for Ollama.
  • Complete the uninstall confirmation.
  • Confirm Ollama no longer appears in the installed applications list.

Delete the project folder:

  • Close Visual Studio Code so the project files are released.
  • Use Windows search to open File Explorer.
  • Go to the folder that contains ai-bug-triage.
  • Move ai-bug-triage to the Recycle Bin.
  • Open the Recycle Bin.
  • Permanently delete only ai-bug-triage from the Recycle Bin.

Cleanup complete. Your computer no longer holds the project folder, Ollama installation, or selected model.

Nice Work!

Nice Work!

You did it! You built a Python terminal assistant that turns messy bug reports into validated triage records using local Ollama inference. The workflow keeps reports local while placing risky decisions behind a human review gate.

You've learned how to:

  • Expose the limits of a rule-based baseline by showing how literal keyword matching misses the impact of duplicate billing.
  • Send bug reports through a local REST API and receive a triage record constrained by JSON Schema.
  • Reject unsafe model output with defensive validation. Prove deterministic behavior with four automated tests that never call the model.
  • Secret Mission: Add a deterministic human-review gate that escalates critical or incomplete triage results. Prove the gate with three additional tests.

Ready to quiz yourself?