Code-First AI Incident Investigator

Build a Python agent that investigates synthetic incident logs with safe tools.

Introduction

30 Second Summary

When checkout starts failing, the cause is often buried across several files. A quick guess can sound convincing without proving anything.

In this project, you will build a terminal-based AI agent in Python. You will watch it call controlled local tools before it diagnoses a synthetic checkout failure from two evidence files.

What You'll Build

Picture the terminal tracing each tool call before your investigator cites the exact evidence behind the checkout failure.

By the end of this project, you'll have:

  • A runnable prompt-only baseline that shows why a model cannot inspect local evidence without tools.
  • A safe local toolset that handles every evidence operation inside a strict filesystem boundary.
  • A visible function calling loop that traces each tool selection before producing a diagnosis grounded in checkout.log plus runbook.txt.
  • Secret Mission: Add a focused search tool that returns matching evidence with filenames plus line numbers.

Are there any prerequisites?

You need basic Python reading skills plus a Google account. No prior code-first agent experience is required. The project walks you through the Windows setup from scratch.

Before We Start

This is your moment to commit to an evidence-driven local incident investigator before the hands-on work begins. It will examine synthetic checkout logs and a runbook so Gemini must use controlled tools before reaching a conclusion.

Set Up the Agent Workspace

A code-first investigator depends on a reproducible local runtime. A mismatched Python installation can load a different SDK from the one your project expects.

This step builds an isolated workspace in Visual Studio Code with a pinned Google Gen AI SDK. It also keeps your Gemini Developer API credential outside the project code.

In this step, get ready to:
  • Install the required workspace tools.
  • Create an isolated virtual environment with the pinned SDK.
  • Store the Gemini API key in your Windows user environment.
Install the workspace tools

The project uses Python 3.15.0 so every command runs against the same language version. Start by checking the version available in Windows PowerShell.

  • Press the Windows key to open the search bar.
  • Type Windows PowerShell and press Enter.
  • Check your installed Python version by running this command:
python --version

What does this command do?

The python command starts the default Python installation. The version option prints its version without starting an interactive session.

✔️ I see the required Python version

Your terminal shows Python 3.15.0. That runtime is ready to support the investigator.

ⓧ I see an older Python version

An older runtime can remain installed while the Python Install Manager adds the current stable release. Managed runtimes take priority over unmanaged installations.

  • Open the official Python Windows download page in your browser.
  • Download the Python Install Manager.
  • Run the downloaded installer.
  • Approve the Windows permission request if one appears.
  • Complete the installation.
  • Close your current PowerShell window.
  • Open a new Windows PowerShell window through Windows search.
  • Check which Python version is now active by running:
python --version

What should I see?

The install manager selects the current stable runtime. You should now see Python 3.15.0.

Still seeing the older version?

Use the official Windows Python troubleshooting guide to check for an older path entry or an inactive Python app alias.

Ask for help with the version conflict: Why does Windows PowerShell still use my older Python installation?

ⓧ Command not found

Windows needs the Python Install Manager before the global command becomes available. The manager can install the current stable runtime when you first invoke Python.

  • Open the official Python Windows download page in your browser.
  • Download the Python Install Manager.
  • Run the downloaded installer.
  • Approve the Windows permission request if one appears.
  • Complete the installation.
  • Close your current PowerShell window.
  • Open a new Windows PowerShell window through Windows search.
  • Install the current stable runtime if prompted by running:
python --version

What should I see?

The first invocation can trigger a runtime installation. Once it finishes, you should see Python 3.15.0.

Python command still missing?

Close every PowerShell window after the installation. A fresh terminal reloads the command path.

Ask for installation help: Why is the Python command unavailable after installing the Python Install Manager?

Visual Studio Code provides the editor and terminal you will use throughout the project. Check whether Windows can find it before you create the workspace.

  • Press the Windows key to open the search bar.
  • Type Visual Studio Code into the search bar.

✔️ Visual Studio Code appears

Windows can find Visual Studio Code. The terminal command becomes your next check when you open the project folder.

  • Close the Windows search panel.

ⓧ Visual Studio Code is missing

The Windows User Setup adds Visual Studio Code to your command path. A new terminal can then open the current folder in the editor.

  • Open the official Visual Studio Code Windows setup guide in your browser.
  • Download the Windows User Setup.
  • Run the downloaded installer.
  • Approve the Windows permission request if one appears.
  • Complete the installation.
  • Close every open PowerShell window so the command path can refresh.

Installation did not complete?

Check whether Windows blocked the installer or your account lacks permission to install applications.

Ask for setup help: How do I troubleshoot a failed Visual Studio Code User Setup on Windows?

Your project needs a named folder on the Desktop so every later file has a predictable location. The first command group creates that location and lists it for confirmation.

  • Return to your open Windows PowerShell window.
  • Create the ai-incident-investigator folder on your Desktop by running these commands:
cd ~/Desktop
mkdir ai-incident-investigator
ls

What do these commands do?

  • The first command moves PowerShell to your Desktop.
  • The second command creates the project folder.
  • The final command lists the Desktop contents so you can confirm the folder exists.

You should see ai-incident-investigator in the folder listing.

Folder missing from the list?

Check that the folder name contains hyphens between all three words. Confirm that PowerShell is showing your Desktop path.

Ask for folder help: Why did my PowerShell folder command fail?

  • Enter the new project folder in PowerShell by running these commands:
cd ai-incident-investigator
code .

What do these commands do?

PowerShell moves into ai-incident-investigator. The Visual Studio Code command opens that current folder as your workspace.

You should see Visual Studio Code open with ai-incident-investigator named in the Explorer sidebar.

Code command not found?

Close PowerShell after installing Visual Studio Code. Open a fresh PowerShell window so it can load the updated command path.

Ask for command help: Why can PowerShell not find the Visual Studio Code command?

A virtual environment keeps this project's packages separate from every other Python project on your computer. Activating .venv makes its interpreter the default for this terminal.

  • Click Terminal in the Visual Studio Code menu bar.
  • Click New Terminal.
  • Create and activate .venv by running these commands:
py -m venv .venv
.\.venv\Scripts\Activate.ps1

What do these commands do?

  • The first command creates an isolated Python environment inside .venv.
  • The second command activates that environment in the current PowerShell terminal.

You should see (.venv) at the start of the terminal prompt. That prefix confirms the isolated interpreter is active.

Activation script blocked?

PowerShell can block local activation scripts through its execution policy. The current-user fallback enables locally created scripts without changing the machine-wide policy.

  • Enable local scripts for your Windows account by running:
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

This updates the execution policy for your user account. PowerShell may ask you to confirm the change.

  • Approve the policy change if prompted.
  • Activate the virtual environment again by running:
.\.venv\Scripts\Activate.ps1

You should now see (.venv) at the start of the prompt.

Ask for activation help: Why is my PowerShell virtual environment activation still blocked?

Pin and install the SDK

A dependency pin makes the workspace reproducible. The requirements.txt file records the exact SDK release used by every later step.

  • Click the New File icon in the Visual Studio Code Explorer sidebar.
  • Enter requirements.txt as the file name.
  • Pin the required SDK version by pasting this line:
google-genai==2.31.0

What does this requirement do?

The double equals sign pins the Google Gen AI SDK to release 2.31.0. Future installs now resolve to the same package version.

  • Save requirements.txt by pressing Ctrl+S.
  • Confirm that the editor shows exactly one dependency line.

Requirement line looks different?

Check the package name for its hyphen. Confirm that the version uses two equals signs.

Ask for dependency help: Why is my requirements file not pinning the expected SDK version?

✔️ Awesome, I've got everything!

Great. Double-check that requirements.txt is saved before installing the package.

ⓧ I'd like to double check the full code

google-genai==2.31.0

What should I compare?

Your full file should contain this exact dependency pin. No other requirements belong in the file at this stage.

The activated environment directs package installation into .venv. This keeps the SDK isolated from system-wide Python packages.

  • Install the pinned dependency from requirements.txt by running:
python -m pip install -r requirements.txt

What does this command do?

The active Python interpreter runs pip against the saved requirements file. The package is installed inside the current virtual environment.

The terminal should finish with a successful installation message for the pinned Google Gen AI SDK and its dependencies.

Package installation failed?

Confirm that (.venv) appears in the prompt. Check that your internet connection can reach the Python package registry.

Ask for installation help: Why did the pinned Google Gen AI SDK installation fail in my virtual environment?

  • Confirm that Python can import the installed SDK by running:
python -c "from google import genai"

What does this command do?

Python executes the documented SDK import in a short command. A successful import returns control to PowerShell without an error.

You should return to the (.venv) prompt with no traceback. That quiet result confirms the package can be imported.

Import produced an error?

Check that the install command completed inside the active virtual environment. Confirm that your requirements file uses the package name google-genai.

Ask for import help: Why can Python not import the Google Gen AI SDK after installation?

Store the Gemini API key

An API key authorizes requests to Gemini. This credential deserves extra care because anyone with the value can use it.

You will store the key in your Windows user environment. The key stays out of requirements.txt and every future Python file.

  • Open the official Gemini API key guide in your browser.
  • Follow its link to Google AI Studio.
  • Accept the terms if Google AI Studio prompts you as a new user.
  • Create a Gemini API key from the API Keys page.
  • Copy the new key to your clipboard.

Protect the key

The key appears as a live credential when it is created. Keep it out of screenshots and project files.

  • Press the Windows key to open the search bar.
  • Type Control Panel and press Enter.
  • Select System.
  • Select Advanced System Settings.
  • Select the Advanced tab.
  • Click Environment Variables....
  • Click New... in the user variables section.
  • Enter GEMINI_API_KEY in the Variable name field.

Why use the user environment?

A user-scoped environment variable persists across PowerShell sessions for your Windows account. New processes can read the key without loading it from a project file.

  • Paste the API key into the Variable value field.
  • Click OK in the new variable dialog.
  • Click OK in the environment variables window.
  • Click OK in the system properties window.

When does the key become available?

Your existing terminal keeps the environment values it loaded when it started. A newly opened PowerShell session receives GEMINI_API_KEY from the Windows user environment.

The final check uses a fresh process to prove that the credential persisted. It checks only whether the variable exists, so the secret never appears in the terminal.

  • Close Visual Studio Code.
  • Press the Windows key to open the search bar.
  • Type Windows PowerShell and press Enter.
  • Return to the project with an active virtual environment by running these commands:
cd ~/Desktop/ai-incident-investigator
.\.venv\Scripts\Activate.ps1
code .

What do these commands do?

The new terminal enters the existing project folder. It activates the saved virtual environment before reopening the folder in Visual Studio Code.

Before you run the final checks, do you expect the new PowerShell session to confirm all three parts of the setup?

  • Verify the API key variable and both pinned versions by running:
Test-Path Env:GEMINI_API_KEY
python --version
python -m pip show google-genai

What do these checks prove?

  • The first check confirms that the new terminal inherited the API key variable without printing its value.
  • The second check confirms which Python runtime the active environment uses.
  • The final check displays the installed SDK package details.

You should see True for the environment variable check. You should also see Python 3.15.0 and SDK version 2.31.0.

One of the final checks failed?

A false environment result means the new terminal did not inherit the user variable. Reopen the environment variables window to confirm the variable name.

A missing package result usually means .venv was not activated. Look for (.venv) at the start of the prompt.

Ask for verification help: Which part of my agent workspace setup is failing?

Your workspace now has the exact runtime, isolated SDK, and protected credential it needs. Next up, you will give the model an incident question and watch a prompt-only investigation fall short.

Let a Prompt-Only Investigator Fail

Your Visual Studio Code workspace is ready to send requests to Gemini. You now need a baseline that reveals what a plain model request can actually access.

A plain request receives only the text your program sends. In this step, the incident files stay local while the model gets only the question.

In this step, get ready to:
  • Create two synthetic evidence files for the checkout incident.
  • Build a prompt-only script that sends the incident question to Gemini.
  • Run the baseline to expose its missing evidence access.
Create the synthetic incident evidence

Real incident logs can contain private data. These synthetic files give you a repeatable incident without exposing anyone's systems.

  • Select the ai-incident-investigator folder in the Explorer sidebar from earlier.
  • Click the New Folder icon in the Explorer toolbar.
  • Enter data as the folder name.
  • Press Enter.

You will see the new data folder beneath ai-incident-investigator in the Explorer sidebar.

  • Select the data folder in the Explorer sidebar.
  • Click the New File icon in the Explorer toolbar.
  • Enter checkout.log as the file name.
  • Press Enter.
  • Fill data/checkout.log with the synthetic application events by pasting the evidence below:
2026-10-18T09:00:00Z INFO service=checkout request_id=req-1001 message="checkout started"
2026-10-18T09:00:01Z INFO service=checkout request_id=req-1001 message="inventory reserved"
2026-10-18T09:00:02Z WARN service=checkout request_id=req-1001 message="database connection wait exceeded 2000ms"
2026-10-18T09:00:05Z ERROR service=checkout request_id=req-1001 code=DB_POOL_EXHAUSTED message="no database connection available after 5000ms"
2026-10-18T09:00:05Z ERROR service=checkout request_id=req-1001 status=503 message="checkout failed"
2026-10-18T09:00:06Z INFO service=checkout request_id=req-1002 message="checkout started"
2026-10-18T09:00:08Z WARN service=checkout request_id=req-1002 message="database connection wait exceeded 2000ms"
2026-10-18T09:00:11Z ERROR service=checkout request_id=req-1002 code=DB_POOL_EXHAUSTED message="no database connection available after 5000ms"

What does this evidence show?

The log follows two checkout requests through normal activity before both requests wait for a database connection. Each request ends with an ERROR containing DB_POOL_EXHAUSTED.

One entry connects that failure sequence to status 503. These repeated events give the investigator evidence to compare across separate request IDs.

  • Press Ctrl+S to save data/checkout.log.
  • Check the Explorer sidebar for checkout.log inside data.

You will see checkout.log listed inside the data folder with no unsaved-change indicator on its editor tab.

Is the checkout log missing?

Check that checkout.log sits directly inside data. A file beside the folder has the wrong location.

If the tab still shows an unsaved-change indicator, press Ctrl+S again.

Help me check why my checkout.log file is missing from the data folder.

  • Select the data folder in the Explorer sidebar.
  • Click the New File icon in the Explorer toolbar.
  • Enter runbook.txt as the file name.
  • Press Enter.
  • Fill data/runbook.txt with the incident guidance by pasting the runbook below:
Checkout incident runbook

Signal: DB_POOL_EXHAUSTED
Meaning: The checkout service could not obtain a database connection before its timeout.
Checks:
1. Confirm repeated DB_POOL_EXHAUSTED entries across separate request IDs.
2. Compare connection demand with the configured pool size.
3. Check for slow or leaked database connections.
First actions:
1. Reduce incoming checkout traffic or temporarily increase safe connection capacity.
2. Inspect connection release paths and long-running database queries before making the capacity change permanent.

How does the runbook help?

The runbook gives DB_POOL_EXHAUSTED an operational meaning. It also defines checks that separate observed signals from possible causes.

Its first actions give the investigator a bounded response plan. The final diagnosis can now connect log evidence to approved next steps.

  • Press Ctrl+S to save data/runbook.txt.
  • Check the Explorer sidebar for runbook.txt inside data.

You will see both checkout.log and runbook.txt inside the data folder.

Does the runbook look incomplete?

Confirm that the file starts with Checkout incident runbook and ends with the guidance about long-running database queries.

Help me compare my runbook with the expected evidence file.

Build the prompt-only baseline

A useful baseline isolates exactly what reaches the model. This first script sends one incident question and prints the returned text.

  • Select the ai-incident-investigator folder in the Explorer sidebar.
  • Click the New File icon in the Explorer toolbar.
  • Enter investigator.py as the file name.
  • Press Enter.
  • Fill investigator.py with the prompt-only request by pasting the code below:
from google import genai

MODEL_ID = "gemini-3.8-flash"


def main() -> None:
    question = (
        "Investigate why checkout requests are returning 503 errors. "
        "Cite the evidence you used."
    )

    with genai.Client() as client:
        interaction = client.interactions.create(
            model=MODEL_ID,
            input=question,
        )

    print(interaction.output_text)


if __name__ == "__main__":
    main()

What does this code do?

  • The genai import gives the script access to the installed SDK.
  • The MODEL_ID constant selects gemini-3.8-flash for the request.
  • The question variable contains the only incident information sent to the model.
  • The genai.Client() context manager creates the client and closes it after the request.
  • The interaction.output_text value contains the model's final response.
  • Press Ctrl+S to save investigator.py.
  • Check the Explorer sidebar for investigator.py beside requirements.txt.

You will see investigator.py at the top level of ai-incident-investigator.

Is the Python file in the wrong place?

Move investigator.py beside requirements.txt if you created it inside data. The evidence files are the only files that belong inside data.

Help me verify the file layout for my incident investigator.

This checkpoint lets you compare every project file at the end of the build.

✔️ Awesome, I've got everything!

  • Confirm that requirements.txt and investigator.py appear at the top level of ai-incident-investigator.
  • Confirm that checkout.log and runbook.txt appear inside data.

ⓧ I'd like to double check the full code

google-genai==2.31.0

What should this file contain?

This file pins the Google Gen AI SDK to version 2.31.0. Your installed dependency stays reproducible.

2026-10-18T09:00:00Z INFO service=checkout request_id=req-1001 message="checkout started"
2026-10-18T09:00:01Z INFO service=checkout request_id=req-1001 message="inventory reserved"
2026-10-18T09:00:02Z WARN service=checkout request_id=req-1001 message="database connection wait exceeded 2000ms"
2026-10-18T09:00:05Z ERROR service=checkout request_id=req-1001 code=DB_POOL_EXHAUSTED message="no database connection available after 5000ms"
2026-10-18T09:00:05Z ERROR service=checkout request_id=req-1001 status=503 message="checkout failed"
2026-10-18T09:00:06Z INFO service=checkout request_id=req-1002 message="checkout started"
2026-10-18T09:00:08Z WARN service=checkout request_id=req-1002 message="database connection wait exceeded 2000ms"
2026-10-18T09:00:11Z ERROR service=checkout request_id=req-1002 code=DB_POOL_EXHAUSTED message="no database connection available after 5000ms"

What should this file contain?

This file contains eight timestamped checkout events. The sequence includes normal entries plus repeated database connection failures.

Checkout incident runbook

Signal: DB_POOL_EXHAUSTED
Meaning: The checkout service could not obtain a database connection before its timeout.
Checks:
1. Confirm repeated DB_POOL_EXHAUSTED entries across separate request IDs.
2. Compare connection demand with the configured pool size.
3. Check for slow or leaked database connections.
First actions:
1. Reduce incoming checkout traffic or temporarily increase safe connection capacity.
2. Inspect connection release paths and long-running database queries before making the capacity change permanent.

What should this file contain?

This file explains the incident signal. It also supplies three checks plus two first actions.

from google import genai

MODEL_ID = "gemini-3.8-flash"


def main() -> None:
    question = (
        "Investigate why checkout requests are returning 503 errors. "
        "Cite the evidence you used."
    )

    with genai.Client() as client:
        interaction = client.interactions.create(
            model=MODEL_ID,
            input=question,
        )

    print(interaction.output_text)


if __name__ == "__main__":
    main()

What should this file contain?

This script sends the incident question to Gemini. It prints the returned text after closing the client.

Run the baseline and inspect the gap

Before you run this, predict whether the model can quote evidence that your Python code never supplied.

  • Run the prompt-only investigator in the activated PowerShell terminal from earlier by running this command:
python investigator.py

What does this run prove?

You will see a model response in the terminal. You will not see any trace lines beginning with [tool].

The response may sound plausible, but the model received only the incident question. This version has no code path that lists or reads data/checkout.log or data/runbook.txt.

  • Compare the response with the exact events in data/checkout.log.
  • Check whether the response cites the runbook guidance from data/runbook.txt.
  • Inspect the terminal for any line beginning with [tool].

The baseline produces fluent text without a trace of local evidence access. Any matching diagnosis comes from the question's wording instead of inspected files.

Did the request fail before returning text?

Confirm that the terminal still shows the activated .venv environment. Use the newly opened PowerShell session from the setup step so it can access GEMINI_API_KEY.

Check that investigator.py matches the full-code checkpoint. A missing parenthesis or changed import prevents the request from running.

Help me diagnose why my prompt-only investigator did not return a model response.

That failure is useful. Your model request works while its response still lacks inspected evidence.

Next, you will give the investigator narrow local tools that can read the synthetic evidence without exposing the rest of your filesystem.

Build Safe Local Evidence Tools

Your prompt-only baseline produced a Gemini response. It also proved the model has no route into data\checkout.log or data\runbook.txt.

This step gives Python a narrow filesystem boundary. Each local tool checks the requested filename before reading evidence. This boundary becomes the safe foundation for function calling.

In this step, get ready to:
  • Restrict evidence access to direct .log or .txt files inside data.
  • Implement tools for listing files, reading evidence, and counting log levels.
  • Prove the local tools work without sending a request to Gemini.
Create the filesystem boundary

An unrestricted reader could expose files that have nothing to do with the incident. The resolve_incident_file() function creates one gate for every filename.

Why use one shared boundary?

Every reading tool passes through the same validation function. A single gate keeps the safety rules consistent as the tool surface grows.

The gate accepts a filename only. It also checks the resolved parent directory and the allowed file suffixes.

  • Switch back to investigator.py in your editor.
  • Find this opening code:
from google import genai

MODEL_ID = "gemini-3.8-flash"

What are you locating?

This is the current import and model configuration from the prompt-only baseline. You are about to extend it with argument handling and safe path configuration.

  • Replace that opening code with this expanded setup:
import sys
from pathlib import Path

from google import genai

MODEL_ID = "gemini-3.8-flash"
DATA_DIR = (Path(__file__).parent / "data").resolve()
ALLOWED_SUFFIXES = {".log", ".txt"}

What does this setup do?

  • The sys import lets the script detect the local test flag.
  • DATA_DIR stores the resolved location of the existing data folder.
  • ALLOWED_SUFFIXES limits readable evidence to log and text files.
  • Find def main() -> None: near the bottom of investigator.py.
  • Insert these boundary and listing functions directly above that line:
def resolve_incident_file(filename: str) -> Path:
    if Path(filename).name != filename:
        raise ValueError("Use a filename only, not a path.")

    candidate = (DATA_DIR / filename).resolve()
    if candidate.parent != DATA_DIR:
        raise ValueError("File access is limited to the data directory.")
    if candidate.suffix.lower() not in ALLOWED_SUFFIXES:
        raise ValueError("Only .log and .txt files are allowed.")
    if not candidate.is_file():
        raise FileNotFoundError(f"No incident file named {filename}.")
    return candidate


def list_incident_files() -> dict[str, object]:
    files = sorted(
        path.name
        for path in DATA_DIR.iterdir()
        if path.is_file() and path.suffix.lower() in ALLOWED_SUFFIXES
    )
    return {"files": files}

How does the boundary work?

  • resolve_incident_file() rejects directory components before resolving the requested file.
  • The parent check confirms that the resolved file remains a direct child of DATA_DIR.
  • list_incident_files() returns only allowed files in sorted order.
  • Save investigator.py.

Before you run this check, do you think the existing main() function knows what --test-tools means?

  • Test the current script from the activated PowerShell terminal by running:
python investigator.py --test-tools

What happened at this checkpoint?

You still see a model response because the existing main() function ignores the new flag. The boundary functions loaded without stopping the script.

This result exposes the remaining gap. The script needs a deterministic test route that calls the local functions directly.

Did the script stop before responding?

Check that both imports appear above the Google import. Confirm that DATA_DIR and ALLOWED_SUFFIXES sit below MODEL_ID.

Compare the indentation inside resolve_incident_file() and list_incident_files().

Help me fix the new filesystem boundary in investigator.py.

Add evidence reading and counting

The boundary decides which file is safe. The evidence tools can now reuse that decision for two focused jobs.

  • Find def main() -> None: again.
  • Insert these evidence functions directly above that line:
def read_incident_file(filename: str) -> dict[str, object]:
    path = resolve_incident_file(filename)
    return {
        "filename": path.name,
        "content": path.read_text(encoding="utf-8"),
    }


def count_log_levels(filename: str) -> dict[str, object]:
    path = resolve_incident_file(filename)
    if path.suffix.lower() != ".log":
        raise ValueError("Log-level counting requires a .log file.")

    counts = {"INFO": 0, "WARN": 0, "ERROR": 0}
    for line in path.read_text(encoding="utf-8").splitlines():
        for level in counts:
            if f" {level} " in line:
                counts[level] += 1

    return {"filename": path.name, "counts": counts}

What do these tools return?

  • read_incident_file() returns the validated filename with its text content.
  • count_log_levels() accepts only a validated log file.
  • The nested loops count lines containing the INFO, WARN, or ERROR marker.
  • Save investigator.py.
  • Confirm the new functions parse by running the same checkpoint command:
python investigator.py --test-tools

What does this run confirm?

The script reaches the existing model request again. That response confirms Python parsed the reading and counting functions successfully.

Did the new functions block the script?

Check that both functions sit above main(). Make sure every nested block uses consistent indentation.

Help me debug the evidence reading and log counting functions.

Wire the deterministic test path

A deterministic test path calls the local functions with known inputs. This lets you verify file safety and counting before a model can choose any tool.

  • Select the existing main() function through the final main() call.
  • Replace that selection with this local test path:
def run_tool_tests() -> None:
    print(list_incident_files())
    print(read_incident_file("checkout.log"))
    print(read_incident_file("runbook.txt"))
    print(count_log_levels("checkout.log"))


def main() -> None:
    if len(sys.argv) > 1 and sys.argv[1] == "--test-tools":
        run_tool_tests()
        return

    print("Run python investigator.py --test-tools to verify local evidence tools.")


if __name__ == "__main__":
    main()

How does the test path work?

  • run_tool_tests() lists the safe evidence files.
  • The same function reads both existing evidence files.
  • It finishes by counting the levels in checkout.log.
  • main() runs those checks only when the terminal argument matches --test-tools.
  • Save investigator.py.

✔️ Awesome, I've got everything!

Your local evidence functions and test route are ready. Make sure investigator.py is saved.

ⓧ I'd like to double check the full code

import sys
from pathlib import Path

from google import genai

MODEL_ID = "gemini-3.8-flash"
DATA_DIR = (Path(__file__).parent / "data").resolve()
ALLOWED_SUFFIXES = {".log", ".txt"}


def resolve_incident_file(filename: str) -> Path:
    if Path(filename).name != filename:
        raise ValueError("Use a filename only, not a path.")

    candidate = (DATA_DIR / filename).resolve()
    if candidate.parent != DATA_DIR:
        raise ValueError("File access is limited to the data directory.")
    if candidate.suffix.lower() not in ALLOWED_SUFFIXES:
        raise ValueError("Only .log and .txt files are allowed.")
    if not candidate.is_file():
        raise FileNotFoundError(f"No incident file named {filename}.")
    return candidate


def list_incident_files() -> dict[str, object]:
    files = sorted(
        path.name
        for path in DATA_DIR.iterdir()
        if path.is_file() and path.suffix.lower() in ALLOWED_SUFFIXES
    )
    return {"files": files}


def read_incident_file(filename: str) -> dict[str, object]:
    path = resolve_incident_file(filename)
    return {
        "filename": path.name,
        "content": path.read_text(encoding="utf-8"),
    }


def count_log_levels(filename: str) -> dict[str, object]:
    path = resolve_incident_file(filename)
    if path.suffix.lower() != ".log":
        raise ValueError("Log-level counting requires a .log file.")

    counts = {"INFO": 0, "WARN": 0, "ERROR": 0}
    for line in path.read_text(encoding="utf-8").splitlines():
        for level in counts:
            if f" {level} " in line:
                counts[level] += 1

    return {"filename": path.name, "counts": counts}


def run_tool_tests() -> None:
    print(list_incident_files())
    print(read_incident_file("checkout.log"))
    print(read_incident_file("runbook.txt"))
    print(count_log_levels("checkout.log"))


def main() -> None:
    if len(sys.argv) > 1 and sys.argv[1] == "--test-tools":
        run_tool_tests()
        return

    print("Run python investigator.py --test-tools to verify local evidence tools.")


if __name__ == "__main__":
    main()

How should I use this reference?

Compare the function order and indentation with your saved file. The complete reference also shows the exact test inputs used by run_tool_tests().

Before you run the finished test path, what totals do you expect from the eight lines in checkout.log?

  • Verify every local evidence tool by running:
python investigator.py --test-tools

What should you see?

You see checkout.log and runbook.txt in the sorted file list. The next two results contain the synthetic log and runbook text.

The final result reports INFO as 3. It reports WARN as 2. It reports ERROR as 3.

You see local dictionaries instead of a model response. You also see no [tool] trace because Gemini did not choose or execute these checks.

Do the local tool tests fail?

Confirm that the existing evidence files remain inside the data folder beside investigator.py.

If the counts differ, compare data\checkout.log with the supplied eight-line evidence file. Each level must have spaces around it for the counter to match.

Help me fix the local evidence tool test output.

That is the safety layer working. Your script can now inspect the synthetic incident evidence through controlled functions without exposing arbitrary files.

Your deterministic tools are proven. Next, you will let Gemini select them while your Python code keeps control of execution.

Wire the Agentic Tool Loop

Your safe local tools can already list files, read evidence, and count log levels. Those functions still need an orchestration loop before Gemini can select them during an investigation.

In this step, you will expose the tools through function calling. Your program will execute each request locally, return the serialized result, and continue until Gemini produces an evidence-based diagnosis.

In this step, get ready to:
  • Define the evidence-first instructions and tool declarations.
  • Dispatch Gemini function calls through the safe Python functions.
  • Trace each tool call before printing the final diagnosis.
Define the agent's evidence contract

The agent needs a contract that describes its available capabilities. The system instruction also requires evidence before conclusions.

  • In investigator.py, select the imports through the ALLOWED_SUFFIXES line.
  • Replace the selection by pasting this code:
import json
import sys
from pathlib import Path

from google import genai

MODEL_ID = "gemini-3.8-flash"
DATA_DIR = (Path(__file__).parent / "data").resolve()
ALLOWED_SUFFIXES = {".log", ".txt"}
SYSTEM_INSTRUCTION = """You are an incident investigator.
Use the available tools before reaching a conclusion.
Base every claim on tool results, distinguish evidence from inference, cite filenames and exact log text, and recommend no more than two first actions.
Never claim that you inspected a file unless a tool returned its contents.
"""

What does this code do?

  • The json import supports JSON serialization when local tool results return to Gemini.
  • The SYSTEM_INSTRUCTION requires the model to inspect tool results before reaching a conclusion.
  • The instruction also separates observed evidence from inference.
  • Save investigator.py.
  • Confirm the deterministic tool tests still run by entering this command:
python investigator.py --test-tools

What does this check prove?

This command stays on the local test path. Seeing the four familiar results proves the new import and instruction did not break your existing tools.

You should still see the file list, both file contents, and the log-level totals.

Seeing a syntax error?

Check that SYSTEM_INSTRUCTION starts and ends with three double quotes. Confirm that import json sits above import sys.

Compare the top of your file with the code block before changing anything else.

Help me fix the syntax near SYSTEM_INSTRUCTION in investigator.py.

A function declaration tells Gemini what a tool is called and which arguments it accepts. The local mapping controls which Python function actually runs for each declared name.

  • In investigator.py, locate the end of count_log_levels().
  • Insert the first two declarations below that function by pasting this code:
TOOLS = [
    {
        "type": "function",
        "name": "list_incident_files",
        "description": "List the incident evidence files available in the safe data directory.",
        "parameters": {"type": "object", "properties": {}},
    },
    {
        "type": "function",
        "name": "read_incident_file",
        "description": "Read one allowed incident evidence file by filename.",
        "parameters": {
            "type": "object",
            "properties": {
                "filename": {
                    "type": "string",
                    "description": "A filename returned by list_incident_files.",
                }
            },
            "required": ["filename"],
        },
    },
]

How do these declarations work?

  • Each declaration gives Gemini a tool name and a description of its purpose.
  • The parameters schema states whether the tool needs a filename.
  • These declarations describe capabilities without granting direct filesystem access.
  • Find the closing square bracket at the end of TOOLS.
  • Replace that closing bracket by pasting this code:
    {
        "type": "function",
        "name": "count_log_levels",
        "description": "Count INFO, WARN, and ERROR entries in an allowed .log file.",
        "parameters": {
            "type": "object",
            "properties": {
                "filename": {
                    "type": "string",
                    "description": "An allowed .log filename.",
                }
            },
            "required": ["filename"],
        },
    },
]

AVAILABLE_FUNCTIONS = {
    "list_incident_files": list_incident_files,
    "read_incident_file": read_incident_file,
    "count_log_levels": count_log_levels,
}

Why keep a separate function map?

Gemini returns a function name as data. AVAILABLE_FUNCTIONS resolves that name to one of your controlled Python functions.

The model can only reach functions registered in this mapping. Each registered function still enforces the existing boundary around DATA_DIR.

  • Save investigator.py.
  • Check the completed declarations without contacting Gemini by running:
python investigator.py --test-tools

What should remain unchanged?

You should see the same deterministic evidence output as before. This confirms that the declarations and function map load without changing the safe local behavior.

Your tool contract is now connected to the exact functions you tested in the previous step.

Tool tests no longer start?

Check the commas between the three objects in TOOLS. Confirm that AVAILABLE_FUNCTIONS begins after the closing square bracket.

Help me compare my TOOLS list and AVAILABLE_FUNCTIONS mapping with the expected structure.

Why use the direct API loop?

A visual tool such as n8n can manage much of the orchestration for you. The direct Interactions API exposes each request, local execution, result, and continuation.

Implementing the loop yourself makes the control boundary visible in code.

Build the dispatch and continuation loop

Gemini does not execute your local functions. It returns a structured request that your program must inspect and dispatch.

The loop repeats that exchange until the interaction contains final text instead of another function call. An eight-turn limit prevents an investigation from continuing forever.

  • In investigator.py, locate run_tool_tests().
  • Insert the start of run_agent() below run_tool_tests() by pasting this code:
def run_agent(question: str) -> str:
    current_input: object = question
    previous_interaction_id = None

    with genai.Client(http_options={"api_version": "v1"}) as client:
        for _ in range(8):
            interaction = client.interactions.create(
                model=MODEL_ID,
                input=current_input,
                tools=TOOLS,
                system_instruction=SYSTEM_INSTRUCTION,
                previous_interaction_id=previous_interaction_id,
            )

How does the interaction start?

  • The client uses the stable v1 API.
  • The first current_input value contains the learner's incident question.
  • Every interaction receives the tools and evidence-first instruction again.
  • The previous_interaction_id value starts empty because no earlier interaction exists.
  • Place your cursor immediately after the closing parenthesis of client.interactions.create().
  • Add the dispatch logic at the same indentation level as interaction by pasting this code:
            function_results = []
            for step in interaction.steps:
                if step.type != "function_call":
                    continue

                print(f"[tool] {step.name}({step.arguments})")
                function = AVAILABLE_FUNCTIONS.get(step.name)

                try:
                    if function is None:
                        result = {"error": f"Unknown tool: {step.name}"}
                    else:
                        result = function(**step.arguments)
                except Exception as exc:
                    result = {"error": str(exc)}

What does the dispatcher control?

  • The loop ignores interaction steps that are not function calls.
  • The [tool] trace exposes the selected function and its arguments.
  • The mapping retrieves only a registered local function.
  • The exception handler converts a local failure into a result the model can inspect.
  • Save investigator.py.
  • Check the file for syntax problems by running the local tests:
python investigator.py --test-tools

Why use the local test path here?

The local test path imports the entire file before running the deterministic tools. Seeing all four results proves the new loop structure parses correctly without starting an API investigation.

You should see the two filenames, both evidence files, and the INFO, WARN, and ERROR totals.

Seeing an indentation problem?

Keep function_results inside the eight-turn loop. Keep the try block inside the loop over interaction.steps.

Help me fix the nesting inside run_agent without changing its behavior.

A local return value must be serialized before it can become the next interaction input. The continuation ID keeps the tool request and tool result inside one server-side conversation.

  • Place your cursor after the except Exception as exc block in run_agent().
  • Complete the result and continuation logic by pasting this code:
                function_results.append(
                    {
                        "type": "function_result",
                        "name": step.name,
                        "call_id": step.id,
                        "result": [
                            {"type": "text", "text": json.dumps(result)}
                        ],
                    }
                )

            if not function_results:
                return interaction.output_text or "The model returned no final text."

            current_input = function_results
            previous_interaction_id = interaction.id

    raise RuntimeError("The agent exceeded the eight-turn safety limit.")

How does the loop continue?

  • Each function_result links back to the requested call through step.id.
  • The local dictionary becomes text through json.dumps().
  • An interaction with no function results returns the model's final answer.
  • An interaction with tool results becomes the input for the next turn.
Connect the loop to the terminal

The orchestration loop now exists, but main() still prints the local-test reminder. The final edit collects an incident question and passes it into run_agent().

  • In main(), find this placeholder line:
    print("Run python investigator.py --test-tools to verify local evidence tools.")

Why replace this line?

This line belongs to the deterministic test stage. The completed entry point needs to accept a question and send it through the agent loop.

  • Replace the placeholder line with this completed terminal flow:
    default_question = (
        "Investigate why checkout requests are returning 503 errors. "
        "Inspect the available evidence, identify the most likely cause, "
        "and recommend the first two actions. Cite filenames and exact log text."
    )
    question = input("Incident question: ").strip() or default_question
    print("\nInvestigation\n")
    print(run_agent(question))

How does the terminal flow work?

  • The default question asks for a cause, evidence, and two first actions.
  • A blank response at the prompt selects that default question.
  • A typed response lets you investigate a different question with the same safe tools.
  • The final print displays the evidence-based answer returned by run_agent().
  • Save investigator.py.
  • Compare your completed file with the reference below.

✔️ Awesome, I've got everything!

Your file now contains the complete three-tool agent loop. Make sure investigator.py is saved before running it.

ⓧ I'd like to double check the full code

import json
import sys
from pathlib import Path

from google import genai

MODEL_ID = "gemini-3.8-flash"
DATA_DIR = (Path(__file__).parent / "data").resolve()
ALLOWED_SUFFIXES = {".log", ".txt"}
SYSTEM_INSTRUCTION = """You are an incident investigator.
Use the available tools before reaching a conclusion.
Base every claim on tool results, distinguish evidence from inference, cite filenames and exact log text, and recommend no more than two first actions.
Never claim that you inspected a file unless a tool returned its contents.
"""


def resolve_incident_file(filename: str) -> Path:
    if Path(filename).name != filename:
        raise ValueError("Use a filename only, not a path.")

    candidate = (DATA_DIR / filename).resolve()
    if candidate.parent != DATA_DIR:
        raise ValueError("File access is limited to the data directory.")
    if candidate.suffix.lower() not in ALLOWED_SUFFIXES:
        raise ValueError("Only .log and .txt files are allowed.")
    if not candidate.is_file():
        raise FileNotFoundError(f"No incident file named {filename}.")
    return candidate


def list_incident_files() -> dict[str, object]:
    files = sorted(
        path.name
        for path in DATA_DIR.iterdir()
        if path.is_file() and path.suffix.lower() in ALLOWED_SUFFIXES
    )
    return {"files": files}


def read_incident_file(filename: str) -> dict[str, object]:
    path = resolve_incident_file(filename)
    return {
        "filename": path.name,
        "content": path.read_text(encoding="utf-8"),
    }


def count_log_levels(filename: str) -> dict[str, object]:
    path = resolve_incident_file(filename)
    if path.suffix.lower() != ".log":
        raise ValueError("Log-level counting requires a .log file.")

    counts = {"INFO": 0, "WARN": 0, "ERROR": 0}
    for line in path.read_text(encoding="utf-8").splitlines():
        for level in counts:
            if f" {level} " in line:
                counts[level] += 1

    return {"filename": path.name, "counts": counts}


TOOLS = [
    {
        "type": "function",
        "name": "list_incident_files",
        "description": "List the incident evidence files available in the safe data directory.",
        "parameters": {"type": "object", "properties": {}},
    },
    {
        "type": "function",
        "name": "read_incident_file",
        "description": "Read one allowed incident evidence file by filename.",
        "parameters": {
            "type": "object",
            "properties": {
                "filename": {
                    "type": "string",
                    "description": "A filename returned by list_incident_files.",
                }
            },
            "required": ["filename"],
        },
    },
    {
        "type": "function",
        "name": "count_log_levels",
        "description": "Count INFO, WARN, and ERROR entries in an allowed .log file.",
        "parameters": {
            "type": "object",
            "properties": {
                "filename": {
                    "type": "string",
                    "description": "An allowed .log filename.",
                }
            },
            "required": ["filename"],
        },
    },
]

AVAILABLE_FUNCTIONS = {
    "list_incident_files": list_incident_files,
    "read_incident_file": read_incident_file,
    "count_log_levels": count_log_levels,
}


def run_tool_tests() -> None:
    print(list_incident_files())
    print(read_incident_file("checkout.log"))
    print(read_incident_file("runbook.txt"))
    print(count_log_levels("checkout.log"))


def run_agent(question: str) -> str:
    current_input: object = question
    previous_interaction_id = None

    with genai.Client(http_options={"api_version": "v1"}) as client:
        for _ in range(8):
            interaction = client.interactions.create(
                model=MODEL_ID,
                input=current_input,
                tools=TOOLS,
                system_instruction=SYSTEM_INSTRUCTION,
                previous_interaction_id=previous_interaction_id,
            )

            function_results = []
            for step in interaction.steps:
                if step.type != "function_call":
                    continue

                print(f"[tool] {step.name}({step.arguments})")
                function = AVAILABLE_FUNCTIONS.get(step.name)

                try:
                    if function is None:
                        result = {"error": f"Unknown tool: {step.name}"}
                    else:
                        result = function(**step.arguments)
                except Exception as exc:
                    result = {"error": str(exc)}

                function_results.append(
                    {
                        "type": "function_result",
                        "name": step.name,
                        "call_id": step.id,
                        "result": [
                            {"type": "text", "text": json.dumps(result)}
                        ],
                    }
                )

            if not function_results:
                return interaction.output_text or "The model returned no final text."

            current_input = function_results
            previous_interaction_id = interaction.id

    raise RuntimeError("The agent exceeded the eight-turn safety limit.")


def main() -> None:
    if len(sys.argv) > 1 and sys.argv[1] == "--test-tools":
        run_tool_tests()
        return

    default_question = (
        "Investigate why checkout requests are returning 503 errors. "
        "Inspect the available evidence, identify the most likely cause, "
        "and recommend the first two actions. Cite filenames and exact log text."
    )
    question = input("Incident question: ").strip() or default_question
    print("\nInvestigation\n")
    print(run_agent(question))


if __name__ == "__main__":
    main()

Before you run the investigator, which safe tools do you expect Gemini to select before it can explain the checkout failures?

  • Start the completed investigator by running this command:
python investigator.py

What happens after this command?

The script waits at the incident-question prompt. Leaving the prompt blank selects the evidence-focused default question.

  • Press Enter at the Incident question prompt.

You should see at least one [tool] trace before the final answer. The diagnosis should refer to checkout.log and runbook.txt.

That closes the agent loop. Gemini can now request evidence while your Python boundary decides what it is allowed to read.

  • Start another investigation by running the script again:
python investigator.py

Why run a second investigation?

A second question confirms that the terminal accepts input beyond the default prompt. The same tool boundary applies to every investigation.

  • Type one follow-up question about the supplied incident evidence.
  • Press Enter to run the investigation.

You should see fresh tool traces followed by an answer grounded in the same two safe evidence files.

Missing tool traces or a diagnosis?

If no [tool] lines appear, confirm that every interaction receives TOOLS and SYSTEM_INSTRUCTION.

If the request cannot start, confirm that your current PowerShell terminal still has the activated .venv and the Gemini API key from earlier.

Help me diagnose why my agent does not show tool traces or return a final diagnosis.

Secret mission

Add Evidence Search with Line Numbers

Full-file reads become noisy as incident evidence grows. Add a focused search tool that returns matching filenames, line numbers, and text without weakening the existing file boundary.

Clean Up Your Resources

Clean Up Your Resources

This project uses the Gemini Developer API free tier, so there are no ongoing project charges after you stop running the agent. Stored free-tier interactions expire automatically after one day.

Resources you used:

  • The ai-incident-investigator folder on your Windows Desktop.
  • A Gemini API key managed through Google AI Studio.
  • A Windows user environment variable named GEMINI_API_KEY.
  • Stored Gemini Interactions API records with one-day automatic retention.

Keep everything running

No action needed. Choose this if you want to continue testing the investigator or reuse its tool loop in another project.

  • Keep the ai-incident-investigator folder on your Desktop.
  • Leave the Gemini API key active in Google AI Studio.
  • Keep GEMINI_API_KEY in your Windows user environment.

Your stored free-tier interaction records still expire automatically after one day. Your Google AI Studio project stays available for future API keys.

Pause - I'll come back to this later

Shut down the active virtual environment to free the terminal. Your project folder remains ready for your return.

  • Leave the active virtual environment by running this command:
deactivate

What does this command do?

The command returns PowerShell to its normal Python environment. The virtual environment marker disappears from your terminal prompt.

  • Close the Visual Studio Code window from earlier if it is still open.
  • Close the PowerShell terminal.
  • Return later by opening the ai-incident-investigator folder in Visual Studio Code.
  • Reactivate the project from its PowerShell terminal by running this command:
.\.venv\Scripts\Activate.ps1

What happens when you return?

The command selects the Python environment stored inside .venv. A new terminal session also loads your Windows user environment variables.

Delete - I don't want to use this again

Deleting the local folder and revoking its API key are permanent changes. Your Google AI Studio project remains available after this cleanup.

  • Copy any code you want to keep to another location.
  • Leave the active virtual environment by running this command:
deactivate

What does this command do?

This command leaves the virtual environment. Your project files remain in the ai-incident-investigator folder until you delete it.

  • Close the PowerShell terminal.
  • Close the Visual Studio Code window from earlier if it is still open.
  • Select the ai-incident-investigator folder on your Windows Desktop.
  • Press the Delete key to move the folder to the Recycle Bin.
  • Open the Recycle Bin from your Windows Desktop.
  • Empty the Recycle Bin to remove the folder permanently.

That is the local cleanup finished. The ai-incident-investigator workspace is now gone from your computer.

  • Visit the API Keys page in Google AI Studio.
  • Find the Gemini API key used for this project.
  • Use the key's delete control to revoke it.
  • Confirm the key removal.

The revoked key can no longer authorize requests. Remove its local environment entry next.

  • Press the Windows key to open search.
  • Search for the user environment variable editor.
  • Open the matching Windows settings result.
  • Select GEMINI_API_KEY from your user variables.
  • Remove the selected environment variable.
  • Confirm the settings change.

Google removes stored free-tier interaction records automatically after one day. Your Google AI Studio project remains available without the deleted key.

Nice Work!

Nice Work!

You did it! Your Python incident investigator now produces an evidence-based diagnosis for the synthetic checkout failure.

You've learned how to:

  • Run a prompt-only baseline that exposes why a Gemini request cannot inspect local evidence without tools. The missing [tool] traces make that limitation visible.
  • Build a safe filesystem boundary that limits reads to direct .log and .txt children of data.
  • Implement a function-calling loop that dispatches Gemini requests to controlled Python functions. The loop returns serialized results to the model. Server-side conversation state carries the investigation across tool turns. Each [tool] trace reveals how the final diagnosis was reached.
  • Secret Mission: Added search_incident_files() to return filenames, line numbers, and matching text without weakening the existing file boundary.

Ready to quiz yourself?