Build an AI Triage Control Room

Introduction

30 Second Summary

A stream of software issues can look manageable until one urgent report gets buried among routine requests. Treating every report the same can hide risk beneath routine work.

In this project, you will turn your Part 2 Gemini classifier into an AI operations control room that sends structured AI output to Slack. Confidence-based escalation automates safe decisions while urgent or uncertain tickets stop for human review.

What You'll Build

Your demo turns a software ticket into a visible priority and confidence decision before a formatted routing message reaches the correct Slack channel.

By the end of this project, you'll have:

• Inspect every decision in the terminal with its chosen priority, team, confidence, and reasoning.
• Control automation risk through rules that send safe work to #team-routing. Urgent or low-confidence work goes to #triage-review for a person to check.
• Prove the architecture with a gate that catches boundary violations locally and in GitHub Actions. Replay 20 real issues in shadow mode to generate results.json and HANDOFF.md with agreement, coverage, review volume, and false automation risk.
• Secret Mission: Schedule a weekly Slack insight that posts the latest evaluation metrics through GitHub Actions.

Are there any prerequisites?

This project builds on Part 2, so you need its completed Gemini router on Linux plus a working virtual environment with Python 3.10+. You also need a GitHub account and an existing GEMINI_API_KEY.

Before We Start

You are building an AI triage control room for operators who route software tickets. Urgent or low-confidence decisions stop for human review so uncertain automation never acts silently.

Create the Slack Control Room

A model decision only helps when operators can see where it went. Your Part 2 router needs shared destinations for automated routing and human review.

In this step, you will create a Slack control room with one channel for automated routing. A second channel keeps urgent or low-confidence tickets visible to reviewers.

In this step, get ready to:

• Create a free workspace with two public routing channels.
• Configure a channel-specific incoming webhook for each destination.
• Prove the local configuration by sending one test message to each channel.

Create your Slack workspace and channels

The control room needs a shared workspace before it can receive routing decisions. Slack Free provides the workspace needed for this project.

• Visit the Slack workspace creation page.
• Enter the email address you want to use for the workspace.
• Click Continue.
• Enter the confirmation code sent to your email.
• Click Create a Workspace.

Your workspace is ready. It now has a home for the control room channels.

Why Use Two Channels?

The team channel is the destination for decisions that meet the automation policy. The review channel is a visible queue for decisions that need a person.

• Click the plus sign in the Slack sidebar.
• Select Channel.
• Enter team-routing as the channel name.
• Choose the public channel option.
• Click Create.

What Will This Channel Show?

The #team-routing channel shows tickets that the policy allows the control room to automate. Keeping these messages together makes automated decisions easy to monitor.

• Click the plus sign in the Slack sidebar again.
• Select Channel.
• Enter triage-review as the channel name.
• Choose the public channel option.
• Click Create.

Good progress. Your workspace now separates routine automation from tickets that require human judgment.

Connect each channel to a webhook

An incoming webhook gives an external program a channel-specific delivery address. Slack fixes each webhook to the channel selected during authorization.

• Create a Slack app named AI Triage Control Room for your new workspace.
• Open Incoming Webhooks in the app settings.
• Turn on Activate Incoming Webhooks.
• Click Add New Webhook to Workspace.
• Choose #team-routing as the destination.
• Select Authorize.

Protect Your Webhook URLs

Each webhook URL contains a secret that grants permission to post into its assigned channel. Keep both URLs out of your repository and public messages.

• Return to Incoming Webhooks in the app settings.
• Click Add New Webhook to Workspace again.
• Choose #triage-review as the destination.
• Select Authorize.

You should now see two webhook entries in the app settings. Each entry should show its assigned channel.

Protect and test the webhook configuration

A local configuration file keeps the webhook URLs outside your repository. Linux file permissions restrict that file to your user account.

• Return to your completed Part 2 repository in VS Code.
• Create a scripts folder at the top level of the repository if that folder is missing.
• Create configure_slack.py inside the scripts folder.
• Add the configuration script by pasting this code into scripts/configure_slack.py:

import json
from getpass import getpass
from pathlib import Path

CONFIG_DIR = Path.home() / ".config" / "ai-triage-control-room"
CONFIG_PATH = CONFIG_DIR / "slack-webhooks.json"


def read_webhook(prompt: str) -> str:
    value = getpass(prompt).strip()
    if not value.startswith("https://"):
        raise SystemExit("A webhook must be an HTTPS URL.")
    return value


def main() -> None:
    CONFIG_DIR.mkdir(parents=True, exist_ok=True)
    config = {
        "team-routing": read_webhook("Webhook for #team-routing: "),
        "triage-review": read_webhook("Webhook for #triage-review: "),
    }
    CONFIG_PATH.write_text(json.dumps(config, indent=2), encoding="utf-8")
    CONFIG_PATH.chmod(0o600)
    print(f"Saved protected Slack configuration to {CONFIG_PATH}")


if __name__ == "__main__":
    main()

What Does This Code Do?

• The CONFIG_PATH value places the file at ~/.config/ai-triage-control-room/slack-webhooks.json.
• The getpass() prompts keep the webhook URLs hidden while you paste them.
• The two configuration keys match the channel destinations used by the control room.
• The chmod(0o600) call limits file access to your Linux user.

This next part handles live webhook credentials. The prompts hide each URL while you paste it.

• Save scripts/configure_slack.py.
• Start the protected configuration prompts by running this command from your repository folder:

python scripts/configure_slack.py

• Copy the webhook URL assigned to #team-routing from the Slack app settings.
• Paste that URL at the first terminal prompt.
• Press Enter.
• Copy the webhook URL assigned to #triage-review from the Slack app settings.
• Paste that URL at the second terminal prompt.
• Press Enter.

You should see a confirmation that the protected Slack configuration was saved under ~/.config/ai-triage-control-room/slack-webhooks.json.

Configuration Not Saving?

• Copy the complete webhook URL from the Slack app settings if the script rejects the value.
• Confirm that each URL starts with https://.
• Run the command from the repository folder that contains the scripts directory.
• Help me debug my Slack webhook configuration.

✔️ My Slack configuration is saved

Your two webhook URLs are stored outside the repository. The protected configuration is ready for the smoke test.

ⓧ I'd like to double check the full code

Compare scripts/configure_slack.py with this complete file.

import json
from getpass import getpass
from pathlib import Path

CONFIG_DIR = Path.home() / ".config" / "ai-triage-control-room"
CONFIG_PATH = CONFIG_DIR / "slack-webhooks.json"


def read_webhook(prompt: str) -> str:
    value = getpass(prompt).strip()
    if not value.startswith("https://"):
        raise SystemExit("A webhook must be an HTTPS URL.")
    return value


def main() -> None:
    CONFIG_DIR.mkdir(parents=True, exist_ok=True)
    config = {
        "team-routing": read_webhook("Webhook for #team-routing: "),
        "triage-review": read_webhook("Webhook for #triage-review: "),
    }
    CONFIG_PATH.write_text(json.dumps(config, indent=2), encoding="utf-8")
    CONFIG_PATH.chmod(0o600)
    print(f"Saved protected Slack configuration to {CONFIG_PATH}")


if __name__ == "__main__":
    main()

The smoke test sends a distinct setup message through each stored webhook. Seeing both messages proves that the channel mappings work before Gemini enters the loop.

• Create smoke_test_slack.py inside the scripts folder.
• Add the smoke test by pasting this code into scripts/smoke_test_slack.py:

from control_room.infrastructure import SlackNotifier


def main() -> None:
    notifier = SlackNotifier()
    notifier.send_text(
        "team-routing",
        "AI Triage Control Room setup complete: automated routing is online.",
    )
    notifier.send_text(
        "triage-review",
        "AI Triage Control Room setup complete: human review is online.",
    )
    print("Sent one test message to each Slack channel.")


if __name__ == "__main__":
    main()

What Does This Code Do?

• The SlackNotifier loads the two webhook URLs from the protected configuration file.
• The first send_text() call targets team-routing.
• The second send_text() call targets triage-review.
• The final terminal message confirms that both posting requests completed.

• Save scripts/smoke_test_slack.py.

Before you run the smoke test, which two Slack channels do you expect to update?

• Send one test message to each destination by running this command from your repository folder:

python scripts/smoke_test_slack.py

Your terminal should report that one test message was sent to each Slack channel. The #team-routing channel should show the automated-routing message.

The #triage-review channel should show the human-review message. Both Slack routes are now live and ready for AI decisions.

Missing a Test Message?

• Run the configuration script again if the protected file was not created.
• Confirm that each webhook entry is assigned to the intended Slack channel.
• Run the smoke test from the repository folder that contains control_room.
• Help me find why one Slack test message is missing.

✔️ Both channels received a message

Your Slack control room is connected. Keep both webhook URLs private while you build the live routing loop.

ⓧ I'd like to double check the full code

Compare scripts/smoke_test_slack.py with this complete file.

from control_room.infrastructure import SlackNotifier


def main() -> None:
    notifier = SlackNotifier()
    notifier.send_text(
        "team-routing",
        "AI Triage Control Room setup complete: automated routing is online.",
    )
    notifier.send_text(
        "triage-review",
        "AI Triage Control Room setup complete: human review is online.",
    )
    print("Sent one test message to each Slack channel.")


if __name__ == "__main__":
    main()

Your two operational destinations are ready. Next, you will send a live Gemini decision into the team channel and expose the risk of routing every ticket automatically.

Send a Live AI Decision to Slack

Your Slack channels can receive messages through the protected webhooks you configured. The control room can now turn that working connection into a live destination for AI decisions.

This step creates the shortest useful path from Gemini to Slack. You will test whether that simple path handles an uncertain ticket safely.

In this step, get ready to:

• Install the pinned control room dependencies.
• Build validated ticket contracts plus Gemini and Slack adapters.
• Run clear and ambiguous tickets through a live workflow.

Install dependencies and define the contracts

The workflow needs predictable data before it can send AI decisions anywhere. Pydantic contracts define the allowed ticket fields and decision values.

• In the VS Code Explorer sidebar, create requirements-control-room.txt at the top level of your Part 2 repository.
• Add the pinned dependencies by copying this content into the file:

google-genai==2.32.0
pydantic==2.14.1

Why Pin These Dependencies?

The Google Gen AI SDK version 2.32.0 provides the Gemini client and structured-output configuration. Pydantic version 2.14.1 validates each model response against your contract.

• Save requirements-control-room.txt.
• Install the dependencies into your active virtual environment by running this command:

python -m pip install -r requirements-control-room.txt

What Does This Command Do?

The -r option tells pip to install every requirement listed in requirements-control-room.txt. The pins keep your environment on the releases used by this project.

You should see the installation finish without a dependency error. Your active virtual environment now contains both pinned libraries.

Having Trouble Installing the Dependencies?

• Confirm that your existing virtual environment is active.
• Check that requirements-control-room.txt contains both dependency lines exactly as shown.
• Help me diagnose this dependency installation failure.

• In the VS Code Explorer sidebar, create a folder named control_room at the top level of your repository.

You should see control_room beside the existing scripts folder in the Explorer sidebar.

• Inside control_room, create __init__.py using the Explorer sidebar.
• Identify the new folder as the control room package by copying this content into control_room/__init__.py:

"""AI Triage Control Room."""

What Does This File Do?

The __init__.py file identifies control_room as a Python package. Its docstring gives the package a concise description.

• Save control_room/__init__.py.
• Confirm that __init__.py appears inside control_room in the Explorer sidebar.

• Inside control_room, create domain.py using the Explorer sidebar.
• Define the ticket and decision contracts by copying this code into control_room/domain.py:

from typing import Literal

from pydantic import BaseModel, Field

Priority = Literal["low", "medium", "high", "urgent"]
Team = Literal["backend", "data", "networking", "developer-experience"]
Destination = Literal["team-routing", "triage-review"]


class Ticket(BaseModel):
    title: str = Field(min_length=3)
    description: str = Field(min_length=3)
    source_url: str | None = None
    expected_team: Team | None = None


class TriageDecision(BaseModel):
    priority: Priority
    team: Team
    confidence: float = Field(ge=0.0, le=1.0)
    reasoning: str = Field(min_length=3)


class RoutingResult(BaseModel):
    ticket: Ticket
    decision: TriageDecision
    destination: Destination
    policy_reason: str

What Do These Contracts Protect?

• The Ticket contract requires meaningful title and description values.
• The TriageDecision contract limits priority and team values to the supported choices.
• The confidence field accepts values from 0.0 through 1.0.
• The RoutingResult contract records the ticket and its chosen Slack destination.

• Save control_room/domain.py.
• Check the VS Code Problems panel for errors in control_room/domain.py.

You should see no syntax errors in the Problems panel. The workflow now has a validated data boundary for every live decision.

Seeing a Domain Error?

• Check that every Literal value has matching quotation marks.
• Confirm that each model field is indented by four spaces.
• Help me fix this domain model error.

Build the Gemini and Slack adapters

Structured output asks Gemini for JSON that matches the TriageDecision schema. The Slack adapter turns that validated decision into a formatted channel message.

• Inside control_room, create infrastructure.py using the Explorer sidebar.
• Add the imports and shared configuration by copying this code into control_room/infrastructure.py:

import json
from pathlib import Path
from urllib.request import Request, urlopen

from google import genai
from google.genai import types

from control_room.domain import Destination, RoutingResult, Ticket, TriageDecision

CONFIG_PATH = (
    Path.home() / ".config" / "ai-triage-control-room" / "slack-webhooks.json"
)
MODEL_ID = "gemini-3.8-flash"

What Does This Foundation Provide?

• The Google imports provide the Gemini client and generation configuration.
• The standard-library URL tools send the Slack request without another dependency.
• The CONFIG_PATH value points to the protected webhook file from the previous step.
• The MODEL_ID value selects gemini-3.8-flash.

• Save control_room/infrastructure.py.
• Check the VS Code Problems panel for import or syntax errors.

You should see no errors for the imports or shared configuration. The adapter now knows which model and local configuration path to use.

• In control_room/infrastructure.py, add GeminiClassifier below MODEL_ID by copying this code:

class GeminiClassifier:
    def __init__(self) -> None:
        self.client = genai.Client()

    def classify(self, ticket: Ticket) -> TriageDecision:
        prompt = f"""
You triage software project issues.
Choose exactly one priority: low, medium, high, urgent.
Choose exactly one team: backend, data, networking, developer-experience.
Return a calibrated confidence from 0.0 to 1.0 and a short reason.

Title: {ticket.title}
Description: {ticket.description}
""".strip()

        response = self.client.models.generate_content(
            model=MODEL_ID,
            contents=prompt,
            config=types.GenerateContentConfig(
                response_mime_type="application/json",
                response_json_schema=TriageDecision.model_json_schema(),
            ),
        )
        if not response.text:
            raise RuntimeError("Gemini returned no text response.")
        return TriageDecision.model_validate_json(response.text)

How Does Structured Classification Work?

• The prompt defines the supported priority and team choices.
• The generation configuration supplies the schema from TriageDecision.model_json_schema().
• The model_validate_json() call validates the returned response before the workflow can use it.

• Save control_room/infrastructure.py.
• Check the VS Code Problems panel for errors inside GeminiClassifier.

You should see no syntax errors in the classifier. Its first live request happens when you run the demo.

• Add the constructor and formatted-message method for SlackNotifier below GeminiClassifier by copying this code:

class SlackNotifier:
    def __init__(self, config_path: Path = CONFIG_PATH) -> None:
        if not config_path.exists():
            raise FileNotFoundError(
                f"Slack configuration not found at {config_path}. "
                "Run python scripts/configure_slack.py first."
            )
        self.webhooks = json.loads(config_path.read_text(encoding="utf-8"))

    def send(self, destination: Destination, result: RoutingResult) -> None:
        message = (
            "*AI Triage Control Room*\n"
            f"*Title:* {result.ticket.title}\n"
            f"*Priority:* {result.decision.priority}\n"
            f"*Team:* {result.decision.team}\n"
            f"*Confidence:* {result.decision.confidence:.2f}\n"
            f"*AI reason:* {result.decision.reasoning}\n"
            f"*Policy:* {result.policy_reason}"
        )
        self.send_text(destination, message)

How Is the Slack Message Prepared?

• The constructor loads both channel-specific webhook URLs from the protected local file.
• The send() method formats the ticket title and decision fields.
• The destination selects the webhook associated with one Slack channel.

• Save control_room/infrastructure.py.
• Confirm that SlackNotifier is aligned with GeminiClassifier at the left edge of the file.

• Inside SlackNotifier, add send_text() directly below send() by copying this code:

    def send_text(self, destination: Destination, text: str) -> None:
        webhook_url = self.webhooks[destination]
        payload = json.dumps({"text": text}).encode("utf-8")
        request = Request(
            webhook_url,
            data=payload,
            headers={"Content-Type": "application/json"},
            method="POST",
        )
        with urlopen(request, timeout=10) as response:
            response_body = response.read().decode("utf-8")
        if response_body != "ok":
            raise RuntimeError(f"Slack returned an unexpected response: {response_body}")

How Does the Message Reach Slack?

• The method selects the webhook for the requested destination.
• The request sends a JSON text payload through an HTTPS POST.
• The response body must equal ok for the post to count as successful.

• Save control_room/infrastructure.py.
• Confirm that send_text() is indented inside SlackNotifier.

Before you test the completed adapter, do you expect both existing Slack channels to receive another setup message?

• Test the completed Slack adapter with the smoke test from earlier by running this command:

python scripts/smoke_test_slack.py

What Does This Check Prove?

The smoke test imports the completed SlackNotifier and uses the protected configuration file. A successful run proves that the refactored adapter still reaches both channel-specific webhooks.

You should see Sent one test message to each Slack channel. in the terminal. You should also see a new setup message in each channel.

Did the Smoke Test Fail?

• Confirm that send_text() remains inside SlackNotifier.
• Check that the protected configuration file still contains entries for both channels.
• Help me debug this Slack adapter failure.

Run the naive live workflow

The temporary demo connects terminal input to Gemini classification and visible Slack delivery. Its destination stays fixed while you compare a clear ticket with an ambiguous one.

• Inside the existing scripts folder, create naive_demo.py using the Explorer sidebar.
• Build the temporary interactive workflow by copying this code into scripts/naive_demo.py:

import json

from control_room.domain import RoutingResult, Ticket
from control_room.infrastructure import GeminiClassifier, SlackNotifier


def main() -> None:
    # Capture one software ticket from the operator.
    title = input("Ticket title: ").strip()
    description = input("Ticket description: ").strip()
    ticket = Ticket(title=title, description=description)

    # Ask Gemini for a validated structured decision.
    decision = GeminiClassifier().classify(ticket)
    print(json.dumps(decision.model_dump(mode="json"), indent=2))

    # Preserve the naive path so every decision reaches the team channel.
    result = RoutingResult(
        ticket=ticket,
        decision=decision,
        destination="team-routing",
        policy_reason="Naive demo sends every decision to the team channel.",
    )
    SlackNotifier().send("team-routing", result)
    print("\nSlack destination: #team-routing")


if __name__ == "__main__":
    main()

What Does the Demo Connect?

• The terminal prompts become a validated Ticket.
• The classifier returns a structured TriageDecision.
• The RoutingResult always records team-routing as its destination.
• The notifier posts the formatted result through the webhook for #team-routing.

• Save scripts/naive_demo.py.

The script asks for a ticket title first. It asks for a description next.

• Start the clear-ticket test by running this command:

python scripts/naive_demo.py

What Happens During This Run?

The script pauses for both ticket fields before contacting Gemini. It then prints the validated decision before posting the formatted result to Slack.

• Enter API returns a 500 error during checkout at the title prompt.
• Enter Customers cannot complete checkout because the backend request fails at the description prompt.

You should see a structured decision containing a priority, team, confidence, and reason. The final terminal line should show Slack destination: #team-routing.

• Return to the #team-routing channel from earlier to confirm that the formatted decision appears there.

That is the first live loop working. A terminal ticket now becomes a validated Gemini decision and a visible Slack message.

Did the Clear Ticket Fail?

• Confirm that your existing GEMINI_API_KEY is available to the active virtual environment.
• Check that both ticket inputs contain at least three characters.
• Help me debug this live triage failure.

Before you run the ambiguous ticket, do you think its confidence can change the Slack destination?

• Start the ambiguous-ticket test by running the demo again:

python scripts/naive_demo.py

What Is This Run Testing?

This run gives the classifier less specific evidence. Comparing the returned confidence with the final destination reveals what the automation currently considers.

• Enter Login sometimes feels odd after an update at the title prompt.
• Enter The behavior is inconsistent and the affected component is unclear at the description prompt.

Look at the confidence in the printed decision. The terminal still reports Slack destination: #team-routing regardless of that confidence.

This Shortfall Is Intentional

The demo records every destination as team-routing. The model can report uncertainty, but that uncertainty has no control over the automation path.

This exposes the false-automation risk in the simplest live integration. Deterministic policy must decide when automation stops for human review.

• Return to #team-routing to confirm that the ambiguous ticket appears as another formatted decision.
• Return to #triage-review to confirm that the ambiguous ticket does not appear there.

✔️ Awesome, I've got everything!

Your dependencies, contracts, adapters, and temporary demo are saved. Both live ticket tests reached #team-routing.

ⓧ I'd like to double check the full code

Compare each new file with the references below. Keep the Slack configuration scripts from the previous step unchanged.

google-genai==2.32.0
pydantic==2.14.1

Check the Dependency Pins

Confirm that both package names and version pins match the reference exactly.

"""AI Triage Control Room."""

Check the Package File

Confirm that the package file contains the control room docstring.

from typing import Literal

from pydantic import BaseModel, Field

Priority = Literal["low", "medium", "high", "urgent"]
Team = Literal["backend", "data", "networking", "developer-experience"]
Destination = Literal["team-routing", "triage-review"]


class Ticket(BaseModel):
    title: str = Field(min_length=3)
    description: str = Field(min_length=3)
    source_url: str | None = None
    expected_team: Team | None = None


class TriageDecision(BaseModel):
    priority: Priority
    team: Team
    confidence: float = Field(ge=0.0, le=1.0)
    reasoning: str = Field(min_length=3)


class RoutingResult(BaseModel):
    ticket: Ticket
    decision: TriageDecision
    destination: Destination
    policy_reason: str

Check the Domain Contracts

Confirm that all three contracts appear in this order. Check the allowed values and field constraints closely.

import json
from pathlib import Path
from urllib.request import Request, urlopen

from google import genai
from google.genai import types

from control_room.domain import Destination, RoutingResult, Ticket, TriageDecision

CONFIG_PATH = (
    Path.home() / ".config" / "ai-triage-control-room" / "slack-webhooks.json"
)
MODEL_ID = "gemini-3.8-flash"


class GeminiClassifier:
    def __init__(self) -> None:
        self.client = genai.Client()

    def classify(self, ticket: Ticket) -> TriageDecision:
        prompt = f"""
You triage software project issues.
Choose exactly one priority: low, medium, high, urgent.
Choose exactly one team: backend, data, networking, developer-experience.
Return a calibrated confidence from 0.0 to 1.0 and a short reason.

Title: {ticket.title}
Description: {ticket.description}
""".strip()

        response = self.client.models.generate_content(
            model=MODEL_ID,
            contents=prompt,
            config=types.GenerateContentConfig(
                response_mime_type="application/json",
                response_json_schema=TriageDecision.model_json_schema(),
            ),
        )
        if not response.text:
            raise RuntimeError("Gemini returned no text response.")
        return TriageDecision.model_validate_json(response.text)


class SlackNotifier:
    def __init__(self, config_path: Path = CONFIG_PATH) -> None:
        if not config_path.exists():
            raise FileNotFoundError(
                f"Slack configuration not found at {config_path}. "
                "Run python scripts/configure_slack.py first."
            )
        self.webhooks = json.loads(config_path.read_text(encoding="utf-8"))

    def send(self, destination: Destination, result: RoutingResult) -> None:
        message = (
            "*AI Triage Control Room*\n"
            f"*Title:* {result.ticket.title}\n"
            f"*Priority:* {result.decision.priority}\n"
            f"*Team:* {result.decision.team}\n"
            f"*Confidence:* {result.decision.confidence:.2f}\n"
            f"*AI reason:* {result.decision.reasoning}\n"
            f"*Policy:* {result.policy_reason}"
        )
        self.send_text(destination, message)

    def send_text(self, destination: Destination, text: str) -> None:
        webhook_url = self.webhooks[destination]
        payload = json.dumps({"text": text}).encode("utf-8")
        request = Request(
            webhook_url,
            data=payload,
            headers={"Content-Type": "application/json"},
            method="POST",
        )
        with urlopen(request, timeout=10) as response:
            response_body = response.read().decode("utf-8")
        if response_body != "ok":
            raise RuntimeError(f"Slack returned an unexpected response: {response_body}")

Check the Infrastructure Adapters

Confirm that the model ID and protected configuration path match the reference. Check that response validation happens before Slack receives the result.

import json

from control_room.domain import RoutingResult, Ticket
from control_room.infrastructure import GeminiClassifier, SlackNotifier


def main() -> None:
    # Capture one software ticket from the operator.
    title = input("Ticket title: ").strip()
    description = input("Ticket description: ").strip()
    ticket = Ticket(title=title, description=description)

    # Ask Gemini for a validated structured decision.
    decision = GeminiClassifier().classify(ticket)
    print(json.dumps(decision.model_dump(mode="json"), indent=2))

    # Preserve the naive path so every decision reaches the team channel.
    result = RoutingResult(
        ticket=ticket,
        decision=decision,
        destination="team-routing",
        policy_reason="Naive demo sends every decision to the team channel.",
    )
    SlackNotifier().send("team-routing", result)
    print("\nSlack destination: #team-routing")


if __name__ == "__main__":
    main()

Check the Temporary Demo

Confirm that the demo prints the validated decision before posting it. Both destination references must remain team-routing in this temporary version.

You now have a live AI triage loop with a visible automation risk. Next, you will add deterministic human-review rules and enforce the architecture boundaries.

Add Human Review and Enforce the Architecture

The direct Gemini to Slack loop exposed a safety gap. An uncertain decision reached the team channel with the same authority as a confident decision.

This step gives the application control over automation. You will also document the runtime boundaries with a C4 container map and enforce the code boundaries through GitHub Actions.

In this step, get ready to:

• Document the runtime containers and code dependency direction.
• Send urgent or low-confidence decisions to human review.
• Enforce the architecture locally and on GitHub.

Map the runtime architecture

A C4 container represents a running application or a data store. The diagram records how the console and replay process communicate with external systems.

A separate code policy assigns responsibilities to the Python modules. This connects the runtime design to dependency rules that your repository can enforce.

• Create the docs folder from the VS Code Explorer sidebar.
• Create architecture.md inside the docs folder.
• Add the container diagram to docs/architecture.md by pasting this code:

# Container diagram for the AI Triage Control Room

```mermaid
flowchart LR
    learner["Person: Learner or operator"]

    subgraph control_room["Software System: AI Triage Control Room"]
        console["Container: Interactive Console<br/>Python<br/>Captures a ticket and shows the routing result"]
        replay["Container: Shadow Replay<br/>Python batch process<br/>Evaluates real issue fixtures without routing each issue"]
        config[("Container: Local Secret Store<br/>Protected JSON file<br/>Stores channel-specific webhook URLs")]
        evidence[("Container: Evaluation Artifacts<br/>JSON and Markdown files<br/>Stores metrics and handoff evidence")]
    end

    gemini["External System: Gemini Developer API"]
    slack["External System: Slack"]

    learner -->|"submits ticket and reads decision"| console
    console -->|"loads webhook URLs from local file"| config
    console -->|"requests structured classification via HTTPS/JSON"| gemini
    console -->|"posts routing message via HTTPS incoming webhook"| slack
    replay -->|"requests structured classifications via HTTPS/JSON"| gemini
    replay -->|"writes replay metrics and handoff"| evidence
    replay -->|"posts one evaluation summary via HTTPS webhook"| slack
```

What does this diagram show?

• The Interactive Console captures one ticket and displays its governed routing result.
• The Shadow Replay evaluates issue fixtures without routing every issue to a team channel.
• The local secret store protects the channel-specific webhook URLs.
• The labeled arrows record each one-way runtime data flow.

• Save docs/architecture.md.
• Render docs/architecture.md with the Markdown preview control in VS Code.

You should see the learner connected to the interactive console. You should also see the console and shadow replay connected to their external systems or data stores.

Diagram not rendering?

• Confirm that the opening code fence includes mermaid.
• Confirm that the closing code fence appears after the final Slack relationship.
• Ask for help with the diagram syntax: Help me find the syntax problem in my container diagram.

• Append the diagram key and dependency policy below the diagram in docs/architecture.md by pasting this code:

Key:

- Person: A human interacting with the system.
- Container: A running application, batch process, or data store.
- External System: A separately owned service used by the control room.
- Arrows: Labeled, one-way runtime communication or data flow.

## Code boundary policy

- `domain.py` owns data contracts and imports no other control-room module.
- `application.py` owns routing policy and may import only the domain.
- `infrastructure.py` owns Gemini and Slack adapters and may import only the domain.
- `interface.py` owns terminal interaction and may import only application and domain.
- `__main__.py` is the composition root where concrete adapters are connected.

Why separate the two views?

The container diagram describes running applications and data stores. The boundary list describes dependency direction inside the Python package.

Together they make the intended architecture visible and testable.

• Save docs/architecture.md.
• Refresh the Markdown preview.

You should now see a key beneath the diagram. You should also see five module responsibility rules under the code boundary policy.

✔️ Awesome, I've got everything!

Your architecture document now shows the runtime boundaries and the permitted dependency direction.

• Keep the rendered preview available for comparison while you build the governed modules.

ⓧ I'd like to double check the full code

Compare docs/architecture.md with this complete reference.

# Container diagram for the AI Triage Control Room

```mermaid
flowchart LR
    learner["Person: Learner or operator"]

    subgraph control_room["Software System: AI Triage Control Room"]
        console["Container: Interactive Console<br/>Python<br/>Captures a ticket and shows the routing result"]
        replay["Container: Shadow Replay<br/>Python batch process<br/>Evaluates real issue fixtures without routing each issue"]
        config[("Container: Local Secret Store<br/>Protected JSON file<br/>Stores channel-specific webhook URLs")]
        evidence[("Container: Evaluation Artifacts<br/>JSON and Markdown files<br/>Stores metrics and handoff evidence")]
    end

    gemini["External System: Gemini Developer API"]
    slack["External System: Slack"]

    learner -->|"submits ticket and reads decision"| console
    console -->|"loads webhook URLs from local file"| config
    console -->|"requests structured classification via HTTPS/JSON"| gemini
    console -->|"posts routing message via HTTPS incoming webhook"| slack
    replay -->|"requests structured classifications via HTTPS/JSON"| gemini
    replay -->|"writes replay metrics and handoff"| evidence
    replay -->|"posts one evaluation summary via HTTPS webhook"| slack
```

Key:

- Person: A human interacting with the system.
- Container: A running application, batch process, or data store.
- External System: A separately owned service used by the control room.
- Arrows: Labeled, one-way runtime communication or data flow.

## Code boundary policy

- `domain.py` owns data contracts and imports no other control-room module.
- `application.py` owns routing policy and may import only the domain.
- `infrastructure.py` owns Gemini and Slack adapters and may import only the domain.
- `interface.py` owns terminal interaction and may import only application and domain.
- `__main__.py` is the composition root where concrete adapters are connected.

What should match?

The complete file contains the diagram first. The key and five code boundary rules follow it.

Build the governed control room

The model still supplies priority and confidence. The application layer now decides whether those values permit automation.

The policy sends urgent decisions to human review. It also escalates confidence values below 0.75.

• Create application.py inside the existing control_room folder.
• Add the service contracts and routing policy to control_room/application.py by pasting this code:

from typing import Protocol

from control_room.domain import Destination, RoutingResult, Ticket, TriageDecision

CONFIDENCE_THRESHOLD = 0.75


class Classifier(Protocol):
    def classify(self, ticket: Ticket) -> TriageDecision:
        ...


class Notifier(Protocol):
    def send(self, destination: Destination, result: RoutingResult) -> None:
        ...


def select_destination(decision: TriageDecision) -> tuple[Destination, str]:
    if decision.priority == "urgent":
        return "triage-review", "Urgent tickets always require human review."
    if decision.confidence < CONFIDENCE_THRESHOLD:
        return (
            "triage-review",
            f"Confidence {decision.confidence:.2f} is below {CONFIDENCE_THRESHOLD:.2f}.",
        )
    return "team-routing", "Decision meets the automation policy."

What does the routing policy do?

• The protocols describe the classifier and notifier capabilities that the service needs.
• The urgent branch requires human review at every confidence level.
• The confidence branch sends values below 0.75 to triage-review.
• Every remaining decision reaches team-routing.

• Save control_room/application.py.
• Confirm that VS Code shows no syntax error markers in control_room/application.py.

Seeing an import or syntax error?

• Confirm that Destination and the three model classes come from control_room.domain.
• Confirm that the low-confidence return closes with a parenthesis.
• Ask for help with the routing policy: Help me debug my application-layer policy.

• Append TriageService below select_destination() in control_room/application.py by pasting this code:

class TriageService:
    def __init__(self, classifier: Classifier, notifier: Notifier) -> None:
        self.classifier = classifier
        self.notifier = notifier

    def triage(self, ticket: Ticket) -> RoutingResult:
        decision = self.classifier.classify(ticket)
        destination, policy_reason = select_destination(decision)
        result = RoutingResult(
            ticket=ticket,
            decision=decision,
            destination=destination,
            policy_reason=policy_reason,
        )
        self.notifier.send(destination, result)
        return result

How does the service govern a ticket?

• The classifier produces a structured TriageDecision.
• The policy chooses the destination and records its reason.
• The notifier receives the completed RoutingResult.
• The returned result gives the console the same evidence that Slack receives.

• Save control_room/application.py.
• Confirm that VS Code shows no syntax error markers in the completed application module.

Service symbols not resolving?

• Confirm that TriageService appears below the complete select_destination() function.
• Confirm that the class indentation uses four spaces inside each method.
• Ask for help with the service flow: Help me trace the application service.

• Create interface.py inside the existing control_room folder.
• Add the interactive console to control_room/interface.py by pasting this code:

import json

from control_room.application import TriageService
from control_room.domain import Ticket


def run_console(service: TriageService) -> None:
    print("AI Triage Control Room")
    title = input("Ticket title: ").strip()
    description = input("Ticket description: ").strip()
    result = service.triage(Ticket(title=title, description=description))
    print("\nStructured decision")
    print(json.dumps(result.model_dump(mode="json"), indent=2))
    print(f"\nSlack destination: #{result.destination}")

What does the console own?

The console owns terminal input and readable JSON output. It displays the destination selected by the application policy.

• Save control_room/interface.py.
• Confirm that VS Code shows no syntax error markers in control_room/interface.py.

Interface import not resolving?

• Confirm that TriageService comes from control_room.application.
• Confirm that Ticket comes from control_room.domain.
• Ask for help with the interface module: Help me debug my console interface.

• Create __main__.py inside the existing control_room folder.
• Connect the service and adapters in control_room/__main__.py by pasting this code:

from control_room.application import TriageService
from control_room.infrastructure import GeminiClassifier, SlackNotifier
from control_room.interface import run_console


def main() -> None:
    service = TriageService(
        classifier=GeminiClassifier(),
        notifier=SlackNotifier(),
    )
    run_console(service)


if __name__ == "__main__":
    main()

Why use a composition root?

The composition root connects the concrete Gemini and Slack adapters to the application service. The policy remains independent of those vendor integrations.

• Save control_room/__main__.py.
• Confirm that VS Code shows no syntax error markers in the package entry point.

Package entry point not resolving?

• Confirm that the file is named __main__.py inside control_room.
• Confirm that GeminiClassifier and SlackNotifier come from the existing infrastructure module.
• Ask for help with the entry point: Help me debug my package entry point.

Before the routine check runs, which Slack destination do you expect the policy to choose?

• Start the governed console by running this command:

python -m control_room

What does this command start?

Python runs the package entry point in control_room/__main__.py. The entry point starts the console with the real classifier and notifier.

• Enter Update README installation example at the ticket title prompt.
• Enter Clarify the setup example for new contributors. at the ticket description prompt.

You should see a structured decision followed by #team-routing as the Slack destination.

• Return to the #team-routing channel from earlier.

You should see the routine ticket with its model decision and automation policy reason. That confirms safe decisions can still move automatically.

Before the ambiguous check runs, do you expect the confidence rule to permit automation?

• Start a new governed console run with this command:

python -m control_room

What changes in this run?

The same service handles the ticket. The ambiguous wording gives the confidence policy a chance to stop an uncertain decision.

• Enter Login sometimes feels odd after an update at the ticket title prompt.
• Enter The behavior is inconsistent and no clear failure is visible. at the ticket description prompt.

When Gemini reports confidence below 0.75, you should see #triage-review as the Slack destination.

• Return to the #triage-review channel from earlier.

You should see the ambiguous ticket with a policy reason that records its confidence below the threshold. The uncertainty now remains visible to a person.

Before the urgent check runs, do you expect high confidence to bypass human review?

• Start the final governed console run with this command:

python -m control_room

What does the urgent check prove?

The urgent branch runs before the confidence branch. Every urgent classification therefore requires human review.

• Enter Production API is down at the ticket title prompt.
• Enter All customer requests fail with server errors. at the ticket description prompt.

You should see an urgent structured decision followed by #triage-review as the Slack destination.

• Return to the #triage-review channel from earlier.

You should see the urgent ticket with the policy reason Urgent tickets always require human review. The application now keeps urgency under human authority.

Ticket reached the wrong channel?

• Read the terminal values for priority and confidence.
• Confirm that select_destination() checks urgent priority before confidence.
• Ask for help tracing the route: Help me trace why this ticket reached the wrong Slack channel.

• Locate scripts/naive_demo.py in the VS Code Explorer sidebar.
• Delete scripts/naive_demo.py with the file context menu.

You should no longer see scripts/naive_demo.py in the Explorer sidebar. The governed package command now replaces the temporary loop.

✔️ Awesome, the governed console works

Your package now sends safe decisions to #team-routing and review cases to #triage-review.

• Keep the three governed modules saved before you add the architecture gate.

ⓧ I'd like to double check the full code

Compare the three new control-room files with these complete references.

from typing import Protocol

from control_room.domain import Destination, RoutingResult, Ticket, TriageDecision

CONFIDENCE_THRESHOLD = 0.75


class Classifier(Protocol):
    def classify(self, ticket: Ticket) -> TriageDecision:
        ...


class Notifier(Protocol):
    def send(self, destination: Destination, result: RoutingResult) -> None:
        ...


def select_destination(decision: TriageDecision) -> tuple[Destination, str]:
    if decision.priority == "urgent":
        return "triage-review", "Urgent tickets always require human review."
    if decision.confidence < CONFIDENCE_THRESHOLD:
        return (
            "triage-review",
            f"Confidence {decision.confidence:.2f} is below {CONFIDENCE_THRESHOLD:.2f}.",
        )
    return "team-routing", "Decision meets the automation policy."


class TriageService:
    def __init__(self, classifier: Classifier, notifier: Notifier) -> None:
        self.classifier = classifier
        self.notifier = notifier

    def triage(self, ticket: Ticket) -> RoutingResult:
        decision = self.classifier.classify(ticket)
        destination, policy_reason = select_destination(decision)
        result = RoutingResult(
            ticket=ticket,
            decision=decision,
            destination=destination,
            policy_reason=policy_reason,
        )
        self.notifier.send(destination, result)
        return result

What should match in the application layer?

The file defines the two protocols first. The routing policy and TriageService follow them.

import json

from control_room.application import TriageService
from control_room.domain import Ticket


def run_console(service: TriageService) -> None:
    print("AI Triage Control Room")
    title = input("Ticket title: ").strip()
    description = input("Ticket description: ").strip()
    result = service.triage(Ticket(title=title, description=description))
    print("\nStructured decision")
    print(json.dumps(result.model_dump(mode="json"), indent=2))
    print(f"\nSlack destination: #{result.destination}")

What should match in the interface?

The interface constructs a Ticket from terminal input. It prints the complete governed result and destination.

from control_room.application import TriageService
from control_room.infrastructure import GeminiClassifier, SlackNotifier
from control_room.interface import run_console


def main() -> None:
    service = TriageService(
        classifier=GeminiClassifier(),
        notifier=SlackNotifier(),
    )
    run_console(service)


if __name__ == "__main__":
    main()

What should match in the entry point?

The entry point constructs the service with the existing adapters. It passes that service to run_console().

Enforce boundaries in GitHub Actions

A written dependency policy can drift away from the code. An abstract syntax tree check converts the allowed imports into an executable architecture rule.

The same script runs in your local repository and on GitHub. Both environments use one definition of a clean dependency boundary.

• Create check_architecture.py inside the existing scripts folder.
• Add the allowed import map and parser to scripts/check_architecture.py by pasting this code:

import ast
from pathlib import Path

PACKAGE_DIR = Path("control_room")
ALLOWED_INTERNAL_IMPORTS = {
    "domain": set(),
    "application": {"control_room.domain"},
    "infrastructure": {"control_room.domain"},
    "interface": {"control_room.application", "control_room.domain"},
}


def internal_imports(path: Path) -> set[str]:
    tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
    imports: set[str] = set()
    for node in ast.walk(tree):
        if isinstance(node, ast.Import):
            imports.update(alias.name for alias in node.names)
        elif isinstance(node, ast.ImportFrom) and node.module:
            imports.add(node.module)
    return {name for name in imports if name.startswith("control_room.")}

How does the import scanner work?

• The allowed import map mirrors the policy in docs/architecture.md.
• The parser converts each Python file into syntax nodes.
• The scanner collects imports beginning with control_room..
• Standard-library and third-party imports stay outside this boundary check.

• Save scripts/check_architecture.py.
• Confirm that VS Code shows no syntax error markers in the scanner.

Scanner syntax not resolving?

• Confirm that every module name in ALLOWED_INTERNAL_IMPORTS matches an existing Python file.
• Confirm that the final set comprehension remains inside internal_imports().
• Ask for help with the scanner: Help me debug my architecture import scanner.

• Append the gate logic below internal_imports() in scripts/check_architecture.py by pasting this code:

def main() -> None:
    violations: list[str] = []
    for module_name, allowed in ALLOWED_INTERNAL_IMPORTS.items():
        path = PACKAGE_DIR / f"{module_name}.py"
        for imported in sorted(internal_imports(path)):
            if imported not in allowed:
                violations.append(
                    f"{path} imports {imported}; allowed internal imports: {sorted(allowed)}"
                )

    if violations:
        print("Architecture policy failed:")
        for violation in violations:
            print(f"- {violation}")
        raise SystemExit(1)

    print(
        "Architecture policy passed: domain, application, infrastructure, "
        "and interface boundaries are clean."
    )


if __name__ == "__main__":
    main()

How does the gate fail safely?

Every forbidden import becomes a readable violation. The nonzero exit prevents a failed policy check from reporting success.

A clean result prints one explicit confirmation. That message becomes the shared evidence for local and remote enforcement.

• Save scripts/check_architecture.py.

Before you run the gate, do you expect the four governed modules to obey every allowed import rule?

• Check the architecture locally by running this command:

python scripts/check_architecture.py

What does this local check prove?

The script parses the current repository files and compares their internal imports with the policy map. A successful exit proves that the checked modules follow the documented dependency direction.

You should see Architecture policy passed: domain, application, infrastructure, and interface boundaries are clean. That is the local architecture gate working.

Architecture policy failed?

• Read the reported file and imported module in the first violation.
• Compare that pair with ALLOWED_INTERNAL_IMPORTS.
• Ask for help with the failed boundary: Help me understand this architecture violation.

• Create the nested .github/workflows folder path from the VS Code Explorer sidebar.
• Create architecture.yml inside .github/workflows.
• Add the remote architecture gate to .github/workflows/architecture.yml by pasting this code:

name: Architecture Policy

on: [push, pull_request]

permissions:
  contents: read

jobs:
  check-boundaries:
    runs-on: ubuntu-latest
    steps:
      - name: Check out the repository
        uses: actions/checkout@v7.0.1
      - name: Set up Python
        uses: actions/setup-python@v7.1.0
        with:
          python-version: "3.10"
      - name: Enforce dependency direction
        run: python scripts/check_architecture.py

What does the workflow enforce?

• The workflow runs on pushes and pull requests.
• The checkout step loads the repository onto the runner.
• The Python setup step provides Python 3.10.
• The final step runs the same architecture command that passed locally.

• Save .github/workflows/architecture.yml.
• Commit the new architecture files with your existing Git workflow.
• Push the commit to your repository's default branch.

The remote run can take a minute to start, so a short queued state is expected.

• Select the Actions tab in your GitHub repository.
• Select the latest Architecture Policy workflow run.

Before you inspect the job, do you expect the remote result to match the local architecture check?

• Confirm that the check-boundaries job is green.

You should see the checkout and Python setup steps complete successfully. You should also see the dependency enforcement step complete successfully.

Strong finish. Your repository now applies the same dependency policy locally and on every push or pull request.

GitHub check not green?

• Select the failed job to reveal its logs.
• Read the first architecture violation printed by the script.
• Confirm that .github/workflows/architecture.yml exists on the branch that triggered the run.
• Ask for help with the workflow failure: Help me debug my Architecture Policy workflow.

✔️ The architecture gate is green

Your local check and GitHub workflow now enforce the same dependency policy.

• Keep the successful workflow run available for your final checkpoint.

ⓧ I'd like to double check the full code

Compare the architecture gate and workflow with these complete references.

import ast
from pathlib import Path

PACKAGE_DIR = Path("control_room")
ALLOWED_INTERNAL_IMPORTS = {
    "domain": set(),
    "application": {"control_room.domain"},
    "infrastructure": {"control_room.domain"},
    "interface": {"control_room.application", "control_room.domain"},
}


def internal_imports(path: Path) -> set[str]:
    tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(path))
    imports: set[str] = set()
    for node in ast.walk(tree):
        if isinstance(node, ast.Import):
            imports.update(alias.name for alias in node.names)
        elif isinstance(node, ast.ImportFrom) and node.module:
            imports.add(node.module)
    return {name for name in imports if name.startswith("control_room.")}


def main() -> None:
    violations: list[str] = []
    for module_name, allowed in ALLOWED_INTERNAL_IMPORTS.items():
        path = PACKAGE_DIR / f"{module_name}.py"
        for imported in sorted(internal_imports(path)):
            if imported not in allowed:
                violations.append(
                    f"{path} imports {imported}; allowed internal imports: {sorted(allowed)}"
                )

    if violations:
        print("Architecture policy failed:")
        for violation in violations:
            print(f"- {violation}")
        raise SystemExit(1)

    print(
        "Architecture policy passed: domain, application, infrastructure, "
        "and interface boundaries are clean."
    )


if __name__ == "__main__":
    main()

What should match in the local gate?

The complete script contains the allowed import map and AST scanner. It ends with the violation check and success message.

name: Architecture Policy

on: [push, pull_request]

permissions:
  contents: read

jobs:
  check-boundaries:
    runs-on: ubuntu-latest
    steps:
      - name: Check out the repository
        uses: actions/checkout@v7.0.1
      - name: Set up Python
        uses: actions/setup-python@v7.1.0
        with:
          python-version: "3.10"
      - name: Enforce dependency direction
        run: python scripts/check_architecture.py

What should match in the remote gate?

The workflow runs on pushes and pull requests. Its final step executes the local architecture script.

Your control room now separates model judgment from automation authority. Next, you will replay real issues in shadow mode and turn the results into an engineering handoff.

Prove the Router on 20 Real Issues

Your governed control room now routes routine tickets automatically. Its architecture gate also passes locally and in GitHub Actions.

A few convincing demonstrations cannot prove operational usefulness. This step uses a shadow evaluation to test the policy against 20 real GitHub issues without sending individual cases into team channels.

In this step, get ready to:

• Create a labeled dataset containing 20 real software issues.
• Replay every issue through Gemini plus the existing review policy.
• Use the generated evidence to make a rollout decision.

Build the 20-issue dataset

A shadow evaluation needs examples with known destinations. Repository membership supplies the expected team label for each fixture.

What does shadow mode prove?

Shadow mode runs the same classifier and review policy without routing each issue into operational team channels. It measures what automation would have done before that behavior affects real work.

Repository membership acts as the comparison label in this project. The result measures agreement with this routing convention.

• In the VS Code file sidebar from earlier, select the repository folder.
• Create a folder named data inside the repository.

You should see the new data folder beside control_room and scripts.

• Select the new data folder in the file sidebar.
• Create a file named issues.json inside data.
• Populate data/issues.json by copying this dataset:

[
  {"repository":"fastapi/fastapi","title":"0.142.0: app fails to start when OTEL_EXPORTER_OTLP_ENDPOINT is set but the opentelemetry extra isn't installed","source_url":"https://github.com/fastapi/fastapi/issues/16417","expected_team":"backend"},
  {"repository":"fastapi/fastapi","title":"app.frontend() with fallback=\"index.html\" returns 404 for client side routes that have a dot in the last path segment (like /users/john.doe or /v/1.2.3)","source_url":"https://github.com/fastapi/fastapi/issues/16010","expected_team":"backend"},
  {"repository":"fastapi/fastapi","title":"0.137.0: `APIRoute.tags` invisible to type checkers in generate_unique_id_function (regression from #15745)","source_url":"https://github.com/fastapi/fastapi/issues/15764","expected_team":"backend"},
  {"repository":"fastapi/fastapi","title":"0.137.0: empty-path route nested under prefix-less outer include raises 'Prefix and path cannot be both empty'","source_url":"https://github.com/fastapi/fastapi/issues/15762","expected_team":"backend"},
  {"repository":"fastapi/fastapi","title":"Discriminated Unions Break When Wrapped in Annotated[Union, Body(...)] in FastAPI 0.124.1+","source_url":"https://github.com/fastapi/fastapi/issues/14508","expected_team":"backend"},
  {"repository":"pydantic/pydantic","title":"Publish the SkipJsonSchema example in the JSON Schema docs","source_url":"https://github.com/pydantic/pydantic/issues/13900","expected_team":"data"},
  {"repository":"pydantic/pydantic","title":"[bug] TypeError: 'MockValSer' object cannot be converted to 'SchemaSerializer'","source_url":"https://github.com/pydantic/pydantic/issues/12282","expected_team":"data"},
  {"repository":"pydantic/pydantic","title":"[Bug] Experimental pipeline .len() adds redundant Python-level validation for native length types","source_url":"https://github.com/pydantic/pydantic/issues/13653","expected_team":"data"},
  {"repository":"pydantic/pydantic","title":"_FieldNameStack/_ModelTypeStack lose their pop() on exception, corrupting later unrelated schema builds","source_url":"https://github.com/pydantic/pydantic/issues/13857","expected_team":"data"},
  {"repository":"pydantic/pydantic","title":"TypedDict with extra_items as PEP 695 type alias to Never or Annotated[Never, ...] crashes with PydanticSchemaGenerationError`","source_url":"https://github.com/pydantic/pydantic/issues/14018","expected_team":"data"},
  {"repository":"encode/httpx","title":"Allow URLs where username or password contains unescaped '@'.","source_url":"https://github.com/encode/httpx/issues/2985","expected_team":"networking"},
  {"repository":"encode/httpx","title":"httpx 0.24 documentation issue for netrc support","source_url":"https://github.com/encode/httpx/issues/2666","expected_team":"networking"},
  {"repository":"encode/httpx","title":"NetRCInfo.netrc_info() doesn't work when run in a systemd service with DynamicUser=true","source_url":"https://github.com/encode/httpx/issues/2532","expected_team":"networking"},
  {"repository":"encode/httpx","title":"Using a Client with a proxy to plain http endpoints returns 200 for the first, and 404 for any others.","source_url":"https://github.com/encode/httpx/issues/2488","expected_team":"networking"},
  {"repository":"encode/httpx","title":"ASGI/WSGI transport does not honor timeouts, but should","source_url":"https://github.com/encode/httpx/issues/2239","expected_team":"networking"},
  {"repository":"fastapi/typer","title":"bug: envvar not working for `typer.Options` with `typer==0.26.0`","source_url":"https://github.com/fastapi/typer/issues/1787","expected_team":"developer-experience"},
  {"repository":"fastapi/typer","title":"Overlap of typer and typer-slim causes package managemers to fail","source_url":"https://github.com/fastapi/typer/issues/1503","expected_team":"developer-experience"},
  {"repository":"fastapi/typer","title":"Version 0.19.1 breaks default_factory that returns empty list","source_url":"https://github.com/fastapi/typer/issues/1353","expected_team":"developer-experience"},
  {"repository":"fastapi/typer","title":"[BUG] help line length miscalculated when using stylized text","source_url":"https://github.com/fastapi/typer/issues/1159","expected_team":"developer-experience"},
  {"repository":"fastapi/typer","title":"Argument(metavar=\"MY_ARG\") has different meaning for the arguments panel compared to the usage text or the non-rich help output","source_url":"https://github.com/fastapi/typer/issues/1156","expected_team":"developer-experience"}
]

What does this dataset contain?

• The first five fixtures come from FastAPI. Their expected team is backend.
• The next five fixtures come from Pydantic. Their expected team is data.
• The next five fixtures come from HTTPX. Their expected team is networking.
• The final five fixtures come from Typer. Their expected team is developer-experience.

• Save data/issues.json.
• Press Cmd+F on macOS or Ctrl+F on Windows.
• Search for "repository" in the file.

The search counter should show 20 matches. That confirms every fixture has a repository label.

Does the dataset show an error?

• Check that the file begins with [ and ends with ].
• Check for a comma after every object except the final object.
• Ask for help with the exact JSON error: Help me find the invalid JSON in data/issues.json without changing the issue titles.

✔️ Awesome, I've got everything!

Your dataset is ready. It contains 20 labeled fixtures across four repositories.

ⓧ I'd like to double check the full code

[
  {"repository":"fastapi/fastapi","title":"0.142.0: app fails to start when OTEL_EXPORTER_OTLP_ENDPOINT is set but the opentelemetry extra isn't installed","source_url":"https://github.com/fastapi/fastapi/issues/16417","expected_team":"backend"},
  {"repository":"fastapi/fastapi","title":"app.frontend() with fallback=\"index.html\" returns 404 for client side routes that have a dot in the last path segment (like /users/john.doe or /v/1.2.3)","source_url":"https://github.com/fastapi/fastapi/issues/16010","expected_team":"backend"},
  {"repository":"fastapi/fastapi","title":"0.137.0: `APIRoute.tags` invisible to type checkers in generate_unique_id_function (regression from #15745)","source_url":"https://github.com/fastapi/fastapi/issues/15764","expected_team":"backend"},
  {"repository":"fastapi/fastapi","title":"0.137.0: empty-path route nested under prefix-less outer include raises 'Prefix and path cannot be both empty'","source_url":"https://github.com/fastapi/fastapi/issues/15762","expected_team":"backend"},
  {"repository":"fastapi/fastapi","title":"Discriminated Unions Break When Wrapped in Annotated[Union, Body(...)] in FastAPI 0.124.1+","source_url":"https://github.com/fastapi/fastapi/issues/14508","expected_team":"backend"},
  {"repository":"pydantic/pydantic","title":"Publish the SkipJsonSchema example in the JSON Schema docs","source_url":"https://github.com/pydantic/pydantic/issues/13900","expected_team":"data"},
  {"repository":"pydantic/pydantic","title":"[bug] TypeError: 'MockValSer' object cannot be converted to 'SchemaSerializer'","source_url":"https://github.com/pydantic/pydantic/issues/12282","expected_team":"data"},
  {"repository":"pydantic/pydantic","title":"[Bug] Experimental pipeline .len() adds redundant Python-level validation for native length types","source_url":"https://github.com/pydantic/pydantic/issues/13653","expected_team":"data"},
  {"repository":"pydantic/pydantic","title":"_FieldNameStack/_ModelTypeStack lose their pop() on exception, corrupting later unrelated schema builds","source_url":"https://github.com/pydantic/pydantic/issues/13857","expected_team":"data"},
  {"repository":"pydantic/pydantic","title":"TypedDict with extra_items as PEP 695 type alias to Never or Annotated[Never, ...] crashes with PydanticSchemaGenerationError`","source_url":"https://github.com/pydantic/pydantic/issues/14018","expected_team":"data"},
  {"repository":"encode/httpx","title":"Allow URLs where username or password contains unescaped '@'.","source_url":"https://github.com/encode/httpx/issues/2985","expected_team":"networking"},
  {"repository":"encode/httpx","title":"httpx 0.24 documentation issue for netrc support","source_url":"https://github.com/encode/httpx/issues/2666","expected_team":"networking"},
  {"repository":"encode/httpx","title":"NetRCInfo.netrc_info() doesn't work when run in a systemd service with DynamicUser=true","source_url":"https://github.com/encode/httpx/issues/2532","expected_team":"networking"},
  {"repository":"encode/httpx","title":"Using a Client with a proxy to plain http endpoints returns 200 for the first, and 404 for any others.","source_url":"https://github.com/encode/httpx/issues/2488","expected_team":"networking"},
  {"repository":"encode/httpx","title":"ASGI/WSGI transport does not honor timeouts, but should","source_url":"https://github.com/encode/httpx/issues/2239","expected_team":"networking"},
  {"repository":"fastapi/typer","title":"bug: envvar not working for `typer.Options` with `typer==0.26.0`","source_url":"https://github.com/fastapi/typer/issues/1787","expected_team":"developer-experience"},
  {"repository":"fastapi/typer","title":"Overlap of typer and typer-slim causes package managemers to fail","source_url":"https://github.com/fastapi/typer/issues/1503","expected_team":"developer-experience"},
  {"repository":"fastapi/typer","title":"Version 0.19.1 breaks default_factory that returns empty list","source_url":"https://github.com/fastapi/typer/issues/1353","expected_team":"developer-experience"},
  {"repository":"fastapi/typer","title":"[BUG] help line length miscalculated when using stylized text","source_url":"https://github.com/fastapi/typer/issues/1159","expected_team":"developer-experience"},
  {"repository":"fastapi/typer","title":"Argument(metavar=\"MY_ARG\") has different meaning for the arguments panel compared to the usage text or the non-rich help output","source_url":"https://github.com/fastapi/typer/issues/1156","expected_team":"developer-experience"}
]

How should the file be organized?

Keep the fixtures in four groups of five. Each object must retain its repository, title, source URL, and expected team.

Run the shadow replay

The replay reuses GeminiClassifier and select_destination from the live control room. This keeps the evaluation aligned with the production path.

• Select the existing scripts folder in the VS Code file sidebar.
• Create a file named replay.py inside scripts.

You should see replay.py beside the existing configuration, smoke-test, and architecture-check scripts.

• Add the replay imports and file locations to scripts/replay.py by copying this code:

import json
from pathlib import Path

from control_room.application import select_destination
from control_room.domain import Ticket
from control_room.infrastructure import GeminiClassifier, SlackNotifier

DATASET_PATH = Path("data/issues.json")
RESULTS_PATH = Path("results.json")
HANDOFF_PATH = Path("HANDOFF.md")
MINUTES_SAVED_PER_AUTOMATED_CASE = 3

What do these values control?

• The imports reuse the existing classifier, ticket contract, routing policy, and Slack adapter.
• The three paths keep the dataset, machine-readable results, and human-readable handoff separate.
• The value in MINUTES_SAVED_PER_AUTOMATED_CASE is an explicit planning assumption. It is not measured return on investment.

• Save scripts/replay.py.
• Confirm that replay.py remains visible inside the scripts folder.

• Add the first runnable replay loop below MINUTES_SAVED_PER_AUTOMATED_CASE = 3 by copying this code:

def main() -> None:
    cases = json.loads(DATASET_PATH.read_text(encoding="utf-8"))
    classifier = GeminiClassifier()
    results: list[dict] = []

    for index, case in enumerate(cases, start=1):
        ticket = Ticket(
            title=case["title"],
            description=(
                f"Real GitHub issue from {case['repository']}. "
                f"Route the issue based on this title: {case['title']}"
            ),
            source_url=case["source_url"],
            expected_team=case["expected_team"],
        )
        decision = classifier.classify(ticket)
        destination, policy_reason = select_destination(decision)
        team_match = decision.team == ticket.expected_team
        false_automation = destination == "team-routing" and not team_match
        print(
            f"[{index:02d}/{len(cases)}] {case['repository']} -> "
            f"{decision.team} -> #{destination}"
        )


if __name__ == "__main__":
    main()

What does the replay loop do?

• Each fixture becomes a validated Ticket with its expected team attached.
• The classifier returns the same structured decision used by the interactive console.
• The routing policy decides whether the case qualifies for automation or human review.
• The progress line exposes the model team plus the policy destination for every case.

Before you run the first replay, do you expect every repository group to map cleanly to its expected team?

• Save scripts/replay.py.
• Run the first shadow replay from the repository folder with this command:

python scripts/replay.py

What should you see?

You should see 20 numbered progress lines. Each line shows the source repository, Gemini's selected team, and the policy destination.

This run makes 20 Gemini requests in sequence. The numbered lines confirm that the replay is continuing to process cases.

Did the first replay stop early?

• Confirm that your existing GEMINI_API_KEY remains available in this terminal session.
• Confirm that you launched the command from the repository folder containing data and scripts.
• Ask for help with the visible failure: Help me debug where scripts/replay.py stopped during the 20-case replay.

Progress lines show that the classifier can process the dataset. The next addition preserves each decision as evidence.

• In scripts/replay.py, find the line that defines false_automation.
• Insert this result record immediately below that line:

        results.append(
            {
                "ticket": ticket.model_dump(mode="json"),
                "decision": decision.model_dump(mode="json"),
                "destination": destination,
                "policy_reason": policy_reason,
                "team_match": team_match,
                "false_automation": false_automation,
            }
        )

What evidence does each record keep?

Each record preserves the original ticket, the structured model decision, the policy destination, and the policy reason. The final two fields show whether the model agreed with the fixture label and whether a mismatch escaped human review.

• Save scripts/replay.py.

• Insert the metric calculations below the replay loop and above if __name__ == "__main__": by copying this code:

    total = len(results)
    matches = sum(result["team_match"] for result in results)
    automated = sum(
        result["destination"] == "team-routing" for result in results
    )
    review_volume = total - automated
    false_automation_count = sum(
        result["false_automation"] for result in results
    )
    summary = {
        "total": total,
        "team_agreement_percent": round(matches / total * 100, 1),
        "automation_coverage_percent": round(automated / total * 100, 1),
        "human_review_volume": review_volume,
        "false_automation_count": false_automation_count,
        "planning_assumption_minutes_saved_per_automated_case": (
            MINUTES_SAVED_PER_AUTOMATED_CASE
        ),
        "estimated_minutes_saved": automated * MINUTES_SAVED_PER_AUTOMATED_CASE,
    }
    RESULTS_PATH.write_text(
        json.dumps({"summary": summary, "cases": results}, indent=2),
        encoding="utf-8",
    )

How are the metrics calculated?

• Team agreement measures how often the model team matches the repository-based expected team.
• Automation coverage measures how often the policy chooses team-routing.
• Human-review volume counts the cases sent to triage-review.
• False automation counts team mismatches that still reached team-routing.

Before you run the updated replay, which metric do you think is most important for deciding whether automation is safe?

• Save scripts/replay.py.
• Generate the machine-readable evaluation evidence with this command:

python scripts/replay.py

What changed after this run?

You should see all 20 progress lines again. You should also see results.json appear in the repository folder.

The file contains a seven-field summary plus a case record for every issue. This turns the replay into auditable evidence.

Is results.json missing?

• Check that the metric block uses four spaces of indentation so it remains inside main().
• Check that the metric block aligns with the for statement instead of its nested body.
• Ask for help with the file-writing stage: Help me fix the results.json generation in scripts/replay.py.

Machine-readable results support auditing. An engineering handoff also needs risks, impact framing, and a rollout recommendation.

• Insert the handoff generation below the RESULTS_PATH.write_text() call:

    handoff = f"""# AI Triage Control Room Handoff

## Shadow evaluation

- Cases: {summary['total']}
- Team agreement: {summary['team_agreement_percent']}%
- Automation coverage: {summary['automation_coverage_percent']}%
- Human-review volume: {summary['human_review_volume']}
- False automation count: {summary['false_automation_count']}

## Potential business impact

Using a planning assumption of {MINUTES_SAVED_PER_AUTOMATED_CASE} minutes saved per safely automated case, this replay estimates {summary['estimated_minutes_saved']} minutes saved across {summary['total']} cases. This is a planning estimate, not measured return on investment.

## Risks and rollout blockers

- The replay uses issue titles and repository context, not complete issue conversations.
- Model confidence is self-reported and has not been calibrated against historical outcomes.
- Any team mismatch routed to automation is a false automation risk that must be reviewed.
- Slack delivery proves routing visibility, not integration with a system of record.

## Recommendation

Keep the workflow in shadow mode until false automation is zero on a larger labeled dataset and confidence thresholds are calibrated with reviewer feedback.
"""
    HANDOFF_PATH.write_text(handoff, encoding="utf-8")

Why generate a handoff?

The handoff separates observed evaluation metrics from a planning assumption. It also records the limitations that should block a premature rollout.

Its recommendation requires zero false automation on a larger labeled dataset. It also requires calibrated confidence thresholds.

• Save scripts/replay.py.

A single summary in Slack makes the evaluation visible without flooding either channel with 20 individual issue messages.

• Insert the summary notification below HANDOFF_PATH.write_text() and above the final module entry point:

    SlackNotifier().send_text(
        "triage-review",
        (
            "*Shadow replay complete*\n"
            f"Cases: {summary['total']}\n"
            f"Team agreement: {summary['team_agreement_percent']}%\n"
            f"Automation coverage: {summary['automation_coverage_percent']}%\n"
            f"Human-review volume: {summary['human_review_volume']}\n"
            f"False automation count: {summary['false_automation_count']}"
        ),
    )
    print(f"Wrote {RESULTS_PATH} and {HANDOFF_PATH}.")

Why send one summary?

The replay keeps individual issues in shadow mode. Only the aggregate metrics are posted to triage-review.

This preserves operational visibility while avoiding 20 simulated routing messages in the team channels.

• Save scripts/replay.py.

✔️ Awesome, I've got everything!

The replay now classifies fixtures, applies the policy, writes both evidence files, and posts one evaluation summary.

ⓧ I'd like to double check the full code

import json
from pathlib import Path

from control_room.application import select_destination
from control_room.domain import Ticket
from control_room.infrastructure import GeminiClassifier, SlackNotifier

DATASET_PATH = Path("data/issues.json")
RESULTS_PATH = Path("results.json")
HANDOFF_PATH = Path("HANDOFF.md")
MINUTES_SAVED_PER_AUTOMATED_CASE = 3


def main() -> None:
    cases = json.loads(DATASET_PATH.read_text(encoding="utf-8"))
    classifier = GeminiClassifier()
    results: list[dict] = []

    for index, case in enumerate(cases, start=1):
        ticket = Ticket(
            title=case["title"],
            description=(
                f"Real GitHub issue from {case['repository']}. "
                f"Route the issue based on this title: {case['title']}"
            ),
            source_url=case["source_url"],
            expected_team=case["expected_team"],
        )
        decision = classifier.classify(ticket)
        destination, policy_reason = select_destination(decision)
        team_match = decision.team == ticket.expected_team
        false_automation = destination == "team-routing" and not team_match
        results.append(
            {
                "ticket": ticket.model_dump(mode="json"),
                "decision": decision.model_dump(mode="json"),
                "destination": destination,
                "policy_reason": policy_reason,
                "team_match": team_match,
                "false_automation": false_automation,
            }
        )
        print(
            f"[{index:02d}/{len(cases)}] {case['repository']} -> "
            f"{decision.team} -> #{destination}"
        )

    total = len(results)
    matches = sum(result["team_match"] for result in results)
    automated = sum(
        result["destination"] == "team-routing" for result in results
    )
    review_volume = total - automated
    false_automation_count = sum(
        result["false_automation"] for result in results
    )
    summary = {
        "total": total,
        "team_agreement_percent": round(matches / total * 100, 1),
        "automation_coverage_percent": round(automated / total * 100, 1),
        "human_review_volume": review_volume,
        "false_automation_count": false_automation_count,
        "planning_assumption_minutes_saved_per_automated_case": (
            MINUTES_SAVED_PER_AUTOMATED_CASE
        ),
        "estimated_minutes_saved": automated * MINUTES_SAVED_PER_AUTOMATED_CASE,
    }
    RESULTS_PATH.write_text(
        json.dumps({"summary": summary, "cases": results}, indent=2),
        encoding="utf-8",
    )

    handoff = f"""# AI Triage Control Room Handoff

## Shadow evaluation

- Cases: {summary['total']}
- Team agreement: {summary['team_agreement_percent']}%
- Automation coverage: {summary['automation_coverage_percent']}%
- Human-review volume: {summary['human_review_volume']}
- False automation count: {summary['false_automation_count']}

## Potential business impact

Using a planning assumption of {MINUTES_SAVED_PER_AUTOMATED_CASE} minutes saved per safely automated case, this replay estimates {summary['estimated_minutes_saved']} minutes saved across {summary['total']} cases. This is a planning estimate, not measured return on investment.

## Risks and rollout blockers

- The replay uses issue titles and repository context, not complete issue conversations.
- Model confidence is self-reported and has not been calibrated against historical outcomes.
- Any team mismatch routed to automation is a false automation risk that must be reviewed.
- Slack delivery proves routing visibility, not integration with a system of record.

## Recommendation

Keep the workflow in shadow mode until false automation is zero on a larger labeled dataset and confidence thresholds are calibrated with reviewer feedback.
"""
    HANDOFF_PATH.write_text(handoff, encoding="utf-8")

    SlackNotifier().send_text(
        "triage-review",
        (
            "*Shadow replay complete*\n"
            f"Cases: {summary['total']}\n"
            f"Team agreement: {summary['team_agreement_percent']}%\n"
            f"Automation coverage: {summary['automation_coverage_percent']}%\n"
            f"Human-review volume: {summary['human_review_volume']}\n"
            f"False automation count: {summary['false_automation_count']}"
        ),
    )
    print(f"Wrote {RESULTS_PATH} and {HANDOFF_PATH}.")


if __name__ == "__main__":
    main()

What should the completed script contain?

The file should load 20 fixtures, classify every case, apply the existing policy, calculate six evaluation values, and preserve every decision. It should then write results.json plus HANDOFF.md before sending one summary.

Read the engineering handoff

The final run produces the decision-ready evidence pack. Its most important question is whether any mismatched decision escaped review.

Before you run the completed replay, do you expect higher automation coverage to increase or decrease false automation risk?

• Run the completed evaluation from the repository folder with this command:

python scripts/replay.py

What should you see?

• The terminal processes all 20 issues with numbered progress lines.
• The final terminal line reports that results.json and HANDOFF.md were written.
• The triage-review channel receives one formatted shadow replay summary.
• The repository contains the JSON evidence plus the Markdown handoff.

You now have a repeatable evaluation instead of a handful of demonstrations. The control room can show how much work it would automate and where that automation could fail.

• Select HANDOFF.md in the VS Code file sidebar.
• Read the Shadow evaluation section to identify automation coverage.
• Compare human-review volume with the total of 20 cases.
• Check the false automation count for mismatches that escaped review.
• Read the Potential business impact section to separate the planning estimate from measured impact.
• Read the Risks and rollout blockers section to identify the limitation that should block rollout.

How should you interpret the evidence?

Useful automation coverage shows that the policy can remove some routine review work. Human-review volume shows the operational load retained by the safety boundary.

Any false automation means a mismatched issue reached the automated path. That result blocks rollout beyond shadow mode.

A zero count remains preliminary evidence. The recommendation still requires a larger labeled dataset plus confidence calibration with reviewer feedback.

Did the final evidence run fail?

• Use the last numbered progress line to identify which fixture completed before the failure.
• Check the local Slack configuration from the earlier step if all 20 cases finish before the exception appears.
• Ask for help with the failing stage: Help me diagnose whether my replay failed during classification, evidence writing, or Slack delivery.

Secret mission

Schedule a Weekly Slack Insight

Your shadow evaluation already captures the signals that matter. In this mission, you will turn the latest committed results into a weekly Slack insight that GitHub Actions can post on schedule.

Clean Up Your Resources

Clean Up Your Resources

Choose whether to keep the control room active, pause its weekly schedule, or remove its resources. Slack Free and the Gemini 3.8 Flash Free Tier can keep this project at $0.

Cost warning

The weekly workflow consumes GitHub Actions runner time. Standard GitHub-hosted runners are free for public repositories.

GitHub Free includes 2,000 standard-runner minutes each month for private repositories. Usage beyond that allowance can incur charges.

• Choose the Pause - I'll come back to this later or Delete - I don't want to use this again tab if recurring runs are no longer useful.

Resources you used:

• A Slack Free workspace containing #team-routing, #triage-review, one Slack app, and two incoming webhooks.
• A protected local webhook configuration file at ~/.config/ai-triage-control-room/slack-webhooks.json.
• The architecture and weekly workflow files in .github/workflows, plus the repository secret SLACK_WEBHOOK_URL.
• The Part 3 source code, scripts, issue dataset, architecture map, dependency pins, results.json, HANDOFF.md, and weekly-slack-payload.json.

Keep everything running

No action is needed. Choose this option if you plan to keep evaluating the router or extending the control room.

• Leave Weekly AI Triage Insights enabled so it can post the committed metrics every Monday at 09:17 UTC.
• Keep the SLACK_WEBHOOK_URL repository secret so the weekly workflow can reach #triage-review.
• Keep ~/.config/ai-triage-control-room/slack-webhooks.json so the local console and replay script can post to Slack.
• Retain results.json and HANDOFF.md as the evidence behind your automation decision.

Pause - I'll come back to this later

Pausing stops the scheduled digest while preserving the local control room, evidence pack, Slack configuration, and architecture gate.

• In your GitHub repository, select the Actions tab.
• Select Weekly AI Triage Insights in the workflows list.
• Open the workflow menu.
• Select Disable workflow.
• Leave Architecture Policy enabled so future pushes still check the code boundaries.
• Keep the local webhook configuration so you can resume without configuring Slack again.

That gives you a clean pause. Scheduled Slack posts stop while your local console and architecture gate stay ready.

Delete - I don't want to use this again

This option removes the Part 3 project files, repository automation, webhook secrets, and Slack control room.

Deleting the local files is permanent, but earlier commits remain available in your repository history.

Delete the repository secret:

• In your GitHub repository, select the Settings tab.
• Select Secrets and variables in the sidebar.
• Select Actions.
• Select SLACK_WEBHOOK_URL from the repository secrets.
• Delete the repository secret.

Remove the repository files and local configuration:

• Switch back to the terminal for your GitHub repository.
• Remove the Part 3 repository files and local Slack configuration by running these commands:

rm -f .github/workflows/architecture.yml .github/workflows/weekly-insights.yml
rm -f control_room/__init__.py control_room/__main__.py
rm -f control_room/application.py control_room/domain.py
rm -f control_room/infrastructure.py control_room/interface.py
rm -f scripts/check_architecture.py scripts/configure_slack.py
rm -f scripts/replay.py scripts/smoke_test_slack.py
rm -f scripts/weekly_summary.py
rm -f data/issues.json docs/architecture.md
rm -f requirements-control-room.txt results.json HANDOFF.md
rm -f weekly-slack-payload.json
rm -f ~/.config/ai-triage-control-room/slack-webhooks.json
rmdir ~/.config/ai-triage-control-room

What gets removed?

• The workflow commands remove the architecture gate and weekly schedule after you push the deletion commit.
• The project-file commands remove the Part 3 control room and its evidence pack.
• The final commands remove the protected webhook file and its empty configuration directory.
• Your existing virtual environment and GEMINI_API_KEY remain untouched.

• Commit these deletions in your existing repository.
• Push the deletion commit to the default branch.
• Return to the repository file list on GitHub.
• Confirm that the deleted Part 3 files no longer appear.
• Return to the repository secrets page.
• Confirm that SLACK_WEBHOOK_URL is no longer listed.

Still see a project file?

• Check that you ran the cleanup commands from the repository folder that contained results.json.
• Confirm that the deletion commit reached the default branch.
• Help me troubleshoot a Part 3 file that remains after this cleanup.

Remove the Slack control room:

• Return to the settings for the Slack app from this project.
• Select Incoming Webhooks.
• Revoke the webhook assigned to #team-routing.
• Revoke the webhook assigned to #triage-review.
• Remove the Slack app from the workspace through its app settings.
• Open the test workspace settings.
• Choose the workspace deletion option.
• Confirm the workspace deletion.

Cleanup complete. You should no longer be able to open #team-routing or #triage-review in the deleted workspace.

Nice Work!

Nice Work!

You did it! Your governed AI triage control room now turns software tickets into policy-checked routes with evidence for human review.

You've learned how to:

• Build a live triage workflow that gets validated structured decisions from Gemini. Send each visible route to Slack through channel-specific webhooks.
• Govern automation with deterministic guardrails based on urgency plus a 0.75 confidence threshold. Document runtime boundaries with a C4 container map. Enforce code dependencies through a local gate mirrored in GitHub Actions.
• Replay 20 real issues through a shadow evaluation. Produce an engineering evidence pack in results.json plus HANDOFF.md. It reports agreement, automation coverage, human-review volume, false automation risk, plus estimated time saved.
• Secret Mission: Schedule a weekly triage insight with GitHub Actions. Post the latest committed evaluation to Slack without exposing the webhook secret.

Ready to quiz yourself?