AI API Risk and Spend Observatory

Build a dashboard that detects AI API spend and key-risk anomalies.

Introduction

30 Second Summary

A sudden jump in software costs can hide inside thousands of ordinary requests. When every row looks similar, risky activity is easy to miss.

In this project, you will build a local dashboard that turns sanitized AI API telemetry into a ranked incident view. You'll investigate simulated spend through masked key identifiers without storing a real credential.

What You'll Build

Picture your local Streamlit dashboard narrowing to oai-dev-44aa as responsive charts expose its simulated spend spike beside a ranked incident explanation.

By the end of this project, you'll have:

  • A provider-neutral telemetry view that compares synthetic Anthropic, OpenAI, and Google activity through one shared schema.
  • Interactive Plotly charts controlled by Streamlit filters for isolating any masked key or workload. You can immediately see its requests, simulated cost, errors, latency, and risk score.
  • A ranked incident table powered by pandas rolling baselines. Each flagged key includes a plain-language reason such as Spend spike vs rolling median.
  • Secret Mission: Add a validated CSV upload path that checks the shared schema before replacing the bundled telemetry.

Are there any prerequisites?

You should be comfortable analyzing data with pandas. Your Windows computer should already have Python, VS Code, and Git installed.

Before We Start

This opening checkpoint locks in the purpose of your local Streamlit dashboard before you start building. You will analyze synthetic AI API telemetry for spend, errors, traffic, latency, rate-limit pressure, and masked-key risk without using or exposing a real credential.

Set Up the Windows Project

Package conflicts can make a local dashboard fail before its analysis even starts. This step gives the project a reproducible Python workspace.

A virtual environment isolates the pinned packages. The version check confirms Streamlit can run on your machine.

You will use VS Code to manage the project files. PowerShell provides the command loop for setup and testing.

In this step, get ready to:
  • Create the Windows workspace and verify your Python version.
  • Build an isolated environment with the pinned project packages.
  • Launch the Streamlit Hello app in your browser.
Create the workspace and verify Python

Streamlit supports Python 3.11 through 3.15. Checking this range now prevents an incompatible runtime from undermining the rest of the project.

  • Press the Windows key to open Windows search.
  • Type Visual Studio Code and press Enter.
  • Click File in the top menu.
  • Select Open Folder....
  • Select your Desktop folder.
  • Select New Folder.
  • Enter ai-api-risk-observatory as the folder name.
  • Select Select Folder.

VS Code now treats ai-api-risk-observatory as the workspace. You will see its name at the top of the Explorer sidebar.

  • Select Yes, I trust the authors if the Workspace Trust dialog appears.
  • Click Terminal in the top menu.
  • Select New Terminal.
  • Open the Launch Profile dropdown if the terminal is not using PowerShell.
  • Select PowerShell from the available terminal profiles.

The integrated terminal starts inside the open workspace. Its path should end with ai-api-risk-observatory.

Before you check, do you expect your installed Python version to fall inside Streamlit's supported range?

  • Check your Python version by running this command:
python --version

What does this command do?

The command prints the installed Python version. It exits after showing the result.

✔️ I see version 3.11 through 3.15

That compatibility check is cleared. Your Python version can run the packages used in this project.

ⓧ I see a version older than 3.11

Your installed version is below Streamlit's supported range. Python 3.15.0 brings the workspace into the supported range.

The installer opens a guided setup. Complete its prompts before returning to VS Code.

  • Visit the official Python downloads page.
  • Download Python 3.15.0 for Windows.
  • Run the downloaded installer.
  • Restart VS Code after the installation completes.
  • Return to the ai-api-risk-observatory workspace.
  • Repeat the version check above in a new PowerShell terminal.

ⓧ Command not found

Windows cannot currently find a Python installation. Installing Python 3.15.0 supplies the supported runtime this project needs.

The installer opens a guided setup. Complete its prompts before returning to VS Code.

  • Visit the official Python downloads page.
  • Download Python 3.15.0 for Windows.
  • Run the downloaded installer.
  • Restart VS Code after the installation completes.
  • Return to the ai-api-risk-observatory workspace.
  • Repeat the version check above in a new PowerShell terminal.
Build the isolated environment

The project pins each package to a known version in requirements.txt. The virtual environment keeps those versions separate from packages used by other Python projects.

  • Select the New File... icon in the Explorer sidebar.
  • Enter requirements.txt as the file name.
  • Add the pinned project packages by pasting these lines into the file:
pandas==3.0.6
streamlit==1.66.0
plotly==7.1.0

What do these packages provide?

  • Pandas 3.0.6 loads the telemetry and calculates grouped metrics.
  • Streamlit 1.66.0 turns the Python analysis into a local browser dashboard.
  • Plotly.py 7.1.0 provides the interactive investigation charts.
  • Save requirements.txt in VS Code.
  • Confirm requirements.txt appears inside ai-api-risk-observatory in the Explorer sidebar.

Don't see requirements.txt?

  • Check that the Explorer sidebar shows ai-api-risk-observatory as the open folder.
  • Check that the file name ends in .txt only once.

Still stuck? Help me create requirements.txt in the correct VS Code workspace.

✔️ Awesome, I've got everything!

Great. Save requirements.txt before creating the virtual environment.

ⓧ I'd like to double check the full code

Compare your file with this complete version.

pandas==3.0.6
streamlit==1.66.0
plotly==7.1.0
  • Create the .venv virtual environment by running this command:
python -m venv .venv

What does this command do?

Python creates an isolated environment inside the .venv folder. Packages installed after activation stay scoped to this project.

  • Confirm the .venv folder appears in the VS Code Explorer.

The new folder is visible proof that the isolated environment exists.

Don't see the .venv folder?

  • Confirm the PowerShell path ends with ai-api-risk-observatory.
  • Check the terminal output for a message about a missing Python installation.

Need help? Help me troubleshoot why Python did not create my .venv folder.

  • Activate the new environment in PowerShell by running this command:
.venv\Scripts\Activate.ps1

What does activation change?

Activation points Python and pip at the copies inside .venv. The environment name appears at the beginning of the PowerShell prompt.

  • Confirm the PowerShell prompt begins with the .venv environment name.

Your next package installation now belongs to this project environment.

Did PowerShell block activation?

PowerShell's script policy can make this part fiddly. The fallback below allows locally created activation scripts for your Windows user account.

  • Update the current user's execution policy by running this command:
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

PowerShell updates the policy for the current user. Respond to its confirmation prompt to apply the change.

  • Reactivate the environment by running this command:
.venv\Scripts\Activate.ps1

The activation command now points the terminal at the project environment. Help me resolve a blocked PowerShell virtual environment activation.

  • Install every package from requirements.txt by running this command:
pip install -r requirements.txt

What happens during installation?

Pip reads each exact pin from requirements.txt. It installs those packages inside the active .venv environment.

Leave PowerShell running until the prompt returns. The completed install gives the project a repeatable dependency set.

When the prompt returns without an installation failure, pandas 3.0.6, Streamlit 1.66.0, and Plotly.py 7.1.0 are available inside .venv.

Did the package installation fail?

  • Confirm the PowerShell prompt still shows the active .venv environment.
  • Confirm requirements.txt contains the three exact package pins from the double-check tab.
  • Confirm your Python version remains inside the supported range from the first check.

Need another pair of eyes? Help me diagnose my pinned package installation.

Launch the Streamlit Hello app

The included Hello app provides a full setup check. A successful launch proves the activated environment can start a local Streamlit server and open its browser interface.

No real API key is needed for this check. The project workspace remains free of credential values.

Before you run this, do you expect the command to stay inside PowerShell or open a page in your browser?

  • Launch the included Streamlit Hello app by running this command:
streamlit hello

What happens when Streamlit starts?

Streamlit starts its included example app through a local server. It opens the Hello app in your default browser.

PowerShell stays occupied while the server is running. This keeps the browser page connected to the local app.

  • Switch to the browser page opened by Streamlit.
  • Confirm the Streamlit Hello app is visible in the browser.

That is the setup working end to end. Your activated environment can now serve a Streamlit app locally.

Didn't the Hello app open?

  • Confirm the PowerShell prompt showed the active .venv environment before you started Streamlit.
  • Try the documented long-form command:
python -m streamlit hello

This form launches Streamlit through the active Python interpreter. It should open the same included Hello app.

Still blocked? Help me launch Streamlit Hello from my activated Windows environment.

  • Return to PowerShell after capturing your screenshot.
  • Press Ctrl+C to stop the Hello app server.

You have cleared the setup hurdle. Streamlit now runs from an isolated Windows environment with the three pinned packages.

Next, you will load the bundled synthetic telemetry and turn its overall totals into your first working dashboard.

Reveal the Telemetry

Your Streamlit environment is ready. The observatory can now move from its demo page to its own data.

Overall totals give you a fast dashboard. They leave suspicious activity buried among 48 events.

First, you create a sanitized telemetry dataset. Then, you build a working dashboard that reveals the totals plus the underlying events.

In this step, get ready to:
  • Create 48 rows of synthetic provider-neutral telemetry.
  • Calculate four baseline metrics from the telemetry.
  • Display the raw events in a working browser dashboard.
Create the synthetic telemetry file

A provider-neutral telemetry schema gives every provider the same columns. The shared structure lets you compare records without storing provider credentials.

This CSV uses invented values in key_id. Every value in cost_usd is synthetic tutorial data.

  • Create a folder named data from the file sidebar in VS Code.

You should see the empty data folder inside ai-api-risk-observatory.

  • Create a file named api_usage.csv inside the data folder.

The blank data/api_usage.csv file should open in your editor.

  • Add the header plus the first 24 telemetry events by pasting this block into data/api_usage.csv:
timestamp,provider,key_id,project,workload,model_tier,requests,tokens,cost_usd,errors,avg_latency_ms,rate_limit_hits
2026-10-13T09:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,120,180000,0.90,1,620,0
2026-10-13T10:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,128,192000,0.96,2,640,0
2026-10-13T11:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,125,188000,0.94,1,610,0
2026-10-13T12:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,132,198000,0.99,2,655,0
2026-10-13T13:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,130,195000,0.98,2,645,0
2026-10-13T14:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,136,204000,1.02,2,670,0
2026-10-13T15:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,620,930000,5.80,46,1300,8
2026-10-13T16:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,140,210000,1.05,3,680,1
2026-10-13T09:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,80,120000,0.40,0,880,0
2026-10-13T10:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,84,126000,0.42,1,900,0
2026-10-13T11:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,86,129000,0.43,0,870,0
2026-10-13T12:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,88,132000,0.44,1,910,0
2026-10-13T13:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,90,135000,0.45,0,895,0
2026-10-13T14:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,92,138000,0.46,1,920,0
2026-10-13T15:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,89,133500,0.45,0,905,0
2026-10-13T16:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,94,141000,0.47,1,930,0
2026-10-13T09:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,200,260000,1.20,3,500,0
2026-10-13T10:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,215,279500,1.29,4,520,0
2026-10-13T11:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,210,273000,1.26,3,510,0
2026-10-13T12:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,220,286000,1.32,4,535,0
2026-10-13T13:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,225,292500,1.35,5,540,0
2026-10-13T14:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,218,283400,1.31,40,980,12
2026-10-13T15:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,230,299000,1.38,4,550,0
2026-10-13T16:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,235,305500,1.41,5,560,0

What Does This Data Establish?

  • The header defines one shared structure for every telemetry event.
  • The Anthropic records contain one support workload plus one document workload.
  • The first OpenAI records contain a production support workload.
  • The masked identifiers support grouping without exposing a secret value.
  • Save data/api_usage.csv.
  • Check the line count in the VS Code status bar.

You should see 25 lines. That count includes one header plus 24 telemetry events.

Does the First Half Look Misaligned?

  • Confirm the first line starts with timestamp.
  • Confirm each record occupies one line.
  • Remove any blank line above the header.

Still seeing uneven columns? Help me check the first half of my telemetry CSV.

The second half adds a development key plus both Google workloads. It completes the six masked identifiers used throughout the project.

  • Append the remaining 24 telemetry events at the end of data/api_usage.csv by pasting this block:
2026-10-13T09:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,30,42000,0.15,0,450,0
2026-10-13T10:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,35,49000,0.17,1,470,0
2026-10-13T11:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,33,46200,0.16,0,455,0
2026-10-13T12:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,38,53200,0.18,1,480,0
2026-10-13T13:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,36,50400,0.17,0,465,0
2026-10-13T14:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,40,56000,0.19,1,490,0
2026-10-13T15:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,39,54600,0.18,0,485,0
2026-10-13T16:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,premium,42,180000,1.20,1,720,0
2026-10-13T09:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,150,210000,0.75,2,570,0
2026-10-13T10:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,155,217000,0.78,2,585,0
2026-10-13T11:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,148,207200,0.74,1,560,0
2026-10-13T12:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,160,224000,0.80,2,590,0
2026-10-13T13:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,700,980000,3.70,18,1180,6
2026-10-13T14:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,165,231000,0.82,2,600,0
2026-10-13T15:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,158,221200,0.79,2,580,0
2026-10-13T16:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,170,238000,0.85,3,610,0
2026-10-13T09:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,45,58500,0.23,0,430,0
2026-10-13T10:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,48,62400,0.24,1,440,0
2026-10-13T11:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,46,59800,0.23,0,425,0
2026-10-13T12:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,50,65000,0.25,1,445,0
2026-10-13T13:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,52,67600,0.26,0,450,0
2026-10-13T14:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,49,63700,0.25,1,438,0
2026-10-13T15:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,53,68900,0.27,0,455,0
2026-10-13T16:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,55,71500,0.28,1,460,0

What Completes the Dataset?

  • The development OpenAI key has a late synthetic cost increase.
  • The production Google key has an injected request spike.
  • The Google lab key provides steadier comparison data.
  • Every provider now contributes eight hourly records for each masked key.
  • Save data/api_usage.csv.
  • Check the completed file's line count in VS Code.

You should see 49 lines. The file now contains one header plus 48 synthetic telemetry events.

Good progress. Your observatory now has a safe dataset that represents Anthropic, OpenAI, and Google without containing a real API key.

Is the Final Row Count Wrong?

  • Confirm the file contains the header only once.
  • Confirm the second block begins immediately after the final production OpenAI record.
  • Confirm the final record ends with gai-lab-18c4 data from 2026-10-13T16:00:00.

Need another pair of eyes? Help me find why my telemetry CSV does not contain 48 data rows.

✔️ Awesome, I've got everything!

Your completed data/api_usage.csv file should contain 48 synthetic events across six masked key identifiers.

ⓧ I'd like to double check the full code

timestamp,provider,key_id,project,workload,model_tier,requests,tokens,cost_usd,errors,avg_latency_ms,rate_limit_hits
2026-10-13T09:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,120,180000,0.90,1,620,0
2026-10-13T10:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,128,192000,0.96,2,640,0
2026-10-13T11:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,125,188000,0.94,1,610,0
2026-10-13T12:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,132,198000,0.99,2,655,0
2026-10-13T13:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,130,195000,0.98,2,645,0
2026-10-13T14:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,136,204000,1.02,2,670,0
2026-10-13T15:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,620,930000,5.80,46,1300,8
2026-10-13T16:00:00,Anthropic,ant-prod-7f2a,customer-support,support-chat,standard,140,210000,1.05,3,680,1
2026-10-13T09:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,80,120000,0.40,0,880,0
2026-10-13T10:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,84,126000,0.42,1,900,0
2026-10-13T11:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,86,129000,0.43,0,870,0
2026-10-13T12:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,88,132000,0.44,1,910,0
2026-10-13T13:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,90,135000,0.45,0,895,0
2026-10-13T14:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,92,138000,0.46,1,920,0
2026-10-13T15:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,89,133500,0.45,0,905,0
2026-10-13T16:00:00,Anthropic,ant-batch-90d1,finance-ops,document-summary,economy,94,141000,0.47,1,930,0
2026-10-13T09:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,200,260000,1.20,3,500,0
2026-10-13T10:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,215,279500,1.29,4,520,0
2026-10-13T11:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,210,273000,1.26,3,510,0
2026-10-13T12:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,220,286000,1.32,4,535,0
2026-10-13T13:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,225,292500,1.35,5,540,0
2026-10-13T14:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,218,283400,1.31,40,980,12
2026-10-13T15:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,230,299000,1.38,4,550,0
2026-10-13T16:00:00,OpenAI,oai-prod-31b8,customer-support,support-chat,premium,235,305500,1.41,5,560,0
2026-10-13T09:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,30,42000,0.15,0,450,0
2026-10-13T10:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,35,49000,0.17,1,470,0
2026-10-13T11:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,33,46200,0.16,0,455,0
2026-10-13T12:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,38,53200,0.18,1,480,0
2026-10-13T13:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,36,50400,0.17,0,465,0
2026-10-13T14:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,40,56000,0.19,1,490,0
2026-10-13T15:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,standard,39,54600,0.18,0,485,0
2026-10-13T16:00:00,OpenAI,oai-dev-44aa,dev-tools,code-assistant,premium,42,180000,1.20,1,720,0
2026-10-13T09:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,150,210000,0.75,2,570,0
2026-10-13T10:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,155,217000,0.78,2,585,0
2026-10-13T11:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,148,207200,0.74,1,560,0
2026-10-13T12:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,160,224000,0.80,2,590,0
2026-10-13T13:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,700,980000,3.70,18,1180,6
2026-10-13T14:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,165,231000,0.82,2,600,0
2026-10-13T15:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,158,221200,0.79,2,580,0
2026-10-13T16:00:00,Google,gai-prod-872e,customer-support,support-chat,standard,170,238000,0.85,3,610,0
2026-10-13T09:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,45,58500,0.23,0,430,0
2026-10-13T10:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,48,62400,0.24,1,440,0
2026-10-13T11:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,46,59800,0.23,0,425,0
2026-10-13T12:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,50,65000,0.25,1,445,0
2026-10-13T13:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,52,67600,0.26,0,450,0
2026-10-13T14:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,49,63700,0.25,1,438,0
2026-10-13T15:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,53,68900,0.27,0,455,0
2026-10-13T16:00:00,Google,gai-lab-18c4,dev-tools,code-assistant,economy,55,71500,0.28,1,460,0

How to Use This Reference

Compare the header plus all 48 records with your saved file. The first data row should use ant-prod-7f2a.

The final data row should use gai-lab-18c4. This confirms that both pasted sections are present.

Build the baseline metrics

The pandas library reads the CSV into a table that Python can calculate from. The first app view uses that table to summarize volume plus simulated spend.

  • Create a file named app.py from the file sidebar in VS Code.

The blank app.py file should appear beside requirements.txt inside ai-api-risk-observatory.

  • Build the dashboard shell by pasting this code into app.py:
import pandas as pd
import streamlit as st

st.set_page_config(
    page_title="AI API Key Risk and Spend Observatory",
    layout="wide",
)
st.title("AI API Key Risk and Spend Observatory")
st.caption(
    "Synthetic telemetry only. Costs and detection thresholds are tutorial values, "
    "not vendor prices, quotas, or security guarantees."
)

What Does This Code Do?

  • The imports make pandas plus Streamlit available to the app.
  • The page configuration gives the dashboard a browser title plus a wide layout.
  • The visible title identifies the observatory.
  • The caption keeps the tutorial values separate from vendor claims.
  • Save app.py.

Before you run the app, what do you expect the browser to show from this page shell?

  • Start the dashboard from the activated PowerShell environment by running this command:
streamlit run app.py

What Does This Command Do?

This starts the local Streamlit app from app.py. It also opens the app in your default browser.

You should see the observatory title plus a caption explaining that the telemetry is synthetic. The PowerShell terminal remains occupied while the local app is running.

Did the Dashboard Fail to Open?

  • Confirm the virtual environment from the previous step is still active.
  • Confirm app.py sits directly inside ai-api-risk-observatory.
  • Confirm PowerShell is using the ai-api-risk-observatory folder.

Still blocked? Help me troubleshoot why my local Streamlit page does not open.

The page shell proves the app can run. It now needs calculations that turn 48 rows into a quick operational summary.

  • Add the loading plus metric code below the existing caption in app.py by pasting this block:
data = pd.read_csv("data/api_usage.csv")
data["timestamp"] = pd.to_datetime(data["timestamp"], errors="raise")

total_requests = int(sum(data["requests"]))
total_cost = float(sum(data["cost_usd"]))
total_errors = int(sum(data["errors"]))
error_rate = total_errors / total_requests * 100 if total_requests else 0
telemetry_rows = len(data)

metric_columns = st.columns(4)
metric_columns[0].metric("Requests", f"{total_requests:,}")
metric_columns[1].metric("Simulated cost", f"${total_cost:,.2f}")
metric_columns[2].metric("Error rate", f"{error_rate:.1f}%")
metric_columns[3].metric("Telemetry rows", telemetry_rows)

How Are the Metrics Calculated?

  • The CSV loader turns data/api_usage.csv into the data table.
  • The timestamp conversion checks that each timestamp can be parsed.
  • The total variables sum requests plus synthetic cost plus errors.
  • The four metric columns present the calculated values as dashboard cards.
  • Save app.py.

Before you return to the browser, which metric do you think confirms that the complete CSV loaded?

  • Return to the browser tab running your observatory.

You should see four metric cards. The Telemetry rows card should show 48.

The cards confirm that the dataset loaded. They do not identify which masked key produced the unusual events.

Are the Metrics Missing?

  • Confirm the saved CSV path is data/api_usage.csv.
  • Confirm the CSV header includes requests plus cost_usd plus errors.
  • Confirm app.py uses the same column spellings as the CSV header.

Need help with the calculation? Help me diagnose why my four Streamlit metrics are missing or incorrect.

Add the raw event table

The raw table exposes every event behind the totals. Scanning all 48 records also demonstrates why totals plus unranked rows are a weak incident view.

  • Add the raw telemetry table at the end of app.py by pasting this block:
st.caption("Raw synthetic telemetry")
st.dataframe(
    data,
    width="stretch",
    hide_index=True,
    alt="Synthetic API telemetry events with masked key identifiers.",
)

What Does This Table Add?

  • The caption tells viewers that the table contains synthetic telemetry.
  • The dataframe displays every column from the loaded CSV.
  • The stretched width gives the operational fields more horizontal space.
  • The hidden index keeps the focus on the event data.
  • Save app.py.

Before you check the result, do you think the deliberately unusual events will stand out immediately among all 48 rows?

  • Return to the browser tab running the dashboard.
  • Scroll through the raw synthetic telemetry table.

You should see 48 telemetry rows beneath the four metrics. The table should contain records labeled Anthropic, OpenAI, and Google.

The injected spikes are present in the table. Finding the affected masked keys still requires careful scanning.

Is the Raw Table Missing?

  • Confirm the table code appears after the four metric calls in app.py.
  • Confirm the local Streamlit process is still running in PowerShell.
  • Confirm the browser shows the latest saved version of the app.

Still missing the table? Help me find why my Streamlit dataframe is not displaying the loaded telemetry.

✔️ Awesome, I've got everything!

Your app.py file should now load the synthetic CSV. It should render four metrics plus the complete raw event table.

ⓧ I'd like to double check the full code

import pandas as pd
import streamlit as st

st.set_page_config(
    page_title="AI API Key Risk and Spend Observatory",
    layout="wide",
)
st.title("AI API Key Risk and Spend Observatory")
st.caption(
    "Synthetic telemetry only. Costs and detection thresholds are tutorial values, "
    "not vendor prices, quotas, or security guarantees."
)

data = pd.read_csv("data/api_usage.csv")
data["timestamp"] = pd.to_datetime(data["timestamp"], errors="raise")

total_requests = int(sum(data["requests"]))
total_cost = float(sum(data["cost_usd"]))
total_errors = int(sum(data["errors"]))
error_rate = total_errors / total_requests * 100 if total_requests else 0
telemetry_rows = len(data)

metric_columns = st.columns(4)
metric_columns[0].metric("Requests", f"{total_requests:,}")
metric_columns[1].metric("Simulated cost", f"${total_cost:,.2f}")
metric_columns[2].metric("Error rate", f"{error_rate:.1f}%")
metric_columns[3].metric("Telemetry rows", telemetry_rows)

st.caption("Raw synthetic telemetry")
st.dataframe(
    data,
    width="stretch",
    hide_index=True,
    alt="Synthetic API telemetry events with masked key identifiers.",
)

What Should Match?

Compare this reference with your saved app.py file from top to bottom. The data loading block should appear between the page caption plus the metric calculations.

The raw dataframe block should be the final section. Matching that order keeps every variable available before Streamlit renders it.

That is your first observability view working. Next, you will turn the unranked events into a key-level incident ranking.

Rank Key-Level Incidents

Your Streamlit dashboard now proves that all 48 telemetry events load safely. However, an incident responder still has to hunt through the raw rows to find the masked key that needs attention.

This step uses pandas to score fixed error conditions. It then groups those scores into a ranked key-level view. The app also prepares Plotly Express for the investigation views that follow.

In this step, get ready to:
  • Score high error rates and rate-limit pressure with fixed tutorial rules.
  • Group event-level signals into a ranked masked-key summary.
  • Inspect the incident ranking to uncover the limitation of static rules.
Score fixed incident conditions

A fixed incident rule checks every event against the same condition. The 10 percent error threshold is a project-specific teaching choice. A positive rate_limit_hits value supplies the second condition.

How the score works

A high error rate contributes two points to the event's risk_score. Rate-limit pressure contributes one point.

These weights help the table prioritize tutorial incidents. They do not represent provider quotas or universal security rules.

  • In app.py, find the import lines at the top of the file.
  • Replace those lines with this import block and the two reason helpers:
import pandas as pd
import plotly.express as px
import streamlit as st


def explain_risk(error_rate, rate_limit_hits):
    reasons = []
    if error_rate >= 10:
        reasons.append("High error rate")
    if rate_limit_hits > 0:
        reasons.append("Rate-limit pressure")
    return ", ".join(reasons) if reasons else "Within baseline"


def combine_reasons(values):
    active_reasons = sorted(
        {value for value in values if value != "Within baseline"}
    )
    return "; ".join(active_reasons) if active_reasons else "Within baseline"

What do these helpers do?

  • The explain_risk() helper converts fixed conditions into readable incident reasons.
  • The combine_reasons() helper removes repeated reasons when several events belong to the same masked key.
  • The plotly.express import keeps the charting library available for the investigation layer.
  • In app.py, find the timestamp parsing line.
  • Add the event-level calculations directly below that line:
data["error_rate_pct"] = data["errors"] / data["requests"] * 100

data["risk_score"] = [
    int(error_rate >= 10) * 2
    + int(rate_limit_hits > 0)
    for error_rate, rate_limit_hits in zip(
        data["error_rate_pct"],
        data["rate_limit_hits"],
    )
]
data["reason"] = [
    explain_risk(error_rate, rate_limit_hits)
    for error_rate, rate_limit_hits in zip(
        data["error_rate_pct"],
        data["rate_limit_hits"],
    )
]

What do the fixed rules do?

  • The error_rate_pct column converts each event's error count into a percentage of its requests.
  • The risk_score list evaluates both tutorial conditions for every row.
  • The reason list explains each score with readable text such as High error rate.
  • Save app.py.
  • Return to the browser tab running the dashboard.
  • Scroll to the right side of the raw telemetry table.

Streamlit reruns the app after the save. You should see the new error_rate_pct, risk_score, and reason columns in the raw table.

New columns missing?

  • Check that the new calculations sit below the line that creates data["timestamp"].
  • Check that both helper functions sit above the page configuration code.
  • Confirm that app.py is saved before checking the browser.

Still stuck? Help me debug the missing incident columns.

Group and rank masked keys

Event scores reveal individual problems. An incident view becomes useful when those events are grouped by masked key and operational context.

Named aggregation gives every result column a clear purpose. Sorting by total risk score places the strongest fixed-rule signals first.

  • In app.py, find the four metric lines that end with the telemetry row metric.
  • Add the key-level aggregation below those metric lines:
key_summary = (
    data.groupby(
        ["provider", "key_id", "project", "workload", "model_tier"],
        as_index=False,
    )
    .agg(
        total_requests=("requests", "sum"),
        total_cost_usd=("cost_usd", "sum"),
        avg_latency_ms=("avg_latency_ms", "mean"),
        max_error_rate_pct=("error_rate_pct", "max"),
        rate_limit_hits=("rate_limit_hits", "sum"),
        risk_score=("risk_score", "sum"),
        reasons=("reason", combine_reasons),
    )
    .sort_values(
        ["risk_score", "total_cost_usd"],
        ascending=[False, False],
    )
)

What does this aggregation measure?

  • The grouping columns keep different providers and operational contexts separate.
  • The named calculations total requests and simulated cost for each grouped key.
  • The summary keeps average latency and the highest event error rate.
  • The final sort ranks higher risk scores first. Simulated cost breaks ties.
  • Add the ranked incident table directly below the key_summary calculation:
st.caption("Ranked incident investigation table")
st.dataframe(
    key_summary[
        [
            "provider",
            "key_id",
            "project",
            "workload",
            "model_tier",
            "total_requests",
            "total_cost_usd",
            "avg_latency_ms",
            "max_error_rate_pct",
            "rate_limit_hits",
            "risk_score",
            "reasons",
        ]
    ],
    width="stretch",
    hide_index=True,
    alt="Masked API key incident ranking.",
)

Why show key-level incidents?

The raw table preserves event detail. The ranked table compresses repeated events into one operational summary for each masked key and model tier.

The reasons column connects each score to an observable condition. An investigator can see why a row deserves attention.

  • Save app.py.
  • Return to the browser tab running the dashboard.

The app reruns with a ranked incident investigation table above the raw telemetry. You should see masked keys ordered by risk score and simulated cost.

Incident table not rendering?

  • Check that key_summary is defined before the ranked table tries to display it.
  • Check that every aggregation source matches a column in data/api_usage.csv.
  • Check the commas after each named aggregation entry.

Need another set of eyes? Help me troubleshoot the key-level aggregation.

✔️ Awesome, I've got everything!

Your saved app.py now scores fixed conditions and renders the key-level incident ranking.

ⓧ I'd like to double check the full code

import pandas as pd
import plotly.express as px
import streamlit as st


def explain_risk(error_rate, rate_limit_hits):
    reasons = []
    if error_rate >= 10:
        reasons.append("High error rate")
    if rate_limit_hits > 0:
        reasons.append("Rate-limit pressure")
    return ", ".join(reasons) if reasons else "Within baseline"


def combine_reasons(values):
    active_reasons = sorted(
        {value for value in values if value != "Within baseline"}
    )
    return "; ".join(active_reasons) if active_reasons else "Within baseline"


st.set_page_config(
    page_title="AI API Key Risk and Spend Observatory",
    layout="wide",
)
st.title("AI API Key Risk and Spend Observatory")
st.caption(
    "Synthetic telemetry only. Costs and detection thresholds are tutorial values, "
    "not vendor prices, quotas, or security guarantees."
)

data = pd.read_csv("data/api_usage.csv")
data["timestamp"] = pd.to_datetime(data["timestamp"], errors="raise")
data["error_rate_pct"] = data["errors"] / data["requests"] * 100

data["risk_score"] = [
    int(error_rate >= 10) * 2
    + int(rate_limit_hits > 0)
    for error_rate, rate_limit_hits in zip(
        data["error_rate_pct"],
        data["rate_limit_hits"],
    )
]
data["reason"] = [
    explain_risk(error_rate, rate_limit_hits)
    for error_rate, rate_limit_hits in zip(
        data["error_rate_pct"],
        data["rate_limit_hits"],
    )
]

total_requests = int(sum(data["requests"]))
total_cost = float(sum(data["cost_usd"]))
total_errors = int(sum(data["errors"]))
error_rate = total_errors / total_requests * 100 if total_requests else 0
telemetry_rows = len(data)

metric_columns = st.columns(4)
metric_columns[0].metric("Requests", f"{total_requests:,}")
metric_columns[1].metric("Simulated cost", f"${total_cost:,.2f}")
metric_columns[2].metric("Error rate", f"{error_rate:.1f}%")
metric_columns[3].metric("Telemetry rows", telemetry_rows)

key_summary = (
    data.groupby(
        ["provider", "key_id", "project", "workload", "model_tier"],
        as_index=False,
    )
    .agg(
        total_requests=("requests", "sum"),
        total_cost_usd=("cost_usd", "sum"),
        avg_latency_ms=("avg_latency_ms", "mean"),
        max_error_rate_pct=("error_rate_pct", "max"),
        rate_limit_hits=("rate_limit_hits", "sum"),
        risk_score=("risk_score", "sum"),
        reasons=("reason", combine_reasons),
    )
    .sort_values(
        ["risk_score", "total_cost_usd"],
        ascending=[False, False],
    )
)

st.caption("Ranked incident investigation table")
st.dataframe(
    key_summary[
        [
            "provider",
            "key_id",
            "project",
            "workload",
            "model_tier",
            "total_requests",
            "total_cost_usd",
            "avg_latency_ms",
            "max_error_rate_pct",
            "rate_limit_hits",
            "risk_score",
            "reasons",
        ]
    ],
    width="stretch",
    hide_index=True,
    alt="Masked API key incident ranking.",
)

st.caption("Raw synthetic telemetry")
st.dataframe(
    data,
    width="stretch",
    hide_index=True,
    alt="Synthetic API telemetry events with masked key identifiers.",
)
Inspect the ranked incidents

The ranking now turns fixed conditions into an investigation queue. The final check reveals which patterns those conditions can prioritize.

Before you check, which masked key do you expect at the top? Do you expect every unusual event to rank highly?

  • Return to the ranked incident investigation table in the browser.
  • Locate the row for oai-prod-31b8.

You should see oai-prod-31b8 near the top. Its maximum error rate exceeds 10 percent. It also records rate-limit hits.

That result proves the fixed rules can elevate a key with known reliability and pressure signals.

  • Locate the row for oai-dev-44aa.
  • Compare its risk score with its late simulated cost value.

The late simulated cost spike for oai-dev-44aa remains difficult to prioritize because its error rate stays below the fixed threshold. It also has no rate-limit hits.

What did the static rules miss?

A fixed rule can recognize conditions that cross its chosen threshold. The oai-dev-44aa event looks unusual only when compared with that key's earlier behavior.

This intended shortfall creates the case for a local baseline. Each key needs its own recent history for spend and traffic comparisons.

Your dashboard now ranks masked keys with explained fixed-rule signals. Next up, you'll compare each key with its own rolling history to expose unusual spend and traffic.

Detect Baseline Anomalies and Investigate

Your fixed rules now expose keys with high error rates or rate-limit pressure. The ranked table makes those incidents easier to prioritize.

A static threshold still misses behavior that is unusual only for one key. In this step, pandas creates a rolling median for each masked key. Plotly turns the resulting spend and latency data into an interactive investigation.

In this step, get ready to:
  • Calculate per-key rolling baselines for simulated cost and request traffic.
  • Filter the dashboard by provider, masked key, or workload.
  • Investigate explained incidents with interactive charts.
Calculate each key's local baseline

A local baseline compares each event with the recent history of the same masked key. The four-event rolling window begins producing a median after three observations.

Why use a rolling median?

A median resists distortion from a single large spike. Grouping by key_id also preserves the normal scale of each key.

The ratio threshold of 2.5 is a tutorial rule. It demonstrates relative detection without claiming a provider-wide security limit.

  • In app.py, keep the existing import lines.
  • Replace the code between the imports and st.set_page_config with the next blocks in order:
REQUIRED_COLUMNS = {
    "timestamp",
    "provider",
    "key_id",
    "project",
    "workload",
    "model_tier",
    "requests",
    "tokens",
    "cost_usd",
    "errors",
    "avg_latency_ms",
    "rate_limit_hits",
}

What does the column contract do?

The REQUIRED_COLUMNS set defines the shared schema expected from every telemetry source. load_data uses it to identify missing fields before the dashboard performs any analysis.

  • Replace the existing explain_risk helper with this expanded version:
def explain_risk(cost_ratio, request_ratio, error_rate, rate_limit_hits):
    reasons = []
    if cost_ratio >= 2.5:
        reasons.append("Spend spike vs rolling median")
    if request_ratio >= 2.5:
        reasons.append("Traffic spike vs rolling median")
    if error_rate >= 10:
        reasons.append("High error rate")
    if rate_limit_hits > 0:
        reasons.append("Rate-limit pressure")
    return ", ".join(reasons) if reasons else "Within baseline"

How are incidents explained?

  • The cost ratio adds a spend-spike reason when the event reaches the tutorial threshold.
  • The request ratio adds a traffic-spike reason using the same relative threshold.
  • The existing error and rate-limit checks remain part of the explanation.
  • Replace the existing combine_reasons helper with this version:
def combine_reasons(values):
    active_reasons = sorted(
        {value for value in values if value != "Within baseline"}
    )
    return "; ".join(active_reasons) if active_reasons else "Within baseline"

Why combine the reasons?

Several events for one key can trigger the same rule. combine_reasons removes duplicates before the grouped incident table displays them.

  • Start the new load_data function below combine_reasons with this block:
def load_data(source):
    data = pd.read_csv(source)
    missing_columns = REQUIRED_COLUMNS - set(data.columns)
    if missing_columns:
        missing_text = ", ".join(sorted(missing_columns))
        raise ValueError(f"Missing required columns: {missing_text}")

    data["timestamp"] = pd.to_datetime(data["timestamp"], errors="raise")
    data = data.sort_values(["key_id", "timestamp"]).reset_index(drop=True)
    data["error_rate_pct"] = data["errors"] / data["requests"] * 100

    data["cost_baseline"] = data.groupby("key_id")["cost_usd"].transform(
        lambda values: values.rolling(window=4, min_periods=3).median()
    )
    data["request_baseline"] = data.groupby("key_id")["requests"].transform(
        lambda values: values.rolling(window=4, min_periods=3).median()
    )

How are the baselines calculated?

  • The schema check raises a readable error when a required column is absent.
  • Sorting by key_id and timestamp puts every key's events into chronological order.
  • The grouped transformations return one rolling cost median and one rolling request median for each event.
  • Continue load_data with the ratio and risk-score calculations:
    data["cost_ratio"] = data["cost_usd"] / data["cost_baseline"]
    data["request_ratio"] = data["requests"] / data["request_baseline"]

    data["risk_score"] = [
        int(cost_ratio >= 2.5) * 3
        + int(request_ratio >= 2.5) * 3
        + int(error_rate >= 10) * 2
        + int(rate_limit_hits > 0)
        for cost_ratio, request_ratio, error_rate, rate_limit_hits in zip(
            data["cost_ratio"],
            data["request_ratio"],
            data["error_rate_pct"],
            data["rate_limit_hits"],
        )
    ]

How does the score change?

Each ratio measures the current event against its key's rolling median. Spend and traffic anomalies receive three points each because this step focuses on detecting those local changes.

The fixed tutorial rules remain active. The combined score lets several warning signals increase an event's priority.

  • Finish load_data with the explained reason and return value:
    data["reason"] = [
        explain_risk(cost_ratio, request_ratio, error_rate, rate_limit_hits)
        for cost_ratio, request_ratio, error_rate, rate_limit_hits in zip(
            data["cost_ratio"],
            data["request_ratio"],
            data["error_rate_pct"],
            data["rate_limit_hits"],
        )
    ]
    return data

What completes the data pipeline?

Each event receives a readable reason based on the signals that fired. Returning the enriched dataframe makes one validated pipeline available to the rest of the app.

  • Replace the old inline CSV loading and event scoring below the safety caption with this function call:
data = load_data("data/api_usage.csv")

What does this line change?

The dashboard now loads its bundled telemetry through load_data. Every later metric receives validated timestamps, rolling ratios, scores, and explanations.

  • Save app.py.

Streamlit reruns the existing dashboard. You should see oai-dev-44aa gain the explanation Spend spike vs rolling median in the current incident table.

Dashboard stopped after the baseline edit?

  • Check that every continuation inside load_data retains four spaces of indentation.
  • Check that the old inline risk_score and reason calculations were removed.
  • Check that data = load_data("data/api_usage.csv") remains below the page caption.

Still stuck? Help me debug the rolling-baseline pipeline.

Filter the investigation

An investigation becomes more useful when every total and incident responds to the same scope. The filters progressively narrow providers, masked keys, and workloads before the dashboard recalculates its findings.

  • In app.py, replace the current totals and key_summary section with the next four blocks in order:
filter_columns = st.columns(3)
providers = sorted(set(data["provider"]))
with filter_columns[0]:
    selected_providers = st.multiselect(
        "Provider",
        providers,
        default=providers,
    )

filtered = data[
    [provider in selected_providers for provider in data["provider"]]
]
key_options = sorted(set(filtered["key_id"]))
with filter_columns[1]:
    selected_keys = st.multiselect(
        "Masked key ID",
        key_options,
        default=key_options,
    )

How does progressive filtering begin?

The provider selection creates the first filtered dataframe. The masked-key options then come from that reduced set, which keeps irrelevant keys out of the second filter.

filtered = filtered[
    [key_id in selected_keys for key_id in filtered["key_id"]]
]
workload_options = sorted(set(filtered["workload"]))
with filter_columns[2]:
    selected_workloads = st.multiselect(
        "Workload",
        workload_options,
        default=workload_options,
    )

filtered = filtered[
    [workload in selected_workloads for workload in filtered["workload"]]
]

if filtered.empty:
    st.error("No telemetry matches the selected filters.")
    st.stop()

Why stop on an empty result?

The workload selection completes the filtered dataset used by the dashboard. The empty check prevents later totals and charts from trying to summarize a selection with no matching events.

total_requests = int(sum(filtered["requests"]))
total_cost = float(sum(filtered["cost_usd"]))
total_errors = int(sum(filtered["errors"]))
error_rate = total_errors / total_requests * 100 if total_requests else 0
telemetry_rows = len(filtered)
risk_events = int(sum(filtered["risk_score"] > 0))

metric_columns = st.columns(5)
metric_columns[0].metric("Requests", f"{total_requests:,}")
metric_columns[1].metric("Simulated cost", f"${total_cost:,.2f}")
metric_columns[2].metric("Error rate", f"{error_rate:.1f}%")
metric_columns[3].metric("Telemetry rows", telemetry_rows)
metric_columns[4].metric("Risk events", risk_events)

What do the filtered metrics show?

All five metrics now summarize filtered instead of the full telemetry dataset. The new risk-event total counts event rows with a score above zero.

key_summary = (
    filtered.groupby(
        ["provider", "key_id", "project", "workload", "model_tier"],
        as_index=False,
    )
    .agg(
        total_requests=("requests", "sum"),
        total_cost_usd=("cost_usd", "sum"),
        avg_latency_ms=("avg_latency_ms", "mean"),
        max_error_rate_pct=("error_rate_pct", "max"),
        rate_limit_hits=("rate_limit_hits", "sum"),
        risk_score=("risk_score", "sum"),
        max_cost_ratio=("cost_ratio", "max"),
        max_request_ratio=("request_ratio", "max"),
        reasons=("reason", combine_reasons),
    )
    .sort_values(
        ["risk_score", "total_cost_usd"],
        ascending=[False, False],
    )
)

How does the ranking respond?

The named aggregation now runs against the filtered events. Maximum cost and request ratios preserve the strongest local anomaly seen for each grouped key.

Sorting by risk score keeps the most concerning keys first. Simulated cost breaks ties between keys with the same score.

  • Save app.py.
  • Select one value in the Provider filter.

You should see the request total, simulated cost, error rate, telemetry rows, and risk-event count update for that provider. The incident ranking should also remove keys from the other providers.

Filters are not changing the totals?

  • Check that each metric reads from filtered.
  • Check that key_summary starts with filtered.groupby.
  • Check that the empty-selection condition appears before the metric calculations.

Need another pair of eyes? Help me trace the filter flow.

Chart and verify the incidents

The ranked table identifies priority. The charts add time and latency context so you can investigate when simulated spend changed and how each key compares.

  • Replace the existing table-rendering section after key_summary with the next four blocks in order:
hourly_cost = filtered.groupby(
    ["timestamp", "provider"],
    as_index=False,
).agg(cost_usd=("cost_usd", "sum"))

cost_chart = px.line(
    hourly_cost,
    x="timestamp",
    y="cost_usd",
    color="provider",
    markers=True,
    title="Hourly simulated spend",
)
st.plotly_chart(
    cost_chart,
    width="stretch",
    alt="Hourly simulated spend by provider.",
)

What does the spend chart reveal?

The hourly aggregation adds simulated cost within each provider and timestamp. The line chart makes sudden spend changes visible while preserving provider-level color groups.

latency_chart = px.scatter(
    key_summary,
    x="avg_latency_ms",
    y="total_cost_usd",
    color="provider",
    size="total_requests",
    hover_name="key_id",
    hover_data=["risk_score", "max_error_rate_pct", "rate_limit_hits"],
    title="Latency versus simulated spend by masked key",
)
st.plotly_chart(
    latency_chart,
    width="stretch",
    alt="Average latency versus simulated spend for each masked key.",
)

What does the scatter chart reveal?

Each point represents one grouped masked key. Its position compares average latency with total simulated cost.

Point size represents request volume. Hover details connect the visual pattern with its risk score, maximum error rate, and rate-limit pressure.

st.caption("Ranked incident investigation table")
st.dataframe(
    key_summary[
        [
            "provider",
            "key_id",
            "project",
            "workload",
            "model_tier",
            "total_requests",
            "total_cost_usd",
            "avg_latency_ms",
            "max_error_rate_pct",
            "rate_limit_hits",
            "max_cost_ratio",
            "max_request_ratio",
            "risk_score",
            "reasons",
        ]
    ],
    width="stretch",
    hide_index=True,
    alt="Masked API key incident ranking.",
)

What completes the incident view?

The incident table shows the strongest cost and request ratios alongside the fixed-rule signals. The reasons column turns those numbers into an investigation summary.

st.caption("Filtered raw telemetry")
st.dataframe(
    filtered[
        [
            "timestamp",
            "provider",
            "key_id",
            "project",
            "workload",
            "model_tier",
            "requests",
            "tokens",
            "cost_usd",
            "errors",
            "avg_latency_ms",
            "rate_limit_hits",
            "risk_score",
            "reason",
        ]
    ],
    width="stretch",
    hide_index=True,
    alt="Filtered synthetic API telemetry events.",
)

Why keep the raw events?

The filtered raw table lets you inspect the event rows behind a ranked incident. It keeps the score and reason beside the operational values that triggered them.

✔️ Awesome, I've got everything!

Save app.py before running the final investigation.

ⓧ I'd like to double check the full code

import pandas as pd
import plotly.express as px
import streamlit as st

REQUIRED_COLUMNS = {
    "timestamp",
    "provider",
    "key_id",
    "project",
    "workload",
    "model_tier",
    "requests",
    "tokens",
    "cost_usd",
    "errors",
    "avg_latency_ms",
    "rate_limit_hits",
}


def explain_risk(cost_ratio, request_ratio, error_rate, rate_limit_hits):
    reasons = []
    if cost_ratio >= 2.5:
        reasons.append("Spend spike vs rolling median")
    if request_ratio >= 2.5:
        reasons.append("Traffic spike vs rolling median")
    if error_rate >= 10:
        reasons.append("High error rate")
    if rate_limit_hits > 0:
        reasons.append("Rate-limit pressure")
    return ", ".join(reasons) if reasons else "Within baseline"


def combine_reasons(values):
    active_reasons = sorted(
        {value for value in values if value != "Within baseline"}
    )
    return "; ".join(active_reasons) if active_reasons else "Within baseline"


def load_data(source):
    data = pd.read_csv(source)
    missing_columns = REQUIRED_COLUMNS - set(data.columns)
    if missing_columns:
        missing_text = ", ".join(sorted(missing_columns))
        raise ValueError(f"Missing required columns: {missing_text}")

    data["timestamp"] = pd.to_datetime(data["timestamp"], errors="raise")
    data = data.sort_values(["key_id", "timestamp"]).reset_index(drop=True)
    data["error_rate_pct"] = data["errors"] / data["requests"] * 100

    data["cost_baseline"] = data.groupby("key_id")["cost_usd"].transform(
        lambda values: values.rolling(window=4, min_periods=3).median()
    )
    data["request_baseline"] = data.groupby("key_id")["requests"].transform(
        lambda values: values.rolling(window=4, min_periods=3).median()
    )
    data["cost_ratio"] = data["cost_usd"] / data["cost_baseline"]
    data["request_ratio"] = data["requests"] / data["request_baseline"]

    data["risk_score"] = [
        int(cost_ratio >= 2.5) * 3
        + int(request_ratio >= 2.5) * 3
        + int(error_rate >= 10) * 2
        + int(rate_limit_hits > 0)
        for cost_ratio, request_ratio, error_rate, rate_limit_hits in zip(
            data["cost_ratio"],
            data["request_ratio"],
            data["error_rate_pct"],
            data["rate_limit_hits"],
        )
    ]
    data["reason"] = [
        explain_risk(cost_ratio, request_ratio, error_rate, rate_limit_hits)
        for cost_ratio, request_ratio, error_rate, rate_limit_hits in zip(
            data["cost_ratio"],
            data["request_ratio"],
            data["error_rate_pct"],
            data["rate_limit_hits"],
        )
    ]
    return data


st.set_page_config(
    page_title="AI API Key Risk and Spend Observatory",
    layout="wide",
)
st.title("AI API Key Risk and Spend Observatory")
st.caption(
    "Synthetic telemetry only. Costs and detection thresholds are tutorial values, "
    "not vendor prices, quotas, or security guarantees."
)

data = load_data("data/api_usage.csv")

filter_columns = st.columns(3)
providers = sorted(set(data["provider"]))
with filter_columns[0]:
    selected_providers = st.multiselect(
        "Provider",
        providers,
        default=providers,
    )

filtered = data[
    [provider in selected_providers for provider in data["provider"]]
]
key_options = sorted(set(filtered["key_id"]))
with filter_columns[1]:
    selected_keys = st.multiselect(
        "Masked key ID",
        key_options,
        default=key_options,
    )

filtered = filtered[
    [key_id in selected_keys for key_id in filtered["key_id"]]
]
workload_options = sorted(set(filtered["workload"]))
with filter_columns[2]:
    selected_workloads = st.multiselect(
        "Workload",
        workload_options,
        default=workload_options,
    )

filtered = filtered[
    [workload in selected_workloads for workload in filtered["workload"]]
]

if filtered.empty:
    st.error("No telemetry matches the selected filters.")
    st.stop()

total_requests = int(sum(filtered["requests"]))
total_cost = float(sum(filtered["cost_usd"]))
total_errors = int(sum(filtered["errors"]))
error_rate = total_errors / total_requests * 100 if total_requests else 0
telemetry_rows = len(filtered)
risk_events = int(sum(filtered["risk_score"] > 0))

metric_columns = st.columns(5)
metric_columns[0].metric("Requests", f"{total_requests:,}")
metric_columns[1].metric("Simulated cost", f"${total_cost:,.2f}")
metric_columns[2].metric("Error rate", f"{error_rate:.1f}%")
metric_columns[3].metric("Telemetry rows", telemetry_rows)
metric_columns[4].metric("Risk events", risk_events)

key_summary = (
    filtered.groupby(
        ["provider", "key_id", "project", "workload", "model_tier"],
        as_index=False,
    )
    .agg(
        total_requests=("requests", "sum"),
        total_cost_usd=("cost_usd", "sum"),
        avg_latency_ms=("avg_latency_ms", "mean"),
        max_error_rate_pct=("error_rate_pct", "max"),
        rate_limit_hits=("rate_limit_hits", "sum"),
        risk_score=("risk_score", "sum"),
        max_cost_ratio=("cost_ratio", "max"),
        max_request_ratio=("request_ratio", "max"),
        reasons=("reason", combine_reasons),
    )
    .sort_values(
        ["risk_score", "total_cost_usd"],
        ascending=[False, False],
    )
)

hourly_cost = filtered.groupby(
    ["timestamp", "provider"],
    as_index=False,
).agg(cost_usd=("cost_usd", "sum"))

cost_chart = px.line(
    hourly_cost,
    x="timestamp",
    y="cost_usd",
    color="provider",
    markers=True,
    title="Hourly simulated spend",
)
st.plotly_chart(
    cost_chart,
    width="stretch",
    alt="Hourly simulated spend by provider.",
)

latency_chart = px.scatter(
    key_summary,
    x="avg_latency_ms",
    y="total_cost_usd",
    color="provider",
    size="total_requests",
    hover_name="key_id",
    hover_data=["risk_score", "max_error_rate_pct", "rate_limit_hits"],
    title="Latency versus simulated spend by masked key",
)
st.plotly_chart(
    latency_chart,
    width="stretch",
    alt="Average latency versus simulated spend for each masked key.",
)

st.caption("Ranked incident investigation table")
st.dataframe(
    key_summary[
        [
            "provider",
            "key_id",
            "project",
            "workload",
            "model_tier",
            "total_requests",
            "total_cost_usd",
            "avg_latency_ms",
            "max_error_rate_pct",
            "rate_limit_hits",
            "max_cost_ratio",
            "max_request_ratio",
            "risk_score",
            "reasons",
        ]
    ],
    width="stretch",
    hide_index=True,
    alt="Masked API key incident ranking.",
)

st.caption("Filtered raw telemetry")
st.dataframe(
    filtered[
        [
            "timestamp",
            "provider",
            "key_id",
            "project",
            "workload",
            "model_tier",
            "requests",
            "tokens",
            "cost_usd",
            "errors",
            "avg_latency_ms",
            "rate_limit_hits",
            "risk_score",
            "reason",
        ]
    ],
    width="stretch",
    hide_index=True,
    alt="Filtered synthetic API telemetry events.",
)

How to use this reference

Compare the placement of each helper, filter, metric, chart, and table with your saved file. Your app.py should match this reference exactly.

  • Save app.py.

Before you rerun the app, predict which anomaly explanation each masked key will receive.

  • Start the completed dashboard in the activated PowerShell environment by running this command:
streamlit run app.py

What does this command do?

This command starts the specified Streamlit app. Your browser should open the complete local observability dashboard.

You should see five metrics, three filters, two Plotly charts, the explained incident ranking, and the filtered raw telemetry table.

  • Remove the existing selections from the Masked key ID filter.
  • Select oai-dev-44aa in the Masked key ID filter.

The incident explanation should include Spend spike vs rolling median.

  • Replace the masked-key selection with gai-prod-872e.

The incident explanation should include Spend spike vs rolling median and Traffic spike vs rolling median.

You have the full investigation loop working. Each masked key now carries an explanation grounded in its own recent history.

Missing a chart or anomaly explanation?

  • Check that both charts receive filtered data or the filtered key_summary.
  • Check that the cost and request baselines group rows by key_id.
  • Check that both ratio conditions use the tutorial value 2.5.

Still investigating? Help me find the missing anomaly signal.

Secret mission

Accept a Replacement Telemetry CSV

Your dashboard currently analyzes one bundled telemetry file. Extend it with a CSV uploader that accepts replacement data, validates the shared schema, and shows a readable error when required columns are missing.

Clean Up Your Resources

Clean Up Your Resources

Your Streamlit observability app runs locally with synthetic telemetry, so its only resources are files and a local process. It creates no ongoing costs, so choose whether to keep it active, pause it for later, or delete the files.

Resources you used:

  • Your local Streamlit process running app.py in the activated PowerShell virtual environment.
  • Your ai-api-risk-observatory folder, including app.py, requirements.txt, data/api_usage.csv, and the .venv environment.

Keep everything running

No action needed. Choose this if you want to keep demonstrating the dashboard or testing replacement telemetry files.

  • Leave the PowerShell terminal running while you continue using the dashboard.
  • Keep the ai-api-risk-observatory folder so the app, synthetic dataset, and virtual environment remain together.

Pause - I'll come back to this later

Stop the local process when you finish working while preserving every project file for another session.

  • Close the PowerShell terminal that is running app.py.
  • Keep the ai-api-risk-observatory folder in its current location.

The local dashboard server stops when the terminal closes. Your project files remain ready for another session.

Delete - I don't want to use this again

This option removes the running process and the complete project folder. No cloud cleanup is required.

  • Close the PowerShell terminal that is running app.py.
  • Use the Windows file manager to locate the ai-api-risk-observatory folder you created in Step 1.
  • Delete the ai-api-risk-observatory folder through the file manager's context menu.

This removes app.py, requirements.txt, data/api_usage.csv, and .venv. The local dashboard cannot run after the folder is deleted.

Nice Work!

Nice Work!

You made it! Your local Streamlit observability dashboard now turns sanitized provider-neutral telemetry into ranked masked-key incidents. Real credentials stay outside the app.

You've learned how to:

  • Built a local Streamlit dashboard for investigating synthetic AI API telemetry. Added Plotly charts that connect latency with simulated spend.
  • Used pandas aggregation to turn raw events into a ranked masked-key incident table. Fixed tutorial rules expose high error rates plus rate-limit pressure.
  • Applied rolling-median baselines to each key's history. Explained incidents reveal key-specific spend spikes. They also reveal traffic spikes.
  • Completed the Secret Mission by adding validated CSV replacement uploads. Invalid schemas now produce readable missing-column errors.

Ready to quiz yourself?