Build a Real-Time Scam Detection API
Deploy a phishing detection SaaS on GCP that scores URLs against live threat feeds.
Introduction
30 Second Summary
Everyone has received a suspicious text or email claiming to be from their bank, a delivery service, or a government agency. These scams cost Americans $15.9 billion in 2025 alone, with losses rising 430% since 2020.
In this project, you will build PhishGuard, a real-time scam detection API that scores suspect messages and URLs against live threat intelligence feeds and returns instant verdicts to protect users before they click.
What You'll Build
PhishGuard is a serverless detection service deployed on Google Cloud Functions that checks suspect URLs and messages against 3.8M+ tracked malicious indicators from OpenPhish and URLhaus, then scores them using a heuristic classifier behind a swappable model gateway.
By the end of this project, you'll have:
- A deployed detection API on GCP free tier that returns structured verdicts (MALICIOUS / SUSPICIOUS / CLEAN) with confidence scores, signal explanations, and recommended actions.
- A complete pre-artifact set including an Architecture Decision Record, C4 diagrams, OpenAPI contract, and implementation plan produced before writing any service code.
- A validated system with precision/recall evaluation, latency report, and a stakeholder presentation projecting the path from zero to 1M users at $1/user.
- Secret Mission: Build a takedown-case routing layer that generates structured abuse reports for confirmed malicious infrastructure and gates on human review.
Are there any prerequisites?
You will need a GCP account with billing enabled (everything in this project stays within the free tier) and a Cursor subscription. All other tools, including Python, Terraform, and the gcloud CLI, will be installed in Step 1.
Set Up Your Environment
Before you can build a real-time detection engine, you need the right tools on your machine. Python runs the detection service. Terraform deploys your infrastructure to GCP. The gcloud CLI connects your local machine to Google Cloud.
In this step, you will install all three tools, register for free threat intelligence API keys, and create the directory structure for your PhishGuard project.
In this step, get ready to:
- Install Python 3.11+, Terraform, and the gcloud CLI on Windows.
- Enable the Google Safe Browsing API and register for a free URLhaus API key.
- Create the project directory structure and configure your environment.
Install your development tools
You need three tools to build and deploy PhishGuard. Let's verify whether each one is already installed, and install any that are missing.
- Press the Windows key to open your search bar.
- Type Cursor and press Enter to open it.
- Open Cursor's integrated terminal by pressing Ctrl+` (backtick key, usually above Tab).
What is Cursor's integrated terminal?
The integrated terminal is a command-line panel built into Cursor. It lets you run commands without leaving your editor. You will use it for all terminal commands in this project.
First, check whether Python 3.11 or higher is installed.
- Check your Python version by running this command:
python --version✔️ I see version 3.11 or higher
Python is ready. Move on to the Terraform check below.
ⓧ I see a version lower than 3.11
You have Python installed, but you need version 3.11 or higher for this project.
- Open python.org/downloads in your browser.
- Download the latest Python 3 Windows installer (64-bit).
- Run the installer. On the first screen, check the box labeled Add Python to PATH before clicking Install Now.
- Close and reopen your terminal in Cursor (Ctrl+` twice), then run python --version again to confirm.
ⓧ Command not found
Python is not installed yet.
- Open python.org/downloads in your browser.
- Download the latest Python 3 Windows installer (64-bit).
- Run the installer. On the first screen, check the box labeled Add Python to PATH before clicking Install Now.
- Close and reopen your terminal in Cursor (Ctrl+` twice), then run python --version again to confirm it prints 3.11 or higher.
Next, check whether Terraform is installed.
- Check your Terraform version by running:
terraform version✔️ I see version 1.0 or higher
Terraform is ready. Move on to the gcloud CLI check below.
ⓧ I see an older version
You need Terraform 1.0 or higher. Download the latest version to replace your current one.
- Open developer.hashicorp.com/terraform/install in your browser.
- Under the Windows section, download the AMD64 binary zip.
- Extract terraform.exe from the zip and replace the old file in the same folder on your PATH.
- Close and reopen your terminal, then run terraform version to confirm the version is 1.0 or higher.
ⓧ Command not found
Terraform is not installed yet.
- Open developer.hashicorp.com/terraform/install in your browser.
- Under the Windows section, download the AMD64 binary zip.
- Create a folder called C:\tools if it does not already exist.
- Extract terraform.exe from the zip into C:\tools\.
- Add C:\tools\ to your system PATH:
- Close and reopen your terminal in Cursor, then run terraform version to confirm.
Finally, check whether the gcloud CLI is installed.
- Check for gcloud by running:
gcloud version✔️ I see a version number
The gcloud CLI is ready. Continue to the authentication step below.
ⓧ Command not found
The gcloud CLI is not installed yet.
- Open cloud.google.com/sdk/docs/install in your browser.
- Download the Google Cloud CLI installer for Windows.
- Run the installer and accept the defaults.
- When the installer finishes, it will ask to run gcloud init. Close that prompt for now.
- Close and reopen your terminal in Cursor, then run gcloud version to confirm it prints a version number.
Now authenticate gcloud and enable the GCP APIs your project needs.
- Sign in to Google Cloud by running:
gcloud auth login- A browser window opens asking you to sign in with your Google account. Sign in and click Allow.
- Set your active GCP project by running:
gcloud config set project [[GCP_PROJECT_ID="your-gcp-project-id"]]- Enable the four APIs required for Cloud Functions deployment by running:
gcloud services enable cloudfunctions.googleapis.com cloudbuild.googleapis.com run.googleapis.com cloudscheduler.googleapis.com- Verify the APIs are enabled by running:
gcloud services list --enabled --filter="NAME:cloudfunctions OR NAME:cloudbuild OR NAME:run OR NAME:cloudscheduler"You should see all four services listed in the output.
Seeing a permissions error?
Make sure you are signed in to the Google account that owns the GCP project with billing enabled. The account needs the Editor or Owner role on the project.
If the issue persists, help me debug my gcloud authentication.
Register for threat feed API keys
PhishGuard enriches suspect URLs against two live threat feeds. You need a free API key from Google Safe Browsing (for phishing detection) and one from URLhaus (for malware distribution).
- Enable the Safe Browsing API on your GCP project by running:
gcloud services enable safebrowsing.googleapis.com- Open console.cloud.google.com in your browser.
- Select APIs & Services from the left navigation menu.
- Click Credentials.
- Click Create Credentials at the top, then select API key.
- Copy your new API key and keep it somewhere safe for the next substep.
What is Google Safe Browsing?
Google Safe Browsing maintains constantly updated lists of unsafe web resources, including phishing (social engineering) sites and malware distributors. It powers the warnings you see in Chrome, Firefox, and Safari when visiting a dangerous page.
- Open urlhaus.abuse.ch in your browser.
- Click API in the navigation bar to view the API documentation.
- Register for a free account at the abuse.ch portal.
- Obtain your Auth-Key from your account settings.
- Copy your Auth-Key and keep it safe.
What is URLhaus?
URLhaus is operated by abuse.ch in partnership with Spamhaus. It tracks over 3.8 million malicious URLs and updates its CSV/JSON dumps every 5 minutes. Together with Google Safe Browsing, it gives PhishGuard coverage of both phishing (credential theft) and malware distribution (payload delivery).
Initialize your project repository
With your tools installed and API keys in hand, it is time to create the project structure. PhishGuard uses four directories to separate concerns: infrastructure, source code, evaluation, and documentation.
- Move to your Desktop by running this command:
cd ~\Desktop- Create your project folder and move into it by running these commands:
mkdir phishguard
cd phishguard- Create the four subdirectories by running:
mkdir infra, src, eval, docs- Confirm the directories exist by running:
lsYou should see infra, src, eval, and docs listed in the output.
- Create the .env file to store your API keys by running:
New-Item .env- Open the phishguard folder in Cursor. Click File in the top menu bar.
- Click Open Folder.
- Navigate to Desktop > phishguard and click Select Folder.
- In the Explorer sidebar, click on .env to open it.
- Paste the following content into .env, replacing the placeholder values with your actual keys and project ID:
GOOGLE_SAFEBROWSING_API_KEY=your-safe-browsing-api-key-here
URLHAUS_AUTH_KEY=your-urlhaus-auth-key-here
GCP_PROJECT_ID=your-gcp-project-id-hereWhat does this file do?
The .env file stores environment-specific configuration that should never be committed to version control. Your detection service reads these values at runtime to authenticate with Google Safe Browsing, URLhaus, and GCP.
- Create the .gitignore file to protect your secrets. In the Explorer sidebar, right-click in the file list and select New File.
- Name it .gitignore and press Enter.
- Paste the following content into .gitignore:
.env
__pycache__/
*.pyc
.terraform/
terraform.tfstate
terraform.tfstate.backup
.terraform.lock.hclWhat does this file do?
The .gitignore tells Git to never track these files. It protects your API keys in .env, avoids committing Python bytecode, and keeps Terraform's local state files out of your repository.
- Save both files (Ctrl+S).
✔️ Awesome, I've got everything!
Great. Double check you have saved both .env and .gitignore.
ⓧ I'd like to double check the full code
Here is the complete content of each file for comparison.
.env:
GOOGLE_SAFEBROWSING_API_KEY=<your_safe_browsing_api_key>
URLHAUS_AUTH_KEY=<your_urlhaus_auth_key>
GCP_PROJECT_ID=<your_project_id>.gitignore:
.env
__pycache__/
*.pyc
.terraform/
terraform.tfstate
terraform.tfstate.backup
.terraform.lock.hclFinally, set up your Cursor Composer workspace for parallel agent execution.
- Open Cursor Composer by pressing Ctrl+I.
- Open three separate Composer context windows. Label them in your mind as:
What is Gentik Systems multi-agent parallelization?
Instead of writing all code sequentially, you architect the solution, then dispatch three Cursor Composer agents to execute in parallel. You review and unblock. The agents execute. This is how you hit a 16-minute build target for a system that would take hours sequentially.
🙋♀️ Don't see Composer in Cursor?
Make sure your Cursor subscription is active and the app is up to date. Press Ctrl+I to toggle the Composer panel.
If you are still having trouble, help me open Cursor Composer.
Your environment is fully configured. Next up, you will write the pre-artifacts that define what PhishGuard will be before writing a single line of service code.
Create Pre-Artifacts
Your environment is ready. Before writing a single line of service code, you need to design the system.
Professional engineering teams document decisions before implementation. This prevents rework, aligns parallel contributors, and creates an audit trail. In this step, you will produce the full design artifact set that your three Cursor Composer agents will consume in Step 3.
In this step, get ready to:
- Write an Architecture Decision Record documenting cloud choice, detection pattern, and pricing model.
- Create C4 architecture diagrams in Mermaid covering Context, Container, and Component levels.
- Define the OpenAPI contract, data model, SLI/SLO targets, and implementation plan.
Write the Architecture Decision Record
An ADR captures the reasoning behind key technical choices. When you or your agents revisit the project in six months, the ADR explains why you chose GCP over AWS, heuristics over an LLM, and a 1/user pricing model.
- Create a new file called docs/adr.md in Cursor by pasting the following content:
# Architecture Decision Record: PhishGuard Detection Engine
## Status
Accepted
## Context
### Global Impact
- $15.9 billion in reported fraud losses in 2025 (FTC, highest on record)
- 3 million consumer fraud reports submitted (FTC, 2025)
- 3.8 million phishing attacks observed globally (APWG, 2025)
- Losses have risen 430% since 2020
- Older adults (60+) lose $2.4 billion per year in reported fraud
### Problem Statement
We need a real-time detection system that scores suspect messages and URLs
against live threat intelligence feeds, returning structured verdicts at
sub-second latency with zero infrastructure cost at MVP scale.
## Decisions
### 1. Cloud Platform: GCP Cloud Functions (2nd gen)
**Chosen over**: AWS Lambda, Azure Functions
**Rationale**:
- 2M free invocations/month (vs 1M on AWS Lambda)
- HTTP triggers included at no extra cost (AWS API Gateway adds $1-3.50/M requests)
- Lower cold starts (~200ms for Python)
- Up to 1000 concurrent requests per instance
- 60-minute HTTP timeout ceiling
### 2. Detection Pattern: Fast-Path + Heuristic Scorer + Model Gateway
**Pattern**: Known-bad indicators resolve instantly from cached feed data
(fast path). Unknown indicators pass through a heuristic scorer behind a
model gateway interface.
**Rationale**:
- Fast-path eliminates latency and compute cost for known threats
- Model gateway interface allows swapping the scorer without changing
ingestion or verdict layers
- Heuristic scorer has zero per-request cost (no LLM API dependency)
### 3. Threat Feeds: OpenPhish + URLhaus
**OpenPhish**: AI-based phishing detection, 12-hour feed updates, plain text
format, no registration required (https://openphish.com/feed.txt)
**URLhaus** (abuse.ch/Spamhaus): Malware URL-focused, 5-minute dump updates,
3.8M+ URLs tracked
**Rationale**: Together they cover both phishing (credential theft) and malware
distribution (payload delivery). Downloadable feeds enable local fast-path
lookups without API rate limits or registration barriers.
### 4. Human-in-the-Loop Gate
**Decision**: All automated actions (takedown requests, blocking) require human
approval before execution.
**Rationale**: False positives blocking legitimate communications cause real
harm. Precision must be provably high before any automated enforcement.
### 5. SaaS Pricing: 1/user/month
**Decision**: Accessible pricing at 1 currency unit per user per month.
**Rationale**: The mission is protection, not profit maximization. The operator
curates and maintains threat datasets (the recurring value). Dataset-as-product
model: more users = better dataset = better detection = more users.
## Consequences
- Zero infrastructure cost at MVP scale (GCP free tier)
- Sub-second detection for known indicators via fast path
- Graceful degradation: if feeds are unavailable, heuristic scorer still functions
- No LLM dependency at runtime (cost-free, latency-predictable)
- Human gate prevents false-positive harm but adds latency to enforcement actionsWhat does this document capture?
Each numbered decision records the choice, what it was chosen over, and the rationale. This is the standard ADR format used by engineering teams at companies like Google and Spotify.
The global impact framing grounds the project in real-world urgency. The $15.9B figure comes directly from FTC Congressional testimony (March 2026). The 3.8M phishing attacks figure comes from APWG's Q4 2025 Trends Report.
- Save docs/adr.md.
- Confirm the file appears in the docs/ folder in Cursor's Explorer sidebar.
File not appearing in docs/?
Make sure you created the file inside the docs directory, not at the project root. The path should be docs/adr.md, not just adr.md.
help me create the ADR file in the correct directory
Create C4 architecture diagrams
The C4 model visualizes software architecture at different zoom levels. Context shows the system boundary and external actors. Container shows major runtime components. Component shows internals of a single container.
- Create a new file called docs/c4_diagrams.md and paste the following Mermaid diagrams:
# C4 Architecture Diagrams: PhishGuard
## Level 1: System Context
```mermaid
C4Context
title System Context Diagram for PhishGuard
Person(endUser, "End User", "Submits suspect messages/URLs for analysis")
Person(guardian, "Guardian", "Reviews flagged cases and approves actions")
System(phishguard, "PhishGuard Detection System", "Scores suspect inputs against threat feeds and heuristics")
System_Ext(msgSource, "Suspect Message Source", "Email/SMS/Chat delivering suspect content")
System_Ext(openphish, "OpenPhish Feed", "AI-based phishing URL feed")
System_Ext(urlhaus, "URLhaus Feed", "Malware URL database by abuse.ch")
System_Ext(hostingProviders, "Hosting Providers", "Receive abuse reports for takedown")
Rel(endUser, phishguard, "Submits suspect message/URL")
Rel(guardian, phishguard, "Reviews flagged cases")
Rel(msgSource, phishguard, "Delivers suspect content")
Rel(phishguard, openphish, "Queries feed data")
Rel(phishguard, urlhaus, "Queries feed data")
Rel(phishguard, hostingProviders, "Sends abuse reports")
Rel(phishguard, endUser, "Returns verdict")
```
## Level 2: Container
```mermaid
C4Container
title Container Diagram for PhishGuard
Person(endUser, "End User", "")
System_Boundary(pg, "PhishGuard System") {
Container(cf, "Detection API", "Python, Cloud Functions 2nd gen", "HTTP endpoint accepting detection requests")
ContainerDb(feedCache, "Threat Feed Cache", "Cloud Storage", "Stores downloaded OpenPhish and URLhaus feeds")
Container(gateway, "Model Gateway", "Python", "Swappable scorer interface")
ContainerDb(eventStore, "Event Store", "Cloud Storage JSON", "Logs detection events for metering")
}
System_Ext(openphish, "OpenPhish", "")
System_Ext(urlhaus, "URLhaus", "")
Rel(endUser, cf, "POST /detect", "HTTPS")
Rel(cf, feedCache, "Fast-path lookup")
Rel(cf, gateway, "Scores unknown indicators")
Rel(cf, eventStore, "Logs detection events")
Rel(feedCache, openphish, "Refreshed every 12 hours")
Rel(feedCache, urlhaus, "Refreshed every 5 min")
```
## Level 3: Component
```mermaid
C4Component
title Component Diagram for Detection API
Container(feedCache, "Threat Feed Cache", "", "")
Container_Boundary(cf, "Detection API (Cloud Function)") {
Component(ingestion, "Ingestion Handler", "Python", "Parses and validates incoming requests")
Component(enricher, "Feed Enricher", "Python", "Looks up indicators in cached feeds")
Component(scorer, "Heuristic Scorer", "Python", "Extracts features and computes weighted score")
Component(formatter, "Verdict Formatter", "Python", "Structures response per OpenAPI contract")
Component(fastpath, "Fast-Path Cache", "Python dict", "In-memory lookup for known-bad indicators")
}
Rel(ingestion, enricher, "Passes parsed input")
Rel(enricher, fastpath, "Cache lookup")
Rel(enricher, formatter, "Known-bad hit: instant verdict")
Rel(enricher, scorer, "Unknown: score with heuristics")
Rel(scorer, formatter, "Returns confidence + signals")
Rel(fastpath, feedCache, "Loaded from feed cache")
```
## SaaS Tenant Flow
```mermaid
sequenceDiagram
participant User
participant API as PhishGuard API
participant FP as Fast-Path Cache
participant Scorer as Heuristic Scorer
participant Meter as Usage Meter
User->>API: POST /detect (API key + payload)
API->>API: Validate API key, identify tenant
API->>FP: Lookup URL/domain in feed cache
alt Known-bad indicator
FP-->>API: Cache hit (MALICIOUS)
else Unknown indicator
FP-->>API: Cache miss
API->>Scorer: Score with heuristics
Scorer-->>API: Confidence + signals
end
API->>Meter: Log detection event (tenant, timestamp)
API-->>User: Verdict response
```Why four diagrams?
The Context diagram shows stakeholders what the system connects to. The Container diagram shows developers what runtime components exist. The Component diagram shows engineers how the Detection API is internally structured.
The sequence diagram is not a standard C4 level. It shows the runtime flow for a single detection request, including the fast-path optimization and usage metering that make this a SaaS-ready service.
- Save docs/c4_diagrams.md.
- To verify the Mermaid syntax renders correctly, paste one of the fenced code blocks into the Mermaid Live Editor at mermaid.live. You should see the diagram render without errors.
Mermaid diagram not rendering?
The most common issue is indentation. Mermaid C4 diagrams use spaces (not tabs) for indentation. Make sure there are no tab characters in your file.
Also check that each diagram block starts with the correct keyword: C4Context, C4Container, or C4Component. These are case-sensitive.
help me fix my Mermaid C4 diagram syntax
Define the OpenAPI contract, data model, and implementation plan
The remaining pre-artifacts define what the API accepts and returns, what data shapes flow through the system, what performance targets must be met, and how the three agents will divide the work.
- Create docs/openapi.yaml with the OpenAPI 3.1 contract for the /detect endpoint:
openapi: "3.1.0"
info:
title: PhishGuard Detection API
version: "1.0.0"
description: Real-time phishing and scam detection service
paths:
/detect:
post:
summary: Analyze a suspect message or URL for phishing indicators
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/DetectionRequest"
responses:
"200":
description: Detection verdict returned
content:
application/json:
schema:
$ref: "#/components/schemas/DetectionResponse"
components:
schemas:
DetectionRequest:
type: object
properties:
message:
type: string
description: Suspect message text to analyze
url:
type: string
description: Suspect URL to check against threat feeds
DetectionResponse:
type: object
properties:
verdict:
$ref: "#/components/schemas/Verdict"
confidence:
type: number
minimum: 0.0
maximum: 1.0
description: Confidence score for the verdict
signals:
type: array
items:
type: string
description: List of signals that contributed to the verdict
recommended_action:
type: string
enum: [block, warn, allow]
description: Recommended action based on the verdict
feed_hits:
type: array
items:
$ref: "#/components/schemas/FeedHit"
description: Matching entries from threat intelligence feeds
Verdict:
type: string
enum: [MALICIOUS, SUSPICIOUS, CLEAN]
description: Detection verdict for the analyzed input
FeedHit:
type: object
properties:
feed_name:
type: string
description: Name of the threat feed (openphish, urlhaus)
indicator:
type: string
description: The matched URL or domain
threat_type:
type: string
description: Type of threat (phishing, malware_download)
last_seen:
type: string
format: date-time
description: When the indicator was last observed
DetectionEvent:
type: object
properties:
event_id:
type: string
format: uuid
timestamp:
type: string
format: date-time
input_url:
type: string
input_message:
type: string
verdict:
$ref: "#/components/schemas/Verdict"
confidence:
type: number
signals:
type: array
items:
type: string
feed_hits:
type: array
items:
$ref: "#/components/schemas/FeedHit"
processing_time_ms:
type: number
tenant_id:
type: stringWhat does this contract define?
The contract specifies a single POST /detect endpoint. The request accepts a message field (text to analyze) and a url field (URL to check against feeds).
The response returns a verdict (MALICIOUS, SUSPICIOUS, or CLEAN), a confidence score, contributing signals[], a recommended_action, and any feed_hits[] from threat intelligence.
- Save docs/openapi.yaml.
- Now create docs/data_model.md with the data model definitions:
# Data Model: PhishGuard
## Core Schemas
### DetectionEvent
Represents a single detection request and its result. Each event is a
countable unit for usage metering.
| Field | Type | Description |
|-------|------|-------------|
| event_id | UUID | Unique identifier for this detection event |
| timestamp | datetime | When the detection was processed |
| input_url | string | URL submitted for analysis (optional) |
| input_message | string | Message text submitted for analysis (optional) |
| verdict | Verdict | MALICIOUS, SUSPICIOUS, or CLEAN |
| confidence | float (0-1) | Confidence score for the verdict |
| signals | string[] | Contributing signals |
| feed_hits | FeedHit[] | Matching threat feed entries |
| processing_time_ms | float | End-to-end processing time in milliseconds |
| tenant_id | string | API key owner (for multi-tenant metering) |
### FeedHit
Represents a match against a threat intelligence feed.
| Field | Type | Description |
|-------|------|-------------|
| feed_name | string | Source feed identifier (openphish, urlhaus) |
| indicator | string | The matched URL or domain |
| threat_type | string | Classification (phishing, malware_download) |
| last_seen | datetime | When the indicator was last observed in the feed |
### Verdict
Enumeration of possible detection outcomes.
| Value | Meaning | Trigger |
|-------|---------|----------|
| MALICIOUS | Confirmed threat | Feed hit with verified status OR high-confidence heuristic score |
| SUSPICIOUS | Possible threat | Medium-confidence heuristic score, no feed confirmation |
| CLEAN | No threat detected | No feed hits AND low heuristic score |
## Event Schemas
### DetectionRequestEvent
Emitted when a new detection request arrives. Used for audit trail and
rate limiting.
### DetectionCompleteEvent
Emitted when a verdict is returned. Used for usage metering and evaluation.
### FeedRefreshEvent
Emitted when threat feed cache is updated. Used for monitoring feed health.- Save docs/data_model.md.
- Create docs/sli_slo.md with the SLI/SLO definitions:
# SLI/SLO Definitions: PhishGuard
## Service Level Indicators (SLIs)
### Detection Latency
- **Measurement**: End-to-end processing time from request received to response sent
- **Unit**: Milliseconds
- **Source**: processing_time_ms field in DetectionEvent
### Detection Precision
- **Measurement**: True positives / (True positives + False positives)
- **Unit**: Ratio (0-1)
- **Source**: Evaluation against labeled dataset
### Detection Recall
- **Measurement**: True positives / (True positives + False negatives)
- **Unit**: Ratio (0-1)
- **Source**: Evaluation against labeled dataset
## Service Level Objectives (SLOs)
| SLI | Objective | Rationale |
|-----|-----------|----------|
| Detection latency (p95) | < 500ms | Real-time detection must feel instant to the user |
| Precision | >= 0.8 | Max 20% false positive rate to maintain user trust |
| Recall | >= 0.8 | Catch at least 80% of actual threats |
## Measurement Strategy
- **Latency**: Measured per-request, aggregated to p50/p95/p99 in evaluation runs
- **Precision/Recall**: Measured against a labeled evaluation set after each deployment
- **Monitoring**: Cloud Functions built-in metrics for invocation count, error rate, latency- Save docs/sli_slo.md.
- Finally, create docs/implementation_plan.md with the agent roster and parallel execution strategy:
# Implementation Plan: PhishGuard
## Execution Strategy: Gentik Systems Multi-Agent Parallelization
Three Cursor Composer agents execute simultaneously under operator review.
Target wall-clock time: 16 minutes.
## Agent Roster
### Agent 1: Infrastructure (Terraform)
**Scope**: /infra directory
**Deliverables**:
- google_cloudfunctions2_function resource
- google_cloud_run_service_iam_member (public access)
- google_storage_bucket (feed cache)
- google_cloud_scheduler_job (hourly feed refresh)
- terraform.tfvars for dev and prod environments
- Variables and provider configuration
**Success criteria**: `terraform validate` passes, `terraform plan` shows
expected resources
### Agent 2: Detection Service (Python)
**Scope**: /src directory
**Deliverables**:
- Cloud Function HTTP entry point (main.py)
- Fast-path layer (feed_lookup.py): URL/domain lookup against cached feeds
- Heuristic scorer (scorer.py): feature extraction and weighted aggregation
- Model gateway interface (gateway.py): swappable scorer abstraction
- Verdict formatter (verdict.py): response structured per OpenAPI contract
- Requirements file (requirements.txt)
**Success criteria**: Local invocation returns correct verdicts for
known-bad, suspicious, and benign inputs
### Agent 3: Evaluation Harness
**Scope**: /eval directory
**Deliverables**:
- Labeled dataset (20 samples: 10 phishing, 5 malware, 5 benign)
- Evaluation script: precision, recall, F1 computation
- Latency measurement script: p50/p95/p99 calculation
- Results output format
**Success criteria**: Evaluation script runs against local endpoint,
produces precision/recall/F1 report
## Parallel Execution Timeline
| Minute | Agent 1 (Infra) | Agent 2 (Service) | Agent 3 (Eval) |
|--------|-----------------|-------------------|----------------|
| 0-4 | Provider + function config | Entry point + fast-path | Dataset assembly |
| 4-8 | Storage + scheduler | Heuristic scorer | Evaluation script |
| 8-12 | IAM + tfvars | Model gateway + verdict | Latency script |
| 12-16 | Validate + plan | Integration test | Dry run |
## Operator Role
- Architect the system (pre-artifacts, this document)
- Assign agent context windows
- Review agent output at each checkpoint
- Unblock cross-agent dependencies
- Final integration reviewWhy document the agent roster?
Each agent gets a defined scope (directory), explicit deliverables, and measurable success criteria. This prevents agents from stepping on each other's work or duplicating effort.
The timeline shows how three agents working in parallel compress 48 minutes of sequential work into 16 minutes of wall-clock time. This is the Gentik Systems multi-agent parallelization method.
- Save docs/implementation_plan.md.
- Verify all six files exist by checking your docs/ folder in Cursor's Explorer sidebar. You should see: adr.md, c4_diagrams.md, data_model.md, implementation_plan.md, openapi.yaml, and sli_slo.md.
✔️ Awesome, I've got all six files!
Great. Double check that no files landed in the project root by accident. All six should be inside the docs/ directory.
ⓧ I'd like to double check the full code
Here is the complete list of files and their key contents for comparison:
- docs/adr.md - Architecture Decision Record with 5 decisions (GCP, fast-path pattern, OpenPhish+URLhaus, human gate, 1/user pricing) plus global impact context.
- docs/c4_diagrams.md - Four Mermaid diagrams: C4Context, C4Container, C4Component, and a sequence diagram for the SaaS tenant flow.
- docs/openapi.yaml - OpenAPI 3.1 contract with POST /detect endpoint, DetectionRequest, DetectionResponse, Verdict, FeedHit, and DetectionEvent schemas.
- docs/data_model.md - DetectionEvent, FeedHit, and Verdict tables plus three event schemas (DetectionRequestEvent, DetectionCompleteEvent, FeedRefreshEvent).
- docs/sli_slo.md - Three SLIs (latency, precision, recall) with SLO targets: p95 < 500ms, precision >= 0.8, recall >= 0.8.
- docs/implementation_plan.md - Three-agent roster (infra, service, eval) with scope, deliverables, success criteria, and a 16-minute execution timeline.
Your design is complete. No service code has been written yet. Every architectural choice is recorded, every interface is defined, and every agent knows its scope. Next up, you will unleash all three agents in parallel to build the detection system in 16 minutes.
Build the Detection Service
Your pre-artifacts are complete. The architecture is documented, the OpenAPI contract defines the API surface, and the implementation plan assigns each agent its responsibilities.
Now you execute. Three Cursor Composer agents will build simultaneously: one for infrastructure, one for the detection service, one for the evaluation harness. You orchestrate, review, and unblock. This is Gentik Systems multi-agent parallelization. The target is 16 minutes of wall-clock time.
In this step, get ready to:
- Direct Agent 1 to produce the Terraform infrastructure in /infra.
- Direct Agent 2 to implement the full detection service in /src.
- Direct Agent 3 to build the evaluation harness in /eval.
Launch Agent 1: infrastructure
Agent 1 owns the /infra directory. It produces the Terraform configuration that deploys the detection function to GCP Cloud Functions 2nd gen, creates the feed cache bucket, and schedules hourly feed refreshes.
- In the Cursor Agents Window, open a new agent tab for Agent 1 (infrastructure).
- Use /worktree or a separate context window so this agent works in isolation.
- Paste the following prompt into Agent 1 to kick off the infrastructure build:
You are Agent 1 (Infrastructure). Build the Terraform configuration in /infra with these files:
1. variables.tf - Define: project_id (string), region (string, default us-central1), environment (string, default dev).
2. main.tf - Resources: google_cloudfunctions2_function (phishguard-detect), google_cloud_run_service_iam_member (allUsers invoker), google_storage_bucket (phishguard-feed-cache), google_cloud_scheduler_job (hourly feed refresh hitting /refresh-feeds).
3. terraform.tfvars - Dev environment values with placeholder project_id.
Use the google provider ~> 5.0. The function uses python311 runtime, entry_point "detect", 256M memory, 60s timeout. The scheduler runs on cron "0 * * * *". The bucket has a 1-day lifecycle rule for cache freshness.Agent 1 should produce three files. Review each against the reference below.
- Confirm Agent 1 created infra/variables.tf with three variables by running:
cat infra/variables.tf- Confirm Agent 1 created infra/main.tf with all four resources by running:
cat infra/main.tf- Validate the Terraform configuration by running:
cd infra
terraform init
terraform validateYou should see Success! The configuration is valid. in the output.
What did Agent 1 produce?
The Terraform configuration defines four GCP resources that work together:
- google_storage_bucket holds cached threat feed data with a 1-day expiry so stale feeds auto-clean.
- google_cloudfunctions2_function is the detection endpoint itself, running Python 3.11.
- google_cloud_run_service_iam_member makes the endpoint publicly accessible (required for an API).
- google_cloud_scheduler_job triggers a feed refresh every hour to keep the cache current.
🙋♀️ Terraform validate failing?
- Make sure you ran terraform init first. Without it, the Google provider is not downloaded.
- Check that variables.tf defines project_id, region, and environment as strings. Missing variables cause reference errors.
Why is my Terraform configuration not validating?
Launch Agent 2: detection service
Agent 2 owns /src. It implements the full detection pipeline: fast-path feed lookup, heuristic scorer, model gateway, verdict formatter, and metering hook. The fast-path-before-model pattern means known-bad URLs resolve instantly from the feed cache without ever calling the scorer.
- In the Cursor Agents Window, open a second agent tab for Agent 2 (detection service).
- Paste the following prompt into Agent 2:
You are Agent 2 (Detection Service). Build the detection pipeline in /src with these files:
1. requirements.txt - Dependencies: functions-framework, requests, python-Levenshtein, python-dotenv.
2. model_gateway.py - Abstract DetectionModel base class with score(features) method. HeuristicModel implementation that delegates to the scorer.
3. fast_path.py - Load OpenPhish and URLhaus JSON cache files from /src/data/. check_fast_path(url) returns instant MALICIOUS verdict with feed_hits if URL is in either cache, None otherwise.
4. heuristic_scorer.py - Feature extraction: calculate_url_entropy (Shannon entropy normalized 0-1), check_suspicious_tld (known bad TLDs like .xyz, .top, .tk, .ml, .ga), check_brand_impersonation (Levenshtein distance on brand names), check_urgency_language (regex patterns), check_domain_age_heuristic (structural signals). Weighted aggregation with score() producing verdict MALICIOUS/SUSPICIOUS/CLEAN based on thresholds 0.7/0.4.
5. verdict_formatter.py - format_verdict() returning dict with verdict, confidence, signals[], recommended_action, feed_hits[] per OpenAPI contract.
6. metering.py - log_detection_event() logs each request as a JSON metering event with timestamp, verdict, confidence.
7. main.py - Cloud Function entry point using @functions_framework.http decorator on detect(request). Routes POST requests through fast_path first, then heuristic scorer via model gateway, then verdict formatter. Calls metering hook. Returns 405 for non-POST, 400 if no url or message provided.
Use inline comments explaining each section. Reference docs/openapi.yaml for response schema.Agent 2 produces seven files. The most critical are main.py (the entry point), fast_path.py (the fast path), and heuristic_scorer.py (the scoring engine).
Why fast-path before model?
Known-bad URLs that already exist in OpenPhish or URLhaus feeds can be resolved in microseconds from a local set lookup. Calling the scorer on these is wasted compute. The fast path returns an instant MALICIOUS verdict with 0.99 confidence.
Only URLs that are NOT in any feed proceed to the heuristic scorer. This is how production systems like Norton Genie achieve sub-second response times on 500K+ submissions.
- Once Agent 2 finishes, verify the dependencies file exists by running:
cat src/requirements.txtYou should see functions-framework, requests, python-Levenshtein, and python-dotenv listed.
- Install the dependencies locally by running:
cd src
pip install -r requirements.txt- Create a test data directory and a minimal phishing cache to verify the fast path by running:
mkdir -p src/data
echo '[{"url": "http://paypal-secure-login.xyz/verify"}]' > src/data/openphish_cache.json
echo '[]' > src/data/urlhaus_cache.json- Start the function locally using the Functions Framework by running:
cd src
functions-framework --target=detectThe function starts at localhost:8080 by default.
- Open a second terminal and test with a known-phishing URL by running:
curl -X POST http://localhost:8080 -H "Content-Type: application/json" -d "{\"url\": \"http://paypal-secure-login.xyz/verify\"}"You should see a JSON response with "verdict": "MALICIOUS" and feed_hits populated.
- Test with a suspicious message (not in any feed) by running:
curl -X POST http://localhost:8080 -H "Content-Type: application/json" -d "{\"url\": \"http://amaz0n-security.xyz/verify\", \"message\": \"Your account will be suspended. Act now to verify your identity immediately.\"}"You should see "verdict": "SUSPICIOUS" with heuristic signals listed.
- Test with a benign URL by running:
curl -X POST http://localhost:8080 -H "Content-Type: application/json" -d "{\"url\": \"https://www.google.com\", \"message\": \"Check out this article about cloud computing.\"}"You should see "verdict": "CLEAN" with an empty signals array.
- Stop the local server with Ctrl+C in the terminal running the Functions Framework.
What does the heuristic scorer measure?
The scorer extracts five features from each input, each normalized to a 0-1 range:
- URL entropy (15% weight): Shannon entropy detects randomly generated domains.
- Suspicious TLD (20% weight): Known scam TLDs like .xyz, .top, .tk.
- Brand impersonation (25% weight): Levenshtein distance catches misspelled brand names like "amaz0n" or "paypa1".
- Urgency language (25% weight): Regex patterns match phrases like "act now", "account locked", "verify your identity".
- Domain age heuristic (15% weight): Structural signals (long subdomains, numeric sequences) indicate throwaway domains.
A weighted sum above 0.7 triggers MALICIOUS. Between 0.4 and 0.7 triggers SUSPICIOUS. Below 0.4 is CLEAN.
🙋♀️ Not seeing the expected verdict?
- If the fast-path test returns SUSPICIOUS instead of MALICIOUS, check that src/data/openphish_cache.json contains the exact URL you are testing with.
- If you see a ModuleNotFoundError, make sure you ran pip install -r requirements.txt from within the src directory.
- If curl returns connection refused, confirm the Functions Framework is still running in the other terminal.
Help me debug why my PhishGuard detection service is returning the wrong verdict locally.
Launch Agent 3: evaluation harness
Agent 3 owns /eval. It builds a labeled dataset of 20 samples and two scripts that measure detection quality and response latency against the SLO targets defined in docs/sli_slo.md.
- In the Cursor Agents Window, open a third agent tab for Agent 3 (evaluation).
- Paste the following prompt into Agent 3:
You are Agent 3 (Evaluation Harness). Build the test and measurement tools in /eval with these files:
1. labeled_samples.json - 20 labeled samples: 10 known-phishing URLs (using suspicious TLDs, brand impersonation patterns, urgency messages), 5 known-malware URLs, 5 benign URLs (google.com, github.com, python.org, wikipedia.org, stackoverflow.com). Each sample has url, message, and label fields.
2. evaluate.py - Loads labeled_samples.json, sends each to the detection endpoint (default http://localhost:8080), classifies MALICIOUS/SUSPICIOUS as positive and CLEAN as negative, computes precision, recall, F1. Prints SLO check (precision >= 0.8, recall >= 0.8).
3. latency.py - Sends 10 sequential requests to the endpoint, measures response time in ms per request, computes p50/p95/p99. Prints SLO check (p95 < 500ms).
Both scripts use the requests library and default to http://localhost:8080 as the endpoint URL.- Once Agent 3 finishes, verify the labeled samples exist by running:
cat eval/labeled_samples.json | python -c "import json,sys; data=json.load(sys.stdin); print(f'{len(data)} samples loaded')"You should see 20 samples loaded in the output.
- Verify the evaluation script is importable by running:
python -c "import eval.evaluate; print('evaluate.py OK')"- Verify the latency script is importable by running:
python -c "import eval.latency; print('latency.py OK')"What makes a good labeled dataset?
The 20 samples deliberately cover three threat categories: phishing (credential theft URLs), malware (payload distribution), and benign (well-known legitimate sites). This forces the scorer to distinguish genuine threats from safe content.
The phishing samples use the same TLD and brand-impersonation patterns the heuristic scorer detects. This validates that the scorer's feature extraction actually fires on realistic threats, not just synthetic test strings.
🙋♀️ Import errors on the eval scripts?
- Make sure you are running the commands from the project root directory, not from inside /eval.
- If you see "No module named eval", create an empty eval/__init__.py file so Python treats eval as a package.
Help me fix import errors with my evaluation harness scripts.
✔️ Awesome, I've got everything!
Great. Make sure you have stopped the local Functions Framework server before moving on.
ⓧ I'd like to double check the full code
Here are the key files produced by each agent. Compare your agent output against these references.
infra/variables.tf
variable "project_id" {
description = "GCP project ID"
type = string
}
variable "region" {
description = "GCP region for resources"
type = string
default = "us-central1"
}
variable "environment" {
description = "Deployment environment (dev or prod)"
type = string
default = "dev"
}infra/terraform.tfvars
# Dev environment values
project_id = "your-gcp-project-id"
region = "us-central1"
environment = "dev"infra/main.tf
terraform {
required_providers {
google = {
source = "hashicorp/google"
version = "~> 5.0"
}
}
}
provider "google" {
project = var.project_id
region = var.region
}
# Storage bucket for cached threat feed data
resource "google_storage_bucket" "feed_cache" {
name = "${var.project_id}-phishguard-feed-cache"
location = var.region
uniform_bucket_level_access = true
lifecycle_rule {
condition {
age = 1
}
action {
type = "Delete"
}
}
}
# Cloud Function for detection endpoint
resource "google_cloudfunctions2_function" "phishguard_detect" {
name = "phishguard-detect-${var.environment}"
location = var.region
build_config {
runtime = "python311"
entry_point = "detect"
source {
storage_source {
bucket = google_storage_bucket.feed_cache.name
object = "source/function-source.zip"
}
}
}
service_config {
max_instance_count = 10
available_memory = "256M"
timeout_seconds = 60
environment_variables = {
ENVIRONMENT = var.environment
FEED_BUCKET = google_storage_bucket.feed_cache.name
}
}
}
# Allow unauthenticated access to the function
resource "google_cloud_run_service_iam_member" "invoker" {
location = var.region
service = google_cloudfunctions2_function.phishguard_detect.name
role = "roles/run.invoker"
member = "allUsers"
}
# Scheduler job for hourly feed refresh
resource "google_cloud_scheduler_job" "feed_refresh" {
name = "phishguard-feed-refresh-${var.environment}"
region = var.region
schedule = "0 * * * *"
http_target {
http_method = "POST"
uri = "${google_cloudfunctions2_function.phishguard_detect.service_config[0].uri}/refresh-feeds"
}
}src/requirements.txt
functions-framework
requests
python-Levenshtein
python-dotenvsrc/model_gateway.py
"""Swappable model gateway interface.
The gateway defines a standard interface for detection models.
The MVP uses a heuristic scorer. Future versions can swap in
an ML model without changing the ingestion or verdict layers.
"""
from abc import ABC, abstractmethod
class DetectionModel(ABC):
"""Abstract base class for detection models."""
@abstractmethod
def score(self, features: dict) -> dict:
"""Score extracted features and return a verdict.
Args:
features: Dictionary of extracted features.
Returns:
Dictionary with verdict, confidence, and signals.
"""
pass
class HeuristicModel(DetectionModel):
"""Routes scoring to the heuristic scorer."""
def __init__(self, scorer):
self.scorer = scorer
def score(self, features: dict) -> dict:
"""Score using the heuristic scorer."""
return self.scorer.score(features)src/fast_path.py
"""Fast-path layer for known-bad indicator lookup.
Checks URLs and domains against locally-cached OpenPhish
and URLhaus feeds. Known-bad indicators get an instant
MALICIOUS verdict without calling the scorer.
"""
import os
import json
# Cache for loaded feed data
_openphish_cache = set()
_urlhaus_cache = set()
def load_feeds():
"""Load threat feed data into memory cache."""
global _openphish_cache, _urlhaus_cache
# Load OpenPhish data from local cache file
openphish_path = os.path.join(os.path.dirname(__file__), "data", "openphish_cache.json")
if os.path.exists(openphish_path):
with open(openphish_path, "r") as f:
data = json.load(f)
_openphish_cache = set(entry.get("url", "") for entry in data)
# Load URLhaus data from local cache file
urlhaus_path = os.path.join(os.path.dirname(__file__), "data", "urlhaus_cache.json")
if os.path.exists(urlhaus_path):
with open(urlhaus_path, "r") as f:
data = json.load(f)
_urlhaus_cache = set(entry.get("url", "") for entry in data)
def check_fast_path(url: str) -> dict | None:
"""Check if a URL is in the known-bad feeds.
Args:
url: The URL to check against threat feeds.
Returns:
A verdict dict if found in feeds, None otherwise.
"""
if not url:
return None
feed_hits = []
# Check OpenPhish cache
if url in _openphish_cache:
feed_hits.append({
"feed": "OpenPhish",
"indicator": url,
"threat_type": "phishing"
})
# Check URLhaus cache
if url in _urlhaus_cache:
feed_hits.append({
"feed": "URLhaus",
"indicator": url,
"threat_type": "malware_distribution"
})
# If any feed matched, return instant MALICIOUS verdict
if feed_hits:
return {
"verdict": "MALICIOUS",
"confidence": 0.99,
"signals": [f"URL found in {hit['feed']} ({hit['threat_type']})" for hit in feed_hits],
"recommended_action": "block",
"feed_hits": feed_hits
}
return Nonesrc/heuristic_scorer.py
"""Heuristic scorer with feature extraction and weighted aggregation.
Extracts features: URL entropy, suspicious TLD, brand impersonation
(Levenshtein distance), urgency language patterns, domain age heuristic.
Produces a confidence-scored verdict.
"""
import math
import re
from urllib.parse import urlparse
from collections import Counter
import Levenshtein
# Known suspicious TLDs commonly used in phishing
SUSPICIOUS_TLDS = {
".xyz", ".top", ".club", ".work", ".buzz", ".tk",
".ml", ".ga", ".cf", ".gq", ".info", ".click",
".link", ".support", ".review", ".zip", ".mov"
}
# Brand names commonly impersonated in phishing
TARGET_BRANDS = [
"paypal", "apple", "microsoft", "google", "amazon",
"netflix", "facebook", "instagram", "chase", "wellsfargo",
"bankofamerica", "citibank", "usps", "fedex", "dhl"
]
# Urgency language patterns in phishing messages
URGENCY_PATTERNS = [
r"act\s+now", r"immediate(ly)?", r"urgent", r"suspend",
r"verify\s+your", r"confirm\s+your", r"account\s+locked",
r"unusual\s+activity", r"expire[sd]?", r"limited\s+time",
r"click\s+here\s+to", r"within\s+24\s+hours", r"final\s+warning"
]
# Feature weights for scoring
WEIGHTS = {
"url_entropy": 0.15,
"suspicious_tld": 0.20,
"brand_impersonation": 0.25,
"urgency_language": 0.25,
"domain_age_heuristic": 0.15
}
def calculate_url_entropy(url: str) -> float:
"""Calculate Shannon entropy of the URL string.
High entropy indicates randomly generated domains.
"""
if not url:
return 0.0
# Count character frequencies
freq = Counter(url)
length = len(url)
# Calculate Shannon entropy
entropy = -sum(
(count / length) * math.log2(count / length)
for count in freq.values()
)
# Normalize to 0-1 range (max entropy for printable ASCII)
max_entropy = math.log2(min(length, 95))
if max_entropy == 0:
return 0.0
return min(entropy / max_entropy, 1.0)
def check_suspicious_tld(url: str) -> float:
"""Check if the URL uses a known suspicious TLD.
Returns 1.0 if suspicious, 0.0 if not.
"""
try:
parsed = urlparse(url if "://" in url else f"http://{url}")
domain = parsed.netloc or parsed.path
for tld in SUSPICIOUS_TLDS:
if domain.endswith(tld):
return 1.0
except Exception:
pass
return 0.0
def check_brand_impersonation(url: str) -> float:
"""Check for brand impersonation using Levenshtein distance.
Looks for brand names appearing in subdomains or paths
with slight misspellings.
"""
try:
parsed = urlparse(url if "://" in url else f"http://{url}")
domain = parsed.netloc or parsed.path
domain_lower = domain.lower()
# Extract domain parts for analysis
parts = domain_lower.replace(".", " ").replace("-", " ").split()
for part in parts:
for brand in TARGET_BRANDS:
# Exact match in subdomain/path (not the actual domain)
if part == brand and brand not in domain_lower.split(".")[-2:-1]:
return 0.9
# Close misspelling (Levenshtein distance 1-2)
distance = Levenshtein.distance(part, brand)
if 0 < distance <= 2 and len(part) > 3:
return 0.8
except Exception:
pass
return 0.0
def check_urgency_language(message: str) -> float:
"""Check for urgency language patterns common in phishing.
Returns a score based on how many patterns match.
"""
if not message:
return 0.0
message_lower = message.lower()
matches = sum(
1 for pattern in URGENCY_PATTERNS
if re.search(pattern, message_lower)
)
# Normalize: 3+ matches is maximum suspicion
return min(matches / 3.0, 1.0)
def check_domain_age_heuristic(url: str) -> float:
"""Heuristic for domain age based on URL characteristics.
Newly registered domains often use certain patterns.
This is a heuristic proxy when WHOIS is unavailable.
"""
if not url:
return 0.0
signals = 0
try:
parsed = urlparse(url if "://" in url else f"http://{url}")
domain = parsed.netloc or parsed.path
# Long random-looking subdomain chains suggest new throwaway domains
parts = domain.split(".")
if len(parts) > 3:
signals += 1
# Numeric sequences in domain
if re.search(r"\d{4,}", domain):
signals += 1
# Very long domain names
if len(domain) > 40:
signals += 1
except Exception:
pass
return min(signals / 2.0, 1.0)
def extract_features(url: str, message: str) -> dict:
"""Extract all heuristic features from the input.
Args:
url: The suspect URL to analyze.
message: The suspect message text to analyze.
Returns:
Dictionary of feature names to scores (0.0-1.0).
"""
return {
"url_entropy": calculate_url_entropy(url),
"suspicious_tld": check_suspicious_tld(url),
"brand_impersonation": check_brand_impersonation(url),
"urgency_language": check_urgency_language(message),
"domain_age_heuristic": check_domain_age_heuristic(url)
}
def score(features: dict) -> dict:
"""Compute weighted aggregation and produce a verdict.
Args:
features: Dictionary of feature scores from extract_features.
Returns:
Dictionary with verdict, confidence, and signals.
"""
# Weighted aggregation
total_score = sum(
features.get(feature, 0.0) * weight
for feature, weight in WEIGHTS.items()
)
# Determine verdict based on score thresholds
if total_score >= 0.7:
verdict = "MALICIOUS"
recommended_action = "block"
elif total_score >= 0.4:
verdict = "SUSPICIOUS"
recommended_action = "warn"
else:
verdict = "CLEAN"
recommended_action = "allow"
# Collect active signals for the response
signals = []
if features.get("url_entropy", 0) > 0.7:
signals.append("High URL entropy (possible random domain)")
if features.get("suspicious_tld", 0) > 0:
signals.append("Suspicious TLD detected")
if features.get("brand_impersonation", 0) > 0:
signals.append("Possible brand impersonation")
if features.get("urgency_language", 0) > 0:
signals.append("Urgency language patterns detected")
if features.get("domain_age_heuristic", 0) > 0:
signals.append("Domain age heuristic triggered")
return {
"verdict": verdict,
"confidence": round(total_score, 3),
"signals": signals,
"recommended_action": recommended_action
}src/verdict_formatter.py
"""Formats detection results per the OpenAPI contract.
Response structure: verdict, confidence, signals[],
recommended_action, feed_hits[].
"""
def format_verdict(verdict: str, confidence: float, signals: list,
recommended_action: str, feed_hits: list = None) -> dict:
"""Format the detection result per OpenAPI contract.
Args:
verdict: MALICIOUS, SUSPICIOUS, or CLEAN.
confidence: Score between 0.0 and 1.0.
signals: List of triggered signal descriptions.
recommended_action: block, warn, or allow.
feed_hits: List of threat feed match details.
Returns:
Formatted response dictionary matching OpenAPI schema.
"""
return {
"verdict": verdict,
"confidence": confidence,
"signals": signals or [],
"recommended_action": recommended_action,
"feed_hits": feed_hits or []
}src/metering.py
"""Usage-metering hook for SaaS billing.
Logs each detection request as a countable event.
In production, this would publish to a billing pipeline.
"""
import json
import logging
from datetime import datetime, timezone
logger = logging.getLogger(__name__)
def log_detection_event(request_data: dict, verdict: dict) -> dict:
"""Log a detection event for usage metering.
Args:
request_data: The incoming request data.
verdict: The detection verdict response.
Returns:
The metering event record.
"""
event = {
"event_type": "detection_request",
"timestamp": datetime.now(timezone.utc).isoformat(),
"verdict": verdict.get("verdict", "UNKNOWN"),
"confidence": verdict.get("confidence", 0.0),
"has_url": bool(request_data.get("url")),
"has_message": bool(request_data.get("message")),
"feed_hits_count": len(verdict.get("feed_hits", []))
}
# Log the metering event
logger.info(f"METERING: {json.dumps(event)}")
return eventsrc/main.py
"""Cloud Function HTTP entry point for PhishGuard detection.
Routes requests to fast_path, heuristic_scorer, and verdict_formatter.
Calls the metering hook for each detection event.
"""
import json
import functions_framework
from fast_path import load_feeds, check_fast_path
from heuristic_scorer import extract_features, score
from model_gateway import HeuristicModel
from verdict_formatter import format_verdict
from metering import log_detection_event
# Load threat feeds into memory on cold start
load_feeds()
class _HeuristicScorer:
"""Adapter to use heuristic_scorer.score via model gateway."""
def score(self, features):
return score(features)
# Initialize model gateway with heuristic scorer
model = HeuristicModel(_HeuristicScorer())
@functions_framework.http
def detect(request):
"""HTTP entry point for detection requests.
Accepts POST with JSON body: { "url": "...", "message": "..." }
Returns JSON verdict per OpenAPI contract.
"""
# Parse request
if request.method != "POST":
return json.dumps({"error": "Method not allowed"}), 405
request_data = request.get_json(silent=True) or {}
url = request_data.get("url", "")
message = request_data.get("message", "")
if not url and not message:
return json.dumps({"error": "Provide url or message field"}), 400
# Fast path: check against threat feed cache
fast_result = check_fast_path(url)
if fast_result:
verdict_response = format_verdict(
verdict=fast_result["verdict"],
confidence=fast_result["confidence"],
signals=fast_result["signals"],
recommended_action=fast_result["recommended_action"],
feed_hits=fast_result["feed_hits"]
)
log_detection_event(request_data, verdict_response)
return json.dumps(verdict_response), 200
# Heuristic scoring via model gateway
features = extract_features(url, message)
scorer_result = model.score(features)
verdict_response = format_verdict(
verdict=scorer_result["verdict"],
confidence=scorer_result["confidence"],
signals=scorer_result["signals"],
recommended_action=scorer_result["recommended_action"],
feed_hits=[]
)
# Log metering event
log_detection_event(request_data, verdict_response)
return json.dumps(verdict_response), 200eval/labeled_samples.json
[
{"url": "http://paypal-secure-login.xyz/verify", "message": "", "label": "phishing"},
{"url": "http://apple-id-confirm.top/account", "message": "Your Apple ID has been locked. Click here to verify immediately.", "label": "phishing"},
{"url": "http://microsoft-support-team.click/reset", "message": "", "label": "phishing"},
{"url": "http://netflix-billing-update.buzz/payment", "message": "Your Netflix account will expire in 24 hours. Update now.", "label": "phishing"},
{"url": "http://chase-security-alert.work/login", "message": "", "label": "phishing"},
{"url": "http://amaz0n-order-confirm.tk/track", "message": "Unusual activity on your account. Act now to avoid suspension.", "label": "phishing"},
{"url": "http://googIe-drive-share.ml/document", "message": "", "label": "phishing"},
{"url": "http://wellsfarg0-alert.ga/secure", "message": "Verify your identity within 24 hours or your account will be suspended.", "label": "phishing"},
{"url": "http://fedex-delivery-notice.cf/track", "message": "Your package is held. Confirm your address immediately.", "label": "phishing"},
{"url": "http://instagram-verify.gq/confirm", "message": "", "label": "phishing"},
{"url": "http://malware-download.example.com/trojan.exe", "message": "", "label": "malware"},
{"url": "http://exploit-kit.example.net/payload.bin", "message": "", "label": "malware"},
{"url": "http://ransomware-drop.example.org/encrypt.dll", "message": "", "label": "malware"},
{"url": "http://botnet-c2.example.com/beacon", "message": "", "label": "malware"},
{"url": "http://cryptominer-install.example.net/mine.sh", "message": "", "label": "malware"},
{"url": "https://www.google.com", "message": "Check out this article about cloud computing.", "label": "benign"},
{"url": "https://github.com/user/repo", "message": "", "label": "benign"},
{"url": "https://docs.python.org/3/tutorial/", "message": "Here is the Python tutorial link you asked for.", "label": "benign"},
{"url": "https://www.wikipedia.org/wiki/Machine_learning", "message": "", "label": "benign"},
{"url": "https://stackoverflow.com/questions/12345", "message": "Found this answer to your coding question.", "label": "benign"}
]eval/evaluate.py
"""Evaluation script: precision, recall, F1 against labeled samples.
Runs each sample through the detection endpoint and compares
the returned verdict to the known label.
"""
import json
import requests
# Detection endpoint URL
ENDPOINT = "http://localhost:8080"
def load_samples(path="labeled_samples.json"):
"""Load labeled samples from JSON file."""
with open(path, "r") as f:
return json.load(f)
def classify_verdict(verdict: str) -> str:
"""Map verdict to binary classification.
MALICIOUS or SUSPICIOUS = positive (detected as threat).
CLEAN = negative (not a threat).
"""
if verdict in ("MALICIOUS", "SUSPICIOUS"):
return "positive"
return "negative"
def expected_class(label: str) -> str:
"""Map sample label to binary classification.
phishing or malware = positive.
benign = negative.
"""
if label in ("phishing", "malware"):
return "positive"
return "negative"
def run_evaluation(endpoint: str, samples: list) -> dict:
"""Run evaluation against all samples.
Args:
endpoint: The detection API URL.
samples: List of labeled sample dicts.
Returns:
Dictionary with precision, recall, F1, and details.
"""
true_positives = 0
false_positives = 0
false_negatives = 0
true_negatives = 0
results = []
for sample in samples:
payload = {"url": sample.get("url", ""), "message": sample.get("message", "")}
response = requests.post(f"{endpoint}/detect", json=payload)
result = response.json()
predicted = classify_verdict(result.get("verdict", "CLEAN"))
actual = expected_class(sample["label"])
if predicted == "positive" and actual == "positive":
true_positives += 1
elif predicted == "positive" and actual == "negative":
false_positives += 1
elif predicted == "negative" and actual == "positive":
false_negatives += 1
else:
true_negatives += 1
results.append({
"url": sample.get("url"),
"label": sample["label"],
"verdict": result.get("verdict"),
"confidence": result.get("confidence"),
"correct": predicted == actual
})
# Calculate metrics
precision = true_positives / (true_positives + false_positives) if (true_positives + false_positives) > 0 else 0.0
recall = true_positives / (true_positives + false_negatives) if (true_positives + false_negatives) > 0 else 0.0
f1 = 2 * (precision * recall) / (precision + recall) if (precision + recall) > 0 else 0.0
return {
"precision": round(precision, 3),
"recall": round(recall, 3),
"f1": round(f1, 3),
"true_positives": true_positives,
"false_positives": false_positives,
"false_negatives": false_negatives,
"true_negatives": true_negatives,
"total_samples": len(samples),
"results": results
}
if __name__ == "__main__":
samples = load_samples()
metrics = run_evaluation(ENDPOINT, samples)
print(f"\nEvaluation Results ({metrics['total_samples']} samples)")
print(f"{'='*40}")
print(f"Precision: {metrics['precision']}")
print(f"Recall: {metrics['recall']}")
print(f"F1 Score: {metrics['f1']}")
print(f"\nConfusion Matrix:")
print(f" True Positives: {metrics['true_positives']}")
print(f" False Positives: {metrics['false_positives']}")
print(f" True Negatives: {metrics['true_negatives']}")
print(f" False Negatives: {metrics['false_negatives']}")
# Check against SLO floors
print(f"\nSLO Check:")
print(f" Precision >= 0.8: {'PASS' if metrics['precision'] >= 0.8 else 'FAIL'}")
print(f" Recall >= 0.8: {'PASS' if metrics['recall'] >= 0.8 else 'FAIL'}")eval/latency.py
"""Latency measurement: 10 sequential requests, p50/p95/p99.
Sends test requests to the detection endpoint and calculates
percentile response times.
"""
import time
import json
import requests
# Detection endpoint URL
ENDPOINT = "http://localhost:8080"
# Test payload for latency measurement
TEST_PAYLOAD = {
"url": "http://suspicious-test-domain.xyz/login",
"message": "Please verify your account immediately."
}
def measure_latency(endpoint: str, payload: dict, num_requests: int = 10) -> dict:
"""Measure response latency over multiple requests.
Args:
endpoint: The detection API URL.
payload: Request body to send.
num_requests: Number of sequential requests.
Returns:
Dictionary with p50, p95, p99 latencies in ms.
"""
latencies = []
for i in range(num_requests):
start = time.time()
response = requests.post(f"{endpoint}/detect", json=payload)
elapsed = (time.time() - start) * 1000 # Convert to ms
latencies.append(elapsed)
if response.status_code != 200:
print(f" Request {i+1}: ERROR ({response.status_code})")
else:
print(f" Request {i+1}: {elapsed:.1f}ms")
# Sort for percentile calculation
latencies.sort()
n = len(latencies)
def percentile(p):
index = int(p / 100 * n) - 1
return latencies[max(0, min(index, n - 1))]
return {
"p50": round(percentile(50), 1),
"p95": round(percentile(95), 1),
"p99": round(percentile(99), 1),
"min": round(min(latencies), 1),
"max": round(max(latencies), 1),
"num_requests": num_requests
}
if __name__ == "__main__":
print(f"Measuring latency ({10} requests)...")
print(f"Endpoint: {ENDPOINT}")
print(f"{'='*40}")
results = measure_latency(ENDPOINT, TEST_PAYLOAD)
print(f"\nLatency Results:")
print(f" p50: {results['p50']}ms")
print(f" p95: {results['p95']}ms")
print(f" p99: {results['p99']}ms")
print(f" Min: {results['min']}ms")
print(f" Max: {results['max']}ms")
# Check against SLO target
print(f"\nSLO Check:")
print(f" p95 < 500ms: {'PASS' if results['p95'] < 500 else 'FAIL'}")All three agents have delivered. Your detection service runs locally, returns correct verdicts across all three scenarios, and the Terraform configuration validates successfully. Next up, you will deploy this to GCP and run the evaluation suite against the live endpoint.
Deploy and Validate
Your detection service passed every local test. Three Cursor Composer agents built the infrastructure config, detection pipeline, and evaluation harness in parallel. Now it's time to prove this system works where it matters: on a live GCP endpoint handling real HTTP requests over the public internet.
Terraform will push your entire infrastructure stack to GCP in a single apply. Once deployed, you'll validate the live system against the SLO floors defined in your pre-artifacts: precision and recall at 0.8 or above, latency p95 under 500ms.
In this step, get ready to:
- Deploy PhishGuard to GCP Cloud Functions via Terraform.
- Validate precision, recall, and latency against SLO targets.
- Create an operational runbook for the deployed system.
Deploy to the dev environment
Local tests prove the code logic works. Deploying to GCP proves it works under real conditions: network latency, cold starts, and live threat feed connections.
- Navigate into your infra directory and initialize Terraform by running these commands:
cd infra
terraform initWhat does terraform init do?
This command downloads the Google Cloud provider plugin that Terraform needs to manage GCP resources. It reads your main.tf configuration and prepares the working directory.
🙋♀️ Seeing a provider error?
Make sure your main.tf specifies the google provider with a valid version constraint. Also confirm your internet connection is active, since Terraform downloads provider binaries from the registry.
Help me debug my terraform init error
- Preview the infrastructure changes by running:
terraform planYou should see a plan showing resources to create: a Cloud Function, a Cloud Storage bucket, a Cloud Scheduler job, and an IAM binding. Review the output to confirm it matches your expectations.
- Deploy the infrastructure to GCP by running:
terraform apply- Terraform will show the plan again and ask for confirmation. Type yes and press Enter to proceed.
- Wait for the apply to complete. This typically takes 1-3 minutes as GCP provisions the Cloud Function and associated resources.
- Note the function URL from the Terraform output. It will appear as a URL ending in .run.app since Cloud Functions 2nd gen runs on Cloud Run infrastructure.
What just happened?
Terraform created four resources in your GCP project:
- phishguard-detect Cloud Function (your detection API endpoint).
- phishguard-feed-cache Cloud Storage bucket (stores cached threat feed data).
- A Cloud Scheduler job that triggers hourly feed refreshes.
- An IAM binding that allows unauthenticated HTTP access to the function.
🙋♀️ Terraform apply failed?
Common causes:
- APIs not enabled: confirm you ran the gcloud services enable commands from Step 1.
- Billing not linked: GCP requires an active billing account even for free-tier resources.
- Region mismatch: check that the region in terraform.tfvars matches a region that supports Cloud Functions 2nd gen.
Help me fix my terraform apply error on GCP
- If you need to retrieve the function URL later, run this command:
gcloud functions describe phishguard-detect --gen2 --format="value(serviceConfig.uri)"- Test the live endpoint with a curl request to confirm it responds. Replace the URL with your deployed function URL:
curl -X POST [[FUNCTION_URL="your deployed function URL ending in .run.app"]] \
-H "Content-Type: application/json" \
-d '{"url": "http://testsafebrowsing.appspot.com/s/phishing.html", "message": ""}'You should see a JSON response containing a verdict field (MALICIOUS, SUSPICIOUS, or CLEAN), a confidence score, and a signals array.
Getting a timeout or connection error?
The first request after deployment may take longer due to a cold start (approximately 200ms for Python on GCP 2nd gen). If the request times out completely, confirm the IAM binding allows unauthenticated access and that the function deployed successfully.
Why is my deployed Cloud Function not responding?
Run the evaluation suite against the live endpoint
A single successful curl proves the endpoint is alive. The evaluation suite proves it actually detects threats accurately and responds fast enough to be useful in real time.
- Navigate to the eval directory and run the evaluation script against your live endpoint:
cd ../eval
python evaluate.py [[FUNCTION_URL="your deployed function URL ending in .run.app"]]The script sends all 20 labeled samples from labeled_samples.json to your live endpoint and computes precision, recall, and F1. You should see all three metrics printed to the terminal.
What do these metrics mean?
Precision measures how many of the items flagged as malicious were actually malicious. A precision of 0.8 means at most 20% of flagged items are false positives.
Recall measures how many actual threats were caught. A recall of 0.8 means the system catches at least 80% of real threats.
F1 is the harmonic mean of precision and recall. It balances both into a single score.
- Confirm precision >= 0.8 and recall >= 0.8 in the output. These are the SLO floors defined in docs/sli_slo.md.
- Run the latency measurement script:
python latency.py [[FUNCTION_URL="your deployed function URL ending in .run.app"]]This sends 10 sequential requests and computes p50, p95, and p99 latency. Confirm p95 is under 500ms to pass the latency SLO.
Precision or recall below 0.8?
If the fast-path layer is not finding known-bad URLs, the feed cache may not be populated yet. Wait for the first Cloud Scheduler trigger (up to 1 hour) or manually trigger a feed refresh, then rerun the evaluation.
If latency p95 exceeds 500ms, the first few requests may include cold starts. Discard the first request as warmup and rerun latency.py.
Help me debug my evaluation results
Create the operational runbook
A deployed system without documentation is a liability. The runbook captures everything an operator needs to deploy, tear down, promote, and scale PhishGuard.
- Navigate back to your docs directory and create runbook.md by running:
cd ../docs
touch runbook.md- Open runbook.md in Cursor and add the deploy commands section:
# PhishGuard Operational Runbook
## Deploy Commands
```bash
cd infra
terraform init
terraform plan
terraform apply
```
Confirm deployment:
```bash
gcloud functions describe phishguard-detect --gen2
```Why document deploy commands?
Runbooks eliminate tribal knowledge. Anyone on the team can deploy, verify, or roll back without asking the original author. This is foundational for SaaS operations.
- Add the teardown and environment promotion sections below the deploy commands:
## Teardown
```bash
cd infra
terraform destroy
```
Confirm removal:
```bash
gcloud functions list
```
The function should no longer appear in the output.
## Environment Promotion (Dev to Prod)
1. Update `terraform.tfvars` to set `environment = "prod"`
2. Run `terraform plan` to review production changes
3. Run `terraform apply` to deploy to production
4. Run evaluation suite against the prod endpoint to confirm SLO compliance- Add the feed-refresh schedule and SaaS scaling path sections:
## Feed Refresh Schedule
- Cloud Scheduler triggers feed cache refresh every hour
- OpenPhish feed updates every 12 hours
- URLhaus dumps update every 5 minutes (fetched hourly to stay within rate limits)
- Feed data stored in `phishguard-feed-cache` Cloud Storage bucket
## SaaS Scaling Path
| Users | Infrastructure Cost | Monthly Revenue | Notes |
|-------|-------------------|-----------------|-------|
| 100 | $0 (free tier) | 100/mo | Validation phase |
| 1,000 | $0 (free tier) | 1K/mo | First operator hire feasible |
| 10,000 | ~$15/mo | 10K/mo | Full-time operation, 99.7% gross margin |
| 100,000 | ~$200/mo | 100K/mo | Enterprise contracts, multi-region |
| 1,000,000 | ~$5K/mo | 1M/mo | Platform status |
### Key Scaling Changes
- At 3,300+ users: exceed GCP free tier (2M invocations/month), begin paying per invocation
- At 10K+ users: add regional redundancy, implement response caching
- At 100K+ users: dedicated instances with min-instance count, multi-region deploymentWhy include the scaling path?
The scaling path connects infrastructure decisions to business outcomes. It shows stakeholders exactly when costs change and proves the unit economics work at every growth stage. The GCP free tier covers up to approximately 3,300 users before any infrastructure cost kicks in.
- Save runbook.md.
- Confirm the runbook covers all five sections: deploy commands, teardown, environment promotion, feed-refresh schedule, and SaaS scaling path.
✔️ Awesome, I've got everything!
Great. Double check you've saved runbook.md in your docs folder.
ⓧ I'd like to double check the full code
Here is the complete runbook.md file:
# PhishGuard Operational Runbook
## Deploy Commands
```bash
cd infra
terraform init
terraform plan
terraform apply
```
Confirm deployment:
```bash
gcloud functions describe phishguard-detect --gen2
```
## Teardown
```bash
cd infra
terraform destroy
```
Confirm removal:
```bash
gcloud functions list
```
The function should no longer appear in the output.
## Environment Promotion (Dev to Prod)
1. Update `terraform.tfvars` to set `environment = "prod"`
2. Run `terraform plan` to review production changes
3. Run `terraform apply` to deploy to production
4. Run evaluation suite against the prod endpoint to confirm SLO compliance
## Feed Refresh Schedule
- Cloud Scheduler triggers feed cache refresh every hour
- OpenPhish feed updates every 12 hours
- URLhaus dumps update every 5 minutes (fetched hourly to stay within rate limits)
- Feed data stored in `phishguard-feed-cache` Cloud Storage bucket
## SaaS Scaling Path
| Users | Infrastructure Cost | Monthly Revenue | Notes |
|-------|-------------------|-----------------|-------|
| 100 | $0 (free tier) | 100/mo | Validation phase |
| 1,000 | $0 (free tier) | 1K/mo | First operator hire feasible |
| 10,000 | ~$15/mo | 10K/mo | Full-time operation, 99.7% gross margin |
| 100,000 | ~$200/mo | 100K/mo | Enterprise contracts, multi-region |
| 1,000,000 | ~$5K/mo | 1M/mo | Platform status |
### Key Scaling Changes
- At 3,300+ users: exceed GCP free tier (2M invocations/month), begin paying per invocation
- At 10K+ users: add regional redundancy, implement response caching
- At 100K+ users: dedicated instances with min-instance count, multi-region deploymentYour detection API is live, validated against SLO floors, and documented with a complete operational runbook. Next, you'll compile the post-artifact bundle and build a stakeholder presentation that shows what PhishGuard looks like at 10K, 100K, and 1M users.
Build Post-Artifacts and Stakeholder Presentation
Your detection system is deployed, passing SLO floors, and responding within latency targets. Now you need to prove it.
Post-artifacts transform a working prototype into a credible product. In this step, you'll compile evidence from your test results, document system risks honestly, and build a stakeholder presentation backed by real numbers from the FTC and APWG.
In this step, get ready to:
- Compile the post-artifact bundle (evaluation report, latency report, threat model, model card).
- Document user-scale projections from 100 to 1M users with infrastructure costs and revenue.
- Prepare a 5-slide stakeholder presentation with sourced impact numbers and honest limitations.
Compile the post-artifact bundle
Post-artifacts are the engineering evidence that proves your system works as claimed. Each document serves a different audience: the evaluation report proves SLO compliance, the latency report proves real-time performance, the threat model shows you understand failure modes, and the model card describes what the scorer can and cannot do.
- Create the evaluation report by creating a new file docs/evaluation_report.md with the following content:
# Evaluation Report
## Summary
- **System**: PhishGuard Detection API (phishguard-detect)
- **Dataset**: 20 labeled samples (10 phishing, 5 malware, 5 benign)
- **Feeds**: PhishTank (Cisco Talos) + URLhaus (abuse.ch/Spamhaus)
- **Date**: [Your evaluation run date]
## Results
| Metric | Value | SLO Target | Status |
|--------|-------|------------|--------|
| Precision | [Your value] | >= 0.8 | [PASS/FAIL] |
| Recall | [Your value] | >= 0.8 | [PASS/FAIL] |
| F1 Score | [Your value] | - | - |
## Confusion Matrix
- True Positives: [count]
- False Positives: [count]
- True Negatives: [count]
- False Negatives: [count]
## Analysis
- Fast-path resolved [X] of 15 malicious samples instantly from feed cache.
- Heuristic scorer handled novel samples not in feed databases.
- Feed coverage: 7.6M+ tracked malicious indicators across both feeds.
## Conclusion
System meets SLO floors. Dataset is small (20 samples) but validates
end-to-end pipeline correctness. Production requires continuous
evaluation with adversarial examples.What does this report contain?
The evaluation report captures precision, recall, and F1 from your eval/evaluate.py run. The pass/fail column compares each metric against the SLO floors defined in docs/sli_slo.md (precision >= 0.8, recall >= 0.8).
Fill in the bracketed values with your actual results from the evaluation you ran in the previous step.
- Create the latency report by creating a new file docs/latency_report.md with the following content:
# Latency Report
## Summary
- **Method**: 10 sequential requests to live endpoint
- **Target**: p95 < 500ms
- **Endpoint**: phishguard-detect (GCP Cloud Functions 2nd gen)
- **Date**: [Your latency test date]
## Results
| Percentile | Latency (ms) | Target | Status |
|------------|-------------|--------|--------|
| p50 | [Your value] | - | - |
| p95 | [Your value] | < 500ms | [PASS/FAIL] |
| p99 | [Your value] | - | - |
## Notes
- First request includes cold start (~200ms for Python on GCP 2nd gen).
- Subsequent requests benefit from warm instance reuse.
- Fast-path lookups (feed cache hits) resolve significantly faster than
heuristic scorer invocations.
## Conclusion
System meets latency SLO. The fast-path-before-model pattern keeps
known-indicator lookups in the low milliseconds. Scorer-dependent
verdicts stay well under the 500ms target.Where do these numbers come from?
These are your actual results from running eval/latency.py against the live endpoint in the previous step. Replace each bracketed placeholder with the real values from your test output.
- Create the threat model by creating a new file docs/threat_model.md with the following content:
# Threat Model
## Overview
This document identifies threats to the PhishGuard detection system
and documents mitigations for each.
## Threat 1: False Positives Blocking Legitimate Messages
**Risk**: Heuristic scorer flags legitimate URLs as SUSPICIOUS or
MALICIOUS, causing users to distrust the system or block valid
communications.
**Likelihood**: Medium. URL entropy and brand-name distance checks
can trigger on legitimate short URLs or new brand domains.
**Mitigation**:
- Human-in-the-loop gate before any automated action.
- Confidence threshold: only MALICIOUS verdicts with confidence > 0.9
trigger recommended blocking actions.
- Feedback loop: false positive reports trigger scorer reweighting.
- SLO floor (precision >= 0.8) caps false positive rate at 20%.
## Threat 2: Feed Outages
**Risk**: PhishTank or URLhaus feeds become unavailable. Fast-path
lookups return no results, forcing all traffic through the scorer.
**Likelihood**: Low-Medium. Both feeds have had historical outages.
**Mitigation**:
- Local feed cache (GCP Cloud Storage bucket) retains last-known-good
data even if the upstream feed is unreachable.
- Scheduler job retries on failure.
- System degrades gracefully: scorer still produces verdicts without
feed enrichment, just with lower confidence.
- Monitoring: alert if feed cache age exceeds 24 hours.
## Threat 3: Model Drift
**Risk**: Phishing tactics evolve. The heuristic scorer's feature
weights become stale as attackers adapt (new TLDs, new impersonation
techniques, AI-generated phishing content).
**Likelihood**: High over 90+ days without retraining.
**Mitigation**:
- Model gateway interface allows hot-swapping the scorer without
changing ingestion or verdict layers.
- Continuous evaluation against fresh labeled samples.
- Feed updates (hourly for PhishTank, every 5 minutes for URLhaus)
keep the fast-path current even if the scorer drifts.
- Adversarial feedback loop: confirmed detections feed back as new
signal for scorer reweighting.
## Threat 4: Adversarial Evasion
**Risk**: Attackers deliberately craft URLs and messages to bypass
detection (homoglyph attacks, URL encoding, redirect chains,
legitimate-domain hosting).
**Likelihood**: High for targeted attacks against known detection
systems.
**Mitigation**:
- Multiple detection layers (feed lookup + heuristic scoring) require
evasion of both simultaneously.
- URL normalization before lookup (decode percent-encoding, resolve
redirects, canonicalize domains).
- Feature diversity in scorer makes single-vector evasion harder.
- Honest limitation: a sufficiently motivated attacker using novel
infrastructure not yet in any feed WILL evade detection until the
feed catches up. This is an inherent limitation of signature-based
and heuristic approaches.
## Residual Risk
This system shortens time-to-detection, not eliminates fraud. It does
not replace user education, bank fraud controls, or law enforcement.
The detection rate depends on feed freshness and scorer calibration.Why document what the system cannot do?
Honest threat modeling builds stakeholder trust. Acknowledging that a motivated attacker with novel infrastructure can evade detection shows engineering maturity. It also scopes expectations: this system reduces detection time from hours to sub-second, not eliminates fraud entirely.
- Create the model card by creating a new file docs/model_card.md with the following content:
# Model Card: PhishGuard Heuristic Scorer
## Model Description
The heuristic scorer is a weighted feature aggregation system behind
a swappable model gateway interface. It extracts features from suspect
URLs and messages, assigns weighted scores, and produces a
confidence-scored verdict.
## What It Does
- Extracts features: URL entropy, suspicious TLD detection, brand
impersonation via Levenshtein distance, urgency language patterns,
domain age heuristic.
- Produces weighted aggregate score mapped to verdict levels
(MALICIOUS / SUSPICIOUS / CLEAN).
- Returns confidence value (0.0 - 1.0) with each verdict.
- Operates behind model gateway interface for future hot-swap to
ML classifiers.
## What It Does NOT Do
- Does not use machine learning or neural networks (pure heuristic).
- Does not analyze email headers or SMTP metadata.
- Does not perform dynamic analysis (sandbox execution, screenshot
comparison).
- Does not detect zero-day phishing on novel infrastructure with no
heuristic signals.
- Does not replace human judgment for final action decisions.
## Known Limitations
- Feature weights are static and manually tuned. They will drift as
attack patterns evolve.
- Levenshtein distance for brand impersonation has a limited brand
list. New brands require manual addition.
- Urgency language patterns are English-only in the MVP.
- Domain age heuristic requires DNS lookup which may add latency for
uncached domains.
## Evaluation Results
- Dataset: 20 labeled samples (10 phishing, 5 malware, 5 benign)
- Precision: [Your value] (SLO: >= 0.8)
- Recall: [Your value] (SLO: >= 0.8)
- F1: [Your value]
- See docs/evaluation_report.md for full breakdown.
## Intended Use
- Real-time scoring of suspect URLs and messages submitted via API.
- First-pass triage before human review.
- NOT intended as sole decision-maker for blocking communications.
## Ethical Considerations
- False positives can block legitimate messages. Human gate mitigates.
- System inspects message content only with user consent.
- No personal data stored beyond the detection event lifecycle.What is a model card?
A model card is a standardized document describing what an AI or scoring system does, what it does not do, and its known limitations. It builds transparency for anyone integrating with or evaluating the system. Fill in the evaluation results from your actual test run.
- Save all four files in your docs/ directory.
- Confirm they exist by running:
ls docs/You should see evaluation_report.md, latency_report.md, threat_model.md, and model_card.md alongside your existing pre-artifact files.
✔️ Awesome, I've got everything!
Great. Make sure you've filled in the bracketed placeholders in the evaluation and latency reports with your actual test results from Step 4.
ⓧ I'd like to double check the full code
Your docs/ directory should now contain these files (in addition to pre-artifacts from Step 2):
- evaluation_report.md with precision, recall, F1, confusion matrix, and pass/fail against SLO.
- latency_report.md with p50/p95/p99 and pass/fail against 500ms target.
- threat_model.md with four threats (false positives, feed outages, model drift, adversarial evasion) and mitigations.
- model_card.md describing the heuristic scorer, what it does not do, limitations, and evaluation results.
Document user-scale projections
Stakeholders want to know what happens when real users start paying. This projection maps infrastructure cost against revenue at each order of magnitude, using GCP's free tier as the foundation.
- Create the user-scale projection by creating a new file docs/user_scale_projection.md with the following content:
# User-Scale Projection
## Pricing Model
1 per month per user (currency-agnostic, localized at point of sale).
Accessible by design. The mission is protection, not profit
maximization.
## Infrastructure: GCP Cloud Functions 2nd Gen
- Free tier: 2M invocations/month, 400,000 GB-seconds compute
- Each detection request = 1 invocation
- Estimated invocations per user per month: ~600 (20/day average)
## Projection
| Users | Monthly Invocations | Infra Cost | Revenue | Gross Margin |
|-------|--------------------:|------------|---------|-------------:|
| 100 | 60K | $0 (free tier) | $100/mo | 100% |
| 1,000 | 600K | $0 (free tier) | $1K/mo | 100% |
| 10,000 | 6M | ~$15/mo | $10K/mo | 99.85% |
| 100,000 | 60M | ~$200/mo | $100K/mo | 99.8% |
| 1,000,000 | 600M | ~$5K/mo | $1M/mo | 99.5% |
## Notes by Scale
### 100 Users
- Validates product-market fit.
- Zero infrastructure cost (well within 2M free invocations).
- Revenue covers domain and operational basics.
### 1,000 Users
- Still within free tier (600K << 2M invocations).
- $1K/mo revenue. First operator hire becomes feasible.
- Feed refresh costs remain zero (Cloud Scheduler: 3 free jobs).
### 10,000 Users
- Exceeds free tier. Overage cost ~$15/mo.
- $10K/mo revenue. Full-time operation viable.
- 99.85% gross margin.
### 100,000 Users
- ~$200/mo infrastructure (compute + egress + storage).
- $100K/mo revenue. Team of 3-5. Enterprise contracts begin.
- Multi-region deployment advisable for latency.
### 1,000,000 Users
- ~$5K/mo infrastructure.
- $1M/mo revenue. Platform status. Acquirable asset.
- Federation required for data residency (GDPR, DPDPA).
- Adversarial feedback loop is non-optional at this scale.
## Break-Even
50 users = break-even (covers domain, monitoring, operator time).
Infrastructure remains free until ~3,300 users.Why does infrastructure stay so cheap?
GCP Cloud Functions 2nd gen provides 2M free invocations per month. At an estimated 600 requests per user per month, that free tier supports approximately 3,300 users before any charges apply. The serverless model means you pay nothing when idle and pennies at scale.
- Save docs/user_scale_projection.md.
You won't see a visible change yet. The next substep references these projections in the stakeholder presentation.
Prepare the stakeholder presentation
The stakeholder presentation distills everything into five slides for a global company audience. Every number in this document is sourced from either the FTC, APWG, or your own test results.
- Create the stakeholder presentation by creating a new file docs/stakeholder_presentation.md with the following content:
# Stakeholder Presentation: PhishGuard Detection SaaS
---
## Slide 1: The Problem
**$15.9 billion** in reported fraud losses in 2025 (FTC, highest on
record). A 25% increase over 2024 and a 430% rise since 2020.
- 3 million fraud reports submitted to the FTC in 2025.
- 3.8 million phishing attacks observed globally (APWG, 2025).
- Older adults (60+) lost $2.4 billion in reported fraud.
- Median loss for those 80+: $1,650.
- Social media scam volume increased 51% to 843% quarter-over-quarter
(APWG/ZeroFox, 2025).
- True cost with underreporting: up to $195.9 billion annually
(FTC Congressional testimony, March 2026).
The problem is growing faster than current defenses can respond.
---
## Slide 2: The Solution
**PhishGuard**: Real-time scam detection SaaS.
- Sub-second detection against 7.6M+ tracked malicious indicators.
- Fast-path-before-model: known threats resolve in milliseconds from
feed cache. Only novel threats invoke the scorer.
- Two threat intelligence feeds: PhishTank (Cisco Talos, hourly
updates) + URLhaus (abuse.ch/Spamhaus, updates every 5 minutes).
- Heuristic scorer behind swappable model gateway (future ML drop-in).
- Structured verdicts: MALICIOUS / SUSPICIOUS / CLEAN with confidence,
signals, and recommended action.
- Pricing: 1/user/month. Accessible protection. Dataset-as-product.
---
## Slide 3: Architecture
```
[User/App] --> [PhishGuard API (Cloud Functions 2nd gen)]
|
+-------+-------+
| |
[Fast-Path Cache] [Heuristic Scorer]
(PhishTank + (Model Gateway)
URLhaus feeds) |
| [Verdict Formatter]
+-------+-------+
|
[Structured Response]
(verdict, confidence,
signals, recommended_action)
```
- Stateless per request (SaaS multi-tenant ready).
- Terraform-managed infrastructure (reproducible, teardown-clean).
- Scales to zero when idle (zero cost at rest).
---
## Slide 4: Evidence
| Metric | Result | Target | Status |
|--------|--------|--------|--------|
| Precision | [Your value] | >= 0.8 | [PASS] |
| Recall | [Your value] | >= 0.8 | [PASS] |
| Latency p95 | [Your value]ms | < 500ms | [PASS] |
- Feed coverage: PhishTank + URLhaus = 7.6M+ tracked indicators.
- Deployment: GCP Cloud Functions 2nd gen, Terraform-managed.
- Cost: $0/month at current scale (free tier).
- Dataset: 20 labeled samples (MVP validation). Production requires
continuous evaluation.
---
## Slide 5: Roadmap and Lessons Learned
### Roadmap
1. Takedown workflow: automated abuse report generation with human gate
2. Guardian dashboard: trusted reviewers confirm/reject detections
3. Adversarial hardening: feedback loop, scorer reweighting
4. Model retraining: ML classifier behind existing gateway interface
5. Path to 10K users: multi-region, enterprise API keys, SLA tiers
### Lessons Learned
**What worked**:
- Fast-path-before-model eliminates latency for 60-80% of malicious
indicators (feed cache hits).
- Gentik Systems multi-agent parallelization: 3 parallel Cursor
Composer agents reduced build time to 16 minutes.
- Design-before-build (pre-artifacts) prevented mid-build rework.
**What the operator would change**:
- Larger evaluation dataset from day one (20 samples validates the
pipeline but not edge cases).
- Automated feed-health monitoring before deployment, not after.
**What the system honestly cannot do**:
- Cannot detect phishing on novel infrastructure not yet in any feed.
- Cannot eliminate fraud. Shortens detection time, does not replace
user education or bank controls.
- Cannot force a hosting provider to act on an abuse report.
- Heuristic scorer will drift without adversarial feedback loop
(estimated 30-90 days before significant accuracy loss).Where do these numbers come from?
Every statistic in the Problem slide is sourced from official reports. The $15.9 billion and 3 million reports come from FTC Congressional testimony (March 2026). The 3.8 million phishing attacks come from the APWG Trends Report Q4 2025.
The Evidence slide uses your own test results. Replace the bracketed values with numbers from docs/evaluation_report.md and docs/latency_report.md.
- Save docs/stakeholder_presentation.md.
- Confirm all post-artifact files exist by running:
ls docs/You should see all post-artifact files: evaluation_report.md, latency_report.md, threat_model.md, model_card.md, user_scale_projection.md, and stakeholder_presentation.md alongside your pre-artifacts from Step 2.
Missing a file?
Double-check that you saved each file in the docs/ directory, not in the project root. All post-artifact files must live alongside your existing pre-artifacts (adr.md, c4_diagrams.md, etc.).
help me find where my post-artifact files ended up
✔️ Awesome, I've got everything!
Make sure you've replaced all bracketed placeholders with your actual evaluation and latency results. The stakeholder presentation should have real numbers in every evidence row.
ⓧ I'd like to double check the full code
Your docs/ directory should contain these post-artifact files:
- evaluation_report.md with precision/recall/F1 and pass/fail against SLO floors.
- latency_report.md with p50/p95/p99 and pass/fail against 500ms p95 target.
- threat_model.md covering false positives, feed outages, model drift, and adversarial evasion with mitigations for each.
- model_card.md describing the heuristic scorer, its capabilities, what it does not do, limitations, and evaluation results.
- user_scale_projection.md with projections at 100/1K/10K/100K/1M users including infra cost and revenue.
- stakeholder_presentation.md with 5 slides: Problem, Solution, Architecture, Evidence, Roadmap + Lessons Learned.
Your post-artifact bundle is complete. Every claim in the stakeholder presentation is backed by either official source data or your own test results. The system is ready to present to any audience, with honest limitations documented alongside the evidence.
Secret mission
Add Takedown Case Routing
Your detection system flags malicious infrastructure, but who acts on it? Right now PhishGuard detects and scores. In the real world, that's only half the job. The other half is routing a structured abuse report to the hosting provider so the malicious content can be taken down. Build a takedown-case routing layer that auto-generates abuse reports and gates them on human review before sending.
Clean Up Your Resources
Clean Up Your Resources
Decide whether to keep your resources running, pause them to come back later, or delete them entirely. All GCP resources in this project fall within the free tier at low volume, so there are no ongoing costs unless you exceed 2M invocations per month.
Resources you used:
- GCP Cloud Function (phishguard-detect) with takedown routing layer.
- GCP Cloud Storage bucket (phishguard-feed-cache) for threat feed data.
- GCP Cloud Scheduler job for hourly feed refresh.
- Local project files (infra/, src/, eval/, docs/, .env).
Keep everything running
No action needed. Choose this if you are still actively building or want to keep your live demo endpoint available.
- The Cloud Function scales to zero when idle. You pay nothing unless you exceed 2M invocations per month.
- The Cloud Storage bucket stays within the free tier at low volume.
- The Cloud Scheduler job uses one of your 3 free jobs per account.
- Your local project files remain on your machine.
Pause - I'll come back to this later
Stop the feed refresh to reduce background activity while keeping your function deployed and your files intact.
- Pause the Cloud Scheduler job by running this command (replace the job name with the one defined in your Terraform config):
gcloud scheduler jobs pause [[SCHEDULER_JOB_NAME="your Cloud Scheduler job name from Terraform"]]- The function remains deployed but will not receive fresh threat feed data until you resume the job.
- When you are ready to resume, run:
gcloud scheduler jobs resume [[SCHEDULER_JOB_NAME="your Cloud Scheduler job name from Terraform"]]Delete - I don't want to use this again
Remove all GCP resources and local project files. This cannot be undone.
- Navigate to your infra/ directory and destroy all GCP resources by running:
cd infra
terraform destroy- Terraform will show a plan and ask for confirmation. Type yes to proceed.
- Confirm the function is removed by running:
gcloud functions list- You should see an empty list or no entry for phishguard-detect.
- Delete your local project files. On macOS or Linux:
cd ~/Desktop
rm -rf [[PROJECT_FOLDER="your project folder name"]]- On Windows (PowerShell):
cd ~\Desktop
Remove-Item -Recurse -Force [[PROJECT_FOLDER="your project folder name"]]- Your .env file containing your PhishTank and URLhaus keys is removed along with the project folder.
- Those keys remain active on the provider side. You can revoke them from your PhishTank and abuse.ch account settings if you no longer need them.
That's a Wrap!
That's a Wrap!
Nice work! You just built and deployed a production-grade real-time scam detection SaaS that addresses $15.9 billion in annual reported fraud losses.
You've learned how to:
- Deploy a real-time phishing detection API that scores suspect messages and URLs against 7.6M+ tracked malicious indicators, using the fast-path-before-model pattern to resolve known-bad indicators in milliseconds from a local feed cache.
- Architect a complete system using design-before-build methodology with pre-artifacts (ADR, C4 diagrams, OpenAPI contract, data model) and post-artifacts (evaluation report, latency report, threat model, stakeholder presentation) backing every claim with evidence.
- Deploy reproducible infrastructure via Terraform on GCP Cloud Functions free tier with a validated SaaS foundation, measured SLI/SLO compliance, and a clear scaling path from zero to 1M users at $1/user.
- Secret Mission: Built a takedown-case routing layer that generates structured abuse reports in RFC 2142 format and gates on human review, reducing time-to-takedown-request from days to minutes.
Ready to quiz yourself?