Build a Shadow API Detection CLI

Discover undocumented API endpoints by analyzing JavaScript with a headless browser.

Introduction

30 Second Summary

Every web application loads JavaScript that quietly reveals where its APIs live. Some of those endpoints are undocumented, unprotected, and invisible to the security team. Attackers find them by reading the code your browser downloads.

In this project, you will build ShadowHawk, a Python CLI that launches a headless browser, captures all the JavaScript a page loads, extracts API endpoints hidden in that code, and probes them for security misconfigurations.

What You'll Build

A command-line security tool that automates shadow API discovery and vulnerability detection against a local test target.

By the end of this project, you'll have:

  • A working CLI tool that discovers undocumented API endpoints from JavaScript loaded by a real browser.
  • Automated security probes that detect CORS misconfigurations and missing authentication on discovered endpoints.
  • A scope guard and structured JSON report output suitable for CI/CD integration.
  • Secret Mission: Add reflected-Origin CORS detection that catches the more dangerous pattern where servers echo back any Origin header.

Are there any prerequisites?

You should be comfortable with Python basics like functions, classes, and async syntax. A basic understanding of HTTP requests and APIs will also help you follow along.

No prior security tooling experience is needed. The project introduces each concept as you build it.

Before We Start

ShadowHawk works as a pipeline. It captures JavaScript from a live web page, extracts API endpoints from that JavaScript, then probes those endpoints for security issues.

Before you can build any of those stages, you need a target to test against and a properly structured Python project. In this step, you will launch a vulnerable web application in Docker and scaffold your project with all the dependencies it needs.

In this step, get ready to:
  • Start OWASP Juice Shop as your authorized test target.
  • Initialize a Python project with uv and install dependencies.
  • Create the package directory structure with placeholder modules.
Start OWASP Juice Shop as your authorized test target

OWASP Juice Shop is a deliberately vulnerable web application designed for security testing. It is the target ShadowHawk will scan throughout this project.

  • Open your terminal. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Pull the Juice Shop Docker image by running this command:
docker pull bkimminich/juice-shop

What is OWASP Juice Shop?

OWASP Juice Shop is a free, open-source application maintained by the Open Web Application Security Project. It contains dozens of intentional security flaws, including CORS misconfigurations and exposed API endpoints.

Security professionals use it as a safe, legal target for testing tools. Running it locally means you are only scanning your own machine.

  • Start the Juice Shop container in detached mode by running:
docker run -d --rm -p 127.0.0.1:3000:3000 bkimminich/juice-shop

What does this command do?

  • -d runs the container in the background so you can keep using this terminal.
  • --rm automatically removes the container when you stop it.
  • -p 127.0.0.1:3000:3000 maps port 3000 on your machine to port 3000 inside the container.
  • Wait about 15-30 seconds for Juice Shop to start up, then verify it is running by opening http://localhost:3000 in your browser.

You should see the Juice Shop storefront page with products displayed.

Page not loading?

  • Make sure Docker (via Colima) is running. Run colima status to check.
  • The container can take up to 30 seconds to start serving. Try refreshing after a short wait.
  • Run docker ps to confirm the container is listed as running.

Help me debug why Juice Shop is not loading at localhost:3000.

Initialize the Python project and install dependencies

With your test target running, you can now set up the ShadowHawk project itself. You will use uv to create a new Python project and install the four libraries the pipeline needs.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create a new project called shadowhawk and move into it by running these commands:
uv init shadowhawk
cd shadowhawk

What did uv init create?

The uv init command created a shadowhawk directory with a default pyproject.toml, a main.py sample file, a README.md, and a .python-version file. You will replace the default pyproject.toml with a custom configuration next.

  • Open the project in your editor. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Cursor and press Enter to open it.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Select the shadowhawk folder on your Desktop, then click Open.
  • Open pyproject.toml from the Explorer sidebar.
  • Replace the entire contents of pyproject.toml with the following:
[project]
name = "shadowhawk"
version = "0.1.0"
description = "Shadow API detection CLI - discovers undocumented endpoints and probes for misconfigurations"
requires-python = ">=3.11"
dependencies = [
    "click==8.4.1",
    "playwright==1.60.0",
    "httpx==0.28.1",
    "pydantic==2.13.4",
]

[project.scripts]
shadowhawk = "shadowhawk.cli:main"

[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"

[tool.setuptools.packages.find]
where = ["src"]

What does this configuration do?

  • dependencies lists the four libraries: Click (CLI framework), Playwright (browser automation), httpx (HTTP client), and Pydantic (data validation and JSON serialization).
  • [project.scripts] registers a shadowhawk command that maps to the main function in shadowhawk.cli.
  • [tool.setuptools.packages.find] tells setuptools to look for packages inside the src/ directory.
  • Save pyproject.toml.
  • Delete the main.py file that uv init created (you will not need it). Run this in the terminal:
rm main.py
  • Install all the dependencies by running:
uv sync

You should see uv resolving and installing the four packages along with their sub-dependencies.

  • Download the Chromium browser binary that Playwright needs by running:
uv run playwright install chromium

This downloads a Chromium binary that Playwright will control for headless browsing. It may take a minute depending on your connection speed.

Seeing a dependency resolution error?

  • Make sure you saved pyproject.toml before running uv sync.
  • Check that your requires-python field says >=3.11. A typo here can prevent resolution.
  • Confirm your Python version with uv run python --version. It should show 3.12.13 or higher.

Help me fix a uv sync error in my ShadowHawk project.

Create the package directory structure

ShadowHawk uses a src layout, which keeps source code separate from project metadata. Each module in the package will hold one stage of the pipeline.

  • Create the src/shadowhawk/ package directory by running:
mkdir -p src/shadowhawk
  • Create the __init__.py file that marks this directory as a Python package. In Cursor, right-click the src/shadowhawk folder in the Explorer sidebar.
  • Click New File.
  • Name it __init__.py and paste the following content:
"""ShadowHawk - Shadow API Detection CLI."""

__version__ = "0.1.0"

What does __init__.py do?

This file tells Python that the shadowhawk directory is a package that can be imported. The __version__ variable gives the tool a version number that the CLI will display later.

  • Create the remaining placeholder modules. Run these commands in your terminal:
touch src/shadowhawk/cli.py
touch src/shadowhawk/scope.py
touch src/shadowhawk/capture.py
touch src/shadowhawk/extract.py
touch src/shadowhawk/probe.py
touch src/shadowhawk/models.py
  • Open each file in Cursor and add # placeholder as the only content. This ensures each module is a valid Python file.
  • Save all files.
  • Re-run uv sync so the build system picks up the new package directory:
uv sync
  • Confirm the files exist by running:
ls src/shadowhawk/

You should see __init__.py, cli.py, scope.py, capture.py, extract.py, probe.py, and models.py in the output.

  • Verify the package is installed correctly by running:
uv run python -c "import shadowhawk; print(shadowhawk.__version__)"

You should see 0.1.0 printed in the terminal. This confirms that uv installed your package and Python can import it.

Seeing a ModuleNotFoundError?

  • Make sure your src/shadowhawk/__init__.py file exists and is not empty.
  • Run uv sync again after creating the src/shadowhawk/ directory. The build system needs the package directory to exist before it can install it.
  • Check that [tool.setuptools.packages.find] in your pyproject.toml has where = ["src"].

Help me fix a ModuleNotFoundError when importing shadowhawk.

✔️ Awesome, I've got everything!

Great. Double check you have saved all your files before moving on.

ⓧ I'd like to double check the full code

Here is the complete pyproject.toml:

[project]
name = "shadowhawk"
version = "0.1.0"
description = "Shadow API detection CLI - discovers undocumented endpoints and probes for misconfigurations"
requires-python = ">=3.11"
dependencies = [
    "click==8.4.1",
    "playwright==1.60.0",
    "httpx==0.28.1",
    "pydantic==2.13.4",
]

[project.scripts]
shadowhawk = "shadowhawk.cli:main"

[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"

[tool.setuptools.packages.find]
where = ["src"]

Here is src/shadowhawk/__init__.py:

"""ShadowHawk - Shadow API Detection CLI."""

__version__ = "0.1.0"

All other files in src/shadowhawk/ (cli.py, scope.py, capture.py, extract.py, probe.py, models.py) should each contain only:

# placeholder

Your project is scaffolded and your test target is running. Next up, you will build the scope guard and data models that keep ShadowHawk safe and structured.

Build the Scope Guard and Data Models

Your project scaffold is in place and Juice Shop is running. Now it's time to build the safety layer that ensures ShadowHawk never touches an unauthorized target.

A security tool without scope enforcement is a liability. In this step, you'll build a scope guard that acts as a gatekeeper, define the data models that flow through your pipeline, and wire both into a working CLI skeleton.

In this step, get ready to:
  • Implement a scope guard that rejects unauthorized targets.
  • Define Pydantic data models for endpoints, findings, and reports.
  • Wire scope validation into the CLI and make your first Git commit.
Build the scope guard

The scope guard is ShadowHawk's first line of defense. It maintains an allowlist of hostnames that the tool is authorized to scan, and raises an error if anything else is attempted.

  • In Cursor, open src/shadowhawk/scope.py from the Explorer sidebar.
  • Replace the placeholder comment with the following scope guard implementation:
"""Scope guard - ensures ShadowHawk only touches authorized targets."""

from urllib.parse import urlparse


class ScopeError(Exception):
    """Raised when a target is outside the authorized scope."""

    pass


DEFAULT_ALLOWLIST = {"localhost", "127.0.0.1"}


def validate_target(url: str, allowlist: set[str] | None = None) -> str:
    """Validate that a URL is within the authorized scope.

    Returns the parsed hostname if valid, raises ScopeError otherwise.
    """
    if allowlist is None:
        allowlist = DEFAULT_ALLOWLIST.copy()

    parsed = urlparse(url)
    hostname = parsed.hostname

    if not hostname:
        raise ScopeError(f"Could not parse hostname from URL: {url}")

    if hostname not in allowlist:
        raise ScopeError(
            f"Target '{hostname}' is NOT in the authorized scope.\n"
            f"Allowed targets: {', '.join(sorted(allowlist))}\n"
            f"ShadowHawk will not send any requests to unauthorized targets."
        )

    return hostname

What does this code do?

ScopeError is a custom exception that signals an out-of-scope target. The DEFAULT_ALLOWLIST set only includes localhost and 127.0.0.1 by default. This means ShadowHawk will refuse to send requests anywhere except your local machine.

The validate_target() function parses the URL, extracts the hostname, and checks it against the allowlist. If the hostname is missing or not authorized, it raises a ScopeError with a clear message explaining why the scan was blocked.

Why a scope guard as the very first module?

Professional security tools can cause real damage if pointed at the wrong target. Building the safety mechanism first ensures that no subsequent code can accidentally send requests to unauthorized hosts. Every pipeline stage will pass through this gate before touching the network.

  • Save scope.py.
  • Verify the scope guard rejects unauthorized targets by running this command in your terminal:
uv run python -c "from shadowhawk.scope import validate_target; validate_target('https://example.com')"

You should see a traceback ending with ScopeError and the message explaining that example.com is NOT in the authorized scope.

Seeing ModuleNotFoundError?

Make sure you're running the command from the root of your shadowhawk project folder (the one containing pyproject.toml). The uv run command needs to be in the project root to find the package.

Also check that your file is saved at exactly src/shadowhawk/scope.py. The directory structure must match what pyproject.toml expects.

Help me debug my ShadowHawk scope guard import error

Define the pipeline data models

Every stage of ShadowHawk's pipeline passes structured records to the next. Pydantic models define the shape of these records and provide built-in JSON serialization for the final report.

  • Open src/shadowhawk/models.py in Cursor.
  • Replace the placeholder with the imports and the two core pipeline models:
"""Data models for the ShadowHawk pipeline."""

from datetime import datetime, timezone
from pydantic import BaseModel, Field


class Endpoint(BaseModel):
    """A discovered API endpoint extracted from JavaScript."""

    path: str
    source_url: str
    evidence_snippet: str
    line_number: int | None = None


class Finding(BaseModel):
    """A security finding produced by a probe."""

    finding_type: str
    endpoint: str
    severity: str = "medium"
    evidence: str
    timestamp: str = Field(
        default_factory=lambda: datetime.now(timezone.utc).isoformat()
    )

What do these models represent?

Endpoint holds a discovered API path along with its source (which JS file it came from), a code snippet showing the match, and the line number. The extractor stage produces these.

Finding captures a security issue found during probing. The timestamp field uses a default_factory to auto-populate with the current UTC time when a finding is created.

  • Add the container models below the Finding class in the same file:
class CapturedScript(BaseModel):
    """A JavaScript asset captured from the target."""

    url: str
    content: str
    content_hash: str


class ScanReport(BaseModel):
    """The final scan report."""

    target: str
    scan_timestamp: str = Field(
        default_factory=lambda: datetime.now(timezone.utc).isoformat()
    )
    total_scripts_captured: int = 0
    total_endpoints_discovered: int = 0
    total_findings: int = 0
    endpoints: list[Endpoint] = Field(default_factory=list)
    findings: list[Finding] = Field(default_factory=list)

What do these models represent?

CapturedScript stores a JavaScript asset grabbed by the browser, including a SHA-256 hash for deduplication. The capture engine produces these.

ScanReport is the top-level container that holds everything from a complete scan. This is what gets serialized to shadowhawk-report.json at the end of a run.

  • Save models.py.
  • Verify the models load correctly by running this in your terminal:
uv run python -c "from shadowhawk.models import ScanReport; print(ScanReport(target='http://localhost:3000').model_dump_json(indent=2))"

You should see a JSON object with the target URL, a timestamp, and zeroed-out counters. This confirms Pydantic is parsing and serializing correctly.

Seeing a validation error?

Check that your ScanReport class has target: str as a required field (no default). Pydantic will raise a validation error if a required field is missing from the constructor call.

Help me debug my Pydantic model import or validation error

✔️ Awesome, I've got everything!

Great. Make sure you've saved models.py before moving on.

ⓧ I'd like to double check the full code

"""Data models for the ShadowHawk pipeline."""

from datetime import datetime, timezone
from pydantic import BaseModel, Field


class Endpoint(BaseModel):
    """A discovered API endpoint extracted from JavaScript."""

    path: str
    source_url: str
    evidence_snippet: str
    line_number: int | None = None


class Finding(BaseModel):
    """A security finding produced by a probe."""

    finding_type: str
    endpoint: str
    severity: str = "medium"
    evidence: str
    timestamp: str = Field(
        default_factory=lambda: datetime.now(timezone.utc).isoformat()
    )


class CapturedScript(BaseModel):
    """A JavaScript asset captured from the target."""

    url: str
    content: str
    content_hash: str


class ScanReport(BaseModel):
    """The final scan report."""

    target: str
    scan_timestamp: str = Field(
        default_factory=lambda: datetime.now(timezone.utc).isoformat()
    )
    total_scripts_captured: int = 0
    total_endpoints_discovered: int = 0
    total_findings: int = 0
    endpoints: list[Endpoint] = Field(default_factory=list)
    findings: list[Finding] = Field(default_factory=list)
Wire the scope guard into the CLI

The scope guard module exists, but nothing calls it yet. You need to connect it to the Click-based CLI so that every command validates the target before proceeding.

  • Open src/shadowhawk/cli.py in Cursor.
  • Replace the placeholder with the CLI implementation that uses your scope guard:
"""ShadowHawk CLI - the main entry point."""

import sys

import click

from shadowhawk import __version__
from shadowhawk.scope import ScopeError, validate_target


@click.group()
@click.version_option(version=__version__, prog_name="shadowhawk")
def main():
    """ShadowHawk - Shadow API Detection CLI."""
    pass


@main.command()
@click.option("--target", required=True, help="Target URL to scan (must be in scope).")
def scan(target: str):
    """Run a full scan (stub)."""
    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)
    click.echo(f"[*] Target validated: {target}")


@main.command()
@click.option("--target", required=True, help="Target URL to capture JS from.")
def capture(target: str):
    """Capture JavaScript from a target (stub)."""
    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)
    click.echo(f"[*] Target validated: {target}")

What does this code do?

@click.group() makes main() a command group that hosts subcommands. The scan and capture functions are registered as subcommands via @main.command().

Both commands follow the same pattern: call validate_target() first, catch a ScopeError if the target is unauthorized, and exit with a non-zero code. If validation passes, the command proceeds. Right now they're stubs that just confirm the target is valid.

  • Save cli.py.
  • Test that the CLI rejects an unauthorized target by running:
uv run shadowhawk scan --target https://example.com

You should see a SCOPE VIOLATION message explaining that example.com is not in the authorized scope.

  • Now confirm that a valid localhost target passes by running:
uv run shadowhawk scan --target http://localhost:3000

You should see [*] Target validated: http://localhost:3000 printed to the terminal.

Seeing 'No such command' for scan?

Check that your pyproject.toml entry point is shadowhawk = "shadowhawk.cli:main" under [project.scripts]. The main function must be decorated with @click.group() for subcommands to register.

Help me debug why my shadowhawk CLI subcommand is not found

✔️ Awesome, I've got everything!

Great. Make sure all three files are saved before committing.

ⓧ I'd like to double check the full code

"""ShadowHawk CLI - the main entry point."""

import sys

import click

from shadowhawk import __version__
from shadowhawk.scope import ScopeError, validate_target


@click.group()
@click.version_option(version=__version__, prog_name="shadowhawk")
def main():
    """ShadowHawk - Shadow API Detection CLI."""
    pass


@main.command()
@click.option("--target", required=True, help="Target URL to scan (must be in scope).")
def scan(target: str):
    """Run a full scan (stub)."""
    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)
    click.echo(f"[*] Target validated: {target}")


@main.command()
@click.option("--target", required=True, help="Target URL to capture JS from.")
def capture(target: str):
    """Capture JavaScript from a target (stub)."""
    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)
    click.echo(f"[*] Target validated: {target}")
  • Make your first commit by running these commands:
git add -A && git commit -m "feat: scope guard and data models"

Why commit now?

Each commit marks a working milestone in your pipeline. This first commit locks in the safety layer and data contracts. If something breaks in a later step, you can always roll back to this known-good state.

Your safety layer is locked in and the data models are ready. Next up, you'll build the JavaScript capture engine that uses Playwright to intercept every script a page loads at runtime.

Capture JavaScript with Playwright

Your scope guard keeps ShadowHawk from scanning unauthorized targets. Now it needs eyes.

Modern web apps load most of their logic as external JavaScript bundles and inline scripts. These scripts often contain hardcoded API endpoint paths that never appear in any public documentation. To find those hidden endpoints, ShadowHawk needs to watch everything the browser downloads when it visits a page.

In this step, get ready to:
  • Build a Playwright-based capture engine that intercepts all JavaScript responses.
  • Capture inline script content from the page DOM after load.
  • Wire the capture engine into the CLI and test it against Juice Shop.
Implement the capture engine

When a browser loads a page, it fetches dozens of JavaScript files over the network. ShadowHawk needs to intercept every one of those responses, read the body, and store it for analysis.

Why Playwright instead of Selenium?

Playwright has built-in response interception via page.on("response") which lets you capture JavaScript assets as they stream in. This includes lazy-loaded chunks that only appear at runtime.

Selenium requires an external proxy setup for the same capability. Playwright also auto-manages browser binaries, so there is nothing extra to configure.

  • Open src/shadowhawk/capture.py in Cursor and replace the placeholder with the following capture engine:
"""JavaScript capture engine using Playwright."""

import hashlib

from playwright.sync_api import sync_playwright

from shadowhawk.models import CapturedScript


def capture_scripts(target_url: str, timeout_ms: int = 30000) -> list[CapturedScript]:
    """Launch a browser, navigate to target, and capture all JS assets."""
    scripts: dict[str, CapturedScript] = {}

    with sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        context = browser.new_context()
        page = context.new_page()

        # Intercept every HTTP response and keep the ones with JS content
        def handle_response(response):
            content_type = response.headers.get("content-type", "")
            if "javascript" in content_type:
                try:
                    body = response.text()
                    content_hash = hashlib.sha256(body.encode()).hexdigest()
                    if content_hash not in scripts:
                        scripts[content_hash] = CapturedScript(
                            url=response.url,
                            content=body,
                            content_hash=content_hash,
                        )
                except Exception:
                    pass

        page.on("response", handle_response)
        page.goto(target_url, wait_until="networkidle", timeout=timeout_ms)

        # Grab inline <script> tags that don't have a src attribute
        inline_scripts = page.eval_on_selector_all(
            "script:not([src])",
            "elements => elements.map(el => el.textContent || '')",
        )

        for script_content in inline_scripts:
            if script_content.strip():
                content_hash = hashlib.sha256(script_content.encode()).hexdigest()
                if content_hash not in scripts:
                    scripts[content_hash] = CapturedScript(
                        url=f"{target_url}#inline",
                        content=script_content,
                        content_hash=content_hash,
                    )

        browser.close()

    return list(scripts.values())

What does this code do?

  • sync_playwright() launches a headless Chromium browser controlled entirely by Python.
  • page.on("response", handle_response) registers a callback that fires for every HTTP response the browser receives. The callback filters by content-type and keeps only JavaScript.
  • Each script body is hashed with SHA-256. If two scripts produce the same hash, only one copy is stored. This deduplication prevents bloated results from CDN mirrors.
  • page.eval_on_selector_all("script:not([src])", ...) runs JavaScript inside the browser to extract all inline <script> content after the page finishes loading.
  • wait_until="networkidle" tells Playwright to wait until no new network requests have fired for 500ms, ensuring lazy-loaded bundles are captured.
  • Save capture.py.

Seeing an import error for playwright?

  • Confirm you installed the Playwright browsers earlier by running uv run playwright install chromium again. The Python package and the browser binary are separate installs.
  • Make sure you are running commands from your project root (the folder containing pyproject.toml).

Why can't Playwright find the browser binary?

Understand inline script capture

External scripts arrive over the network and are caught by the response handler. But many apps also embed JavaScript directly in their HTML as inline <script> tags.

These inline scripts never trigger a network response because they are already part of the HTML document. That is why capture.py uses a second pass after the page loads: eval_on_selector_all queries the DOM for every <script> tag without a src attribute and pulls out its text content.

Why deduplicate by hash?

Some frameworks inject the same script content both inline and as an external file. Hashing ensures ShadowHawk never processes the same code twice, keeping extraction results clean.

Wire up the CLI capture command and test

The capture engine is ready. Now you need to connect it to the shadowhawk capture CLI command so you can run it against Juice Shop.

  • Open src/shadowhawk/cli.py in Cursor.
  • Find this import line near the top of the file:
from shadowhawk.scope import ScopeError, validate_target
  • Add the capture import directly above it, so both import lines look like this:
from shadowhawk.capture import capture_scripts
from shadowhawk.scope import ScopeError, validate_target
  • Now find the existing capture command at the bottom of the file:
@main.command()
@click.option("--target", required=True, help="Target URL to capture JS from.")
def capture(target: str):
    """Capture JavaScript from a target (stub)."""
    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)
    click.echo(f"[*] Target validated: {target}")
  • Replace the entire capture command with this updated version that calls the capture engine and prints results:
@main.command()
@click.option("--target", required=True, help="Target URL to capture JS from.")
def capture(target: str):
    """Capture JavaScript from a target (discovery only, no probes)."""
    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)

    # Run the capture engine and display results
    scripts = capture_scripts(target)
    click.echo(f"Captured {len(scripts)} unique scripts:")
    total_size = 0
    for s in scripts:
        size = len(s.content)
        total_size += size
        click.echo(f"  {s.url} ({size:,} bytes)")
    click.echo(f"Total: {total_size:,} bytes")

What changed in the CLI?

  • The docstring changed from "stub" to its final description.
  • After scope validation passes, it calls capture_scripts(target) to launch the browser and collect JavaScript.
  • It prints a summary: one line per script (URL and size in bytes) plus a total.
  • Save cli.py.
  • Test the capture command against Juice Shop by running:
uv run shadowhawk capture --target http://localhost:3000

You should see output listing multiple JavaScript files with their URLs and sizes. Juice Shop typically loads several bundled scripts totaling hundreds of kilobytes.

Seeing zero scripts captured?

  • Confirm Juice Shop is still running by opening http://localhost:3000 in your browser. If it is not responding, restart it with docker run --rm -p 127.0.0.1:3000:3000 bkimminich/juice-shop.
  • If you see a timeout error, increase the timeout by adding a longer wait. The default 30 seconds is usually enough, but slow machines may need more.
  • Check that uv run playwright install chromium completed successfully. The capture engine cannot work without the Chromium binary.

Help me debug why no scripts are being captured from Juice Shop.

  • Commit your progress by running:
git add -A && git commit -m "feat: browser-driven JS capture"

✔️ Awesome, I've got everything!

Great. Double check you have saved both capture.py and cli.py before moving on.

ⓧ I'd like to double check the full code

Here is the full src/shadowhawk/capture.py:

"""JavaScript capture engine using Playwright."""

import hashlib

from playwright.sync_api import sync_playwright

from shadowhawk.models import CapturedScript


def capture_scripts(target_url: str, timeout_ms: int = 30000) -> list[CapturedScript]:
    """Launch a browser, navigate to target, and capture all JS assets."""
    scripts: dict[str, CapturedScript] = {}

    with sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        context = browser.new_context()
        page = context.new_page()

        def handle_response(response):
            content_type = response.headers.get("content-type", "")
            if "javascript" in content_type:
                try:
                    body = response.text()
                    content_hash = hashlib.sha256(body.encode()).hexdigest()
                    if content_hash not in scripts:
                        scripts[content_hash] = CapturedScript(
                            url=response.url,
                            content=body,
                            content_hash=content_hash,
                        )
                except Exception:
                    pass

        page.on("response", handle_response)
        page.goto(target_url, wait_until="networkidle", timeout=timeout_ms)

        inline_scripts = page.eval_on_selector_all(
            "script:not([src])",
            "elements => elements.map(el => el.textContent || '')",
        )

        for script_content in inline_scripts:
            if script_content.strip():
                content_hash = hashlib.sha256(script_content.encode()).hexdigest()
                if content_hash not in scripts:
                    scripts[content_hash] = CapturedScript(
                        url=f"{target_url}#inline",
                        content=script_content,
                        content_hash=content_hash,
                    )

        browser.close()

    return list(scripts.values())

And here is the full src/shadowhawk/cli.py:

"""ShadowHawk CLI - the main entry point."""

import sys

import click

from shadowhawk import __version__
from shadowhawk.capture import capture_scripts
from shadowhawk.scope import ScopeError, validate_target


@click.group()
@click.version_option(version=__version__, prog_name="shadowhawk")
def main():
    """ShadowHawk - Shadow API Detection CLI."""
    pass


@main.command()
@click.option("--target", required=True, help="Target URL to scan (must be in scope).")
def scan(target: str):
    """Run a full scan (stub)."""
    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)
    click.echo(f"[*] Target validated: {target}")


@main.command()
@click.option("--target", required=True, help="Target URL to capture JS from.")
def capture(target: str):
    """Capture JavaScript from a target (discovery only, no probes)."""
    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)

    scripts = capture_scripts(target)
    click.echo(f"Captured {len(scripts)} unique scripts:")
    total_size = 0
    for s in scripts:
        size = len(s.content)
        total_size += size
        click.echo(f"  {s.url} ({size:,} bytes)")
    click.echo(f"Total: {total_size:,} bytes")

Your capture engine is live and pulling JavaScript straight from the browser. Next up, you will parse all that captured code to extract the hidden API endpoints buried inside it.

Extract API Endpoints from JavaScript

Your capture engine grabs every JavaScript file that Juice Shop loads at runtime. The next challenge is finding the API endpoints buried inside all that code.

JavaScript bundles are full of fetch() calls, axios requests, and string literals referencing API paths. In this step, you will build a regex-powered extractor that scans every captured script and pulls out those paths automatically.

In this step, get ready to:
  • Define regex patterns that match common JavaScript API call signatures.
  • Build an extraction function that runs those patterns across all captured scripts.
  • Wire extraction into the scan command and test against Juice Shop.
Define regex patterns for API call signatures

JavaScript code uses several conventions to make HTTP requests. A fetch("/api/users") call, an axios.get("/rest/products") call, and an Angular $http.get("/api/items") call all embed an API path as a string literal. Your regex patterns will capture that path from each style.

  • Open src/shadowhawk/extract.py in Cursor and replace the placeholder comment with the following code:
"""Endpoint extraction from captured JavaScript."""

import re

from shadowhawk.models import Endpoint

ENDPOINT_PATTERNS = [
    re.compile(r"""(?:fetch|axios\.get|axios\.post|axios\.put|axios\.delete)\s*\(\s*['"`]([/][^'"`\s]+)['"`]"""),
    re.compile(r"""\$http\.(?:get|post|put|delete)\s*\(\s*['"`]([/][^'"`\s]+)['"`]"""),
    re.compile(r"""\.(?:get|post|put|delete|patch)\s*\(\s*['"`]([/][^'"`\s]+)['"`]"""),
    re.compile(r"""['"`](/(?:api|rest|v[0-9]+)/[^'"`\s]+)['"`]"""),
]

What do these patterns match?

  • Pattern 1 catches fetch(), axios.get(), axios.post(), axios.put(), and axios.delete() calls where the first argument is a path starting with /.
  • Pattern 2 catches Angular's $http service calls.
  • Pattern 3 is a broad catch-all for any method-chained .get(), .post(), etc. on HTTP client instances.
  • Pattern 4 catches raw string literals that look like API paths (/api/..., /rest/..., /v1/...) even when not inside a function call.
Build the extraction function

Each pattern captures a path. The extraction function runs every pattern against every line of every captured script, deduplicates by path, and collects 50 characters of surrounding context as evidence.

  • Add the following function below the ENDPOINT_PATTERNS list in extract.py:

def extract_endpoints(scripts: list) -> list[Endpoint]:
    """Extract API endpoints from captured scripts using regex patterns."""
    seen_paths: set[str] = set()
    endpoints: list[Endpoint] = []

    for script in scripts:
        lines = script.content.split("\n")
        for line_num, line in enumerate(lines, start=1):
            for pattern in ENDPOINT_PATTERNS:
                for match in pattern.finditer(line):
                    path = match.group(1)
                    if path in seen_paths:
                        continue
                    seen_paths.add(path)

                    start = max(0, match.start() - 50)
                    end = min(len(line), match.end() + 50)
                    snippet = line[start:end].strip()

                    endpoints.append(
                        Endpoint(
                            path=path,
                            source_url=script.url,
                            evidence_snippet=snippet,
                            line_number=line_num,
                        )
                    )

    return endpoints

What does this function do?

  • It splits each script's content into individual lines so it can report the line number where each endpoint was found.
  • For every regex match, it grabs 50 characters before and after the match as an evidence snippet. This gives context for where the endpoint appeared in the source.
  • The seen_paths set ensures each unique path appears only once in the results, even if it is referenced multiple times across different scripts.
  • Save extract.py.

Seeing a syntax error in extract.py?

  • Make sure the ENDPOINT_PATTERNS list ends with a closing ] and has a blank line before def extract_endpoints.
  • Each re.compile() line uses triple-quoted raw strings (r"""..."""). Make sure you have three quotes on each side, not two.
  • help me debug a syntax error in my extract.py regex patterns

✔️ Awesome, I've got everything!

Great. Make sure you have saved extract.py before moving on.

ⓧ I'd like to double check the full code

"""Endpoint extraction from captured JavaScript."""

import re

from shadowhawk.models import Endpoint

ENDPOINT_PATTERNS = [
    re.compile(r"""(?:fetch|axios\.get|axios\.post|axios\.put|axios\.delete)\s*\(\s*['"`]([/][^'"`\s]+)['"`]"""),
    re.compile(r"""\$http\.(?:get|post|put|delete)\s*\(\s*['"`]([/][^'"`\s]+)['"`]"""),
    re.compile(r"""\.(?:get|post|put|delete|patch)\s*\(\s*['"`]([/][^'"`\s]+)['"`]"""),
    re.compile(r"""['"`](/(?:api|rest|v[0-9]+)/[^'"`\s]+)['"`]"""),
]


def extract_endpoints(scripts: list) -> list[Endpoint]:
    """Extract API endpoints from captured scripts using regex patterns."""
    seen_paths: set[str] = set()
    endpoints: list[Endpoint] = []

    for script in scripts:
        lines = script.content.split("\n")
        for line_num, line in enumerate(lines, start=1):
            for pattern in ENDPOINT_PATTERNS:
                for match in pattern.finditer(line):
                    path = match.group(1)
                    if path in seen_paths:
                        continue
                    seen_paths.add(path)

                    start = max(0, match.start() - 50)
                    end = min(len(line), match.end() + 50)
                    snippet = line[start:end].strip()

                    endpoints.append(
                        Endpoint(
                            path=path,
                            source_url=script.url,
                            evidence_snippet=snippet,
                            line_number=line_num,
                        )
                    )

    return endpoints
Wire extraction into the scan command

The scan command currently only validates the target. Now you will update it to run the full capture-then-extract pipeline and print discovered endpoints.

  • Open src/shadowhawk/cli.py in Cursor.
  • Find this import line near the top:
from shadowhawk.capture import capture_scripts
from shadowhawk.scope import ScopeError, validate_target
  • Add the extract_endpoints import so the imports look like this:
from shadowhawk.capture import capture_scripts
from shadowhawk.extract import extract_endpoints
from shadowhawk.scope import ScopeError, validate_target
  • Now find the existing scan command function and replace it entirely with the following:
@main.command()
@click.option("--target", required=True, help="Target URL to scan (must be in scope).")
def scan(target: str):
    """Run a full scan: capture JS, extract endpoints."""
    click.echo(f"[*] ShadowHawk v{__version__}")
    click.echo(f"[*] Target: {target}")
    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)

    click.echo("[*] Stage 1: Capturing JavaScript...")
    scripts = capture_scripts(target)
    click.echo(f"    Found {len(scripts)} unique scripts")

    click.echo("[*] Stage 2: Extracting endpoints...")
    endpoints = extract_endpoints(scripts)
    click.echo(f"    Discovered {len(endpoints)} unique endpoints")
    for ep in endpoints:
        click.echo(f"      {ep.path}  (from {ep.source_url})")

What does the updated scan command do?

  • Stage 1 calls capture_scripts() to launch a headless browser and collect all JavaScript from the target.
  • Stage 2 feeds those scripts into extract_endpoints() and prints each discovered path alongside its source URL.
  • The scope guard still runs first. If the target is not on the allowlist, the scan aborts before any network activity.
  • Save cli.py.

✔️ Awesome, I've got everything!

Great. Make sure you have saved cli.py before testing.

ⓧ I'd like to double check the full code

"""ShadowHawk CLI - the main entry point."""

import sys

import click

from shadowhawk import __version__
from shadowhawk.capture import capture_scripts
from shadowhawk.extract import extract_endpoints
from shadowhawk.scope import ScopeError, validate_target


@click.group()
@click.version_option(version=__version__, prog_name="shadowhawk")
def main():
    """ShadowHawk - Shadow API Detection CLI."""
    pass


@main.command()
@click.option("--target", required=True, help="Target URL to scan (must be in scope).")
def scan(target: str):
    """Run a full scan: capture JS, extract endpoints."""
    click.echo(f"[*] ShadowHawk v{__version__}")
    click.echo(f"[*] Target: {target}")
    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)

    click.echo("[*] Stage 1: Capturing JavaScript...")
    scripts = capture_scripts(target)
    click.echo(f"    Found {len(scripts)} unique scripts")

    click.echo("[*] Stage 2: Extracting endpoints...")
    endpoints = extract_endpoints(scripts)
    click.echo(f"    Discovered {len(endpoints)} unique endpoints")
    for ep in endpoints:
        click.echo(f"      {ep.path}  (from {ep.source_url})")


@main.command()
@click.option("--target", required=True, help="Target URL to capture JS from.")
def capture(target: str):
    """Capture JavaScript from a target (discovery only, no probes)."""
    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)

    scripts = capture_scripts(target)
    click.echo(f"Captured {len(scripts)} unique scripts:")
    total_size = 0
    for s in scripts:
        size = len(s.content)
        total_size += size
        click.echo(f"  {s.url} ({size:,} bytes)")
    click.echo(f"Total: {total_size:,} bytes")
  • Test the scan command against Juice Shop by running:
uv run shadowhawk scan --target http://localhost:3000

You should see ShadowHawk launch a headless browser, capture JavaScript bundles, then print a list of discovered endpoints. Look for paths like /rest/products, /api/Feedbacks, and /rest/user/login. These are undocumented API endpoints that do not appear in any official Juice Shop documentation.

Not seeing any endpoints?

  • Make sure Juice Shop is still running. Open http://localhost:3000 in your browser to confirm.
  • If the scan finds scripts but zero endpoints, double-check that your regex patterns use triple-quoted raw strings (r"""...""") and that the character classes include single quotes, double quotes, and backticks.
  • If the scan hangs during capture, try increasing the timeout or restarting the Juice Shop container with docker restart.
  • help me debug why ShadowHawk scan is not finding endpoints

Why are these shadow APIs?

These endpoints exist in the running application's JavaScript but are not listed in any official API documentation. This is what makes them "shadow" APIs. An attacker can find them the same way ShadowHawk does, and if they lack proper authentication or input validation, they become attack vectors.

Your tool just automated a reconnaissance technique that security researchers perform manually with browser DevTools.

  • Commit your progress by running:
git add -A && git commit -m "feat: endpoint extraction from JS"

Your extraction pipeline is finding real shadow APIs. Next up, you will build security probes that test each discovered endpoint for CORS misconfigurations and missing authentication.

Add Security Probes and JSON Reporting

Your endpoint extractor is pulling API paths out of Juice Shop's JavaScript. But discovering that /api/Feedbacks exists is only half the story. The real question is whether those endpoints have security misconfigurations that an attacker could exploit.

In this step, you'll build two security probes that test each discovered endpoint with safe, read-only HTTP requests. One checks for CORS wildcard misconfigurations. The other checks if endpoints leak data without requiring authentication. You'll wire these probes into the scan pipeline and output a structured JSON report.

In this step, get ready to:
  • Implement CORS wildcard and missing-auth security probes.
  • Add rate-limited probe execution that respects target servers.
  • Wire probes into the CLI with JSON report output.
Implement the CORS wildcard and missing-auth probes

A CORS wildcard means a server responds with Access-Control-Allow-Origin: *, telling browsers that any website can make requests to it. That is a security misconfiguration because it lets malicious pages read the response data.

The missing-auth probe is simpler. It sends a request with no cookies or tokens and checks if the server hands back a JSON response anyway. If it does, that endpoint is leaking data to unauthenticated users.

  • Open src/shadowhawk/probe.py in Cursor and replace the placeholder comment with the following code:
"""Security probes - safe, read-only checks against discovered endpoints."""

import time

import httpx

from shadowhawk.models import Finding


def probe_cors_wildcard(endpoint_path: str, base_url: str) -> Finding | None:
    """Check if an endpoint returns Access-Control-Allow-Origin: *."""
    url = f"{base_url.rstrip('/')}{endpoint_path}"
    try:
        response = httpx.get(
            url,
            headers={"Origin": "https://evil.example.com"},
            timeout=10.0,
            follow_redirects=True,
        )
        acao = response.headers.get("access-control-allow-origin", "")
        if acao == "*":
            return Finding(
                finding_type="cors_wildcard",
                endpoint=endpoint_path,
                severity="medium",
                evidence=f"Access-Control-Allow-Origin: {acao}",
            )
    except httpx.HTTPError:
        pass
    return None


def probe_missing_auth(endpoint_path: str, base_url: str) -> Finding | None:
    """Check if an endpoint returns data without any authentication."""
    url = f"{base_url.rstrip('/')}{endpoint_path}"
    try:
        response = httpx.get(url, timeout=10.0, follow_redirects=True)
        if response.status_code == 200:
            content_type = response.headers.get("content-type", "")
            if "json" in content_type and len(response.content) > 2:
                body_preview = response.text[:200]
                return Finding(
                    finding_type="missing_auth",
                    endpoint=endpoint_path,
                    severity="high",
                    evidence=f"HTTP 200 with JSON body (no auth): {body_preview}",
                )
    except httpx.HTTPError:
        pass
    return None

What does this code do?

These are two independent probe functions. Each takes an endpoint path and a base URL, makes a single GET request, and returns either a Finding or None.

  • probe_cors_wildcard sends a fake Origin header and checks if the server responds with a wildcard * allowing any origin.
  • probe_missing_auth sends a completely unauthenticated GET. If the server returns HTTP 200 with a JSON body, that data is accessible to anyone without credentials.
  • Both probes only ever use GET (read-only). They never modify data on the target.
  • Don't save yet. You'll add one more function below these two.
Add rate-limited probe execution

Sending requests as fast as possible would overwhelm the target and behave like a denial-of-service attack. A responsible security tool adds deliberate pauses between requests.

  • In probe.py, add the following run_probes function below probe_missing_auth:
def run_probes(
    endpoint_paths: list[str], base_url: str, rate_limit: float = 0.2
) -> list[Finding]:
    """Run all probes against discovered endpoints with rate limiting."""
    findings: list[Finding] = []

    for path in endpoint_paths:
        result = probe_cors_wildcard(path, base_url)
        if result:
            findings.append(result)
        time.sleep(rate_limit)

        result = probe_missing_auth(path, base_url)
        if result:
            findings.append(result)
        time.sleep(rate_limit)

    return findings

What does this code do?

run_probes is the orchestrator. It loops through every discovered endpoint path, runs both probes against each one, and collects any findings.

time.sleep(rate_limit) pauses 0.2 seconds between each request by default. This keeps the tool respectful of the target server. The rate limit is configurable via a CLI flag you'll add next.

  • Save probe.py.
  • You won't see output yet. The next substep wires these probes into the CLI scan command.

✔️ Awesome, I've got everything!

Great. Double check you've saved probe.py before moving on.

ⓧ I'd like to double check the full code

"""Security probes - safe, read-only checks against discovered endpoints."""

import time

import httpx

from shadowhawk.models import Finding


def probe_cors_wildcard(endpoint_path: str, base_url: str) -> Finding | None:
    """Check if an endpoint returns Access-Control-Allow-Origin: *."""
    url = f"{base_url.rstrip('/')}{endpoint_path}"
    try:
        response = httpx.get(
            url,
            headers={"Origin": "https://evil.example.com"},
            timeout=10.0,
            follow_redirects=True,
        )
        acao = response.headers.get("access-control-allow-origin", "")
        if acao == "*":
            return Finding(
                finding_type="cors_wildcard",
                endpoint=endpoint_path,
                severity="medium",
                evidence=f"Access-Control-Allow-Origin: {acao}",
            )
    except httpx.HTTPError:
        pass
    return None


def probe_missing_auth(endpoint_path: str, base_url: str) -> Finding | None:
    """Check if an endpoint returns data without any authentication."""
    url = f"{base_url.rstrip('/')}{endpoint_path}"
    try:
        response = httpx.get(url, timeout=10.0, follow_redirects=True)
        if response.status_code == 200:
            content_type = response.headers.get("content-type", "")
            if "json" in content_type and len(response.content) > 2:
                body_preview = response.text[:200]
                return Finding(
                    finding_type="missing_auth",
                    endpoint=endpoint_path,
                    severity="high",
                    evidence=f"HTTP 200 with JSON body (no auth): {body_preview}",
                )
    except httpx.HTTPError:
        pass
    return None


def run_probes(
    endpoint_paths: list[str], base_url: str, rate_limit: float = 0.2
) -> list[Finding]:
    """Run all probes against discovered endpoints with rate limiting."""
    findings: list[Finding] = []

    for path in endpoint_paths:
        result = probe_cors_wildcard(path, base_url)
        if result:
            findings.append(result)
        time.sleep(rate_limit)

        result = probe_missing_auth(path, base_url)
        if result:
            findings.append(result)
        time.sleep(rate_limit)

    return findings
Wire probes into the scan command and create the README

The probe module is ready but nothing calls it yet. You need to update cli.py to run probes after extraction, print findings, and write a JSON report file using the ScanReport model.

  • Open src/shadowhawk/cli.py and replace its entire contents with the following:
"""ShadowHawk CLI - the main entry point."""

import json
import sys

import click

from shadowhawk import __version__
from shadowhawk.capture import capture_scripts
from shadowhawk.extract import extract_endpoints
from shadowhawk.models import ScanReport
from shadowhawk.probe import run_probes
from shadowhawk.scope import ScopeError, validate_target


@click.group()
@click.version_option(version=__version__, prog_name="shadowhawk")
def main():
    """ShadowHawk - Shadow API Detection CLI.

    Discovers undocumented API endpoints from JavaScript and probes them
    for security misconfigurations. Only scans authorized targets.
    """
    pass


@main.command()
@click.option("--target", required=True, help="Target URL to scan (must be in scope).")
@click.option("--rate-limit", default=0.2, help="Seconds between probe requests.")
@click.option("--output", default="shadowhawk-report.json", help="Output JSON report path.")
def scan(target: str, rate_limit: float, output: str):
    """Run a full scan: capture JS, extract endpoints, probe for issues."""
    click.echo(f"[*] ShadowHawk v{__version__}")
    click.echo(f"[*] Target: {target}")

    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)

    click.echo("[*] Stage 1: Capturing JavaScript...")
    scripts = capture_scripts(target)
    click.echo(f"    Found {len(scripts)} unique scripts")

    click.echo("[*] Stage 2: Extracting endpoints...")
    endpoints = extract_endpoints(scripts)
    click.echo(f"    Discovered {len(endpoints)} unique endpoints")

    for ep in endpoints:
        click.echo(f"      {ep.path}  (from {ep.source_url})")

    click.echo(f"[*] Stage 3: Running security probes (rate limit: {rate_limit}s)...")
    endpoint_paths = [ep.path for ep in endpoints]
    findings = run_probes(endpoint_paths, target, rate_limit)
    click.echo(f"    Produced {len(findings)} findings")

    for f in findings:
        icon = "!!" if f.severity == "high" else "**" if f.severity == "critical" else "--"
        click.echo(f"      [{icon}] {f.finding_type} on {f.endpoint} ({f.severity})")

    report = ScanReport(
        target=target,
        total_scripts_captured=len(scripts),
        total_endpoints_discovered=len(endpoints),
        total_findings=len(findings),
        endpoints=endpoints,
        findings=findings,
    )

    with open(output, "w") as f:
        f.write(report.model_dump_json(indent=2))

    click.echo(f"[*] Report written to {output}")
    click.echo("[*] Scan complete.")


@main.command()
@click.option("--target", required=True, help="Target URL to capture JS from.")
def capture(target: str):
    """Capture JavaScript from a target (discovery only, no probes)."""
    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)

    scripts = capture_scripts(target)
    click.echo(f"Captured {len(scripts)} unique scripts:")
    total_size = 0
    for s in scripts:
        size = len(s.content)
        total_size += size
        click.echo(f"  {s.url} ({size:,} bytes)")
    click.echo(f"Total: {total_size:,} bytes")

What changed in the CLI?

  • Two new imports: ScanReport and run_probes.
  • The scan command gains --rate-limit and --output options.
  • Stage 3 calls run_probes() and prints each finding with a severity icon.
  • A ScanReport is assembled from all pipeline results and written to disk with model_dump_json(), Pydantic's built-in JSON serializer.
  • Save cli.py.
  • Run the full scan against Juice Shop by running this command:
uv run shadowhawk scan --target http://localhost:3000

You should see Stage 1, Stage 2, and Stage 3 output. The findings section should list CORS wildcard and missing-auth issues on endpoints like /rest/products and /api/Feedbacks. At the end, you'll see the report path printed.

Not seeing any findings?

  • Make sure Juice Shop is still running. Check with docker ps and look for the bkimminich/juice-shop container.
  • Confirm the target URL includes the port: http://localhost:3000 not just http://localhost.
  • If you see zero endpoints discovered in Stage 2, the browser may have timed out. Try running the scan again.

help me debug why ShadowHawk's probes aren't finding CORS or missing-auth issues on Juice Shop

  • Verify the JSON report was created by running:
ls shadowhawk-report.json

You should see shadowhawk-report.json listed in the output.

✔️ Awesome, I've got everything!

Great. Double check you've saved both probe.py and cli.py before moving on.

ⓧ I'd like to double check the full code

"""ShadowHawk CLI - the main entry point."""

import json
import sys

import click

from shadowhawk import __version__
from shadowhawk.capture import capture_scripts
from shadowhawk.extract import extract_endpoints
from shadowhawk.models import ScanReport
from shadowhawk.probe import run_probes
from shadowhawk.scope import ScopeError, validate_target


@click.group()
@click.version_option(version=__version__, prog_name="shadowhawk")
def main():
    """ShadowHawk - Shadow API Detection CLI.

    Discovers undocumented API endpoints from JavaScript and probes them
    for security misconfigurations. Only scans authorized targets.
    """
    pass


@main.command()
@click.option("--target", required=True, help="Target URL to scan (must be in scope).")
@click.option("--rate-limit", default=0.2, help="Seconds between probe requests.")
@click.option("--output", default="shadowhawk-report.json", help="Output JSON report path.")
def scan(target: str, rate_limit: float, output: str):
    """Run a full scan: capture JS, extract endpoints, probe for issues."""
    click.echo(f"[*] ShadowHawk v{__version__}")
    click.echo(f"[*] Target: {target}")

    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)

    click.echo("[*] Stage 1: Capturing JavaScript...")
    scripts = capture_scripts(target)
    click.echo(f"    Found {len(scripts)} unique scripts")

    click.echo("[*] Stage 2: Extracting endpoints...")
    endpoints = extract_endpoints(scripts)
    click.echo(f"    Discovered {len(endpoints)} unique endpoints")

    for ep in endpoints:
        click.echo(f"      {ep.path}  (from {ep.source_url})")

    click.echo(f"[*] Stage 3: Running security probes (rate limit: {rate_limit}s)...")
    endpoint_paths = [ep.path for ep in endpoints]
    findings = run_probes(endpoint_paths, target, rate_limit)
    click.echo(f"    Produced {len(findings)} findings")

    for f in findings:
        icon = "!!" if f.severity == "high" else "**" if f.severity == "critical" else "--"
        click.echo(f"      [{icon}] {f.finding_type} on {f.endpoint} ({f.severity})")

    report = ScanReport(
        target=target,
        total_scripts_captured=len(scripts),
        total_endpoints_discovered=len(endpoints),
        total_findings=len(findings),
        endpoints=endpoints,
        findings=findings,
    )

    with open(output, "w") as f:
        f.write(report.model_dump_json(indent=2))

    click.echo(f"[*] Report written to {output}")
    click.echo("[*] Scan complete.")


@main.command()
@click.option("--target", required=True, help="Target URL to capture JS from.")
def capture(target: str):
    """Capture JavaScript from a target (discovery only, no probes)."""
    try:
        validate_target(target)
    except ScopeError as e:
        click.echo(f"[!] SCOPE VIOLATION: {e}", err=True)
        sys.exit(1)

    scripts = capture_scripts(target)
    click.echo(f"Captured {len(scripts)} unique scripts:")
    total_size = 0
    for s in scripts:
        size = len(s.content)
        total_size += size
        click.echo(f"  {s.url} ({size:,} bytes)")
    click.echo(f"Total: {total_size:,} bytes")

Now create a README so anyone cloning this repo knows how to use ShadowHawk responsibly.

  • Create a file called README.md in the root of your shadowhawk project folder (next to pyproject.toml) and paste the following:
# ShadowHawk

Shadow API Detection CLI - discovers undocumented endpoints and probes for misconfigurations.

## Usage

```
uv run shadowhawk scan --target http://localhost:3000
uv run shadowhawk capture --target http://localhost:3000
```

## Architecture

```
Target URL
   |
   v
[Scope Guard] -- rejects unauthorized targets
   |
   v
[JS Capture] -- Playwright headless browser
   |
   v
[Endpoint Extractor] -- regex pattern matching
   |
   v
[Security Probes] -- CORS, missing auth
   |
   v
[JSON Report]
```

## Disclaimer

For authorized use only. Only scan targets you have explicit permission to test.

What's in the README?

  • Usage examples showing the two CLI commands.
  • A text-based architecture diagram showing the five pipeline stages.
  • A disclaimer that makes it clear this tool is for authorized testing only. This is essential for any offensive security tool you publish.
  • Save README.md.
  • Confirm it exists by running:
ls README.md

You should see README.md listed in the output.

  • Commit the probe module, updated CLI, and report output by running:
git add src/shadowhawk/probe.py src/shadowhawk/cli.py
git commit -m "feat: security probes and JSON reporting"
  • Commit the README by running:
git add README.md
git commit -m "docs: add README with usage and disclaimer"

Why two separate commits?

Keeping the README in its own commit separates documentation from feature code. This makes your Git history easier to navigate. Reviewers can see exactly which commit introduced the probes vs. which added docs.

Secret mission

Add Reflected-Origin CORS Detection

Your current CORS probe only catches the trivial Access-Control-Allow-Origin: * wildcard. The more dangerous pattern is when a server reflects back whatever Origin you send, effectively allowing any site to make credentialed requests. Add a reflected-Origin probe that catches this subtler and more critical misconfiguration.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing cloud costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • OWASP Juice Shop Docker container (running at localhost:3000).
  • Juice Shop Docker image (bkimminich/juice-shop).
  • Playwright Chromium browser binary.
  • The shadowhawk project folder on your Desktop.
  • The shadowhawk-report.json output file.

Keep everything running

No action needed. Choose this if you are still actively building on ShadowHawk or plan to continue testing against Juice Shop.

  • Juice Shop will keep running in its terminal tab until you close it or press Ctrl+C.
  • Your project folder, report file, and Playwright browser binary all remain on disk.

Pause - I'll come back to this later

Stop the running container to free up memory and CPU. Your project files stay on disk for next time.

  • Switch to the terminal where Juice Shop is running and press Ctrl+C to stop it.
  • The --rm flag in the original run command means the container is automatically removed when stopped. No manual cleanup needed.
  • To restart later, run the same Docker command from the project steps.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each item below.

  • Stop the Juice Shop container by pressing Ctrl+C in its terminal tab.
  • Remove the Juice Shop Docker image by running:
docker rmi bkimminich/juice-shop
  • Remove the Playwright browser binary by running:
uv run playwright uninstall
  • Delete the project folder from your Desktop by running:
cd ~/Desktop
rm -rf shadowhawk
  • On Windows, use this command instead:
cd ~\Desktop
rmdir /s /q shadowhawk
  • If you generated a shadowhawk-report.json outside the project folder, delete it manually from wherever you saved it.

Nice Work!

Nice Work!

You just built ShadowHawk, a production-grade shadow API detection CLI that discovers undocumented endpoints and probes them for security misconfigurations.

You've learned how to:

  • Built a scope-guarded CLI with Click that enforces an allowlist and refuses to scan unauthorized targets.
  • Used Playwright to launch a headless browser and capture all JavaScript a page loads at runtime, including lazy-loaded bundles and inline scripts.
  • Extracted undocumented API endpoints from JavaScript using regex pattern matching and probed them for CORS wildcard misconfigurations and missing authentication, outputting structured JSON reports.
  • Secret Mission: Detected reflected-Origin CORS vulnerabilities, the more dangerous and harder-to-find misconfiguration pattern where a server echoes back any Origin it receives.

Ready to quiz yourself?