Engineer a Harness for a Python REST API

Build a five-subsystem AI agent harness and measure its impact on reliability.

Introduction

30 Second Summary

AI coding agents can write impressive code, but without structure they skip steps, declare victory early, and ship broken features. The difference between unreliable output and production-ready results is not the model you choose. It is the system you build around it.

In this project, you will build a five-subsystem harness that governs how Claude Code works inside a Python repository. You will then run a controlled experiment proving the harness makes the agent dramatically more reliable.

What You'll Build

A portfolio-ready GitHub repository containing a complete harness system, a Python FastAPI REST API built by Claude Code under harness governance, and before/after evidence measuring the improvement.

By the end of this project, you'll have:

  • A complete five-subsystem harness (AGENTS.md, feature list, init script, progress log, and verification commands) governing an AI coding agent.
  • A controlled experiment comparing a prompt-only agent run against a harnessed run on the same coding task, with a measured verification gap proving the difference.
  • A working FastAPI REST API with automated tests as proof of harness-governed output.
  • Secret Mission: Add an evaluator rubric and session handoff mechanism to bring your harness to production-grade.

What is harness engineering?

Harness engineering is the discipline of designing the constraints, feedback loops, and verification systems that wrap around an AI model to make it reliable. The core idea is that an agent equals a model plus a harness. Your harness is what turns a capable-but-chaotic model into a dependable coding partner.

Set Up the Project Repository

Before you build the harness, you need a project workspace with version control. Git branches let you run the same task twice and compare the results side by side.

In this step, you'll create the project directory, install the Python dependencies, and write the task prompt that Claude Code will receive in both runs.

In this step, get ready to:
  • Create the project directory with Git and a Python virtual environment.
  • Write the baseline task prompt for Claude Code.
  • Set up comparison branches and verify Claude Code works.
Create the project directory and initialize Git

Every harness engineering demo needs a clean workspace where the only variable between runs is the harness itself. You'll create a dedicated folder with Git tracking and a Python virtual environment to isolate your dependencies.

  • Move to your Desktop and create the project folder by running these commands:
cd ~/Desktop
mkdir harness-engineering-demo
cd harness-engineering-demo
  • Initialize a Git repository and create a Python virtual environment by running:
git init -b main
python -m venv venv

What do these commands do?

  • git init -b main creates a new Git repository with main as the default branch name.
  • python -m venv venv creates an isolated Python environment in a folder called venv so your project's packages don't conflict with system-wide installs.
  • Activate the virtual environment by running:

macOS

source venv/bin/activate

Windows

.\venv\Scripts\Activate.ps1

Linux

source venv/bin/activate

Your terminal prompt should now show (venv) at the beginning, confirming the virtual environment is active.

Don't see (venv) in your prompt?

  • On Windows, if you see an error about scripts being disabled, you may need to update your PowerShell execution policy. Search for the official Microsoft documentation on Set-ExecutionPolicy for your version of PowerShell.
  • On macOS/Linux, make sure you typed source before the path. Running the script directly without source activates it in a subshell that closes immediately.

Help me activate my Python virtual environment.

  • Create a file called requirements.txt in the harness-engineering-demo folder and paste this content:
fastapi
uvicorn
pytest
httpx

What are these dependencies?

  • FastAPI is the web framework Claude Code will use to build the REST API.
  • uvicorn is the server that runs FastAPI applications.
  • pytest is the testing framework for automated verification.
  • httpx lets pytest send HTTP requests to your API during tests.
  • Save requirements.txt.

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

fastapi
uvicorn
pytest
httpx
  • Install the project dependencies by running:
pip install -r requirements.txt

You should see output confirming that fastapi, uvicorn, pytest, and httpx (plus their sub-dependencies) installed successfully.

Seeing install errors?

  • Make sure (venv) appears in your terminal prompt. If it doesn't, re-run the activate command from the previous step.
  • If you see a permissions error, you're likely installing to the system Python instead of the virtual environment.

Help me fix pip install errors in my virtual environment.

  • Confirm your tools are ready by running:
python --version
git --version

You should see version numbers for both Python and Git printed in your terminal.

Create the baseline prompt file

The core of this experiment is giving Claude Code the exact same task twice. You need a single prompt file that stays identical across both runs so the only variable is whether the harness is present.

  • Create a file called task-prompt.md in the harness-engineering-demo folder and paste this content:
# Task: Build a FastAPI Task Management API

Build a REST API for task management using FastAPI with the following features:

1. CRUD endpoints for tasks (create, read, update, delete)
2. Filter tasks by status (todo, in-progress, done)
3. Input validation with proper error responses (400 for invalid input, 404 for missing tasks)
4. Automated tests for all endpoints using pytest

Each task should have: id (auto-generated), title (required string), description (optional string), status (enum: todo/in-progress/done, default: todo), created_at (auto-generated timestamp).

The API should run with: uvicorn app.main:app --reload
Tests should run with: python -m pytest -v

Why a separate prompt file?

By storing the task in its own file, you guarantee both runs receive identical instructions. If you typed the prompt manually each time, small wording differences could explain any performance gap. The file eliminates that variable.

  • Save task-prompt.md.

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

# Task: Build a FastAPI Task Management API

Build a REST API for task management using FastAPI with the following features:

1. CRUD endpoints for tasks (create, read, update, delete)
2. Filter tasks by status (todo, in-progress, done)
3. Input validation with proper error responses (400 for invalid input, 404 for missing tasks)
4. Automated tests for all endpoints using pytest

Each task should have: id (auto-generated), title (required string), description (optional string), status (enum: todo/in-progress/done, default: todo), created_at (auto-generated timestamp).

The API should run with: uvicorn app.main:app --reload
Tests should run with: python -m pytest -v
Create the baseline branch and verify Claude Code works

With both files in place, you'll commit the initial state and create the branch where the unharnessed run will happen. Then you'll confirm Claude Code can operate inside this project directory.

  • Commit your initial files to the main branch by running:
git add .
git commit -m "Initial project setup with requirements and task prompt"
  • Create the baseline branch by running:
git branch baseline-no-harness

Why a separate branch?

The baseline-no-harness branch will hold whatever Claude Code produces when given only the raw prompt. The main branch will hold the harnessed run. Keeping them on separate branches lets anyone visiting your repo compare the two approaches directly.

  • Verify both branches exist by running:
git branch

You should see * main and baseline-no-harness listed. The asterisk shows you're currently on main.

  • Open Claude Code in your project directory by running:
claude

Claude Code starts an interactive session in your terminal. You should see a prompt where you can type messages.

  • Verify Claude Code can read your project files by typing this message into the Claude Code prompt:
List the files in this directory

Claude Code should respond with a list that includes requirements.txt, task-prompt.md, and the venv folder.

  • Exit Claude Code by typing /exit or pressing Ctrl+C.

Claude Code not responding?

  • Make sure you're running claude from inside the harness-engineering-demo directory, not from your Desktop.
  • If you see an authentication error, confirm your API key is still valid by checking your Anthropic dashboard.

Help me get Claude Code working in my project directory.

Your workspace is ready with both branches in place. In the next step, you'll run Claude Code on the raw task prompt with no harness and record what goes wrong.

Run the Baseline (No Harness)

Your environment is set up and ready to go. Now comes the experiment that gives your harness engineering its power: running the agent with zero guidance and documenting exactly what happens.

Think of this step as the "before" photo. You will give Claude Code the exact same task you will give it later with a harness, but this time with nothing but a raw prompt. The failures you record here become your measurable proof that harness engineering works.

In this step, get ready to:
  • Run Claude Code on the task with only a raw prompt and no harness files.
  • Verify what the agent actually produced versus what it claimed.
  • Document the verification gap and commit the baseline for later comparison.
Execute the prompt-only run

A baseline establishes what happens when the agent operates without constraints. The only variable between this run and the harnessed run later will be the harness files. Everything else stays identical: same task, same model, same project directory.

  • Switch to the baseline branch by running this command:
git checkout baseline-no-harness
  • Note the current time. You will use this to measure how long the agent takes.
  • In your Claude Code session, send the following message to give the agent the raw task with no other context:
Read task-prompt.md and implement everything it describes.
  • Let the agent work. Do not interrupt, guide, or correct it.
  • Note the time again when the agent declares it is done.

What should I watch for?

Pay attention to the agent's behavior patterns as it works. Does it tackle features one at a time or jump between them? Does it run tests before declaring victory? Does it skip anything?

You do not need to memorize everything. Just keep a mental note of anything that seems rushed or incomplete. You will record these observations formally in the next substep.

Record failures and measure the verification gap

The verification gap is the proportion of features an agent claims are done but that are actually broken. This is the core metric that proves whether a harness makes a measurable difference.

What is a verification gap?

The formula is: (features claimed done but actually broken) / (total features claimed done) x 100.

A verification gap of 50% means the agent said it finished four features, but two of them do not actually work. The higher the gap, the less reliable the agent is without governance.

  • After the agent claims completion, run the test suite independently to check what actually passes:
python -m pytest -v

What does this command do?

This runs pytest in verbose mode, showing each individual test and whether it passed or failed. If the agent did not create tests at all, you will see a "no tests ran" message.

  • Try starting the server to check if the API actually runs:
uvicorn app.main:app --host 127.0.0.1 --port 8000
  • If the server starts, open a second terminal and try a quick endpoint test. Then stop the server with Ctrl+C.
  • If the server fails to start, note the error. This counts as a broken feature.

Server won't start or module not found?

If you see an import error or "module not found", the agent likely did not create the expected file structure (app/main.py). That is valid data for your baseline. Record it as a failure.

help me understand why the agent's code fails to start

  • Create a file called baseline-results.md in your project root with the following template:
# Baseline Run Results (No Harness)

## Run Details
- Date: [fill in]
- Duration: [fill in]
- Agent: Claude Code
- Harness: NONE (prompt-only)

## What the Agent Claimed Was Done
[Record what the agent said it completed]

## Actual Verification Results

| Feature | Claimed Done? | Actually Works? | Evidence |
|---------|--------------|-----------------|----------|
| CRUD endpoints | | | |
| Status filtering | | | |
| Input validation | | | |
| Automated tests | | | |

## Verification Gap
Formula: (claimed done but broken) / (total claimed done) x 100

**Verification Gap: ____%**

## Observations
- Did the agent run tests before claiming done?
- Did it skip any features?
- Did it declare victory early?
- Did it work on multiple features simultaneously?

[Record your observations here]

What does each section capture?

  • Run Details records the controlled conditions so both runs are comparable.
  • What the Agent Claimed captures the agent's self-reported completion. Copy its final summary message here.
  • Actual Verification Results is your honest assessment after running pytest and testing endpoints manually.
  • Verification Gap is the key metric. Fill in the percentage after completing the table.
  • Observations captures behavioral patterns you noticed during the run.
  • Fill in each section of baseline-results.md honestly based on what you observed and verified.
  • For each feature in the table, mark whether the agent claimed it was done, whether it actually works, and paste any error messages or test output as evidence.
  • Calculate the verification gap percentage and fill it in.

Honesty makes the portfolio piece

A high verification gap is not a failure on your part. It is exactly the evidence you need. The worse the baseline performs, the more dramatic (and impressive) your harnessed results will be in comparison.

✔️ Awesome, I've got everything!

Great. Double check you have saved baseline-results.md with your observations and verification gap filled in.

ⓧ I'd like to double check the full code

Here is the complete template for baseline-results.md. Your filled-in version should follow this structure with your actual observations added:

# Baseline Run Results (No Harness)

## Run Details
- Date: [fill in]
- Duration: [fill in]
- Agent: Claude Code
- Harness: NONE (prompt-only)

## What the Agent Claimed Was Done
[Record what the agent said it completed]

## Actual Verification Results

| Feature | Claimed Done? | Actually Works? | Evidence |
|---------|--------------|-----------------|----------|
| CRUD endpoints | | | |
| Status filtering | | | |
| Input validation | | | |
| Automated tests | | | |

## Verification Gap
Formula: (claimed done but broken) / (total claimed done) x 100

**Verification Gap: ____%**

## Observations
- Did the agent run tests before claiming done?
- Did it skip any features?
- Did it declare victory early?
- Did it work on multiple features simultaneously?

[Record your observations here]
Commit and switch branches

The baseline branch preserves everything the agent produced without a harness. You need this untouched for the side-by-side comparison later.

  • Stage and commit all files the agent created, plus your results document, by running these commands:
git add -A
git commit -m "Baseline run: prompt-only, no harness"

Why commit everything?

Even if the agent produced broken or incomplete code, commit it all. This branch is your "before" evidence. Anyone reviewing your portfolio repo can check out this branch and see exactly what an unharnessed agent produces.

  • Record any final observations about total time spent and agent behavior in baseline-results.md before moving on.
  • If you updated the file, commit the change:
git add baseline-results.md
git commit -m "Add final baseline observations"
  • Switch back to the main branch for the harnessed run by running:
git checkout main
  • Confirm you are on main by running:
git branch

You should see an asterisk (*) next to main in the output.

Still seeing baseline files?

If you see the agent's code files after switching to main, make sure you committed everything on the baseline branch first. Uncommitted changes follow you across branches.

help me fix my git branch situation

You now have hard evidence of what an unharnessed agent produces. Next up, you will build the five-subsystem harness that transforms this same task from unreliable to verifiable.

Build the Five-Subsystem Harness

In the previous step, you ran Claude Code with nothing but a raw prompt and documented the gaps. Now you know exactly where an unharnessed agent fails.

This step is the core engineering work. You will build five files that together form a complete harness governing how Claude Code operates inside your repository. Each file maps to one or more of the five harness subsystems: instructions, state, verification, scope, and lifecycle.

In this step, get ready to:
  • Create AGENTS.md and feature_list.json to define what the agent must do and in what order.
  • Build an initialization script and clean-state checklist to control the agent's environment and session boundaries.
  • Set up a progress log so the agent records evidence of its work.
Build the Instruction and Scope subsystems

The Instruction subsystem tells the agent how to behave. The Scope subsystem tells it what to work on and in what order. Together, these two files replace the vague prompt you used in the baseline run with explicit, verifiable rules.

  • Create a new file called AGENTS.md in your harness-engineering-demo folder and paste the following content:
# Project: Harness Engineering Demo - Task Management API

## Overview
A FastAPI REST API for task management, built to demonstrate harness engineering principles.

## Tech Stack
- Python 3.x
- FastAPI (web framework)
- Uvicorn (ASGI server)
- Pytest + HTTPX (testing)

## First-Run Commands
1. Run init.ps1 to verify environment health
2. Run `python -m pytest -v` to check existing test state

## Hard Constraints
- Work on ONE feature at a time from feature_list.json
- Never mark a feature as "pass" without running verification and recording evidence
- Always run `python -m pytest -v` before claiming any feature is complete
- If tests fail, fix them before moving to the next feature
- Update claude-progress.md after completing each feature
- Update feature_list.json status after verification passes

## Verification Commands
- Import check: `python -c "from app.main import app; print('OK')"`
- Test suite: `python -m pytest -v`
- Server startup: `uvicorn app.main:app --host 127.0.0.1 --port 8000` (verify it starts without errors, then stop)

## Project Structure
```
harness-engineering-demo/
├── app/
│   ├── __init__.py
│   └── main.py
├── tests/
│   ├── __init__.py
│   └── test_api.py
├── AGENTS.md (this file)
├── feature_list.json
├── init.ps1
├── clean-state-checklist.md
├── claude-progress.md
└── requirements.txt
```

## Workflow
1. Read feature_list.json to find the first feature with status "not_started"
2. Implement ONLY that feature
3. Run verification commands
4. If verification passes, update feature_list.json status to "pass" and add evidence
5. Update claude-progress.md
6. Move to the next feature
7. Before ending: complete clean-state-checklist.md checks

What does this file do?

AGENTS.md is the instruction manual that Claude Code reads automatically when it opens your project directory. It replaces hope with structure.

  • The Hard Constraints section forces one-feature-at-a-time execution. In your baseline run, the agent likely tried to tackle everything at once.
  • The Verification Commands section gives the agent concrete commands to prove its work. No more "I'm done" without evidence.
  • The Workflow section defines the exact loop the agent follows for each feature.
  • Save AGENTS.md.

Why AGENTS.md and not CLAUDE.md?

AGENTS.md is an open standard supported by multiple AI coding agents, not just Claude Code. Using it means your harness works with Google Antigravity, Kimchi, and future agents too. Claude Code reads both AGENTS.md and CLAUDE.md from the project root.

Next, you need a structured list of features that the agent works through one at a time.

  • Create a new file called feature_list.json in your harness-engineering-demo folder and paste the following content:
{
  "project": "task-management-api",
  "features": [
    {
      "id": "F1",
      "name": "CRUD endpoints for tasks",
      "status": "not_started",
      "acceptance_criteria": "POST /tasks creates a task, GET /tasks returns all tasks, GET /tasks/{id} returns one task, PUT /tasks/{id} updates a task, DELETE /tasks/{id} deletes a task",
      "verification_command": "python -m pytest tests/test_api.py -v -k crud",
      "evidence": ""
    },
    {
      "id": "F2",
      "name": "Filter tasks by status",
      "status": "not_started",
      "acceptance_criteria": "GET /tasks?status=todo returns only tasks with status todo. Same for in-progress and done.",
      "verification_command": "python -m pytest tests/test_api.py -v -k filter",
      "evidence": ""
    },
    {
      "id": "F3",
      "name": "Input validation with error responses",
      "status": "not_started",
      "acceptance_criteria": "POST /tasks with missing title returns 422. PUT /tasks/{id} with invalid status returns 422. GET /tasks/{nonexistent_id} returns 404.",
      "verification_command": "python -m pytest tests/test_api.py -v -k validation",
      "evidence": ""
    },
    {
      "id": "F4",
      "name": "Automated tests for all endpoints",
      "status": "not_started",
      "acceptance_criteria": "pytest test suite covers all CRUD operations, filtering, and error cases. All tests pass.",
      "verification_command": "python -m pytest -v --tb=short",
      "evidence": ""
    }
  ]
}

What does this file do?

This is the Scope subsystem. Each feature has a unique ID, acceptance criteria that define "done", and a verification command the agent must run to prove completion.

The status field starts at not_started and changes to pass only when the agent records evidence. The evidence field stays empty until the agent fills it with actual test output.

  • Save feature_list.json.
  • Confirm both files exist by running:
ls AGENTS.md feature_list.json

You should see both filenames listed in the output.

Don't see one of the files?

  • Make sure you saved both files in the harness-engineering-demo root folder, not inside a subfolder.
  • Check that the filenames are exact. AGENTS.md must be uppercase. feature_list.json uses an underscore, not a hyphen.

Help me troubleshoot why my harness files aren't showing up in my project directory.

Build the Environment and Lifecycle subsystems

The Environment subsystem ensures the project is in a healthy state before work begins. The Lifecycle subsystem defines when the agent is allowed to commit or end a session. These two files automate the "before" and "after" checks that a senior engineer would normally enforce in code review.

  • Create a new file called init.ps1 in your harness-engineering-demo folder and paste the following content:
# Harness Initialization Script
# Run this at the start of every agent session

Write-Host "=== Harness Init: Environment Health Check ===" -ForegroundColor Cyan

# Check Python
Write-Host "`n[1/4] Python version:" -ForegroundColor Yellow
python --version

# Activate virtual environment
Write-Host "`n[2/4] Activating virtual environment..." -ForegroundColor Yellow
if (Test-Path ".\venv\Scripts\Activate.ps1") {
    .\venv\Scripts\Activate.ps1
    Write-Host "Virtual environment activated." -ForegroundColor Green
} else {
    Write-Host "ERROR: Virtual environment not found. Run: python -m venv venv" -ForegroundColor Red
    exit 1
}

# Install dependencies
Write-Host "`n[3/4] Installing dependencies..." -ForegroundColor Yellow
pip install -r requirements.txt --quiet
Write-Host "Dependencies installed." -ForegroundColor Green

# Run existing tests
Write-Host "`n[4/4] Running existing tests..." -ForegroundColor Yellow
python -m pytest -v --tb=short 2>$null
if ($LASTEXITCODE -eq 5) {
    Write-Host "No tests found yet (expected on first run)." -ForegroundColor Yellow
} elseif ($LASTEXITCODE -eq 0) {
    Write-Host "All tests pass." -ForegroundColor Green
} else {
    Write-Host "WARNING: Some tests are failing. Fix before starting new work." -ForegroundColor Red
}

Write-Host "`n=== Harness Init Complete ===" -ForegroundColor Cyan
Write-Host "Read feature_list.json to find your next task." -ForegroundColor White

What does this script do?

This is the Environment subsystem's entry point. Every time the agent starts a session, it runs this script to verify the project is healthy before writing any code.

  • Steps 1-3 guarantee the environment is ready (correct Python, venv active, packages installed).
  • Step 4 runs any existing tests so the agent knows the current pass/fail state. Exit code 5 means "no tests yet" which is expected on the first run.
  • Save init.ps1.

macOS or Linux users

This project uses a .ps1 script because Claude Code can execute PowerShell cross-platform. If you prefer bash, you can create an equivalent init.sh later. For now, keep init.ps1 as-is so the agent finds the file named in AGENTS.md.

Now create the Lifecycle boundary. This checklist defines what must be true before the agent is allowed to commit code or end a session.

  • Create a new file called clean-state-checklist.md in your harness-engineering-demo folder and paste the following content:
# Clean State Checklist

Complete ALL items before ending a session or committing code.

- [ ] All tests pass (`python -m pytest -v` exits with code 0)
- [ ] No debug print statements left in code
- [ ] No hardcoded test values in production code
- [ ] feature_list.json statuses are accurate (no "pass" without evidence)
- [ ] claude-progress.md updated with this session's work
- [ ] Server starts without errors (`uvicorn app.main:app` runs clean)
- [ ] Git status is clean (all changes committed or intentionally staged)

What does this file do?

This is the Lifecycle subsystem's exit gate. Without it, the agent in your baseline run could declare "done" and stop. With it, the agent must prove seven conditions are met before ending work. It is the difference between "I think I'm done" and "I can verify I'm done."

  • Save clean-state-checklist.md.
  • Confirm both new files exist by running:
ls init.ps1 clean-state-checklist.md

You should see both filenames in the output.

Seeing a "No such file" error?

  • Double-check you are in the harness-engineering-demo directory. Run pwd to confirm.
  • Check the filename for init.ps1. The extension is .ps1 (PowerShell), not .sh or .py.

Help me figure out why my init.ps1 or clean-state-checklist.md file isn't showing up.

Build the State and Feedback subsystems

The State subsystem gives the agent a place to record what it accomplished, what failed, and what comes next. Without this, the agent has no memory between features and no way to self-audit.

  • Create a new file called claude-progress.md in your harness-engineering-demo folder and paste the following content:
# Progress Log

## Session Template

### Session [N] - [Date]
**Features attempted:** [list]
**Features completed (with evidence):** [list with test output or verification proof]
**Blockers:** [any issues encountered]
**Next steps:** [what to pick up next session]
**Verification gap:** [anything claimed done but not actually verified]

---

(Sessions will be logged below as work progresses)

What does this file do?

This is the State subsystem. The agent fills in a new session entry each time it works. The Verification gap field forces the agent to self-report any claims it made without proof. Combined with the Verification Commands in AGENTS.md, this creates a complete Feedback loop.

  • Save claude-progress.md.

All five harness files are now in place. Time to commit them to the main branch so Claude Code finds them on the next run.

  • Verify all five files exist by running:
ls AGENTS.md feature_list.json init.ps1 clean-state-checklist.md claude-progress.md

You should see all five filenames listed with no errors.

  • Commit all harness files to the main branch by running these commands:
git add AGENTS.md feature_list.json init.ps1 clean-state-checklist.md claude-progress.md
git commit -m "Add five-subsystem harness files"

What did you just commit?

Your main branch now contains the complete harness. When Claude Code opens this directory in the next step, it automatically reads AGENTS.md and discovers the rules, workflow, and verification commands that govern its behavior.

Here is how each file maps to the five-subsystem model:

  • Instructions: AGENTS.md (hard constraints, workflow).
  • Scope: feature_list.json (what to build, in what order).
  • Environment: init.ps1 (health check, dependency install).
  • Lifecycle: clean-state-checklist.md (exit gate).
  • State + Feedback: claude-progress.md (progress log) and the verification commands in AGENTS.md.

Git commit failed?

  • If you see "nothing to commit", check that you saved all five files before running git add.
  • If you see "not a git repository", confirm you are inside the harness-engineering-demo folder where you ran git init in Step 1.
  • If you see a message about configuring your identity, run git config user.email and git config user.name with your details first.

Help me fix a git commit error when trying to commit my harness files.

✔️ Awesome, I've got everything!

Great. Double check you have saved all five files and that your commit went through cleanly.

ⓧ I'd like to double check the full code

Here are all five harness files for comparison. Make sure yours match exactly.

# Project: Harness Engineering Demo - Task Management API

## Overview
A FastAPI REST API for task management, built to demonstrate harness engineering principles.

## Tech Stack
- Python 3.x
- FastAPI (web framework)
- Uvicorn (ASGI server)
- Pytest + HTTPX (testing)

## First-Run Commands
1. Run init.ps1 to verify environment health
2. Run `python -m pytest -v` to check existing test state

## Hard Constraints
- Work on ONE feature at a time from feature_list.json
- Never mark a feature as "pass" without running verification and recording evidence
- Always run `python -m pytest -v` before claiming any feature is complete
- If tests fail, fix them before moving to the next feature
- Update claude-progress.md after completing each feature
- Update feature_list.json status after verification passes

## Verification Commands
- Import check: `python -c "from app.main import app; print('OK')"`
- Test suite: `python -m pytest -v`
- Server startup: `uvicorn app.main:app --host 127.0.0.1 --port 8000` (verify it starts without errors, then stop)

## Project Structure
```
harness-engineering-demo/
├── app/
│   ├── __init__.py
│   └── main.py
├── tests/
│   ├── __init__.py
│   └── test_api.py
├── AGENTS.md (this file)
├── feature_list.json
├── init.ps1
├── clean-state-checklist.md
├── claude-progress.md
└── requirements.txt
```

## Workflow
1. Read feature_list.json to find the first feature with status "not_started"
2. Implement ONLY that feature
3. Run verification commands
4. If verification passes, update feature_list.json status to "pass" and add evidence
5. Update claude-progress.md
6. Move to the next feature
7. Before ending: complete clean-state-checklist.md checks
{
  "project": "task-management-api",
  "features": [
    {
      "id": "F1",
      "name": "CRUD endpoints for tasks",
      "status": "not_started",
      "acceptance_criteria": "POST /tasks creates a task, GET /tasks returns all tasks, GET /tasks/{id} returns one task, PUT /tasks/{id} updates a task, DELETE /tasks/{id} deletes a task",
      "verification_command": "python -m pytest tests/test_api.py -v -k crud",
      "evidence": ""
    },
    {
      "id": "F2",
      "name": "Filter tasks by status",
      "status": "not_started",
      "acceptance_criteria": "GET /tasks?status=todo returns only tasks with status todo. Same for in-progress and done.",
      "verification_command": "python -m pytest tests/test_api.py -v -k filter",
      "evidence": ""
    },
    {
      "id": "F3",
      "name": "Input validation with error responses",
      "status": "not_started",
      "acceptance_criteria": "POST /tasks with missing title returns 422. PUT /tasks/{id} with invalid status returns 422. GET /tasks/{nonexistent_id} returns 404.",
      "verification_command": "python -m pytest tests/test_api.py -v -k validation",
      "evidence": ""
    },
    {
      "id": "F4",
      "name": "Automated tests for all endpoints",
      "status": "not_started",
      "acceptance_criteria": "pytest test suite covers all CRUD operations, filtering, and error cases. All tests pass.",
      "verification_command": "python -m pytest -v --tb=short",
      "evidence": ""
    }
  ]
}
# Harness Initialization Script
# Run this at the start of every agent session

Write-Host "=== Harness Init: Environment Health Check ===" -ForegroundColor Cyan

# Check Python
Write-Host "`n[1/4] Python version:" -ForegroundColor Yellow
python --version

# Activate virtual environment
Write-Host "`n[2/4] Activating virtual environment..." -ForegroundColor Yellow
if (Test-Path ".\venv\Scripts\Activate.ps1") {
    .\venv\Scripts\Activate.ps1
    Write-Host "Virtual environment activated." -ForegroundColor Green
} else {
    Write-Host "ERROR: Virtual environment not found. Run: python -m venv venv" -ForegroundColor Red
    exit 1
}

# Install dependencies
Write-Host "`n[3/4] Installing dependencies..." -ForegroundColor Yellow
pip install -r requirements.txt --quiet
Write-Host "Dependencies installed." -ForegroundColor Green

# Run existing tests
Write-Host "`n[4/4] Running existing tests..." -ForegroundColor Yellow
python -m pytest -v --tb=short 2>$null
if ($LASTEXITCODE -eq 5) {
    Write-Host "No tests found yet (expected on first run)." -ForegroundColor Yellow
} elseif ($LASTEXITCODE -eq 0) {
    Write-Host "All tests pass." -ForegroundColor Green
} else {
    Write-Host "WARNING: Some tests are failing. Fix before starting new work." -ForegroundColor Red
}

Write-Host "`n=== Harness Init Complete ===" -ForegroundColor Cyan
Write-Host "Read feature_list.json to find your next task." -ForegroundColor White
# Clean State Checklist

Complete ALL items before ending a session or committing code.

- [ ] All tests pass (`python -m pytest -v` exits with code 0)
- [ ] No debug print statements left in code
- [ ] No hardcoded test values in production code
- [ ] feature_list.json statuses are accurate (no "pass" without evidence)
- [ ] claude-progress.md updated with this session's work
- [ ] Server starts without errors (`uvicorn app.main:app` runs clean)
- [ ] Git status is clean (all changes committed or intentionally staged)
# Progress Log

## Session Template

### Session [N] - [Date]
**Features attempted:** [list]
**Features completed (with evidence):** [list with test output or verification proof]
**Blockers:** [any issues encountered]
**Next steps:** [what to pick up next session]
**Verification gap:** [anything claimed done but not actually verified]

---

(Sessions will be logged below as work progresses)

Your harness is locked in. Next up, you will run Claude Code on the exact same task from the baseline, but this time with all five subsystem files guiding its behavior.

Run with the Harness

Your harness is fully assembled. Five files now sit in your repository, each mapping to a subsystem that governs how Claude Code works.

Now comes the moment of truth. You will run Claude Code on the exact same task from task-prompt.md. The only difference between this run and the baseline is the harness. Same model, same prompt, different environment. This controlled comparison is what makes the results meaningful.

In this step, get ready to:
  • Launch Claude Code with the harness and observe how it follows the constraints.
  • Watch the agent work through all four features using the one-at-a-time workflow.
  • Verify and document the harnessed run results for comparison.
Initialize and run

The harness works because the agent reads its instruction files before starting work. By pointing Claude Code at AGENTS.md and feature_list.json first, you give it the constraints, workflow, and verification commands that were missing in the baseline run.

  • Confirm you are on the main branch by running:
git branch
  • You should see * main in the output, indicating you are on the correct branch.
  • Start a timer. You will compare the total time against your baseline run.
  • Open Claude Code in the harness-engineering-demo directory by running:
claude
  • Give Claude Code the following prompt to kick off the harnessed run:
Read AGENTS.md and feature_list.json, then run init.ps1 and begin work on the first unfinished feature.

Why this specific prompt?

This prompt explicitly tells the agent to read the harness files before doing anything else. Unlike the baseline run where the agent received only the task description, this time the agent gets the constraints, workflow rules, and verification commands up front.

The init.ps1 script runs the environment health check, confirming dependencies are installed and tests are in a known state before any coding begins.

Note the time

Write down the time you started. You will record this in your results file later to compare against the baseline duration.

Observe harness-governed behavior

This is where the harness earns its value. Instead of jumping in and guessing, you will watch the agent and note how its behavior differs from the baseline run.

  • Watch the agent work and note whether it follows these harness-governed behaviors:
  • Let the agent complete all four features. Do not interrupt or guide it.
  • When the agent declares it is done, check whether it followed the clean-state-checklist.md before finishing.

What should I see?

A well-harnessed agent will create the app/ directory with __init__.py and main.py. It will also create the tests/ directory with __init__.py and test_api.py. You should see it run python -m pytest -v after each feature and only move on when tests pass.

Compare this to the baseline. Did the baseline agent run tests before claiming done? Did it skip features or work on multiple at once? These differences are your evidence.

Agent not following the harness rules?

If the agent ignores the constraints in AGENTS.md (e.g., working on multiple features at once), remind it: "Check AGENTS.md Hard Constraints. Work on one feature at a time." This is still a valid harnessed run because real harness engineering includes mid-session corrections.

If the agent fails to read AGENTS.md entirely, re-issue the prompt: "Read AGENTS.md first, then follow the workflow section."

help me get Claude Code to follow my AGENTS.md instructions

Verify and document results

The agent has declared itself done. Now you independently verify its claims, just like you did for the baseline run. Trust but verify.

  • Run the test suite independently to verify all tests pass:
python -m pytest -v
  • You should see all tests passing with a summary line showing no failures.
  • Open feature_list.json and check that all four features show "status": "pass" with evidence recorded.
  • Open claude-progress.md and check that the agent logged its session with features attempted, completed, and verification proof.
  • Stop your timer and record the total duration of the harnessed run.

What if some tests fail?

If tests fail, that is still valid data. Record exactly what passed and what failed. The comparison against the baseline is what matters. Even a partial improvement demonstrates harness value.

If you want to push for full completion, you can ask Claude Code to fix the failing tests while still following the harness workflow. Record any additional time this takes separately.

  • Create harnessed-results.md documenting the harnessed run. Use this template:
# Harnessed Run Results

## Run Details
- Date: [fill in]
- Duration: [fill in]
- Agent: Claude Code
- Harness: AGENTS.md + feature_list.json + init.ps1 + clean-state-checklist.md + claude-progress.md

## What the Agent Completed
[Record what features the agent built and in what order]

## Verification Results

| Feature | Status in feature_list.json | Tests Pass? | Evidence |
|---------|----------------------------|-------------|----------|
| CRUD endpoints | | | |
| Status filtering | | | |
| Input validation | | | |
| Automated tests | | | |

## Verification Gap
Formula: (claimed done but broken) / (total claimed done) x 100

**Verification Gap: ____%**

## Behavior Differences from Baseline
- Did the agent work on one feature at a time? [yes/no + notes]
- Did it run verification commands before claiming done? [yes/no + notes]
- Did it update progress logs? [yes/no + notes]
- Did it follow the clean-state-checklist? [yes/no + notes]
- Total time compared to baseline: [faster/slower by how much]

## Summary
[Your overall assessment: how did the harness change agent behavior and output quality?]

What does this file capture?

This file is the companion to baseline-results.md. Together they form the before/after evidence that proves your harness works. Fill in every field based on what you actually observed during the run.

The "Behavior Differences" section is especially important. It captures the qualitative changes (did it follow constraints?) alongside the quantitative ones (verification gap percentage).

  • Fill in the template with your actual observations from the harnessed run.
  • Save harnessed-results.md.
  • Commit all changes on the main branch by running:
git add .
git commit -m "Harnessed run complete: all features verified"
  • Verify the commit succeeded. You should see a summary of files changed including the new app/ and tests/ directories, updated feature_list.json, and your harnessed-results.md.

Git commit shows nothing to commit?

If the agent already committed its own changes, that is fine. Your harnessed-results.md file should still show as untracked. Run git status to see what is uncommitted, then add and commit just that file.

help me commit my harnessed results when the agent already committed its changes

You now have hard evidence on both branches. The baseline sits on baseline-no-harness and the harnessed output lives on main. Next up, you will write the comparison README that tells this story to anyone who visits your repository.

Compare and Document

Your harnessed run is complete and the tests pass. You now have two branches telling two very different stories about the same agent doing the same task.

The final piece is making those stories visible to anyone who visits your repository. A clear README with before/after evidence transforms this from a personal experiment into a portfolio-ready demonstration of harness engineering competence.

In this step, get ready to:
  • Write a comparison README with the five-subsystem model, results table, and verification gap numbers.
  • Commit, tag, and push both branches to GitHub.
Write the comparison README

Your README is the front door to this repository. It needs to answer three questions for any visitor: what is harness engineering, what files compose your harness, and what measurable difference did the harness make.

  • Create a file called README.md in the root of your harness-engineering-demo folder, then paste this template:
# Harness Engineering Demo

A controlled experiment proving that harness design, not model choice, is what makes AI coding agent output reliable.

## What Is Harness Engineering?

A harness is every piece of code, configuration, and execution logic that wraps around an AI model to make it reliable. This project implements the five-subsystem harness model:

| Subsystem | File | Purpose |
|-----------|------|---------|
| Instructions | AGENTS.md | Hard constraints, workflow rules, verification commands |
| Scope | feature_list.json | One-feature-at-a-time task boundaries |
| Environment | init.ps1 | Repeatable session setup and health checks |
| State | claude-progress.md | Persistent memory across sessions |
| Lifecycle | clean-state-checklist.md | Exit criteria before committing |

## The Experiment

Same task. Same model (Claude Code). Same prompt. Only variable: the harness.

**Task:** Build a FastAPI REST API with CRUD endpoints, status filtering, input validation, and automated tests.

## Results

| Metric | Baseline (No Harness) | Harnessed Run |
|--------|----------------------|---------------|
| Duration | [[BASELINE_DURATION="e.g. 8 minutes"]] | [[HARNESSED_DURATION="e.g. 12 minutes"]] |
| Features claimed done | [[BASELINE_CLAIMED="e.g. 4/4"]] | [[HARNESSED_CLAIMED="e.g. 4/4"]] |
| Features actually working | [[BASELINE_WORKING="e.g. 2/4"]] | [[HARNESSED_WORKING="e.g. 4/4"]] |
| Verification gap | [[BASELINE_GAP="e.g. 50%"]] | [[HARNESSED_GAP="e.g. 0%"]] |

## Verification Gap Formula

Verification Gap = (claimed done but broken) / (total claimed done) x 100

- Baseline: [[BASELINE_GAP_CALC="e.g. 2/4 = 50%"]]
- Harnessed: [[HARNESSED_GAP_CALC="e.g. 0/4 = 0%"]]

## Key Observations

- [[OBSERVATION_1="e.g. Baseline agent declared victory without running tests"]]
- [[OBSERVATION_2="e.g. Harnessed agent worked one feature at a time"]]
- [[OBSERVATION_3="e.g. Harnessed agent self-corrected after failing verification"]]

## Repository Structure

- `main` branch: Harnessed code + all harness files
- `baseline-no-harness` branch: Unharnessed agent output for comparison

## How to Reproduce

1. Clone this repo
2. Switch to `baseline-no-harness` to see the prompt-only output
3. Switch to `main` to see the harnessed output
4. Compare `baseline-results.md` with `harnessed-results.md`

What does this README do?

The README serves as your portfolio evidence. It explains the five-subsystem model with a table mapping each subsystem to its file. The results table shows concrete numbers from both runs so anyone can see the improvement at a glance.

The verification gap formula gives visitors the exact metric you used. The observations section documents qualitative behavior differences you noticed between the two runs.

  • Fill in each placeholder with your actual numbers from baseline-results.md and harnessed-results.md.
  • Replace the observation placeholders with specific behaviors you noticed during each run.
  • Save README.md.

Use your own numbers

The power of this README comes from real data. If your baseline run actually produced 3/4 working features, record that honestly. A 25% verification gap is still meaningful evidence that the harness improved reliability.

Not sure what numbers to use?

Open baseline-results.md and count how many features the agent claimed were done versus how many actually passed when you tested them manually. That ratio is your baseline verification gap.

For the harnessed run, check feature_list.json. Every feature with status pass and recorded evidence counts as verified. Help me calculate my verification gap from my baseline and harnessed results.

Final commit and push

With the README complete, your repository has everything it needs to demonstrate harness engineering to anyone who visits it.

  • Commit the README to your main branch by running these commands:
git add README.md
git commit -m "Add comparison README with verification gap evidence"
  • Tag the final state of your project by running:
git tag v1.0-harness-complete

Why tag the final state?

A Git tag marks a specific point in your history as significant. Anyone cloning this repo can check out v1.0-harness-complete to see the exact state where all harness files, passing tests, and documentation are in place.

  • Verify both branches exist by running:
git branch
  • You should see both main and baseline-no-harness listed in the output.
  • Push both branches and the tag to GitHub by running:
git push origin main
git push origin baseline-no-harness
git push origin v1.0-harness-complete

What if I don't have a remote set up?

If you have not yet created a GitHub repository, create one on GitHub first, then add the remote:

git remote add origin [[YOUR_REPO_URL="https://github.com/YOUR_USERNAME/harness-engineering-demo.git"]]

Then re-run the push commands above. If you want to skip GitHub for now, your local repo with both branches and the tag is still a complete deliverable.

Push rejected or branch not found?

If git push is rejected, check that the remote URL is correct with git remote -v. Make sure you created the GitHub repository as empty (no README or license initialized on GitHub's side).

If baseline-no-harness does not appear in git branch output, switch to it and verify it has your baseline code: git checkout baseline-no-harness. Then switch back with git checkout main. Help me fix my git push.

Secret mission

Add an Evaluator Rubric and Session Handoff

Your harness works, but it still trusts the agent's own judgment about quality. What if the agent could evaluate its own work against a rubric before claiming completion? And what happens when a session ends mid-task? Upgrade your harness to production-grade with role separation and multi-session continuity.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. The Claude Code API usage ($0.50-$2.00) was a one-time spend during your agent runs. Decide whether to keep your files for your portfolio or delete them.

Resources you used:

  • Local harness-engineering-demo directory with Python virtual environment and Git repository.
  • Git branches: main and baseline-no-harness (both local, stored inside the project directory).
  • Claude Code API credits consumed during the two agent runs (already spent, no ongoing charge).

Keep everything running

No action needed. Choose this if you plan to push the repository to GitHub as a portfolio piece or continue experimenting with harness files.

  • Your harness-engineering-demo folder stays on your Desktop with all harness files, both branches, and the working API.
  • Push to GitHub when ready to share it on your resume or LinkedIn.
  • The virtual environment uses approximately 50-100 MB of disk space.

Pause - I'll come back to this later

Nothing to pause. All resources are local files with no running processes or cloud services. Your project will be exactly where you left it whenever you return.

  • If you want to free up a small amount of disk space without losing your work, you can delete just the virtual environment folder by running:
cd ~/Desktop/harness-engineering-demo
rm -rf venv

On Windows, run this in PowerShell instead:

cd ~\Desktop\harness-engineering-demo
Remove-Item -Recurse -Force venv

When you come back, recreate it with python -m venv venv and reinstall dependencies with pip install -r requirements.txt.

Delete - I don't want to use this again

Remove the entire project directory and all its contents. This deletes your harness files, the API code, both Git branches, and the virtual environment.

  • On macOS or Linux, run:
cd ~/Desktop
rm -rf harness-engineering-demo
  • On Windows, run this in PowerShell:
cd ~\Desktop
Remove-Item -Recurse -Force harness-engineering-demo
  • Confirm deletion by running ls (macOS/Linux) or dir (Windows). You should no longer see harness-engineering-demo in the listing.
  • The Claude Code API credits you used are already consumed. There is nothing to cancel or deactivate.

Nice Work!

Nice Work!

Nice work! You just engineered a complete harness system that transforms how an AI coding agent operates. More importantly, you have the measured evidence to prove it.

You've learned how to:

  • Engineer a five-subsystem harness (instructions, state, verification, scope, lifecycle) that governs AI agent behavior through files like AGENTS.md, feature_list.json, and init.ps1.
  • Run a controlled experiment comparing prompt-only versus harnessed agent runs on the same task, isolating the harness as the only variable.
  • Measure the verification gap (features claimed done but actually broken) and document quantitative improvement in a portfolio-ready README.md.
  • Secret Mission: Added an evaluator rubric and session handoff mechanism for production-grade multi-session continuity with role separation.

Ready to quiz yourself?