Build an Observable Health-Check Service

Prove liveness vs readiness with Flask, OpenTelemetry, and a kill drill.

Introduction

30 Second Summary

Every system people depend on can be running without being ready to serve. A service that reports "all good" while its database is dead sends users into errors no one can explain.

In this project, you will build a Flask service with two health-check endpoints that answer genuinely different questions. You will kill a dependency process and watch one endpoint hold steady while the other flips to failure.

What You'll Build

You will start a service and its backing store in two terminals, kill the store with Ctrl+C, and immediately see readiness return 503 while liveness holds at 200.

By the end of this project, you'll have:

  • Kill a process and prove the readiness contract: liveness holds at 200 while readiness flips to 503, with response times under 200ms.
  • Trace a request through three nested spans in a JSON lines file, verified by a five-line assertion script that exits non-zero on failure.
  • Score AI-generated design docs against a sealed key you wrote before the build, measuring coverage as kept over kept-plus-missed.
  • Secret Mission: Add graceful shutdown drain so the service announces it is leaving before it is gone.

Are there any prerequisites?

You need a text editor, a terminal, and access to any AI client (Claude, ChatGPT, or a local model). Python 3.11+ and git are verified during setup with install paths provided if a check fails.

Before We Start

Before any code or configuration, take a moment to lock in what you are about to build and why it matters.

Set Up the Project

Setup lives inside the 45-minute project clock, not beside it. A gated check covers your text editor, version control, and the language runtime.

On a machine that ran a Python project before, the marginal setup here is one pip install command. You verify first and take the install branch only if a check fails.

In this step, get ready to:
  • Verify your tools are installed (text editor, Git, Python 3.11+).
  • Create and initialize the project folder with Git.
  • Install dependencies and create the project's standing rules and config files.
Run the setup gate

Before writing any code, confirm your machine has everything the project needs. You need a text editor, Git, and Python 3.11 or newer.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS/Linux) or PowerShell (Windows) and press Enter to open it.

Confirm you have a text editor (VS Code, Notepad++, vim, nano, or any editor that can save plain text files). If you do not have one, install VS Code from https://code.visualstudio.com.

  • Check that Git is installed by running this command:
git --version

✔️ I see a version number

Git is ready. Move on to the Python check below.

ⓧ Command not found

Install Git for your operating system.

Mac

  • Install the Xcode command line tools by running:
xcode-select --install

A dialog will appear asking you to confirm. Click Install and wait for it to finish.

Windows

  • Install Git by running:
winget install Git.Git

Close and reopen your terminal after the install finishes, then run git --version again to confirm.

Linux

  • Install Git by running:
sudo apt install git

On Fedora, use sudo dnf install git instead.

  • Check your Python version by running:

Mac

python3 --version

Windows

python --version

Linux

python3 --version

✔️ I see version 3.11 or higher

Python is ready. Move on to creating the project folder below.

ⓧ I see an older version

Your system has Python installed, but the project requires 3.11+ for compatibility with all dependencies.

Mac

  • Install a newer Python alongside your existing one by running:
brew install python@3.12

After installing, run python3 --version again. If it still shows the old version, Homebrew's bin directory may not be first in your PATH. Run which python3 to check which executable is being used.

Windows

  • Install Python 3.12 by running:
winget install Python.Python.3.12

Close and reopen your terminal, then run python --version again to confirm.

Linux

  • Install Python 3.12 by running:
sudo apt install python3.12 python3.12-venv

On Fedora, use sudo dnf install python3.12 instead.

ⓧ Command not found

Python is not installed yet.

Mac

  • Install Python 3.12 by running:
brew install python@3.12

After installing, run python3 --version to confirm it prints 3.12 or higher.

Windows

  • Install Python 3.12 by running:
winget install Python.Python.3.12

Close and reopen your terminal, then run python --version to confirm.

Linux

  • Install Python 3.12 by running:
sudo apt install python3.12 python3.12-venv

On Fedora, use sudo dnf install python3.12 instead.

Once all three checks pass, create your project folder.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and initialize Git by running these commands:
mkdir health-check-service
cd health-check-service
git init
git config core.autocrlf input

What does this do?

The mkdir command creates the folder. git init turns it into a Git repository.

The core.autocrlf input setting converts Windows-style line endings (CRLF) to Unix-style (LF) on commit. This ensures files written on Windows behave the same on macOS and Linux.

  • Confirm the repo is initialized by running:
git status

You should see a message starting with "On branch" and "No commits yet". This confirms your project folder is ready.

Seeing 'not a git repository'?

Make sure you are inside the health-check-service folder. Run pwd (macOS/Linux) or cd (Windows) to check your current directory.

If you are in the wrong folder, run cd ~/Desktop/health-check-service and try git init again. Help me fix my git init issue

Install dependencies and create the checklist

You'll create several files in this substep, so open your project folder in a text editor first. The project uses Flask for the HTTP service and the OpenTelemetry SDK for tracing, both pinned to exact versions in a requirements file.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type the name of your text editor (e.g. VS Code, Cursor, or whichever editor you use) and press Enter to open it.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Select the health-check-service folder on your Desktop, then click Open.

Don't have a text editor yet?

Any editor that can save plain text files works. If you need one, install VS Code from code.visualstudio.com. The File > Open Folder steps above work the same in VS Code, Cursor, and most code editors.

  • Create a file called requirements.txt in your health-check-service folder with this content:
flask==3.1.3
opentelemetry-sdk==1.44.0
opentelemetry-api==1.44.0

What does this file do?

Each line pins a package to an exact version. flask==3.1.3 is the web framework. opentelemetry-sdk==1.44.0 and opentelemetry-api==1.44.0 provide the tracing library.

  • Create and activate a virtual environment by running these commands:

Mac

python3 -m venv .venv
source .venv/bin/activate

Windows

python -m venv .venv
.venv\Scripts\Activate.ps1

Linux

python3 -m venv .venv
source .venv/bin/activate

You should see (.venv) at the start of your terminal prompt. This confirms the virtual environment is active.

What is a virtual environment?

A virtual environment is an isolated Python installation inside your project folder. Packages you install with pip land in .venv/ instead of your system Python.

This keeps your project's dependencies separate from every other Python project on your machine. If you close your terminal later, run the activate command again before using pip or running the service.

  • Install the packages by running:
pip install -r requirements.txt

You should see pip download and install Flask, OpenTelemetry, and their sub-dependencies. The output ends with "Successfully installed" followed by the package names.

Seeing 'pip: command not found'?

Try pip3 install -r requirements.txt instead. On some systems, the pip command for Python 3 is named pip3.

Help me fix my pip install issue

  • Create a file called TASKS.md in your health-check-service folder with this content:
# Tasks

- [ ] Setup: git init, requirements installed, standing rules committed, sealed key written
- [ ] Design: requirements brief, C4 view, build plan, four ADRs committed (AI conflation caught and corrected)
- [ ] Build: store.py and app.py committed, both processes start, /livez returns 200
- [ ] Drill and Score: /livez < 200ms, store killed, /readyz 503, /livez 200, check_spans.py PASS, audit scored, readout.html drafted
- [ ] Close Out: teach-back and AAR written, tree clean, tagged v1.0.0

---

Measured time: _____ minutes
AI coverage score: kept ____ / (kept ____ + missed ____) = ____%

What is TASKS.md?

This is your checklist for the entire project. One box per deliverable, ticked as each commit lands. The measured time and AI coverage score go at the bottom before the final commit.

  • Create a file called .gitattributes in the same folder with this single line:
* text=auto eol=lf

What does .gitattributes do here?

This forces LF line endings in the working tree on all platforms. Combined with the core.autocrlf setting, every contributor gets identical files regardless of operating system.

  • Confirm your files are in place by running:
ls

You should see requirements.txt, TASKS.md, and .gitattributes listed in the output.

Don't see .gitattributes?

Files starting with a dot are hidden by default. Run ls -a to show all files including hidden ones. You should see .gitattributes and the .git folder.

Create standing rules, environment config, and commit

The standing rules file tells any AI agent exactly what constraints to follow when working in this repo. It codifies the Twelve-Factor App principles and the health endpoint contract as machine-readable requirements.

  • Create a file called standing-rules.md in your health-check-service folder. Start with the header and twelve-factor constraints:
# Standing Rules for AI Agents

This file constrains every AI agent working in this repository.

## Twelve-Factor Constraints

1. **Codebase**: One repo, tracked in git, no vendored dependencies.
2. **Dependencies**: Declared in requirements.txt with pinned versions.
3. **Config**: Every machine-specific value comes from the environment. Missing vars crash at startup with a named error, not a silent default.
4. **Backing Services**: The store is a separate process attached via STORE_HOST and STORE_PORT. Never import it, never embed it.
5. **Build/Release/Run**: Not demonstrated in this build (single machine).
6. **Processes**: The service is stateless. No in-memory session. No written state beyond the append-only trace file.
7. **Port Binding**: The service binds SERVICE_PORT from environment.
8. **Concurrency**: Not demonstrated (single process).
9. **Disposability**: Fast startup (no warm-up phase). Crash-only design.
10. **Dev/Prod Parity**: Discussed in ADR-04, not demonstrated. This build uses a stub, not a real store.
11. **Logs**: Spans are written as an event stream (one JSON object per line to a file). No log framework, no rotation.
12. **Admin Processes**: Not applicable.

What are standing rules?

Standing rules are constraints you give to an AI before asking it to generate code. They prevent the AI from making common design mistakes like embedding secrets in code or bundling dependencies inside the service.

  • Below the twelve-factor section in the same file, add the health endpoint contract and instrumentation contract:
## Health Endpoint Contract

- `/livez` answers ONLY whether the process is running. It touches NO dependency. It returns 200 if the event loop can respond.
- `/readyz` opens a TCP socket to STORE_HOST:STORE_PORT. It returns 200 if the socket connects, 503 if it cannot. This is the readiness contract.
- These are two separate endpoints with genuinely different logic. A single /health endpoint that checks everything is explicitly rejected.

## Instrumentation Contract

- Spans are written to the file named in TRACE_FILE.
- Each span is one JSON object on one line (indent=None).
- The trace_id is inside the span's context object, not at the top level.
- One request to `/` must produce at least three spans sharing one trace_id.

Why spell out the health endpoint contract here?

AI models predictably suggest a single /health endpoint that checks everything. That conflates liveness and readiness.

Writing the split explicitly in standing rules prevents the AI from making that mistake when it generates code later.

  • Below the instrumentation contract in the same file, add the store stub contract and operator instructions:
## Store stub contract (store.py)

Implement this file as specified. Do not "improve" it.

- Use stdlib `http.server.HTTPServer` and `BaseHTTPRequestHandler` only.
- Bind `("127.0.0.1", PORT)`, not `""` or `0.0.0.0`.
- `PORT = int(os.environ.get("STORE_PORT", "9001"))`.
  The default `9001` is allowed for this stub only. Factor III (crash on missing config) applies to `app.py`, not `store.py`.
- On start, print exactly: `Store listening on 127.0.0.1:{PORT}`
- Any GET returns 200 and `{"status":"ok"}`.
- Override `log_message` so access logs are silent.
- `serve_forever()` is the intended run. The process must stay up. It is not stuck.
- Ctrl+C: `sys.exit(0)`.
- Do not add `json.dumps`, `Content-Length`, a `main()` wrapper, or a FATAL exit for missing `STORE_PORT`.

## How to run the store

`$env:STORE_PORT="9001"; python store.py`

Leave that terminal open. You should see `Store listening on 127.0.0.1:9001`.

Probe from a second terminal. Ctrl+C stops it.

Why lock the store stub in standing rules?

Factor III says "crash if a var is missing." The store snippet defaults STORE_PORT to 9001 and prints a listening line. Without this contract, an AI agent will "fix" the snippet by dropping the default and the print, making the command look hung.

The one sentence that prevents the rewrite is: Factor III applies to app.py, not store.py.

  • Save standing-rules.md and confirm it exists by running:
ls standing-rules.md

You should see the filename printed back. This confirms the file was saved in the right location.

✔️ Awesome, I've got everything!

Great. Make sure you saved your file with all five sections: Twelve-Factor Constraints, Health Endpoint Contract, Instrumentation Contract, Store stub contract, and How to run the store.

ⓧ I'd like to double check the full code

Scroll up and confirm your standing-rules.md file contains these sections in order:

  • The header: "# Standing Rules for AI Agents" and the intro sentence.
  • "## Twelve-Factor Constraints" with items 1 through 12.
  • "## Health Endpoint Contract" with three bullet points about /livez and /readyz.
  • "## Instrumentation Contract" with four bullet points about spans and trace_id.
  • "## Store stub contract (store.py)" with nine bullet points specifying the exact behavior. Confirm the line "Factor III (crash on missing config) applies to app.py, not store.py" is present.
  • "## How to run the store" with the run command, expected output, and Ctrl+C instruction.
  • Create a file called .env.example with the four environment variables the service needs:
STORE_HOST=127.0.0.1
STORE_PORT=9001
TRACE_FILE=traces.jsonl
SERVICE_PORT=8080

What is .env.example for?

This file documents every environment variable the service expects. It is committed to the repo so any collaborator knows what to set. The actual values live in the environment at runtime, never in committed code.

  • Create a file called .gitignore with these two exclusions:
.key/
traces.jsonl
  • Confirm both hidden files exist by running:
ls -a

You should see .env.example and .gitignore alongside your other files and the .git folder.

  • Create the sealed key directory by running:
mkdir .key
  • Create a file called sealed-key.md inside the .key/ folder with this placeholder content:
Sealed key file created (contents private, excluded from git via .gitignore).

What is the sealed key?

The sealed key is a file you will fill in before the AI writes code. It lists which file proves which twelve-factor principle.

After the build, you compare the AI's audit claims against this key to calculate a coverage score. It lives in .key/ which is excluded from git, so the AI never sees it during the build.

  • Verify the sealed key directory exists by running:
ls .key/

You should see sealed-key.md listed.

  • Stage and commit all your files by running these commands:
git add .
git commit -m "init: standing rules, requirements, task checklist, env example"

What does this commit include?

This bundles requirements.txt, TASKS.md, .gitattributes, standing-rules.md, .env.example, and .gitignore into one commit.

The .key/ folder is excluded by .gitignore, so the sealed key stays private and off the commit log.

Before you run the final check, what do you expect the Python import command to print if both packages installed correctly?

  • Verify your full setup by running:

Mac

python3 -c "from importlib.metadata import version; print(version('flask'), version('opentelemetry-sdk'))"

Windows

python -c "from importlib.metadata import version; print(version('flask'), version('opentelemetry-sdk'))"

Linux

python3 -c "from importlib.metadata import version; print(version('flask'), version('opentelemetry-sdk'))"

You should see 3.1.3 1.44.0 printed. This confirms both Flask and OpenTelemetry installed at the correct versions.

  • Confirm your commit landed by running:
git status

You should see "nothing to commit, working tree clean". Your project folder is fully initialized with one commit.

Seeing 'ModuleNotFoundError'?

The pip install may have targeted a different Python version than the one you are checking. Try running python3 -m pip install -r requirements.txt to ensure the packages land in the correct Python's site-packages.

Help me fix my import error after pip install

Your environment is ready and your constraints are committed. Next up, you will prompt the AI to draft the design artifacts that define what the service must do.

Design with AI

Your project folder is initialized, your dependencies are installed, and your standing rules are committed. Now you need design decisions on paper before any code exists.

Every implementation choice in this project traces back to a numbered acceptance criterion and a decision record. The AI will draft these documents, but your job is design reviewer. AI models predictably conflate liveness and readiness into one endpoint. Catching that is the work.

In this step, get ready to:
  • Prompt the AI to draft a requirements brief, four Architecture Decision Records, a C4 container view, and a build plan.
  • Review and correct the AI's conflation of liveness with readiness.
  • Commit the corrected design artifacts to your repository.
Create the docs folder

The corrected documents need to land in a docs/ folder inside your project. Each document gets its own file.

  • Create the docs/ folder inside your health-check-service folder by running:
mkdir docs
  • Copy the AI's corrected output into six files. Use your text editor to create each one inside docs/:
  • docs/requirements-brief.md containing the numbered acceptance criteria.
  • docs/c4-container.md containing the C4 container view showing the Flask service and store stub processes.
  • docs/build-plan.md containing the five-task plan with minute budgets.
  • docs/adr-01-readiness-contract.md through docs/adr-04-spans-to-file.md containing the four corrected ADRs.
Prompt the AI for design artifacts

The standing rules you committed in Step 1 constrain what the AI can produce. You will feed them directly to your AI client so every generated document obeys the same contract.

  • Open your AI client (Claude, ChatGPT, or a local model).
  • Copy the entire contents of standing-rules.md and paste it into a new conversation.
  • Below the pasted rules, add the following prompt:
Based on these standing rules, produce the following:

1. A requirements brief with numbered acceptance criteria for a health-check service that implements these constraints.

2. Four Architecture Decision Records:
   - ADR-01: Readiness Contract (liveness vs readiness)
   - ADR-02: Config from Environment
   - ADR-03: Store as Separate Process
   - ADR-04: Spans to File

Each ADR must contain: Title, Status (Accepted), Context, Decision, Alternative Rejected, Tradeoff Accepted, Reversal Trigger.

3. A C4 container view showing the processes and their connections.

4. A build plan with five tasks and minute budgets summing to 30.

What is an Architecture Decision Record?

An ADR is a short document that records a single design decision: what you chose, what you rejected, and what would make you reverse it. Four ADRs means four decisions locked before any code is written.

  • Send the prompt and wait for the AI to generate its response.
# Requirements Brief
AC-1: The service crashes at startup if STORE_HOST, STORE_PORT, TRACE_FILE, or SERVICE_PORT is missing.
AC-2: /livez returns 200 without contacting any dependency.
AC-3: /readyz returns 200 if the store is reachable via TCP, 503 if not.
...

# ADR-01: Readiness Contract
Title: Readiness Contract
Status: Accepted
Context: ...
Decision: /livez checks process health only. /readyz checks store via TCP socket.
Alternative Rejected: A single /health endpoint.
Tradeoff Accepted: Two endpoints to maintain instead of one.
Reversal Trigger: ...

# ADR-02, ADR-03, ADR-04 (same seven-field structure)

# C4 Container View
[User] --> [Flask Service :8080] --> [Store Stub :9001]
                |-> traces.jsonl

# Build Plan
1. Checklist and decision records (5 min)
2. Config to environment (4 min)
3. Endpoints and spans (6 min)
4. Drill, score, readout (5 min)
5. Close-out (10 min)
Total: 30 min

Before you review the AI's output, ask yourself: did the AI distinguish between liveness and readiness with genuinely different logic, or did it merge them into one endpoint?

Catch the conflation and correct the ADRs

Look at the AI's ADR-01 output now. Most AI models will propose a single /health endpoint that checks everything, or they will put the store dependency check inside liveness.

This conflation is exactly the defect this project exists to fix. When liveness checks dependencies and the store blips for 30 seconds, every pod restarts simultaneously. That is a restart storm, not graceful degradation.

Why is this conflation dangerous?

In Kubernetes, a failed liveness probe restarts the container. A failed readiness probe removes it from traffic but keeps it running.

If liveness checks the database and the database blips, every container restarts at once. That turns a 30-second outage into a full service collapse. The split means the service stays alive and simply stops receiving new traffic until the store recovers.

  • If the AI proposed a single /health endpoint or placed the store check inside liveness, send this correction:
ADR-01 must explicitly distinguish liveness from readiness:
- /livez checks ONLY process health. It touches NO dependency. Returns 200 if the process can respond.
- /readyz opens a TCP socket to the store. Returns 200 if reachable, 503 if not.
These are two separate endpoints with genuinely different logic. A single /health endpoint is rejected. Revise ADR-01 to reflect this split.
  • Send the correction and confirm the AI revises ADR-01 to separate the two endpoints completely.
  • Verify each ADR contains all seven fields: Title, Status (Accepted), Context, Decision, Alternative Rejected, Tradeoff Accepted, Reversal Trigger.
  • Verify the build plan lists five tasks with minute budgets summing to 30: checklist and decision records (5 min), config to environment (4 min), endpoints and spans (6 min), drill and score and readout (5 min), close-out (10 min).
# ADR-01: Readiness Contract

Title: Readiness Contract (Liveness vs Readiness)
Status: Accepted
Context: Health checks must distinguish between "process is alive" and "process can serve traffic."
Decision: Two separate endpoints with genuinely different logic.
  - /livez returns 200 if the process can respond. It touches NO dependency.
  - /readyz opens a TCP socket to STORE_HOST:STORE_PORT. Returns 200 if reachable, 503 if not.
Alternative Rejected: A single /health endpoint that checks everything.
Tradeoff Accepted: Two endpoints to maintain instead of one.
Reversal Trigger: If the orchestrator does not support separate liveness/readiness probes.
Save and commit the design artifacts
  • Confirm all six files exist by running:
ls docs/

You should see: adr-01-readiness-contract.md, adr-02-config-from-environment.md, adr-03-store-as-process.md, adr-04-spans-to-file.md, build-plan.md, c4-container.md, and requirements-brief.md.

Missing a file?

Make sure you saved each file inside the docs/ folder, not in the project root. Check that the filenames match exactly (lowercase, hyphens, .md extension).

Help me figure out why my docs/ folder is missing files.

  • Open TASKS.md in your text editor.
  • Change the Setup line from - [ ] to - [x] to mark Setup complete.
  • Change the Design line from - [ ] to - [x] to mark Design complete.
  • Save TASKS.md.
  • Commit all design artifacts by running these commands:
git add docs/ TASKS.md
git commit -m "design: requirements brief, C4 view, build plan, ADRs 01-04"

What does this commit do?

This commit locks the design decisions into version history before any code exists. When you implement in the next step, every choice traces back to a committed ADR.

Before you verify, what do you expect git log --oneline to show?

  • Verify your commit history by running:
git log --oneline

You should see at least two commits: the init: commit from Step 1 and the design: commit you just made.

Only seeing one commit?

If the git commit command said "nothing to commit," make sure you ran git add docs/ TASKS.md first. Check that your files are inside docs/ and not the project root.

Help me figure out why my git commit did not work.

Your design decisions are locked in version history. Next up, you will implement the service against these ADRs and prove that /livez and /readyz actually behave differently when the store dies.

Build the Service

The design is done. Four ADRs define what the service must do, and the standing rules constrain how it does it. Now the AI writes code against those artifacts.

The service must enforce three twelve-factor principles at runtime. Factor III means missing environment variables crash the process at startup with a named error. Factor IV means the store is reached over a TCP socket, never imported. Factor XI means spans are written as one JSON object per line to a file.

In this step, get ready to:
  • Prompt the AI to implement store.py and app.py against the standing rules.
  • Review the AI's output for three specific violations and correct them.
  • Start both processes and confirm /livez returns 200.
Prompt the AI to implement both files

Your AI needs the standing rules as context and a clear specification of what to build. The prompt should name both files and their responsibilities.

  • Switch back to your AI client.
  • Paste the full contents of standing-rules.md into the conversation as context.
  • Below the pasted rules, add the following prompt:
Based on the standing rules above, produce two Python files:

1. store.py
   - A stdlib http.server.HTTPServer binding to STORE_PORT from the environment.
   - Returns 200 with {"status":"ok"} on any GET.
   - Suppresses access logs (override log_message).
   - Exits cleanly on Ctrl+C.

2. app.py
   - A Flask app that reads STORE_HOST, STORE_PORT, TRACE_FILE, SERVICE_PORT from os.environ.
   - Exits with a fatal message at module level if any variable is missing (Factor III).
   - Configures OpenTelemetry with ConsoleSpanExporter writing to the trace file.
   - The formatter must use span.to_json(indent=None) + "\n" so each span is one line.
   - Three routes:
     /livez — returns 200 with {"status":"alive"}. Touches NO dependency.
     /readyz — opens a TCP socket to STORE_HOST:STORE_PORT with a 1-second timeout. Returns 200 if reachable, 503 if not.
     / — produces at least three nested spans sharing one trace_id.
   - Runs Flask on SERVICE_PORT.
The AI should give you two complete Python files.

store.py (short, stdlib only):
  - Reads STORE_PORT from environment
  - Returns 200 {"status":"ok"} on GET
  - Suppresses access logs
  - Handles Ctrl+C cleanly

app.py (longer, Flask + OpenTelemetry):
  - Module-level config crash using os.environ.get, empty counts as missing (not inside a route)
  - ConsoleSpanExporter with formatter using indent=None (not indent=4)
  - /livez returns 200 with ZERO references to the store, tracer, or TRACE_FILE
  - /readyz opens TCP socket to STORE_HOST:STORE_PORT
  - / produces 3 nested spans sharing one trace_id
  - Runs on SERVICE_PORT

CRITICAL CHECK: If /livez calls _check_store(), references STORE_HOST, or wraps in tracer.start_as_current_span, that violates the liveness contract. Fix it.
Review the AI's code and save store.py

The store stub is small enough to verify in one pass. Compare the AI's output against this reference and correct any differences.

  • Create store.py in your health-check-service folder with the corrected code:
"""Backing store stub: stdlib HTTP server bound to STORE_PORT from environment."""
import os
import sys
from http.server import HTTPServer, BaseHTTPRequestHandler

PORT = int(os.environ.get("STORE_PORT", "9001"))


class StoreHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200)
        self.send_header("Content-Type", "application/json")
        self.end_headers()
        self.wfile.write(b'{"status":"ok"}')

    def log_message(self, format, *args):
        pass


if __name__ == "__main__":
    server = HTTPServer(("127.0.0.1", PORT), StoreHandler)
    print(f"Store listening on 127.0.0.1:{PORT}")
    try:
        server.serve_forever()
    except KeyboardInterrupt:
        sys.exit(0)

What does this code do?

  • PORT reads from the environment with a fallback to 9001. This is the store's port, not the service's.
  • log_message is overridden to suppress access logs. Without this, every health check floods the terminal.
  • The KeyboardInterrupt handler exits cleanly on Ctrl+C. You will use that kill to prove the readiness contract in Step 4.
  • Save store.py.
  • Start the store in your terminal by running this command:

Mac

STORE_PORT=9001 python store.py

Windows

$env:STORE_PORT="9001"; python store.py

You should see Store listening on 127.0.0.1:9001 printed. The terminal is now occupied by this process. Leave it running.

Don't see the listening message?

  • If you see Address already in use, another process is using port 9001. Close any other servers you have running.
  • If you see a SyntaxError, confirm you are running Python 3.11 or newer. The f-string syntax requires it.
  • Why won't my store start?
Review the AI's app.py and verify three critical patterns

The service file is larger and the AI is more likely to violate the standing rules. Check these three patterns before saving.

Pattern 1: The config crash must happen at module level, not inside a route. The exact idiom can vary (a helper function, a list comprehension, or direct checks), but two things are non-negotiable: it uses os.environ.get and treats empty strings as missing. Look for something like this near the top of the file:

# Factor III: crash on missing config
def _require(name):
    value = os.environ.get(name)
    if not value:
        print(f"FATAL: missing required environment variable {name}", file=sys.stderr)
        sys.exit(1)
    return value

STORE_HOST = _require("STORE_HOST")
STORE_PORT = int(_require("STORE_PORT"))
TRACE_FILE = _require("TRACE_FILE")
SERVICE_PORT = int(_require("SERVICE_PORT"))

Pattern 2: The formatter must pass indent=None. If the AI omits this argument, the default indent=4 produces multi-line JSON that breaks the span assertion script later.

exporter = ConsoleSpanExporter(
    out=trace_file,
    formatter=lambda span: span.to_json(indent=None) + "\n",
)

Pattern 3: The /livez route must contain zero references to the store and zero tracer or TRACE_FILE I/O. No _check_store() call, no STORE_HOST, no socket, no tracer.start_as_current_span. It returns 200 if the process can respond. @app.get vs @app.route does not matter.

# @app.get works identically to @app.route
@app.route("/livez")
def livez():
    """Liveness: process is running. Touches no dependency."""
    return jsonify({"status": "alive"}), 200

Why must /livez ignore the store?

In Kubernetes, if the liveness probe fails, the container is restarted. If liveness checks the database and the database blips for 30 seconds, every pod restarts simultaneously. That turns a transient outage into a restart storm.

Readiness failing only removes the pod from receiving traffic. The pod stays alive and recovers when the dependency comes back. That is the split ADR-01 defines.

  • Correct any violations in your AI's output, then create app.py in your health-check-service folder. The full file should contain these sections in order:
  • Imports: os, sys, socket, Flask's jsonify, and the OpenTelemetry SDK modules (TracerProvider, SimpleSpanProcessor, ConsoleSpanExporter, Resource, SERVICE_NAME).
  • Config crash: a module-level check using os.environ.get with sys.exit() that treats empty as missing, followed by the four variable bindings.
  • OpenTelemetry setup: Resource, TracerProvider, file-backed ConsoleSpanExporter with the indent=None formatter, and a SimpleSpanProcessor.
  • Flask app with _check_store() helper (TCP socket connect with 1-second timeout).
  • Three routes: /livez (no dependency, no tracer span), /readyz (socket check), / (three nested spans).
  • The if __name__ == "__main__" block running Flask on SERVICE_PORT.
  • Save app.py.

Before starting the service, confirm the config crash works. Open a new terminal (keep the store running in the first one) and run the service without any environment variables:

Mac

python app.py

Windows

python app.py

You should see a FATAL message naming a missing environment variable and the process exits immediately. That is Factor III working.

Service started instead of crashing?

  • If the service starts without crashing, the config validation is inside a route function instead of at module level. Move the config check (the _require helper or equivalent) above the Flask app definition so it runs on import.
  • If you have leftover environment variables from earlier work, they may satisfy the check. Run env | grep STORE (bash/zsh) or Get-ChildItem Env:STORE* (PowerShell) to check.
  • Help me fix the config crash in app.py.
Start the service and verify /livez

Now start the service with all four environment variables set. Use the same terminal where the config crash just ran.

  • Start the service by running this command:

Mac

STORE_HOST=127.0.0.1 STORE_PORT=9001 TRACE_FILE=traces.jsonl SERVICE_PORT=8080 python app.py

Windows

$env:STORE_HOST="127.0.0.1"; $env:STORE_PORT="9001"; $env:TRACE_FILE="traces.jsonl"; $env:SERVICE_PORT="8080"; python app.py

You should see Flask's startup message indicating the service is running on port 8080. Leave this terminal running too.

Before you send the request: do you expect /livez to return 200 or 503 right now? Why?

  • Open a third terminal (keep both the store and service running in their own terminals). Request /livez by running:

Mac

curl http://localhost:8080/livez

Windows

curl.exe http://127.0.0.1:8080/livez

You should see {"status":"alive"} with HTTP 200. Liveness passes because it only checks that the process can respond. The store being up or down is irrelevant to this endpoint.

Connection refused on port 8080?

  • Check that the service terminal shows Flask running. If it crashed, look for a traceback.
  • If you see an ImportError for Flask or OpenTelemetry, confirm you installed the packages from requirements.txt in Step 1.
  • My service won't start on port 8080. Help me debug.

✔️ Awesome, I've got everything!

Great. Double check that both store.py and app.py are saved before committing.

ⓧ I'd like to double check the full code

store.py should match the 25-line reference shown in the previous substep (the full file is there).

app.py should contain these sections in order. Scroll up and confirm each one is present:

  • Module docstring and imports (os, sys, socket, Flask, OpenTelemetry modules).
  • A module-level config crash that exits on missing or empty environment variables (Pattern 1 above).
  • Four os.environ reads assigning STORE_HOST, STORE_PORT, TRACE_FILE, SERVICE_PORT.
  • OpenTelemetry setup with ConsoleSpanExporter using indent=None formatter (Pattern 2 above).
  • Flask app, _check_store() helper with TCP socket and 1-second timeout.
  • Three routes: /livez with no store reference and no tracer span (Pattern 3 above), /readyz with nested check_store span, and / with three nested spans.
  • The if __name__ == "__main__" block calling app.run(host="0.0.0.0", port=SERVICE_PORT).
Commit the implementation
  • In your third terminal (the one you used for curl), commit both files by running:
git add store.py app.py
git commit -m "feat: service with /livez, /readyz, store stub, OTel spans to file"
  • Open TASKS.md and tick the Build box by changing - [ ] Build to - [x] Build.
  • Save TASKS.md.

Confirm the commit landed by running:

git log --oneline -1

You should see a commit with the message feat: service with /livez, /readyz, store stub, OTel spans to file.

Commit says 'nothing to commit'?

  • Confirm store.py and app.py are saved in your health-check-service folder (not a subfolder or a parent directory).
  • Run git status to see which files git sees as new or modified.
  • My git commit isn't working. Help me debug.

Both processes are running and liveness is confirmed. Next up, you will kill the store and prove that readiness fails while liveness holds steady.

Prove the Drill, Score the AI, Draft the Readout

Your service has two health endpoints with genuinely different logic committed and running. But code that exists is not code that works. The only proof is a drill: kill the dependency and watch what each endpoint reports.

A service that returns 200 when its store is dead is the exact defect this project exists to prevent. In this step, you'll time liveness, kill the store, confirm the endpoint split holds, verify instrumentation with a deterministic assertion script, and score the AI's audit against your sealed key.

In this step, get ready to:
  • Time /livez and generate trace spans for verification.
  • Kill the store and prove readiness returns 503 while liveness stays 200.
  • Score the AI's audit against your sealed key and draft the stakeholder readout.
Time liveness and generate trace spans

You need two measurements from your running service. First, how fast /livez responds (it must be under 200ms to prove there is no dependency check slowing it down). Second, a request to / that generates three nested spans in the trace file.

  • In Terminal 3 (a new terminal window in your health-check-service folder), time a request to /livez by running the command for your shell:

Mac

time curl -s http://localhost:8080/livez

Check the real line in the output. It should show a value under 0.200s.

Windows

Measure-Command { curl.exe -s http://127.0.0.1:8080/livez }

Check the TotalMilliseconds value in the output. It should be under 200.

Why under 200ms matters

A liveness endpoint that touches no dependency should respond in single-digit milliseconds. Anything near 200ms suggests a network call snuck in. Under 200ms confirms the endpoint is purely local to the process.

  • Request / once to generate trace spans by running:

Mac

curl http://localhost:8080/

Windows

curl.exe http://127.0.0.1:8080/

The response body depends on how your AI implemented the / route. As long as the request completed without an error, it generated trace spans in the file.

  • Confirm spans were written to the trace file by running:

Mac

cat traces.jsonl

Windows

Get-Content traces.jsonl

You should see multiple lines of compact JSON, each containing a trace_id inside a context object. Each line is one span.

Seeing an empty file or no file?

If traces.jsonl does not exist or is empty, confirm your service is still running in your service terminal. The trace file is only written when the service handles a request.

Also verify your TRACE_FILE environment variable is set to traces.jsonl in the terminal where the service runs.

help me debug why traces.jsonl is empty or missing

Kill the store and prove the endpoint split

This is the central assertion of the entire project. When the backing store dies, readiness must fail while liveness stays healthy. That single behavioral split is what lets an orchestrator drain traffic gracefully instead of killing and restarting every pod in a loop.

  • Switch to your store terminal (the one showing Store listening on 127.0.0.1:9001).
  • Press Ctrl+C to kill store.py. You should see the prompt return.
  • Confirm your service terminal still shows app.py running (no prompt, no traceback). Do not press Ctrl+C in that terminal.

Before you request /readyz, predict: what HTTP status code do you expect back now that the store is dead? What about /livez?

Both endpoints say Connection refused?

Connection refused on port 8080 means app.py is not running. You stopped the wrong process. The drill requires store.py dead and app.py alive.

Restart the service in your service terminal, then retry the curl commands:

help me restart app.py for the store-kill drill

  • In your curl terminal, request /readyz by running:

Mac

curl -i http://localhost:8080/readyz

Windows

curl.exe -i http://127.0.0.1:8080/readyz

The output starts with HTTP/1.1 503 SERVICE UNAVAILABLE. The exact JSON body varies depending on how your AI wrote the route, but the 503 status code is the contract. The TCP socket to the store failed, so readiness correctly reports the service cannot serve traffic.

  • Now request /livez in the same curl terminal by running:

Mac

curl -i http://localhost:8080/livez

Windows

curl.exe -i http://127.0.0.1:8080/livez

The output starts with HTTP/1.1 200 OK and the body contains {"status":"alive"}. The store is dead, but liveness is unaffected because it touches no dependency. That is the proof.

Why this split matters in production

In Kubernetes, a failed liveness probe triggers a container restart. A failed readiness probe removes the pod from the service's endpoint list without restarting it.

If liveness checks the database and the database blips for 30 seconds, every pod restarts simultaneously. That is a restart storm. By keeping liveness dependency-free, the orchestrator knows the process is fine and simply stops routing traffic until readiness recovers.

Now you need a deterministic script to prove the spans in your trace file are valid. The script asserts that at least three spans share one trace_id, confirming a single request produced the expected nesting.

  • Create check_spans.py in your health-check-service folder with the following code:
"""Assert at least 3 spans share one trace_id. Exits non-zero on failure."""
import json, sys, collections
spans = [json.loads(line) for line in open(sys.argv[1])]
trace_ids = collections.Counter(s["context"]["trace_id"] for s in spans)
shared = {tid: c for tid, c in trace_ids.items() if c >= 3}
assert shared, f"FAIL: no trace_id with 3+ spans. Got: {dict(trace_ids)}"
print(f"PASS: {len(shared)} trace(s) with 3+ spans")

What does this script do?

It reads every line of the trace file, parses each as JSON, and counts how many spans share each trace_id. If no trace_id has three or more spans, the assertion fails and the script exits non-zero.

A passing result proves that your / route produced the three nested spans defined in app.py.

  • Save check_spans.py and run it against your trace file:
python check_spans.py traces.jsonl

You should see a line starting with PASS. The number of traces depends on how many requests you sent to /. As long as at least one trace has 3+ spans, the script exits with code 0 and the assertion holds.

Seeing FAIL or an assertion error?

If the script reports no trace_id with 3+ spans, your / route may not have been requested while the service was running. Re-start the service (the store can stay dead for this check), request / once, then re-run the script.

If you see a JSON decode error, the trace file may contain multi-line JSON. Check that the formatter in app.py passes indent=None (not the default indent=4). Delete traces.jsonl, restart the service, request /, and try again.

help me debug why check_spans.py is failing

✔️ Awesome, I've got everything!

Great. Make sure you saved check_spans.py and confirmed PASS in the output.

ⓧ I'd like to double check the full code

"""Assert at least 3 spans share one trace_id. Exits non-zero on failure."""
import json, sys, collections
spans = [json.loads(line) for line in open(sys.argv[1])]
trace_ids = collections.Counter(s["context"]["trace_id"] for s in spans)
shared = {tid: c for tid, c in trace_ids.items() if c >= 3}
assert shared, f"FAIL: no trace_id with 3+ spans. Got: {dict(trace_ids)}"
print(f"PASS: {len(shared)} trace(s) with 3+ spans")
Score the AI audit and draft the stakeholder readout

The drill is proven. Now you score how well your AI understands what was built. The sealed key you wrote before any code existed lists which file proves which twelve-factor principle. You will compare the AI's claims against it.

  • Open your AI client and send the following prompt:
Here are the files in my project: store.py, app.py, standing-rules.md, and the four ADRs in docs/.

For each file, answer:
1. Which of the five adjudicable twelve-factor principles (III Config, IV Backing Services, VI Processes, VII Port Binding, XI Logs) does this file satisfy? Name the specific line or pattern that proves it.
2. Does this file contribute to proving the readiness contract (liveness vs readiness split)? If so, how?
  • Compare the AI's output against your sealed key in .key/sealed-key.md.
## app.py
- Factor III (Config): REQUIRED_ENV check at line 12 crashes on missing env vars.
- Factor IV (Backing Services): _check_store() uses socket.connect() to reach store at STORE_HOST:STORE_PORT.
- Factor VII (Port Binding): app.run(port=SERVICE_PORT) binds from environment.
- Factor XI (Logs): ConsoleSpanExporter writes one JSON object per line to TRACE_FILE.
- Readiness Contract: /livez (line 35) returns 200 with no store reference. /readyz (line 42) checks TCP socket.

## store.py
- Factor VII (Port Binding): HTTPServer binds to STORE_PORT from environment.

## standing-rules.md
- Documents the liveness/readiness split as a constraint.

(Your output will differ. Score each claim against your sealed key.)
  • Create a new file called audit-score.md inside the docs/ folder of your health-check-service project.
  • Paste the following template into docs/audit-score.md:
# AI Audit Score

## Counts
- Claims kept:
- Claims rejected:
- Claims missed:
- Coverage: (kept / (kept + rejected + missed)) x 100 = %

## Notes
<!-- Optional: note any surprising claims or gaps your sealed key caught. -->

What counts as kept, rejected, and missed?

Kept means the AI's claim matches your sealed key. Rejected means the AI made a claim your sealed key does not support. Missed means your sealed key lists a principle that the AI did not mention.

  • Score each AI claim against your sealed key and fill in the counts.
  • Calculate the coverage percentage: kept ÷ (kept + rejected + missed) × 100.
  • Save docs/audit-score.md.

Your coverage number is personal to your sealed key and your AI's output. There is no universal right answer. What matters is that you adjudicated each claim rather than accepting them at face value.

  • Send your AI the following prompt to draft the stakeholder readout:
Draft a file called readout.html for this health-check service project.

Requirements:
- Self-contained: all styling inline, no external stylesheet, no CDN links.
- Two audience sections:
  1. Engineering leadership: the endpoint split (/livez vs /readyz), the config boundary (Factor III crash on missing env), and the two failure modes designed out (restart storm, silent default).
  2. Stakeholders: what an on-call engineer can now see at 3AM, what it costs to run (nothing), and one plain sentence on what the build does not cover.
- An honesty note stating: this proves the contract on one machine against a stub, not against a production deployment target.
  • Save the AI's output as readout.html inside your health-check-service folder (the same level as app.py and store.py).
  • Open readout.html by double-clicking it in Finder (macOS) or File Explorer (Windows). Confirm it renders correctly with no broken styles or missing content.
  • Tick the Drill and Score box in TASKS.md.
  • Commit everything by running:
git add check_spans.py docs/audit-score.md readout.html TASKS.md
git commit -m "prove: drill passed, spans verified, audit scored, readout drafted"

What does this commit prove?

This single commit bundles four distinct proofs: timing under 200ms, the store-kill drill (503 vs 200), deterministic span verification, and a scored AI audit. Anyone cloning this repo can re-run check_spans.py and open readout.html to verify the claims.

  • Confirm the commit landed by running:
git log --oneline

You should see your new commit at the top with the message prove: drill passed, spans verified, audit scored, readout drafted.

Git complaining about untracked files?

If git reports untracked files you did not expect, check that .gitignore still contains .key/ and traces.jsonl. Those files should never be committed.

help me fix my git commit for the drill step

The drill is done and the proof is committed. Next up, you'll close the folder: defend each decision in a teach-back, write an honest after-action review, and tag the final version.

Close Out the Project

The drill is proven. The AI is scored. The readout is drafted. Everything that demonstrates the readiness contract is committed and visible.

This step carries nothing but close-out. If time was short earlier, build scope was cut to protect this moment. You will defend each decision aloud, answer four honest questions about what happened, record your numbers, and close the folder clean.

In this step, get ready to:
  • Write a teach-back defending four decisions in sixty seconds each.
  • Write an after-action review answering four honest questions.
  • Record measured time and AI score, commit, and tag the final version.
Write the teach-back

A teach-back is a sixty-second spoken defense of a decision, delivered to one person who is allowed to ask "why not the alternative?" Writing it forces you to articulate the tradeoff, not just the choice.

  • Send your AI the following prompt (or write it yourself):
Create a file called docs/teach-back.md for this health-check service project.

Write a sixty-second spoken defense for each of these four decisions. For each one, name what was chosen, what alternative it beat, and why the tradeoff is worth it:

1. Split endpoints (/livez and /readyz) over a single /health endpoint.
2. Config from environment variables over a committed config file.
3. A local process (store.py) over a container for the backing store.
4. Spans to a file over a vendor telemetry agent.
# Teach-Back: Four Decisions

## 1. Split endpoints over one
Chosen: /livez and /readyz as separate endpoints.
Beat: A single /health endpoint that checks everything.
Why: A shared endpoint turns a 30-second store blip into a restart storm because the orchestrator kills every pod whose liveness fails.

## 2. Environment over committed file
Chosen: Config from os.environ with crash on missing.
Beat: A committed config.json or .env file.
Why: A committed file ties the service to one machine. Environment variables let the same image run anywhere.

## 3. Local process over container
Chosen: store.py as a separate process you can kill with Ctrl+C.
Beat: Running the store in a Docker container.
Why: A killable process is a truer dead dependency than editing a connection string.

## 4. File over vendor agent
Chosen: Spans written to traces.jsonl.
Beat: Sending spans to a vendor telemetry agent.
Why: A file is zero-cost, zero-dependency, and greppable. No agent to install or configure.
  • Save the AI's output to docs/teach-back.md in your health-check-service folder.
  • Confirm the file exists by running:
ls docs/teach-back.md

You should see docs/teach-back.md listed without an error.

File not found?

Make sure you saved the file inside the docs/ directory that already exists from earlier steps. If you accidentally saved it at the project root, move it with mv teach-back.md docs/.

help me find where my teach-back file ended up

Write the after-action review

An after-action review answers four questions with honesty, not optimism. It exists so next time you have a written record of what actually happened.

  • Send your AI the following prompt (or write it yourself):
Create a file called docs/after-action-review.md for this health-check service project.

Answer these four questions honestly:

1. What was meant to happen? (Prove the readiness contract on one machine.)
2. What actually happened? (Record the real result from your drill.)
3. Why does the gap exist? (Name it if there is one. If the drill passed cleanly, say so.)
4. One thing to change next time. (Name it concretely.)

Include an honesty note at the end: this build proves the config boundary, the readiness contract, and the instrumentation on one machine against a stub. It proves nothing about a real deployment target or a real database driver. Dev-to-prod parity is discussed but not demonstrated.
# After-Action Review

## 1. What was meant to happen?
Prove the readiness contract on one machine: /readyz returns 503 when the store is dead, /livez holds at 200.

## 2. What actually happened?
The drill passed. /readyz returned 503 after killing the store. /livez returned 200 throughout. Timing was under 200ms. check_spans.py printed PASS.

## 3. Why does the gap exist?
No gap — the drill passed cleanly. (Or: name the specific issue you hit.)

## 4. One thing to change next time
Write the sealed key with more specific line numbers instead of file-level claims.

## Honesty Note
This build proves the config boundary, the readiness contract, and the instrumentation on one machine against a stub. It proves nothing about a real deployment target or a real database driver. Dev-to-prod parity is discussed but not demonstrated.
  • Save the output to docs/after-action-review.md.
  • Confirm both new files exist by running:
ls docs/teach-back.md docs/after-action-review.md

You should see both filenames printed without errors.

Record your numbers, commit, and tag

The build is done. What remains is recording the truth: how long it took and how well the AI performed against your sealed key. Then the folder closes clean.

  • Open TASKS.md in your text editor.
  • Tick the final checkbox so it reads:
- [x] Close Out: teach-back and AAR written, tree clean, tagged v1.0.0
  • At the bottom of TASKS.md, fill in your actual measured build time (in minutes from when you started Step 1) and your AI coverage score from docs/audit-score.md.
  • Save TASKS.md.

Why record the real numbers?

The measured time and coverage score make the project self-documenting. Anyone who opens this folder later (including future you) can see exactly how long it took and how well the AI performed. Estimates lie. Measurements don't.

  • Commit everything and tag the final version by running these commands:
git add .
git commit -m "close: teach-back, AAR, all tasks complete"
git tag v1.0.0

What do these commands do?

The git add . stages all remaining changes. The commit records them with a meaningful message. The tag v1.0.0 marks this exact commit so you can recover this version at any point in the future.

Pushing the folder to a remote is optional. It happens on your own time, and nothing in this build depends on it.

Before you verify, what do you expect git status to show? Think about whether any files remain untracked or modified.

  • Verify the folder is closed by running:
git status

You should see nothing to commit, working tree clean. Every file is committed. Nothing is left dangling.

  • Confirm the tag exists by running:
git tag

You should see v1.0.0 printed. This is the recoverable version of your complete build.

Tree not clean?

If git status shows untracked or modified files, check whether you saved all your edits to TASKS.md before running git add .. Run git add . and git commit --amend --no-edit to fold the remaining changes into your last commit.

If traces.jsonl shows as untracked, that is expected. It is listed in .gitignore and should not appear in status at all. If it does, double-check your .gitignore contains the line traces.jsonl.

help me get a clean git status

The folder is closed. Every box is ticked. The elapsed time is written down. The tag marks an exact version you can recover at any time.

Secret mission

Add Graceful Shutdown Drain

Your service can already say 'I am sick' when the store dies. But what about when the service itself wants to leave? Right now, Ctrl+C kills the process instantly. Traffic in flight gets dropped. Add a graceful shutdown drain so the service flips readiness to 503 before it exits, giving the orchestrator a 2-second window to stop sending new requests.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources on disk, stop running processes to free up memory, or delete everything entirely.

Resources you used:

  • The health-check-service project folder on your Desktop (all code, docs, git history, and trace data).
  • Python packages installed via pip: flask, opentelemetry-sdk, opentelemetry-api.
  • Running processes (store.py and app.py) if still active in your terminals.

Keep everything running

No action needed. Choose this if you want to keep experimenting with the service or extend it further.

  • Your project folder stays on your Desktop.
  • Your pip packages remain installed.
  • If you closed your terminals, the store and service processes already stopped on their own.

Pause - I'll come back to this later

Stop the running processes to free up memory. Your files and packages stay on disk for next time.

  • Press Ctrl+C in the terminal running the service (app.py). The graceful drain handler will shut it down after 2 seconds.
  • Press Ctrl+C in the terminal running the store (store.py).
  • Close both terminal windows.
  • Your project folder and installed packages remain untouched. Next time, open a terminal in ~/Desktop/health-check-service and start both processes again.

Delete - I don't want to use this again

Remove all project files and uninstall the packages. This is irreversible.

  • Stop any running processes by pressing Ctrl+C in both terminal windows, then close them.
  • Delete the project folder by running this command (macOS/Linux):
rm -rf ~/Desktop/health-check-service

On Windows, run this in PowerShell instead:

Remove-Item -Recurse -Force "$env:USERPROFILE\Desktop\health-check-service"
  • Uninstall the Python packages by running:
pip uninstall flask opentelemetry-sdk opentelemetry-api -y
  • Confirm the folder is gone by running ls ~/Desktop (macOS/Linux) or ls $env:USERPROFILE\Desktop (Windows). You should no longer see health-check-service in the output.

Nice Work!

Nice Work!

You built a service that can say "I am sick" before it is dead, proved it by killing a real process, and defended every decision in writing.

You learned how to:

  • Build a Flask service with separate /livez and /readyz endpoints that respond differently when a dependency dies, proven by killing a real process and observing the behavioral split.
  • Score an AI's design audit against a sealed key, measuring coverage as kept / (kept + missed) rather than trusting claims at face value.
  • Configure OpenTelemetry to write single-line JSON spans to a file and verify instrumentation with a deterministic assertion script that exits non-zero on failure.
  • Secret Mission: Added graceful shutdown drain so the service announces it is leaving before it is gone.

Ready to quiz yourself?