Test Legacy Billing Code with pytest

Build a deterministic pytest suite for a legacy billing workflow.

Introduction

30 Second Summary

A billing change can look safe in one function while quietly breaking retries, reports, or configuration elsewhere. In an unfamiliar codebase, a green test suite earns trust by exercising those connections.

In this project, you will test a prepared legacy billing processor with pytest. You will build a deterministic nine-test suite around its business rules and side effects.

What You'll Build

You run the finished suite and watch nine tests prove the billing workflow across valid payments, rejected invoices, exhausted retries, warning logs, CSV reports, and cached settings.

By the end of this project, you'll have:

  • A boundary matrix you can use to explain exactly which invoice amounts pass or fail.
  • A component-level billing test that runs the real service and CSV writer against isolated temporary files while replacing only the payment boundary.
  • A deterministic test suite that proves retry behavior, warning logs, gateway call contracts, and protection from cached configuration leaks.
  • Secret Mission: Prove a mixed batch containing a successful payment, a validation rejection, and a permanent gateway failure in input order.

Are there any prerequisites?

You should be comfortable reading Python functions, classes, imports, exceptions, and basic pytest tests. The prepared billing application keeps your focus on test strategy inside an unfamiliar codebase.

Before We Start

A legacy billing batch can fail across several connected surfaces: invoice validation, cached JSON settings, payment-gateway calls, retry behavior, warning logs, CSV reports, plus cross-test isolation. Before any setup begins, lock in which missed behavior would create the greatest business risk.

Set Up the Legacy Billing Codebase

Advanced failures are useful only when every run starts from the same Python interpreter. A pinned test runner keeps dependency drift out of your debugging.

This step gives you a reproducible interpreter with a pinned pytest installation. It also assembles the prepared billing package that every later test exercises.

In this step, get ready to:
  • Confirm Python 3.10 or newer is available.
  • Create an isolated environment with pytest 9.1.1.
  • Assemble the legacy billing package and verify its imports.
Confirm the Python version

The terminal inside PyCharm runs commands from your current project folder. Checking its interpreter first prevents the virtual environment from inheriting an unsupported Python version.

  • Select the Terminal tool window at the bottom of PyCharm.
  • Check the available Python version by running this command:
python3 --version

What does this command check?

The command asks the Python 3 executable to print its version. Pytest 9.1.1 requires Python 3.10 or newer.

✔️ I see Python 3.10 or newer

Your interpreter meets the project requirement. That clears the first source of environment drift.

ⓧ I see an older Python version

The existing interpreter is below the project requirement. Install Python 3.14.8 before creating the environment.

  • Open the official Python downloads for macOS.
  • Download the macOS installer listed for Python 3.14.8.
  • Follow the installer prompts to complete the installation.
  • Close the current PyCharm terminal session.
  • Select the Terminal tool window to start a fresh session.
  • Check the new interpreter by running this command:
python3 --version

What should this confirm?

A fresh terminal reloads the command path after installation. Continue when the reported version is Python 3.10 or newer.

ⓧ Command not found

The terminal cannot find a Python 3 executable. The official macOS installer adds the interpreter required by this project.

  • Open the official Python downloads for macOS.
  • Download the macOS installer listed for Python 3.14.8.
  • Follow the installer prompts to complete the installation.
  • Close the current PyCharm terminal session.
  • Select the Terminal tool window to start a fresh session.
  • Check the installed interpreter by running this command:
python3 --version

What should this confirm?

The new terminal should now resolve the Python 3 executable. Continue when the reported version is Python 3.10 or newer.

A virtual environment gives this project its own package installation. The pinned pytest version stays separate from other Python projects on your Mac.

  • Create the env virtual environment by running this command:
python3 -m venv env

What does this command create?

Python creates an env directory containing an isolated interpreter plus its package installer. Project dependencies installed there do not modify your system Python.

  • Activate the new environment by running this command:
source env/bin/activate

What does activation change?

Activation makes this terminal use the interpreter inside env. Commands in this terminal now install packages into the project environment.

  • Install the pinned test dependency by running this command:
python -m pip install pytest==9.1.1

Why pin pytest?

The command installs pytest 9.1.1 through the active environment's interpreter. The exact pin makes the test runner reproducible across machines.

The installation output should finish successfully. Your isolated test environment is now ready to receive the starter code.

Installation failed?

Confirm that you activated env in this terminal. Rerun the installation after activation.

If the failure mentions an unsupported Python version, return to the version check above. Use the newly installed interpreter to recreate env.

Ask for help with the exact terminal output: help me diagnose my pytest installation failure

Build the starter billing package

The starter application separates models, configuration, validation, gateway access, reporting, and orchestration. These module boundaries give your later tests realistic seams to inspect.

  • Select the top-level project directory in PyCharm's Project tool window.
  • Press Alt+Insert to open the new-item menu.

You will see the item types that PyCharm can create inside the selected directory.

  • Select Python Package.
  • Type billing as the package name.

PyCharm now has the package name it needs.

  • Press Enter to create the package.
  • Select billing/__init__.py in the Project tool window.
  • Replace its contents with this package description:
"""Legacy billing training package."""

What does this file do?

The __init__.py file identifies billing as an importable package. Its docstring records the package's purpose.

  • Save billing/__init__.py.

You should see __init__.py inside the billing package.

  • Create billing/models.py with PyCharm's Python File workflow.
  • Paste these billing data models into billing/models.py:
from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class Invoice:
    invoice_id: str
    customer_id: str
    amount_cents: int


@dataclass(frozen=True)
class ChargeResult:
    invoice_id: str
    customer_id: str
    status: str
    reference: str = ""
    reason: str = ""


@dataclass(frozen=True)
class BatchOutcome:
    results: tuple[ChargeResult, ...]
    report_path: Path

What do these models represent?

  • The Invoice model carries the public input for one charge.
  • The ChargeResult model records the status plus any payment reference or failure reason.
  • The BatchOutcome model groups the ordered results with the generated report path.
  • Save billing/models.py.

You should now see models.py inside the billing package.

  • Create billing/config.py with PyCharm's Python File workflow.
  • Paste the settings loader into billing/config.py:
import json
from dataclasses import dataclass
from functools import lru_cache
from pathlib import Path


DEFAULT_SETTINGS_PATH = Path("billing-settings.json")


@dataclass(frozen=True)
class Settings:
    max_invoice_cents: int
    retries: int


@lru_cache(maxsize=8)
def load_settings(path: Path = DEFAULT_SETTINGS_PATH) -> Settings:
    data = json.loads(path.read_text(encoding="utf-8"))
    return Settings(
        max_invoice_cents=int(data["max_invoice_cents"]),
        retries=int(data["retries"]),
    )

How does configuration work?

  • The DEFAULT_SETTINGS_PATH value points to the default JSON settings file.
  • The Settings model stores the invoice limit plus the retry count.
  • The load_settings() function reads the file before converting its values to integers.
  • The lru_cache decorator keeps recently loaded settings in a cache.
  • Save billing/config.py.

You should now see config.py beside models.py.

  • Create billing/domain.py with PyCharm's Python File workflow.
  • Paste the invoice rule into billing/domain.py:
from billing.config import Settings
from billing.models import Invoice


def validation_error(invoice: Invoice, settings: Settings) -> str | None:
    if invoice.amount_cents <= 0:
        return "amount must be positive"
    if invoice.amount_cents > settings.max_invoice_cents:
        return "amount exceeds configured limit"
    return None

What does this rule expose?

The validation_error() function returns a public rejection reason for an invalid invoice. A valid invoice returns None.

  • Save billing/domain.py.

You should now see domain.py inside the billing package.

  • Create billing/gateway.py with PyCharm's Python File workflow.
  • Paste the payment boundary into billing/gateway.py:
def charge_card(customer_id: str, amount_cents: int) -> str:
    raise RuntimeError("real payment gateway is unavailable in this training project")

Why does the gateway raise an error?

The charge_card() function represents the external payment boundary. Its deliberate error prevents this training project from making real charges.

  • Save billing/gateway.py.

You should now see gateway.py inside the billing package.

  • Create billing/reporting.py with PyCharm's Python File workflow.
  • Paste the report writer into billing/reporting.py:
import csv
from collections.abc import Sequence
from pathlib import Path

from billing.models import ChargeResult


FIELDNAMES = ("invoice_id", "customer_id", "status", "reference", "reason")


def write_report(results: Sequence[ChargeResult], output_dir: Path) -> Path:
    output_dir.mkdir(parents=True, exist_ok=True)
    report_path = output_dir / "billing-report.csv"

    with report_path.open("w", encoding="utf-8", newline="") as handle:
        writer = csv.DictWriter(handle, fieldnames=FIELDNAMES)
        writer.writeheader()
        for result in results:
            writer.writerow({"invoice_id": result.invoice_id, "customer_id": result.customer_id, "status": result.status, "reference": result.reference, "reason": result.reason})

    return report_path

How is the report produced?

The write_report() function creates the output directory before writing billing-report.csv.

The real CSV writer preserves a stable column order. Later tests can inspect the file without mocking filesystem behavior.

  • Save billing/reporting.py.

You should now see reporting.py inside the billing package.

  • Create billing/service.py with PyCharm's Python File workflow.
  • Paste the service imports plus logger into billing/service.py:
import logging
from collections.abc import Sequence
from pathlib import Path

from billing.config import DEFAULT_SETTINGS_PATH, load_settings
from billing.domain import validation_error
from billing.gateway import charge_card
from billing.models import BatchOutcome, ChargeResult, Invoice
from billing.reporting import write_report


logger = logging.getLogger(__name__)

What does the service connect?

The imports connect settings, validation, gateway access, result models, and report writing. The module logger records warnings from rejected invoices or failed charge attempts.

  • Save billing/service.py.

You should now see service.py inside the billing package.

  • Append the complete process_batch() function below the logger line:
def process_batch(invoices: Sequence[Invoice], output_dir: Path, settings_path: Path = DEFAULT_SETTINGS_PATH) -> BatchOutcome:
    settings = load_settings(settings_path)
    results: list[ChargeResult] = []

    for invoice in invoices:
        reason = validation_error(invoice, settings)
        if reason is not None:
            logger.warning("invoice %s rejected: %s", invoice.invoice_id, reason)
            results.append(ChargeResult(invoice_id=invoice.invoice_id, customer_id=invoice.customer_id, status="REJECTED", reason=reason))
            continue

        for attempt in range(settings.retries + 1):
            try:
                reference = charge_card(invoice.customer_id, invoice.amount_cents)
            except RuntimeError as error:
                logger.warning("charge attempt %s failed for %s: %s", attempt + 1, invoice.invoice_id, error)
            else:
                results.append(ChargeResult(invoice_id=invoice.invoice_id, customer_id=invoice.customer_id, status="PAID", reference=reference))
                break
        else:
            results.append(ChargeResult(invoice_id=invoice.invoice_id, customer_id=invoice.customer_id, status="FAILED", reason="gateway attempts exhausted"))

    report_path = write_report(results, output_dir)
    return BatchOutcome(results=tuple(results), report_path=report_path)

How does a batch move through the service?

  • The service loads settings once before processing the invoices in input order.
  • Invalid invoices become REJECTED results without reaching the gateway.
  • Valid invoices retry gateway failures according to the configured retry count.
  • The final ordered results are written to the real report before the batch outcome returns.
  • Save billing/service.py.

The billing workflow is now assembled across six production modules.

  • Select the top-level project directory in the Project tool window.
  • Press Alt+Insert to open the new-item menu.

The same menu also creates plain directories for test files.

  • Select Directory.
  • Type tests as the directory name.

PyCharm now has the test-directory name.

  • Press Enter to create the directory.
  • Create an empty tests/.gitkeep file with PyCharm's File workflow.
  • Save tests/.gitkeep without adding any content.

You should see an empty tests directory containing .gitkeep.

✔️ Awesome, I've got everything!

Great. Save every billing module before running the import checks.

ⓧ I'd like to double check the full code

Compare each starter file with the complete versions below. The tests/.gitkeep file remains empty.

"""Legacy billing training package."""

Package checkpoint

This file contains one package docstring.

import json
from dataclasses import dataclass
from functools import lru_cache
from pathlib import Path


DEFAULT_SETTINGS_PATH = Path("billing-settings.json")


@dataclass(frozen=True)
class Settings:
    max_invoice_cents: int
    retries: int


@lru_cache(maxsize=8)
def load_settings(path: Path = DEFAULT_SETTINGS_PATH) -> Settings:
    data = json.loads(path.read_text(encoding="utf-8"))
    return Settings(
        max_invoice_cents=int(data["max_invoice_cents"]),
        retries=int(data["retries"]),
    )

Configuration checkpoint

This file defines the default path, settings model, and cached loader.

from billing.config import Settings
from billing.models import Invoice


def validation_error(invoice: Invoice, settings: Settings) -> str | None:
    if invoice.amount_cents <= 0:
        return "amount must be positive"
    if invoice.amount_cents > settings.max_invoice_cents:
        return "amount exceeds configured limit"
    return None

Domain checkpoint

This file exposes the invoice validation result through one public function.

def charge_card(customer_id: str, amount_cents: int) -> str:
    raise RuntimeError("real payment gateway is unavailable in this training project")

Gateway checkpoint

This file contains the payment boundary used by the service.

from dataclasses import dataclass
from pathlib import Path


@dataclass(frozen=True)
class Invoice:
    invoice_id: str
    customer_id: str
    amount_cents: int


@dataclass(frozen=True)
class ChargeResult:
    invoice_id: str
    customer_id: str
    status: str
    reference: str = ""
    reason: str = ""


@dataclass(frozen=True)
class BatchOutcome:
    results: tuple[ChargeResult, ...]
    report_path: Path

Model checkpoint

This file contains the three immutable billing data models.

import csv
from collections.abc import Sequence
from pathlib import Path

from billing.models import ChargeResult


FIELDNAMES = ("invoice_id", "customer_id", "status", "reference", "reason")


def write_report(results: Sequence[ChargeResult], output_dir: Path) -> Path:
    output_dir.mkdir(parents=True, exist_ok=True)
    report_path = output_dir / "billing-report.csv"

    with report_path.open("w", encoding="utf-8", newline="") as handle:
        writer = csv.DictWriter(handle, fieldnames=FIELDNAMES)
        writer.writeheader()
        for result in results:
            writer.writerow({"invoice_id": result.invoice_id, "customer_id": result.customer_id, "status": result.status, "reference": result.reference, "reason": result.reason})

    return report_path

Reporting checkpoint

This file writes every public result field to the billing report.

import logging
from collections.abc import Sequence
from pathlib import Path

from billing.config import DEFAULT_SETTINGS_PATH, load_settings
from billing.domain import validation_error
from billing.gateway import charge_card
from billing.models import BatchOutcome, ChargeResult, Invoice
from billing.reporting import write_report


logger = logging.getLogger(__name__)


def process_batch(invoices: Sequence[Invoice], output_dir: Path, settings_path: Path = DEFAULT_SETTINGS_PATH) -> BatchOutcome:
    settings = load_settings(settings_path)
    results: list[ChargeResult] = []

    for invoice in invoices:
        reason = validation_error(invoice, settings)
        if reason is not None:
            logger.warning("invoice %s rejected: %s", invoice.invoice_id, reason)
            results.append(ChargeResult(invoice_id=invoice.invoice_id, customer_id=invoice.customer_id, status="REJECTED", reason=reason))
            continue

        for attempt in range(settings.retries + 1):
            try:
                reference = charge_card(invoice.customer_id, invoice.amount_cents)
            except RuntimeError as error:
                logger.warning("charge attempt %s failed for %s: %s", attempt + 1, invoice.invoice_id, error)
            else:
                results.append(ChargeResult(invoice_id=invoice.invoice_id, customer_id=invoice.customer_id, status="PAID", reference=reference))
                break
        else:
            results.append(ChargeResult(invoice_id=invoice.invoice_id, customer_id=invoice.customer_id, status="FAILED", reason="gateway attempts exhausted"))

    report_path = write_report(results, output_dir)
    return BatchOutcome(results=tuple(results), report_path=report_path)

Service checkpoint

This file contains the logger plus the complete batch-processing workflow.

Verify the complete setup

The version check proves the virtual environment contains the pinned test runner. The import check proves every billing module can resolve its dependencies.

Before you run the checks, do you expect both commands to complete without an import error?

  • Verify the test runner and billing import by running these commands:
pytest --version
python -c "from billing.service import process_batch; print('billing codebase ready')"

What do these checks prove?

The first command reports the pytest version installed inside env. The second command imports process_batch through the complete module chain.

You should see pytest 9.1.1 followed by billing codebase ready. Those two lines confirm the environment plus the starter application are reproducible.

Import check failed?

Confirm that the terminal is in the top-level folder shown in PyCharm's Project tool window. The billing directory must be directly inside that folder.

Compare every filename with the full-code tab. A missing __init__.py file or misspelled module name breaks the import chain.

Ask for help with the complete traceback: help me diagnose the billing package import failure

That is the setup locked down: pytest 9.1.1 can import the billing service. Next, you will characterize the invoice boundaries with your first four tests.

Characterize the Billing Rules

Your reproducible billing codebase is ready. Now you can use pytest to establish what the billing rules actually promise.

A characterization test records the valid boundary through the public return value. It also records each invalid boundary through its rejection reason.

In this step, get ready to:
  • Trace the public outcomes of the invoice validation rule.
  • Build a parametrized matrix for four boundary cases.
  • Run the suite to confirm every boundary behaves as recorded.
Trace the public validation boundary

The validation_error() function is a public testing seam. Its returned value describes whether an invoice can continue into payment processing.

  • In the Project sidebar in PyCharm, select billing/domain.py.
  • Trace amount_cents from the Invoice input through each comparison in validation_error().
  • Record a negative amount with the returned reason amount must be positive.
  • Record zero with the returned reason amount must be positive.
  • Record 50_001 with the returned reason amount exceeds configured limit.
  • Record 50_000 with the returned value None.

Why Test Public Outcomes?

A characterization test locks in observable behavior before orchestration changes begin. This lets the suite catch a changed business rule without depending on private implementation details.

Build the parametrized characterization test

Parametrization turns one test function into one case per table row. pytest.mark.parametrize keeps each amount beside its expected public outcome.

  • In PyCharm's Project sidebar, right-click the tests directory.
  • Choose the menu option for creating a Python file.

You'll see a field where you can name the new file.

  • Enter test_validation in the file name field.
  • Confirm the file name.

You'll see an empty tests/test_validation.py file open in the editor.

  • Create the boundary matrix by pasting this code into tests/test_validation.py:
import pytest

from billing.config import Settings
from billing.domain import validation_error
from billing.models import Invoice


@pytest.mark.parametrize(
    ("amount_cents", "expected_reason"),
    [
        (-1, "amount must be positive"),
        (0, "amount must be positive"),
        (50_001, "amount exceeds configured limit"),
        (50_000, None),
    ],
    ids=("negative", "zero", "over-limit", "exact-limit"),
)
def test_validation_boundaries(amount_cents: int, expected_reason: str | None):
    settings = Settings(max_invoice_cents=50_000, retries=1)
    invoice = Invoice("inv-boundary", "cus-boundary", amount_cents)

    assert validation_error(invoice, settings) == expected_reason

What Does This Test Prove?

  • The pytest.mark.parametrize table generates four named cases from the same test body.
  • Settings holds the fixed 50_000 limit for every case.
  • Each Invoice uses stable identifiers. Only the amount changes between cases.
  • The assertion compares the public result from validation_error() with the recorded outcome.
  • Save tests/test_validation.py.
  • Confirm PyCharm no longer marks the file as unsaved.

✔️ Awesome, I've got everything!

Your boundary matrix is saved. Double check that tests/test_validation.py has no unsaved changes.

ⓧ I'd like to double check the full code

Compare your complete tests/test_validation.py file with this reference:

import pytest

from billing.config import Settings
from billing.domain import validation_error
from billing.models import Invoice


@pytest.mark.parametrize(
    ("amount_cents", "expected_reason"),
    [
        (-1, "amount must be positive"),
        (0, "amount must be positive"),
        (50_001, "amount exceeds configured limit"),
        (50_000, None),
    ],
    ids=("negative", "zero", "over-limit", "exact-limit"),
)
def test_validation_boundaries(amount_cents: int, expected_reason: str | None):
    settings = Settings(max_invoice_cents=50_000, retries=1)
    invoice = Invoice("inv-boundary", "cus-boundary", amount_cents)

    assert validation_error(invoice, settings) == expected_reason
Run the boundary matrix

Before you run the suite, consider whether every row will agree with the current production rule.

  • Test all four boundaries in the PyCharm terminal by running:
pytest

What Does This Check?

  • The pytest command discovers the test file inside tests.
  • The parameter table expands test_validation_boundaries into four separate tests.
  • The case IDs make future failures point directly to negative, zero, over-limit, or exact-limit.

You should see 4 passed in the terminal. That's your first fast safety net for the billing codebase.

Seeing a Failed Boundary Case?

  • Use the failing case ID to find the matching row in tests/test_validation.py.
  • Compare expected_reason with the value returned by validation_error().
  • Update the expected row when the assertion diff shows that your assumption differs from production behavior.
  • Keep billing/domain.py unchanged in this step.

Help me diagnose a failing billing boundary case.

Your billing rules now have a fast safety net. Next, you'll test a real batch through its CSV report and payment boundary.

Hit the Wrong Mocking Boundary

Your four pytest characterization cases now lock down the billing rules at their edges. You will use that safety net while testing a complete batch workflow.

A mock can look convincing while leaving a production boundary active. Which module supplies the name that process_batch() actually calls?

In this step, get ready to:
  • Create a reusable factory for isolated settings files.
  • Exercise process_batch() with real configuration files and report output.
  • Trace the gateway name resolved by the service.
Create the settings fixture factory

Each service test needs its own JSON settings file. A fixture factory lets each test choose its limits while tmp_path keeps those files isolated.

  • Use the PyCharm project file tree to create conftest.py inside the existing tests folder.
  • Paste this fixture factory into tests/conftest.py:
import json
from collections.abc import Callable
from pathlib import Path

import pytest


@pytest.fixture
def settings_file(tmp_path: Path) -> Callable[..., Path]:
    def make_settings(max_invoice_cents: int = 50_000, retries: int = 1) -> Path:
        path = tmp_path / "billing-settings.json"
        path.write_text(json.dumps({"max_invoice_cents": max_invoice_cents, "retries": retries}), encoding="utf-8")
        return path

    return make_settings

What Does This Fixture Do?

  • The settings_file fixture returns make_settings(). Tests can call that factory with different limits or retry counts.
  • The tmp_path fixture gives each test a temporary directory. Settings from one test cannot overwrite another test's file.
  • The factory returns a Path object. That path can pass directly into process_batch().
  • Save tests/conftest.py.
  • Confirm the fixture leaves the existing suite green by running:
pytest

What Does This Check Prove?

The command runs all discovered tests. You should still see four passing cases because the new fixture changes no production behavior.

Your original four cases still pass. The fixture can now supply isolated settings to the service tests.

Fixture File Causing a Collection Error?

  • Confirm the filename is exactly tests/conftest.py.
  • Check the indentation inside make_settings() against the reference below.
  • Confirm the active environment still contains pytest.

Ask for help with the fixture error: help me debug my settings_file fixture

✔️ Awesome, I've got everything!

Great. Your settings_file fixture is saved while all four characterization cases remain green.

ⓧ I'd like to double check the full code

Compare your complete tests/conftest.py file with this reference:

import json
from collections.abc import Callable
from pathlib import Path

import pytest


@pytest.fixture
def settings_file(tmp_path: Path) -> Callable[..., Path]:
    def make_settings(max_invoice_cents: int = 50_000, retries: int = 1) -> Path:
        path = tmp_path / "billing-settings.json"
        path.write_text(json.dumps({"max_invoice_cents": max_invoice_cents, "retries": retries}), encoding="utf-8")
        return path

    return make_settings

What Should Match?

The fixture must return the nested factory. The factory must write both settings into the temporary file.

Exercise the real billing workflow

This component test keeps the domain logic and file operations real. The payment call is the only simulated edge while the service writes an actual CSV report.

  • Use the PyCharm project file tree to create test_service.py inside the existing tests folder.
  • Paste this batch test into tests/test_service.py:
from unittest.mock import patch

from billing.models import Invoice
from billing.service import process_batch


def test_batch_writes_paid_report(tmp_path, settings_file):
    invoice = Invoice("inv-1", "cus-1", 2_500)
    settings_path = settings_file(retries=0)

    with patch("billing.gateway.charge_card", return_value="pay-123") as gateway:
        outcome = process_batch([invoice], tmp_path / "reports", settings_path)

    assert [result.status for result in outcome.results] == ["PAID"]
    gateway.assert_called_once_with("cus-1", 2_500)

What Does This Test Cover?

  • The test creates a real Invoice with an amount of 2_500 cents.
  • The fixture writes a real settings file with retries set to 0.
  • The patch requests a temporary replacement for billing.gateway.charge_card.
  • The status assertion demands a public outcome of PAID.
  • The call assertion demands the exact customer ID and amount expected by the gateway contract.
  • Save tests/test_service.py.

Before you run the suite, do you expect the payment replacement to intercept the service call?

  • Test the new component boundary by running:
pytest

What Does the First Run Reveal?

You should see four passing tests plus one failing service test. The status assertion compares ['FAILED'] with ['PAID'].

You hit the seam problem. The FAILED result proves the real gateway raised its runtime failure.

Seeing a Different Result?

  • Confirm the patch target in your test is billing.gateway.charge_card.
  • Confirm the first assertion still expects ["PAID"].
  • Confirm the billing package still contains the prepared gateway implementation.

Ask for help comparing the observed result: help me understand my service test result

✔️ Awesome, I've got everything!

Your service test now exposes the intended boundary failure. Keep the failing assertion unchanged for the investigation.

ⓧ I'd like to double check the full code

Compare your complete tests/test_service.py file with this reference:

from unittest.mock import patch

from billing.models import Invoice
from billing.service import process_batch


def test_batch_writes_paid_report(tmp_path, settings_file):
    invoice = Invoice("inv-1", "cus-1", 2_500)
    settings_path = settings_file(retries=0)

    with patch("billing.gateway.charge_card", return_value="pay-123") as gateway:
        outcome = process_batch([invoice], tmp_path / "reports", settings_path)

    assert [result.status for result in outcome.results] == ["PAID"]
    gateway.assert_called_once_with("cus-1", 2_500)

What Should Match?

The patch target must remain billing.gateway.charge_card in this step. That target preserves the designed failure you are diagnosing.

Trace the service lookup

A direct import binds a name inside the importing module. That lookup namespace determines which object process_batch() calls at runtime.

  • In billing/service.py, locate from billing.gateway import charge_card.
  • In the same file, locate reference = charge_card(invoice.customer_id, invoice.amount_cents).
  • Compare the module containing that bound name with the target used by patch().
  • Leave the assertion and production code unchanged.

Why Does the Patch Miss?

The import binds a charge_card name inside billing.service. The process_batch() function calls that bound name.

The patch modifies the attribute on billing.gateway. The service-bound name continues pointing to the real function.

Before you rerun the suite, which four cases should stay green? Which service behavior should stay red?

  • Confirm the intended failure remains by running the full suite again:
pytest

What Does the Final Check Prove?

Pytest should still report four passing characterization cases plus one failing service test. The service outcome remains FAILED because the patch did not replace the name resolved inside billing.service.

This failure is your evidence. A plausible definition-site patch can leave the real dependency active.

You have proved that mock isolation depends on the lookup site. Next, you will repair that boundary and test retry behavior without hiding wiring mistakes.

Repair the Boundary and Test Retries

The failed batch test gave you useful evidence. The patch changed the gateway module while the service kept calling its imported name.

This step repairs that lookup boundary with an autospecced mock. The corrected test protects the exact payment call.

Next, you will use pytest to prove retry recovery through captured warning logs. Two configuration tests will then probe whether process state survives between tests.

In this step, get ready to:
  • Patch the gateway at the service lookup site.
  • Prove retry recovery through the warning from its failed first attempt.
  • Probe configuration isolation with separate temporary directories.
Patch the service lookup site

A patch isolates code when it replaces the name used at runtime. process_batch() looks up charge_card inside billing.service.

  • Replace the existing test_batch_writes_paid_report function in tests/test_service.py with the code below:
def test_batch_writes_paid_report(tmp_path, settings_file):
    invoice = Invoice("inv-1", "cus-1", 2_500)
    settings_path = settings_file(retries=0)

    with patch("billing.service.charge_card", autospec=True, return_value="pay-123") as gateway:
        outcome = process_batch([invoice], tmp_path / "reports", settings_path)

    assert [result.status for result in outcome.results] == ["PAID"]
    assert outcome.report_path.read_text(encoding="utf-8").splitlines() == ["invoice_id,customer_id,status,reference,reason", "inv-1,cus-1,PAID,pay-123,"]
    gateway.assert_called_once_with("cus-1", 2_500)

What does this test prove?

  • The billing.service.charge_card target replaces the name that process_batch() reads.
  • The autospec=True option checks calls against the real gateway signature.
  • The report assertion leaves the real writer active. This proves the batch produced the expected CSV rows.
  • The final call assertion protects the customer identifier plus the amount sent across the payment boundary.
  • Save tests/test_service.py.

Before you rerun the suite, do you expect the batch status to remain FAILED or become PAID?

  • Check the repaired boundary by running this command:
pytest

What does this run prove?

The pytest command discovers the current validation tests plus the repaired service test. A passing batch test confirms that the service reaches the replacement at its lookup site.

You should see five passing tests. That recovery matters because the mock now isolates the payment boundary while the real report writer remains under test.

Still seeing the batch fail?

  • Confirm that the patch target is exactly billing.service.charge_card.
  • Confirm that return_value="pay-123" remains inside the patch() call.
  • Compare the expected report row with the value in billing-report.csv.

Help me diagnose why the repaired payment patch still fails.

Test retry recovery and warning logs

A retry test needs different behavior on successive calls. An iterable side_effect supplies one outcome per call.

  • Add import logging above the existing mock import in tests/test_service.py.
  • Add the retry test below test_batch_writes_paid_report by copying this code:
def test_transient_failure_is_retried_and_logged(tmp_path, settings_file, caplog):
    invoice = Invoice("inv-9", "cus-9", 4_000)
    settings_path = settings_file(retries=1)

    with caplog.at_level(logging.WARNING, logger="billing.service"):
        with patch("billing.service.charge_card", autospec=True, side_effect=[RuntimeError("timeout"), "pay-9"]) as gateway:
            outcome = process_batch([invoice], tmp_path / "reports", settings_path)

    assert [result.status for result in outcome.results] == ["PAID"]
    assert gateway.call_count == 2
    assert "charge attempt 1 failed for inv-9: timeout" in caplog.text

How does the retry test work?

  • The retries=1 setting permits an initial gateway call plus one retry.
  • The first item in side_effect=[RuntimeError("timeout"), "pay-9"] raises a transient failure. The second item returns a successful payment reference.
  • The caplog.at_level() context captures warnings from the service logger.
  • The gateway.call_count assertion proves that recovery required two calls. The caplog.text assertion proves that the first failure remained observable.
  • Save tests/test_service.py.
  • Check the retry behavior by running this command:
pytest

What does this run prove?

This run executes both service scenarios against the real domain code plus the real report writer. The retry test proves that a transient exception produces a warning before the next gateway call succeeds.

You should see six passing tests. Strong result. The suite now protects the gateway signature plus the retry behavior.

Retry test not passing?

  • Confirm that the settings factory receives retries=1.
  • Confirm that the first sequential outcome is RuntimeError("timeout").
  • Confirm that caplog.at_level() targets the billing.service logger.

Help me debug the retry test and captured warning.

✔️ Awesome, I've got everything!

Your service tests now cover a successful payment plus recovery from a transient gateway failure.

ⓧ I'd like to double check the full code

Compare your complete tests/test_service.py file with this reference:

import logging
from unittest.mock import patch

from billing.models import Invoice
from billing.service import process_batch


def test_batch_writes_paid_report(tmp_path, settings_file):
    invoice = Invoice("inv-1", "cus-1", 2_500)
    settings_path = settings_file(retries=0)

    with patch("billing.service.charge_card", autospec=True, return_value="pay-123") as gateway:
        outcome = process_batch([invoice], tmp_path / "reports", settings_path)

    assert [result.status for result in outcome.results] == ["PAID"]
    assert outcome.report_path.read_text(encoding="utf-8").splitlines() == ["invoice_id,customer_id,status,reference,reason", "inv-1,cus-1,PAID,pay-123,"]
    gateway.assert_called_once_with("cus-1", 2_500)


def test_transient_failure_is_retried_and_logged(tmp_path, settings_file, caplog):
    invoice = Invoice("inv-9", "cus-9", 4_000)
    settings_path = settings_file(retries=1)

    with caplog.at_level(logging.WARNING, logger="billing.service"):
        with patch("billing.service.charge_card", autospec=True, side_effect=[RuntimeError("timeout"), "pay-9"]) as gateway:
            outcome = process_batch([invoice], tmp_path / "reports", settings_path)

    assert [result.status for result in outcome.results] == ["PAID"]
    assert gateway.call_count == 2
    assert "charge attempt 1 failed for inv-9: timeout" in caplog.text

What should match?

The first test protects the successful payment contract plus the report output. The second test protects retry recovery plus the warning left by the failed attempt.

Expose cached configuration state

The default settings path is relative to the current directory. Two tests can therefore ask the same public loader to read separate real JSON files.

  • Create tests/test_config.py beside tests/test_service.py using PyCharm's file creation control.
  • Add the helper plus the first directory scenario by copying this code:
import json
from pathlib import Path

from billing.config import load_settings


def write_default_settings(max_invoice_cents: int) -> None:
    Path("billing-settings.json").write_text(json.dumps({"max_invoice_cents": max_invoice_cents, "retries": 1}), encoding="utf-8")


def test_low_limit_is_loaded_from_current_directory(tmp_path, monkeypatch):
    monkeypatch.chdir(tmp_path)
    write_default_settings(1_000)

    assert load_settings().max_invoice_cents == 1_000

What does this scenario test?

  • The write_default_settings() helper writes a real configuration file in the current directory.
  • The monkeypatch.chdir() call moves the test into its isolated temporary directory.
  • The no-argument load_settings() call exercises DEFAULT_SETTINGS_PATH through the public loader.
  • Save tests/test_config.py.
  • Confirm the first configuration scenario by running this command:
pytest

What does this run prove?

The suite now includes one default-path configuration scenario. Its success proves that the loader can read the settings file from the temporary current directory.

You should see seven passing tests. This is a useful checkpoint because the first isolated directory behaves correctly on its own.

First configuration test failing?

  • Confirm that monkeypatch.chdir(tmp_path) runs before the settings file is written.
  • Confirm that the filename is exactly billing-settings.json.
  • Confirm that the assertion calls load_settings() without a path argument.

Help me debug the first default-path configuration test.

A second temporary directory introduces another valid configuration value. Running both scenarios together tests whether each call observes its own current directory.

  • Add the second configuration test below the first test by copying this code:
def test_high_limit_is_loaded_from_current_directory(tmp_path, monkeypatch):
    monkeypatch.chdir(tmp_path)
    write_default_settings(90_000)

    assert load_settings().max_invoice_cents == 90_000

What changes in this test?

This test uses a fresh tmp_path directory plus a limit of 90_000. It calls the same no-argument loader used by the low-limit test.

  • Save tests/test_config.py.

Before you run the full suite, which result do you expect from two no-argument calls after the current directory changes?

  • Check both configuration scenarios by running this command:
pytest

This failure is intentional

Seven tests pass. The test_high_limit_is_loaded_from_current_directory test receives the low-limit settings from the earlier call.

The failed assertion is the evidence you needed. The cached no-argument result survives after the working directory changes.

Seeing a different suite result?

  • Confirm that the low-limit test appears before the high-limit test in tests/test_config.py.
  • Confirm that both assertions call load_settings() without a path argument.
  • Confirm that tests/conftest.py still contains only the existing settings_file fixture.

Help me reproduce the stale configuration cache between these tests.

✔️ Awesome, I've got everything!

Your two configuration scenarios now expose the cached state that survives between temporary directories.

ⓧ I'd like to double check the full code

Compare your complete tests/test_config.py file with this reference:

import json
from pathlib import Path

from billing.config import load_settings


def write_default_settings(max_invoice_cents: int) -> None:
    Path("billing-settings.json").write_text(json.dumps({"max_invoice_cents": max_invoice_cents, "retries": 1}), encoding="utf-8")


def test_low_limit_is_loaded_from_current_directory(tmp_path, monkeypatch):
    monkeypatch.chdir(tmp_path)
    write_default_settings(1_000)

    assert load_settings().max_invoice_cents == 1_000


def test_high_limit_is_loaded_from_current_directory(tmp_path, monkeypatch):
    monkeypatch.chdir(tmp_path)
    write_default_settings(90_000)

    assert load_settings().max_invoice_cents == 90_000

What should match?

Both tests use separate temporary directories. Both tests also call the cached loader through the same default-path interface.

Your service tests now enforce the payment call contract plus retry behavior. The stale configuration result gives you the evidence needed to remove order dependence next.

Eliminate Cross-Test State Leakage

Your pytest suite now isolates the payment boundary at the service lookup site. The remaining configuration failure shows that the suite still shares process state between cases.

The load_settings() function uses memoization to reuse earlier results. That cached result can outlive the temporary directory that produced it.

In this step, get ready to:
  • Trace why both configuration tests share the same cached path key.
  • Add an autouse yield fixture for cache isolation.
  • Prove that all eight tests pass regardless of collection order.
Trace the cached path key

A cache can return an old result before the function reads the current file. You need to identify the argument that makes both configuration calls look identical.

  • In PyCharm, select billing/config.py from the file tree.
  • Find the @lru_cache(maxsize=8) decorator above load_settings().
  • Trace DEFAULT_SETTINGS_PATH into the default argument on load_settings().
  • Select tests/test_config.py from the file tree.
  • Compare the load_settings() calls in both configuration tests.

Each test changes to a unique tmp_path. Both tests call load_settings() without an explicit path.

Why does the directory change miss the cache?

The monkeypatch.chdir(tmp_path) call changes the current directory for each test. The default argument remains the same Path("billing-settings.json") object.

The cache reuses a result when it receives the same argument key. The settings from the first directory can therefore satisfy the second call before another file is read.

Clear the cache around every test

A shared isolation fixture gives the suite one policy for cached configuration. Its setup clears stale input before a test starts. Its teardown clears any value the test leaves behind.

  • In tests/conftest.py, find the existing import pytest line.
  • Insert the following code between import pytest and the existing @pytest.fixture decorator for settings_file:
from billing.config import load_settings


@pytest.fixture(autouse=True)
def clear_settings_cache():
    load_settings.cache_clear()
    yield
    load_settings.cache_clear()

What does this fixture do?

  • The load_settings import gives the fixture access to the cached function.
  • The autouse=True setting applies the fixture to every test without adding another test parameter.
  • The first load_settings.cache_clear() call removes state left by an earlier test.
  • The yield statement separates setup from teardown.
  • The final load_settings.cache_clear() call removes state created by the current test.
  • Save tests/conftest.py.

Before you run the suite, do you expect the second configuration test to read its own file this time?

  • Run the complete suite from the active virtual environment by running:
pytest

What does this run prove?

Pytest executes the service tests alongside both configuration tests in one process. A passing run proves that each test begins with an empty settings cache.

You should see 8 passed. Your suite now clears settings state around every case.

Still seeing a stale limit?

  • Confirm clear_settings_cache appears above settings_file in tests/conftest.py.
  • Confirm the decorator includes autouse=True.
  • Confirm both cache clears call load_settings.cache_clear().

Help me diagnose why the settings cache still leaks between my pytest tests.

✔️ Awesome, I've got everything!

Your tests/conftest.py file now isolates the settings cache while preserving the existing fixture factory.

ⓧ I'd like to double check the full code

Compare your complete tests/conftest.py file with this cumulative version.

import json
from collections.abc import Callable
from pathlib import Path

import pytest

from billing.config import load_settings


@pytest.fixture(autouse=True)
def clear_settings_cache():
    load_settings.cache_clear()
    yield
    load_settings.cache_clear()


@pytest.fixture
def settings_file(tmp_path: Path) -> Callable[..., Path]:
    def make_settings(max_invoice_cents: int = 50_000, retries: int = 1) -> Path:
        path = tmp_path / "billing-settings.json"
        path.write_text(json.dumps({"max_invoice_cents": max_invoice_cents, "retries": retries}), encoding="utf-8")
        return path

    return make_settings
Prove order-independent behavior

A deterministic suite keeps the same result when test collection order changes. Reversing the configuration tests directly challenges the isolation policy you added.

  • In tests/test_config.py, move the complete test_high_limit_is_loaded_from_current_directory function above test_low_limit_is_loaded_from_current_directory.
  • Save tests/test_config.py.

Before you rerun the suite, do you expect the high-limit test to influence the low-limit test?

  • Test the reversed definition order by running:
pytest

What does the reversed run prove?

The reversed order changes which settings value enters the process first. Another passing run proves that neither value survives into the next test.

You should still see 8 passed. The configuration tests now behave consistently in either order.

Does the reversed order fail?

Check that the fixture clears the cache before yield. Check that it clears the cache again after yield.

Help me find why reversing my configuration tests still changes the result.

The reversed run has tested the isolation policy. Return the configuration file to its cumulative project layout before the final check.

  • Restore tests/test_config.py so it matches the cumulative file reference below.

✔️ Awesome, I've got everything!

Keep the low-limit test above the high-limit test for the final project state.

ⓧ I'd like to double check the full code

Compare your complete tests/test_config.py file with this cumulative version.

import json
from pathlib import Path

from billing.config import load_settings


def write_default_settings(max_invoice_cents: int) -> None:
    Path("billing-settings.json").write_text(json.dumps({"max_invoice_cents": max_invoice_cents, "retries": 1}), encoding="utf-8")


def test_low_limit_is_loaded_from_current_directory(tmp_path, monkeypatch):
    monkeypatch.chdir(tmp_path)
    write_default_settings(1_000)
    assert load_settings().max_invoice_cents == 1_000


def test_high_limit_is_loaded_from_current_directory(tmp_path, monkeypatch):
    monkeypatch.chdir(tmp_path)
    write_default_settings(90_000)
    assert load_settings().max_invoice_cents == 90_000
  • Save tests/test_config.py.

Before the final run, do you expect restoring the original definition order to change any outcome?

  • Verify the final cumulative suite by running:
pytest

What does the final run confirm?

This run uses the final file layout with the cache isolation fixture active. It confirms that the service tests remain intact while both configuration tests read their own files.

You should see 8 passed again. That is suite-level determinism you can defend in a code review.

That shared-state leak is closed. Your eight-test suite now produces the same result regardless of configuration test order.

Secret mission

Prove a Mixed Batch End to End

One billing batch can contain successful payments, validation rejections, plus exhausted gateway retries. Build one test that proves these outcomes preserve input order while producing the correct calls, report rows, plus warning logs.

Clean Up Your Resources

Clean Up Your Resources

Everything you built stays on your Mac. With no cloud services or ongoing costs, your only cleanup decision is whether to keep, pause, or delete the local suite.

Resources you used:

  • The local billing/ package containing the billing workflow.
  • The nine-test suite in tests/, including tests/test_service.py.

Keep everything running

Keeping the suite requires no action. Choose this option if you want to continue testing the billing workflow in PyCharm.

  • Keep billing/ in the current PyCharm project.
  • Keep tests/ with all nine passing tests intact.

Pause - I'll come back to this later

Your test run has already returned to the terminal prompt. Pausing means closing the editor while preserving every local file.

  • Close PyCharm to free the memory used by the editor.
  • Leave billing/ unchanged for your next session.
  • Leave tests/ unchanged so the nine-test suite is ready when you return.

Delete - I don't want to use this again

Starting fresh means removing the local billing package plus its complete test suite. Deleting these folders is permanent, so use this option only when you no longer need your work.

  • Switch back to the PyCharm terminal where you ran the nine-test suite.
  • Remove billing/ plus tests/ by running this command:
rm -rf billing tests

What Does This Command Remove?

The command recursively removes the billing/ package plus the tests/ suite from the current folder. It leaves unrelated folders untouched.

  • Check the PyCharm file tree. You should no longer see billing/ or tests/.

Nice Work!

Nice Work!

You did it! Your nine-test pytest suite now protects the legacy billing workflow from boundary mistakes and shared-state leaks.

You've learned how to:

  • Create a parameterized characterization suite that pins down the valid invoice limit. The suite also records every rejection reason at the billing boundary.
  • Run the real billing service against isolated temporary files. Verify the generated CSV report through public outcomes.
  • Repair a lookup-site patch after observing the wrong boundary fail. Enforce the gateway signature with autospeccing. Prove retry recovery through sequential side effects and captured warnings. Clear cached configuration state around every test so collection order cannot change the result.
  • Secret Mission: Prove a mixed billing batch with one successful payment. Keep one validation rejection away from the gateway. Exhaust both attempts for one permanent failure. Confirm that the public results remain in input order across the real report.

Ready to quiz yourself?