Detect Multiple Incidents with CaseTrace

Extend CaseTrace to correlate multiple security incidents with JSON rules.

Introduction

30 Second Summary

A busy security log can capture several suspicious stories at once. Reading only the first story can leave a complete later incident hidden in plain sight.

In this project, you will tackle Part 2 of the CaseTrace Cybersecurity series by extending CaseTrace so one evidence file produces every qualifying incident through multi-incident correlation. You will finish with a JSON-configured analyzer whose regression tests preserve separate HIGH and CRITICAL outcomes.

What You'll Build

When you run CaseTrace, you see two ordered incident conclusions in the terminal before finding the same CRITICAL and HIGH results in the Markdown report.

By the end of this project, you'll have:

  • A repeatable single-result blind spot where two complete suspicious sequences still produce only one conclusion.
  • Configurable multi-incident detection that reads thresholds and severity labels from rules.json. You can rerun CaseTrace to see one CRITICAL incident plus one HIGH incident.
  • A seven-test regression suite that protects behavior from event loading through the final evidence hash.
  • Secret Mission: An eighth test with minimum_failures tightened from 3 to 4. It proves both three-failure sequences are suppressed.

Are there any prerequisites?

You need Windows 10 or newer with Visual Studio Code installed.

You also need basic Python syntax plus the completed Part 1 workspace from Build an Investigative Log Analyzer.

Before We Start

Your starting point is the exact tested Part 1 CaseTrace implementation. Its preservation contract has three parts: fixed CSV evidence boundaries, intact SHA-256 integrity handling, plus separate HIGH or CRITICAL outcomes.

Restore and Verify Part 1

Part 2 depends on the exact tested Part 1 baseline. A guessed reconstruction could make later failures impossible to attribute.

This step prepares Visual Studio Code with Python 3.15.2 for that baseline. You prove the recovered workspace still passes all five inherited tests before changing any evidence or code.

In this step, get ready to:
  • Prepare Python 3.15.2 through Python install manager 26.4.
  • Recover the exact Part 1 Secret Mission workspace.
  • Prove the inherited five-test baseline still passes.
Install the pinned Python tools

The Python install manager controls which CPython runtime your terminal launches. Pinning the tools gives every later test the same interpreter behavior.

  • Press the Windows key to open Windows search.
  • Type Visual Studio Code.
  • Press Enter to start Visual Studio Code.
  • Select View from the menu bar.
  • Select Terminal to reveal the integrated terminal.
  • Select the terminal profile dropdown in the terminal panel.
  • Select PowerShell.

Expect the installation to take a few minutes while Windows retrieves the manager. A short pause in the terminal is normal.

  • Prepare to confirm any source agreement prompt that WinGet displays.
  • Install Python install manager 26.4 by running this command:
winget install 9NQ7512CXL7T

What does this command do?

  • The winget command asks Windows Package Manager to install a package.
  • The package ID 9NQ7512CXL7T identifies Python install manager 26.4.

That is the first dependency sorted. Windows can now install the runtime that CaseTrace uses.

Manager installation failed?

Check that the PowerShell terminal has internet access. Run the installation command again after any pending Windows prompt has closed.

If WinGet reports that the package is already installed, continue to the runtime installation below.

Help me troubleshoot the Python install manager installation. You can also share the terminal output in the NextWork community.

  • Install the current stable Python runtime by running this command:
py install

What does this command do?

The py install command asks the manager to add the current stable runtime. For this project, that runtime is Python 3.15.2.

The runtime download can take a few minutes. You will return to the PowerShell prompt when the installation finishes.

Before the check, form a quick prediction about whether the runtime list contains Python 3.15.2.

  • Inspect the installed runtimes by running this command:
py list

What does this command prove?

The py list command shows the Python runtimes available through the install manager. The current runtime entry confirms which interpreter later commands can use.

✔️ I see Python 3.15.2

Your pinned Python environment is ready. The terminal can now run the inherited CaseTrace tests with the expected runtime.

ⓧ I see an older Python version

The manager works, but the required runtime is still missing.

  • Return to the PowerShell terminal.
  • Run the runtime installation command shown directly above.
  • Run the runtime list command shown above again.

You should now see an entry containing Python 3.15.2.

ⓧ The py command is unavailable

The terminal has not picked up the install manager command yet.

  • Close Visual Studio Code.
  • Reopen Visual Studio Code through Windows search.
  • Return to the integrated PowerShell terminal.
  • Run the manager installation command shown above again.
  • Run the runtime installation command shown above.
  • Run the runtime list command shown above.

You should now see Python 3.15.2 in the runtime list.

Still missing Python 3.15.2?

Close the current terminal before creating a fresh PowerShell terminal. An older terminal can keep command information from before the installation.

Confirm that the runtime installation completed before checking the list again.

Help me make Python 3.15.2 appear in py list. You can also share the terminal output in the NextWork community.

Recover the exact Part 1 workspace

The recovered workspace must match the Part 1 Secret Mission completion state. That shared starting point makes every later change traceable.

Why does the exact snapshot matter?

The inherited analyzer already defines the accepted CSV evidence boundaries. It also preserves the evidence file with a SHA-256 integrity record.

The snapshot also treats HIGH and CRITICAL as separate valid outcomes. Starting from its passing tests protects those contracts during the multi-incident refactor.

Your next move depends on whether the completed Part 1 folder is still on your machine.

✔️ I have the completed CaseTrace folder

  • Press Ctrl+K Ctrl+O to open the folder picker.
  • Select your completed CaseTrace folder.
  • Select Select Folder.
  • Select Yes, I trust the authors if Visual Studio Code displays the Workspace Trust prompt.

ⓧ I need to recover Part 1

  • Open the canonical Build an Investigative Log Analyzer project.
  • Find its Secret Mission completion snapshot.
  • Restore the complete CaseTrace folder by following that snapshot exactly.
  • Return to Visual Studio Code.
  • Press Ctrl+K Ctrl+O to open the folder picker.
  • Select the recovered CaseTrace folder.
  • Select Select Folder.
  • Select Yes, I trust the authors if Visual Studio Code displays the Workspace Trust prompt.
  • Confirm that the Explorer shows case_trace.py, test_case_trace.py, and data/case_log.csv.

Great, Visual Studio Code now points at the exact artifact set that Part 2 extends.

Verify the inherited baseline

The inherited tests form your regression baseline. Their passing result protects the parser counts, severity outcomes, malformed-row handling, and integrity behavior.

Before you run the suite, predict whether an exact recovery should pass without any edits.

  • Return to the integrated PowerShell terminal from earlier.
  • Run the inherited test suite with this command:
python -m unittest

What does this command prove?

The -m option runs the standard-library unittest module through the active Python interpreter. Test discovery finds the inherited tests in test_case_trace.py.

A passing result proves the recovered analyzer matches the tested Part 1 behavior before Part 2 changes begin.

You should see five inherited tests complete. The output ends with OK.

The inherited tests did not pass?

Confirm that the terminal path ends in your CaseTrace folder. Test discovery cannot find the correct files from another directory.

Check that case_trace.py, test_case_trace.py, and data/case_log.csv match the Part 1 Secret Mission snapshot.

Help me diagnose a failing inherited CaseTrace test.

The passing suite establishes the code baseline. One final runtime check ties that result to the pinned interpreter.

Before this final check, predict whether the listed runtime still matches the interpreter prepared earlier.

  • Inspect the installed runtime list again by running this command:
py list

Why check the runtime again?

This final check records the interpreter available after the workspace test. It prevents a passing result from being mistaken for one produced by an unverified runtime.

You should see Python 3.15.2 in the list. Together with the five tests ending in OK, this confirms the exact Part 1 baseline is ready.

Your tested Part 1 workspace is restored with the pinned runtime. Next, you expose the single-result blind spot with a second complete incident sequence.

Expose the Single-Result Limit

The five inherited tests confirm that your recovered CaseTrace workspace matches Part 1. That trusted baseline gives you a defensible starting point for this experiment.

Security evidence can hold several suspicious sequences at once. This step introduces multi-incident correlation by adding a second complete sequence before testing the inherited analyzer unchanged.

In this step, get ready to:
  • Add a second complete sequence to the CSV evidence.
  • Run the inherited analyzer without changing its correlation logic.
  • Compare the complete evidence with the reported conclusion.
Add a second complete sequence

The existing evidence proves how CaseTrace handles one suspicious sequence. A second user now needs the same failure-to-outbound pattern so you can test how the inherited analyzer behaves with competing conclusions.

  • Switch back to the file tree in Visual Studio Code.
  • Select data/case_log.csv from the recovered CaseTrace workspace.
  • Scroll to the malformed jrivera row at the bottom of the file.
  • Append the six mchen events by pasting this block below that row:
2026-10-27T10:00:00Z,workstation-11,mchen,10.0.0.31,AUTH_FAILURE,invalid password
2026-10-27T10:01:00Z,workstation-11,mchen,10.0.0.31,AUTH_FAILURE,invalid password
2026-10-27T10:02:00Z,workstation-11,mchen,10.0.0.31,AUTH_FAILURE,invalid password
2026-10-27T10:03:00Z,workstation-11,mchen,10.0.0.31,AUTH_SUCCESS,successful sign-in after failures
2026-10-27T10:04:00Z,workstation-11,mchen,10.0.0.31,FILE_ACCESS,quarterly_forecast.xlsx opened
2026-10-27T10:05:00Z,workstation-11,mchen,10.0.0.31,OUTBOUND_CONNECTION,destination synthetic-server-02

What Do These Events Add?

  • The first three rows create three earlier authentication failures for mchen.
  • The fourth row records a successful sign-in on the same workstation.
  • The fifth row records access to quarterly_forecast.xlsx after that success.
  • The final row records a later outbound connection.

The sequence contains no DEVICE_WARNING event. This creates contrasting evidence for the analyzer to evaluate.

  • Save data/case_log.csv.
  • Confirm the file ends with the outbound event at 2026-10-27T10:05:00Z.

The evidence file now contains complete suspicious sequences for two different users. Both sequences progress from authentication failures to an outbound connection.

Do the new rows look incomplete?

  • Check that all six new rows sit below the malformed jrivera row.
  • Check that each new event occupies its own line.
  • Compare every comma-separated value with the full-file reference below.

Ask for help if the sequence still looks different. Help me compare my CaseTrace CSV with the required evidence rows.

✔️ Awesome, I've got everything!

Good. Your evidence file now contains both complete suspicious sequences.

ⓧ I'd like to double check the full code

timestamp,device,user,source_ip,event_type,detail
2026-10-27T08:45:00Z,workstation-02,asmith,10.0.0.12,AUTH_SUCCESS,routine sign-in
2026-10-27T08:50:00Z,sensor-gateway-02,system,10.0.0.50,DEVICE_HEALTH,status normal
2026-10-27T09:00:00Z,workstation-07,jrivera,10.0.0.24,AUTH_FAILURE,invalid password
2026-10-27T09:01:00Z,workstation-07,jrivera,10.0.0.24,AUTH_FAILURE,invalid password
2026-10-27T09:02:00Z,workstation-07,jrivera,10.0.0.24,AUTH_FAILURE,invalid password
2026-10-27T09:03:00Z,workstation-07,jrivera,10.0.0.24,AUTH_SUCCESS,successful sign-in after failures
2026-10-27T09:04:00Z,workstation-07,jrivera,10.0.0.24,FILE_ACCESS,sensitive_plans.zip opened
2026-10-27T09:05:00Z,workstation-07,jrivera,10.0.0.24,OUTBOUND_CONNECTION,destination 203.0.113.77
2026-10-27T09:06:00Z,workstation-07,system,10.0.0.24,DEVICE_WARNING,network adapter reset unexpectedly
2026-10-27T09:07:00Z,workstation-07,,10.0.0.24,AUTH_FAILURE,missing user field
2026-10-27T10:00:00Z,workstation-11,mchen,10.0.0.31,AUTH_FAILURE,invalid password
2026-10-27T10:01:00Z,workstation-11,mchen,10.0.0.31,AUTH_FAILURE,invalid password
2026-10-27T10:02:00Z,workstation-11,mchen,10.0.0.31,AUTH_FAILURE,invalid password
2026-10-27T10:03:00Z,workstation-11,mchen,10.0.0.31,AUTH_SUCCESS,successful sign-in after failures
2026-10-27T10:04:00Z,workstation-11,mchen,10.0.0.31,FILE_ACCESS,quarterly_forecast.xlsx opened
2026-10-27T10:05:00Z,workstation-11,mchen,10.0.0.31,OUTBOUND_CONNECTION,destination synthetic-server-02
Run the unchanged analyzer

The evidence now gives the inherited analyzer two complete sequences to evaluate. Running the existing code reveals whether its conclusion represents the whole file.

Before you run the analyzer, make a quick prediction about whether two complete sequences produce one conclusion or two.

  • Return to the PowerShell terminal in Visual Studio Code from earlier.
  • Test the unchanged analyzer by running this command:
python case_trace.py

What Does This Command Do?

  • The command reads data/case_log.csv through the inherited parser.
  • It sends the valid events through the inherited correlation path.
  • It regenerates output/investigation_report.md from the returned conclusion.

The parser reports 15 valid events. It also reports one malformed row skipped.

The baseline count rises to 10 findings. The correlated conclusion still names only the CRITICAL result for jrivera on workstation-07.

You have made the blind spot visible. The evidence contains a later complete sequence for mchen that never reaches the conclusion.

Why Does the Second Incident Disappear?

The inherited correlate_incident() returns as soon as it builds the first complete match. Processing stops before the later sequence can become another conclusion.

The jrivera sequence includes a later device warning that preserves its CRITICAL outcome. The complete mchen sequence has no warning that would escalate its valid HIGH outcome.

Seeing Different Counts or Two Incidents?

  • Compare data/case_log.csv with the full-file reference if the counts differ from 15 valid events or one malformed row.
  • Restore the inherited Part 1 case_trace.py if the analyzer already reports two incidents.

Use the inherited baseline to keep this experiment defensible. Help me diagnose my unexpected CaseTrace counts or correlation result.

  • Select output/investigation_report.md from the CaseTrace file tree.
  • Locate the correlated conclusion generated by the latest run.
  • Match that conclusion against the two complete sequences in data/case_log.csv.

The Markdown report repeats only the jrivera result. The mchen sequence has no conclusion.

The inherited analyzer now has a reproducible single-result blind spot. Next, you'll move its thresholds and severity labels into a validated configuration file.

Move Rules into JSON

The second incident is now visible in the evidence. The inherited analyzer still returns only the first match.

Hard-coded thresholds make the next refactor difficult to adapt or test. In this step, you'll move the correlation policy into a validated JSON configuration. CaseTrace will keep its inherited single-result behavior during this intermediate check.

In this step, get ready to:
  • Create a configuration file for the failure threshold plus severity labels.
  • Build a loader that rejects malformed rule configurations.
  • Make the analyzer display its loaded rules before the inherited incident result.
Create the rules file

A configuration file separates detection policy from correlation logic. The analyzer can load a threshold or severity label without embedding that value inside the correlation function.

  • Select the CaseTrace workspace folder in the Explorer sidebar.
  • Click the New File icon at the top of the Explorer sidebar.
  • Type rules.json into the file name field.
  • Press Enter to create the file beside case_trace.py.
  • Define the correlation policy by pasting this configuration into rules.json:
{
  "minimum_failures": 3,
  "base_severity": "HIGH",
  "warning_severity": "CRITICAL"
}

What do these rules control?

  • The minimum_failures value requires three earlier authentication failures.
  • The base_severity value labels a complete sequence without a later device warning.
  • The warning_severity value preserves the escalation caused by a later same-device warning.
  • Save rules.json.
  • Confirm the Explorer sidebar lists rules.json beside case_trace.py.

Rules file missing or underlined?

Confirm that rules.json sits directly inside the CaseTrace workspace. A file inside data or output uses the wrong path.

Check every property name for double quotation marks. Confirm that the first two properties end with commas.

Help me check my CaseTrace rules file.

✔️ I have the rules file

Good work. Your saved rules.json now holds the correlation policy that CaseTrace will validate.

ⓧ I'd like to double check the full code

{
  "minimum_failures": 3,
  "base_severity": "HIGH",
  "warning_severity": "CRITICAL"
}
Add the rule loader

CaseTrace must validate configuration before trusting it. The loader will require an object with all three fields plus values of the expected types.

  • Switch back to the case_trace.py tab from earlier.
  • Find import hashlib near the top of the file.
  • Insert the following import directly below it:
import json

Why add this import?

The json module converts the configuration file into Python values. CaseTrace can then validate each rule before correlation uses it.

  • Find the SUSPICIOUS_TYPES set near the top of case_trace.py.
  • Place your cursor below the set's closing brace.
  • Add the required rule names by pasting this tuple:
RULE_FIELDS = (
    "minimum_failures",
    "base_severity",
    "warning_severity",
)

What does this tuple define?

The RULE_FIELDS tuple is the configuration schema. The loader uses it to identify any missing threshold or severity field.

  • Save case_trace.py.
  • Check that the new declarations preserve the existing analyzer by running:
python case_trace.py

What does this check prove?

The analyzer should still report 15 valid events plus one malformed row. It should also report 10 baseline findings plus the inherited CRITICAL result for jrivera.

That complete run confirms the import plus rule-field declaration contain no syntax errors.

Analyzer stopped after the declarations?

Confirm that import json aligns with the other imports. Check that RULE_FIELDS starts at the left edge of the file.

Check that every string in the tuple ends with a comma. Confirm that the tuple closes with a parenthesis.

Help me fix the new rule declarations.

A valid file can still contain unusable policy values. The load_rules() function will reject missing fields plus invalid thresholds or labels before analysis continues.

  • Find def correlate_incident(events): in case_trace.py.
  • Place your cursor directly above that function.
  • Add the configuration loader by pasting this function:
def load_rules(path):
    with path.open(mode="r", encoding="utf-8") as source:
        rules = json.load(source)

    if not isinstance(rules, dict):
        raise ValueError("Rules must be a JSON object.")

    missing = [field for field in RULE_FIELDS if field not in rules]
    if missing:
        raise ValueError("Missing rule field(s): " + ", ".join(missing))

    minimum_failures = rules["minimum_failures"]
    if (
        not isinstance(minimum_failures, int)
        or isinstance(minimum_failures, bool)
        or minimum_failures < 1
    ):
        raise ValueError("minimum_failures must be a positive integer.")

    for field in ("base_severity", "warning_severity"):
        if not isinstance(rules[field], str) or not rules[field].strip():
            raise ValueError(f"{field} must be a non-empty string.")

    return rules

How does the loader protect CaseTrace?

  • The file is opened as UTF-8 text before json.load() converts it into a Python value.
  • The first check requires the top-level value to be an object.
  • The missing-field check compares the object with every name in RULE_FIELDS.
  • The threshold check requires a positive integer while explicitly rejecting Boolean values.
  • The final loop requires both severity labels to contain non-whitespace text.
  • Save case_trace.py.
  • Confirm that the loader has not interrupted the analyzer by running:
python case_trace.py

What should this run confirm?

You should see the same event counts plus the inherited single CRITICAL incident. This confirms that load_rules() is syntactically valid before main() starts using it.

Seeing a syntax or indentation error?

Confirm that load_rules() starts at the left edge of the file. Its internal statements must use consistent four-space indentation.

Check the closing parenthesis beneath the three-part threshold condition. Confirm that return rules remains inside the function.

Help me debug the rule loader.

Connect the rules to main

The execution path now needs to locate the configuration before analysis begins. It will load plus display the rules while continuing to call the inherited correlate_incident(events) function.

  • Find source_path = Path("data/case_log.csv") inside main().
  • Select the path declarations plus the existing source-file check.
  • Replace that selection with this configuration-aware path block:
    source_path = Path("data/case_log.csv")
    rules_path = Path("rules.json")
    report_path = Path("output/investigation_report.md")
    hash_path = Path("output/evidence_hash.sha256")

    if not source_path.exists():
        print("ERROR: data/case_log.csv was not found.")
        return 1

    if not rules_path.exists():
        print("ERROR: rules.json was not found.")
        return 1

What does the new path check do?

The rules_path value points to the configuration beside case_trace.py. The second existence check stops the analyzer with a specific diagnostic when that file is unavailable.

  • Find findings = baseline_findings(events) inside main().
  • Replace that line with these two assignments:
    findings = baseline_findings(events)
    rules = load_rules(rules_path)

When are the rules validated?

The analyzer validates rules.json after loading the evidence. An invalid configuration now stops execution before the inherited correlation result is produced.

  • Save case_trace.py.
  • Confirm that the saved configuration passes validation by running:
python case_trace.py

What does a complete run prove?

The analyzer should reach its existing incident output without a configuration exception. That result confirms that rules.json passed every check in load_rules().

  • Find the loaded-event summary plus baseline-findings print statements inside main().
  • Replace those two statements with this rule-aware output block:
    print(f"Loaded {len(events)} valid events; skipped {skipped} malformed row(s).")
    print(
        f"Rules loaded: minimum failures {rules['minimum_failures']}; "
        f"severities {rules['base_severity']}/{rules['warning_severity']}."
    )
    print(f"Baseline findings: {len(findings)} individual suspicious event(s).")

Why print the policy?

The new summary exposes the exact policy used for the run. This makes the threshold plus both severity outcomes visible before correlation reports its conclusion.

  • Save case_trace.py.

Before you run the final check, which configuration summary do you expect to appear before the inherited incident result?

  • Run the configuration-aware analyzer from the integrated terminal:
python case_trace.py

What should you see?

The terminal prints Rules loaded: minimum failures 3; severities HIGH/CRITICAL. before the inherited correlated conclusion.

You should still see only the CRITICAL result for jrivera on workstation-07. That single result confirms this intermediate version preserves the early-return behavior exposed in the previous step.

Rules line missing or analyzer stopped?

Confirm that rules = load_rules(rules_path) appears before the print statements. Check that the rules summary remains indented inside main().

If configuration loading fails, compare rules.json with the full-file reference above. Check the threshold type plus both severity strings.

Help me diagnose the configuration run.

Your detection policy now lives in a validated configuration file. Next, you'll replace the early return with an ordered collection that preserves both CRITICAL plus HIGH incidents.

Correlate Every Incident

Your JSON rules now separate detection policy from the correlation code. The Python analyzer still follows its inherited one-result path.

This step turns correlation into a collection of complete sequences. Each sequence keeps the severity supported by its own evidence.

In this step, get ready to:
  • Collect every complete sequence in timestamp order.
  • Write each configured severity to the Markdown report.
  • Regenerate the evidence integrity record.
Collect every complete sequence

A multi-incident collector needs to finish scanning every user before it returns. It also needs the configured failure threshold during each sequence check.

  • Switch back to case_trace.py in the Visual Studio Code editor.
  • Find the inherited correlate_incident(events) function.
  • Place your cursor on the blank line immediately after that function.
  • Start the new collector by pasting this first section:
def correlate_incidents(events, rules):
    incidents = []
    users = sorted({event["user"] for event in events})
    minimum_failures = rules["minimum_failures"]

    for user in users:
        user_events = [event for event in events if event["user"] == user]
        failures = [
            event
            for event in user_events
            if event["event_type"] == "AUTH_FAILURE"
        ]
        successes = [
            event
            for event in user_events
            if event["event_type"] == "AUTH_SUCCESS"
        ]

How the collector starts

  • The incidents list holds every completed conclusion until the scan finishes.
  • The sorted user set gives the function a stable scanning order.
  • The minimum_failures value comes from the validated rules dictionary.
  • Continue the same function with the success sequence checks:
        for success in successes:
            device = success["device"]
            earlier_failures = [
                event
                for event in failures
                if event["device"] == device
                and event["timestamp"] < success["timestamp"]
            ]
            sensitive_accesses = [
                event
                for event in user_events
                if event["device"] == device
                and event["event_type"] == "FILE_ACCESS"
                and event["timestamp"] > success["timestamp"]
            ]

            if len(earlier_failures) < minimum_failures or not sensitive_accesses:
                continue

How sequences qualify

  • Each successful sign-in becomes a possible sequence boundary.
  • Earlier failures must belong to the same device.
  • A later sensitive file access must exist before correlation continues.
  • Extend the sequence through its outbound activity:
            sensitive_access = sensitive_accesses[0]
            outbound_connections = [
                event
                for event in user_events
                if event["device"] == device
                and event["event_type"] == "OUTBOUND_CONNECTION"
                and event["timestamp"] > sensitive_access["timestamp"]
            ]

            if not outbound_connections:
                continue

            outbound = outbound_connections[0]
            device_warnings = [
                event
                for event in events
                if event["device"] == device
                and event["event_type"] == "DEVICE_WARNING"
                and event["timestamp"] > outbound["timestamp"]
            ]

How context stays connected

  • The first outbound connection after file access completes the base incident pattern.
  • Device warnings come from all events because their user is system.
  • The warning must occur on the same device after the outbound event.
  • Assign the configured severity to each completed sequence:
            severity = (
                rules["warning_severity"]
                if device_warnings
                else rules["base_severity"]
            )
            reason = (
                f"{minimum_failures} authentication failures were followed by "
                "a success, sensitive file access, and an outbound connection."
            )
            evidence = (
                earlier_failures[-minimum_failures:]
                + [success, sensitive_access, outbound]
            )

How severity follows evidence

  • A later warning selects warning_severity from the rules.
  • A sequence without that warning keeps base_severity.
  • The evidence list retains the configured number of failures plus the three later events.
  • Finish the collector by storing each incident:
            if device_warnings:
                reason += " A later device warning increases operational concern."
                evidence.append(device_warnings[0])

            incidents.append(
                {
                    "severity": severity,
                    "user": user,
                    "device": device,
                    "reason": reason,
                    "evidence": evidence,
                }
            )

    return sorted(
        incidents,
        key=lambda incident: incident["evidence"][0]["timestamp"],
    )

How every incident survives

  • The warning becomes supporting evidence only when it exists.
  • Each completed incident is appended instead of returned immediately.
  • The final sort uses the timestamp of the first retained evidence event.
  • Save case_trace.py.

Before you run the analyzer, do you expect its incident count to change before main() calls the new collector?

  • Check that the new function parses by running this command in the existing PowerShell terminal:
python case_trace.py

What does this check prove?

Python parses the new function when it loads case_trace.py. The unchanged call site still uses the inherited single-result function.

You should still see one correlated incident for jrivera. This successful run confirms the collector has valid syntax before you connect it to the output path.

Seeing a syntax or indentation error?

Check that every continuation begins inside correlate_incidents(). The for success in successes: line needs eight leading spaces.

Compare the closing parentheses around incidents.append() with the snippets above.

Help me fix my correlation function.

Wire the incident list into outputs

The new collector produces a list, so the Markdown report must render each item. The main flow must pass the loaded rules into both correlation and reporting.

  • Find the complete build_report() function in case_trace.py.
  • Select the complete function definition.
  • Replace the selection by pasting these report sections in order:
def build_report(source_path, events, skipped, findings, incidents, digest, rules):
    lines = [
        "# CaseTrace Investigation Report",
        "",
        "Scope: Synthetic training data only.",
        f"Evidence source: `{source_path.as_posix()}`",
        f"SHA-256: `{digest}`",
        f"Valid events: {len(events)}",
        f"Malformed rows skipped: {skipped}",
        f"Baseline findings: {len(findings)}",
        (
            f"Rules: minimum failures {rules['minimum_failures']}; "
            f"base {rules['base_severity']}; "
            f"warning {rules['warning_severity']}"
        ),
        "",
        "## Correlated incidents",
        "",
    ]

How the report header changes

  • The function now receives incidents as a collection.
  • The report records the threshold plus both configured severity labels.
  • The existing evidence counts remain part of the report header.
  • Continue the report function with numbered incident sections:
    if incidents:
        for number, incident in enumerate(incidents, start=1):
            lines.extend(
                [
                    f"### Incident {number}",
                    "",
                    f"Severity: {incident['severity']}",
                    f"User: {incident['user']}",
                    f"Device: {incident['device']}",
                    f"Reason: {incident['reason']}",
                    "",
                ]
            )
    else:
        lines.extend(["No coordinated incidents identified.", ""])

How incidents become report sections

  • The loop creates one numbered heading per incident.
  • Each section preserves its severity plus its user context.
  • The empty-list branch keeps the report readable when no sequence qualifies.
  • Finish the report with the shared timeline plus its limitations:
    lines.extend(["## Timeline", ""])
    for event in events:
        lines.append(
            f"- {event['timestamp']} | {event['device']} | "
            f"{event['user']} | {event['event_type']} | {event['detail']}"
        )

    lines.extend(
        [
            "",
            "## Limitations",
            "",
            "This educational rule set analyzes one known CSV schema and does not "
            "establish identity, intent, or legal conclusions.",
        ]
    )
    return "\n".join(lines) + "\n"

How the report closes

The complete event timeline remains independent from the incident count. The final newline keeps the generated Markdown file well formed.

  • Find the complete main() function near the bottom of case_trace.py.
  • Select the complete function definition.
  • Include the final module entry point in your selection.
  • Replace the selection by pasting these main-flow sections in order:
def main():
    source_path = Path("data/case_log.csv")
    rules_path = Path("rules.json")
    report_path = Path("output/investigation_report.md")
    hash_path = Path("output/evidence_hash.sha256")

    if not source_path.exists():
        print("ERROR: data/case_log.csv was not found.")
        return 1

    if not rules_path.exists():
        print("ERROR: rules.json was not found.")
        return 1

    events, skipped = load_events(source_path)
    findings = baseline_findings(events)
    rules = load_rules(rules_path)
    incidents = correlate_incidents(events, rules)

How main collects incidents

  • The existing file checks still protect both required inputs.
  • The validated rules now enter correlate_incidents() with the parsed events.
  • The result is stored as the incidents list.
  • Continue main() with the multi-incident terminal output:
    print(f"Loaded {len(events)} valid events; skipped {skipped} malformed row(s).")
    print(
        f"Rules loaded: minimum failures {rules['minimum_failures']}; "
        f"severities {rules['base_severity']}/{rules['warning_severity']}."
    )
    print(f"Baseline findings: {len(findings)} individual suspicious event(s).")
    print(f"Correlated incidents: {len(incidents)}.")
    for incident in incidents:
        print(
            f"- {incident['severity']} severity for "
            f"{incident['user']} on {incident['device']}."
        )

    digest = integrity_hash(source_path)
    report = build_report(
        source_path,
        events,
        skipped,
        findings,
        incidents,
        digest,
        rules,
    )

How main exposes every result

  • The incident count comes from the completed collection.
  • The loop prints one conclusion for each incident in sorted order.
  • The report receives the same collection plus the loaded rules.
  • Finish main() with regenerated output files:
    report_path.parent.mkdir(parents=True, exist_ok=True)
    report_path.write_text(report, encoding="utf-8")
    hash_path.write_text(
        f"{digest}  {source_path.as_posix()}\n",
        encoding="utf-8",
    )

    print(f"Integrity record: {hash_path.as_posix()}")
    print(f"Report written: {report_path.as_posix()}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

How the outputs stay trustworthy

The report is rewritten from the current run. The evidence hash is regenerated from the unchanged CSV bytes.

  • Save case_trace.py.

Before you run the updated path, how many conclusions should a collector return from two complete sequences?

  • Run the updated analyzer from the existing PowerShell terminal with this command:
python case_trace.py

What does this run check?

This run sends the loaded rules through the new collector. It also regenerates the report plus the integrity record.

You should see 15 valid events. You should also see one skipped malformed row.

The next lines show 10 baseline findings. The correlated incident count is 2.

The ordered conclusions show CRITICAL for jrivera on workstation-07. They show HIGH for mchen on workstation-11.

That is the blind spot closed. Both complete sequences now reach the terminal with their evidence-supported severities intact.

Still seeing one incident?

Check that main() calls correlate_incidents(events, rules).

Check that incidents.append() remains inside the success loop. The final return sorted() must remain outside the user loop.

Help me find the missing incident.

Verify both conclusions

The successful run proves that the new path works. Removing the inherited single-result function leaves one authoritative correlation implementation.

  • Find the inherited correlate_incident(events) definition in case_trace.py.
  • Select the complete inherited function definition.
  • Delete the selected function.
  • Save case_trace.py.

✔️ Awesome, I've got everything!

Great. Double check that case_trace.py is saved without the inherited single-result function.

ⓧ I'd like to double check the full code

import csv
import hashlib
import json
from pathlib import Path

REQUIRED_FIELDS = (
    "timestamp",
    "device",
    "user",
    "source_ip",
    "event_type",
    "detail",
)

SUSPICIOUS_TYPES = {
    "AUTH_FAILURE",
    "FILE_ACCESS",
    "OUTBOUND_CONNECTION",
}

RULE_FIELDS = (
    "minimum_failures",
    "base_severity",
    "warning_severity",
)


def load_events(path):
    events = []
    skipped = 0

    with path.open(mode="r", encoding="utf-8", newline="") as source:
        reader = csv.DictReader(source)
        for row in reader:
            is_valid = row and all(
                (row.get(field) or "").strip() for field in REQUIRED_FIELDS
            )
            if not is_valid:
                skipped += 1
                continue

            event = {
                field: row[field].strip()
                for field in REQUIRED_FIELDS
            }
            events.append(event)

    return sorted(events, key=lambda event: event["timestamp"]), skipped


def baseline_findings(events):
    return [
        event
        for event in events
        if event["event_type"] in SUSPICIOUS_TYPES
    ]


def load_rules(path):
    with path.open(mode="r", encoding="utf-8") as source:
        rules = json.load(source)

    if not isinstance(rules, dict):
        raise ValueError("Rules must be a JSON object.")

    missing = [field for field in RULE_FIELDS if field not in rules]
    if missing:
        raise ValueError("Missing rule field(s): " + ", ".join(missing))

    minimum_failures = rules["minimum_failures"]
    if (
        not isinstance(minimum_failures, int)
        or isinstance(minimum_failures, bool)
        or minimum_failures < 1
    ):
        raise ValueError("minimum_failures must be a positive integer.")

    for field in ("base_severity", "warning_severity"):
        if not isinstance(rules[field], str) or not rules[field].strip():
            raise ValueError(f"{field} must be a non-empty string.")

    return rules


def correlate_incidents(events, rules):
    incidents = []
    users = sorted({event["user"] for event in events})
    minimum_failures = rules["minimum_failures"]

    for user in users:
        user_events = [event for event in events if event["user"] == user]
        failures = [
            event
            for event in user_events
            if event["event_type"] == "AUTH_FAILURE"
        ]
        successes = [
            event
            for event in user_events
            if event["event_type"] == "AUTH_SUCCESS"
        ]

        for success in successes:
            device = success["device"]
            earlier_failures = [
                event
                for event in failures
                if event["device"] == device
                and event["timestamp"] < success["timestamp"]
            ]
            sensitive_accesses = [
                event
                for event in user_events
                if event["device"] == device
                and event["event_type"] == "FILE_ACCESS"
                and event["timestamp"] > success["timestamp"]
            ]

            if len(earlier_failures) < minimum_failures or not sensitive_accesses:
                continue

            sensitive_access = sensitive_accesses[0]
            outbound_connections = [
                event
                for event in user_events
                if event["device"] == device
                and event["event_type"] == "OUTBOUND_CONNECTION"
                and event["timestamp"] > sensitive_access["timestamp"]
            ]

            if not outbound_connections:
                continue

            outbound = outbound_connections[0]
            device_warnings = [
                event
                for event in events
                if event["device"] == device
                and event["event_type"] == "DEVICE_WARNING"
                and event["timestamp"] > outbound["timestamp"]
            ]

            severity = (
                rules["warning_severity"]
                if device_warnings
                else rules["base_severity"]
            )
            reason = (
                f"{minimum_failures} authentication failures were followed by "
                "a success, sensitive file access, and an outbound connection."
            )
            evidence = (
                earlier_failures[-minimum_failures:]
                + [success, sensitive_access, outbound]
            )

            if device_warnings:
                reason += " A later device warning increases operational concern."
                evidence.append(device_warnings[0])

            incidents.append(
                {
                    "severity": severity,
                    "user": user,
                    "device": device,
                    "reason": reason,
                    "evidence": evidence,
                }
            )

    return sorted(
        incidents,
        key=lambda incident: incident["evidence"][0]["timestamp"],
    )


def integrity_hash(path):
    return hashlib.sha256(path.read_bytes()).hexdigest()


def build_report(source_path, events, skipped, findings, incidents, digest, rules):
    lines = [
        "# CaseTrace Investigation Report",
        "",
        "Scope: Synthetic training data only.",
        f"Evidence source: `{source_path.as_posix()}`",
        f"SHA-256: `{digest}`",
        f"Valid events: {len(events)}",
        f"Malformed rows skipped: {skipped}",
        f"Baseline findings: {len(findings)}",
        (
            f"Rules: minimum failures {rules['minimum_failures']}; "
            f"base {rules['base_severity']}; "
            f"warning {rules['warning_severity']}"
        ),
        "",
        "## Correlated incidents",
        "",
    ]

    if incidents:
        for number, incident in enumerate(incidents, start=1):
            lines.extend(
                [
                    f"### Incident {number}",
                    "",
                    f"Severity: {incident['severity']}",
                    f"User: {incident['user']}",
                    f"Device: {incident['device']}",
                    f"Reason: {incident['reason']}",
                    "",
                ]
            )
    else:
        lines.extend(["No coordinated incidents identified.", ""])

    lines.extend(["## Timeline", ""])
    for event in events:
        lines.append(
            f"- {event['timestamp']} | {event['device']} | "
            f"{event['user']} | {event['event_type']} | {event['detail']}"
        )

    lines.extend(
        [
            "",
            "## Limitations",
            "",
            "This educational rule set analyzes one known CSV schema and does not "
            "establish identity, intent, or legal conclusions.",
        ]
    )
    return "\n".join(lines) + "\n"


def main():
    source_path = Path("data/case_log.csv")
    rules_path = Path("rules.json")
    report_path = Path("output/investigation_report.md")
    hash_path = Path("output/evidence_hash.sha256")

    if not source_path.exists():
        print("ERROR: data/case_log.csv was not found.")
        return 1

    if not rules_path.exists():
        print("ERROR: rules.json was not found.")
        return 1

    events, skipped = load_events(source_path)
    findings = baseline_findings(events)
    rules = load_rules(rules_path)
    incidents = correlate_incidents(events, rules)

    print(f"Loaded {len(events)} valid events; skipped {skipped} malformed row(s).")
    print(
        f"Rules loaded: minimum failures {rules['minimum_failures']}; "
        f"severities {rules['base_severity']}/{rules['warning_severity']}."
    )
    print(f"Baseline findings: {len(findings)} individual suspicious event(s).")
    print(f"Correlated incidents: {len(incidents)}.")
    for incident in incidents:
        print(
            f"- {incident['severity']} severity for "
            f"{incident['user']} on {incident['device']}."
        )

    digest = integrity_hash(source_path)
    report = build_report(
        source_path,
        events,
        skipped,
        findings,
        incidents,
        digest,
        rules,
    )

    report_path.parent.mkdir(parents=True, exist_ok=True)
    report_path.write_text(report, encoding="utf-8")
    hash_path.write_text(
        f"{digest}  {source_path.as_posix()}\n",
        encoding="utf-8",
    )

    print(f"Integrity record: {hash_path.as_posix()}")
    print(f"Report written: {report_path.as_posix()}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

How to use the full-file reference

Compare the collector plus the two output functions with your saved file. The reference contains the exact cumulative analyzer for this step.

Before the final run, which incident do you expect to appear first after sorting by its earliest evidence timestamp?

  • Run the finished analyzer from the existing PowerShell terminal with this command:
python case_trace.py

What does the final run verify?

The command executes the finished collector without the inherited function. It rewrites both files under output from the current evidence.

You should see 15 valid events plus one skipped malformed row. You should also see 10 baseline findings plus two correlated incidents.

The first conclusion is CRITICAL for jrivera on workstation-07. The second is HIGH for mchen on workstation-11.

  • Select output/investigation_report.md in the Visual Studio Code Explorer sidebar.

You should see Incident 1 with the CRITICAL conclusion. You should see Incident 2 with the HIGH conclusion.

  • Select output/evidence_hash.sha256 in the Explorer sidebar.

You should see a hexadecimal SHA-256 digest followed by data/case_log.csv.

Missing a report section or hash record?

Check that build_report() loops over incidents. Its empty-list branch should run only when the collection has no items.

Check that both output-writing calls remain below the build_report() call in main().

Help me verify my generated outputs.

Both incidents now survive correlation with their distinct severities intact. Next up, you will protect these contracts with seven focused regression tests.

Protect the Behavior with Tests

CaseTrace now finds both synthetic incidents while preserving their separate severity outcomes. A later refactor could silently break the inherited counts, negative cases, evidence hashing, or severity escalation.

Regression testing turns each expected behavior into an independent contract. The same Python 3.15.2 runtime in Visual Studio Code can check every contract with one command.

In this step, get ready to:
  • Create shared evidence and rule fixtures for the test suite.
  • Protect the two-incident result with separate HIGH and CRITICAL assertions.
  • Retain the negative-case and evidence-integrity checks.
Build the test foundation

A test fixture prepares the same evidence and rules before every check. This keeps each test focused on one behavior.

  • Switch back to test_case_trace.py in the editor.
  • Replace its contents with the shared imports and paths by pasting this code:
import unittest
from pathlib import Path

from case_trace import baseline_findings
from case_trace import correlate_incidents
from case_trace import integrity_hash
from case_trace import load_events
from case_trace import load_rules

CASE_LOG = Path("data/case_log.csv")
RULES_PATH = Path("rules.json")

What does this foundation provide?

  • unittest provides the test case and assertion tools.
  • CASE_LOG points every test to the expanded synthetic evidence.
  • RULES_PATH points every test to the validated JSON configuration.
  • Save test_case_trace.py.
  • Check that test discovery can import the new foundation by running:
python -m unittest

What does this check prove?

Test discovery loads test_case_trace.py through Python. A completed run confirms that the imports and paths contain no syntax errors.

The command completes without an import failure. Your test module can now load the CaseTrace functions.

Seeing an import failure?

Confirm that every imported function name matches case_trace.py. Check that the terminal still points to the CaseTrace workspace.

Help me fix the test imports.

  • Add the shared fixture and dataset-count test below RULES_PATH by pasting this code:
class CaseTraceTests(unittest.TestCase):
    def setUp(self):
        self.events, self.skipped = load_events(CASE_LOG)
        self.rules = load_rules(RULES_PATH)

    def test_known_dataset_counts(self):
        self.assertEqual(len(self.events), 15)
        self.assertEqual(self.skipped, 1)
        self.assertEqual(len(baseline_findings(self.events)), 10)

What does this test protect?

setUp() loads fresh events and rules before each test. The first test locks in 15 valid events, one rejected row, and 10 baseline findings.

  • Save test_case_trace.py.
  • Run the growing suite with:
python -m unittest

What does this result prove?

The first passing test confirms that the parser still accepts the expanded evidence correctly. It also confirms that the baseline detector still returns the expected count.

You should see one passing test. The inherited parser counts are now protected.

Is the count test failing?

Confirm that data/case_log.csv still contains the complete 16-row dataset including its header. Check that the malformed row still has an empty user field.

Help me trace the count mismatch.

Test configuration and incident outcomes

The new correlation behavior depends on both the configured threshold and the configured severity labels. Separate tests reveal which contract changed when a future edit causes a failure.

  • Add the configuration test below test_known_dataset_counts() by pasting this method:
    def test_loads_rule_configuration(self):
        self.assertEqual(self.rules["minimum_failures"], 3)
        self.assertEqual(self.rules["base_severity"], "HIGH")
        self.assertEqual(self.rules["warning_severity"], "CRITICAL")

What does this test protect?

This test confirms that load_rules() returns the threshold and both labels from rules.json. A configuration regression now fails independently from correlation logic.

  • Save test_case_trace.py.
  • Check the configuration assertion by running:
python -m unittest

What does this result prove?

A passing suite confirms that the analyzer loads the configured threshold and severity labels. It also confirms that the earlier count contract still holds.

You should see two passing tests. The rule configuration is now covered.

Is the rule test failing?

Compare the three values in rules.json with the assertions. Confirm that the threshold remains a number and both severity labels remain strings.

Help me debug the rule assertions.

  • Add the incident-count test below test_loads_rule_configuration() by pasting this method:
    def test_finds_two_incidents(self):
        incidents = correlate_incidents(self.events, self.rules)
        self.assertEqual(len(incidents), 2)

What does this test protect?

This test passes the loaded evidence and rules into correlate_incidents(). The assertion prevents the old single-result blind spot from returning unnoticed.

  • Save test_case_trace.py.
  • Check the multi-incident assertion by running:
python -m unittest

What does this result prove?

A passing suite confirms that both complete evidence sequences become incidents. The collection behavior now has its own regression check.

You should see three passing tests. The early-return bug can no longer hide behind a successful run.

Does the suite find one incident?

Check that the test calls correlate_incidents() with both self.events and self.rules. Confirm that the function appends complete sequences before returning the sorted list.

Help me trace the missing incident.

  • Add the severity-preservation test below test_finds_two_incidents() by pasting this method:
    def test_preserves_high_and_critical_outcomes(self):
        incidents = correlate_incidents(self.events, self.rules)
        by_user = {incident["user"]: incident for incident in incidents}
        self.assertEqual(by_user["jrivera"]["severity"], "CRITICAL")
        self.assertEqual(by_user["jrivera"]["device"], "workstation-07")
        self.assertEqual(by_user["mchen"]["severity"], "HIGH")
        self.assertEqual(by_user["mchen"]["device"], "workstation-11")

What does this test protect?

by_user makes each incident directly addressable by its user. The assertions preserve the CRITICAL warning escalation for jrivera and the HIGH base outcome for mchen.

  • Save test_case_trace.py.
  • Check both severity paths by running:
python -m unittest

What does this result prove?

A passing suite confirms that the warning path remains CRITICAL. It also confirms that a complete sequence without a later warning remains HIGH.

You should see four passing tests. Both valid severity outcomes now have explicit protection.

Is one severity assertion failing?

Confirm that warning escalation checks for a later DEVICE_WARNING on the same device. Confirm that the base path reads base_severity from the loaded rules.

Help me debug the severity paths.

Cover negative cases and integrity

Positive results prove that CaseTrace finds known incidents. Negative and integrity checks prove that it avoids unsupported conclusions while preserving the source-evidence contract.

  • Add the routine-event test below test_preserves_high_and_critical_outcomes() by pasting this method:
    def test_routine_events_do_not_create_incident(self):
        routine_events = [
            event
            for event in self.events
            if event["user"] == "asmith"
            or event["event_type"] == "DEVICE_HEALTH"
        ]
        self.assertEqual(correlate_incidents(routine_events, self.rules), [])

What does this test protect?

This test isolates the routine sign-in and device-health events. An empty incident list confirms that CaseTrace needs the complete suspicious sequence before producing a conclusion.

  • Save test_case_trace.py.
  • Check the negative case by running:
python -m unittest

What does this result prove?

A passing suite confirms that ordinary activity does not become an incident. This guards against false conclusions from incomplete evidence.

You should see five passing tests. The routine-event boundary remains intact.

Do routine events create an incident?

Check the event filter for asmith and DEVICE_HEALTH. Confirm that the assertion passes the filtered list into correlate_incidents().

Help me inspect the negative case.

  • Add the malformed-row test below test_routine_events_do_not_create_incident() by pasting this method:
    def test_malformed_row_is_rejected(self):
        users = {event["user"] for event in self.events}
        self.assertNotIn("", users)
        self.assertEqual(self.skipped, 1)

What does this test protect?

The user set must exclude the empty value from the malformed CSV row. The skipped count confirms that the parser rejected exactly one row.

  • Save test_case_trace.py.
  • Check malformed-row rejection by running:
python -m unittest

What does this result prove?

A passing suite confirms that incomplete evidence never enters the valid event collection. It also preserves the parser's skipped-row count.

You should see six passing tests. The malformed evidence boundary is protected.

Is the empty user still present?

Check that load_events() validates every required field before appending an event. Confirm that the malformed CSV row still has no user value.

Help me trace malformed-row handling.

  • Add the repeatable-hash test below test_malformed_row_is_rejected() by pasting this method:
    def test_hash_is_repeatable_for_unchanged_source(self):
        first_digest = integrity_hash(CASE_LOG)
        second_digest = integrity_hash(CASE_LOG)
        self.assertEqual(first_digest, second_digest)

What does this test protect?

The test hashes the unchanged evidence twice. Matching digests confirm that the SHA-256 integrity function returns a repeatable record for the same bytes.

  • Save test_case_trace.py.
  • Check the integrity assertion by running:
python -m unittest

What does this result prove?

A passing suite confirms that unchanged source evidence produces the same digest on repeated reads. All seven behavior checks are now present.

You should see seven passing tests. The evidence-integrity contract remains stable.

Is the hash test failing?

Confirm that both digest calls use the same CASE_LOG path. Check that no other process changes data/case_log.csv between the two reads.

Help me debug the integrity check.

  • Add the direct-run entry point after the CaseTraceTests class by pasting this code:
if __name__ == "__main__":
    unittest.main()

What does this entry point do?

unittest.main() runs the suite when the test file is executed directly. Test discovery can still run the same methods through the module command.

  • Save test_case_trace.py.

Before you run the final check, do you expect every parser, configuration, correlation, negative-case, and integrity contract to pass?

  • Run the complete regression suite with:
python -m unittest

What does the final run prove?

One command now checks seven independent contracts against the real project fixtures. A future behavior change identifies its broken contract through the failing test method.

The terminal reports Ran 7 tests followed by OK. That success proves the multi-incident refactor preserved the inherited boundaries.

Is one of the seven tests failing?

Read the failing method name first. Match that name to the parser, rule loader, correlation function, negative case, or hash function that it protects.

Help me trace the failing test.

✔️ Awesome, I've got everything!

Great work. Your seven-test suite now protects both incident results and the evidence boundaries around them.

ⓧ I'd like to double check the full code

import unittest
from pathlib import Path

from case_trace import baseline_findings
from case_trace import correlate_incidents
from case_trace import integrity_hash
from case_trace import load_events
from case_trace import load_rules

CASE_LOG = Path("data/case_log.csv")
RULES_PATH = Path("rules.json")


class CaseTraceTests(unittest.TestCase):
    def setUp(self):
        self.events, self.skipped = load_events(CASE_LOG)
        self.rules = load_rules(RULES_PATH)

    def test_known_dataset_counts(self):
        self.assertEqual(len(self.events), 15)
        self.assertEqual(self.skipped, 1)
        self.assertEqual(len(baseline_findings(self.events)), 10)

    def test_loads_rule_configuration(self):
        self.assertEqual(self.rules["minimum_failures"], 3)
        self.assertEqual(self.rules["base_severity"], "HIGH")
        self.assertEqual(self.rules["warning_severity"], "CRITICAL")

    def test_finds_two_incidents(self):
        incidents = correlate_incidents(self.events, self.rules)
        self.assertEqual(len(incidents), 2)

    def test_preserves_high_and_critical_outcomes(self):
        incidents = correlate_incidents(self.events, self.rules)
        by_user = {incident["user"]: incident for incident in incidents}
        self.assertEqual(by_user["jrivera"]["severity"], "CRITICAL")
        self.assertEqual(by_user["jrivera"]["device"], "workstation-07")
        self.assertEqual(by_user["mchen"]["severity"], "HIGH")
        self.assertEqual(by_user["mchen"]["device"], "workstation-11")

    def test_routine_events_do_not_create_incident(self):
        routine_events = [
            event
            for event in self.events
            if event["user"] == "asmith"
            or event["event_type"] == "DEVICE_HEALTH"
        ]
        self.assertEqual(correlate_incidents(routine_events, self.rules), [])

    def test_malformed_row_is_rejected(self):
        users = {event["user"] for event in self.events}
        self.assertNotIn("", users)
        self.assertEqual(self.skipped, 1)

    def test_hash_is_repeatable_for_unchanged_source(self):
        first_digest = integrity_hash(CASE_LOG)
        second_digest = integrity_hash(CASE_LOG)
        self.assertEqual(first_digest, second_digest)


if __name__ == "__main__":
    unittest.main()

How should I compare this file?

Check the imports and shared paths first. Then compare the seven test methods in order before confirming the final direct-run entry point.

That locks in the behavior you built. CaseTrace can now evolve without silently losing incidents, severity context, or evidence integrity.

Secret mission

Prove the Threshold Is Configurable

Your analyzer already reads its failure threshold from `rules.json`. This mission adds an eighth regression test that raises the threshold to `4` in memory. Both three-failure incidents should disappear while the baseline configuration remains intact.

Clean Up Your Resources

Clean Up Your Resources

CaseTrace uses local files only, so it creates no ongoing charges. Choose whether to keep the workspace, close it for later, or delete it.

Resources you used:

  • The local CaseTrace workspace, including case_trace.py, test_case_trace.py, rules.json, data/case_log.csv, output/investigation_report.md, and output/evidence_hash.sha256.

Keep everything running

No action is needed. Choose this option if you want to keep testing correlation rules or preserve the workspace for the risk-scoring follow-on project.

  • Keep the CaseTrace workspace on your computer.
  • Retain the output folder as the investigator-readable record of your current evidence.

Pause - I'll come back to this later

Pausing means closing the editor while leaving every CaseTrace file intact. No service needs to be stopped.

  • Save every open CaseTrace file in Visual Studio Code.
  • Close Visual Studio Code.

Delete - I don't want to use this again

Deleting the workspace removes your analyzer, synthetic evidence, configuration, tests, report, and hash record. Copy anything important before removing the folder.

  • Copy any CaseTrace files you want to keep to another folder.
  • Press the Windows key to open the search bar.
  • Type File Explorer into the search bar.
  • Press Enter to open File Explorer.
  • Navigate to the location that contains the CaseTrace folder.
  • Select the CaseTrace folder.
  • Press the Delete key.
  • Confirm the deletion if Windows asks for confirmation.
  • Check that the CaseTrace folder no longer appears in File Explorer.

Nice Work!

Nice Work!

Outstanding work! CaseTrace now uses configurable multi-incident correlation while preserving separate HIGH and CRITICAL outcomes.

You've learned how to:

  • Reproduced the early-return blind spot with a second complete incident sequence. Replaced the single result with an ordered collection from correlate_incidents().
  • Built a validated JSON configuration in rules.json. Kept HIGH and CRITICAL as distinct severity outcomes.
  • Generated an investigator-readable report containing both incidents. Preserved evidence integrity with a repeatable SHA-256 record. Protected the analyzer with seven regression tests.
  • Secret Mission: Added an eighth test that raises minimum_failures to 4. Proved both three-failure sequences are suppressed.

Ready to quiz yourself?