Build a Detection Rule Lifecycle Pipeline

Manage SIEM detection rules as code with CI/CD, testing, and lifecycle states.

Introduction

30 Second Summary

Imagine writing a fire alarm rule for your building, but having no way to test it, no record of who changed the sensitivity last month, and no warning when the sensor model gets swapped out. That is what most security teams face with their threat detection rules.

In this project, you will build a complete detection-as-code infrastructure that manages SIEM detection rules through Git, validates them automatically with a CI pipeline, and documents every tuning decision along the way.

What You'll Build

When you push a new detection rule to your repository, you will watch a pipeline automatically validate its syntax, convert it to Elastic query language, and test it against sample events. All without touching the SIEM UI.

By the end of this project, you'll have:

  • A Sigma rule you authored from scratch that converts to Elastic EQL for your cluster, with fixture tests that prove it catches real attack patterns and ignores benign activity.
  • A GitHub Actions CI pipeline that lints, converts, and tests every rule on every pull request, catching silent breakage from schema changes that syntax validation alone would miss.
  • A documented shadow-mode-to-production workflow with explicit promotion criteria, tuning templates, and decision records so every rule change is traceable and evidence-based.
  • Secret Mission: Automate the entire rule authoring and validation workflow into a single Claude Code slash command that your team can reuse for every future detection rule.

Are there any prerequisites?

None. This guide walks you through installing every tool you need. You will set up sigma-cli, the Elasticsearch backend plugin, and your repository structure from scratch in Step 1.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters for your team's detection capability.

Set Up the Detection Engineering Toolchain

Detection rules that live only in a SIEM's UI cannot be versioned, tested, or shared across teams. To treat rules as proper code, you need a local authoring environment that can validate and convert them before they ever touch production.

In this step, you will install sigma-cli for writing portable detection rules, set up a Git repository with an enforced folder structure, and push it to GitHub so your team can collaborate on rules the same way developers collaborate on application code.

In this step, get ready to:
  • Install sigma-cli and the Elasticsearch backend plugin in a Python virtual environment.
  • Initialize a Git repository with the detection-as-code folder structure.
  • Create a README documenting your repository structure and push to GitHub.
Create a virtual environment and install sigma-cli

Sigma is a SIEM-agnostic rule format. You write a rule once in Sigma, then convert it to whatever query language your SIEM speaks.

  • Open your terminal by pressing Cmd+Space (macOS) or the Windows key (Windows), then typing Terminal and pressing Enter.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir detection-rules
cd detection-rules
  • Create a Python virtual environment by running:
python3 -m venv .venv
  • Activate the virtual environment:

macOS

source .venv/bin/activate

Windows

.venv\Scripts\Activate.ps1

Your terminal prompt now shows (.venv) at the beginning, confirming the virtual environment is active.

What is a virtual environment?

A virtual environment is an isolated Python installation that keeps this project's packages separate from your system Python. This prevents version conflicts between projects.

  • Install sigma-cli and the Elasticsearch backend library by running:
pip install sigma-cli pySigma-backend-elasticsearch

You will see pip downloading and installing several packages. Wait for it to finish and return you to the prompt.

  • Register the Elasticsearch backend plugin with sigma-cli by running:
sigma plugin install elasticsearch
  • Confirm the plugin installed correctly by running:
sigma plugin list

You should see a table that includes an elasticsearch row with type backend and state stable.

Don't see elasticsearch in the list?

  • Make sure your virtual environment is active (you see (.venv) in your prompt). If not, re-run the activate command above.
  • Try running sigma plugin install elasticsearch again. It may have failed silently due to a network timeout.
  • help me troubleshoot sigma plugin install
Initialize the Git repository with detection-as-code folder structure

A detection-as-code repository needs an enforced structure so every team member puts rules, tests, and documentation in predictable locations.

  • Initialize a Git repository and create the folder structure by running these commands:
git init
mkdir -p rules/windows/process_creation
mkdir -p tests
mkdir -p docs/decisions
mkdir -p golden
mkdir -p scripts

What do these folders hold?

  • rules/ stores Sigma rules organized by log source (e.g., windows/process_creation/).
  • tests/ holds fixture events used to validate that rules match (or don't match) expected data.
  • docs/decisions/ contains decision records explaining why each rule exists and how it should be tuned.
  • golden/ stores known-good EQL conversion outputs for detecting silent drift.
  • scripts/ holds automation and test scripts that the CI pipeline runs.
  • Verify the folder structure was created by running:
ls

You should see docs, golden, rules, scripts, and tests listed in the output.

Create the README and push to GitHub

The README is the first thing a team member sees when they open the repository. It documents where everything lives and how to contribute a new rule.

  • Create README.md in the root of your detection-rules folder.
  • Open it in VS Code and paste the following content:
# Detection Rule Repository

## Structure

- `rules/` - Sigma detection rules organized by log source
- `tests/` - Test fixture events (true/false positives per rule)
- `docs/` - Lifecycle documentation and tuning templates
- `docs/decisions/` - Decision records for each detection rule
- `scripts/` - Automation and test scripts
- `golden/` - Known-good EQL outputs for drift detection

## Lifecycle States

- **experimental** - Newly authored, untested in production
- **test** - Shadow mode (non-alerting), under observation
- **stable** - Production, generating analyst alerts
- **deprecated** - Archived with documented retirement reason
## Contribution Workflow

1. Create a feature branch from main
2. Author a Sigma rule with full metadata
3. Add test fixtures (true_positive.json, false_positive.json)
4. Run `sigma check` to validate syntax
5. Run `sigma convert` to verify EQL output
6. Create a decision document in docs/decisions/
7. Open a pull request (CI validates automatically)
8. Obtain peer review before merging

What does this README communicate?

The structure section tells contributors exactly where to place new files. The lifecycle states section defines the four stages every rule passes through: experimental, test, stable, and deprecated.

The contribution workflow gives new team members a step-by-step process for adding rules safely. Every step has a purpose: validation catches syntax errors, fixtures catch logic errors, and peer review catches design errors.

  • Save README.md.
  • Stage all files and create the initial commit by running:
git add .
git commit -m "feat: initialize detection-as-code repository structure"

Git confirms the commit with a message showing the files added.

  • Open your browser and go to github.com.
  • Click the + icon in the top right corner, then select New repository.
  • Enter detection-rules as the repository name.
  • Leave all other settings at their defaults (do not initialize with a README since you already have one).
  • Click Create repository.

GitHub shows a page with setup instructions. You will use the commands for pushing an existing repository.

  • Back in your terminal, connect your local repository to GitHub and push by running:
git remote add origin [[YOUR_REPO_URL="https://github.com/YOUR_USERNAME/detection-rules.git"]]
git branch -M main
git push -u origin main

Your terminal shows the push progress and confirms the branch is set to track origin/main.

Push rejected or authentication error?

  • If you see a permission denied error, make sure you are signed into the correct GitHub account. GitHub may prompt you to authenticate via browser.
  • If the push is rejected because the remote contains work you don't have, you may have accidentally initialized the GitHub repo with a README. Delete the repo on GitHub, recreate it without any initialization, and try again.
  • help me fix my git push to GitHub
  • Refresh the GitHub page in your browser.

You should see your README.md rendered with the structure, lifecycle states, and contribution workflow sections.

✔️ Awesome, I've got everything!

Great. Double check you have saved your README.md and that it appears on your GitHub repository page.

ⓧ I'd like to double check the full code

Your README.md should contain three sections: Structure (listing all six directories), Lifecycle States (four bullet points: experimental, test, stable, deprecated), and Contribution Workflow (eight numbered steps from branch creation through peer review). Scroll up and confirm each section is present.

Before you think about what comes next, what do you think will happen if you run sigma check --help?

  • Verify that sigma-cli is fully operational by running these commands:
sigma check --help
sigma convert --help

Both commands print usage information listing their available flags and arguments. This confirms sigma-cli is installed correctly and the Elasticsearch backend is available for conversion.

Your toolchain is ready. Next up, you will author your first Sigma detection rule and see how it converts into an Elastic EQL query your cluster can run.

Author Your First Sigma Rule and Convert to EQL

Your detection engineering toolchain is installed and your repository structure is in place. Now it is time to write your first real detection rule.

A rule that lives only in your head or directly in a SIEM UI cannot be versioned, tested, or shared across your team. Writing it as a Sigma rule and converting it to Elastic EQL proves the format works for your specific cluster.

In this step, get ready to:
  • Author a Sigma rule detecting encoded PowerShell execution with full metadata.
  • Validate syntax with sigma check and convert to Elastic EQL.
  • Create a decision document and commit on a feature branch.
Author the Sigma rule metadata

Your first detection rule targets a real-world attack technique: encoded PowerShell execution (ATT&CK T1059.001). Attackers use encoded commands to hide malicious payloads in process logs.

  • Create the subdirectory for your rule by running this command:
mkdir -p rules/windows/process_creation
  • Create a new file called proc_creation_win_encoded_powershell.yml inside rules/windows/process_creation/ and paste the following metadata section:
title: Suspicious Encoded PowerShell Command Line
id: f26c6093-6f14-4b12-800f-0571728ff8e1
status: experimental
description: >
  Detects execution of PowerShell with encoded command line arguments,
  which is commonly used by attackers to obfuscate malicious payloads.
references:
  - https://attack.mitre.org/techniques/T1059/001/
author: Detection Engineering Team
date: 2026-07-29
modified: 2026-07-29
tags:
  - attack.execution
  - attack.t1059.001

What does this metadata do?

  • id is a unique UUID that identifies this rule across every system that imports it.
  • status: experimental marks this as a new rule entering the lifecycle. It has not been tested in production yet.
  • tags map to MITRE ATT&CK technique IDs so analysts can cross-reference detections with the framework.
Add the detection logic

The metadata describes what the rule is. The detection logic describes what it looks for in log data.

  • In the same file, directly below the tags section, paste the following detection logic:
logsource:
  category: process_creation
  product: windows
detection:
  selection_img:
    Image|endswith:
      - '\powershell.exe'
      - '\pwsh.exe'
  selection_enc:
    CommandLine|contains:
      - '-enc '
      - '-EncodedCommand '
      - '-ec '
  condition: selection_img and selection_enc
falsepositives:
  - Legitimate administration scripts using encoding for special characters
  - Software deployment tools that encode parameters
level: medium

What does the detection logic do?

  • logsource tells Sigma which log category and product this rule applies to. The conversion pipeline uses this to select the right field mappings.
  • selection_img matches process creation events where the executable ends with powershell.exe or pwsh.exe.
  • selection_enc matches command lines containing encoding flags like -enc or -EncodedCommand.
  • condition: selection_img and selection_enc requires BOTH selections to match. A process must be PowerShell AND use an encoding flag.
  • falsepositives documents known benign triggers so analysts do not waste time investigating expected behaviour.
  • Save proc_creation_win_encoded_powershell.yml.

Before you run validation, take a moment to predict: do you think sigma check will pass on the first try, or will it flag something?

  • Validate the rule syntax by running this command:
sigma check rules/windows/process_creation/proc_creation_win_encoded_powershell.yml

You should see a summary line reporting 0 errors, 0 condition errors, and 0 issues.

Seeing errors in the output?

  • Check that your YAML indentation uses spaces, not tabs. YAML is strict about whitespace.
  • Make sure the condition field references the exact selection names (selection_img and selection_enc). A typo here causes a condition error.
  • Confirm you have no trailing whitespace or missing colons after field names.

help me fix my sigma check errors

✔️ Awesome, I've got everything!

Great. Double check that you have saved the file. Both the metadata section and the detection logic should be in one continuous YAML file with no blank lines between sections.

ⓧ I'd like to double check the full code

Scroll up and confirm your file contains both chunks in order: the metadata section (title through tags) followed immediately by the detection logic section (logsource through level). The complete file should have 32 lines total.

Convert to Elastic EQL

Your rule passes syntax validation. Now convert it into the query language your Elastic Security cluster actually runs.

  • Convert the rule to EQL format by running this command:
sigma convert -t elasticsearch -p ecs_windows -f eql rules/windows/process_creation/proc_creation_win_encoded_powershell.yml

The terminal prints a valid EQL query. Notice how Sigma field names have been translated to ECS fields: Image became process.executable and CommandLine became process.command_line.

Why do the field names change?

Sigma uses its own generic field names (like Image and CommandLine) so rules stay portable across different SIEMs. The ecs_windows pipeline maps those generic names to the Elastic Common Schema (ECS) fields that your Elastic cluster indexes.

This is the power of detection-as-code. You write once in Sigma, and the pipeline handles the translation to whatever backend your team uses.

Command fails with a backend or pipeline error?

  • Run sigma plugin list and confirm elasticsearch appears in the installed backends.
  • Make sure your virtual environment is still activated. If the prompt does not show the venv name, reactivate it.

help me fix my sigma convert error

Create the decision document and commit

Every detection rule needs a decision record that captures the reasoning behind it. This is how future engineers understand why the rule exists without reverse-engineering the YAML.

  • Create the decision document by pasting this into a new file at docs/decisions/001-encoded-powershell.md:
# Decision: Detect Encoded PowerShell Execution

Date: 2026-07-29
Status: experimental
Rule ID: f26c6093-6f14-4b12-800f-0571728ff8e1
ATT&CK: T1059.001 - Command and Scripting Interpreter: PowerShell

## Context

Encoded PowerShell commands are a common adversary technique to obfuscate
malicious payloads and evade basic command-line inspection. Attackers use
-EncodedCommand (or shortened aliases -enc, -ec) to pass Base64-encoded
scripts that are not human-readable in process logs.

## Decision

Detect any execution of powershell.exe or pwsh.exe where the command line
contains encoding-related flags. This targets the delivery mechanism rather
than specific payloads, giving broad coverage at the cost of potential
false positives from legitimate encoded scripts.

What does a decision document capture?

  • Context explains the threat landscape and why this technique matters to your environment.
  • Decision states the detection approach you chose and its trade-offs.
  • This document lives in Git alongside the rule, so every teammate can trace the reasoning without asking the original author.
  • Add the remaining sections below the Decision section in the same file:
## Known False Positive Sources

- SCCM/ConfigMgr task sequences that encode parameters
- Deployment tools (PDQ Deploy, Ansible WinRM) using encoding for escaping
- Internal automation scripts that encode complex parameters

## Initial Tuning Plan

1. Deploy in shadow mode for 14 days
2. Expect high volume from known automation. Plan exclusions for:
   - Service accounts matching svc-* pattern
   - Specific deployment servers by hostname
3. After exclusions, target less than 10% FP rate
4. Consider adding corroborating signal (network connection after execution)
   if FP rate remains high after initial tuning

## Alternatives Considered

- Decoding and inspecting the Base64 payload: more precise but requires
  additional processing infrastructure not available in SIEM query layer.
- Alerting only on unsigned PowerShell: would miss attacks using the
  legitimate signed PowerShell binary (which is the common case).
  • Save 001-encoded-powershell.md.
  • Confirm both new files exist by running:
ls rules/windows/process_creation/ docs/decisions/

You should see proc_creation_win_encoded_powershell.yml in the first listing and 001-encoded-powershell.md in the second.

  • Create a feature branch and commit your work by running these commands:
git checkout -b detect/t1059.001-encoded-powershell
git add rules/windows/process_creation/proc_creation_win_encoded_powershell.yml docs/decisions/001-encoded-powershell.md
git commit -m "feat(detection): add encoded PowerShell execution rule [ATT&CK T1059.001]"

Why a feature branch?

Detection-as-code follows the same workflow as application code. Each new rule lives on its own branch and enters main through a pull request. This gives teammates a chance to review the detection logic, check false positive coverage, and verify the decision document before anything goes live.

Git commit fails?

  • If Git asks you to configure your name and email first, run git config user.name and git config user.email with your details, then retry the commit.
  • Make sure you are in the repository root directory (the folder containing rules/ and docs/).

help me fix my git commit error

You now have a versioned, validated detection rule and a documented decision record on a feature branch. Next up, you will build the CI pipeline that automates this validation on every pull request.

Build the CI Pipeline with Automated Quality Gates

Your rule converts cleanly and passes syntax validation locally. But what happens when five engineers author rules every week?

Manual validation does not scale. Without automated checks, a syntactically valid but semantically broken rule can reach production undetected. In this step, you will build a GitHub Actions pipeline that catches what humans miss consistently.

In this step, get ready to:
  • Create test fixture events that prove your rule triggers (and does not trigger) correctly.
  • Write a pytest-based test script that validates rules against fixtures automatically.
  • Create a GitHub Actions workflow that lints, converts, and tests every rule on pull request.
Create the true positive test fixture

Every detection rule needs proof that it works. A true positive fixture is a synthetic event that your rule must match. You will store it alongside the rule so the test suite can verify detection logic automatically.

  • Create the test fixture directory by running this command:
mkdir -p tests/proc_creation_win_encoded_powershell
  • Create a file called true_positive.json inside that directory. Paste in the following event:
{
  "process": {
    "executable": "C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe",
    "name": "powershell.exe",
    "command_line": "powershell.exe -EncodedCommand SQBuAHYAbwBrAGUALQBXAGUAYgBSAGUAcQB1AGUAcwB0",
    "pid": 4792
  },
  "user": {
    "name": "jsmith"
  },
  "host": {
    "name": "WKS-PC042"
  },
  "event": {
    "category": ["process"],
    "type": ["start"],
    "kind": "event"
  },
  "@timestamp": "2026-07-29T14:23:01.000Z"
}

What does this fixture represent?

This is a process creation event in ECS (Elastic Common Schema) format. The command_line field contains -EncodedCommand followed by a Base64 string. That is exactly the pattern your Sigma rule targets.

Notice the fields use ECS naming (process.executable), not Sigma naming (Image). The test script will convert the Sigma rule to a Lucene query using ECS mappings, then match it against this event.

  • Save true_positive.json.
  • Confirm the file exists by running:
ls tests/proc_creation_win_encoded_powershell/

You should see true_positive.json in the output.

Create the false positive test fixture

A rule that fires on everything is useless. The false positive fixture is a normal PowerShell event that your rule must not match.

  • Create a file called false_positive.json in the same tests/proc_creation_win_encoded_powershell/ directory. Paste in this event:
{
  "process": {
    "executable": "C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\powershell.exe",
    "name": "powershell.exe",
    "command_line": "powershell.exe -ExecutionPolicy Bypass -File C:\\Scripts\\backup.ps1",
    "pid": 3201
  },
  "user": {
    "name": "svc-backup"
  },
  "host": {
    "name": "SRV-BACKUP01"
  },
  "event": {
    "category": ["process"],
    "type": ["start"],
    "kind": "event"
  },
  "@timestamp": "2026-07-29T02:00:00.000Z"
}

Why this event should not match

This event runs powershell.exe with -ExecutionPolicy Bypass -File. It uses PowerShell but does not contain any encoding flags (-enc, -EncodedCommand, -ec). A legitimate backup script running from a service account is exactly the kind of event your rule must ignore.

  • Save false_positive.json.
  • Confirm both fixtures exist by running:
ls tests/proc_creation_win_encoded_powershell/

You should see both true_positive.json and false_positive.json listed.

Only seeing one file?

Make sure you saved the second file inside tests/proc_creation_win_encoded_powershell/ and not in the repository root. Check the filename matches exactly (no extra extension like .json.txt).

help me find my missing fixture file

Write the fixture test script

The test script is the engine that ties rules to fixtures. It loads each Sigma rule, converts it to a Lucene query using the ecs_windows pipeline, then asserts it matches true positives and rejects false positives.

  • Install pytest in your virtual environment by running:
pip install pytest
  • Create the scripts/ directory and a file called test_fixtures.py inside it by running:
mkdir -p scripts
touch scripts/test_fixtures.py
  • Open scripts/test_fixtures.py and paste in the imports, constants, and discovery function:
"""Test detection rules against fixture events using pySigma."""

import json
import re
from pathlib import Path

import pytest
from sigma.collection import SigmaCollection
from sigma.backends.elasticsearch import LuceneBackend
from sigma.pipelines.elasticsearch.windows import ecs_windows


RULES_DIR = Path("rules")
TESTS_DIR = Path("tests")


def get_rule_test_pairs():
    """Discover all rules that have corresponding test fixtures."""
    pairs = []
    for rule_path in RULES_DIR.rglob("*.yml"):
        test_dir = TESTS_DIR / rule_path.stem
        if test_dir.exists():
            pairs.append((rule_path, test_dir))
    return pairs

What does this code do?

  • The imports bring in pySigma's SigmaCollection (loads rules), LuceneBackend (converts to Lucene queries), and ecs_windows (maps Sigma fields to ECS fields).
  • RULES_DIR and TESTS_DIR point to the folder structure you created.
  • get_rule_test_pairs() discovers rules that have a matching test directory, pairing them automatically. Add a new rule with fixtures and the tests pick it up without editing the script.
  • Below get_rule_test_pairs(), add the Lucene query matcher function. Paste this as the first part:
def lucene_matches_event(lucene_query: str, event: dict) -> bool:
    """
    Simplified Lucene query matcher for fixture testing.
    Checks that field:value pairs in the query match the event.
    Handles wildcards (*) at start/end of values.
    """
    matches = re.findall(r'(\S+?):\(?"?([^")\s]+)"?\)?', lucene_query)
    if not matches:
        return False

    for field, value in matches:
        field_parts = field.replace("\\", "").split(".")
        current = event
        for part in field_parts:
            if isinstance(current, dict) and part in current:
                current = current[part]
            else:
                return False
  • Still inside the for loop of lucene_matches_event, add the value comparison logic directly below the return False you just pasted:
        event_value = str(current).lower()
        check_value = value.strip('"').lower()

        if check_value.startswith("*") and check_value.endswith("*"):
            if check_value.strip("*") not in event_value:
                return False
        elif check_value.startswith("*"):
            if not event_value.endswith(check_value.lstrip("*")):
                return False
        elif check_value.endswith("*"):
            if not event_value.startswith(check_value.rstrip("*")):
                return False
        elif event_value != check_value:
            return False

    return True

What does lucene_matches_event do?

This is a simplified Lucene query matcher. It extracts field:value pairs from the converted query, navigates the nested event dictionary to find each field, and compares values with wildcard support.

A production system would use Elasticsearch's _validate/query API. For this project, the simplified matcher demonstrates the test-driven detection pattern without requiring a running cluster.

  • Below lucene_matches_event, add the true positive test function:
@pytest.mark.parametrize("rule_path,test_dir", get_rule_test_pairs())
def test_true_positive(rule_path, test_dir):
    """Rule must match the true positive fixture event."""
    tp_file = test_dir / "true_positive.json"
    if not tp_file.exists():
        pytest.skip(f"No true_positive.json for {rule_path.name}")

    rule_collection = SigmaCollection.load_ruleset([str(rule_path)])
    backend = LuceneBackend(ecs_windows())
    queries = backend.convert(rule_collection)

    with open(tp_file) as f:
        event = json.load(f)

    assert len(queries) > 0, f"No queries generated for {rule_path.name}"
    assert any(
        lucene_matches_event(q, event) for q in queries
    ), f"Rule {rule_path.name} did not match true positive fixture"
  • Finally, add the false positive test function below test_true_positive:
@pytest.mark.parametrize("rule_path,test_dir", get_rule_test_pairs())
def test_false_positive(rule_path, test_dir):
    """Rule must NOT match the false positive fixture event."""
    fp_file = test_dir / "false_positive.json"
    if not fp_file.exists():
        pytest.skip(f"No false_positive.json for {rule_path.name}")

    rule_collection = SigmaCollection.load_ruleset([str(rule_path)])
    backend = LuceneBackend(ecs_windows())
    queries = backend.convert(rule_collection)

    with open(fp_file) as f:
        event = json.load(f)

    assert len(queries) > 0, f"No queries generated for {rule_path.name}"
    assert not any(
        lucene_matches_event(q, event) for q in queries
    ), f"Rule {rule_path.name} incorrectly matched false positive fixture"

What do these test functions do?

  • Both tests are parametrized. pytest automatically discovers every rule that has a matching test directory and runs both tests against it.
  • test_true_positive asserts the converted query matches the true positive event. If it does not, the rule is broken.
  • test_false_positive asserts the converted query does not match the false positive event. If it does, the rule is too broad.
  • Save scripts/test_fixtures.py.
  • Run the fixture tests locally by running:
pytest scripts/test_fixtures.py -v

You should see two tests pass: test_true_positive and test_false_positive, both marked PASSED in the output.

Seeing import errors or test failures?

  • If you see ModuleNotFoundError, confirm your virtual environment is activated and that you installed sigma-cli, pySigma-backend-elasticsearch, and pytest in it.
  • If test_true_positive fails, check that your true_positive.json contains -EncodedCommand in the command_line field.
  • If both tests are skipped, the test directory name must match the rule filename stem exactly: proc_creation_win_encoded_powershell.

help me debug my failing fixture tests

✔️ Awesome, I've got everything!

Great. Make sure scripts/test_fixtures.py is saved.

ⓧ I'd like to double check the full code

Your scripts/test_fixtures.py should contain these sections in order. Scroll up and confirm each is present:

  • The docstring, imports (json, re, Path, pytest, SigmaCollection, LuceneBackend, ecs_windows), and RULES_DIR/TESTS_DIR constants.
  • The get_rule_test_pairs() function that discovers rules with matching test directories.
  • The lucene_matches_event() function with regex extraction, field traversal, and wildcard comparison.
  • The test_true_positive parametrized test function.
  • The test_false_positive parametrized test function.
Create the GitHub Actions workflow

Now you will automate these checks so they run on every pull request. The workflow has three jobs: lint (validate syntax), convert (produce EQL output), and test (run fixture tests). If any job fails, the PR cannot merge.

  • Create the workflow directory by running:
mkdir -p .github/workflows
  • Create a file called detection-pipeline.yml inside .github/workflows/. Paste in the workflow name, trigger, and lint job:
name: Detection Rule Pipeline

on:
  pull_request:
    paths:
      - 'rules/**'
      - 'tests/**'
      - 'scripts/**'

jobs:
  lint:
    name: Validate Sigma Rules
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Install sigma-cli
        run: pip install sigma-cli pySigma-backend-elasticsearch

      - name: Install Elasticsearch backend
        run: sigma plugin install elasticsearch

      - name: Validate all rules
        run: |
          find rules/ -name "*.yml" -exec sigma check {} +

What does this do?

  • The on.pull_request.paths filter means this workflow only runs when a PR touches files in rules/, tests/, or scripts/.
  • The lint job installs sigma-cli and runs sigma check against every .yml file in rules/. If any rule has invalid syntax or missing required fields, the job fails.
  • Below the lint job (still inside the jobs: block), add the convert job:
  convert:
    name: Convert to Elastic EQL
    runs-on: ubuntu-latest
    needs: lint
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Install sigma-cli
        run: pip install sigma-cli pySigma-backend-elasticsearch

      - name: Install Elasticsearch backend
        run: sigma plugin install elasticsearch

      - name: Convert rules to EQL
        run: |
          mkdir -p output/eql
          for rule in $(find rules/ -name "*.yml"); do
            rulename=$(basename "$rule" .yml)
            sigma convert -t elasticsearch -p ecs_windows -f eql "$rule" > "output/eql/${rulename}.eql"
            echo "--- Converted: $rulename ---"
            cat "output/eql/${rulename}.eql"
          done

      - name: Upload EQL artifacts
        uses: actions/upload-artifact@v4
        with:
          name: eql-output
          path: output/eql/

What does the convert job do?

  • needs: lint means this job only runs after lint passes.
  • It loops through every rule, converts it to EQL using the ecs_windows pipeline, and saves the output. The converted EQL is uploaded as a build artifact you can download from the PR.
  • Below the upload step (still inside the convert job's steps:), add the golden file drift check. Then add the test job as a sibling of convert:
      - name: Check golden file drift
        run: |
          if [ -d "golden/" ]; then
            for golden in golden/*.eql; do
              rulename=$(basename "$golden")
              if [ -f "output/eql/$rulename" ]; then
                if ! diff -q "$golden" "output/eql/$rulename" > /dev/null 2>&1; then
                  echo "::warning::DRIFT DETECTED in $rulename - conversion output differs from golden file"
                  diff "$golden" "output/eql/$rulename" || true
                fi
              fi
            done
          fi

  test:
    name: Fixture Tests
    runs-on: ubuntu-latest
    needs: lint
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'

      - name: Install dependencies
        run: pip install sigma-cli pySigma-backend-elasticsearch pytest

      - name: Install Elasticsearch backend
        run: sigma plugin install elasticsearch

      - name: Run fixture tests
        run: pytest scripts/test_fixtures.py -v

What do these additions do?

  • The golden file drift check compares the current EQL output against a stored "golden" version. If the output changed (from a backend update or pipeline change), it flags a warning. You will create the golden file in the next step.
  • The test job runs pytest scripts/test_fixtures.py -v in CI. This is the same command you just ran locally. If fixtures do not pass, the PR is blocked.
  • Save .github/workflows/detection-pipeline.yml.
  • Verify the file is in the right location by running:
ls .github/workflows/

You should see detection-pipeline.yml listed.

Directory not found?

The .github folder starts with a dot, making it hidden by default. Make sure you ran mkdir -p .github/workflows from the repository root (the same directory that contains rules/ and tests/).

help me fix my workflow file location

✔️ Awesome, I've got everything!

Great. Make sure the file is saved.

ⓧ I'd like to double check the full code

Your .github/workflows/detection-pipeline.yml should contain these sections in order. Scroll up and confirm each is present:

  • The name: and on: pull_request: trigger with paths rules/**, tests/**, scripts/**.
  • The lint job with sigma check on all .yml rules.
  • The convert job (needs lint) with EQL conversion, artifact upload, and golden file drift check.
  • The test job (needs lint) with pytest scripts/test_fixtures.py -v.
Push and verify the pipeline runs

Everything is ready locally. Time to push your feature branch and see the pipeline run against a real pull request.

  • Stage and commit all the new files by running:
git add tests/ scripts/ .github/
git commit -m "feat(ci): add fixture tests and detection pipeline workflow"
  • Push your feature branch to GitHub by running:
git push -u origin HEAD
  • Open your repository on GitHub in your browser.
  • Click the Compare & pull request button that appears for your recently pushed branch.
  • Click Create pull request.

Before you check the Actions tab, what do you expect to happen? Will all three jobs pass on the first run?

  • Click the Actions tab in your repository to watch the workflow run.

You should see the "Detection Rule Pipeline" workflow running with green checkmarks appearing for the lint, convert, and test jobs. The convert job also uploads an eql-output artifact you can download to see the converted EQL.

Workflow not appearing?

  • The workflow file must be at exactly .github/workflows/detection-pipeline.yml in the repository root. If you accidentally nested it deeper, the Actions tab will not pick it up.
  • If the workflow appears but a job fails, click the failed job to see logs. Common causes: a typo in the YAML indentation, or the paths filter not matching the files you changed.

help me debug my GitHub Actions workflow

Your detection rules now have automated quality gates. Every future rule goes through the same gauntlet: syntax validation, EQL conversion, and fixture testing. Next up, you will discover what happens when a schema change silently invalidates a passing rule.

Catch Schema Evolution That Silently Kills Detection

Your CI pipeline is green. Your rule validates cleanly and converts to valid EQL. Everything looks production-ready.

But what happens when a parser update silently renames a field in your index? The rule still compiles. The conversion still succeeds. The detection, however, matches nothing. This is the most dangerous failure mode in detection engineering: silent breakage with no error.

In this step, get ready to:
  • Simulate a schema evolution event and observe the silent breakage it causes.
  • Prove that fixture testing catches what syntax validation cannot.
  • Add golden-file drift detection to your CI pipeline.
Simulate a schema evolution event

In the real world, Elastic Agent parser updates sometimes rename fields between versions. Your rule still converts to valid syntax, but the output query references a field that no longer exists in your index.

  • Create a pipelines/ directory in your project root by running this command:
mkdir pipelines
  • Create a custom pipeline file that simulates a parser version change by remapping CommandLine to process.args instead of process.command_line. Create pipelines/ecs_windows_v2.yml with this content:
name: ECS Windows v2 (simulated parser change)
priority: 20
transformations:
  - id: image_field_mapping
    type: field_name_mapping
    mapping:
      Image: process.executable
  - id: commandline_field_mapping
    type: field_name_mapping
    mapping:
      CommandLine: process.args

What does this pipeline do?

The standard ecs_windows pipeline maps the Sigma field CommandLine to the ECS field process.command_line. This custom pipeline maps it to process.args instead, simulating what happens when an Elastic Agent parser update renames a field.

The field_name_mapping transformation type tells pySigma to replace Sigma rule field names with target-specific field names during conversion. The priority value controls processing order when multiple pipelines are chained.

  • Now convert your rule using the new pipeline by running:
sigma convert -t elasticsearch -p pipelines/ecs_windows_v2.yml -f eql rules/windows/process_creation/proc_creation_win_encoded_powershell.yml

Before you look at the output, think about what you expect to see. Will this command succeed or fail?

The command succeeds. The output is a valid-looking EQL query. But look closely at the field names. You should see process.args where you previously saw process.command_line. The query compiles, but it references a field that does not exist in your current index.

Why is this so dangerous?

Every validation layer passes. sigma check validates the rule's Sigma syntax, not the converted output. The conversion produces syntactically valid EQL. No tool raises an alarm. But when Elastic executes this query against your index, it matches zero events because the field process.args does not exist in your data.

This is silent breakage. Your detection goes blind with no error, no alert, and no indication that anything changed.

Prove that fixture testing catches silent breakage

You just saw that syntax validation and conversion both pass. Now run the fixture test to see if it catches the problem.

  • Before you run this, predict: will your fixture test pass or fail against the new pipeline's output? Why?
  • Run the fixture test by executing:
pytest scripts/test_fixtures.py -v

The test fails. Your test_true_positive test reports that the rule did not match the true positive fixture. This is the designed result.

Why did the test fail?

Your true_positive.json fixture contains a process.command_line field. But the test script uses the ecs_windows pipeline (the standard one), so the Lucene query references process.command_line and the test still passes against the current pipeline.

The key insight is this: if your production index were updated to use process.args (as the new pipeline simulates), you would need to update both the pipeline AND your fixtures to reflect reality. A fixture test that stays pinned to actual index field names catches the moment when your conversion drifts away from your data.

What did this prove?

Syntax validation passes. Conversion succeeds. But fixture testing catches the breakage because it tests the query against real event structures. This is why every rule needs fixture tests. They are the last line of defence against silent schema drift.

  • Create a decision record documenting this finding. Create docs/decisions/002-schema-drift-detection.md and include: the problem (schema changes break detections silently), what you proved (fixture tests catch what validation cannot), and the fix (golden-file drift detection, which you will add next).
  • Save the file.
Add golden-file drift detection to your CI pipeline

Fixture tests catch breakage locally, but you need an automated way to detect when a conversion output changes unexpectedly. A golden file stores the last known-good EQL output. If the output ever differs, CI flags it for human review.

  • First, generate and save the known-good EQL output. Create the golden/ directory and save the output by running:
mkdir golden
sigma convert -t elasticsearch -p ecs_windows -f eql rules/windows/process_creation/proc_creation_win_encoded_powershell.yml > golden/proc_creation_win_encoded_powershell.eql
  • Verify the golden file was created by running:
cat golden/proc_creation_win_encoded_powershell.eql

You should see EQL output referencing process.executable and process.command_line. This is your baseline.

What is a golden file?

A golden file is the same pattern as snapshot testing in software engineering. It stores the expected output of a process. If the output ever changes, the diff is surfaced for review rather than silently deployed.

In detection engineering, the golden file stores the last known-good EQL conversion. If a pipeline update, backend upgrade, or field mapping change alters the output, the CI diff tells you immediately.

  • Now add the drift detection step to your CI workflow. Open .github/workflows/detection-pipeline.yml and find the convert job. Locate the step named Upload EQL artifacts. Add this new step directly after it:
      - name: Check golden file drift
        run: |
          if [ -d "golden/" ]; then
            for golden in golden/*.eql; do
              rulename=$(basename "$golden")
              if [ -f "output/eql/$rulename" ]; then
                if ! diff -q "$golden" "output/eql/$rulename" > /dev/null 2>&1; then
                  echo "::warning::DRIFT DETECTED in $rulename - conversion output differs from golden file"
                  diff "$golden" "output/eql/$rulename" || true
                fi
              fi
            done
          fi

What does this step do?

  • It checks whether a golden/ directory exists in the repo.
  • For each .eql golden file, it compares the freshly converted output (in output/eql/) against the stored baseline.
  • If the files differ, it emits a ::warning:: annotation on the pull request. This flags the change for human review rather than blocking the build.
  • The || true at the end ensures the step does not fail the workflow. Drift is a signal, not an error.
  • Save .github/workflows/detection-pipeline.yml.

YAML indentation wrong?

The new step must be indented at the same level as the other - name: entries under the steps: key of the convert job. If your workflow fails to parse, check that each - name: line starts with exactly 6 spaces.

help me fix YAML indentation in my GitHub Actions workflow

  • Commit all your changes (the golden file, the pipeline file, the decision record, and the updated workflow) by running:
git add golden/ pipelines/ docs/decisions/002-schema-drift-detection.md .github/workflows/detection-pipeline.yml
git commit -m "feat(ci): add golden file drift detection for schema evolution"

Now verify the full picture. You have proven that schema evolution breaks detection silently, that fixture testing catches what validation misses, and that golden-file drift detection automates the early warning in CI.

  • Confirm your golden file contains the correct baseline by running:
cat golden/proc_creation_win_encoded_powershell.eql

You should see EQL output that references process.command_line (not process.args). This is the known-good output that CI will compare against on every future pull request.

✔️ Awesome, I've got everything!

Great. Double check you have saved all your files and committed your changes.

ⓧ I'd like to double check the full code

Your golden/proc_creation_win_encoded_powershell.eql file should contain:

process where process.executable : ("*\\powershell.exe", "*\\pwsh.exe") and process.command_line : ("*-enc *", "*-EncodedCommand *", "*-ec *")

Your workflow's drift detection step (in the convert job, after the upload artifact step) should contain:

      - name: Check golden file drift
        run: |
          if [ -d "golden/" ]; then
            for golden in golden/*.eql; do
              rulename=$(basename "$golden")
              if [ -f "output/eql/$rulename" ]; then
                if ! diff -q "$golden" "output/eql/$rulename" > /dev/null 2>&1; then
                  echo "::warning::DRIFT DETECTED in $rulename - conversion output differs from golden file"
                  diff "$golden" "output/eql/$rulename" || true
                fi
              fi
            done
          fi

You have experienced the most dangerous failure mode in detection engineering firsthand. Next up, you will build the workflow that takes a rule from shadow mode observation to production alerting, with data-driven tuning criteria at every stage.

Implement Shadow Mode and Threshold Tuning Workflow

Your CI pipeline catches syntax and conversion errors automatically. But a rule that passes CI is not necessarily production-ready.

Without observing how a rule behaves against real traffic, you cannot know its false-positive rate in your specific environment. Shadow mode lets you observe matches without generating analyst alerts. Data-driven tuning replaces guesswork with evidence.

In this step, get ready to:
  • Create lifecycle documentation defining promotion criteria for each state transition.
  • Create a tuning template for shadow-mode baseline analysis.
  • Promote your rule from experimental to test status with a documented state transition.
Create the lifecycle documentation

Every detection rule in your repository moves through a defined set of states. Without explicit promotion criteria, rules either languish in limbo or get pushed to production without evidence they work.

  • Create a new file called docs/lifecycle.md by running:
touch docs/lifecycle.md
  • Open docs/lifecycle.md in VS Code and paste the following content defining the four lifecycle states:
# Detection Rule Lifecycle

## States

### experimental
- Rule is newly authored and untested in production data.
- CI must pass (lint + convert + fixture tests).
- Peer review required before promotion.

### test (shadow mode)
- Rule is deployed in non-alerting mode.
- Matches are logged but do not page analysts.
- Minimum observation period: 7 days.
- Promotion criteria: false-positive rate below 10% based on sample review.

### stable (production)
- Rule generates analyst-facing alerts.
- Changes require PR with passing CI and peer review.
- Quarterly revalidation required.

### deprecated
- Rule is archived, not deleted.
- Commit message must state retirement reason.
- Tombstone tag added. Rule file remains in repository history.

What do these states mean?

Each state represents a confidence level. Experimental means untested against real data. Test means actively observing in shadow mode. Stable means alerting analysts. Deprecated means archived (never deleted, so you keep Git history).

  • Below the deprecated section, add the promotion criteria checklists:
## Promotion Criteria

### experimental to test
- [ ] CI pipeline passes (lint, convert, fixture tests)
- [ ] Peer review approved
- [ ] Decision document created in docs/decisions/
- [ ] Golden file committed

### test to stable
- [ ] Shadow mode active for 7+ days
- [ ] Tuning template completed with baseline data
- [ ] False-positive rate below 10%
- [ ] Top noisy entities documented and exclusions applied
- [ ] Reviewer sign-off on tuning record

### stable to deprecated
- [ ] Retirement reason documented (obsolete technique, superseded, unacceptable FP rate)
- [ ] Replacement rule referenced if applicable
- [ ] Archival commit with tombstone tag
  • Confirm the file is taking shape by running:
cat docs/lifecycle.md | wc -l

You should see a line count around 40, confirming both sections are in place.

  • Add the baseline analysis workflow at the bottom of the file:
## Baseline Analysis Workflow

1. Deploy rule in non-alerting mode (shadow/test state)
2. Collect match volume per day for minimum 7 days (30 days recommended)
3. Identify top 5 entities (users, hosts, processes) by match count
4. Sample 20 matches and categorize: true positive, benign true positive, false positive
5. Compute: TP rate = TPs / (TPs + FPs)
6. Apply tuning based on FP categories:
   - Known tool activity: suppress by process name, review every 90 days
   - Authorized admin: suppress by username for that action, quarterly review
   - Threshold too low: add count or time window requirement
   - Logic too broad: add corroborating conditions
7. Re-run sample after tuning to verify improvement
8. Document in tuning template and request promotion review

Why 7 days minimum?

Seven days captures at least one full business week of activity patterns. Weekend vs. weekday traffic often looks very different. The recommended 30-day period catches monthly cycles (patch Tuesdays, month-end batch jobs) but 7 days is the minimum before making a promotion decision.

  • Save docs/lifecycle.md.
  • Verify the complete file by running:
cat docs/lifecycle.md

You should see all three sections: States, Promotion Criteria, and Baseline Analysis Workflow with 8 numbered steps.

File looks incomplete?

Make sure you appended each section below the previous one, not overwriting. Check that the Baseline Analysis Workflow section appears after the Promotion Criteria section.

Why is my lifecycle.md missing sections after I saved?

Create the tuning template

During shadow mode, detection engineers need a structured way to record what they observe. The tuning template standardizes this process so that promotion decisions are backed by data, not gut feeling.

  • Create a new file called docs/tuning-template.md and paste the header and baseline metrics section:
# Tuning Record: [Rule Title]

Rule ID: [UUID]
Rule file: [path]
Analyst: [name]
Shadow mode start: [date]
Shadow mode end: [date]

## Baseline Metrics

- Total matches observed: [count]
- Observation period: [days]
- Average daily matches: [count/day]

## Top Entities by Match Volume

| Rank | Entity Type | Entity Value | Match Count | Classification |
|------|-------------|--------------|-------------|----------------|
| 1    |             |              |             |                |
| 2    |             |              |             |                |
| 3    |             |              |             |                |
| 4    |             |              |             |                |
| 5    |             |              |             |                |

What goes in the Top Entities table?

During shadow mode, you identify which users, hosts, or processes trigger the rule most often. The top 5 entities by match count reveal whether your rule is catching real attackers or just noisy automation. Classifying each entity (true positive, benign, false positive) drives your tuning decisions.

  • Below the Top Entities table, add the false positive analysis and tuning sections:
## False Positive Analysis

Sample size: [n] matches reviewed

| Category | Count | Percentage | Resolution |
|----------|-------|------------|------------|
| Known tool activity |  |  |  |
| Authorized admin |  |  |  |
| Threshold too low |  |  |  |
| Logic too broad |  |  |  |
| True positive |  |  |  |

## Applied Tuning Changes

| Change | Rationale | Risk of blind spot |
|--------|-----------|-------------------|
|        |           |                   |
  • Confirm both documentation files exist by running:
ls docs/

You should see decisions/, lifecycle.md, and tuning-template.md listed.

  • Finally, add the metrics and promotion decision section at the bottom of docs/tuning-template.md:
## Metrics After Tuning

- TP rate before: [%]
- TP rate after: [%]
- Daily match volume before: [count]
- Daily match volume after: [count]

## Promotion Decision

- [ ] Promote to stable
- [ ] Extend shadow mode (reason: )
- [ ] Retire rule (reason: )

Reviewer: [name]
Date: [date]

Why track the risk of blind spot?

Every exclusion you add during tuning narrows what the rule can see. The "Risk of blind spot" column in the Applied Tuning Changes table forces you to document what you might miss. This prevents over-tuning where a rule is so heavily suppressed it stops catching real attacks.

  • Save docs/tuning-template.md.
  • Verify the template is complete by running:
cat docs/tuning-template.md

You should see six sections: header fields, Baseline Metrics, Top Entities table, False Positive Analysis table, Applied Tuning Changes table, Metrics After Tuning, and the Promotion Decision checklist.

Missing tables or garbled formatting?

Markdown tables require the header separator row (the line with dashes and pipes). Make sure each table has the |------|---| separator row immediately below the column headers.

Help me fix the formatting in my tuning-template.md

Promote your rule to shadow mode

Your lifecycle documentation defines the criteria. Now walk through the first state transition: promoting your encoded PowerShell rule from experimental to test (shadow mode).

  • Open rules/windows/process_creation/proc_creation_win_encoded_powershell.yml in VS Code.
  • Find this line:
status: experimental
  • Change experimental to test, so the line now reads:
status: test
  • Scroll to the very bottom of the file (after the level: medium line) and add a custom field recording when shadow mode started:
shadow_start: 2026-07-29

Why add shadow_start as a custom field?

Sigma allows custom fields that are not part of the official schema. The shadow_start field records exactly when the rule entered shadow mode. Your promotion criteria require 7+ days in shadow mode before advancing to stable. This field makes that verifiable from the rule file itself, without digging through Git history.

  • Save the rule file.
  • Before you re-run the conversion, predict: will the EQL output change because you updated the status field?
  • Re-run the sigma conversion to verify your rule still converts cleanly:
sigma convert -t elasticsearch -p ecs_windows -f eql rules/windows/process_creation/proc_creation_win_encoded_powershell.yml

The EQL output is identical to before. The status and shadow_start fields are metadata. They do not affect the detection logic or the converted query.

  • Run the fixture tests to confirm nothing broke:
pytest scripts/test_fixtures.py -v

Both tests should pass. The rule's detection logic has not changed, only its lifecycle metadata.

  • Stage all new and modified files and commit with a descriptive message:
git add docs/lifecycle.md docs/tuning-template.md rules/windows/process_creation/proc_creation_win_encoded_powershell.yml
git commit -m "lifecycle(test): promote encoded PowerShell rule to shadow mode

- Add docs/lifecycle.md with state definitions and promotion criteria
- Add docs/tuning-template.md for baseline analysis records
- Update rule status from experimental to test
- Add shadow_start: 2026-07-29"
  • Push your feature branch and create a pull request:
git push origin HEAD
  • Open your repository on GitHub.
  • Click Compare & pull request on the banner that appears for your recently pushed branch.
  • Title the PR: Promote encoded PowerShell rule to shadow mode.
  • In the description, note that this PR demonstrates the full workflow: rule authoring, CI validation, fixture testing, and lifecycle state transition.
  • Click Create pull request.

What do you think will happen when the CI pipeline runs on this PR? Will it pass or fail?

The Actions workflow triggers automatically. Watch the Checks section on your PR page. All three jobs (lint, convert, test) should show green checkmarks. The convert job's drift detection step compares your golden file against the fresh conversion output. Since the EQL did not change, no drift warning appears.

CI failing on your PR?

If the lint job fails, make sure your rule file is valid YAML. A common mistake is incorrect indentation on the shadow_start field. It should be at the top level (same indentation as level), not nested under another key.

If the test job fails, check that you did not accidentally modify the detection section when editing the status. The fixture tests validate the detection logic, not the metadata.

Help me debug my failing CI pipeline on this PR

✔️ Awesome, I've got everything!

Great. Double check you have saved both docs/lifecycle.md and docs/tuning-template.md, and that your rule file shows status: test with shadow_start: 2026-07-29 at the bottom.

ⓧ I'd like to double check the full code

Here is the complete docs/lifecycle.md. Scroll up and confirm your file matches the three sections: States, Promotion Criteria, and Baseline Analysis Workflow (in that order).

Confirm your rule file ends with these lines:

falsepositives:
  - Legitimate administration scripts using encoding for special characters
  - Software deployment tools that encode parameters
level: medium
shadow_start: 2026-07-29

And the status line near the top reads:

status: test

Your detection rule now has a fully documented lifecycle. You have explicit criteria for every state transition, a structured template for recording shadow-mode observations, and a live PR demonstrating the promotion workflow with passing CI. Every future rule on your team follows this same path: author, validate, observe, tune, promote.

Secret mission

Automate the Lifecycle with a Claude Code Slash Command

You have built every component of a detection rule lifecycle by hand. What if a single command could orchestrate the entire workflow? In this mission, you will create a Claude Code slash command that accepts an ATT&CK technique, generates a complete Sigma rule with fixtures, runs validation, and prepares the commit. You will also add tuning and promotion paths so your team can manage rules without remembering every step.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally with a free GitHub repository, so there are no ongoing costs.

Resources you used:

  • Local project folder on your Desktop (contains your virtual environment, sigma-cli packages, rule files, and all documentation).
  • GitHub repository (free tier, no ongoing charges).
  • GitHub Actions minutes consumed on pull request pushes (free tier: 2,000 minutes/month for private repos, unlimited for public repos).

Keep everything running

No action needed. Choose this if you plan to continue using this repository as your team's detection rule library.

  • Your repository is ready for new rules. Use the /detect slash command in Claude Code to keep authoring detection rules.
  • Your CI pipeline triggers automatically on every pull request that touches rules/ or tests/.
  • Your virtual environment stays active until you close the terminal or run deactivate.

Pause - I'll come back to this later

Free up terminal state while keeping all files intact. The repository sits idle with no cost.

  • Deactivate your Python virtual environment by running:
deactivate
  • When you return, navigate back to your project folder and reactivate the environment by running:
cd ~/Desktop/[[PROJECT_FOLDER="your project folder name"]]
source .venv/bin/activate
  • Your GitHub repository, CI pipeline, rules, and documentation all remain exactly as you left them.

Delete - I don't want to use this again

Remove all project resources from your machine and GitHub.

  • Deactivate your virtual environment first by running:
deactivate
  • Delete the local project folder by running:
cd ~/Desktop
rm -rf [[PROJECT_FOLDER="your project folder name"]]
  • Confirm the folder is gone by running:
ls
  • Your project folder should no longer appear in the output.
  • To delete the GitHub repository, open the repository page on GitHub.
  • Click Settings in the top navigation bar.
  • Scroll to the bottom of the General page and find the Danger Zone section.
  • Click Delete this repository.
  • Click I want to delete this repository, then click I have read and understand these effects.
  • Type the full repository name into the confirmation field and click Delete this repository to confirm.

Nice Work!

Nice Work!

You just built a production-grade detection engineering infrastructure that manages the full lifecycle of SIEM detection rules as code. Every rule you write from now on is versioned, tested, and promoted through a documented pipeline.

Here's what you learned:

  • Built a Git-managed detection rule repository with enforced structure, authored a Sigma rule with full metadata, and converted it to Elastic EQL targeting your cluster's ECS field mappings.
  • Created a GitHub Actions CI pipeline that lints, converts, and runs fixture tests on every pull request. Then experienced the silent schema breakage problem firsthand and solved it with golden-file drift detection.
  • Documented a complete shadow-mode-to-production promotion workflow with explicit criteria, a tuning template driven by observed data, and decision records that capture the rationale behind every change.
  • Secret Mission: Automated the entire detection engineering workflow into a Claude Code slash command that handles new rule authoring, tuning, and lifecycle promotion in a single repeatable prompt.

Ready to quiz yourself?