Detection Rule Validation Pipeline

Build a CI pipeline that validates Sigma detection rules on every push.

Introduction

30 Second Summary

Security teams write detection rules to spot threats in their environment. When a single typo in a rule silently breaks detection coverage, attackers slip through unnoticed.

In this project, you will build an automated validation pipeline that catches broken detection rules before they ever reach production. You will wire up local linting tools and a GitHub Actions workflow that validates every rule on every push.

What You'll Build

You will push a detection rule to GitHub and watch your CI pipeline light up green or red within seconds, telling you exactly whether the rule is valid or broken and pinpointing the exact line and field that caused the failure.

By the end of this project, you'll have:

  • A detection rules repository where pushing a Sigma-format rule triggers automated CI/CD validation of its YAML syntax and schema compliance.
  • The ability to read a CI failure log and identify the exact file, line number, and error message for both syntax errors and schema violations.
  • A local validation workflow that catches the same errors at your terminal before you push, so you never wait for CI to tell you something is broken.
  • Secret Mission: Add a pre-commit hook that runs validation automatically before every commit, making it impossible to push a broken rule.

Are there any prerequisites?

You will need a GitHub account, Git installed on your machine, and VS Code as your code editor. Step 1 walks you through installing everything else you need, including Python 3 and the validation tools.

Before We Start

Before We Start...

Before we dive in, let's lock in what you're about to build and why it matters.

Detection engineers write rules that tell security tools what to look for. When those rules break, threats slip through unnoticed. This project gives you a pipeline that catches broken rules automatically, before they ever reach production.

Set Up Local Validation Tools

Without local tooling, every time you write a detection rule you would push it to GitHub and wait for CI to tell you whether it is broken. That feedback loop is slow and frustrating.

In this step you will install two CLI validators on your machine: yamllint for YAML syntax checking and check-jsonschema for schema compliance. You will also create the repository skeleton that holds your detection rules.

In this step, get ready to:
  • Install Python 3, pipx, yamllint, and check-jsonschema on your machine.
  • Initialize a Git repository with the detection rules directory structure.
  • Download the official Sigma JSON Schema for rule validation.
Install Python 3 and pipx via Homebrew

Both yamllint and check-jsonschema are Python-based CLI tools. They require Python 3.10 or higher to run.

  • Press Cmd+Space to open Spotlight, type Terminal, and press Enter to open it.
  • Check whether Python 3 is already installed by running this command:
python3 --version

✔️ I see version 3.10 or higher

Python is ready. Skip ahead to the pipx install below.

ⓧ I see a version lower than 3.10

Your system has an older Python that does not meet the 3.10+ requirement. Install the latest Python via Homebrew by running:

brew install python3

Homebrew may show a dependency summary and ask you to confirm. Press y to proceed.

After installation, run python3 --version again. You should now see 3.10 or higher.

ⓧ Command not found

Python 3 is not installed yet. Install it via Homebrew by running:

brew install python3

Homebrew may show a dependency summary and ask you to confirm. Press y to proceed.

After installation, run python3 --version again. You should see a version number of 3.10 or higher.

  • Install pipx by running:
brew install pipx

What is pipx?

pipx installs Python CLI tools into isolated environments so they do not conflict with each other or your system packages. It is the recommended way to install standalone command-line tools like yamllint and check-jsonschema on macOS.

  • Add the pipx binary directory to your shell PATH by running:
pipx ensurepath
  • Close your terminal window completely, then open a new terminal so the PATH change takes effect.

Why close and reopen the terminal?

The pipx ensurepath command modifies your shell configuration file. Your current session still uses the old PATH. Opening a fresh terminal loads the updated configuration.

  • Install yamllint as an isolated CLI tool by running:
pipx install yamllint
  • Install check-jsonschema by running:
pipx install check-jsonschema

Both commands print a success message showing the installed version and the path where the binary was placed.

  • Confirm both tools are accessible by running:
yamllint --version
check-jsonschema --version

You should see a version number printed for each command. As long as both produce output without errors, your tools are ready.

Command not found after installing?

Make sure you opened a brand-new terminal window after running pipx ensurepath. The PATH update only applies to new sessions.

If you still see an error, run pipx ensurepath again, close the terminal, and open a fresh one.

Help me troubleshoot why yamllint or check-jsonschema is not found after pipx install.

Initialize the detection rules repository

Your detection rules need a home. You will create a Git repository with the directory structure that the CI pipeline expects later.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir detection-rules-pipeline
cd detection-rules-pipeline
  • Initialize a Git repository and create the required directory structure by running:
git init
mkdir -p rules schemas .github/workflows
  • Confirm the directories were created by running:
ls -la

You should see rules, schemas, and .github listed in the output alongside the .git directory.

  • Create placeholder files so Git can track the empty directories, then commit the skeleton:
touch rules/.gitkeep schemas/.gitkeep .github/workflows/.gitkeep
git add .
git commit -m "Initial skeleton: directory structure"

Git confirms the commit with a short summary of the files added.

Seeing an error about user identity?

If Git asks you to set your name and email before committing, run these two commands with your own details, then retry the commit:

git config user.name "Your Name"
git config user.email "your@email.com"

Help me configure Git so I can make my first commit.

Download the Sigma JSON Schema

The check-jsonschema tool validates files against a JSON Schema. The Sigma project publishes an official schema that defines what a valid detection rule looks like.

  • Download the official Sigma detection rule schema into your schemas/ directory by running:
curl -o schemas/sigma-detection-rule-schema.json https://raw.githubusercontent.com/SigmaHQ/sigma-specification/refs/heads/main/json-schema/sigma-detection-rule-schema.json

You should see a progress indicator followed by a download complete message from curl.

What does this schema define?

The Sigma JSON Schema enforces the structure every detection rule must follow. It requires three top-level fields: title, logsource, and detection. The detection object itself must contain a condition field.

Any rule missing these required fields will fail validation with a clear error message pointing to the exact missing property.

  • Before you run the next command, what do you expect to see if the download succeeded? What about if the file was corrupted?
  • Verify the downloaded file is valid JSON by running:
python3 -c "import json; json.load(open('schemas/sigma-detection-rule-schema.json')); print('Valid JSON')"

You should see Valid JSON printed. If the file were corrupted or truncated, Python would raise a JSONDecodeError instead.

Seeing a JSONDecodeError?

The download was incomplete or corrupted. Delete the file and re-download it:

rm schemas/sigma-detection-rule-schema.json
curl -o schemas/sigma-detection-rule-schema.json https://raw.githubusercontent.com/SigmaHQ/sigma-specification/refs/heads/main/json-schema/sigma-detection-rule-schema.json

Help me fix a corrupted Sigma JSON Schema download.

  • Open the project in VS Code by running:
code .

VS Code opens with the detection-rules-pipeline folder loaded in the Explorer sidebar. You can see rules/, schemas/, and .github/ in the file tree.

Your local validation toolkit is ready. Next up, you will write your first Sigma detection rule and see both validators confirm it passes cleanly.

Write and Validate Your First Detection Rule

Your local tools are installed and the repository skeleton is ready. Now you need something to validate.

Before you can trust your pipeline to catch errors, you need to know what "green" looks like. In this step you will write a valid Sigma detection rule from scratch, then run both validators against it to establish a clean baseline.

In this step, get ready to:
  • Write a Sigma detection rule that detects whoami.exe execution on Windows.
  • Configure and run yamllint to validate YAML syntax.
  • Run check-jsonschema to validate the rule against the Sigma JSON Schema.
Write a Sigma detection rule

A Sigma rule is a structured YAML file that describes suspicious activity in log data. Every valid Sigma rule must include three required fields: title, logsource, and detection (with a condition inside it).

  • In VS Code, right-click the rules folder in the **Explorer** sidebar.
  • Select **New File** and name it whoami_execution.yml.
  • Paste the following detection rule into the file:
title: Whoami Command Execution
id: e34a5272-f6e4-4117-9e5b-4b1f4a5e0e4c
status: test
description: Detects execution of whoami.exe which may indicate reconnaissance activity
author: Detection Engineering Team
date: 2026-07-29
references:
    - https://attack.mitre.org/techniques/T1033/
logsource:
    category: process_creation
    product: windows
detection:
    selection:
        Image|endswith: '\whoami.exe'
    condition: selection
falsepositives:
    - Legitimate administrator activity
    - Inventory scripts
level: low
tags:
    - attack.discovery
    - attack.t1033

What does this rule detect?

This rule watches for process creation events where the executable path ends with \whoami.exe. Attackers commonly run whoami early in a compromise to discover which account they are operating under.

The detection block defines a search identifier called selection, and the condition field references it. The tags field maps to MITRE ATT&CK technique T1033 (System Owner/User Discovery).

  • Save rules/whoami_execution.yml.
  • Confirm the file exists by running this in your terminal:
ls rules/

You should see whoami_execution.yml listed in the output.

Don't see the file?

Make sure you created the file inside the rules/ directory, not in the project root. If it is in the wrong location, drag it into the rules folder in VS Code's Explorer sidebar.

Help me figure out why my rule file isn't showing up.

Configure yamllint and validate syntax

Sigma rules use 4-space indentation, but yamllint's default config expects 2 spaces. You need a custom configuration file so yamllint knows what to enforce.

  • In VS Code, right-click the project root in the **Explorer** sidebar (the top-level folder).
  • Select **New File** and name it .yamllint.yml.
  • Paste the following configuration:
extends: default

rules:
  line-length:
    max: 150
    level: warning
  indentation:
    spaces: 4
    indent-sequences: true
  document-start: disable
  truthy:
    check-keys: false

What does this configuration do?

  • indentation: spaces: 4 tells yamllint to expect 4-space indentation, matching the Sigma convention.
  • document-start: disable stops yamllint from requiring a --- at the top of each file. Sigma rules do not use document-start markers.
  • truthy: check-keys: false prevents yamllint from flagging keys like status: test as truthy values.
  • line-length: max: 150 raises the line length limit because detection rule descriptions can be verbose.
  • Save .yamllint.yml.
  • Run yamllint against your rules directory:
yamllint rules/

You should see no output at all. yamllint follows the Unix Rule of Silence: no news is good news. An empty response means zero errors and zero warnings.

Seeing indentation or truthy errors?

If yamllint reports problems, it means it is not picking up your .yamllint.yml config. Confirm the file is named exactly .yamllint.yml (with the leading dot) and sits in the same directory where you ran the command.

Help me troubleshoot yamllint not finding my config file.

✔️ Awesome, I've got everything!

Great. Make sure you have saved both files before moving on.

ⓧ I'd like to double check the full code

Here is the complete .yamllint.yml file:

extends: default

rules:
  line-length:
    max: 150
    level: warning
  indentation:
    spaces: 4
    indent-sequences: true
  document-start: disable
  truthy:
    check-keys: false

And here is the complete rules/whoami_execution.yml file:

title: Whoami Command Execution
id: e34a5272-f6e4-4117-9e5b-4b1f4a5e0e4c
status: test
description: Detects execution of whoami.exe which may indicate reconnaissance activity
author: Detection Engineering Team
date: 2026-07-29
references:
    - https://attack.mitre.org/techniques/T1033/
logsource:
    category: process_creation
    product: windows
detection:
    selection:
        Image|endswith: '\whoami.exe'
    condition: selection
falsepositives:
    - Legitimate administrator activity
    - Inventory scripts
level: low
tags:
    - attack.discovery
    - attack.t1033
Validate against the Sigma JSON Schema

Passing yamllint proves your file is syntactically valid YAML. It does not prove it is a valid Sigma rule. The JSON Schema you downloaded in Step 1 defines exactly which fields a Sigma rule must have and what types they accept.

Before you run this, what do you think the output will be: an error message or a success confirmation?

  • Validate your rule against the Sigma schema by running:
check-jsonschema --schemafile schemas/sigma-detection-rule-schema.json --verbose rules/whoami_execution.yml

You should see ok -- validation done printed in green. This confirms your rule satisfies every requirement in the Sigma specification: the three required fields are present, the detection object contains a condition, and all values match their expected types.

Why two validation layers?

yamllint catches formatting and syntax issues (bad indentation, trailing spaces, duplicate keys). check-jsonschema catches structural issues (missing required fields, wrong value types, invalid enum values). A file can pass yamllint and still fail schema validation if it is missing the detection block, for example.

You will see this distinction clearly in Step 4 when you deliberately break rules in both ways.

Seeing a schema validation error?

Double-check that your rule file has all three required top-level fields: title, logsource, and detection. Inside detection, confirm you have condition: selection.

Help me understand why my rule is failing schema validation.

  • Commit your progress by running these commands:
git add .
git commit -m "Add first detection rule and yamllint config"

You should see a commit confirmation with the files listed.

You now have a validated baseline. You know exactly what clean output looks like from both tools. Next up, you will wire these same commands into a GitHub Actions workflow so validation runs automatically on every push.

Build the CI Pipeline

In the previous step, you proved that your local tools catch errors before you push. The problem is that running yamllint and check-jsonschema locally is voluntary. Nothing stops you from skipping those checks and pushing a broken rule straight to the repository.

A CI/CD pipeline removes that choice. By wiring up GitHub Actions to run both validation commands on every push, broken rules can never merge into main. In this step, you will create that pipeline and watch it pass on your first push.

In this step, get ready to:
  • Create a GitHub Actions workflow with two validation jobs (YAML lint and schema validation).
  • Push your repository to GitHub and watch the pipeline run.
  • Read the successful CI log output and understand its structure.
Create the GitHub Actions workflow file

GitHub Actions workflows live in the .github/workflows/ directory. Each workflow is a YAML file that defines when to run and what jobs to execute.

  • In VS Code, create a new file at .github/workflows/validate-rules.yml and paste the following workflow header and first job:
name: Validate Detection Rules

on:
  push:
    paths:
      - 'rules/**'
      - 'schemas/**'
      - '.github/workflows/validate-rules.yml'
  pull_request:
    paths:
      - 'rules/**'
      - 'schemas/**'

jobs:
  lint-yaml:
    name: YAML Lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - name: Install yamllint
        run: pip install yamllint==1.38.0
      - name: Run yamllint
        run: yamllint --strict rules/

What does this workflow do so far?

  • on: push: paths: triggers the workflow only when files in rules/, schemas/, or the workflow file itself change. Unrelated pushes skip CI entirely.
  • The lint-yaml job installs the exact same version of yamllint you use locally (1.38.0) and runs yamllint --strict rules/ to catch syntax errors. The --strict flag treats warnings as errors so nothing slips through.
  • Below the lint-yaml job (after the last run: yamllint --strict rules/ line), add a blank line and paste the second job:
  validate-schema:
    name: Schema Validation
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.12'
      - name: Install check-jsonschema
        run: pip install check-jsonschema==0.37.4
      - name: Validate rules against Sigma schema
        run: check-jsonschema --schemafile schemas/sigma-detection-rule-schema.json rules/*.yml

What does the second job do?

The validate-schema job installs check-jsonschema (0.37.4) and validates every .yml file in rules/ against your Sigma JSON Schema. This catches rules that are valid YAML but violate the Sigma specification (like a missing detection field).

Both jobs run in parallel on separate runners. If either fails, the overall workflow fails and the push is flagged red.

  • Save validate-rules.yml.

Indentation looking off?

YAML is whitespace-sensitive. The validate-schema key must be indented at exactly 2 spaces (aligned with lint-yaml above it). Each nested level adds 2 more spaces. If your editor auto-indents with 4 spaces, adjust manually.

help me fix my workflow YAML indentation

✔️ Awesome, I've got everything!

Great. Make sure you have saved validate-rules.yml before moving on.

ⓧ I'd like to double check the full code

Your .github/workflows/validate-rules.yml file should contain both chunks in order:

  • Chunk 1: The workflow name, on: triggers (push and pull_request with path filters), and the full lint-yaml job (checkout, setup-python, install yamllint 1.38.0, run yamllint).
  • Chunk 2: The full validate-schema job (checkout, setup-python, install check-jsonschema 0.37.4, run check-jsonschema with the schema file path).
  • Both jobs sit under jobs: at 2-space indent. Confirm the file has 41 lines total and no extra blank lines at the end.
Push the repository to GitHub

Your workflow file is ready. Now you need to push the entire repository to GitHub so Actions can pick it up.

  • Stage and commit the workflow file by running these commands in your terminal:
git add .github/workflows/validate-rules.yml
git commit -m "Add CI validation workflow"
  • In your browser, navigate to your GitHub Enterprise instance.
  • Click the + icon in the top-right corner and select New repository.
  • Name the repository detection-rules and leave it set to Private.
  • Do not initialize with a README, .gitignore, or license (your local repo already has commits).
  • Click Create repository.
  • Copy the HTTPS URL shown on the quick setup page.
  • Back in your terminal, connect the remote and push by running:
git remote add origin [[REPO_URL="your GitHub repository HTTPS URL"]]
git push -u origin main

What does this push trigger?

Because your push includes files under rules/ and .github/workflows/, the path filter in your workflow matches. GitHub Actions automatically picks up the validate-rules.yml workflow and starts both jobs.

Before you check the result, predict: do you think both jobs will pass or fail? You validated locally in the previous step, so what would you expect?

  • In your browser, navigate to your detection-rules repository on GitHub.
  • Click the Actions tab at the top of the repository page.
  • You should see a workflow run titled Add CI validation workflow (matching your commit message). Wait for it to complete.

Both the YAML Lint and Schema Validation jobs display green checkmarks. Your rule passed both checks in CI, exactly as it did locally.

Seeing a failing workflow or no workflow at all?

  • If the Actions tab shows no runs, confirm your push included the .github/workflows/validate-rules.yml file. Run git log --stat -1 to check which files were in your last commit.
  • If a job fails with a Python or pip error, double-check that python-version: '3.12' is quoted as a string in your workflow file (unquoted numbers can be truncated by YAML).
  • If you see error: src refspec main does not match any when pushing, your default branch may be named master. Run git branch to check, then push with that name instead.

help me debug my GitHub Actions workflow run

Read the successful CI log output

Seeing green is satisfying, but you also need to understand the log structure. When something breaks later, you will navigate these same logs to find the error.

  • Click the workflow run name (Add CI validation workflow) to open the run summary.
  • You will see two job boxes: YAML Lint and Schema Validation. Click YAML Lint to expand its steps.
  • Click the Run yamllint step to see its output. You should see yamllint executed with no output lines (no errors means no output for yamllint in strict mode).
  • Go back and click Schema Validation, then expand the Validate rules against Sigma schema step. You should see a success message confirming your rule file passed schema validation.

What is the log structure telling you?

Every CI log follows this hierarchy: workflow run, then job name, then step name, then command output. When CI fails, you navigate this same tree to find the exact step that errored and the command output that explains why.

Notice that each step shows the command it ran and its exit code. An exit code of 0 means success. Any non-zero exit code means failure and stops the job.

Your pipeline is live. Every push that touches a detection rule now gets validated automatically. Next up, you will deliberately break rules to see what CI failures look like and learn to read the error output.

Break Rules and Diagnose CI Failures

Your CI pipeline passed green in the last step. That means it works when everything is correct.

But the whole point of CI is to catch mistakes. Until you have seen it fail and learned to read the error output, you do not actually trust the pipeline. In this step, you will deliberately introduce two classes of error so you can distinguish them and read failure logs with confidence.

In this step, get ready to:
  • Introduce a YAML syntax error and observe CI fail on the lint step.
  • Introduce a schema violation and observe CI fail on the validation step.
  • Compare both failure logs and learn to distinguish the two error types.
Create a branch and introduce a YAML syntax error

You will write a new detection rule with broken indentation. This simulates a common mistake when editing YAML by hand.

  • Switch back to your terminal and create a new branch by running this command:
git checkout -b test/broken-rules
  • Create a new rule file with a deliberate indentation error by running:
cat > rules/suspicious_encoded_powershell.yml << 'EOF'
title: Suspicious Encoded PowerShell Command Line
id: 1f5e1433-9f19-4187-9f4e-0e5b3d5b2b57
status: test
description: Detects PowerShell invoked with encoded command parameters
author: Detection Engineering Team
date: 2026-07-29
references:
    - https://attack.mitre.org/techniques/T1059/001/
logsource:
    category: process_creation
    product: windows
detection:
    selection_img:
        Image|endswith:
            - '\powershell.exe'
            - '\pwsh.exe'
  selection_flag:
        CommandLine|contains:
            - '-enc '
            - '-EncodedCommand'
            - ' -e '
    condition: selection_img and selection_flag
falsepositives:
    - Legitimate admin scripts using EncodedCommand for quoting safety
level: medium
tags:
    - attack.execution
    - attack.t1059.001
EOF

Where is the error?

Look at line 17: selection_flag: is indented with 2 spaces instead of the required 4. This breaks the structure of the detection mapping because yamllint expects consistent 4-space indentation as configured in your .yamllint.yml.

  • Commit and push this broken rule by running:
git add rules/suspicious_encoded_powershell.yml
git commit -m "Add encoded PowerShell rule (broken)"
git push origin test/broken-rules

Before you check the Actions tab, predict: which CI job do you think will fail, and what kind of error message will it show?

  • Return to your repository on GitHub Enterprise.
  • Click the Actions tab.
  • Click the most recent workflow run (it will show a red X).
  • Click the YAML Lint job, then expand the Run yamllint step.

You will see an error that includes the file path, a line number and column, and a message about wrong indentation. The yamllint output format looks like rules/suspicious_encoded_powershell.yml: 17:3 error wrong indentation: expected 4 but found 2 (indentation) pointing you to the exact location of the problem.

How to read a yamllint error

  • The filename tells you which rule file is broken.
  • The line:column pair tells you the exact position in the file.
  • The message describes what went wrong (expected indentation vs. what was found).
  • The rule name in parentheses (e.g. indentation) tells you which yamllint rule flagged the issue.

Don't see a failed run?

Make sure you pushed to a branch that touches files under rules/. The workflow only triggers on changes to rules/**, schemas/**, or the workflow file itself.

Also confirm you pushed to the remote (check for the branch on GitHub).

help me debug

Fix the syntax error but introduce a schema violation

Now you will fix the indentation so the file is valid YAML, but remove the entire detection block. This creates a file that yamllint will pass (it is syntactically correct YAML) but check-jsonschema will reject (the Sigma schema requires a detection field).

  • Replace the rule file with a schema-invalid version by running:
cat > rules/suspicious_encoded_powershell.yml << 'EOF'
title: Suspicious Encoded PowerShell Command Line
id: 1f5e1433-9f19-4187-9f4e-0e5b3d5b2b57
status: test
description: Detects PowerShell invoked with encoded command parameters
author: Detection Engineering Team
date: 2026-07-29
references:
    - https://attack.mitre.org/techniques/T1059/001/
logsource:
    category: process_creation
    product: windows
falsepositives:
    - Legitimate admin scripts using EncodedCommand for quoting safety
level: medium
tags:
    - attack.execution
    - attack.t1059.001
EOF

What changed?

The indentation is now correct everywhere (consistent 4-space indent). But the entire detection block (with selection_img, selection_flag, and condition) is gone. The Sigma JSON Schema lists detection as a required field, so check-jsonschema will reject this file even though it is perfectly valid YAML.

  • Commit and push the updated file by running:
git add rules/suspicious_encoded_powershell.yml
git commit -m "Fix YAML syntax, remove detection block"
git push origin test/broken-rules

Before you check the Actions tab again, predict: will the lint job pass this time? Which job will fail now?

  • Return to the Actions tab on GitHub Enterprise.
  • Click the latest workflow run (another red X).
  • Confirm the YAML Lint job now shows a green checkmark.
  • Click the Schema Validation job, then expand the Validate rules against Sigma schema step.

You will see a different kind of error this time. The check-jsonschema output format shows the filename, a JSON path, and a message stating that a required property is missing. It will tell you that detection is a required property.

How to read a check-jsonschema error

  • The output starts with a header line stating validation errors were encountered.
  • The filename and JSON path ($ for the root object) identify which file and where in the document the error is.
  • The message names the missing required property. In this case detection is missing from the top-level object.
  • Unlike yamllint, there is no line:column reference because the error is structural (a missing field), not positional (a malformed character).

Lint job still failing?

If the YAML Lint job still shows red, you may have a leftover indentation issue. Open rules/suspicious_encoded_powershell.yml in VS Code and confirm every nested key uses exactly 4 spaces (no tabs, no 2-space indentation).

help me fix my YAML indentation

Compare the two failure logs side by side

You now have two failed CI runs in your Actions history. Each one failed at a different stage with a different error format.

  • Click the Actions tab to see the full run history.
  • Open the first failed run (the YAML syntax error) in a new browser tab.
  • Open the second failed run (the schema violation) in another tab.
  • Compare the error output from each.

What is the difference between these two error types?

A yamllint error means the file is not valid YAML at all. The parser cannot make sense of the structure. The error message gives you a specific line number and column where parsing broke down, plus the rule name that caught it.

A check-jsonschema error means the file is valid YAML but does not conform to the Sigma rule specification. The YAML parsed fine, but the resulting data structure is missing a field the schema requires. The error message names the missing property rather than a line number.

When you see a CI failure in the future, this distinction tells you exactly where to look: syntax errors point to a specific character in the file, while schema errors point to a missing or invalid field in the logical structure.

You have seen CI catch two completely different classes of error. You can now read the failure output and immediately know whether you have a YAML syntax problem or a schema compliance problem. Next up, you will fix both errors locally before pushing, closing the full validation loop.

Fix Locally and Re-Push Green

In the previous step, you saw CI catch two different classes of error and learned to read the failure logs. Now it is time to close the loop.

The real detection engineering workflow is: validate locally, fix, then push confidently. In this step you will reproduce the CI error on your own machine, fix the broken rule, confirm it passes locally, and then push to see CI go green.

In this step, get ready to:
  • Run local validation and confirm the same schema error CI caught.
  • Fix the broken rule file and verify it passes locally.
  • Push the fixed branch to GitHub and confirm CI passes green.
Reproduce the CI error locally

CI told you the rule file failed schema validation. Before fixing anything, confirm you can see the exact same error on your own machine.

  • Switch back to your terminal.
  • Run yamllint against the rules directory to confirm YAML syntax is still clean:
yamllint --strict rules/

You should see no output and no error. The file is syntactically valid YAML, so yamllint has nothing to report.

Before you run the next command, what do you expect to happen when you validate against the Sigma schema?

  • Run check-jsonschema against the rules to see the schema violation:
check-jsonschema --schemafile schemas/sigma-detection-rule-schema.json rules/*.yml

You will see an error reporting that a required property is missing from rules/suspicious_encoded_powershell.yml. This is the same schema validation failure CI reported.

What does this confirm?

Running the same tools locally produces the same errors CI produces. This proves you never need to push and wait to discover whether a rule is broken.

Fix the broken rule and re-validate

The schema requires every Sigma rule to include a detection object with a condition field. You need to restore the complete detection block with proper 4-space indentation.

  • Open rules/suspicious_encoded_powershell.yml in VS Code.
  • Replace the entire file content with the corrected rule below:
title: Suspicious Encoded PowerShell Command Line
id: 1f5e1433-9f19-4187-9f4e-0e5b3d5b2b57
status: test
description: Detects PowerShell invoked with encoded command parameters
author: Detection Engineering Team
date: 2026-07-29
references:
    - https://attack.mitre.org/techniques/T1059/001/
logsource:
    category: process_creation
    product: windows
detection:
    selection_img:
        Image|endswith:
            - '\powershell.exe'
            - '\pwsh.exe'
    selection_flag:
        CommandLine|contains:
            - '-enc '
            - '-EncodedCommand'
            - ' -e '
    condition: selection_img and selection_flag
falsepositives:
    - Legitimate admin scripts using EncodedCommand for quoting safety
level: medium
tags:
    - attack.execution
    - attack.t1059.001

What does the detection block do?

  • selection_img matches processes whose image path ends with powershell.exe or pwsh.exe.
  • selection_flag matches command lines containing encoded-command flags.
  • condition combines both selections with a logical AND, meaning both must be true for the rule to fire.
  • Save rules/suspicious_encoded_powershell.yml.
  • Re-run both validation commands locally to confirm the fix works:
yamllint --strict rules/
check-jsonschema --schemafile schemas/sigma-detection-rule-schema.json rules/*.yml

Both commands should exit silently with no errors. Zero output means zero problems.

Still seeing a validation error?

  • Check that the detection block uses exactly 4-space indentation, matching the rest of the file.
  • Confirm the condition field exists inside the detection object. The schema requires it.
  • Make sure you saved the file before re-running the commands.

Why is my rule still failing schema validation?

✔️ Awesome, I've got everything!

Great. Double check you have saved rules/suspicious_encoded_powershell.yml and both commands exited with no output.

ⓧ I'd like to double check the full code

Here is the complete rules/suspicious_encoded_powershell.yml file:

title: Suspicious Encoded PowerShell Command Line
id: 1f5e1433-9f19-4187-9f4e-0e5b3d5b2b57
status: test
description: Detects PowerShell invoked with encoded command parameters
author: Detection Engineering Team
date: 2026-07-29
references:
    - https://attack.mitre.org/techniques/T1059/001/
logsource:
    category: process_creation
    product: windows
detection:
    selection_img:
        Image|endswith:
            - '\powershell.exe'
            - '\pwsh.exe'
    selection_flag:
        CommandLine|contains:
            - '-enc '
            - '-EncodedCommand'
            - ' -e '
    condition: selection_img and selection_flag
falsepositives:
    - Legitimate admin scripts using EncodedCommand for quoting safety
level: medium
tags:
    - attack.execution
    - attack.t1059.001
Push the fixed branch and confirm CI passes green

Local validation passes cleanly. Time to push the fix and let CI confirm it on its side.

  • Stage and commit the fix by running these commands:
git add rules/suspicious_encoded_powershell.yml
git commit -m "Fix: restore detection block in encoded powershell rule"
  • Push the branch to GitHub by running:
git push

Before you check the Actions tab, do you expect both CI jobs to pass this time?

  • Switch back to your browser and open the Actions tab in your GitHub repository.
  • Click on the latest workflow run triggered by your push.

You should see green checkmarks on both the YAML Lint and Schema Validation jobs. That confirms CI agrees with your local tools: the rule is valid.

  • Merge the branch into main by running:
git checkout main
git merge --no-ff -m "Merge fix for encoded powershell rule" -
git push

Why merge with --no-ff?

The --no-ff flag creates a merge commit even when a fast-forward is possible. This preserves the branch history so you can see exactly which commits were part of the fix.

CI still showing red?

  • Confirm you pushed the correct branch (the one with the fix, not an older commit).
  • Check the Actions tab for the exact error. If it mentions a file path, open that file locally and re-run the matching validation command.
  • Verify the workflow file paths filter includes rules/** so it triggers on your changes.

Why is my CI still failing after I fixed the rule locally?

You have completed the full detection engineering loop: break, diagnose, fix locally, re-push green. This is the workflow professional detection engineers use every day. Next up, you will take it one step further by making it impossible to even commit a broken rule.

Secret mission

Add Pre-Commit Hooks for Automatic Validation

You have local validation tools and a CI pipeline. But what if you forget to run the tools before pushing? In this secret mission, you will configure pre-commit hooks that run validation automatically before every commit, making it impossible to push a broken rule.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally with a free GitHub repository, so there are no ongoing costs.

Resources you used:

  • GitHub Enterprise repository (free, no ongoing cost).
  • Local project folder containing rules/, schemas/, and .github/workflows/.
  • CLI tools installed via pipx: yamllint, check-jsonschema, and pre-commit.

Keep everything running

No action needed. Choose this if you plan to use this repository as a template for your real detection rules.

  • Your GitHub repository stays available as a starting point for future detection rules.
  • Your local CLI tools (yamllint, check-jsonschema, pre-commit) remain available for other projects.
  • The pre-commit hooks stay active in this repository, validating every future commit automatically.

Pause - I'll come back to this later

Since there are no running processes or paid resources, there is nothing to pause. Your repository and local files will be waiting for you when you return.

  • No action required. Everything is already in a paused state.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow these steps to clean up everything.

  • Delete the GitHub repository: open your repository on GitHub Enterprise, click Settings, scroll down to the Danger Zone section, and click Delete this repository. Type the repository name to confirm.
  • Delete the local project folder by running this command (replace the folder name with your actual project folder):
cd ~/Desktop && rm -rf [[PROJECT_FOLDER="your project folder name"]]
  • Uninstall the CLI tools you installed via pipx by running:
pipx uninstall yamllint
pipx uninstall check-jsonschema
pipx uninstall pre-commit

Nice Work!

Nice Work!

Nice work! You have just built a detection engineering validation pipeline that catches broken rules before they ever reach production.

You've learned how to:

  • Build a detection rules repository using the Sigma YAML format and validate rules locally with yamllint and check-jsonschema.
  • Create a GitHub Actions CI pipeline that automatically validates rules on every push, distinguishing between YAML syntax errors and schema validation errors.
  • Complete the full break, diagnose, fix locally, re-push green loop that professional detection engineers use daily to read CI failure logs and pinpoint the exact file, line, and error.
  • Secret Mission: Configured pre-commit hooks to block broken rules before they are even committed.

Ready to quiz yourself?