NIDS YAML Rule Engine with ATT&CK

Refactor a Python NIDS into a declarative YAML rule engine with MITRE ATT&CK tagging.

Introduction

30 Second Summary

Every professional security operations center writes detection rules in structured formats that other tools can read, test, and share. If your detections only live inside hardcoded Python functions, they are invisible to the rest of the security ecosystem.

In this project, you will refactor your existing NIDS detections into a YAML-based rule engine inspired by the Sigma rule format. Each detection becomes a standalone file tagged with its MITRE ATT&CK technique ID, and every alert is emitted as structured JSON ready for SIEM ingestion.

What You'll Build

A declarative detection engineering framework that reads Sigma-inspired YAML rule files from a directory, evaluates live packet data against those rules, and outputs structured JSON-lines alerts enriched with ATT&CK context.

By the end of this project, you'll have:

  • A rules/ directory containing 4 YAML detection rule files that replace hardcoded Python logic with declarative configuration.
  • A rule engine that loads, validates, and evaluates rules at startup, so adding a detection means adding a file instead of editing code.
  • Structured JSON-lines output to alerts.jsonl with timestamps, ATT&CK technique IDs, severity levels, and evidence fields ready for SIEM ingestion.
  • Secret Mission: Generate an ATT&CK Navigator layer JSON and visualize your detection coverage as a heatmap on the ATT&CK matrix.

Are there any prerequisites?

This project builds on an existing Python/Scapy NIDS implementation with ARP spoof, SYN flood, ICMP flood, and port scan detections. You should already have a working NIDS codebase with a Rich dashboard and alerts.log output running on lo0 or en0.

Set Up ATT&CK Mapping

Your NIDS currently fires alerts for port scans, ARP spoofing, SYN floods, and ICMP floods. Those detections live as hardcoded Python functions with no connection to industry threat frameworks.

In this step, you will install the library needed for YAML rule parsing, scaffold the project structure, and map each of your existing detections to its official MITRE ATT&CK technique ID.

In this step, get ready to:
  • Install PyYAML for parsing YAML detection rules.
  • Create the rules/ directory and supporting files.
  • Map 4 existing detections to MITRE ATT&CK technique IDs and tactics.
Install PyYAML in your virtual environment

Your rule engine will read detection rules from YAML files. PyYAML is the library that parses YAML into Python dictionaries so your engine can evaluate them at runtime.

  • Install PyYAML by running this command in your terminal:
pip install pyyaml==6.0.3
  • Verify the installation by running:
python -c "import yaml; print(yaml.__version__)"

You should see 6.0.3 printed in the terminal.

Seeing a ModuleNotFoundError?

Make sure your virtual environment is active. You should see the environment name in parentheses at the start of your terminal prompt.

If pip installed the package but the verify command fails, confirm you are running python from the same environment where you ran pip.

help me fix a ModuleNotFoundError for PyYAML in my virtual environment

Create the project structure

The rule engine needs three things: a directory to hold individual YAML rule files, a Python module for the engine logic, and an output file for structured alerts.

  • Create the project structure by running these commands:
mkdir rules
touch rule_engine.py
touch alerts.jsonl

What do these files do?

  • rules/ is the directory where each detection rule lives as its own YAML file. Adding a new detection means adding a file here.
  • rule_engine.py will contain the engine that loads and evaluates rules against packet data.
  • alerts.jsonl will store structured JSON-lines alerts ready for SIEM ingestion.
  • Confirm the files exist by running:
ls

You should see rules, rule_engine.py, and alerts.jsonl in the output alongside your existing NIDS files.

Missing a file?

If rules does not appear, check that you ran mkdir rules (not touch rules). mkdir creates a directory while touch creates an empty file.

help me create the rules directory and project files for my NIDS rule engine

Map your detections to MITRE ATT&CK

MITRE ATT&CK is a knowledge base that catalogues adversary techniques observed in real-world attacks. Each technique has a unique ID, a name, and belongs to one or more tactics (the adversary's goal).

Your four existing detections each correspond to a specific ATT&CK technique. This mapping is the foundation for every rule file you will write in the next step.

  • Review the reference table below. This is the mapping you will embed into your YAML rules:
  • Port Scan -> T1046 (Network Service Discovery) -> Tactic: Discovery.
  • ARP Spoofing -> T1557.002 (ARP Cache Poisoning) -> Tactics: Credential Access, Collection.
  • SYN Flood -> T1498.001 (Direct Network Flood) -> Tactic: Impact.
  • ICMP Flood -> T1498.001 (Direct Network Flood) -> Tactic: Impact.

What is a tactic vs. a technique?

A tactic is the adversary's goal. Discovery means the attacker is trying to learn about the network. Impact means the attacker is trying to disrupt availability.

A technique is the specific method used to achieve that goal. T1046 (Network Service Discovery) is one way an attacker performs Discovery. T1498.001 (Direct Network Flood) is one way an attacker causes Impact.

Why do SYN Flood and ICMP Flood share a technique ID?

Both are volumetric flooding attacks that overwhelm network resources. ATT&CK groups them under the same sub-technique (T1498.001) because the adversary goal and method are identical. The difference is the protocol used, not the technique.

Your environment is ready and your ATT&CK mapping is defined. Next up, you will write the actual YAML rule files that encode these detections as declarative configuration.

Write YAML Detection Rules

You now have a clear MITRE ATT&CK mapping for each of your four detections. The next step is turning that mapping into something your NIDS can read at startup: declarative YAML rule files.

In enterprise security teams, detection engineers write rules in formats like Sigma rather than editing Python code directly. This keeps detection logic separate from engine logic. In this step, you will write four Sigma-inspired YAML rules that your rule engine will load in the next step.

In this step, get ready to:
  • Create a port scan detection rule using a Sigma-inspired YAML format.
  • Write three more rules for ARP spoofing, SYN flood, and ICMP flood detections.
  • Validate all rules load correctly with PyYAML.
Create the port scan detection rule

Each detection rule lives in its own YAML file inside the rules/ directory. The format follows Sigma's structure: metadata at the top (title, ID, description), a detection section describing what to look for, and tags at the bottom mapping to ATT&CK techniques.

  • In VS Code, right-click the rules folder in the Explorer sidebar.
  • Select New File and name it port_scan.yml.
  • Paste in the following detection rule:
title: Network Service Discovery via Port Scan
id: nids-001
description: Detects a single source IP scanning multiple unique destination ports within a time window, indicating network reconnaissance.
references:
    - https://attack.mitre.org/techniques/T1046/
author: Detection Engineer
date: 2026-07-10
logsource:
    category: network_traffic
    product: custom_nids
detection:
    protocol: tcp
    flags: S
    condition: threshold
    threshold:
        field: dst_port
        unique_count: 10
        per: src_ip
        timewindow: 10
level: high
falsepositives:
    - Legitimate vulnerability scanners
    - Network discovery tools used by IT staff
tags:
    - attack.discovery
    - attack.t1046

What does this rule define?

This rule follows a Sigma-inspired structure adapted for live packet inspection rather than log-event matching. Here is what each section does:

  • title, id, description provide human-readable metadata identifying the detection.
  • logsource declares this is a network traffic rule processed by your custom NIDS.
  • detection holds the actual logic: match TCP SYN packets, and trigger when a single source hits 10 or more unique destination ports within 10 seconds.
  • tags maps the rule to ATT&CK using the attack.t1046 convention. Your rule engine will parse this to extract the technique ID.
  • level sets alert severity. Sigma defines five levels: informational, low, medium, high, and critical.

Why Sigma-inspired instead of actual Sigma?

Sigma is designed for SIEM log-source queries, not live packet inspection. Your rules borrow Sigma's structure (title, detection, level, tags) but adapt the detection section for network packet fields like protocol, flags, and thresholds.

This means you get the same workflow ("add a file, not edit code") without needing the full pySigma conversion pipeline.

  • Save rules/port_scan.yml.

YAML syntax error when saving?

  • YAML is indentation-sensitive. Make sure each nested level uses exactly 4 spaces, not tabs.
  • Check that list items under tags and falsepositives start with - (dash followed by a space).
  • Help me fix a YAML indentation error in my detection rule.
Create the remaining detection rules

The remaining three rules follow the same structure. Each targets a different protocol and uses a different detection condition type.

  • Create a new file called arp_spoof.yml inside the rules folder with the following content:
title: ARP Cache Poisoning Detected
id: nids-002
description: Detects a known IP address claiming a different MAC address than previously observed, indicating ARP spoofing.
references:
    - https://attack.mitre.org/techniques/T1557/002/
author: Detection Engineer
date: 2026-07-10
logsource:
    category: network_traffic
    product: custom_nids
detection:
    protocol: arp
    condition: arp_mismatch
    arp_mismatch:
        track: ip_to_mac
        alert_on: mac_change
level: critical
falsepositives:
    - DHCP reassignment causing legitimate MAC changes
    - Virtual machine migration
tags:
    - attack.credential_access
    - attack.t1557.002

What is different about this rule?

Instead of a threshold condition, this rule uses arp_mismatch. The rule engine will track IP-to-MAC mappings and alert when a known IP suddenly claims a different MAC address. The severity is critical because ARP spoofing enables man-in-the-middle attacks.

  • Save rules/arp_spoof.yml.
  • Create a new file called syn_flood.yml inside the rules folder with the following content:
title: SYN Flood Denial of Service
id: nids-003
description: Detects a high volume of TCP SYN packets from a single source within a short time window, indicating a SYN flood attack.
references:
    - https://attack.mitre.org/techniques/T1498/001/
author: Detection Engineer
date: 2026-07-10
logsource:
    category: network_traffic
    product: custom_nids
detection:
    protocol: tcp
    flags: S
    condition: threshold
    threshold:
        field: packet_count
        count: 100
        per: src_ip
        timewindow: 10
level: high
falsepositives:
    - High-traffic web servers receiving burst connections
    - Load testing tools
tags:
    - attack.impact
    - attack.t1498.001
  • Save rules/syn_flood.yml.
  • Create a new file called icmp_flood.yml inside the rules folder with the following content:
title: ICMP Flood Denial of Service
id: nids-004
description: Detects a high volume of ICMP packets from a single source within a short time window, indicating an ICMP flood attack.
references:
    - https://attack.mitre.org/techniques/T1498/001/
author: Detection Engineer
date: 2026-07-10
logsource:
    category: network_traffic
    product: custom_nids
detection:
    protocol: icmp
    condition: threshold
    threshold:
        field: packet_count
        count: 100
        per: src_ip
        timewindow: 10
level: high
falsepositives:
    - Network monitoring tools using ICMP
    - Legitimate ping sweeps by IT staff
tags:
    - attack.impact
    - attack.t1498.001

How do the flood rules differ from the port scan rule?

The port scan rule counts unique destination ports (using unique_count). The flood rules count raw packet_count (using count). Both use the threshold condition type, but the rule engine will handle them differently based on which field they track.

Both SYN flood and ICMP flood map to T1498.001 (Direct Network Flood) because they detect volumetric traffic patterns at the network level.

  • Save rules/icmp_flood.yml.

File not appearing in the rules folder?

  • Make sure you right-clicked the rules folder (not the project root) when creating each file. Files must be inside rules/, not alongside it.
  • Confirm the file extension is .yml, not .yaml. The rule engine will look for *.yml files specifically.
  • My YAML rule files aren't showing up in the rules folder.

✔️ Awesome, I've got everything!

Great. Make sure all four files are saved before moving on to validation.

ⓧ I'd like to double check the full code

Here are all four rule files for comparison. Make sure your files match these exactly.

title: Network Service Discovery via Port Scan
id: nids-001
description: Detects a single source IP scanning multiple unique destination ports within a time window, indicating network reconnaissance.
references:
    - https://attack.mitre.org/techniques/T1046/
author: Detection Engineer
date: 2026-07-10
logsource:
    category: network_traffic
    product: custom_nids
detection:
    protocol: tcp
    flags: S
    condition: threshold
    threshold:
        field: dst_port
        unique_count: 10
        per: src_ip
        timewindow: 10
level: high
falsepositives:
    - Legitimate vulnerability scanners
    - Network discovery tools used by IT staff
tags:
    - attack.discovery
    - attack.t1046
title: ARP Cache Poisoning Detected
id: nids-002
description: Detects a known IP address claiming a different MAC address than previously observed, indicating ARP spoofing.
references:
    - https://attack.mitre.org/techniques/T1557/002/
author: Detection Engineer
date: 2026-07-10
logsource:
    category: network_traffic
    product: custom_nids
detection:
    protocol: arp
    condition: arp_mismatch
    arp_mismatch:
        track: ip_to_mac
        alert_on: mac_change
level: critical
falsepositives:
    - DHCP reassignment causing legitimate MAC changes
    - Virtual machine migration
tags:
    - attack.credential_access
    - attack.t1557.002
title: SYN Flood Denial of Service
id: nids-003
description: Detects a high volume of TCP SYN packets from a single source within a short time window, indicating a SYN flood attack.
references:
    - https://attack.mitre.org/techniques/T1498/001/
author: Detection Engineer
date: 2026-07-10
logsource:
    category: network_traffic
    product: custom_nids
detection:
    protocol: tcp
    flags: S
    condition: threshold
    threshold:
        field: packet_count
        count: 100
        per: src_ip
        timewindow: 10
level: high
falsepositives:
    - High-traffic web servers receiving burst connections
    - Load testing tools
tags:
    - attack.impact
    - attack.t1498.001
title: ICMP Flood Denial of Service
id: nids-004
description: Detects a high volume of ICMP packets from a single source within a short time window, indicating an ICMP flood attack.
references:
    - https://attack.mitre.org/techniques/T1498/001/
author: Detection Engineer
date: 2026-07-10
logsource:
    category: network_traffic
    product: custom_nids
detection:
    protocol: icmp
    condition: threshold
    threshold:
        field: packet_count
        count: 100
        per: src_ip
        timewindow: 10
level: high
falsepositives:
    - Network monitoring tools using ICMP
    - Legitimate ping sweeps by IT staff
tags:
    - attack.impact
    - attack.t1498.001
Validate all rules load correctly

Before building the rule engine, you need to confirm that PyYAML can parse every rule file without errors. A single misplaced indent or missing colon will cause a silent failure later.

  • Verify all 4 rules load successfully by running this command in your terminal:
python -c "import yaml, glob
for f in glob.glob('rules/*.yml'):
    with open(f) as fh:
        rule = yaml.safe_load(fh)
        print(f\"{rule['id']}: {rule['title']}\")"

What does this command do?

  • glob.glob('rules/*.yml') finds every .yml file in the rules/ directory.
  • yaml.safe_load(fh) parses the YAML safely (no arbitrary code execution). If the YAML is malformed, this line throws an error.
  • The print statement confirms each rule's ID and title loaded correctly.

You should see output similar to this (the order may vary):

nids-001: Network Service Discovery via Port Scan
nids-002: ARP Cache Poisoning Detected
nids-003: SYN Flood Denial of Service
nids-004: ICMP Flood Denial of Service

Seeing a YAML parsing error?

  • The error message will include a line number. Open the file it mentions and check that line for missing colons, incorrect indentation, or stray characters.
  • If you see fewer than 4 lines printed, one or more files may have the wrong extension. Confirm each file ends with .yml.
  • If you see a KeyError for id or title, make sure those fields exist at the top level of the file (no indentation).
  • Help me debug a YAML parsing error in my detection rules.

All four detection rules are valid and ready to be consumed by a rule engine. Next up, you will build that engine to load these rules at startup and evaluate live packets against them.

Build the Rule Engine

Your YAML rules now define what to detect. The next piece is an engine that reads those rules at startup and evaluates every incoming packet against them.

In this step, you will build a RuleEngine class that loads all four YAML rules, checks each packet's metadata against the loaded conditions, and returns structured alert dictionaries tagged with MITRE ATT&CK technique IDs.

In this step, get ready to:
  • Build a Rule class that parses YAML data and extracts ATT&CK metadata.
  • Implement a RuleEngine class with threshold and ARP mismatch evaluation.
  • Integrate the engine into your existing NIDS main loop.
Create the Rule class

Each YAML rule file contains a title, detection conditions, severity level, and ATT&CK tags. You need a Python class that takes this raw dictionary and exposes clean attributes your engine can use for evaluation and alert generation.

  • Open rule_engine.py in VS Code. The file is currently empty.
  • Add the imports and the full Rule class by pasting this code:
import yaml
import glob
import time
from collections import defaultdict


class Rule:
    def __init__(self, data):
        self.title = data['title']
        self.id = data['id']
        self.description = data.get('description', '')
        self.level = data['level']
        self.tags = data.get('tags', [])
        self.detection = data['detection']
        self.technique_id = self._extract_technique_id()
        self.technique_name = self._get_technique_name()
        self.tactic = self._extract_tactic()

    def _extract_technique_id(self):
        # Parse the attack.tXXXX tag into a proper technique ID
        for tag in self.tags:
            if tag.startswith('attack.t'):
                raw = tag.replace('attack.', '')
                parts = raw.split('.')
                if len(parts) == 2:
                    return parts[0].upper() + '.' + parts[1]
                return raw.upper()
        return 'UNKNOWN'

    def _get_technique_name(self):
        # Map technique IDs to human-readable names
        mapping = {
            'T1046': 'Network Service Discovery',
            'T1557.002': 'ARP Cache Poisoning',
            'T1498.001': 'Direct Network Flood',
            'T1110': 'Brute Force',
        }
        return mapping.get(self.technique_id, 'Unknown Technique')

    def _extract_tactic(self):
        # Pull the tactic tag (non-technique attack.* tag)
        tactic_tags = [t for t in self.tags if t.startswith('attack.') and not t.startswith('attack.t')]
        if tactic_tags:
            return tactic_tags[0].replace('attack.', '')
        return 'unknown'

What does this code do?

The Rule class takes the raw dictionary from yaml.safe_load() and exposes structured attributes.

  • _extract_technique_id() parses tags like attack.t1557.002 into T1557.002. It handles both simple IDs and sub-technique IDs with a dot separator.
  • _get_technique_name() uses a lookup dictionary to map technique IDs to their official ATT&CK names.
  • _extract_tactic() finds the first non-technique tag (like attack.discovery) and strips the prefix to give you a clean tactic name.
  • Save rule_engine.py.
  • You won't see visible output yet. The next chunk adds the RuleEngine class that uses Rule to load your YAML files.
Implement the RuleEngine loading logic

The RuleEngine class is responsible for scanning the rules/ directory, loading each YAML file into a Rule object, and maintaining state trackers for time-window-based detections.

  • In rule_engine.py, add the following RuleEngine class below the Rule class:
class RuleEngine:
    def __init__(self, rules_dir='rules'):
        self.rules = []
        self.trackers = defaultdict(lambda: defaultdict(list))
        self._load_rules(rules_dir)

    def _load_rules(self, rules_dir):
        # Load all YAML rule files from the rules directory
        for filepath in glob.glob(f'{rules_dir}/*.yml'):
            with open(filepath) as f:
                data = yaml.safe_load(f)
                self.rules.append(Rule(data))
        print(f'[RuleEngine] Loaded {len(self.rules)} rules')

What does this code do?

The __init__ method creates an empty list for rules and a nested defaultdict for tracking packet counts over time windows. It then calls _load_rules() immediately.

The _load_rules() method uses glob to find every .yml file in the rules directory. Each file is parsed with yaml.safe_load() and wrapped in a Rule object. The confirmation message prints how many rules loaded successfully.

  • Save rule_engine.py.
  • Test the loading by running this in your terminal:
python -c "from rule_engine import RuleEngine; engine = RuleEngine()"
  • You should see [RuleEngine] Loaded 4 rules printed to the terminal.

Seeing a ModuleNotFoundError?

  • Make sure your virtual environment is active. You should see (venv) in your terminal prompt.
  • Confirm you are running the command from the project root directory where rule_engine.py and the rules/ folder both exist.
  • Help me debug my virtual environment and file structure
Implement the evaluation logic

The evaluate() method is the core dispatcher. It receives a packet metadata dictionary, checks protocol and flag filters as early exits, then routes to the correct condition handler based on each rule's condition field.

  • Add the evaluate method inside the RuleEngine class, below _load_rules:
    def evaluate(self, packet_meta):
        triggered = []
        current_time = time.time()

        for rule in self.rules:
            detection = rule.detection
            protocol = detection.get('protocol', '')

            # Early exit: skip rules that don't match the packet protocol
            if packet_meta.get('protocol', '') != protocol:
                continue

            # Early exit: skip rules requiring specific TCP flags
            if 'flags' in detection:
                if packet_meta.get('flags', '') != detection['flags']:
                    continue

            condition = detection.get('condition', '')

            # Route to the correct condition handler
            if condition == 'threshold':
                alert = self._check_threshold(rule, packet_meta, current_time)
                if alert:
                    triggered.append(alert)

            elif condition == 'arp_mismatch':
                alert = self._check_arp_mismatch(rule, packet_meta)
                if alert:
                    triggered.append(alert)

        return triggered

What does this code do?

For every loaded rule, evaluate() first checks whether the packet matches the rule's protocol and flags. This is a fast filter that skips irrelevant rules without doing expensive state lookups.

If the packet passes those checks, the method dispatches to either _check_threshold (for port scans, SYN floods, ICMP floods) or _check_arp_mismatch (for ARP spoofing). Each handler returns an alert dictionary if the condition triggers, or None otherwise.

  • Save rule_engine.py.
  • You won't see a change yet. The next chunk implements the threshold handler that evaluate() calls.
  • Add the _check_threshold method below evaluate:
    def _check_threshold(self, rule, packet_meta, current_time):
        threshold_cfg = rule.detection['threshold']
        field = threshold_cfg['field']
        per = threshold_cfg['per']
        timewindow = threshold_cfg['timewindow']
        tracker_key = f'{rule.id}_{packet_meta.get(per, "unknown")}'

        # Build an entry for the current packet
        entry = {
            'time': current_time,
            'value': packet_meta.get(field, packet_meta.get('dst_port', 0)),
        }

        # Prune entries outside the time window
        self.trackers[tracker_key]['entries'] = [
            e for e in self.trackers[tracker_key].get('entries', [])
            if current_time - e['time'] < timewindow
        ]
        self.trackers[tracker_key]['entries'].append(entry)

        entries = self.trackers[tracker_key]['entries']

        # Check unique port count (port scan detection)
        if field == 'dst_port':
            unique_count = threshold_cfg.get('unique_count', 10)
            unique_ports = set(e['value'] for e in entries)
            if len(unique_ports) >= unique_count:
                self.trackers[tracker_key]['entries'] = []
                return self._build_alert(rule, packet_meta, {
                    'unique_ports_scanned': len(unique_ports),
                    'timewindow_seconds': timewindow,
                })
        # Check raw packet count (flood detection)
        elif field == 'packet_count':
            count = threshold_cfg.get('count', 100)
            if len(entries) >= count:
                self.trackers[tracker_key]['entries'] = []
                return self._build_alert(rule, packet_meta, {
                    'packet_count': len(entries),
                    'timewindow_seconds': timewindow,
                })

        return None

How does threshold detection work?

This method implements a sliding time window. It tracks entries per source IP (the per field from the YAML rule) and prunes any entries older than the timewindow value.

For port scans, it counts unique destination ports. For SYN/ICMP floods, it counts total packets. When the threshold is exceeded, the tracker resets and an alert is generated. This is the same logic your hardcoded detections used, but now driven by the YAML parameters.

  • Save rule_engine.py.
  • You won't see a change yet. The next chunk adds the ARP mismatch handler and alert builder.
  • Add the _check_arp_mismatch and _build_alert methods below _check_threshold:
    def _check_arp_mismatch(self, rule, packet_meta):
        src_ip = packet_meta.get('src_ip', '')
        src_mac = packet_meta.get('src_mac', '')

        if not src_ip or not src_mac:
            return None

        # Maintain an ARP table mapping IPs to MACs
        tracker_key = f'{rule.id}_arp_table'
        arp_table = self.trackers[tracker_key]

        # Alert if a known IP appears with a different MAC
        if src_ip in arp_table and arp_table[src_ip] != src_mac:
            old_mac = arp_table[src_ip]
            arp_table[src_ip] = src_mac
            return self._build_alert(rule, packet_meta, {
                'ip': src_ip,
                'old_mac': old_mac,
                'new_mac': src_mac,
            })

        arp_table[src_ip] = src_mac
        return None

    def _build_alert(self, rule, packet_meta, evidence):
        # Construct a standardized alert dictionary
        return {
            'rule_id': rule.id,
            'rule_name': rule.title,
            'severity': rule.level,
            'technique_id': rule.technique_id,
            'technique_name': rule.technique_name,
            'tactic': rule.tactic,
            'src_ip': packet_meta.get('src_ip', 'N/A'),
            'dst_ip': packet_meta.get('dst_ip', 'N/A'),
            'evidence': evidence,
        }

What do these methods do?

_check_arp_mismatch() maintains a mapping of IP addresses to MAC addresses. When it sees a known IP claim a different MAC, it fires an alert. This is the declarative version of your existing ARP spoofing detection.

_build_alert() creates a standardized alert dictionary that every triggered rule returns. It includes the rule ID, technique ID, severity, source/destination IPs, and detection-specific evidence. This structure makes downstream processing consistent regardless of which rule triggered.

  • Save rule_engine.py.
  • Test the full engine by running this in your terminal:
python -c "from rule_engine import RuleEngine; engine = RuleEngine(); print([r.technique_id for r in engine.rules])"
  • You should see [RuleEngine] Loaded 4 rules followed by a list of technique IDs like ['T1046', 'T1557.002', 'T1498.001', 'T1498.001'] (order may vary).

Seeing fewer than 4 rules?

  • Confirm all four .yml files exist in the rules/ directory. Run ls rules/ and verify you see port_scan.yml, arp_spoof.yml, syn_flood.yml, and icmp_flood.yml.
  • Check for YAML indentation errors. Open each file and make sure tags: is at the top level, not nested under detection:.
  • Help me debug my rules directory

✔️ Awesome, I've got everything!

Great. Double check you have saved rule_engine.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete rule_engine.py file up to this point:

import yaml
import glob
import time
from collections import defaultdict


class Rule:
    def __init__(self, data):
        self.title = data['title']
        self.id = data['id']
        self.description = data.get('description', '')
        self.level = data['level']
        self.tags = data.get('tags', [])
        self.detection = data['detection']
        self.technique_id = self._extract_technique_id()
        self.technique_name = self._get_technique_name()
        self.tactic = self._extract_tactic()

    def _extract_technique_id(self):
        for tag in self.tags:
            if tag.startswith('attack.t'):
                raw = tag.replace('attack.', '')
                parts = raw.split('.')
                if len(parts) == 2:
                    return parts[0].upper() + '.' + parts[1]
                return raw.upper()
        return 'UNKNOWN'

    def _get_technique_name(self):
        mapping = {
            'T1046': 'Network Service Discovery',
            'T1557.002': 'ARP Cache Poisoning',
            'T1498.001': 'Direct Network Flood',
            'T1110': 'Brute Force',
        }
        return mapping.get(self.technique_id, 'Unknown Technique')

    def _extract_tactic(self):
        tactic_tags = [t for t in self.tags if t.startswith('attack.') and not t.startswith('attack.t')]
        if tactic_tags:
            return tactic_tags[0].replace('attack.', '')
        return 'unknown'


class RuleEngine:
    def __init__(self, rules_dir='rules'):
        self.rules = []
        self.trackers = defaultdict(lambda: defaultdict(list))
        self._load_rules(rules_dir)

    def _load_rules(self, rules_dir):
        for filepath in glob.glob(f'{rules_dir}/*.yml'):
            with open(filepath) as f:
                data = yaml.safe_load(f)
                self.rules.append(Rule(data))
        print(f'[RuleEngine] Loaded {len(self.rules)} rules')

    def evaluate(self, packet_meta):
        triggered = []
        current_time = time.time()

        for rule in self.rules:
            detection = rule.detection
            protocol = detection.get('protocol', '')

            if packet_meta.get('protocol', '') != protocol:
                continue

            if 'flags' in detection:
                if packet_meta.get('flags', '') != detection['flags']:
                    continue

            condition = detection.get('condition', '')

            if condition == 'threshold':
                alert = self._check_threshold(rule, packet_meta, current_time)
                if alert:
                    triggered.append(alert)

            elif condition == 'arp_mismatch':
                alert = self._check_arp_mismatch(rule, packet_meta)
                if alert:
                    triggered.append(alert)

        return triggered

    def _check_threshold(self, rule, packet_meta, current_time):
        threshold_cfg = rule.detection['threshold']
        field = threshold_cfg['field']
        per = threshold_cfg['per']
        timewindow = threshold_cfg['timewindow']
        tracker_key = f'{rule.id}_{packet_meta.get(per, "unknown")}'

        entry = {
            'time': current_time,
            'value': packet_meta.get(field, packet_meta.get('dst_port', 0)),
        }

        self.trackers[tracker_key]['entries'] = [
            e for e in self.trackers[tracker_key].get('entries', [])
            if current_time - e['time'] < timewindow
        ]
        self.trackers[tracker_key]['entries'].append(entry)

        entries = self.trackers[tracker_key]['entries']

        if field == 'dst_port':
            unique_count = threshold_cfg.get('unique_count', 10)
            unique_ports = set(e['value'] for e in entries)
            if len(unique_ports) >= unique_count:
                self.trackers[tracker_key]['entries'] = []
                return self._build_alert(rule, packet_meta, {
                    'unique_ports_scanned': len(unique_ports),
                    'timewindow_seconds': timewindow,
                })
        elif field == 'packet_count':
            count = threshold_cfg.get('count', 100)
            if len(entries) >= count:
                self.trackers[tracker_key]['entries'] = []
                return self._build_alert(rule, packet_meta, {
                    'packet_count': len(entries),
                    'timewindow_seconds': timewindow,
                })

        return None

    def _check_arp_mismatch(self, rule, packet_meta):
        src_ip = packet_meta.get('src_ip', '')
        src_mac = packet_meta.get('src_mac', '')

        if not src_ip or not src_mac:
            return None

        tracker_key = f'{rule.id}_arp_table'
        arp_table = self.trackers[tracker_key]

        if src_ip in arp_table and arp_table[src_ip] != src_mac:
            old_mac = arp_table[src_ip]
            arp_table[src_ip] = src_mac
            return self._build_alert(rule, packet_meta, {
                'ip': src_ip,
                'old_mac': old_mac,
                'new_mac': src_mac,
            })

        arp_table[src_ip] = src_mac
        return None

    def _build_alert(self, rule, packet_meta, evidence):
        return {
            'rule_id': rule.id,
            'rule_name': rule.title,
            'severity': rule.level,
            'technique_id': rule.technique_id,
            'technique_name': rule.technique_name,
            'tactic': rule.tactic,
            'src_ip': packet_meta.get('src_ip', 'N/A'),
            'dst_ip': packet_meta.get('dst_ip', 'N/A'),
            'evidence': evidence,
        }
Integrate the rule engine into your NIDS

Your rule engine is ready. Now you need to wire it into your existing NIDS main script so that packet processing goes through rule_engine.evaluate() instead of your hardcoded detection functions.

  • Open your main NIDS script in VS Code.
  • Add this import at the top of the file:
from rule_engine import RuleEngine
  • Initialize the engine near the top of your script (after your other setup code):
# Initialize the YAML rule engine
engine = RuleEngine()
  • In your packet processing callback, build a packet_meta dictionary from the Scapy packet and pass it to engine.evaluate(packet_meta). The dictionary should include keys like protocol, flags, src_ip, dst_ip, dst_port, and src_mac.
  • When evaluate() returns triggered alerts, pass them to your Rich dashboard display. Each alert dictionary includes rule_name and technique_id that you can show in the alert output.

Why wrap instead of replace?

You can keep your existing detection functions alongside the rule engine during testing. Once you confirm the engine produces the same alerts, you can remove the hardcoded functions entirely. This gives you a safe migration path without losing any detections during the refactor.

  • Save your main NIDS script.
  • Start your NIDS against the loopback interface by running it with lo0 as the interface.
  • In a second terminal window, trigger a port scan by running:
sudo nmap -sS localhost
  • Switch back to the terminal running your NIDS. You should see an alert appear in the Rich dashboard showing the rule title (Network Service Discovery via Port Scan) and technique ID (T1046).

No alert appearing in the dashboard?

  • Make sure nmap ran with sudo. The -sS flag requires root privileges to send raw SYN packets.
  • Verify your NIDS is sniffing on lo0 (loopback). Traffic to localhost travels over this interface.
  • Check that the packet_meta dictionary you pass to evaluate() includes protocol set to tcp and flags set to S for SYN packets.
  • Help me debug the rule engine integration

Your rule engine is live. Every detection is now driven by a YAML file instead of hardcoded Python logic. Next up, you will add structured JSON-lines output so these alerts are ready for SIEM ingestion.

Emit Structured JSON-Lines Alerts

Your rule engine detects threats and enriches alerts with ATT&CK context. But right now, those alerts only live in the Rich dashboard and the plain-text alerts.log.

Enterprise SOCs ingest alerts from a structured format that machines can parse without guessing. JSON-lines (one JSON object per line) is the standard format for SIEM ingestion pipelines. In this step, you will build an emitter that writes each triggered alert as a structured JSON-lines record to alerts.jsonl.

In this step, get ready to:
  • Create an AlertEmitter class that writes structured JSON-lines records.
  • Wire the emitter into your NIDS detection pipeline alongside the existing Rich dashboard.
  • Trigger a port scan and verify alerts.jsonl contains valid structured output.
Create the AlertEmitter module

Each alert your rule engine produces contains rich context: rule ID, technique ID, severity, and evidence. The emitter's job is to stamp each alert with a UTC timestamp and append it as a single JSON line to alerts.jsonl.

  • Open alert_emitter.py in VS Code and add the following code:
import json
from datetime import datetime, timezone


class AlertEmitter:
    def __init__(self, jsonl_path='alerts.jsonl'):
        self.jsonl_path = jsonl_path

    def emit(self, alert):
        record = {
            'timestamp': datetime.now(timezone.utc).isoformat(),
            'rule_id': alert['rule_id'],
            'rule_name': alert['rule_name'],
            'severity': alert['severity'],
            'technique_id': alert['technique_id'],
            'technique_name': alert['technique_name'],
            'tactic': alert['tactic'],
            'src_ip': alert['src_ip'],
            'dst_ip': alert['dst_ip'],
            'evidence': alert['evidence'],
        }
        with open(self.jsonl_path, 'a') as f:
            f.write(json.dumps(record) + '\n')
        return record

What does this code do?

  • datetime.now(timezone.utc).isoformat() stamps each alert with a UTC timestamp in ISO 8601 format. This is the universal format SIEMs expect.
  • The emit method assembles a flat dictionary from the alert dict your rule engine already produces, so no transformation is needed upstream.
  • Opening the file in append mode ('a') means each new alert adds a line without overwriting previous ones.
  • json.dumps(record) + '\n' writes exactly one JSON object per line. This is the JSON-lines format that tools like jq, Elastic, and Splunk can ingest natively.
  • Save alert_emitter.py.

Why JSON-lines instead of a JSON array?

A JSON array requires reading the entire file into memory to append a new entry. JSON-lines lets you append a single line without touching the rest of the file.

This matters when your NIDS runs for hours and accumulates thousands of alerts. It also lets downstream tools like tail -f stream alerts in real time.

Wire the emitter into your detection pipeline

Your main NIDS script already calls rule_engine.evaluate(packet_metadata) and displays results in the Rich dashboard. You need to add the emitter as a parallel output channel without replacing anything.

  • In your main NIDS script, add this import near the top of the file:
# Import the structured alert emitter
from alert_emitter import AlertEmitter
  • Below your existing initialization code (where you create the RuleEngine instance), instantiate the emitter:
# Initialize the JSON-lines emitter for SIEM-ready output
emitter = AlertEmitter()
  • Find the section of your main loop where you process the results of rule_engine.evaluate(). Add a call to emitter.emit(alert) for each triggered alert:
# Emit each triggered alert to alerts.jsonl
for alert in triggered_alerts:
    emitter.emit(alert)

What does this wiring do?

Every time a rule fires, the alert now flows to two destinations in parallel: the Rich dashboard (for live monitoring) and alerts.jsonl (for SIEM ingestion). Your existing alerts.log continues to work as before.

This pattern of multiple output channels from a single detection pipeline is how production NIDS systems like Suricata and Zeek operate.

  • Save your main NIDS script.

Seeing a ModuleNotFoundError?

  • Make sure alert_emitter.py is in the same directory as your main NIDS script. Python imports from the current working directory.
  • Check that the filename is exactly alert_emitter.py (not alertemitter.py or alert-emitter.py).
  • Confirm your virtual environment is active before running the script.

Help me debug why my AlertEmitter import isn't working.

Test the full pipeline

Time to prove the pipeline works end-to-end. You will trigger a port scan detection and then inspect the structured output file.

  • Start your NIDS by running it against lo0 (loopback) in one terminal window.
  • Open a second terminal and trigger a port scan against localhost by running:
nmap -sS localhost
  • Wait for the alert to appear in your Rich dashboard. Once you see it, stop the NIDS with Ctrl+C.
  • Inspect the structured output by running:
cat alerts.jsonl | python -m json.tool --no-ensure-ascii

What should I see?

You should see a pretty-printed JSON object with all the required fields populated. The output looks something like this:

{
    "timestamp": "2026-07-10T14:32:01.123456+00:00",
    "rule_id": "nids-001",
    "rule_name": "Network Service Discovery via Port Scan",
    "severity": "high",
    "technique_id": "T1046",
    "technique_name": "Network Service Discovery",
    "tactic": "discovery",
    "src_ip": "127.0.0.1",
    "dst_ip": "127.0.0.1",
    "evidence": {
        "unique_ports_scanned": 10,
        "timewindow_seconds": 10
    }
}

Every field is machine-parseable. A SIEM can index on technique_id, filter by severity, and correlate by src_ip without any regex parsing.

  • Verify that alerts.jsonl contains at least one line with the fields technique_id, rule_name, severity, src_ip, and timestamp.
  • Confirm your Rich dashboard still displayed the alert as before. Both output channels should work in parallel.

alerts.jsonl is still empty?

  • Make sure you ran nmap with -sS (SYN scan). A regular ping or connect scan might not trigger the port scan rule's threshold of 10 unique ports.
  • Check that emitter.emit(alert) is inside the loop that processes triggered alerts, not outside it.
  • Verify alerts.jsonl has write permissions. Run ls -la alerts.jsonl to check.

Help me debug why my alerts.jsonl file is empty after running a port scan.

✔️ Awesome, I've got everything!

Great. Double check you have saved alert_emitter.py and your main NIDS script.

ⓧ I'd like to double check the full code

Here is the complete alert_emitter.py for comparison:

import json
from datetime import datetime, timezone


class AlertEmitter:
    def __init__(self, jsonl_path='alerts.jsonl'):
        self.jsonl_path = jsonl_path

    def emit(self, alert):
        record = {
            'timestamp': datetime.now(timezone.utc).isoformat(),
            'rule_id': alert['rule_id'],
            'rule_name': alert['rule_name'],
            'severity': alert['severity'],
            'technique_id': alert['technique_id'],
            'technique_name': alert['technique_name'],
            'tactic': alert['tactic'],
            'src_ip': alert['src_ip'],
            'dst_ip': alert['dst_ip'],
            'evidence': alert['evidence'],
        }
        with open(self.jsonl_path, 'a') as f:
            f.write(json.dumps(record) + '\n')
        return record

Your NIDS now speaks two languages: a human-friendly Rich dashboard for live monitoring, and a machine-friendly JSON-lines stream ready for SIEM ingestion. Next up, you will document your detection coverage in a professional README table that maps every rule to its ATT&CK technique.

Document Detection Coverage in README

Your rule engine is working, alerts are flowing to alerts.jsonl, and the Rich dashboard is still showing detections in real time. The missing piece is documentation that communicates your coverage to other engineers.

A professional detection engineering team always maintains a coverage table mapping detections to the MITRE ATT&CK framework. In this step, you will add that coverage table, document your structured output schema, and write a recipe so future contributors can add detections without touching code.

In this step, get ready to:
  • Add a Detection Coverage table to your README mapping each rule to its ATT&CK technique.
  • Document the JSON-lines output schema with a sample alert.
  • Write a 3-step recipe for adding new detections without code changes.
Add the Detection Coverage table

A coverage table gives anyone reading your README an instant view of what your NIDS detects and how each detection maps to the ATT&CK matrix. This is the same format enterprise SOCs use to communicate detection posture.

  • Open README.md in VS Code.
  • Add a new section heading and the coverage table by pasting the following markdown:
## Detection Coverage

| Rule ID | Detection | ATT&CK Technique | Technique Name | Tactic | Severity |
|---------|-----------|-------------------|----------------|--------|----------|
| nids-001 | Port Scan | T1046 | Network Service Discovery | Discovery | High |
| nids-002 | ARP Spoofing | T1557.002 | ARP Cache Poisoning | Credential Access | Critical |
| nids-003 | SYN Flood | T1498.001 | Direct Network Flood | Impact | High |
| nids-004 | ICMP Flood | T1498.001 | Direct Network Flood | Impact | High |

What does this table communicate?

Each row ties a rule ID to an ATT&CK technique. This lets a SOC manager instantly see which adversary behaviors your NIDS covers.

Notice that nids-003 and nids-004 both map to T1498.001. Two different attack patterns (SYN flood vs ICMP flood) are both categorized as Direct Network Flood because ATT&CK groups them by adversary objective, not by packet type.

  • Save README.md.
  • Preview the markdown in VS Code by pressing Cmd+Shift+V (macOS) or Ctrl+Shift+V (Windows). You should see a formatted table with 4 rows of detection data.

Table not rendering correctly in preview?

Make sure each row has the same number of pipe characters (|) as the header row. Markdown tables break if any row has a different column count.

Check that the separator row (the line with dashes) is directly below the header row with no blank line between them.

Help me fix my markdown table formatting.

Document the Structured Output schema and Rule Format

Anyone consuming your alerts.jsonl output (a SIEM, a log aggregator, a downstream script) needs to know the exact field names and types. A sample alert makes this concrete.

  • Below the Detection Coverage table, add a Structured Output section by pasting this markdown:
## Structured Output

The NIDS emits alerts as JSON-lines to `alerts.jsonl`. Each line contains:

| Field | Type | Description |
|-------|------|-------------|
| timestamp | string | ISO 8601 UTC timestamp |
| rule_id | string | Unique rule identifier (e.g., nids-001) |
| rule_name | string | Human-readable detection name |
| severity | string | Alert severity: high or critical |
| technique_id | string | MITRE ATT&CK technique ID |
| technique_name | string | ATT&CK technique display name |
| tactic | string | ATT&CK tactic category |
| src_ip | string | Source IP that triggered the alert |
| dst_ip | string | Destination IP (or N/A for broadcast) |
| evidence | object | Detection-specific details (port count, packet count, etc.) |

### Sample Alert

```json
{
  "timestamp": "2026-07-10T14:32:01.456789+00:00",
  "rule_id": "nids-001",
  "rule_name": "Network Service Discovery via Port Scan",
  "severity": "high",
  "technique_id": "T1046",
  "technique_name": "Network Service Discovery",
  "tactic": "discovery",
  "src_ip": "192.168.1.105",
  "dst_ip": "192.168.1.1",
  "evidence": {
    "unique_ports_scanned": 15,
    "timewindow_seconds": 10
  }
}
```

Why document the schema this way?

The field table tells a SIEM engineer what to expect without reading your source code. The sample alert gives them a concrete example to validate their parser against.

In Project 2, you will ingest this exact alerts.jsonl format into Elastic. Having the schema documented now means you can hand this file to a teammate and they can build dashboards without asking you questions.

  • Next, add a Rule Format section that explains the YAML schema conventions. Paste this below the Structured Output section:
## Rule Format

Detection rules use a Sigma-inspired YAML format adapted for network packet analysis. Each rule file in `rules/` must contain:

| Field | Required | Description |
|-------|----------|-------------|
| title | Yes | Human-readable detection name |
| id | Yes | Unique rule identifier (format: nids-NNN) |
| description | No | What the rule detects and why |
| detection | Yes | Condition configuration (see below) |
| level | Yes | Severity: informational, low, medium, high, critical |
| tags | Yes | ATT&CK mapping in attack.tXXXX format |

### Supported Condition Types

- **threshold**: Triggers when a count exceeds a limit within a time window. Fields: `field`, `count` or `unique_count`, `per`, `timewindow`.
- **arp_mismatch**: Triggers when a known IP claims a different MAC address. Fields: `track`, `alert_on`.

What does this section do?

The Rule Format section is your internal developer documentation. It tells anyone who wants to write a new rule exactly what fields are required and what condition types the engine supports.

  • Save README.md.
  • Preview the markdown again. You should see the schema table, a formatted JSON sample alert inside a code fence, and the Rule Format section with its condition types listed.

JSON sample alert not showing as a code block?

Make sure the triple backticks (```json) are on their own line with no leading spaces. The closing triple backticks also need their own line.

Help me debug my README markdown formatting.

Add the "How to Add a Detection" recipe

The whole point of a declarative rule engine is that adding a detection means adding a file, not editing code. This section makes that workflow explicit for future contributors.

  • Below the Rule Format section, add the contributor recipe by pasting this markdown:
## How to Add a Detection

Adding a new detection requires no code changes. Follow these 3 steps:

1. **Create a new `.yml` file** in the `rules/` directory. Use an existing rule as a template and assign the next sequential ID (e.g., `nids-005`).

2. **Choose a detection condition type** that matches your detection logic:
   - Use `threshold` for volume-based detections (packet floods, scans).
   - Use `arp_mismatch` for ARP-based detections.

3. **Restart the NIDS.** The rule engine loads all `.yml` files from `rules/` at startup. Your new rule is active immediately after restart.

Why is this recipe important?

This 3-step recipe is the payoff of the entire refactoring project. Before the rule engine, adding a detection meant writing Python code, testing it, and debugging edge cases.

Now it means creating one YAML file. A junior analyst who has never written Python can contribute detections. That is the power of declarative engineering.

  • Save README.md.
  • Preview the full README. You should see four distinct sections: Detection Coverage (the table), Structured Output (schema table plus sample JSON), Rule Format (field requirements and condition types), and How to Add a Detection (the 3-step numbered recipe).

Missing sections or broken formatting?

Confirm each section starts with a level-2 heading (## with a space after the hashes). Missing the space after ## causes markdown to render it as plain text.

Check that there are blank lines between each section heading and the content below it. Markdown requires a blank line before and after headings.

Help me troubleshoot my README sections.

✔️ Awesome, I've got everything!

Great. Double check you have saved README.md with all four sections.

ⓧ I'd like to double check the full code

Here is the complete README content you should have added in this step:

## Detection Coverage

| Rule ID | Detection | ATT&CK Technique | Technique Name | Tactic | Severity |
|---------|-----------|-------------------|----------------|--------|----------|
| nids-001 | Port Scan | T1046 | Network Service Discovery | Discovery | High |
| nids-002 | ARP Spoofing | T1557.002 | ARP Cache Poisoning | Credential Access | Critical |
| nids-003 | SYN Flood | T1498.001 | Direct Network Flood | Impact | High |
| nids-004 | ICMP Flood | T1498.001 | Direct Network Flood | Impact | High |

## Structured Output

The NIDS emits alerts as JSON-lines to `alerts.jsonl`. Each line contains:

| Field | Type | Description |
|-------|------|-------------|
| timestamp | string | ISO 8601 UTC timestamp |
| rule_id | string | Unique rule identifier (e.g., nids-001) |
| rule_name | string | Human-readable detection name |
| severity | string | Alert severity: high or critical |
| technique_id | string | MITRE ATT&CK technique ID |
| technique_name | string | ATT&CK technique display name |
| tactic | string | ATT&CK tactic category |
| src_ip | string | Source IP that triggered the alert |
| dst_ip | string | Destination IP (or N/A for broadcast) |
| evidence | object | Detection-specific details (port count, packet count, etc.) |

### Sample Alert

```json
{
  "timestamp": "2026-07-10T14:32:01.456789+00:00",
  "rule_id": "nids-001",
  "rule_name": "Network Service Discovery via Port Scan",
  "severity": "high",
  "technique_id": "T1046",
  "technique_name": "Network Service Discovery",
  "tactic": "discovery",
  "src_ip": "192.168.1.105",
  "dst_ip": "192.168.1.1",
  "evidence": {
    "unique_ports_scanned": 15,
    "timewindow_seconds": 10
  }
}
```

## Rule Format

Detection rules use a Sigma-inspired YAML format adapted for network packet analysis. Each rule file in `rules/` must contain:

| Field | Required | Description |
|-------|----------|-------------|
| title | Yes | Human-readable detection name |
| id | Yes | Unique rule identifier (format: nids-NNN) |
| description | No | What the rule detects and why |
| detection | Yes | Condition configuration (see below) |
| level | Yes | Severity: informational, low, medium, high, critical |
| tags | Yes | ATT&CK mapping in attack.tXXXX format |

### Supported Condition Types

- **threshold**: Triggers when a count exceeds a limit within a time window. Fields: `field`, `count` or `unique_count`, `per`, `timewindow`.
- **arp_mismatch**: Triggers when a known IP claims a different MAC address. Fields: `track`, `alert_on`.

## How to Add a Detection

Adding a new detection requires no code changes. Follow these 3 steps:

1. **Create a new `.yml` file** in the `rules/` directory. Use an existing rule as a template and assign the next sequential ID (e.g., `nids-005`).

2. **Choose a detection condition type** that matches your detection logic:
   - Use `threshold` for volume-based detections (packet floods, scans).
   - Use `arp_mismatch` for ARP-based detections.

3. **Restart the NIDS.** The rule engine loads all `.yml` files from `rules/` at startup. Your new rule is active immediately after restart.

Your NIDS is now fully documented with professional detection engineering standards. The coverage table, structured output schema, and contributor recipe make this a portfolio-ready project that speaks the same language as enterprise SOC teams. Next up, take on the Secret Mission to visualize your coverage on the ATT&CK Navigator matrix.

Secret mission

ATT&CK Navigator Layer Export

You have structured alerts and a coverage table. But what if you could visualize your detection coverage directly on the ATT&CK matrix? In this mission, you will generate a Navigator layer JSON from your rule set and load it into MITRE's web app to see exactly which techniques your NIDS covers.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources for the next project or delete them entirely.

Resources you created:

  • rules/ directory with 4 YAML detection rule files.
  • rule_engine.py module.
  • alert_emitter.py module.
  • alerts.jsonl structured output file.
  • generate_layer.py and coverage_layer.json (Secret Mission).
  • PyYAML 6.0.3 package installed in your virtual environment.

Keep everything running

No action needed. Choose this if you are continuing to Project 2 (SIEM integration with Elastic).

  • Your alerts.jsonl output is exactly what you will ingest into Elastic in the next project.
  • The rule engine, YAML rules, and ATT&CK Navigator layer are all ready to extend with new detections.
  • Your virtual environment stays active with PyYAML installed.

Pause - I'll come back to this later

Since everything is local, there is nothing to pause. Your files will stay on disk until you return.

  • Deactivate your virtual environment by running:
deactivate

When you come back, reactivate the virtual environment and you are ready to go.

Delete - I don't want to use this again

Remove all project files created during this refactor and uninstall the added dependency.

  • Remove the rule engine files and rules directory by running:
rm -rf rules/
rm rule_engine.py
rm alert_emitter.py
rm alerts.jsonl
rm generate_layer.py
rm coverage_layer.json
  • Uninstall PyYAML from your virtual environment by running:
pip uninstall pyyaml -y
  • If you also want to remove the README changes, revert README.md to its previous state using your version control, or delete the Detection Coverage, Structured Output, Rule Format, How to Add a Detection, and Coverage Heatmap sections manually.

Nice Work!

Nice Work!

You just transformed a hardcoded packet sniffer into a professional detection engineering framework. Your NIDS now speaks the same language as enterprise SOCs, with declarative YAML rules, MITRE ATT&CK mapping, and SIEM-ready structured output.

You've learned how to:

  • Map network detections to MITRE ATT&CK technique IDs (T1046, T1557.002, T1498.001) and document coverage in a professional README table.
  • Build a Sigma-inspired YAML rule engine where adding a detection means adding a file to the rules/ directory, not editing code.
  • Emit structured JSON-lines alerts with full ATT&CK context (technique ID, technique name, tactic, and severity) ready for SIEM ingestion.
  • Secret Mission: Generated an ATT&CK Navigator layer JSON and visualized your detection coverage heatmap on the ATT&CK matrix.

Ready to quiz yourself?