AI-Powered Network Intrusion Detection

Train a Random Forest model to classify network traffic and detect attacks.

Introduction

30 Second Summary

Every time you shop online, send a message, or stream a video, your data travels through networks that are constantly under attack. Security teams need systems that can automatically spot malicious traffic hiding among millions of normal connections.

In this project, you will build a machine learning-powered Network Intrusion Detection System that classifies network traffic as normal or malicious across four attack categories. You'll train a model on real-world data and create an interactive dashboard to visualize its results.

What You'll Build

A complete intrusion detection system using a Random Forest classifier trained on 125,973 labeled network connections, paired with an interactive Streamlit dashboard for visualization and live predictions.

By the end of this project, you'll have:

  • A trained Random Forest model that classifies network traffic into normal or 4 attack categories (DoS, Probe, R2L, U2R) with measurable accuracy.
  • A data preprocessing pipeline that handles 41 network features including both numeric and categorical data using scikit-learn.
  • An interactive Streamlit dashboard with attack distribution charts, confusion matrix, feature importance visualization, and a live prediction interface.
  • Secret Mission: Add an Isolation Forest anomaly detector to catch novel zero-day attacks the supervised model has never seen.

Are there any prerequisites?

You need basic Python knowledge (variables, loops, and functions). No prior machine learning or cybersecurity experience is required.

Set Up Your Python Environment

Before you can train a machine learning model to detect network intrusions, you need a working Python environment with the right packages installed.

In this step, you will verify your Python version, create an isolated project environment, and install the machine learning and visualization libraries your detection system needs.

In this step, get ready to:
  • Verify Python 3.11 or higher is installed.
  • Create a project folder and virtual environment.
  • Install the ML and visualization packages.
Verify your Python version

This project uses scikit-learn 1.9.0 and pandas 3.0.3, both of which require Python 3.11 or higher.

  • Press the Windows key to open your search bar.
  • Type Command Prompt and press Enter to open it.
  • Check your Python version by running this command:
python --version

✔️ I see version 3.11 or higher

Your Python version is compatible. Move on to the next substep.

ⓧ I see an older version

Your installed Python is too old for the packages this project uses. You need to install a newer version alongside it.

  • Go to python.org/downloads in your browser.
  • Click the large Download Python button at the top of the page.
  • Run the installer once it downloads.
  • On the first installer screen, check the box that says Add Python to PATH before clicking Install Now.
  • Close your Command Prompt and reopen it, then run python --version again to confirm the new version shows up.

Older version still showing?

Windows may still point to the old Python. Try running python3 --version instead. If the new version appears there, use python3 in place of python for the rest of this project.

ⓧ Command not found

Python is not installed on your system yet. Let's install it.

  • Go to python.org/downloads in your browser.
  • Click the large Download Python button at the top of the page.
  • Run the installer once it downloads.
  • On the first installer screen, check the box that says Add Python to PATH before clicking Install Now.
  • Close your Command Prompt and reopen it, then verify by running:
python --version

You should see a version number of 3.11 or higher.

Create your project folder and virtual environment

A virtual environment keeps your project's packages separate from the rest of your system. This prevents version conflicts between different Python projects.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir nids-project
cd nids-project

What is nids-project?

NIDS stands for Network Intrusion Detection System. This folder will hold all your project files including the ML model, dataset, and dashboard code.

  • Open this folder in VS Code by running:
code .

'code' is not recognized?

If code . does not work, open VS Code manually. Press the Windows key, type Visual Studio Code, and press Enter. Then click File, then Open Folder, and select the nids-project folder on your Desktop.

If VS Code is not installed, download it from code.visualstudio.com/Download and install it first.

help me open my project folder in VS Code

VS Code opens with your empty project folder in the sidebar.

  • Open a terminal inside VS Code by pressing Ctrl+` (backtick key, usually above Tab).
  • Create a virtual environment by running:
python -m venv venv

What does this command do?

This creates a folder called venv inside your project. It contains a self-contained copy of Python and a place to install packages without affecting your system-wide Python installation.

  • Activate the virtual environment by running:

Windows

venv\Scripts\activate

macOS

source venv/bin/activate

Linux

source venv/bin/activate

You should see (venv) appear at the start of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

On Windows, you may get a script execution error. Run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned in PowerShell, then try the activate command again.

help me activate my Python virtual environment

Install the required packages

Your detection system needs packages for machine learning, data processing, and building the dashboard.

  • Install all required packages by running:
pip install scikit-learn pandas numpy streamlit matplotlib seaborn joblib

What are these packages?

  • scikit-learn provides the machine learning algorithms (Random Forest, Isolation Forest) and data preprocessing tools.
  • pandas handles loading and manipulating the network traffic dataset.
  • numpy provides fast numerical operations used by the other packages.
  • streamlit builds the interactive dashboard with pure Python.
  • matplotlib and seaborn create charts and visualizations for the dashboard.
  • joblib saves and loads your trained model so you do not have to retrain every time.

The install may take a minute or two. You will see progress bars as each package downloads.

  • Verify the installation by running:
python -c "import sklearn; print(sklearn.__version__)"

You should see 1.9.0 (or a similar version number) printed in your terminal.

Seeing a ModuleNotFoundError?

Make sure your virtual environment is active. You should see (venv) at the start of your terminal prompt. If not, run the activate command from the previous substep, then run the pip install command again.

help me fix my Python package installation

Your Python environment is ready to go. Next up, you will download the NSL-KDD dataset and explore what real network traffic data looks like.

Download and Explore the NSL-KDD Dataset

Your environment is set up and ready to go. Now you need data to train your intrusion detection model.

Every machine learning system starts with a dataset. You will use the NSL-KDD dataset, a collection of 125,973 labeled network connections that researchers use to benchmark intrusion detection systems. Each record captures 41 features of a single network connection (how long it lasted, how many bytes moved, what protocol it used) plus a label telling you whether it was normal traffic or one of four attack types.

In this step, get ready to:
  • Download the NSL-KDD training and test files from a GitHub repository.
  • Create a Python script that loads and labels the data.
  • Run the script to explore how the dataset is structured.
Download the NSL-KDD dataset files

The NSL-KDD dataset is a cleaned-up version of the original KDD Cup 1999 dataset. It removes duplicate records so your model does not get biased toward repeated patterns. The original source at the University of New Brunswick is no longer available, so you will download the files from a GitHub mirror.

  • Create a data/ folder inside your nids-project directory by running this command in your VS Code terminal:
mkdir data
  • Download the training dataset into your data/ folder by running:
curl -L -o data/KDDTrain+.txt https://raw.githubusercontent.com/jmnwong/NSL-KDD-Dataset/master/KDDTrain%2B.txt
  • Download the test dataset by running:
curl -L -o data/KDDTest+.txt https://raw.githubusercontent.com/jmnwong/NSL-KDD-Dataset/master/KDDTest%2B.txt
  • Confirm the files exist by running:
ls data
  • You should see KDDTrain+.txt and KDDTest+.txt in the output.

What is NSL-KDD?

NSL-KDD is the standard benchmark dataset for network intrusion detection research. Each row represents a single network connection with 41 features describing its behavior (duration, bytes transferred, error rates, protocol type, and more).

Every connection is labeled with a specific attack name (like neptune or portsweep) or normal. You will map these specific names into 5 broader categories: normal, DoS, Probe, R2L, and U2R.

Download failing or curl not found?

If curl is not recognized, you can download the files manually. Open https://github.com/jmnwong/NSL-KDD-Dataset in your browser, click on KDDTrain+.txt, then click the download button. Do the same for KDDTest+.txt. Move both files into your data/ folder.

help me download the NSL-KDD dataset files into my project

Create the data loader script

The raw dataset files have no column headers and use specific attack names like neptune or buffer_overflow. You need a script that assigns proper column names, maps those specific attack names to the 5 broader categories your model will learn, and prints a summary so you can verify everything loaded correctly.

  • Create a new file called data_loader.py in the root of your nids-project folder (not inside data/).
  • Add the imports and column names by copying this code into data_loader.py:
import pandas as pd
import numpy as np

# NSL-KDD column names (41 features + label + difficulty)
COLUMN_NAMES = [
    'duration', 'protocol_type', 'service', 'flag', 'src_bytes',
    'dst_bytes', 'land', 'wrong_fragment', 'urgent', 'hot',
    'num_failed_logins', 'logged_in', 'num_compromised', 'root_shell',
    'su_attempted', 'num_root', 'num_file_creations', 'num_shells',
    'num_access_files', 'num_outbound_cmds', 'is_host_login',
    'is_guest_login', 'count', 'srv_count', 'serror_rate',
    'srv_serror_rate', 'rerror_rate', 'srv_rerror_rate', 'same_srv_rate',
    'diff_srv_rate', 'srv_diff_host_rate', 'dst_host_count',
    'dst_host_srv_count', 'dst_host_same_srv_rate', 'dst_host_diff_srv_rate',
    'dst_host_same_src_port_rate', 'dst_host_srv_diff_host_rate',
    'dst_host_serror_rate', 'dst_host_srv_serror_rate',
    'dst_host_rerror_rate', 'dst_host_srv_rerror_rate',
    'label', 'difficulty'
]

What does this code do?

The COLUMN_NAMES list defines all 43 columns in the raw dataset: 41 network features, plus the attack label and a difficulty score. The TXT files have no header row, so pandas needs these names to know which column is which.

  • Add the attack category mapping below the column names in data_loader.py:
# Attack type mapping to categories
ATTACK_CATEGORIES = {
    'normal': 'normal',
    'back': 'dos', 'land': 'dos', 'neptune': 'dos', 'pod': 'dos',
    'smurf': 'dos', 'teardrop': 'dos', 'mailbomb': 'dos',
    'apache2': 'dos', 'processtable': 'dos', 'udpstorm': 'dos',
    'ipsweep': 'probe', 'nmap': 'probe', 'portsweep': 'probe',
    'satan': 'probe', 'mscan': 'probe', 'saint': 'probe',
    'ftp_write': 'r2l', 'guess_passwd': 'r2l', 'imap': 'r2l',
    'multihop': 'r2l', 'phf': 'r2l', 'spy': 'r2l', 'warezclient': 'r2l',
    'warezmaster': 'r2l', 'sendmail': 'r2l', 'named': 'r2l',
    'snmpgetattack': 'r2l', 'snmpguess': 'r2l', 'xlock': 'r2l',
    'xsnoop': 'r2l', 'worm': 'r2l',
    'buffer_overflow': 'u2r', 'loadmodule': 'u2r', 'perl': 'u2r',
    'rootkit': 'u2r', 'httptunnel': 'u2r', 'ps': 'u2r',
    'sqlattack': 'u2r', 'xterm': 'u2r'
}

CATEGORICAL_FEATURES = ['protocol_type', 'service', 'flag']
NUMERIC_FEATURES = [col for col in COLUMN_NAMES[:41] if col not in CATEGORICAL_FEATURES]

What are the 5 attack categories?

The dictionary maps each specific attack name to one of 5 categories your model will learn to classify:

  • normal - legitimate, harmless traffic.
  • dos (Denial of Service) - flooding attacks that overwhelm a target.
  • probe - surveillance and scanning to find vulnerabilities.
  • r2l (Remote to Local) - unauthorized access from a remote machine.
  • u2r (User to Root) - privilege escalation from normal user to admin.

The CATEGORICAL_FEATURES list identifies the 3 text-based columns. The remaining 38 columns are numeric. This split matters later when you preprocess the data for training.

  • Add the loader function and the main block at the bottom of data_loader.py:
def load_nslkdd(filepath):
    """Load NSL-KDD dataset from TXT file."""
    df = pd.read_csv(filepath, names=COLUMN_NAMES, header=None)
    df['attack_category'] = df['label'].map(ATTACK_CATEGORIES).fillna('unknown')
    df = df[df['attack_category'] != 'unknown']
    df.drop(columns=['difficulty'], inplace=True)
    return df


if __name__ == '__main__':
    print("Loading NSL-KDD Training Data...")
    train_df = load_nslkdd('data/KDDTrain+.txt')

    print(f"\nDataset shape: {train_df.shape}")
    print(f"\nAttack category distribution:")
    print(train_df['attack_category'].value_counts())
    print(f"\nSample rows:")
    print(train_df.head())
    print(f"\nFeature types:")
    print(f"  Numeric features: {len(NUMERIC_FEATURES)}")
    print(f"  Categorical features: {len(CATEGORICAL_FEATURES)}")

What does load_nslkdd() do?

This function reads the CSV-formatted TXT file using the column names you defined (since the file has no header row). It creates a new attack_category column by mapping each specific attack name to one of the 5 categories. Any attack names not in the dictionary get filtered out as unknown. Finally, it drops the difficulty column since your model will not use it.

  • Save data_loader.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before running it in the next substep.

ⓧ I'd like to double check the full code

import pandas as pd
import numpy as np

# NSL-KDD column names (41 features + label + difficulty)
COLUMN_NAMES = [
    'duration', 'protocol_type', 'service', 'flag', 'src_bytes',
    'dst_bytes', 'land', 'wrong_fragment', 'urgent', 'hot',
    'num_failed_logins', 'logged_in', 'num_compromised', 'root_shell',
    'su_attempted', 'num_root', 'num_file_creations', 'num_shells',
    'num_access_files', 'num_outbound_cmds', 'is_host_login',
    'is_guest_login', 'count', 'srv_count', 'serror_rate',
    'srv_serror_rate', 'rerror_rate', 'srv_rerror_rate', 'same_srv_rate',
    'diff_srv_rate', 'srv_diff_host_rate', 'dst_host_count',
    'dst_host_srv_count', 'dst_host_same_srv_rate', 'dst_host_diff_srv_rate',
    'dst_host_same_src_port_rate', 'dst_host_srv_diff_host_rate',
    'dst_host_serror_rate', 'dst_host_srv_serror_rate',
    'dst_host_rerror_rate', 'dst_host_srv_rerror_rate',
    'label', 'difficulty'
]

# Attack type mapping to categories
ATTACK_CATEGORIES = {
    'normal': 'normal',
    'back': 'dos', 'land': 'dos', 'neptune': 'dos', 'pod': 'dos',
    'smurf': 'dos', 'teardrop': 'dos', 'mailbomb': 'dos',
    'apache2': 'dos', 'processtable': 'dos', 'udpstorm': 'dos',
    'ipsweep': 'probe', 'nmap': 'probe', 'portsweep': 'probe',
    'satan': 'probe', 'mscan': 'probe', 'saint': 'probe',
    'ftp_write': 'r2l', 'guess_passwd': 'r2l', 'imap': 'r2l',
    'multihop': 'r2l', 'phf': 'r2l', 'spy': 'r2l', 'warezclient': 'r2l',
    'warezmaster': 'r2l', 'sendmail': 'r2l', 'named': 'r2l',
    'snmpgetattack': 'r2l', 'snmpguess': 'r2l', 'xlock': 'r2l',
    'xsnoop': 'r2l', 'worm': 'r2l',
    'buffer_overflow': 'u2r', 'loadmodule': 'u2r', 'perl': 'u2r',
    'rootkit': 'u2r', 'httptunnel': 'u2r', 'ps': 'u2r',
    'sqlattack': 'u2r', 'xterm': 'u2r'
}

CATEGORICAL_FEATURES = ['protocol_type', 'service', 'flag']
NUMERIC_FEATURES = [col for col in COLUMN_NAMES[:41] if col not in CATEGORICAL_FEATURES]


def load_nslkdd(filepath):
    """Load NSL-KDD dataset from TXT file."""
    df = pd.read_csv(filepath, names=COLUMN_NAMES, header=None)
    df['attack_category'] = df['label'].map(ATTACK_CATEGORIES).fillna('unknown')
    df = df[df['attack_category'] != 'unknown']
    df.drop(columns=['difficulty'], inplace=True)
    return df


if __name__ == '__main__':
    print("Loading NSL-KDD Training Data...")
    train_df = load_nslkdd('data/KDDTrain+.txt')

    print(f"\nDataset shape: {train_df.shape}")
    print(f"\nAttack category distribution:")
    print(train_df['attack_category'].value_counts())
    print(f"\nSample rows:")
    print(train_df.head())
    print(f"\nFeature types:")
    print(f"  Numeric features: {len(NUMERIC_FEATURES)}")
    print(f"  Categorical features: {len(CATEGORICAL_FEATURES)}")
Run the data loader and explore the output

Now run your script to confirm the data loads correctly and explore how the dataset is structured.

  • Run the data loader script by executing this command in your terminal:
python data_loader.py
  • You should see output showing the dataset shape, attack category distribution, sample rows, and feature type counts.

What should I see?

The output tells you several important things about your data:

  • Dataset shape shows approximately 125,973 rows (network connections) and 42 columns (41 features + the new attack_category column).
  • Attack category distribution shows how many examples exist per class. You will notice normal and dos have thousands of examples, while r2l and u2r have far fewer. This imbalance affects detection accuracy later.
  • Feature types confirms 38 numeric features and 3 categorical features.

What do the 41 features represent?

Each network connection is described by features like duration (how long the connection lasted), src_bytes and dst_bytes (data transferred), protocol_type (TCP, UDP, or ICMP), service (which service was accessed), and flag (connection status).

There are also derived features like serror_rate (percentage of SYN errors), count (connections to the same host), and srv_count (connections to the same service). These traffic patterns are what your model will use to tell normal traffic apart from attacks.

Seeing a FileNotFoundError?

Make sure you are running the script from your nids-project folder (not from inside data/). The script looks for data/KDDTrain+.txt relative to where you run it. Check that both KDDTrain+.txt and KDDTest+.txt exist inside your data/ folder.

help me fix a FileNotFoundError when running data_loader.py

Your data is loaded and you understand its structure. Next up, you will preprocess these features and train a Random Forest model to classify each connection as normal or malicious.

Preprocess Data and Train the Random Forest Model

In the previous step, you explored the NSL-KDD dataset and saw its 41 features spread across 125,973 network connections.

Now you need to transform that raw data into something a machine learning model can learn from. Numeric features like duration and src_bytes need to be scaled so large values don't dominate. Categorical features like protocol_type need to be converted into numbers the model can process. In this step, you will build a preprocessing pipeline and train a Random Forest classifier that detects attacks across all 5 categories.

In this step, get ready to:
  • Build a preprocessing pipeline that handles mixed feature types.
  • Train a Random Forest model on the processed training data.
  • Save the trained model and pipeline to disk for your dashboard.
Create the preprocessing pipeline

The NSL-KDD dataset has a mix of feature types. A ColumnTransformer lets you apply different transformations to different columns and combine the results into a single feature matrix.

  • In VS Code, create a new file called train_model.py inside your nids-project folder.
  • Add the following imports and preprocessing function:
import pandas as pd
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, OneHotEncoder, LabelEncoder
from sklearn.compose import ColumnTransformer
from sklearn.metrics import classification_report, accuracy_score, confusion_matrix
import joblib
import os

from data_loader import load_nslkdd, CATEGORICAL_FEATURES, NUMERIC_FEATURES


def build_preprocessor():
    """Build a ColumnTransformer for mixed feature types."""
    preprocessor = ColumnTransformer(
        transformers=[
            ('num', StandardScaler(), NUMERIC_FEATURES),
            ('cat', OneHotEncoder(handle_unknown='ignore', sparse_output=False), CATEGORICAL_FEATURES)
        ]
    )
    return preprocessor

What does this code do?

The imports pull in everything you need from scikit-learn for preprocessing, training, and evaluation. The from data_loader import ... line reuses the constants and loader function you built in the previous step.

The build_preprocessor() function creates a ColumnTransformer with two parallel paths. StandardScaler normalizes the 38 numeric features so they all have the same scale. OneHotEncoder converts the 3 categorical features into binary columns the model can process.

  • Save train_model.py.
  • You won't see a change yet. The next chunk builds the training function that uses this pipeline.
Add the training function

The training function loads data, encodes labels as integers, splits into training and test sets, preprocesses features, trains the model, and prints evaluation metrics.

  • In train_model.py, add the following train() function below build_preprocessor():
def train():
    """Train the Random Forest model on NSL-KDD data."""
    print("Loading training data...")
    train_df = load_nslkdd('data/KDDTrain+.txt')

    feature_cols = NUMERIC_FEATURES + CATEGORICAL_FEATURES
    X = train_df[feature_cols]
    y = train_df['attack_category']

    # Encode labels
    label_encoder = LabelEncoder()
    y_encoded = label_encoder.fit_transform(y)

    # Split data
    X_train, X_test, y_train, y_test = train_test_split(
        X, y_encoded, test_size=0.2, random_state=42, stratify=y_encoded
    )

    # Preprocess
    print("Preprocessing features...")
    preprocessor = build_preprocessor()
    X_train_processed = preprocessor.fit_transform(X_train)
    X_test_processed = preprocessor.transform(X_test)

    # Train Random Forest
    print("Training Random Forest classifier...")
    rf_model = RandomForestClassifier(
        n_estimators=100,
        max_depth=20,
        min_samples_split=5,
        random_state=42,
        n_jobs=-1
    )
    rf_model.fit(X_train_processed, y_train)

    # Evaluate
    y_pred = rf_model.predict(X_test_processed)
    accuracy = accuracy_score(y_test, y_pred)
    print(f"\nModel Accuracy: {accuracy:.4f}")
    print("\nClassification Report:")
    print(classification_report(
        y_test, y_pred,
        target_names=label_encoder.classes_
    ))

What does this code do?

LabelEncoder converts the 5 text labels (normal, dos, probe, r2l, u2r) into integers the classifier can work with. train_test_split holds back 20% of the data for testing, with stratify ensuring each attack category is proportionally represented in both sets.

The preprocessor fits on training data only, then transforms both sets. This prevents data leakage where test data statistics influence the model.

The RandomForestClassifier uses 100 decision trees (n_estimators=100) that each learn different patterns. n_jobs=-1 uses all your CPU cores for faster training. The classification report shows precision, recall, and F1-score for each attack type.

  • Save train_model.py.
  • You won't see a change yet. The next chunk adds model saving and the entry point so you can run the script.
Save models to disk and run training

Your dashboard will need the trained model, preprocessor, and label encoder without retraining every time. Saving them with joblib lets you load them instantly later.

  • In train_model.py, add the following code directly below the classification_report print statement (still inside the train() function):
    # Save models
    os.makedirs('models', exist_ok=True)
    joblib.dump(rf_model, 'models/rf_model.joblib')
    joblib.dump(preprocessor, 'models/preprocessor.joblib')
    joblib.dump(label_encoder, 'models/label_encoder.joblib')

    # Save test data for dashboard
    cm = confusion_matrix(y_test, y_pred)
    joblib.dump(cm, 'models/confusion_matrix.joblib')
    joblib.dump((X_test, y_test, y_pred), 'models/test_results.joblib')

    print("\nModels saved to models/ directory.")
    return rf_model, preprocessor, label_encoder


if __name__ == '__main__':
    train()

What does this code do?

os.makedirs('models', exist_ok=True) creates the models/ directory if it does not already exist. joblib.dump() serializes each Python object to a file so it can be loaded later without retraining.

The confusion matrix and test results are saved separately so the dashboard can display performance metrics without re-running predictions. The if __name__ == '__main__' block means the training only runs when you execute this file directly.

  • Save train_model.py.

✔️ Awesome, I've got everything!

Great. Make sure you have saved train_model.py before running it.

ⓧ I'd like to double check the full code

import pandas as pd
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, OneHotEncoder, LabelEncoder
from sklearn.compose import ColumnTransformer
from sklearn.metrics import classification_report, accuracy_score, confusion_matrix
import joblib
import os

from data_loader import load_nslkdd, CATEGORICAL_FEATURES, NUMERIC_FEATURES


def build_preprocessor():
    """Build a ColumnTransformer for mixed feature types."""
    preprocessor = ColumnTransformer(
        transformers=[
            ('num', StandardScaler(), NUMERIC_FEATURES),
            ('cat', OneHotEncoder(handle_unknown='ignore', sparse_output=False), CATEGORICAL_FEATURES)
        ]
    )
    return preprocessor


def train():
    """Train the Random Forest model on NSL-KDD data."""
    print("Loading training data...")
    train_df = load_nslkdd('data/KDDTrain+.txt')

    feature_cols = NUMERIC_FEATURES + CATEGORICAL_FEATURES
    X = train_df[feature_cols]
    y = train_df['attack_category']

    # Encode labels
    label_encoder = LabelEncoder()
    y_encoded = label_encoder.fit_transform(y)

    # Split data
    X_train, X_test, y_train, y_test = train_test_split(
        X, y_encoded, test_size=0.2, random_state=42, stratify=y_encoded
    )

    # Preprocess
    print("Preprocessing features...")
    preprocessor = build_preprocessor()
    X_train_processed = preprocessor.fit_transform(X_train)
    X_test_processed = preprocessor.transform(X_test)

    # Train Random Forest
    print("Training Random Forest classifier...")
    rf_model = RandomForestClassifier(
        n_estimators=100,
        max_depth=20,
        min_samples_split=5,
        random_state=42,
        n_jobs=-1
    )
    rf_model.fit(X_train_processed, y_train)

    # Evaluate
    y_pred = rf_model.predict(X_test_processed)
    accuracy = accuracy_score(y_test, y_pred)
    print(f"\nModel Accuracy: {accuracy:.4f}")
    print("\nClassification Report:")
    print(classification_report(
        y_test, y_pred,
        target_names=label_encoder.classes_
    ))

    # Save models
    os.makedirs('models', exist_ok=True)
    joblib.dump(rf_model, 'models/rf_model.joblib')
    joblib.dump(preprocessor, 'models/preprocessor.joblib')
    joblib.dump(label_encoder, 'models/label_encoder.joblib')

    # Save test data for dashboard
    cm = confusion_matrix(y_test, y_pred)
    joblib.dump(cm, 'models/confusion_matrix.joblib')
    joblib.dump((X_test, y_test, y_pred), 'models/test_results.joblib')

    print("\nModels saved to models/ directory.")
    return rf_model, preprocessor, label_encoder


if __name__ == '__main__':
    train()
  • Run the training script in your terminal by running this command:
python train_model.py
  • Wait for training to complete (this takes 10-30 seconds depending on your machine).
  • You should see output showing "Loading training data...", "Preprocessing features...", "Training Random Forest classifier...", then a Model Accuracy score followed by a full classification report with precision, recall, and F1-score for each attack category.
  • The final line should say "Models saved to models/ directory."
  • Confirm the model files were saved by running:
ls models/
  • You should see rf_model.joblib, preprocessor.joblib, label_encoder.joblib, confusion_matrix.joblib, and test_results.joblib in the output.

Seeing an import error?

  • Make sure your virtual environment is activated. You should see (venv) at the start of your terminal prompt.
  • Check that data_loader.py is in the same folder as train_model.py. Both should be directly inside nids-project.
  • If you see a scikit-learn import error, re-run pip install scikit-learn with your venv active.

Help me debug my train_model.py import error

What should I see in the classification report?

The classification report shows how well the model detects each attack type. You will see high precision and recall for DoS and Probe attacks because the training set has many examples of those.

R2L and U2R attacks will have lower scores because the dataset contains far fewer examples of those categories. This imbalance is a real challenge in production intrusion detection systems.

Your model is trained and saved. Next up, you will build an interactive Streamlit dashboard to visualize these results and test the model with live predictions.

Build the Streamlit Dashboard

You have a trained model sitting in your models/ folder. Right now, the only way to interact with it is through Python scripts in your terminal.

In this step, you will build an interactive Streamlit dashboard that loads your saved model and lets you explore its performance visually. The dashboard will have four pages: an overview of attack categories, a model performance breakdown, a feature importance chart, and a live prediction interface.

In this step, get ready to:
  • Create the app skeleton with model loading and sidebar navigation.
  • Build the Model Performance and Feature Importance visualization pages.
  • Build a Live Prediction page with interactive controls.
Create the app skeleton and Overview page

Streamlit apps are plain Python scripts that Streamlit re-runs from top to bottom every time the user interacts with a widget. You will create a single app.py file that loads your saved model files once (using caching) and routes the user to different pages via a sidebar.

Why Streamlit instead of Flask?

Flask would require you to write HTML templates, CSS, and JavaScript to display charts and handle form inputs. Streamlit produces interactive dashboards with pure Python. Since this project focuses on machine learning, Streamlit lets you spend time on the data science rather than frontend code.

  • Create a new file called app.py in your nids-project folder, then paste the following code:
import streamlit as st
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import joblib

from data_loader import CATEGORICAL_FEATURES, NUMERIC_FEATURES

# Page config
st.set_page_config(
    page_title="AI Network Intrusion Detection System",
    page_icon="\ud83d\udd12",
    layout="wide"
)

# Load models
@st.cache_resource
def load_models():
    rf_model = joblib.load('models/rf_model.joblib')
    preprocessor = joblib.load('models/preprocessor.joblib')
    label_encoder = joblib.load('models/label_encoder.joblib')
    cm = joblib.load('models/confusion_matrix.joblib')
    test_results = joblib.load('models/test_results.joblib')
    return rf_model, preprocessor, label_encoder, cm, test_results


rf_model, preprocessor, label_encoder, cm, test_results = load_models()
X_test, y_test, y_pred = test_results

# Sidebar navigation
st.sidebar.title("Navigation")
page = st.sidebar.radio("Go to", ["Overview", "Model Performance", "Feature Importance", "Live Prediction"])

What does this code do?

  • st.set_page_config sets the browser tab title, icon, and uses a wide layout for charts.
  • @st.cache_resource tells Streamlit to load the model files only once and reuse them across reruns. Without caching, the app would reload all 5 joblib files every time you click a button.
  • st.sidebar.radio creates a radio button group in the sidebar that controls which page content is displayed.
  • Save app.py.

You won't see a change yet. The next chunk adds the Overview page that displays when the app launches.

  • Add the Overview page below the sidebar code in app.py:
if page == "Overview":
    st.title("AI-Powered Network Intrusion Detection System")
    st.markdown("""
    This system uses a **Random Forest** machine learning model trained on the
    **NSL-KDD** dataset to classify network traffic into 5 categories:
    """)

    col1, col2, col3, col4, col5 = st.columns(5)
    col1.metric("Normal", "Benign Traffic")
    col2.metric("DoS", "Denial of Service")
    col3.metric("Probe", "Surveillance")
    col4.metric("R2L", "Remote to Local")
    col5.metric("U2R", "User to Root")

    from sklearn.metrics import accuracy_score
    accuracy = accuracy_score(y_test, y_pred)
    st.metric("Overall Model Accuracy", f"{accuracy:.2%}")

    st.subheader("Attack Category Distribution (Training Data)")
    from data_loader import load_nslkdd
    train_df = load_nslkdd('data/KDDTrain+.txt')
    fig, ax = plt.subplots(figsize=(8, 4))
    train_df['attack_category'].value_counts().plot(kind='bar', ax=ax, color='steelblue')
    ax.set_xlabel("Attack Category")
    ax.set_ylabel("Count")
    ax.set_title("Distribution of Attack Types in Training Data")
    plt.tight_layout()
    st.pyplot(fig)

What does this code do?

  • st.columns(5) creates five columns to display each traffic category as a metric card.
  • accuracy_score(y_test, y_pred) computes accuracy from the test results you saved during training.
  • The bar chart loads the full training dataset and plots how many connections fall into each attack category. This gives SOC analysts a quick view of the threat landscape.
  • Save app.py.
  • Launch the dashboard by running this command in your terminal:
streamlit run app.py
  • Your browser should open automatically to http://localhost:8501. You should see the title, five category metric cards, an accuracy percentage, and a bar chart showing the attack distribution.

Dashboard not loading?

  • Make sure your virtual environment is activated. If you see a ModuleNotFoundError, your terminal is using the system Python instead of the venv where you installed the packages.
  • Check that you are running the command from inside the nids-project folder. Streamlit needs to find app.py, data_loader.py, and the models/ directory in the same location.
  • If port 8501 is already in use, stop the other process or run with a different port: streamlit run app.py --server.port 8502.

help me debug why my Streamlit app won't launch

Build the Model Performance and Feature Importance pages

The Overview page gives a high-level summary. Now you will add two analytical pages that let you drill into how well the model performs on each attack type and which network features drive its decisions.

  • In app.py, add the Model Performance page directly below the Overview if block:
elif page == "Model Performance":
    st.title("Model Performance")

    from sklearn.metrics import accuracy_score
    accuracy = accuracy_score(y_test, y_pred)
    st.metric("Accuracy", f"{accuracy:.2%}")

    st.subheader("Confusion Matrix")
    fig, ax = plt.subplots(figsize=(8, 6))
    sns.heatmap(
        cm, annot=True, fmt='d', cmap='Blues',
        xticklabels=label_encoder.classes_,
        yticklabels=label_encoder.classes_,
        ax=ax
    )
    ax.set_xlabel("Predicted")
    ax.set_ylabel("Actual")
    ax.set_title("Confusion Matrix")
    plt.tight_layout()
    st.pyplot(fig)

    st.subheader("Per-Class Metrics")
    from sklearn.metrics import classification_report
    report = classification_report(
        y_test, y_pred,
        target_names=label_encoder.classes_,
        output_dict=True
    )
    report_df = pd.DataFrame(report).transpose()
    st.dataframe(report_df.style.format("{:.3f}"))

What is a confusion matrix?

A confusion matrix shows where the model gets confused. Each row is the actual attack category, each column is what the model predicted. Diagonal cells are correct predictions. Off-diagonal cells reveal which attack types get misclassified as something else.

The per-class metrics table shows precision (how many flagged alerts are real), recall (how many real attacks are caught), and F1-score (the balance between both). These are the same metrics SOC teams use to evaluate detection tools.

  • Save app.py. In your browser, click **Model Performance** in the sidebar.
  • You should see an accuracy metric at the top, a blue heatmap grid showing the confusion matrix, and a table with precision, recall, and F1-score for each category.

Seeing a blank page when clicking Model Performance?

  • Check that the elif keyword starts at the same indentation level as the if page == "Overview": line above it. Python is sensitive to indentation.
  • If the page shows an error about cm being undefined, confirm that the load_models() function at the top of your file returns all 5 values and that you are unpacking them correctly.

help me fix my Model Performance page

  • Now add the Feature Importance page directly below the Model Performance elif block:
elif page == "Feature Importance":
    st.title("Feature Importance")
    st.markdown("Which network features matter most for detecting attacks?")

    # Get feature names after preprocessing
    cat_features = preprocessor.named_transformers_['cat'].get_feature_names_out(CATEGORICAL_FEATURES)
    all_features = np.concatenate([NUMERIC_FEATURES, cat_features])

    importances = rf_model.feature_importances_
    indices = np.argsort(importances)[-15:]

    fig, ax = plt.subplots(figsize=(10, 6))
    ax.barh(range(15), importances[indices], color='steelblue')
    ax.set_yticks(range(15))
    ax.set_yticklabels(all_features[indices])
    ax.set_xlabel("Importance Score")
    ax.set_title("Top 15 Most Important Features for Attack Detection")
    plt.tight_layout()
    st.pyplot(fig)

What does feature importance tell you?

Random Forest calculates how much each feature reduces prediction errors across all its decision trees. A high importance score means the model relies heavily on that feature to distinguish attack types from normal traffic.

For SOC analysts, this reveals which network indicators are most diagnostic. Features like src_bytes, dst_bytes, and service often rank high because attack traffic has distinctive byte patterns and targets specific services.

  • Save app.py. Click **Feature Importance** in the sidebar.
  • You should see a horizontal bar chart showing the top 15 features ranked by importance score.

Chart not showing all 15 bars?

  • Make sure the preprocessor.named_transformers_['cat'].get_feature_names_out(CATEGORICAL_FEATURES) line is copied exactly. A typo in named_transformers_ (note the trailing underscore) will cause an AttributeError.
  • Verify your models/preprocessor.joblib file was saved after training completed successfully in the previous step.

help me fix my Feature Importance page

Build the Live Prediction page

The final page turns your dashboard into a hands-on detection tool. It lets users adjust network connection parameters with sliders and dropdowns, then see the model's real-time classification with a confidence score.

  • Add the Live Prediction page below the Feature Importance elif block in app.py:
elif page == "Live Prediction":
    st.title("Live Traffic Prediction")
    st.markdown("Adjust the network connection parameters and click Predict.")

    col1, col2 = st.columns(2)

    with col1:
        duration = st.slider("Duration (seconds)", 0, 60000, 0)
        src_bytes = st.number_input("Source Bytes", 0, 1000000000, 0)
        dst_bytes = st.number_input("Destination Bytes", 0, 1000000000, 0)
        count = st.slider("Count (connections to same host)", 0, 511, 1)
        srv_count = st.slider("Srv Count (connections to same service)", 0, 511, 1)

    with col2:
        protocol_type = st.selectbox("Protocol Type", ['tcp', 'udp', 'icmp'])
        service = st.selectbox("Service", ['http', 'smtp', 'ftp', 'ftp_data', 'telnet',
                                           'private', 'domain_u', 'eco_i', 'other'])
        flag = st.selectbox("Flag", ['SF', 'S0', 'REJ', 'RSTR', 'SH', 'RSTO', 'S1', 'S2', 'OTH'])
        serror_rate = st.slider("SYN Error Rate", 0.0, 1.0, 0.0)
        same_srv_rate = st.slider("Same Service Rate", 0.0, 1.0, 1.0)

    if st.button("Predict", type="primary"):
        # Build feature vector with defaults for unset features
        input_data = {feat: 0 for feat in NUMERIC_FEATURES}
        input_data['duration'] = duration
        input_data['src_bytes'] = src_bytes
        input_data['dst_bytes'] = dst_bytes
        input_data['count'] = count
        input_data['srv_count'] = srv_count
        input_data['serror_rate'] = serror_rate
        input_data['same_srv_rate'] = same_srv_rate
        input_data['protocol_type'] = protocol_type
        input_data['service'] = service
        input_data['flag'] = flag

        feature_cols = NUMERIC_FEATURES + CATEGORICAL_FEATURES
        input_df = pd.DataFrame([input_data])[feature_cols]

        processed = preprocessor.transform(input_df)
        prediction = rf_model.predict(processed)[0]
        probabilities = rf_model.predict_proba(processed)[0]

        predicted_label = label_encoder.inverse_transform([prediction])[0]
        confidence = probabilities[prediction]

        if predicted_label == 'normal':
            st.success(f"Classification: **{predicted_label.upper()}** (Confidence: {confidence:.1%})")
        else:
            st.error(f"ALERT - Classification: **{predicted_label.upper()}** (Confidence: {confidence:.1%})")

        st.subheader("Confidence per Category")
        prob_df = pd.DataFrame({
            'Category': label_encoder.classes_,
            'Confidence': probabilities
        }).sort_values('Confidence', ascending=False)
        st.bar_chart(prob_df.set_index('Category'))

What does this code do?

  • The left column has sliders and number inputs for numeric features like duration, src_bytes, and count. The right column has dropdown menus for categorical features like protocol_type and service.
  • When the user clicks **Predict**, the code builds a feature dictionary with zeros for the 38 numeric features, then overwrites the ones the user adjusted. This ensures the model receives all 41 features it expects.
  • predict_proba returns the model's confidence for each category. A green success box appears for normal traffic. A red error box appears for detected attacks.
  • Save app.py. Click **Live Prediction** in the sidebar.
  • You should see two columns of input controls (sliders, number inputs, and dropdowns) plus a blue **Predict** button at the bottom.
  • Click **Predict** with the default values. You should see a green classification result showing NORMAL with a high confidence percentage, and a bar chart breaking down confidence per category.

Predict button not returning a result?

  • If you see an error about column mismatch or shape, double-check that the feature_cols = NUMERIC_FEATURES + CATEGORICAL_FEATURES line uses the same order your model was trained with.
  • Make sure the input_df = pd.DataFrame([input_data])[feature_cols] line has the square brackets around input_data to create a single-row DataFrame.

help me debug my Live Prediction page

✔️ Awesome, I've got everything!

Great. Double check you have saved app.py and that all four sidebar pages load without errors.

ⓧ I'd like to double check the full code

Here is the complete app.py file. Compare your code against this:

import streamlit as st
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import joblib

from data_loader import CATEGORICAL_FEATURES, NUMERIC_FEATURES

# Page config
st.set_page_config(
    page_title="AI Network Intrusion Detection System",
    page_icon="\ud83d\udd12",
    layout="wide"
)

# Load models
@st.cache_resource
def load_models():
    rf_model = joblib.load('models/rf_model.joblib')
    preprocessor = joblib.load('models/preprocessor.joblib')
    label_encoder = joblib.load('models/label_encoder.joblib')
    cm = joblib.load('models/confusion_matrix.joblib')
    test_results = joblib.load('models/test_results.joblib')
    return rf_model, preprocessor, label_encoder, cm, test_results


rf_model, preprocessor, label_encoder, cm, test_results = load_models()
X_test, y_test, y_pred = test_results

# Sidebar navigation
st.sidebar.title("Navigation")
page = st.sidebar.radio("Go to", ["Overview", "Model Performance", "Feature Importance", "Live Prediction"])

if page == "Overview":
    st.title("AI-Powered Network Intrusion Detection System")
    st.markdown("""
    This system uses a **Random Forest** machine learning model trained on the
    **NSL-KDD** dataset to classify network traffic into 5 categories:
    """)

    col1, col2, col3, col4, col5 = st.columns(5)
    col1.metric("Normal", "Benign Traffic")
    col2.metric("DoS", "Denial of Service")
    col3.metric("Probe", "Surveillance")
    col4.metric("R2L", "Remote to Local")
    col5.metric("U2R", "User to Root")

    from sklearn.metrics import accuracy_score
    accuracy = accuracy_score(y_test, y_pred)
    st.metric("Overall Model Accuracy", f"{accuracy:.2%}")

    st.subheader("Attack Category Distribution (Training Data)")
    from data_loader import load_nslkdd
    train_df = load_nslkdd('data/KDDTrain+.txt')
    fig, ax = plt.subplots(figsize=(8, 4))
    train_df['attack_category'].value_counts().plot(kind='bar', ax=ax, color='steelblue')
    ax.set_xlabel("Attack Category")
    ax.set_ylabel("Count")
    ax.set_title("Distribution of Attack Types in Training Data")
    plt.tight_layout()
    st.pyplot(fig)

elif page == "Model Performance":
    st.title("Model Performance")

    from sklearn.metrics import accuracy_score
    accuracy = accuracy_score(y_test, y_pred)
    st.metric("Accuracy", f"{accuracy:.2%}")

    st.subheader("Confusion Matrix")
    fig, ax = plt.subplots(figsize=(8, 6))
    sns.heatmap(
        cm, annot=True, fmt='d', cmap='Blues',
        xticklabels=label_encoder.classes_,
        yticklabels=label_encoder.classes_,
        ax=ax
    )
    ax.set_xlabel("Predicted")
    ax.set_ylabel("Actual")
    ax.set_title("Confusion Matrix")
    plt.tight_layout()
    st.pyplot(fig)

    st.subheader("Per-Class Metrics")
    from sklearn.metrics import classification_report
    report = classification_report(
        y_test, y_pred,
        target_names=label_encoder.classes_,
        output_dict=True
    )
    report_df = pd.DataFrame(report).transpose()
    st.dataframe(report_df.style.format("{:.3f}"))

elif page == "Feature Importance":
    st.title("Feature Importance")
    st.markdown("Which network features matter most for detecting attacks?")

    # Get feature names after preprocessing
    cat_features = preprocessor.named_transformers_['cat'].get_feature_names_out(CATEGORICAL_FEATURES)
    all_features = np.concatenate([NUMERIC_FEATURES, cat_features])

    importances = rf_model.feature_importances_
    indices = np.argsort(importances)[-15:]

    fig, ax = plt.subplots(figsize=(10, 6))
    ax.barh(range(15), importances[indices], color='steelblue')
    ax.set_yticks(range(15))
    ax.set_yticklabels(all_features[indices])
    ax.set_xlabel("Importance Score")
    ax.set_title("Top 15 Most Important Features for Attack Detection")
    plt.tight_layout()
    st.pyplot(fig)

elif page == "Live Prediction":
    st.title("Live Traffic Prediction")
    st.markdown("Adjust the network connection parameters and click Predict.")

    col1, col2 = st.columns(2)

    with col1:
        duration = st.slider("Duration (seconds)", 0, 60000, 0)
        src_bytes = st.number_input("Source Bytes", 0, 1000000000, 0)
        dst_bytes = st.number_input("Destination Bytes", 0, 1000000000, 0)
        count = st.slider("Count (connections to same host)", 0, 511, 1)
        srv_count = st.slider("Srv Count (connections to same service)", 0, 511, 1)

    with col2:
        protocol_type = st.selectbox("Protocol Type", ['tcp', 'udp', 'icmp'])
        service = st.selectbox("Service", ['http', 'smtp', 'ftp', 'ftp_data', 'telnet',
                                           'private', 'domain_u', 'eco_i', 'other'])
        flag = st.selectbox("Flag", ['SF', 'S0', 'REJ', 'RSTR', 'SH', 'RSTO', 'S1', 'S2', 'OTH'])
        serror_rate = st.slider("SYN Error Rate", 0.0, 1.0, 0.0)
        same_srv_rate = st.slider("Same Service Rate", 0.0, 1.0, 1.0)

    if st.button("Predict", type="primary"):
        # Build feature vector with defaults for unset features
        input_data = {feat: 0 for feat in NUMERIC_FEATURES}
        input_data['duration'] = duration
        input_data['src_bytes'] = src_bytes
        input_data['dst_bytes'] = dst_bytes
        input_data['count'] = count
        input_data['srv_count'] = srv_count
        input_data['serror_rate'] = serror_rate
        input_data['same_srv_rate'] = same_srv_rate
        input_data['protocol_type'] = protocol_type
        input_data['service'] = service
        input_data['flag'] = flag

        feature_cols = NUMERIC_FEATURES + CATEGORICAL_FEATURES
        input_df = pd.DataFrame([input_data])[feature_cols]

        processed = preprocessor.transform(input_df)
        prediction = rf_model.predict(processed)[0]
        probabilities = rf_model.predict_proba(processed)[0]

        predicted_label = label_encoder.inverse_transform([prediction])[0]
        confidence = probabilities[prediction]

        if predicted_label == 'normal':
            st.success(f"Classification: **{predicted_label.upper()}** (Confidence: {confidence:.1%})")
        else:
            st.error(f"ALERT - Classification: **{predicted_label.upper()}** (Confidence: {confidence:.1%})")

        st.subheader("Confidence per Category")
        prob_df = pd.DataFrame({
            'Category': label_encoder.classes_,
            'Confidence': probabilities
        }).sort_values('Confidence', ascending=False)
        st.bar_chart(prob_df.set_index('Category'))

Your dashboard is live. Next up, you will run thorough tests on the Live Prediction page by simulating real attack patterns and verifying the model catches them.

Test and Validate Your Detection System

You have a trained model, a preprocessing pipeline, and a full dashboard ready to go. Now it is time to launch the app and see if your detection system actually works.

In this step, you will run your Streamlit dashboard, verify every page loads correctly, and stress-test the Live Prediction page by simulating real attack patterns.

In this step, get ready to:
  • Launch the Streamlit dashboard and verify all four pages load.
  • Test the Live Prediction page with simulated DoS attack and normal traffic values.
  • Record your model's accuracy, top features, and hardest-to-detect attack types.
Launch the Streamlit app

Your dashboard code is complete and the trained model files are sitting in the models/ directory. Streamlit serves your app as a local web page you can interact with in the browser.

  • Make sure your virtual environment is still active (you should see (venv) in your terminal prompt).
  • Launch the dashboard by running this command:
streamlit run app.py

What does this command do?

Streamlit starts a local web server and opens your default browser to the app (usually at http://localhost:8501). The terminal stays occupied running the server. You interact with the app in the browser, not the terminal.

  • Your browser should open automatically. If it does not, copy the URL shown in the terminal output and paste it into your browser.
  • You should see the Overview page with the title "AI-Powered Network Intrusion Detection System", five category metric cards, the overall model accuracy percentage, and a bar chart showing attack category distribution.
  • In the left sidebar, click Model Performance.
  • Verify you see the accuracy metric, a blue confusion matrix heatmap, and a per-class metrics table below it.
  • Click Feature Importance in the sidebar.
  • Verify you see a horizontal bar chart showing the top 15 most important features for attack detection.
  • Click Live Prediction in the sidebar.
  • Verify you see sliders and dropdowns on the left and right columns, plus a purple Predict button.

App not loading?

If the browser shows an error or the page is blank, check your terminal for a traceback. Common causes:

  • Make sure you are running the command from inside the nids-project folder (where app.py lives).
  • Confirm the models/ directory contains all five .joblib files. If not, re-run python train_model.py first.
  • Check that your virtual environment is active. If you see a ModuleNotFoundError, the packages are installed in the venv but you are running outside it.

Help me troubleshoot why my Streamlit app is not loading. I ran streamlit run app.py and see an error in the terminal.

Test live predictions with attack patterns

A detection system is only useful if it correctly identifies threats. You will simulate a DoS attack pattern and verify the model flags it, then test with normal traffic to confirm it does not over-alert.

  • Make sure you are on the Live Prediction page in the sidebar.
  • Set the following values to simulate a DoS attack:
  • Drag Count (connections to same host) to 400.
  • Drag Srv Count (connections to same service) to 400.
  • Select tcp for Protocol Type.
  • Drag SYN Error Rate to 1.0.
  • Click the Predict button.

You should see a red alert banner classifying this traffic as DOS with a high confidence percentage. A bar chart below shows confidence scores across all five categories.

Why these values simulate a DoS attack

A Denial of Service attack floods a target with connections. High count and srv_count values mean hundreds of connections hit the same host and service in a short window.

A SYN error rate of 1.0 means every connection attempt fails to complete the TCP handshake. This is the signature of a SYN flood attack.

  • Now test with normal traffic values. Reset the sliders:
  • Drag Count back to 1.
  • Drag Srv Count back to 1.
  • Drag SYN Error Rate back to 0.0.
  • Drag Same Service Rate to 1.0.
  • Select SF for Flag (means the connection completed successfully).
  • Click Predict again.

You should see a green success banner classifying this traffic as NORMAL with high confidence.

Model classifying everything the same?

If every prediction returns the same category regardless of input values, your model file may be corrupted or out of sync with the preprocessor.

  • Stop the Streamlit server by pressing Ctrl+C in the terminal.
  • Re-run python train_model.py to regenerate all model files.
  • Relaunch with streamlit run app.py and try again.

Help me debug why my model gives the same prediction regardless of input values.

Record your model's performance insights

A detection system in a real SOC needs documented performance baselines. Take a moment to record what your model can and cannot do.

  • Click Overview in the sidebar and note the Overall Model Accuracy percentage.
  • Click Feature Importance and note the top 3 features in the bar chart. These are the network attributes your model relies on most heavily for classification.
  • Click Model Performance and look at the per-class metrics table.
  • Find the rows for r2l and u2r. Notice that their F1-scores are lower than the other categories.

Why are R2L and U2R harder to detect?

R2L (Remote to Local) and U2R (User to Root) attacks have far fewer training samples than DoS or Probe. The model has seen thousands of DoS examples but only dozens of U2R examples.

This is a real-world challenge in cybersecurity ML. Rare but dangerous attacks are the hardest to catch because the model has so little data to learn from. Production systems address this with oversampling, cost-sensitive learning, or anomaly detection (which you will explore in the Secret Mission).

  • Take a screenshot of the Overview page showing the accuracy metric and attack distribution chart. This is your portfolio-ready proof that you built a working IDS.

Your detection system is live and validated. You have confirmed it correctly classifies both attack traffic and normal traffic, and you understand its strengths and limitations. Next up, a Secret Mission to add zero-day detection for the novel attacks your supervised model has never seen.

Secret mission

Add Zero-Day Detection with Isolation Forest

Your Random Forest is great at catching known attack patterns. But what about brand-new attacks it has never seen before? Real Security Operations Centers use a second model trained only on normal traffic to flag anything that looks unusual. Add an Isolation Forest anomaly detector to catch novel zero-day attacks.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally, so there are no ongoing costs.

Resources you used:

  • The nids-project folder containing your Python scripts, trained models, dataset, and virtual environment.
  • The Streamlit development server (if still running in your terminal).

Keep everything running

No action needed. Choose this if you are still actively building or want to experiment with the dashboard further.

  • Your nids-project folder stays on your Desktop with all scripts, models, and data intact.
  • Your virtual environment (venv) and all installed packages remain ready to use.
  • Python and VS Code remain installed on your system for future projects.

Pause - I'll come back to this later

Shut down the running Streamlit process to free up memory. Your files stay on disk so you can pick up where you left off.

  • Switch to the terminal where Streamlit is running and press Ctrl+C to stop the server.
  • Close VS Code. Your project files remain saved in the nids-project folder.
  • When you return, reactivate your virtual environment with venv\Scripts\activate and run streamlit run app.py to restart the dashboard.

Delete - I don't want to use this again

Remove all project files, trained models, and the virtual environment. This frees up disk space and leaves no project artifacts behind.

  • If Streamlit is still running, switch to that terminal and press Ctrl+C to stop it.
  • Close VS Code.
  • Open a new terminal and navigate to the folder containing your project by running:
cd ~/Desktop
  • Delete the entire project folder. On Windows (Command Prompt), run:
rmdir /s /q nids-project
  • On macOS or Linux, run:
rm -rf nids-project
  • Confirm the folder is gone by running:
ls
  • You should no longer see nids-project in the output. All Python scripts, trained models, the NSL-KDD dataset, and the virtual environment are now removed.

Nice Work!

Nice Work!

You just built a complete AI-powered Network Intrusion Detection System from scratch. Your system classifies network traffic across 5 categories using a trained Random Forest model and displays results in an interactive dashboard.

You've learned how to:

  • Trained a Random Forest classifier on 125,973 labeled network connections from the NSL-KDD dataset, achieving high accuracy across normal traffic and 4 attack types (DoS, Probe, R2L, U2R).
  • Built a data preprocessing pipeline using scikit-learn ColumnTransformer to handle mixed feature types, encoding categorical features with OneHotEncoder and scaling numeric features with StandardScaler.
  • Created an interactive Streamlit dashboard with a confusion matrix heatmap, feature importance chart, and a live prediction interface for classifying network traffic in real time.
  • Secret Mission: Added zero-day detection capability using an Isolation Forest anomaly detector that catches novel attacks the supervised model has never seen.

Ready to quiz yourself?