Website Response Time Anomaly Detector

Use Isolation Forest ML to detect unusual website response time patterns.

Introduction

30 Second Summary

Every website slows down sometimes. The tricky part is knowing whether a slowdown is normal traffic or a sign that something is breaking. Production monitoring tools like Datadog and AWS CloudWatch solve this with machine learning, automatically learning what "normal" looks like and alerting engineers when behavior drifts outside that pattern.

In this project, you will build a Python-based anomaly detector that monitors website response times and uses the Isolation Forest algorithm to flag unusual patterns automatically.

What You'll Build

A complete ML-powered monitoring pipeline that collects response times, engineers time-series features, trains an unsupervised model, and visualizes detected anomalies in a two-panel chart.

By the end of this project, you'll have:

  • A working anomaly detection script that flags unusual website response times using Isolation Forest.
  • A professional two-panel visualization showing detected anomalies over a simulated 24-hour monitoring period.
  • A reusable monitoring pipeline (collect, engineer features, train, detect, visualize) applicable to any time-series data.
  • Secret Mission: Extend the batch detector into a real-time continuous monitoring loop with live alerting.

Are there any prerequisites?

This project assumes intermediate Python experience. You should be comfortable writing functions, installing packages with pip, and working with libraries. Familiarity with pandas DataFrames is helpful but not required.

Before We Start

You are about to build a tool that uses machine learning to spot unusual website response times automatically. Before writing any detection logic, you need the right Python environment in place.

In this step, you will verify that Python 3.11 or newer is installed, create an isolated project folder, and install the five libraries your anomaly detector depends on.

In this step, get ready to:
  • Verify Python 3.11+ is installed on your machine.
  • Create a project folder with a virtual environment.
  • Install the required Python packages.
Verify your Python version

The ML libraries in this project require Python 3.11 or newer. Let's confirm you have the right version before going further.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Visual Studio Code and press Enter to open it.
  • Open the built-in terminal by pressing Ctrl+` (the backtick key above Tab).
  • Check your Python version by running this command:

macOS

python3 --version

Windows

python --version

✔️ I see version 3.11 or higher

You should see output like Python 3.11.x or Python 3.12.x (or any version above 3.11). You are all set to continue.

ⓧ I see an older version

If you see Python 3.10 or lower, you need a newer version. On macOS, the system Python is often outdated even if you installed a newer one separately.

  • Download Python 3.11+ from https://www.python.org/downloads/.
  • Run the installer. On Windows, check Add Python to PATH on the first screen before clicking Install Now.
  • Close and reopen VS Code so the terminal picks up the new Python.
  • Run the version command again to confirm it now shows 3.11 or higher.

macOS still showing the old version?

After installing from python.org, macOS adds the new binary as python3.11 or python3.12. Try running the specific version command (e.g. python3.12 --version) to confirm it is installed. Use that versioned command for the rest of this project wherever you see python3.

ⓧ Command not found

Python is not installed on your machine yet.

  • Go to https://www.python.org/downloads/ and download the latest Python 3.11+ installer for your operating system.
  • Run the installer. On Windows, check Add Python to PATH on the first screen before clicking Install Now.
  • Close and reopen VS Code so the terminal picks up the new installation.
  • Run the version command again to confirm you see 3.11 or higher.
Create your project folder and virtual environment

A virtual environment gives your project its own isolated set of packages. This keeps your system Python clean and avoids version conflicts between projects.

  • Move to your Desktop and create the project folder by running these commands:

macOS

cd ~/Desktop
mkdir response-monitor
cd response-monitor

Windows

cd ~/Desktop
mkdir response-monitor
cd response-monitor
  • Create and activate a virtual environment by running:

macOS

python3 -m venv venv
source venv/bin/activate

Windows

python -m venv venv
venv\Scripts\activate

How do I know the virtual environment is active?

When the virtual environment is active, your terminal prompt shows (venv) at the beginning of the line. This means all pip install commands will install packages into this project only.

  • Confirm you are inside the response-monitor folder and the environment is active by checking for (venv) in your prompt.

Not seeing (venv) in your prompt?

  • On Windows, if you see an error about running scripts being disabled, run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned and then try the activate command again.
  • Make sure you ran the activate command from inside the response-monitor folder where the venv directory was created.

Why isn't my virtual environment activating?

Install the required packages

Your anomaly detector uses five libraries for HTTP requests, data manipulation, numerical computation, machine learning, and visualization.

  • Install all five packages by running this command:
pip install requests==2.34.2 pandas==3.0.3 numpy==2.4.6 scikit-learn==1.9.0 matplotlib==3.10.9

What are these packages?

  • requests makes HTTP calls to websites and measures response times.
  • pandas organizes data into tables (DataFrames) for easy manipulation.
  • numpy provides fast numerical operations and array math.
  • scikit-learn contains the Isolation Forest algorithm you will use for anomaly detection.
  • matplotlib creates charts and visualizations of your results.
  • Verify the installation succeeded by running:
python -c "import sklearn; print(sklearn.__version__)"

You should see 1.9.0 printed in the terminal.

Seeing an error during install?

  • Make sure (venv) is visible in your prompt. If not, reactivate the virtual environment before running pip install.
  • If you see a permissions error, you are likely installing to the system Python. Activate the venv first.
  • If a package fails to build, confirm your Python version is 3.11 or higher. These package versions require it.

Why is my pip install failing?

Your environment is ready and loaded with the libraries you need. Next up, you will build a data collector that measures real website response times.

Build the Data Collector

Before your anomaly detector can spot unusual patterns, it needs data to analyze. In this step, you will build the data collection layer of your monitoring pipeline.

You will write functions that measure real website response times, collect them into a structured dataset, and generate a realistic 24-hour historical record with injected anomalies for training.

In this step, get ready to:
  • Measure a single website response time with error handling.
  • Collect batches of response times into a pandas DataFrame.
  • Generate a synthetic 24-hour dataset with injected anomalies for model training.
Create monitor.py and measure a single response time

Every monitoring system starts by measuring one thing: how long a request takes. You will create the project file and write a function that times a single HTTP request.

  • Create a new file called monitor.py in your response-monitor folder by running:
touch monitor.py
  • Confirm the file exists by running:
ls

You should see monitor.py listed alongside the venv folder.

  • Open monitor.py in VS Code by clicking on it in the Explorer sidebar.
  • Add the import statements your project needs by pasting this code at the top of the file:
import requests
import pandas as pd
import numpy as np
from sklearn.ensemble import IsolationForest
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
from datetime import datetime, timedelta
import time

What do these imports do?

  • requests makes HTTP calls to websites so you can measure response times.
  • pandas organizes your data into tables (DataFrames) for easy manipulation.
  • numpy handles numerical operations like generating random noise and sine waves.
  • IsolationForest and StandardScaler are imported now but used in a later step for the ML model.
  • matplotlib creates the final visualization chart.

Now add the first data collection function below the imports. This function measures how long a single HTTP request takes to complete.

  • Add the following code below the imports in monitor.py:


# --- Data Collection ---

def collect_response_time(url, timeout=10):
    """Measure response time for a single request to a URL."""
    try:
        start = time.time()
        response = requests.get(url, timeout=timeout)
        elapsed = time.time() - start
        return {
            "timestamp": datetime.now(),
            "url": url,
            "response_time": elapsed,
            "status_code": response.status_code,
        }
    except requests.exceptions.RequestException:
        return {
            "timestamp": datetime.now(),
            "url": url,
            "response_time": timeout,
            "status_code": 0,
        }

What does this function do?

The function records the time before and after an HTTP GET request using time.time(). The difference is the response time in seconds.

If the request fails (timeout, DNS error, connection refused), the except block returns the full timeout value with a status code of 0. This prevents your script from crashing on network errors.

  • Save monitor.py.
  • Test your function by running this command in the terminal:
python -c "from monitor import collect_response_time; print(collect_response_time('https://httpbin.org/get'))"

You should see a dictionary printed with a timestamp, url, response_time (a small decimal like 0.2-0.5), and status_code of 200.

Seeing a ModuleNotFoundError?

  • Make sure your virtual environment is activated. You should see (venv) at the start of your terminal prompt.
  • If it is not active, run venv\Scripts\activate (Windows) or source venv/bin/activate (macOS/Linux) and try again.
  • If you see a timeout or connection error instead, check your internet connection. The function is calling a live website.

Why is my function returning a timeout or connection error?

Collect response times in batches

A single measurement is noisy. To get a reliable baseline, you need to collect multiple samples across different URLs.

  • Add the collect_batch function below collect_response_time in monitor.py:


def collect_batch(urls, num_requests=5):
    """Collect multiple response time measurements."""
    results = []
    for _ in range(num_requests):
        for url in urls:
            result = collect_response_time(url)
            results.append(result)
            time.sleep(0.5)
    return pd.DataFrame(results)

What does this function do?

This function loops through a list of URLs, pinging each one num_requests times. It collects all results into a list, then converts them into a pandas DataFrame with columns for timestamp, URL, response time, and status code.

The time.sleep(0.5) pause between requests avoids overwhelming the target server. This is good practice when making repeated HTTP calls.

  • Save monitor.py.
  • Test the batch collector with a single request by running:
python -c "from monitor import collect_batch; print(collect_batch(['https://httpbin.org/get'], num_requests=1))"

You should see a single-row DataFrame printed with columns timestamp, url, response_time, and status_code.

Seeing a NameError for pd?

  • Check that import pandas as pd is at the top of your file.
  • Make sure you saved monitor.py after adding the imports.

Help me fix my collect_batch function

Generate a synthetic 24-hour dataset

Training a machine learning model requires lots of data. Collecting 24 hours of real measurements would take, well, 24 hours.

Instead, you will generate synthetic data that mimics realistic patterns: a daily cycle (websites are slower during peak hours), random noise, and a few extreme spikes that represent the anomalies your model needs to learn to catch.

  • Add the generate_historical_data function below collect_batch in monitor.py:


def generate_historical_data(base_response_time=0.2, hours=24, interval_seconds=60):
    """Generate synthetic historical response time data with realistic patterns."""
    num_points = int((hours * 3600) / interval_seconds)
    timestamps = [
        datetime.now() - timedelta(seconds=interval_seconds * (num_points - i))
        for i in range(num_points)
    ]

    np.random.seed(42)

    # Daily sinusoidal pattern
    hour_of_day = np.array([t.hour for t in timestamps])
    daily_pattern = 0.05 * np.sin(2 * np.pi * hour_of_day / 24)

    # Normal noise
    noise = np.random.normal(0, 0.03, num_points)

    response_times = base_response_time + daily_pattern + noise
    response_times = np.maximum(response_times, 0.01)

    # Inject anomalies (about 3% of data)
    anomaly_indices = np.random.choice(
        num_points, size=int(num_points * 0.03), replace=False
    )
    response_times[anomaly_indices] *= np.random.uniform(
        3, 8, size=len(anomaly_indices)
    )

    df = pd.DataFrame({"timestamp": timestamps, "response_time": response_times})

    return df, anomaly_indices

What does this function do?

This function creates 1440 fake data points (one per minute for 24 hours) that simulate real monitoring data.

  • A sinusoidal pattern adds a daily cycle. Response times rise and fall like real web traffic.
  • Random noise from np.random.normal makes each point slightly different, like real network jitter.
  • 3% of points get multiplied by 3-8x to create obvious spikes. These are the anomalies your model will learn to detect.
  • np.random.seed(42) makes the random data reproducible. You will get the same results every time you run it.

Why synthetic data instead of real data?

ML models need hundreds or thousands of data points to learn patterns. Generating synthetic data lets you prototype the full pipeline in minutes rather than waiting a full day for real measurements.

This is standard practice when building monitoring systems. You validate the approach on synthetic data first, then swap in real collection once you know the pipeline works.

  • Save monitor.py.
  • Verify the data generator works by running:
python -c "from monitor import generate_historical_data; df, idx = generate_historical_data(); print(f'Data points: {df.shape[0]}, Anomalies injected: {len(idx)}')"

You should see: Data points: 1440, Anomalies injected: 43.

Seeing fewer data points or a different anomaly count?

  • Make sure you copied the function parameters exactly: hours=24 and interval_seconds=60 produce 1440 points.
  • The anomaly count of 43 comes from int(1440 * 0.03). If you see a different number, check that np.random.seed(42) appears before the anomaly injection.

Help me debug my generate_historical_data function

✔️ Awesome, I've got everything!

Great. Double check you have saved monitor.py with all three functions.

ⓧ I'd like to double check the full code

Here is the complete monitor.py so far:

import requests
import pandas as pd
import numpy as np
from sklearn.ensemble import IsolationForest
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
from datetime import datetime, timedelta
import time


# --- Data Collection ---

def collect_response_time(url, timeout=10):
    """Measure response time for a single request to a URL."""
    try:
        start = time.time()
        response = requests.get(url, timeout=timeout)
        elapsed = time.time() - start
        return {
            "timestamp": datetime.now(),
            "url": url,
            "response_time": elapsed,
            "status_code": response.status_code,
        }
    except requests.exceptions.RequestException:
        return {
            "timestamp": datetime.now(),
            "url": url,
            "response_time": timeout,
            "status_code": 0,
        }


def collect_batch(urls, num_requests=5):
    """Collect multiple response time measurements."""
    results = []
    for _ in range(num_requests):
        for url in urls:
            result = collect_response_time(url)
            results.append(result)
            time.sleep(0.5)
    return pd.DataFrame(results)


def generate_historical_data(base_response_time=0.2, hours=24, interval_seconds=60):
    """Generate synthetic historical response time data with realistic patterns."""
    num_points = int((hours * 3600) / interval_seconds)
    timestamps = [
        datetime.now() - timedelta(seconds=interval_seconds * (num_points - i))
        for i in range(num_points)
    ]

    np.random.seed(42)

    # Daily sinusoidal pattern
    hour_of_day = np.array([t.hour for t in timestamps])
    daily_pattern = 0.05 * np.sin(2 * np.pi * hour_of_day / 24)

    # Normal noise
    noise = np.random.normal(0, 0.03, num_points)

    response_times = base_response_time + daily_pattern + noise
    response_times = np.maximum(response_times, 0.01)

    # Inject anomalies (about 3% of data)
    anomaly_indices = np.random.choice(
        num_points, size=int(num_points * 0.03), replace=False
    )
    response_times[anomaly_indices] *= np.random.uniform(
        3, 8, size=len(anomaly_indices)
    )

    df = pd.DataFrame({"timestamp": timestamps, "response_time": response_times})

    return df, anomaly_indices

Your data collection layer is complete. Next up, you will engineer time-series features that give your ML model the temporal context it needs to spot anomalies.

Engineer Time-Series Features

Your data collector can generate 1,440 data points representing a full day of monitoring. The Isolation Forest model you'll train in the next step treats each row independently. It has no built-in awareness of time passing between measurements.

To give the model temporal context, you'll transform each raw response time into a richer row of statistics. Rolling averages, lag values, and rate of change all encode recent history into a single observation. This is called feature engineering, and it's what makes time-series anomaly detection work.

In this step, get ready to:
  • Define a feature columns list that keeps training and prediction consistent.
  • Write a feature engineering function that adds rolling statistics, lag values, and rate of change.
  • Test the feature pipeline on your historical dataset.
Define the feature columns list

Before writing the function that creates features, you'll declare exactly which columns the model should use. Keeping this in a single list ensures your training step and detection step always reference the same features.

  • In monitor.py, scroll to the bottom of the file (after the generate_historical_data function).
  • Add the following section comment and feature columns list:
# --- Feature Engineering ---

FEATURE_COLUMNS = [
    "response_time",
    "rolling_mean_5",
    "rolling_std_5",
    "rolling_mean_20",
    "rolling_std_20",
    "lag_1",
    "lag_5",
    "diff_1",
    "hour",
]

What does this list do?

FEATURE_COLUMNS is a module-level constant that names every column the model will train on. It includes the raw response_time plus eight engineered features you'll create next.

Defining it once prevents a common bug where training uses a different feature set than prediction. Both steps will reference this same list.

  • Save monitor.py.

You won't see a change yet. The next chunk uses this list inside the feature engineering function.

Write the feature engineering function

Each feature encodes a different slice of recent history. Rolling means capture trend, rolling standard deviations capture volatility, lag values capture what happened a moment ago, and the diff captures how fast things are changing.

  • In monitor.py, add the following function directly below the FEATURE_COLUMNS list:
def engineer_features(df):
    """Create time-series features for anomaly detection."""
    df = df.copy()
    df = df.sort_values("timestamp").reset_index(drop=True)

    # Rolling statistics
    df["rolling_mean_5"] = df["response_time"].rolling(window=5, min_periods=1).mean()
    df["rolling_std_5"] = (
        df["response_time"].rolling(window=5, min_periods=1).std().fillna(0)
    )
    df["rolling_mean_20"] = (
        df["response_time"].rolling(window=20, min_periods=1).mean()
    )
    df["rolling_std_20"] = (
        df["response_time"].rolling(window=20, min_periods=1).std().fillna(0)
    )

    # Lag features
    df["lag_1"] = df["response_time"].shift(1).fillna(df["response_time"].iloc[0])
    df["lag_5"] = df["response_time"].shift(5).fillna(df["response_time"].iloc[0])

    # Rate of change
    df["diff_1"] = df["response_time"].diff().fillna(0)

    # Hour of day
    df["hour"] = pd.to_datetime(df["timestamp"]).dt.hour

    return df

What does this function do?

  • df.copy() avoids mutating the original DataFrame. This keeps your raw data intact for debugging.
  • Rolling statistics (window 5 and 20) capture short-term and medium-term trends. A spike in rolling_std_5 means recent response times are volatile.
  • Lag features (lag_1, lag_5) tell the model what happened 1 and 5 observations ago. A sudden jump from a normal lag value signals something changed.
  • diff_1 is the rate of change between consecutive points. Large positive or negative diffs indicate rapid spikes or drops.
  • hour captures the daily cycle. Some hours naturally have higher traffic and longer response times.

Why use fillna(0) after rolling std?

The first observation in a rolling window of size 5 has only one data point. You cannot compute a standard deviation from a single value, so pandas returns NaN. Filling with 0 tells the model "no volatility measured yet" rather than leaving a gap that would break training.

  • Save monitor.py.

You won't see visible output from saving alone. The next substep runs the function to confirm it works.

Seeing a red underline in your editor?

If VS Code highlights pd or np as unresolved, make sure your Python interpreter is set to the venv environment. Press Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows), type Python: Select Interpreter, and choose the one inside your response-monitor/venv folder.

help me fix unresolved import warnings in VS Code

Test the feature engineering pipeline

Running the function on your synthetic dataset confirms that all nine feature columns are populated and the DataFrame shape is correct.

  • In your terminal, test the feature engineering by running this command:
python -c "from monitor import generate_historical_data, engineer_features; df, _ = generate_historical_data(); featured = engineer_features(df); print(f'Shape: {featured.shape}'); print(featured.columns.tolist())"
  • You should see output showing a shape of (1440, 10) and a list of 10 column names including rolling_mean_5, lag_1, diff_1, and hour.

What do those numbers mean?

1,440 rows equals one data point per minute for 24 hours. The 10 columns are the original timestamp and response_time plus the 8 engineered features. Every row now carries enough context for the model to judge whether it looks normal.

Seeing a ModuleNotFoundError?

Make sure your terminal is inside the response-monitor folder and your virtual environment is active (you should see (venv) at the start of your prompt). If not, run venv\Scripts\activate (Windows) or source venv/bin/activate (macOS/Linux) to reactivate it.

help me fix my module import error

✔️ Awesome, I've got everything!

Great. Double check you've saved monitor.py with both the FEATURE_COLUMNS list and the engineer_features function.

ⓧ I'd like to double check the full code

Here's the complete monitor.py file up to this point:

import requests
import pandas as pd
import numpy as np
from sklearn.ensemble import IsolationForest
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
from datetime import datetime, timedelta
import time


# --- Data Collection ---

def collect_response_time(url, timeout=10):
    """Measure response time for a single request to a URL."""
    try:
        start = time.time()
        response = requests.get(url, timeout=timeout)
        elapsed = time.time() - start
        return {
            "timestamp": datetime.now(),
            "url": url,
            "response_time": elapsed,
            "status_code": response.status_code,
        }
    except requests.exceptions.RequestException:
        return {
            "timestamp": datetime.now(),
            "url": url,
            "response_time": timeout,
            "status_code": 0,
        }


def collect_batch(urls, num_requests=5):
    """Collect multiple response time measurements."""
    results = []
    for _ in range(num_requests):
        for url in urls:
            result = collect_response_time(url)
            results.append(result)
            time.sleep(0.5)
    return pd.DataFrame(results)


def generate_historical_data(base_response_time=0.2, hours=24, interval_seconds=60):
    """Generate synthetic historical response time data with realistic patterns."""
    num_points = int((hours * 3600) / interval_seconds)
    timestamps = [
        datetime.now() - timedelta(seconds=interval_seconds * (num_points - i))
        for i in range(num_points)
    ]

    np.random.seed(42)

    # Daily sinusoidal pattern
    hour_of_day = np.array([t.hour for t in timestamps])
    daily_pattern = 0.05 * np.sin(2 * np.pi * hour_of_day / 24)

    # Normal noise
    noise = np.random.normal(0, 0.03, num_points)

    response_times = base_response_time + daily_pattern + noise
    response_times = np.maximum(response_times, 0.01)

    # Inject anomalies (about 3% of data)
    anomaly_indices = np.random.choice(
        num_points, size=int(num_points * 0.03), replace=False
    )
    response_times[anomaly_indices] *= np.random.uniform(
        3, 8, size=len(anomaly_indices)
    )

    df = pd.DataFrame({"timestamp": timestamps, "response_time": response_times})

    return df, anomaly_indices


# --- Feature Engineering ---

FEATURE_COLUMNS = [
    "response_time",
    "rolling_mean_5",
    "rolling_std_5",
    "rolling_mean_20",
    "rolling_std_20",
    "lag_1",
    "lag_5",
    "diff_1",
    "hour",
]


def engineer_features(df):
    """Create time-series features for anomaly detection."""
    df = df.copy()
    df = df.sort_values("timestamp").reset_index(drop=True)

    # Rolling statistics
    df["rolling_mean_5"] = df["response_time"].rolling(window=5, min_periods=1).mean()
    df["rolling_std_5"] = (
        df["response_time"].rolling(window=5, min_periods=1).std().fillna(0)
    )
    df["rolling_mean_20"] = (
        df["response_time"].rolling(window=20, min_periods=1).mean()
    )
    df["rolling_std_20"] = (
        df["response_time"].rolling(window=20, min_periods=1).std().fillna(0)
    )

    # Lag features
    df["lag_1"] = df["response_time"].shift(1).fillna(df["response_time"].iloc[0])
    df["lag_5"] = df["response_time"].shift(5).fillna(df["response_time"].iloc[0])

    # Rate of change
    df["diff_1"] = df["response_time"].diff().fillna(0)

    # Hour of day
    df["hour"] = pd.to_datetime(df["timestamp"]).dt.hour

    return df

Your feature pipeline is ready. Next up, you'll train an Isolation Forest model on these features and use it to flag anomalies automatically.

Train the Model and Detect Anomalies

You have a feature-rich DataFrame that encodes temporal context into every data point. Now you need a model that can learn what "normal" looks like and flag anything that deviates.

In this step, you will train an Isolation Forest model on the engineered features, use it to classify each data point as normal or anomalous, and wire the full pipeline together so you can run it end-to-end.

In this step, get ready to:
  • Write a training function that scales features and fits an Isolation Forest model.
  • Write a detection function that labels data points as normal or anomalous.
  • Build a main block that runs the entire pipeline from data collection to anomaly summary.
Write the training function

The Isolation Forest algorithm works by randomly partitioning data with decision trees. Anomalies are easier to isolate, so they end up with shorter paths through the tree.

Before training, the features need to be scaled to the same range. Response times are measured in fractions of a second while hour-of-day ranges from 0 to 23. Without scaling, the larger-valued features would dominate the random splits.

  • In monitor.py, scroll down below the engineer_features function.
  • Add a new section comment and the train_detector function by copying the following code:
# --- Anomaly Detection ---


def train_detector(df, contamination=0.03):
    """Train an Isolation Forest model on engineered features."""
    features = df[FEATURE_COLUMNS]

    scaler = StandardScaler()
    features_scaled = scaler.fit_transform(features)

    model = IsolationForest(
        n_estimators=100,
        contamination=contamination,
        random_state=42,
    )
    model.fit(features_scaled)

    return model, scaler

What does this code do?

StandardScaler normalizes each feature column to have a mean of 0 and a standard deviation of 1. This prevents features with larger numeric ranges from biasing the random splits in the trees.

IsolationForest builds 100 random trees (n_estimators=100) that attempt to isolate each observation. The contamination=0.03 parameter tells the model to expect about 3% of data points to be anomalies, which matches the synthetic anomaly rate you injected earlier.

  • Save monitor.py.

Why return both the model and the scaler?

When you score new data later, it must be scaled using the same statistics (mean and standard deviation) that the model was trained on. Returning the scaler alongside the model keeps them paired for consistent predictions.

Write the detection function

The trained model can now classify any new data point. You need a function that scales incoming data with the fitted scaler and asks the model for predictions.

  • Below the train_detector function, add the detect_anomalies function:
def detect_anomalies(df, model, scaler):
    """Use trained model to detect anomalies in new data."""
    features = df[FEATURE_COLUMNS]
    features_scaled = scaler.transform(features)

    df = df.copy()
    df["anomaly"] = model.predict(features_scaled)
    df["anomaly_score"] = model.decision_function(features_scaled)
    df["is_anomaly"] = df["anomaly"] == -1

    return df

What does this code do?

model.predict() returns -1 for outliers and 1 for inliers. The function stores this in the anomaly column.

model.decision_function() returns a continuous anomaly score. Negative scores indicate outliers. Positive scores indicate inliers. The lower the score, the more anomalous the data point.

The is_anomaly boolean column makes it easy to filter anomalies later for visualization and reporting.

  • Save monitor.py.

Seeing a NameError?

If you see an error mentioning FEATURE_COLUMNS is not defined, make sure your new functions are placed below the FEATURE_COLUMNS list in the file. Python reads top to bottom, so the list must be defined before any function that references it.

help me fix a NameError in my anomaly detection code

Wire the full pipeline together

Your functions are ready. Now you need a main block that orchestrates the entire flow: collect live baseline data, generate a historical dataset, engineer features, train the model, and print a detection summary.

  • Below the detect_anomalies function, add the main execution block:
# --- Main ---

if __name__ == "__main__":
    print("=" * 60)
    print("  Website Response Time Anomaly Detector")
    print("=" * 60)

    # Collect live samples for baseline
    print("\n[1/4] Collecting live response time samples...")
    urls = [
        "https://httpbin.org/get",
        "https://jsonplaceholder.typicode.com/posts/1",
    ]
    live_sample = collect_batch(urls, num_requests=3)
    baseline = live_sample["response_time"].median()
    print(f"  Baseline response time: {baseline:.3f}s")

    # Generate historical dataset
    print("\n[2/4] Generating historical dataset (simulating 24h of monitoring)...")
    historical_df, injected_anomalies = generate_historical_data(
        base_response_time=baseline, hours=24, interval_seconds=60
    )
    print(f"  Generated {len(historical_df)} data points")
    print(f"  Injected {len(injected_anomalies)} synthetic anomalies")

    # Engineer features
    print("\n[3/4] Engineering time-series features...")
    featured_df = engineer_features(historical_df)
    print(f"  Features: {FEATURE_COLUMNS}")

    # Train model and detect anomalies
    print("\n[4/4] Training Isolation Forest model...")
    model, scaler = train_detector(featured_df, contamination=0.03)
    results_df = detect_anomalies(featured_df, model, scaler)

    detected = results_df["is_anomaly"].sum()
    print(
        f"  Anomalies detected: {detected}/{len(results_df)}"
        f" ({100 * detected / len(results_df):.1f}%)"
    )

    # Show top anomalies
    print("\n  Top 5 anomalies by score:")
    top_anomalies = results_df[results_df["is_anomaly"]].nsmallest(
        5, "anomaly_score"
    )
    for _, row in top_anomalies.iterrows():
        print(
            f"    {row['timestamp']:%Y-%m-%d %H:%M}"
            f" - Response: {row['response_time']:.3f}s"
            f" (score: {row['anomaly_score']:.3f})"
        )

What does this code do?

  • Step 1 pings two public APIs (httpbin.org and jsonplaceholder.typicode.com) to measure a real baseline response time.
  • Step 2 generates 1440 synthetic data points (one per minute for 24 hours) centered on that real baseline, with 3% injected anomalies.
  • Step 3 adds rolling statistics, lag values, and rate-of-change features to the dataset.
  • Step 4 trains the Isolation Forest, detects anomalies, and prints both the total count and the 5 most anomalous data points (sorted by lowest anomaly score).
  • Save monitor.py.
  • Run the full pipeline by executing this command in your terminal:
python monitor.py

The script takes about 30 seconds to run because it collects live HTTP samples before generating and analyzing data.

You should see output showing all four stages completing, ending with a summary like "Anomalies detected: 43/1440 (3.0%)" followed by the top 5 anomalies with timestamps, response times, and scores.

Script hangs on step 1?

The live collection step makes real HTTP requests to public APIs. If your network is slow or the APIs are temporarily unreachable, the requests may time out after 10 seconds each. Wait up to 2 minutes for the full batch to complete.

If the script never progresses past "Collecting live response time samples", check that your internet connection is active and that your firewall is not blocking outbound HTTP requests.

help me debug why my monitor.py script hangs during data collection

✔️ Awesome, I've got everything!

Great. Double check that you have saved monitor.py and that your terminal output shows all four pipeline stages completing successfully.

ⓧ I'd like to double check the full code

Here is the complete monitor.py file at the end of this step:

import requests
import pandas as pd
import numpy as np
from sklearn.ensemble import IsolationForest
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
from datetime import datetime, timedelta
import time


# --- Data Collection ---

def collect_response_time(url, timeout=10):
    """Measure response time for a single request to a URL."""
    try:
        start = time.time()
        response = requests.get(url, timeout=timeout)
        elapsed = time.time() - start
        return {
            "timestamp": datetime.now(),
            "url": url,
            "response_time": elapsed,
            "status_code": response.status_code,
        }
    except requests.exceptions.RequestException:
        return {
            "timestamp": datetime.now(),
            "url": url,
            "response_time": timeout,
            "status_code": 0,
        }


def collect_batch(urls, num_requests=5):
    """Collect multiple response time measurements."""
    results = []
    for _ in range(num_requests):
        for url in urls:
            result = collect_response_time(url)
            results.append(result)
            time.sleep(0.5)
    return pd.DataFrame(results)


def generate_historical_data(base_response_time=0.2, hours=24, interval_seconds=60):
    """Generate synthetic historical response time data with realistic patterns."""
    num_points = int((hours * 3600) / interval_seconds)
    timestamps = [
        datetime.now() - timedelta(seconds=interval_seconds * (num_points - i))
        for i in range(num_points)
    ]

    np.random.seed(42)

    # Daily sinusoidal pattern
    hour_of_day = np.array([t.hour for t in timestamps])
    daily_pattern = 0.05 * np.sin(2 * np.pi * hour_of_day / 24)

    # Normal noise
    noise = np.random.normal(0, 0.03, num_points)

    response_times = base_response_time + daily_pattern + noise
    response_times = np.maximum(response_times, 0.01)

    # Inject anomalies (about 3% of data)
    anomaly_indices = np.random.choice(
        num_points, size=int(num_points * 0.03), replace=False
    )
    response_times[anomaly_indices] *= np.random.uniform(
        3, 8, size=len(anomaly_indices)
    )

    df = pd.DataFrame({"timestamp": timestamps, "response_time": response_times})

    return df, anomaly_indices


# --- Feature Engineering ---

FEATURE_COLUMNS = [
    "response_time",
    "rolling_mean_5",
    "rolling_std_5",
    "rolling_mean_20",
    "rolling_std_20",
    "lag_1",
    "lag_5",
    "diff_1",
    "hour",
]


def engineer_features(df):
    """Create time-series features for anomaly detection."""
    df = df.copy()
    df = df.sort_values("timestamp").reset_index(drop=True)

    # Rolling statistics
    df["rolling_mean_5"] = df["response_time"].rolling(window=5, min_periods=1).mean()
    df["rolling_std_5"] = (
        df["response_time"].rolling(window=5, min_periods=1).std().fillna(0)
    )
    df["rolling_mean_20"] = (
        df["response_time"].rolling(window=20, min_periods=1).mean()
    )
    df["rolling_std_20"] = (
        df["response_time"].rolling(window=20, min_periods=1).std().fillna(0)
    )

    # Lag features
    df["lag_1"] = df["response_time"].shift(1).fillna(df["response_time"].iloc[0])
    df["lag_5"] = df["response_time"].shift(5).fillna(df["response_time"].iloc[0])

    # Rate of change
    df["diff_1"] = df["response_time"].diff().fillna(0)

    # Hour of day
    df["hour"] = pd.to_datetime(df["timestamp"]).dt.hour

    return df


# --- Anomaly Detection ---


def train_detector(df, contamination=0.03):
    """Train an Isolation Forest model on engineered features."""
    features = df[FEATURE_COLUMNS]

    scaler = StandardScaler()
    features_scaled = scaler.fit_transform(features)

    model = IsolationForest(
        n_estimators=100,
        contamination=contamination,
        random_state=42,
    )
    model.fit(features_scaled)

    return model, scaler


def detect_anomalies(df, model, scaler):
    """Use trained model to detect anomalies in new data."""
    features = df[FEATURE_COLUMNS]
    features_scaled = scaler.transform(features)

    df = df.copy()
    df["anomaly"] = model.predict(features_scaled)
    df["anomaly_score"] = model.decision_function(features_scaled)
    df["is_anomaly"] = df["anomaly"] == -1

    return df


# --- Main ---

if __name__ == "__main__":
    print("=" * 60)
    print("  Website Response Time Anomaly Detector")
    print("=" * 60)

    # Collect live samples for baseline
    print("\n[1/4] Collecting live response time samples...")
    urls = [
        "https://httpbin.org/get",
        "https://jsonplaceholder.typicode.com/posts/1",
    ]
    live_sample = collect_batch(urls, num_requests=3)
    baseline = live_sample["response_time"].median()
    print(f"  Baseline response time: {baseline:.3f}s")

    # Generate historical dataset
    print("\n[2/4] Generating historical dataset (simulating 24h of monitoring)...")
    historical_df, injected_anomalies = generate_historical_data(
        base_response_time=baseline, hours=24, interval_seconds=60
    )
    print(f"  Generated {len(historical_df)} data points")
    print(f"  Injected {len(injected_anomalies)} synthetic anomalies")

    # Engineer features
    print("\n[3/4] Engineering time-series features...")
    featured_df = engineer_features(historical_df)
    print(f"  Features: {FEATURE_COLUMNS}")

    # Train model and detect anomalies
    print("\n[4/4] Training Isolation Forest model...")
    model, scaler = train_detector(featured_df, contamination=0.03)
    results_df = detect_anomalies(featured_df, model, scaler)

    detected = results_df["is_anomaly"].sum()
    print(
        f"  Anomalies detected: {detected}/{len(results_df)}"
        f" ({100 * detected / len(results_df):.1f}%)"
    )

    # Show top anomalies
    print("\n  Top 5 anomalies by score:")
    top_anomalies = results_df[results_df["is_anomaly"]].nsmallest(
        5, "anomaly_score"
    )
    for _, row in top_anomalies.iterrows():
        print(
            f"    {row['timestamp']:%Y-%m-%d %H:%M}"
            f" - Response: {row['response_time']:.3f}s"
            f" (score: {row['anomaly_score']:.3f})"
        )

Your anomaly detector is working. Next up, you will build a visualization that makes the detected anomalies easy to interpret at a glance.

Visualize the Results

Your model can now detect anomalies and score each data point. Numbers in a terminal are useful for debugging, but a chart tells the story at a glance.

In this step, you will build a two-panel matplotlib visualization that shows response times with anomalies highlighted in red, plus a confidence score panel that reveals how "sure" the model is about each detection.

In this step, get ready to:
  • Write a visualization function that creates a 2-panel chart.
  • Hook the visualization into your main pipeline.
  • Run the full script end-to-end and save the chart as an image.
Write the plot_anomalies function

A good monitoring chart needs two things: a view of the raw data with anomalies called out, and a view of the model's confidence scores so you can see how close borderline points are to the threshold.

  • In monitor.py, find the # --- Main --- comment near the bottom of your file.
  • Add the following visualization section directly above that comment, leaving a blank line above and below:
# --- Visualization ---


def plot_anomalies(df):
    """Plot response times with anomalies highlighted."""
    fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(12, 8), sharex=True)

    # Top panel: Response times with anomalies
    normal = df[~df["is_anomaly"]]
    anomalies = df[df["is_anomaly"]]

    ax1.plot(
        normal["timestamp"],
        normal["response_time"],
        "b.",
        markersize=2,
        label="Normal",
    )
    ax1.scatter(
        anomalies["timestamp"],
        anomalies["response_time"],
        c="red",
        s=30,
        label="Anomaly",
        zorder=5,
    )
    ax1.set_ylabel("Response Time (seconds)")
    ax1.set_title("Website Response Time Monitoring - Anomaly Detection")
    ax1.legend()
    ax1.grid(True, alpha=0.3)

    # Bottom panel: Anomaly scores
    ax2.plot(df["timestamp"], df["anomaly_score"], "g-", linewidth=0.5)
    ax2.axhline(y=0, color="red", linestyle="--", label="Anomaly Threshold")
    ax2.fill_between(
        df["timestamp"],
        df["anomaly_score"],
        0,
        where=(df["anomaly_score"] < 0),
        alpha=0.3,
        color="red",
    )
    ax2.set_ylabel("Anomaly Score")
    ax2.set_xlabel("Time")
    ax2.legend()
    ax2.grid(True, alpha=0.3)

    plt.tight_layout()
    plt.savefig("anomaly_detection_results.png", dpi=150, bbox_inches="tight")
    plt.show()
    print("Plot saved to anomaly_detection_results.png")

What does this code do?

This function creates a figure with two vertically stacked panels that share the same x-axis (time).

  • The top panel splits data into normal and anomalies using the is_anomaly column. Normal points appear as tiny blue dots. Anomalies appear as larger red circles so they pop visually.
  • The bottom panel plots the anomaly score from decision_function() as a green line. Points below the red dashed line at y=0 are the ones the model flagged as anomalies. The red fill makes it easy to spot where the model is most concerned.
  • The function saves the chart at 150 dpi and also calls plt.show() to display it in a popup window.
  • Save monitor.py.

You will not see a change yet. The next substep hooks this function into your main pipeline.

Seeing a syntax error on save?

  • Make sure the function is placed above the # --- Main --- comment, not inside the if __name__ == "__main__": block.
  • Check that the def plot_anomalies(df): line has no extra indentation. It should start at column 0.

Still stuck? Help me fix a syntax error in my plot_anomalies function in monitor.py.

Call the visualization from your main block

The function exists, but nothing calls it yet. You need to add two lines at the end of your __main__ block so the pipeline finishes with a chart.

  • In monitor.py, scroll to the very bottom of your if __name__ == "__main__": block.
  • Find the last line of the top anomalies loop. It currently ends with the print(f" {row['timestamp']...") line inside the for loop.
  • Add the following code below that for loop, at the same indent level as the rest of the __main__ block:
    # Visualize
    print("\nGenerating visualization...")
    plot_anomalies(results_df)

    print("\nDone! Check anomaly_detection_results.png for results.")

What does this code do?

These lines call your new plot_anomalies function with the results_df DataFrame that already contains the is_anomaly and anomaly_score columns from the previous step. The final print confirms the image file was saved.

  • Save monitor.py.
Run the complete script end-to-end

Everything is wired together. Time to run the full pipeline from data collection through visualization.

  • Run the script by entering this command in your terminal:
python monitor.py

What should I see?

The script takes about 30-60 seconds to run because it makes live HTTP requests in the first stage. You should see four progress stages print in order:

  • [1/4] Collecting live response time samples followed by a baseline time.
  • [2/4] Generating historical dataset showing 1440 data points and injected anomalies.
  • [3/4] Engineering time-series features listing all feature names.
  • [4/4] Training Isolation Forest model with a detection summary and top 5 anomalies.

A matplotlib window pops up showing your two-panel chart. Close it to let the script finish. The terminal then prints Plot saved to anomaly_detection_results.png.

  • Close the matplotlib popup window after viewing the chart.
  • Confirm the image was saved by running:
ls

You should see anomaly_detection_results.png listed alongside monitor.py and the venv folder.

Script hangs or no chart appears?

  • If the script seems stuck during stage [1/4], it is waiting for HTTP responses. Check your internet connection and wait up to 60 seconds.
  • If you see KeyError: 'is_anomaly', make sure plot_anomalies(results_df) is called after detect_anomalies has run, not before.
  • If no popup appears but the file saves successfully, your matplotlib backend may not support GUI display. The image file is still valid. Open it from your file explorer.

Need more help? Help me debug why my monitor.py script is not producing a chart.

✔️ Awesome, I've got everything!

Great. Double check you have saved monitor.py and that anomaly_detection_results.png exists in your project folder.

ⓧ I'd like to double check the full code

Here is the complete monitor.py at the end of this step:

import requests
import pandas as pd
import numpy as np
from sklearn.ensemble import IsolationForest
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
from datetime import datetime, timedelta
import time


# --- Data Collection ---

def collect_response_time(url, timeout=10):
    """Measure response time for a single request to a URL."""
    try:
        start = time.time()
        response = requests.get(url, timeout=timeout)
        elapsed = time.time() - start
        return {
            "timestamp": datetime.now(),
            "url": url,
            "response_time": elapsed,
            "status_code": response.status_code,
        }
    except requests.exceptions.RequestException:
        return {
            "timestamp": datetime.now(),
            "url": url,
            "response_time": timeout,
            "status_code": 0,
        }


def collect_batch(urls, num_requests=5):
    """Collect multiple response time measurements."""
    results = []
    for _ in range(num_requests):
        for url in urls:
            result = collect_response_time(url)
            results.append(result)
            time.sleep(0.5)
    return pd.DataFrame(results)


def generate_historical_data(base_response_time=0.2, hours=24, interval_seconds=60):
    """Generate synthetic historical response time data with realistic patterns."""
    num_points = int((hours * 3600) / interval_seconds)
    timestamps = [
        datetime.now() - timedelta(seconds=interval_seconds * (num_points - i))
        for i in range(num_points)
    ]

    np.random.seed(42)

    # Daily sinusoidal pattern
    hour_of_day = np.array([t.hour for t in timestamps])
    daily_pattern = 0.05 * np.sin(2 * np.pi * hour_of_day / 24)

    # Normal noise
    noise = np.random.normal(0, 0.03, num_points)

    response_times = base_response_time + daily_pattern + noise
    response_times = np.maximum(response_times, 0.01)

    # Inject anomalies (about 3% of data)
    anomaly_indices = np.random.choice(
        num_points, size=int(num_points * 0.03), replace=False
    )
    response_times[anomaly_indices] *= np.random.uniform(
        3, 8, size=len(anomaly_indices)
    )

    df = pd.DataFrame({"timestamp": timestamps, "response_time": response_times})

    return df, anomaly_indices


# --- Feature Engineering ---

FEATURE_COLUMNS = [
    "response_time",
    "rolling_mean_5",
    "rolling_std_5",
    "rolling_mean_20",
    "rolling_std_20",
    "lag_1",
    "lag_5",
    "diff_1",
    "hour",
]


def engineer_features(df):
    """Create time-series features for anomaly detection."""
    df = df.copy()
    df = df.sort_values("timestamp").reset_index(drop=True)

    # Rolling statistics
    df["rolling_mean_5"] = df["response_time"].rolling(window=5, min_periods=1).mean()
    df["rolling_std_5"] = (
        df["response_time"].rolling(window=5, min_periods=1).std().fillna(0)
    )
    df["rolling_mean_20"] = (
        df["response_time"].rolling(window=20, min_periods=1).mean()
    )
    df["rolling_std_20"] = (
        df["response_time"].rolling(window=20, min_periods=1).std().fillna(0)
    )

    # Lag features
    df["lag_1"] = df["response_time"].shift(1).fillna(df["response_time"].iloc[0])
    df["lag_5"] = df["response_time"].shift(5).fillna(df["response_time"].iloc[0])

    # Rate of change
    df["diff_1"] = df["response_time"].diff().fillna(0)

    # Hour of day
    df["hour"] = pd.to_datetime(df["timestamp"]).dt.hour

    return df


# --- Anomaly Detection ---


def train_detector(df, contamination=0.03):
    """Train an Isolation Forest model on engineered features."""
    features = df[FEATURE_COLUMNS]

    scaler = StandardScaler()
    features_scaled = scaler.fit_transform(features)

    model = IsolationForest(
        n_estimators=100,
        contamination=contamination,
        random_state=42,
    )
    model.fit(features_scaled)

    return model, scaler


def detect_anomalies(df, model, scaler):
    """Use trained model to detect anomalies in new data."""
    features = df[FEATURE_COLUMNS]
    features_scaled = scaler.transform(features)

    df = df.copy()
    df["anomaly"] = model.predict(features_scaled)
    df["anomaly_score"] = model.decision_function(features_scaled)
    df["is_anomaly"] = df["anomaly"] == -1

    return df


# --- Visualization ---


def plot_anomalies(df):
    """Plot response times with anomalies highlighted."""
    fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(12, 8), sharex=True)

    # Top panel: Response times with anomalies
    normal = df[~df["is_anomaly"]]
    anomalies = df[df["is_anomaly"]]

    ax1.plot(
        normal["timestamp"],
        normal["response_time"],
        "b.",
        markersize=2,
        label="Normal",
    )
    ax1.scatter(
        anomalies["timestamp"],
        anomalies["response_time"],
        c="red",
        s=30,
        label="Anomaly",
        zorder=5,
    )
    ax1.set_ylabel("Response Time (seconds)")
    ax1.set_title("Website Response Time Monitoring - Anomaly Detection")
    ax1.legend()
    ax1.grid(True, alpha=0.3)

    # Bottom panel: Anomaly scores
    ax2.plot(df["timestamp"], df["anomaly_score"], "g-", linewidth=0.5)
    ax2.axhline(y=0, color="red", linestyle="--", label="Anomaly Threshold")
    ax2.fill_between(
        df["timestamp"],
        df["anomaly_score"],
        0,
        where=(df["anomaly_score"] < 0),
        alpha=0.3,
        color="red",
    )
    ax2.set_ylabel("Anomaly Score")
    ax2.set_xlabel("Time")
    ax2.legend()
    ax2.grid(True, alpha=0.3)

    plt.tight_layout()
    plt.savefig("anomaly_detection_results.png", dpi=150, bbox_inches="tight")
    plt.show()
    print("Plot saved to anomaly_detection_results.png")


# --- Main ---

if __name__ == "__main__":
    print("=" * 60)
    print("  Website Response Time Anomaly Detector")
    print("=" * 60)

    # Collect live samples for baseline
    print("\n[1/4] Collecting live response time samples...")
    urls = [
        "https://httpbin.org/get",
        "https://jsonplaceholder.typicode.com/posts/1",
    ]
    live_sample = collect_batch(urls, num_requests=3)
    baseline = live_sample["response_time"].median()
    print(f"  Baseline response time: {baseline:.3f}s")

    # Generate historical dataset
    print("\n[2/4] Generating historical dataset (simulating 24h of monitoring)...")
    historical_df, injected_anomalies = generate_historical_data(
        base_response_time=baseline, hours=24, interval_seconds=60
    )
    print(f"  Generated {len(historical_df)} data points")
    print(f"  Injected {len(injected_anomalies)} synthetic anomalies")

    # Engineer features
    print("\n[3/4] Engineering time-series features...")
    featured_df = engineer_features(historical_df)
    print(f"  Features: {FEATURE_COLUMNS}")

    # Train model and detect anomalies
    print("\n[4/4] Training Isolation Forest model...")
    model, scaler = train_detector(featured_df, contamination=0.03)
    results_df = detect_anomalies(featured_df, model, scaler)

    detected = results_df["is_anomaly"].sum()
    print(
        f"  Anomalies detected: {detected}/{len(results_df)}"
        f" ({100 * detected / len(results_df):.1f}%)"
    )

    # Show top anomalies
    print("\n  Top 5 anomalies by score:")
    top_anomalies = results_df[results_df["is_anomaly"]].nsmallest(
        5, "anomaly_score"
    )
    for _, row in top_anomalies.iterrows():
        print(
            f"    {row['timestamp']:%Y-%m-%d %H:%M}"
            f" - Response: {row['response_time']:.3f}s"
            f" (score: {row['anomaly_score']:.3f})"
        )

    # Visualize
    print("\nGenerating visualization...")
    plot_anomalies(results_df)

    print("\nDone! Check anomaly_detection_results.png for results.")

Secret mission

Add Real-Time Continuous Monitoring

Your batch detector works great on historical data. But real monitoring systems don't wait for a full day's data before raising an alarm. Can you extend your detector into a live loop that watches a URL in real-time and alerts you the moment something looks wrong?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your project folder for future experimentation or delete it entirely.

Resources you used:

  • Virtual environment folder (venv) inside your project directory.
  • Generated image file (anomaly_detection_results.png) saved by your script.
  • Python packages installed inside the virtual environment (requests, pandas, numpy, scikit-learn, matplotlib).

Keep everything running

No action needed. Choose this if you plan to keep experimenting with the anomaly detector or want to tweak parameters like contamination rate or feature windows.

  • Your response-monitor folder stays on your machine with all code and the virtual environment intact.
  • To resume later, open the folder in VS Code and reactivate the virtual environment.

Pause - I'll come back to this later

Since this project is entirely local with no running processes or cloud resources, pausing is the same as keeping. Your files sit quietly on disk with no ongoing cost.

  • Deactivate your virtual environment by running:
deactivate
  • Close the VS Code window. Your project will be exactly as you left it when you return.

Delete - I don't want to use this again

Remove the entire project folder to free up disk space. This deletes your code, the virtual environment, and the generated visualization.

  • Deactivate your virtual environment first by running:
deactivate
  • Move out of the project folder by running:
cd ..
  • Delete the response-monitor folder.

On macOS or Linux:

rm -rf response-monitor

On Windows:

rmdir /s /q response-monitor
  • Confirm the folder is gone by running ls (macOS/Linux) or dir (Windows). You should no longer see response-monitor in the listing.

That's a Wrap!

That's a Wrap!

Nice work. You just built a machine learning-powered monitoring tool that automatically detects unusual website response times using the same core technique behind production systems like Datadog and AWS CloudWatch.

You've learned how to:

  • Build a data collection pipeline that measures real website response times and generates realistic synthetic monitoring data with injected anomalies for training.
  • Engineer time-series features (rolling statistics, lag values, rate of change) that encode temporal context into each observation for an ML model.
  • Train an Isolation Forest model that learns normal response patterns without labeled data and visualize detected anomalies in a professional two-panel chart.
  • Secret Mission: Extended the batch detector into a real-time monitoring loop with live alerting and on-stop visualization.

Ready to quiz yourself?