FX Rate Anomaly Detection Pipeline

Introduction

30 Second Summary

Currency exchange rates shift every second of the trading day. When something unusual happens, like a sudden spike in volatility or a breakdown in how currencies normally move together, businesses need to know immediately so they can protect their positions.

In this project, you will build an end-to-end FX analytics pipeline that pulls real market data, detects statistical anomalies, and surfaces findings in an interactive dashboard with actionable business recommendations.

What You'll Build

A Python data pipeline that ingests live FX rate data via yfinance, detects volatility anomalies using Z-scores and Bollinger Bands, and presents findings through an interactive Streamlit dashboard.

By the end of this project, you'll have:

• A data ingestion pipeline pulling 2 years of daily FX rate data for 4 major currency pairs.
• A statistical analysis engine computing rolling volatility, Z-scores, Bollinger Bands, and cross-pair correlations to detect anomalies.
• An interactive Streamlit dashboard with filterable charts and a business recommendations panel that translates quantitative findings into plain-English advice.
• Secret Mission: Add a correlation regime detector that flags when historically correlated pairs decouple, and generate an automated PDF report of findings.

Are there any prerequisites?

This project assumes you are comfortable with Python and pandas. Familiarity with financial markets concepts like FX pairs and volatility is helpful but not strictly required.

Set Up Your Project Environment

Before you write any analysis code, you need a clean project structure that keeps dependencies isolated and files organised.

In this step, you will create a dedicated folder, initialise a Git repository, set up a Python virtual environment, and install the packages your pipeline needs.

In this step, get ready to:

• Create the project directory and initialise a Git repository with a virtual environment.
• Install all required Python packages (yfinance, pandas, numpy, scipy, plotly, streamlit).
• Create the initial project file structure.

Create the project directory and initialise Git

• Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
• Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.

• Move to your Desktop by running this command:

cd ~/Desktop

• Create your project folder and move into it by running these commands:

mkdir fx-anomaly-detection
cd fx-anomaly-detection

• Initialise a Git repository inside your new folder by running:

git init

You should see a message like Initialized empty Git repository in .../fx-anomaly-detection/.git/.

• Create a Python virtual environment by running:

python3 -m venv venv

What is a virtual environment?

A virtual environment is an isolated Python installation inside your project folder. Packages you install here stay separate from your system Python, preventing version conflicts between projects.

• Activate the virtual environment so that all packages install into it:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the start of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

On Windows, if you see an error about execution policies, run Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned first, then try the activate command again.

On macOS, make sure you used python3 (not python) when creating the venv.

help me activate my Python virtual environment

Install the required Python packages

Your pipeline depends on six packages. Each handles a different part of the workflow, from pulling market data to rendering interactive charts.

• Install all project dependencies by running:

pip install yfinance pandas numpy scipy plotly streamlit

This will take a minute or two as pip downloads and installs each package along with its dependencies.

What do these packages do?

• yfinance pulls live FX rate data from Yahoo Finance. No API key required.
• pandas and numpy handle data manipulation and numerical computation.
• scipy provides advanced statistical functions.
• plotly builds interactive charts with hover details and zoom.
• streamlit turns a Python script into a live web dashboard with zero frontend code.

• Verify the installation succeeded by running:

python3 -c "import yfinance, pandas, numpy, scipy, plotly, streamlit; print('All packages installed successfully')"

You should see All packages installed successfully printed in your terminal.

Seeing an import error or install failure?

Make sure your virtual environment is still active. Check for (venv) in your prompt. If it disappeared, re-run the activate command from the previous substep.

If a single package failed, try installing it individually. For example: pip install yfinance. This helps isolate which package has the issue.

help me fix my pip install errors for yfinance pandas numpy scipy plotly streamlit

Create the project files

Your project will have three Python modules and a README. Each module handles one responsibility: data ingestion, statistical analysis, and the dashboard.

• Create all four project files by running:

macOS

touch data_pipeline.py analysis.py dashboard.py README.md

Windows

New-Item data_pipeline.py, analysis.py, dashboard.py, README.md

• Confirm the files exist by running:

ls

You should see data_pipeline.py, analysis.py, dashboard.py, README.md, and a venv/ folder in the output.

• Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
• Type your editor name (e.g. VS Code) and press Enter to open it.
• In your editor, open the fx-anomaly-detection folder from your Desktop.

You should see all four files listed in your editor's file explorer panel. They are empty for now.

Your environment is ready. Next up, you will build the data ingestion pipeline that pulls live FX market data into your project.

Build the FX Data Ingestion Pipeline

Your project environment is ready to go. Now it is time to build the first core module: a data pipeline that pulls real FX market data from the internet into your project.

Every analytics project starts with reliable data. In this step, you will use yfinance to fetch 2 years of daily FX rates for 4 major currency pairs, validate the data quality, and cache it locally so you never need to re-download during development.

In this step, get ready to:

• Write a data fetch function that pulls daily close prices for GBP/USD, EUR/USD, USD/JPY, and EUR/GBP.
• Add data quality checks to catch missing values and stale data.
• Calculate daily returns and cache the cleaned dataset to CSV.

Write the data fetch function

The yfinance library lets you pull financial data from Yahoo Finance without an API key. FX pairs use a special ticker format where you append =X to the symbol (e.g., GBPUSD=X).

• Open data_pipeline.py in your code editor.
• Add the imports and the data fetch function by pasting this code:

import yfinance as yf
import pandas as pd
import numpy as np
from datetime import datetime, timedelta
import os


def fetch_fx_data(pairs: list[str], period_years: int = 2) -> pd.DataFrame:
    """Fetch daily close prices for FX pairs from Yahoo Finance."""
    end_date = datetime.now()
    start_date = end_date - timedelta(days=period_years * 365)

    tickers = {
        "GBP/USD": "GBPUSD=X",
        "EUR/USD": "EURUSD=X",
        "USD/JPY": "USDJPY=X",
        "EUR/GBP": "EURGBP=X",
    }

    selected_tickers = {k: v for k, v in tickers.items() if k in pairs}

    data = yf.download(
        list(selected_tickers.values()),
        start=start_date.strftime("%Y-%m-%d"),
        end=end_date.strftime("%Y-%m-%d"),
        interval="1d",
        auto_adjust=True,
        progress=False,
    )

    # Extract close prices
    if len(selected_tickers) == 1:
        close_data = data[["Close"]].copy()
        close_data.columns = list(selected_tickers.keys())
    else:
        close_data = data["Close"].copy()
        close_data.columns = [
            name
            for name, ticker in selected_tickers.items()
            if ticker in close_data.columns
        ]

    return close_data

What does this code do?

This function handles the connection between your project and Yahoo Finance's FX data.

• The tickers dictionary maps human-readable pair names (like "GBP/USD") to Yahoo Finance ticker symbols (like "GBPUSD=X").
• The yf.download() call fetches 2 years of daily data for all 4 pairs in a single request.
• The conditional at the end handles a quirk of yfinance: when you download a single ticker, the DataFrame structure differs from multi-ticker downloads.

• Save data_pipeline.py.

You will not see a change yet. The function is defined but nothing calls it until you add the pipeline runner later in this step.

Seeing a red underline on the import?

If your editor shows an error on import yfinance, your editor may not be using the virtual environment's Python interpreter. Select the interpreter from your venv folder in your editor's Python interpreter settings.

help me fix my Python interpreter in VS Code

Add data quality checks

Real-world financial data often has gaps. Weekends and bank holidays leave missing rows. Sometimes a data feed goes stale and repeats the same price for days. Your pipeline needs to detect these issues before analysis.

• In data_pipeline.py, add the data quality function below the fetch_fx_data() function:

def check_data_quality(df: pd.DataFrame) -> dict:
    """Run data quality checks and return a report."""
    report = {
        "total_rows": len(df),
        "date_range": f"{df.index.min()} to {df.index.max()}",
        "missing_values": df.isnull().sum().to_dict(),
        "missing_pct": (df.isnull().sum() / len(df) * 100).to_dict(),
        "stale_data_days": {},
    }

    # Check for stale data (same value repeated 3+ days)
    for col in df.columns:
        stale_count = (df[col].diff() == 0).sum()
        report["stale_data_days"][col] = int(stale_count)

    return report

What does this code do?

• The report dictionary captures five quality metrics: total row count, date range, raw missing values, missing percentage, and stale data days.
• The stale data check uses .diff() == 0 to count days where the price did not change at all. This catches frozen data feeds that could distort your analysis.

• Save data_pipeline.py.

You still will not see output yet. The next chunk adds the orchestrator that calls both functions and produces results.

Indentation error?

Make sure this function is at the top level of the file (no indentation on the def line). It should sit below fetch_fx_data() with a blank line separating them.

help me fix my indentation

Calculate returns and run the pipeline

Raw prices are not directly comparable across currency pairs because each trades at a different scale. Daily returns (percentage change day-over-day) normalise all pairs to the same unit, making statistical analysis possible.

• In data_pipeline.py, add the returns calculation, the pipeline orchestrator, and the main execution block below check_data_quality():

def calculate_returns(df: pd.DataFrame) -> pd.DataFrame:
    """Calculate daily percentage returns."""
    returns = df.pct_change().dropna()
    return returns


def run_pipeline(pairs: list[str] = None) -> tuple[pd.DataFrame, pd.DataFrame, dict]:
    """Run the full data pipeline. Returns (prices, returns, quality_report)."""
    if pairs is None:
        pairs = ["GBP/USD", "EUR/USD", "USD/JPY", "EUR/GBP"]

    print("Fetching FX data...")
    prices = fetch_fx_data(pairs)

    print("Running data quality checks...")
    quality_report = check_data_quality(prices)

    # Forward-fill small gaps (weekends already excluded by yfinance)
    prices = prices.ffill()

    print("Calculating returns...")
    returns = calculate_returns(prices)

    # Cache to CSV
    os.makedirs("data", exist_ok=True)
    prices.to_csv("data/fx_prices.csv")
    returns.to_csv("data/fx_returns.csv")
    print(f"Data cached to data/ directory. {quality_report['total_rows']} rows fetched.")

    return prices, returns, quality_report


if __name__ == "__main__":
    prices, returns, report = run_pipeline()
    print("\nData Quality Report:")
    for key, value in report.items():
        print(f"  {key}: {value}")

What does this code do?

• The calculate_returns() function uses .pct_change() to compute the percentage difference between each day and the previous day. The .dropna() removes the first row (which has no prior day to compare against).
• The run_pipeline() function orchestrates the full flow: fetch data, check quality, forward-fill any gaps, compute returns, and save everything to CSV files in a data/ directory.
• The if __name__ == "__main__" block means this code only runs when you execute the file directly. It will not run when other files import functions from this module.

Why forward-fill instead of dropping rows?

The .ffill() method fills missing values with the last known price. This is standard practice for FX data because a missing row usually means the market was closed. The last known price is the best estimate of the current value.

• Save data_pipeline.py.
• Run the pipeline by executing this command in your terminal:

python data_pipeline.py

You should see output similar to this:

Fetching FX data...
Running data quality checks...
Calculating returns...
Data cached to data/ directory. 521 rows fetched.

Data Quality Report:
  total_rows: 521
  date_range: 2024-05-08 00:00:00 to 2026-05-07 00:00:00
  missing_values: {'GBP/USD': 0, 'EUR/USD': 0, 'USD/JPY': 0, 'EUR/GBP': 0}
  missing_pct: {'GBP/USD': 0.0, 'EUR/USD': 0.0, 'USD/JPY': 0.0, 'EUR/GBP': 0.0}
  stale_data_days: {'GBP/USD': 3, 'EUR/USD': 2, 'USD/JPY': 1, 'EUR/GBP': 4}

Will my numbers match exactly?

Your row counts, dates, and stale data numbers will differ from the example above. This is expected because the pipeline fetches live data up to today's date. The key thing is that all four pairs appear and the pipeline completes without errors.

• Confirm the cached files exist by running:

ls data/

You should see fx_prices.csv and fx_returns.csv in the output.

Seeing a connection or download error?

Make sure you have an active internet connection. The yf.download() call needs to reach Yahoo Finance servers.

If you see a JSONDecodeError or empty DataFrame warning, Yahoo Finance may be temporarily rate-limiting requests. Wait 30 seconds and try again.

help me debug my yfinance download error

✔️ Awesome, I've got everything!

Great. Double check you have saved data_pipeline.py and that both CSV files exist in your data/ folder.

ⓧ I'd like to double check the full code

Here is the complete data_pipeline.py file. Compare your file against this:

import yfinance as yf
import pandas as pd
import numpy as np
from datetime import datetime, timedelta
import os


def fetch_fx_data(pairs: list[str], period_years: int = 2) -> pd.DataFrame:
    """Fetch daily close prices for FX pairs from Yahoo Finance."""
    end_date = datetime.now()
    start_date = end_date - timedelta(days=period_years * 365)

    tickers = {
        "GBP/USD": "GBPUSD=X",
        "EUR/USD": "EURUSD=X",
        "USD/JPY": "USDJPY=X",
        "EUR/GBP": "EURGBP=X",
    }

    selected_tickers = {k: v for k, v in tickers.items() if k in pairs}

    data = yf.download(
        list(selected_tickers.values()),
        start=start_date.strftime("%Y-%m-%d"),
        end=end_date.strftime("%Y-%m-%d"),
        interval="1d",
        auto_adjust=True,
        progress=False,
    )

    # Extract close prices
    if len(selected_tickers) == 1:
        close_data = data[["Close"]].copy()
        close_data.columns = list(selected_tickers.keys())
    else:
        close_data = data["Close"].copy()
        close_data.columns = [
            name
            for name, ticker in selected_tickers.items()
            if ticker in close_data.columns
        ]

    return close_data


def check_data_quality(df: pd.DataFrame) -> dict:
    """Run data quality checks and return a report."""
    report = {
        "total_rows": len(df),
        "date_range": f"{df.index.min()} to {df.index.max()}",
        "missing_values": df.isnull().sum().to_dict(),
        "missing_pct": (df.isnull().sum() / len(df) * 100).to_dict(),
        "stale_data_days": {},
    }

    # Check for stale data (same value repeated 3+ days)
    for col in df.columns:
        stale_count = (df[col].diff() == 0).sum()
        report["stale_data_days"][col] = int(stale_count)

    return report


def calculate_returns(df: pd.DataFrame) -> pd.DataFrame:
    """Calculate daily percentage returns."""
    returns = df.pct_change().dropna()
    return returns


def run_pipeline(pairs: list[str] = None) -> tuple[pd.DataFrame, pd.DataFrame, dict]:
    """Run the full data pipeline. Returns (prices, returns, quality_report)."""
    if pairs is None:
        pairs = ["GBP/USD", "EUR/USD", "USD/JPY", "EUR/GBP"]

    print("Fetching FX data...")
    prices = fetch_fx_data(pairs)

    print("Running data quality checks...")
    quality_report = check_data_quality(prices)

    # Forward-fill small gaps (weekends already excluded by yfinance)
    prices = prices.ffill()

    print("Calculating returns...")
    returns = calculate_returns(prices)

    # Cache to CSV
    os.makedirs("data", exist_ok=True)
    prices.to_csv("data/fx_prices.csv")
    returns.to_csv("data/fx_returns.csv")
    print(f"Data cached to data/ directory. {quality_report['total_rows']} rows fetched.")

    return prices, returns, quality_report


if __name__ == "__main__":
    prices, returns, report = run_pipeline()
    print("\nData Quality Report:")
    for key, value in report.items():
        print(f"  {key}: {value}")

Your data pipeline is live and pulling real market data. Next up, you will build the statistical analysis module that scans this data for volatility anomalies and price breakouts.

Perform Statistical Anomaly Detection

Your data pipeline is pulling live FX data and caching clean returns to disk. Now comes the analytical core of the project.

Raw price data tells you what happened. Statistical anomaly detection tells you what was unusual. In this step, you will build functions that measure rolling volatility, compute Z-scores, and flag days where the market behaved in a statistically significant way.

In this step, get ready to:

• Compute 20-day rolling volatility and Z-scores for each currency pair.
• Calculate Bollinger Bands to identify price breakouts.
• Combine both signals into a summary anomalies DataFrame.

Compute rolling volatility and Z-scores

A single day's return tells you very little about market conditions. What matters is whether today's volatility is unusual compared to the recent past.

You will compute a 20-day rolling standard deviation (the industry standard for short-term FX volatility) and then apply a Z-score to measure how extreme the current volatility is relative to its own 100-day history.

• Open analysis.py in your code editor.
• Add the imports and the rolling volatility function by pasting this code:

import pandas as pd
import numpy as np


def compute_rolling_volatility(returns: pd.DataFrame, window: int = 20) -> pd.DataFrame:
    """Compute rolling standard deviation of returns (annualised)."""
    rolling_vol = returns.rolling(window=window).std() * np.sqrt(252)
    return rolling_vol

What does this code do?

The function takes a DataFrame of daily returns and computes a 20-day rolling standard deviation for each column.

Multiplying by np.sqrt(252) annualises the result. There are approximately 252 trading days per year, so this converts a daily volatility reading into an annual figure that traders can compare across timeframes.

• Now add the Z-score function directly below by pasting this code:

def compute_volatility_zscore(returns: pd.DataFrame, window: int = 20) -> pd.DataFrame:
    """Compute Z-score of rolling volatility to detect unusual volatility regimes."""
    rolling_vol = returns.rolling(window=window).std()
    vol_mean = rolling_vol.rolling(window=window * 5).mean()
    vol_std = rolling_vol.rolling(window=window * 5).std()
    zscore = (rolling_vol - vol_mean) / vol_std
    return zscore

What does this code do?

This function answers the question: is today's volatility unusual compared to the last 100 days?

It computes the 20-day rolling vol, then calculates the mean and standard deviation of that vol over a 100-day window (window * 5). The Z-score formula (value - mean) / std tells you how many standard deviations the current reading is from normal. A Z-score above 2 or below -2 means the volatility is statistically unusual.

Why Z-scores on volatility instead of prices?

Prices are non-stationary. They trend up or down over time, which makes Z-scores on raw prices misleading. Volatility, however, tends to revert to a mean. Computing Z-scores on volatility detects when the market is in a genuinely unusual regime, not just trending.

• Save analysis.py.

You won't see a change yet. The next chunk adds Bollinger Bands, and you will test the full file after combining all signals.

Seeing a syntax error?

Make sure both functions are at the top level of the file (no extra indentation). Check that the import pandas as pd and import numpy as np lines are at the very top of analysis.py.

help me debug a syntax error in my analysis.py volatility functions

Calculate Bollinger Bands for price breakout detection

Volatility Z-scores detect unusual volatility regimes. Bollinger Bands detect unusual price levels. Together they give you two independent signals that something noteworthy happened on a given day.

• In analysis.py, add the Bollinger Bands function below compute_volatility_zscore by pasting this code:

def compute_bollinger_bands(
    prices: pd.DataFrame, window: int = 20, num_std: int = 2
) -> dict:
    """Compute Bollinger Bands for each currency pair."""
    bands = {}
    for col in prices.columns:
        sma = prices[col].rolling(window=window).mean()
        std = prices[col].rolling(window=window).std()
        bands[col] = pd.DataFrame(
            {
                "price": prices[col],
                "sma": sma,
                "upper": sma + (num_std * std),
                "lower": sma - (num_std * std),
            },
            index=prices.index,
        )
    return bands

What does this code do?

For each currency pair, the function computes a 20-day Simple Moving Average (SMA) and then draws bands at +/- 2 standard deviations from it.

When a close price lands outside these bands, it means the price moved more than expected given recent behaviour. The function returns a dictionary keyed by pair name, where each value is a DataFrame with price, sma, upper, and lower columns.

• Save analysis.py.

You won't see output yet. The next function ties everything together and produces the anomaly results you can test.

Indentation looking off?

The function signature spans multiple lines. Make sure the closing parenthesis of the arguments is followed by ) -> dict: on its own line, aligned with def. The body inside the function should be indented one level (4 spaces).

help me fix indentation in my Bollinger Bands function

Combine signals into a unified anomaly detector

Now you will write the function that brings both signals together. It will loop through each pair, flag volatility Z-score spikes and Bollinger Band breakouts, and compile them into a single sorted DataFrame.

• In analysis.py, add the detect_anomalies function below compute_bollinger_bands by pasting this code:

def detect_anomalies(
    prices: pd.DataFrame, returns: pd.DataFrame, window: int = 20, z_threshold: float = 2.0
) -> pd.DataFrame:
    """Detect anomalies using volatility Z-scores and Bollinger Band breakouts."""
    vol_zscore = compute_volatility_zscore(returns, window)
    bands = compute_bollinger_bands(prices, window)

    anomalies = []

    for col in prices.columns:
        # Volatility anomalies
        vol_anomaly_mask = vol_zscore[col].abs() > z_threshold
        vol_anomaly_dates = vol_zscore[col][vol_anomaly_mask]

        for date, zscore_val in vol_anomaly_dates.items():
            anomalies.append(
                {
                    "date": date,
                    "pair": col,
                    "type": "Volatility Spike",
                    "severity": "High" if abs(zscore_val) > 3 else "Medium",
                    "z_score": round(zscore_val, 2),
                    "detail": f"Volatility Z-score: {zscore_val:.2f}",
                }
            )

        # Bollinger Band breakouts
        pair_bands = bands[col]
        upper_break = prices[col] > pair_bands["upper"]
        lower_break = prices[col] < pair_bands["lower"]
        breakout_mask = upper_break | lower_break

        for date in prices.index[breakout_mask]:
            price = prices[col].loc[date]
            upper = pair_bands["upper"].loc[date]
            lower = pair_bands["lower"].loc[date]

            if pd.isna(upper) or pd.isna(lower):
                continue

            direction = "above upper" if price > upper else "below lower"
            anomalies.append(
                {
                    "date": date,
                    "pair": col,
                    "type": "Bollinger Breakout",
                    "severity": "Medium",
                    "z_score": None,
                    "detail": f"Price closed {direction} band",
                }
            )

    anomalies_df = pd.DataFrame(anomalies)
    if not anomalies_df.empty:
        anomalies_df = anomalies_df.sort_values("date").reset_index(drop=True)

    return anomalies_df

What does this code do?

This function runs both detection methods for every currency pair and collects results into a single list of dictionaries.

• Volatility spikes: any day where the absolute Z-score exceeds the threshold (default 2.0). Days with |Z| > 3 are labelled "High" severity.
• Bollinger Breakouts: any day where the close price lands outside the upper or lower band. These are all labelled "Medium" severity.
• The pd.isna() check skips early rows where the rolling window hasn't accumulated enough data to compute valid bands.

• Now add the __main__ block at the bottom of analysis.py so you can run the file directly:

if __name__ == "__main__":
    prices = pd.read_csv("data/fx_prices.csv", index_col=0, parse_dates=True)
    returns = pd.read_csv("data/fx_returns.csv", index_col=0, parse_dates=True)

    print("Detecting anomalies...")
    anomalies = detect_anomalies(prices, returns)
    print(f"Found {len(anomalies)} anomalies")

What does this code do?

The __main__ guard means this code only runs when you execute python analysis.py directly. It loads the cached CSVs from your data pipeline, runs the anomaly detection, and prints the count.

• Save analysis.py.
• Run the analysis module by executing this command in your terminal:

python analysis.py

You should see output like Found 87 anomalies (the exact number depends on recent market conditions). Any number above zero confirms your detection logic is working.

Seeing 0 anomalies or a FileNotFoundError?

• If you get FileNotFoundError, make sure you ran python data_pipeline.py in Step 2 first. The analysis module reads from data/fx_prices.csv and data/fx_returns.csv.
• If you see 0 anomalies, double-check that compute_volatility_zscore uses window * 5 (not window * 2 or just window) for the comparison rolling window.
• Check that your virtual environment is activated. You should see (venv) in your terminal prompt.

help me debug my analysis.py anomaly detection output

✔️ Awesome, I've got everything!

Great. Double check you have saved analysis.py and confirmed the anomaly count prints successfully.

ⓧ I'd like to double check the full code

Here is the complete analysis.py as it should look at the end of this step:

import pandas as pd
import numpy as np


def compute_rolling_volatility(returns: pd.DataFrame, window: int = 20) -> pd.DataFrame:
    """Compute rolling standard deviation of returns (annualised)."""
    rolling_vol = returns.rolling(window=window).std() * np.sqrt(252)
    return rolling_vol


def compute_volatility_zscore(returns: pd.DataFrame, window: int = 20) -> pd.DataFrame:
    """Compute Z-score of rolling volatility to detect unusual volatility regimes."""
    rolling_vol = returns.rolling(window=window).std()
    vol_mean = rolling_vol.rolling(window=window * 5).mean()
    vol_std = rolling_vol.rolling(window=window * 5).std()
    zscore = (rolling_vol - vol_mean) / vol_std
    return zscore


def compute_bollinger_bands(
    prices: pd.DataFrame, window: int = 20, num_std: int = 2
) -> dict:
    """Compute Bollinger Bands for each currency pair."""
    bands = {}
    for col in prices.columns:
        sma = prices[col].rolling(window=window).mean()
        std = prices[col].rolling(window=window).std()
        bands[col] = pd.DataFrame(
            {
                "price": prices[col],
                "sma": sma,
                "upper": sma + (num_std * std),
                "lower": sma - (num_std * std),
            },
            index=prices.index,
        )
    return bands


def detect_anomalies(
    prices: pd.DataFrame, returns: pd.DataFrame, window: int = 20, z_threshold: float = 2.0
) -> pd.DataFrame:
    """Detect anomalies using volatility Z-scores and Bollinger Band breakouts."""
    vol_zscore = compute_volatility_zscore(returns, window)
    bands = compute_bollinger_bands(prices, window)

    anomalies = []

    for col in prices.columns:
        # Volatility anomalies
        vol_anomaly_mask = vol_zscore[col].abs() > z_threshold
        vol_anomaly_dates = vol_zscore[col][vol_anomaly_mask]

        for date, zscore_val in vol_anomaly_dates.items():
            anomalies.append(
                {
                    "date": date,
                    "pair": col,
                    "type": "Volatility Spike",
                    "severity": "High" if abs(zscore_val) > 3 else "Medium",
                    "z_score": round(zscore_val, 2),
                    "detail": f"Volatility Z-score: {zscore_val:.2f}",
                }
            )

        # Bollinger Band breakouts
        pair_bands = bands[col]
        upper_break = prices[col] > pair_bands["upper"]
        lower_break = prices[col] < pair_bands["lower"]
        breakout_mask = upper_break | lower_break

        for date in prices.index[breakout_mask]:
            price = prices[col].loc[date]
            upper = pair_bands["upper"].loc[date]
            lower = pair_bands["lower"].loc[date]

            if pd.isna(upper) or pd.isna(lower):
                continue

            direction = "above upper" if price > upper else "below lower"
            anomalies.append(
                {
                    "date": date,
                    "pair": col,
                    "type": "Bollinger Breakout",
                    "severity": "Medium",
                    "z_score": None,
                    "detail": f"Price closed {direction} band",
                }
            )

    anomalies_df = pd.DataFrame(anomalies)
    if not anomalies_df.empty:
        anomalies_df = anomalies_df.sort_values("date").reset_index(drop=True)

    return anomalies_df


if __name__ == "__main__":
    prices = pd.read_csv("data/fx_prices.csv", index_col=0, parse_dates=True)
    returns = pd.read_csv("data/fx_returns.csv", index_col=0, parse_dates=True)

    print("Detecting anomalies...")
    anomalies = detect_anomalies(prices, returns)
    print(f"Found {len(anomalies)} anomalies")

Your analysis engine can now flag statistically unusual market events across all four currency pairs. Next up, you will add cross-currency correlation analysis to detect when historically related pairs start moving independently.

Analyse Cross-Currency Correlations

Your anomaly detection engine can now spot individual pairs behaving unusually. The next question is whether pairs that normally move together are still doing so.

EUR/USD and GBP/USD are historically correlated because both are "Dollar pairs." When that relationship breaks down, hedging strategies that assume correlation can silently fail. In this step, you will build a correlation tracker, detect breakdown events, and generate plain-English business recommendations from your findings.

In this step, get ready to:

• Compute a 60-day rolling Pearson correlation between EUR/USD and GBP/USD and detect breakdown events.
• Classify correlation regimes and generate business recommendations from the analysis.
• Run the updated analysis script and verify breakdown events print with business context notes.

Compute rolling correlation and detect breakdowns

A rolling correlation measures how closely two series move together over a sliding window. A 60-day window is standard in FX markets because it is short enough to catch regime shifts but long enough to be statistically meaningful.

• In analysis.py, find the detect_anomalies function. Scroll down past it to the if __name__ == "__main__": block at the bottom.
• Add the following two functions above the if __name__ block:

def compute_rolling_correlation(
    returns: pd.DataFrame, pair_a: str, pair_b: str, window: int = 60
) -> pd.Series:
    """Compute rolling Pearson correlation between two pairs."""
    correlation = returns[pair_a].rolling(window=window).corr(returns[pair_b])
    return correlation


def detect_correlation_breakdowns(
    returns: pd.DataFrame,
    pair_a: str = "EUR/USD",
    pair_b: str = "GBP/USD",
    window: int = 60,
    breakdown_threshold: float = 0.5,
) -> pd.DataFrame:
    """Detect periods where correlation drops below threshold."""
    rolling_corr = compute_rolling_correlation(returns, pair_a, pair_b, window)

    # Find breakdown events (transition from above to below threshold)
    above = rolling_corr.shift(1) >= breakdown_threshold
    below = rolling_corr < breakdown_threshold
    breakdown_starts = rolling_corr[above & below]

    events = []
    for date, corr_val in breakdown_starts.items():
        events.append(
            {
                "date": date,
                "pair_a": pair_a,
                "pair_b": pair_b,
                "correlation": round(corr_val, 3),
                "regime": "Breakdown",
                "business_note": (
                    f"Potential hedging risk: {pair_b.split('/')[0]} decoupled from "
                    f"{pair_a.split('/')[0]}. Recommend reviewing cross-currency "
                    f"exposure for {pair_b.split('/')[0]}-denominated positions."
                ),
            }
        )

    return pd.DataFrame(events)

What does this code do?

These two functions work together to track how EUR/USD and GBP/USD move in relation to each other.

• compute_rolling_correlation uses pandas' .rolling(window).corr() to calculate Pearson correlation over a sliding 60-day window. The result is a time series showing how the relationship changes day by day.
• detect_correlation_breakdowns finds the exact dates where correlation transitions from above 0.5 to below 0.5. It uses a shift-and-compare pattern to catch only the moment of transition, not every day the correlation stays low.
• Each breakdown event gets a business_note explaining the hedging risk in plain English. This is the kind of translation from numbers to action that analysts deliver to stakeholders.

Why 0.5 as the breakdown threshold?

EUR/USD and GBP/USD historically correlate between 0.6 and 0.8. A drop below 0.5 represents a significant structural break where one pair is no longer tracking the other. For a UK bank with exposure to both currencies, this means hedging one pair with the other no longer works as expected.

• Save analysis.py.

You will not see a change yet. The next chunk adds the regime classifier and recommendations engine that uses these functions.

Seeing a NameError for compute_rolling_correlation?

• Make sure compute_rolling_correlation is defined above detect_correlation_breakdowns in the file. Python reads top-to-bottom, so the function must exist before it is called.
• Check that both functions are placed above the if __name__ == "__main__": block, not inside it.

Why is my correlation function raising an error?

Frame findings as business recommendations

Detecting anomalies is only half the job. A data analyst also needs to translate those signals into language that stakeholders can act on. You will now add a regime classifier and a recommendations engine that does exactly that.

• In analysis.py, add this short classifier function below detect_correlation_breakdowns:

def classify_correlation_regime(correlation_value: float) -> str:
    """Classify correlation into regime buckets."""
    if correlation_value >= 0.7:
        return "Healthy"
    elif correlation_value >= 0.5:
        return "Warning"
    else:
        return "Breakdown"

What does this code do?

This function labels any correlation reading into one of three regimes. "Healthy" means the pairs are tracking normally (above 0.7). "Warning" means correlation is weakening (0.5 to 0.7). "Breakdown" means the relationship has structurally changed (below 0.5). The dashboard will use these labels to colour-code the correlation chart later.

• Save analysis.py.

You will not see a change yet. The next chunk uses this classifier inside the recommendations engine.

• Now add the recommendations engine below classify_correlation_regime:

def generate_business_recommendations(
    anomalies_df: pd.DataFrame,
    rolling_corr: pd.Series,
    rolling_vol: pd.DataFrame,
) -> list[dict]:
    """Generate plain-English business recommendations based on analysis."""
    recommendations = []

    # Check current volatility regime for each pair
    if not rolling_vol.empty:
        latest_vol = rolling_vol.iloc[-1]
        vol_mean = rolling_vol.mean()

        for pair in rolling_vol.columns:
            if latest_vol[pair] > vol_mean[pair] * 1.5:
                recommendations.append(
                    {
                        "priority": "High",
                        "category": "Volatility",
                        "pair": pair,
                        "recommendation": (
                            f"Elevated {pair} volatility detected "
                            f"({latest_vol[pair]:.1%} annualised vs "
                            f"{vol_mean[pair]:.1%} average). "
                            f"Recommend reviewing hedging exposure for "
                            f"{pair.split('/')[0]}-denominated positions."
                        ),
                    }
                )

    # Check correlation regime
    if not rolling_corr.empty:
        current_corr = rolling_corr.iloc[-1]
        regime = classify_correlation_regime(current_corr)

        if regime in ("Warning", "Breakdown"):
            recommendations.append(
                {
                    "priority": "High" if regime == "Breakdown" else "Medium",
                    "category": "Correlation",
                    "pair": "EUR/USD vs GBP/USD",
                    "recommendation": (
                        f"EUR/USD and GBP/USD correlation in {regime} regime "
                        f"(current: {current_corr:.2f}). Cross-currency hedging "
                        f"assumptions may be invalid. Recommend independent risk "
                        f"assessment for each position."
                    ),
                }
            )

    # Check recent anomaly frequency
    if not anomalies_df.empty:
        recent = anomalies_df[
            anomalies_df["date"] > anomalies_df["date"].max() - pd.Timedelta(days=30)
        ]
        if len(recent) > 10:
            recommendations.append(
                {
                    "priority": "Medium",
                    "category": "Market Regime",
                    "pair": "All",
                    "recommendation": (
                        f"{len(recent)} anomalies detected in the last 30 days. "
                        f"Market may be in a high-volatility regime. Recommend "
                        f"tightening risk limits and increasing monitoring frequency."
                    ),
                }
            )

    if not recommendations:
        recommendations.append(
            {
                "priority": "Low",
                "category": "Status",
                "pair": "All",
                "recommendation": (
                    "All monitored pairs operating within normal parameters. "
                    "No immediate action required."
                ),
            }
        )

    return recommendations

What does this code do?

This function pulls together all your analysis into actionable, prioritised recommendations. It checks three things:

• Volatility check: If the latest annualised volatility for any pair exceeds 1.5x its historical average, flag it as high priority.
• Correlation check: If the EUR/USD vs GBP/USD correlation is in Warning or Breakdown regime, flag that hedging assumptions may be invalid.
• Anomaly frequency check: If more than 10 anomalies occurred in the last 30 days, flag the market as potentially in a high-volatility regime.
• If nothing is flagged, the function returns an all-clear status. This ensures the dashboard always has something to display.

• Save analysis.py.

You will not see output yet. Next you will update the test block at the bottom of the file to exercise the new correlation functions.

Seeing an indentation or syntax error?

• Make sure the multi-line f-strings inside the dictionaries use parentheses for continuation, not backslashes. Each line of the f-string should start with f".
• Check that the function is at the top level (no indentation before def) and not accidentally nested inside another function.

Help me debug a syntax error in my generate_business_recommendations function.

Update the test block and verify the output

The if __name__ == "__main__": block at the bottom of analysis.py currently only tests anomaly detection. You need to add correlation breakdown detection so you can verify the new functions work.

• In analysis.py, scroll to the bottom and find this line inside the if __name__ block:

    print(f"Found {len(anomalies)} anomalies")

• Add the following code directly below that line:

    print("\nDetecting correlation breakdowns...")
    breakdowns = detect_correlation_breakdowns(returns)
    print(f"Found {len(breakdowns)} breakdown events")

    if not breakdowns.empty:
        print("\nBreakdown events:")
        print(breakdowns[["date", "correlation", "business_note"]].to_string())

What does this code do?

This calls detect_correlation_breakdowns with the default parameters (EUR/USD vs GBP/USD, 60-day window, 0.5 threshold). It then prints any breakdown events found, including the date, correlation value, and the plain-English business note.

• Save analysis.py.
• Run the analysis script by entering this command in your terminal:

python analysis.py

You should see the anomaly count from before, followed by a line showing how many correlation breakdown events were found. If breakdowns exist, you will also see a table with dates, correlation values, and business notes like "Potential hedging risk: GBP decoupled from EUR."

What if zero breakdown events are found?

That is a valid result. It means EUR/USD and GBP/USD stayed correlated above 0.5 throughout your 2-year window. The script will print "Found 0 breakdown events" and skip the table. Your code is still working correctly.

Seeing a KeyError for EUR/USD or GBP/USD?

• Open data/fx_returns.csv and check the column headers. They should be exactly EUR/USD and GBP/USD. If the headers look different, re-run python data_pipeline.py to regenerate the CSV files.
• Make sure you have an active internet connection. If yfinance failed silently earlier, the CSV may be empty or missing columns.

Why am I getting a KeyError when running analysis.py?

✔️ Awesome, I've got everything!

Great. Double check you have saved analysis.py and that running the script prints both anomaly and breakdown results.

ⓧ I'd like to double check the full code

Here is the complete analysis.py file at this point:

import pandas as pd
import numpy as np


def compute_rolling_volatility(returns: pd.DataFrame, window: int = 20) -> pd.DataFrame:
    """Compute rolling standard deviation of returns (annualised)."""
    rolling_vol = returns.rolling(window=window).std() * np.sqrt(252)
    return rolling_vol


def compute_volatility_zscore(returns: pd.DataFrame, window: int = 20) -> pd.DataFrame:
    """Compute Z-score of rolling volatility to detect unusual volatility regimes."""
    rolling_vol = returns.rolling(window=window).std()
    vol_mean = rolling_vol.rolling(window=window * 5).mean()
    vol_std = rolling_vol.rolling(window=window * 5).std()
    zscore = (rolling_vol - vol_mean) / vol_std
    return zscore


def compute_bollinger_bands(
    prices: pd.DataFrame, window: int = 20, num_std: int = 2
) -> dict:
    """Compute Bollinger Bands for each currency pair."""
    bands = {}
    for col in prices.columns:
        sma = prices[col].rolling(window=window).mean()
        std = prices[col].rolling(window=window).std()
        bands[col] = pd.DataFrame(
            {
                "price": prices[col],
                "sma": sma,
                "upper": sma + (num_std * std),
                "lower": sma - (num_std * std),
            },
            index=prices.index,
        )
    return bands


def detect_anomalies(
    prices: pd.DataFrame, returns: pd.DataFrame, window: int = 20, z_threshold: float = 2.0
) -> pd.DataFrame:
    """Detect anomalies using volatility Z-scores and Bollinger Band breakouts."""
    vol_zscore = compute_volatility_zscore(returns, window)
    bands = compute_bollinger_bands(prices, window)

    anomalies = []

    for col in prices.columns:
        # Volatility anomalies
        vol_anomaly_mask = vol_zscore[col].abs() > z_threshold
        vol_anomaly_dates = vol_zscore[col][vol_anomaly_mask]

        for date, zscore_val in vol_anomaly_dates.items():
            anomalies.append(
                {
                    "date": date,
                    "pair": col,
                    "type": "Volatility Spike",
                    "severity": "High" if abs(zscore_val) > 3 else "Medium",
                    "z_score": round(zscore_val, 2),
                    "detail": f"Volatility Z-score: {zscore_val:.2f}",
                }
            )

        # Bollinger Band breakouts
        pair_bands = bands[col]
        upper_break = prices[col] > pair_bands["upper"]
        lower_break = prices[col] < pair_bands["lower"]
        breakout_mask = upper_break | lower_break

        for date in prices.index[breakout_mask]:
            price = prices[col].loc[date]
            upper = pair_bands["upper"].loc[date]
            lower = pair_bands["lower"].loc[date]

            if pd.isna(upper) or pd.isna(lower):
                continue

            direction = "above upper" if price > upper else "below lower"
            anomalies.append(
                {
                    "date": date,
                    "pair": col,
                    "type": "Bollinger Breakout",
                    "severity": "Medium",
                    "z_score": None,
                    "detail": f"Price closed {direction} band",
                }
            )

    anomalies_df = pd.DataFrame(anomalies)
    if not anomalies_df.empty:
        anomalies_df = anomalies_df.sort_values("date").reset_index(drop=True)

    return anomalies_df


def compute_rolling_correlation(
    returns: pd.DataFrame, pair_a: str, pair_b: str, window: int = 60
) -> pd.Series:
    """Compute rolling Pearson correlation between two pairs."""
    correlation = returns[pair_a].rolling(window=window).corr(returns[pair_b])
    return correlation


def detect_correlation_breakdowns(
    returns: pd.DataFrame,
    pair_a: str = "EUR/USD",
    pair_b: str = "GBP/USD",
    window: int = 60,
    breakdown_threshold: float = 0.5,
) -> pd.DataFrame:
    """Detect periods where correlation drops below threshold."""
    rolling_corr = compute_rolling_correlation(returns, pair_a, pair_b, window)

    # Find breakdown events (transition from above to below threshold)
    above = rolling_corr.shift(1) >= breakdown_threshold
    below = rolling_corr < breakdown_threshold
    breakdown_starts = rolling_corr[above & below]

    events = []
    for date, corr_val in breakdown_starts.items():
        events.append(
            {
                "date": date,
                "pair_a": pair_a,
                "pair_b": pair_b,
                "correlation": round(corr_val, 3),
                "regime": "Breakdown",
                "business_note": (
                    f"Potential hedging risk: {pair_b.split('/')[0]} decoupled from "
                    f"{pair_a.split('/')[0]}. Recommend reviewing cross-currency "
                    f"exposure for {pair_b.split('/')[0]}-denominated positions."
                ),
            }
        )

    return pd.DataFrame(events)


def classify_correlation_regime(correlation_value: float) -> str:
    """Classify correlation into regime buckets."""
    if correlation_value >= 0.7:
        return "Healthy"
    elif correlation_value >= 0.5:
        return "Warning"
    else:
        return "Breakdown"


def generate_business_recommendations(
    anomalies_df: pd.DataFrame,
    rolling_corr: pd.Series,
    rolling_vol: pd.DataFrame,
) -> list[dict]:
    """Generate plain-English business recommendations based on analysis."""
    recommendations = []

    # Check current volatility regime for each pair
    if not rolling_vol.empty:
        latest_vol = rolling_vol.iloc[-1]
        vol_mean = rolling_vol.mean()

        for pair in rolling_vol.columns:
            if latest_vol[pair] > vol_mean[pair] * 1.5:
                recommendations.append(
                    {
                        "priority": "High",
                        "category": "Volatility",
                        "pair": pair,
                        "recommendation": (
                            f"Elevated {pair} volatility detected "
                            f"({latest_vol[pair]:.1%} annualised vs "
                            f"{vol_mean[pair]:.1%} average). "
                            f"Recommend reviewing hedging exposure for "
                            f"{pair.split('/')[0]}-denominated positions."
                        ),
                    }
                )

    # Check correlation regime
    if not rolling_corr.empty:
        current_corr = rolling_corr.iloc[-1]
        regime = classify_correlation_regime(current_corr)

        if regime in ("Warning", "Breakdown"):
            recommendations.append(
                {
                    "priority": "High" if regime == "Breakdown" else "Medium",
                    "category": "Correlation",
                    "pair": "EUR/USD vs GBP/USD",
                    "recommendation": (
                        f"EUR/USD and GBP/USD correlation in {regime} regime "
                        f"(current: {current_corr:.2f}). Cross-currency hedging "
                        f"assumptions may be invalid. Recommend independent risk "
                        f"assessment for each position."
                    ),
                }
            )

    # Check recent anomaly frequency
    if not anomalies_df.empty:
        recent = anomalies_df[
            anomalies_df["date"] > anomalies_df["date"].max() - pd.Timedelta(days=30)
        ]
        if len(recent) > 10:
            recommendations.append(
                {
                    "priority": "Medium",
                    "category": "Market Regime",
                    "pair": "All",
                    "recommendation": (
                        f"{len(recent)} anomalies detected in the last 30 days. "
                        f"Market may be in a high-volatility regime. Recommend "
                        f"tightening risk limits and increasing monitoring frequency."
                    ),
                }
            )

    if not recommendations:
        recommendations.append(
            {
                "priority": "Low",
                "category": "Status",
                "pair": "All",
                "recommendation": (
                    "All monitored pairs operating within normal parameters. "
                    "No immediate action required."
                ),
            }
        )

    return recommendations


if __name__ == "__main__":
    prices = pd.read_csv("data/fx_prices.csv", index_col=0, parse_dates=True)
    returns = pd.read_csv("data/fx_returns.csv", index_col=0, parse_dates=True)

    print("Detecting anomalies...")
    anomalies = detect_anomalies(prices, returns)
    print(f"Found {len(anomalies)} anomalies")

    print("\nDetecting correlation breakdowns...")
    breakdowns = detect_correlation_breakdowns(returns)
    print(f"Found {len(breakdowns)} breakdown events")

    if not breakdowns.empty:
        print("\nBreakdown events:")
        print(breakdowns[["date", "correlation", "business_note"]].to_string())

Your analysis engine is now complete. It detects individual pair anomalies, tracks cross-pair correlation, and translates everything into plain-English recommendations. Next up, you will build an interactive Streamlit dashboard that puts all of this in front of a user with filterable charts and a live recommendations panel.

Build the Interactive Streamlit Dashboard

Your data pipeline fetches live FX data, and your analysis module detects anomalies and correlation breakdowns. Right now, the only way to see those results is by reading raw DataFrames in the terminal.

In this step, you will build an interactive Streamlit dashboard that ties everything together. The dashboard lets you select a currency pair, filter by date, and see anomalies highlighted on live Plotly charts alongside plain-English business recommendations.

In this step, get ready to:

• Create the dashboard layout with a sidebar for pair selection and date filtering, plus key summary metrics.
• Build a Plotly subplot chart with Bollinger Bands, anomaly markers, and a volatility Z-score panel.
• Add a Business Recommendations panel that generates actionable insights from the analysis output.

Create the dashboard layout with sidebar and metrics

Streamlit apps are single Python files that render top-to-bottom. You will set up the page configuration, import your pipeline and analysis modules, then build interactive sidebar controls that filter the data before any charts render.

• Open dashboard.py in your code editor and add the imports and page configuration:

import streamlit as st
import pandas as pd
import plotly.graph_objects as go
from plotly.subplots import make_subplots
from data_pipeline import run_pipeline
from analysis import (
    compute_rolling_volatility,
    compute_volatility_zscore,
    compute_bollinger_bands,
    detect_anomalies,
    compute_rolling_correlation,
    detect_correlation_breakdowns,
    generate_business_recommendations,
)

st.set_page_config(page_title="FX Anomaly Detection Dashboard", layout="wide")
st.title("FX Rate Anomaly Detection & Business Insights")
st.markdown("*Statistical analytics pipeline for identifying market anomalies and generating actionable recommendations*")

What does this code do?

The imports pull in Streamlit for the dashboard framework, Plotly for interactive charts, and your own modules for data and analysis.

st.set_page_config() sets the browser tab title and uses layout="wide" to give charts the full screen width.

You won't see the dashboard yet. The next chunk adds the data loading that makes the page functional.

• Below the page configuration, add the sidebar controls and data loading function:

# --- Sidebar Controls ---
st.sidebar.header("Controls")

# Load or fetch data
@st.cache_data(ttl=3600)
def load_data():
    prices, returns, quality_report = run_pipeline()
    return prices, returns, quality_report

prices, returns, quality_report = load_data()

# Pair selection
available_pairs = list(prices.columns)
selected_pair = st.sidebar.selectbox("Select Currency Pair", available_pairs)

# Date range filter
min_date = prices.index.min().date()
max_date = prices.index.max().date()
date_range = st.sidebar.date_input(
    "Date Range",
    value=(min_date, max_date),
    min_value=min_date,
    max_value=max_date,
)

if len(date_range) == 2:
    start_date, end_date = date_range
    prices_filtered = prices.loc[str(start_date):str(end_date)]
    returns_filtered = returns.loc[str(start_date):str(end_date)]
else:
    prices_filtered = prices
    returns_filtered = returns

What does this code do?

• @st.cache_data(ttl=3600) caches the pipeline output for one hour so the app does not re-download data on every interaction.
• st.sidebar.selectbox creates a dropdown for picking a currency pair.
• st.sidebar.date_input creates a date range picker that filters both prices and returns DataFrames.

• Save dashboard.py.
• Launch the dashboard by running this command in your terminal:

streamlit run dashboard.py

Your browser opens automatically. You should see the title "FX Rate Anomaly Detection & Business Insights" at the top, and a sidebar on the left with a currency pair selector and date range picker.

Why does the first load take a few seconds?

On the first run, Streamlit calls your pipeline which downloads 2 years of FX data from Yahoo Finance. Subsequent refreshes use the cached data and load instantly.

Seeing an import error?

• Make sure you are running the command from inside your fx-anomaly-detection folder where data_pipeline.py and analysis.py live.
• Check that your virtual environment is activated. You should see (venv) at the start of your terminal prompt.

Help me debug my Streamlit import error

• Now add the key metrics section below the sidebar code in dashboard.py:

# --- Key Metrics ---
anomalies = detect_anomalies(prices_filtered, returns_filtered)
rolling_vol = compute_rolling_volatility(returns_filtered)
vol_zscore = compute_volatility_zscore(returns_filtered)
rolling_corr = compute_rolling_correlation(returns_filtered, "EUR/USD", "GBP/USD")

col1, col2, col3, col4 = st.columns(4)
with col1:
    st.metric("Total Anomalies", len(anomalies))
with col2:
    current_vol = rolling_vol[selected_pair].iloc[-1] if not rolling_vol.empty else 0
    st.metric("Current Volatility", f"{current_vol:.1%}")
with col3:
    current_corr = rolling_corr.iloc[-1] if not rolling_corr.empty else 0
    st.metric("EUR/GBP Correlation", f"{current_corr:.2f}")
with col4:
    pair_anomalies = len(anomalies[anomalies["pair"] == selected_pair]) if not anomalies.empty else 0
    st.metric(f"{selected_pair} Anomalies", pair_anomalies)

st.divider()

What does this code do?

This runs the anomaly detection and rolling computations on the filtered data, then displays four summary cards across the top of the page.

• st.columns(4) creates a 4-column layout so the metrics sit side by side.
• st.metric() renders a large number with a label. These update automatically when the user changes the sidebar filters.

• Save dashboard.py and check your browser.

You should see four metric cards below the title showing Total Anomalies, Current Volatility, EUR/GBP Correlation, and the pair-specific anomaly count. Try changing the currency pair in the sidebar to see the values update.

Metrics not showing up?

• Check the Streamlit terminal for errors. A common cause is a missing comma between function arguments in the detect_anomalies() call.
• If you see a KeyError, make sure the sidebar has loaded data before the metrics code runs. The code order matters in Streamlit.

Help me debug my Streamlit metrics section

Build the main visualisation panel

The core of the dashboard is a two-panel Plotly chart. The top panel shows the selected pair's price with Bollinger Bands and red anomaly markers. The bottom panel shows the volatility Z-score so you can see exactly when volatility became statistically unusual.

• Below the st.divider() line, add the Bollinger Bands chart setup:

# --- Main Visualisation: Bollinger Bands + Anomalies ---
st.subheader(f"{selected_pair} Price with Bollinger Bands & Anomalies")

bands = compute_bollinger_bands(prices_filtered)
pair_bands = bands[selected_pair]

fig = make_subplots(rows=2, cols=1, shared_xaxes=True, row_heights=[0.7, 0.3],
                    vertical_spacing=0.05)

# Price and Bollinger Bands
fig.add_trace(
    go.Scatter(x=pair_bands.index, y=pair_bands["upper"], name="Upper Band",
               line=dict(color="rgba(173,216,230,0.5)", dash="dash")),
    row=1, col=1,
)
fig.add_trace(
    go.Scatter(x=pair_bands.index, y=pair_bands["lower"], name="Lower Band",
               line=dict(color="rgba(173,216,230,0.5)", dash="dash"),
               fill="tonexty", fillcolor="rgba(173,216,230,0.1)"),
    row=1, col=1,
)
fig.add_trace(
    go.Scatter(x=pair_bands.index, y=pair_bands["price"], name="Price",
               line=dict(color="#1f77b4", width=1.5)),
    row=1, col=1,
)
fig.add_trace(
    go.Scatter(x=pair_bands.index, y=pair_bands["sma"], name="SMA(20)",
               line=dict(color="orange", width=1)),
    row=1, col=1,
)

What does this code do?

• make_subplots(rows=2) creates a figure with two vertically stacked panels sharing the same x-axis (dates).
• The upper and lower bands are plotted as dashed lines with fill="tonexty" shading the area between them.
• The price line and 20-day SMA overlay on top so you can visually spot where price breaks through the bands.

The chart will not render yet. The next chunk adds the anomaly markers and Z-score subplot, then displays the figure.

• Directly below the Bollinger Bands traces, add the anomaly markers, Z-score subplot, and chart rendering:

# Add anomaly markers
if not anomalies.empty:
    pair_anom = anomalies[anomalies["pair"] == selected_pair]
    if not pair_anom.empty:
        anom_prices = prices_filtered[selected_pair].loc[
            prices_filtered.index.isin(pair_anom["date"])
        ]
        fig.add_trace(
            go.Scatter(x=anom_prices.index, y=anom_prices.values, mode="markers",
                       name="Anomaly", marker=dict(color="red", size=8, symbol="x")),
            row=1, col=1,
        )

# Volatility Z-score subplot
pair_zscore = vol_zscore[selected_pair]
fig.add_trace(
    go.Scatter(x=pair_zscore.index, y=pair_zscore.values, name="Vol Z-Score",
               line=dict(color="#2ca02c", width=1)),
    row=2, col=1,
)
fig.add_hline(y=2, line_dash="dash", line_color="red", row=2, col=1)
fig.add_hline(y=-2, line_dash="dash", line_color="red", row=2, col=1)

fig.update_layout(height=600, showlegend=True, margin=dict(t=30, b=30))
fig.update_yaxes(title_text="Price", row=1, col=1)
fig.update_yaxes(title_text="Z-Score", row=2, col=1)

st.plotly_chart(fig, use_container_width=True)

What does this code do?

• Anomaly dates are plotted as red X markers on the price chart so you can instantly see where breakouts and volatility spikes occurred.
• The bottom panel plots the volatility Z-score as a green line with red dashed threshold lines at +2 and -2.
• st.plotly_chart(fig, use_container_width=True) renders the interactive chart at the full width of the page.

• Save dashboard.py and check your browser.

You should see a two-panel interactive chart. The top panel shows the price line with a shaded Bollinger Band corridor and red X markers on anomaly dates. The bottom panel shows the Z-score line bouncing between the red threshold lines.

Chart not appearing?

• Check the terminal for a NameError. If vol_zscore is not defined, make sure the key metrics section (which computes it) appears before the chart code.
• If you see an empty chart, confirm that your data/fx_prices.csv file has data by opening it in your editor.

Help me fix my Plotly chart in Streamlit

• Below the main chart, add the rolling correlation chart:

# --- Correlation Chart ---
st.subheader("EUR/USD vs GBP/USD Rolling Correlation (60-day)")

fig_corr = go.Figure()
fig_corr.add_trace(
    go.Scatter(x=rolling_corr.index, y=rolling_corr.values, name="Rolling Correlation",
               line=dict(color="#9467bd", width=1.5))
)
fig_corr.add_hline(y=0.7, line_dash="dash", line_color="green",
                   annotation_text="Healthy threshold")
fig_corr.add_hline(y=0.5, line_dash="dash", line_color="orange",
                   annotation_text="Breakdown threshold")
fig_corr.update_layout(height=300, margin=dict(t=30, b=30),
                       yaxis_title="Correlation", yaxis_range=[-0.5, 1.0])
st.plotly_chart(fig_corr, use_container_width=True)

What does this code do?

This chart plots the 60-day rolling Pearson correlation between EUR/USD and GBP/USD. The green line at 0.7 marks the "Healthy" threshold and the orange line at 0.5 marks the "Breakdown" threshold. Drops below orange indicate the pairs have decoupled.

• Save dashboard.py and check your browser.

A purple correlation line chart appears below the main chart with two annotated threshold lines. You can hover over any point to see the exact date and correlation value.

Add the Business Recommendations panel

Charts show what is happening. Recommendations tell the reader what to do about it. This panel translates the quantitative output into plain-English actions a risk manager could act on immediately.

• Below the correlation chart, add the Business Recommendations section:

# --- Business Recommendations ---
st.subheader("Business Recommendations")

recommendations = generate_business_recommendations(anomalies, rolling_corr, rolling_vol)

for rec in recommendations:
    priority_color = {"High": "🔴", "Medium": "🟡", "Low": "🟢"}.get(rec["priority"], "⚪")
    st.markdown(f"{priority_color} **[{rec['priority']}] {rec['category']}** - {rec['pair']}")
    st.markdown(f"> {rec['recommendation']}")
    st.markdown("")

What does this code do?

The generate_business_recommendations() function from your analysis module checks the current volatility regime, correlation state, and recent anomaly frequency. It returns a list of prioritised recommendations.

The loop renders each recommendation with a coloured priority indicator and a blockquote containing the actionable text. High-priority items get a red circle, medium gets yellow, and low gets green.

• Save dashboard.py and check your browser.

You should see a "Business Recommendations" section with one or more coloured priority indicators followed by plain-English advice about hedging exposure or risk limits.

Recommendations section is blank?

• The function always returns at least one recommendation (a green "all normal" status). If nothing renders, check the terminal for a TypeError in the loop.
• Verify that you are passing anomalies, rolling_corr, and rolling_vol in that exact order to the function.

Help me debug my business recommendations panel

• Finally, add the two expandable detail sections at the bottom of the file:

# --- Data Quality Summary ---
with st.expander("Data Quality Report"):
    st.json(quality_report)

# --- Anomalies Table ---
with st.expander(f"All Detected Anomalies ({len(anomalies)} total)"):
    if not anomalies.empty:
        st.dataframe(anomalies, use_container_width=True)
    else:
        st.info("No anomalies detected in the selected date range.")

What does this code do?

• st.expander() creates collapsible sections so the dashboard stays clean. Users click to reveal the details.
• The Data Quality Report shows the raw JSON from your pipeline's quality checks.
• The Anomalies Table shows every detected anomaly as an interactive, sortable DataFrame.

• Save dashboard.py and check your browser.

You should see two collapsible sections at the bottom of the page. Click "All Detected Anomalies" to expand it and confirm you see a table with date, pair, type, severity, and detail columns.

Test the interactivity

Try switching the currency pair in the sidebar to USD/JPY. All four metrics, the Bollinger Bands chart, and the anomalies table update instantly. This reactive behaviour is built into Streamlit with no extra code needed.

✔️ Awesome, I've got everything!

Great. Double check you have saved dashboard.py and that your browser shows all five sections: metrics, Bollinger Bands chart, correlation chart, recommendations, and expanders.

ⓧ I'd like to double check the full code

Here is the complete dashboard.py file. Compare it against yours:

import streamlit as st
import pandas as pd
import plotly.graph_objects as go
from plotly.subplots import make_subplots
from data_pipeline import run_pipeline
from analysis import (
    compute_rolling_volatility,
    compute_volatility_zscore,
    compute_bollinger_bands,
    detect_anomalies,
    compute_rolling_correlation,
    detect_correlation_breakdowns,
    generate_business_recommendations,
)

st.set_page_config(page_title="FX Anomaly Detection Dashboard", layout="wide")
st.title("FX Rate Anomaly Detection & Business Insights")
st.markdown("*Statistical analytics pipeline for identifying market anomalies and generating actionable recommendations*")

# --- Sidebar Controls ---
st.sidebar.header("Controls")

# Load or fetch data
@st.cache_data(ttl=3600)
def load_data():
    prices, returns, quality_report = run_pipeline()
    return prices, returns, quality_report

prices, returns, quality_report = load_data()

# Pair selection
available_pairs = list(prices.columns)
selected_pair = st.sidebar.selectbox("Select Currency Pair", available_pairs)

# Date range filter
min_date = prices.index.min().date()
max_date = prices.index.max().date()
date_range = st.sidebar.date_input(
    "Date Range",
    value=(min_date, max_date),
    min_value=min_date,
    max_value=max_date,
)

if len(date_range) == 2:
    start_date, end_date = date_range
    prices_filtered = prices.loc[str(start_date):str(end_date)]
    returns_filtered = returns.loc[str(start_date):str(end_date)]
else:
    prices_filtered = prices
    returns_filtered = returns

# --- Key Metrics ---
anomalies = detect_anomalies(prices_filtered, returns_filtered)
rolling_vol = compute_rolling_volatility(returns_filtered)
vol_zscore = compute_volatility_zscore(returns_filtered)
rolling_corr = compute_rolling_correlation(returns_filtered, "EUR/USD", "GBP/USD")

col1, col2, col3, col4 = st.columns(4)
with col1:
    st.metric("Total Anomalies", len(anomalies))
with col2:
    current_vol = rolling_vol[selected_pair].iloc[-1] if not rolling_vol.empty else 0
    st.metric("Current Volatility", f"{current_vol:.1%}")
with col3:
    current_corr = rolling_corr.iloc[-1] if not rolling_corr.empty else 0
    st.metric("EUR/GBP Correlation", f"{current_corr:.2f}")
with col4:
    pair_anomalies = len(anomalies[anomalies["pair"] == selected_pair]) if not anomalies.empty else 0
    st.metric(f"{selected_pair} Anomalies", pair_anomalies)

st.divider()

# --- Main Visualisation: Bollinger Bands + Anomalies ---
st.subheader(f"{selected_pair} Price with Bollinger Bands & Anomalies")

bands = compute_bollinger_bands(prices_filtered)
pair_bands = bands[selected_pair]

fig = make_subplots(rows=2, cols=1, shared_xaxes=True, row_heights=[0.7, 0.3],
                    vertical_spacing=0.05)

# Price and Bollinger Bands
fig.add_trace(
    go.Scatter(x=pair_bands.index, y=pair_bands["upper"], name="Upper Band",
               line=dict(color="rgba(173,216,230,0.5)", dash="dash")),
    row=1, col=1,
)
fig.add_trace(
    go.Scatter(x=pair_bands.index, y=pair_bands["lower"], name="Lower Band",
               line=dict(color="rgba(173,216,230,0.5)", dash="dash"),
               fill="tonexty", fillcolor="rgba(173,216,230,0.1)"),
    row=1, col=1,
)
fig.add_trace(
    go.Scatter(x=pair_bands.index, y=pair_bands["price"], name="Price",
               line=dict(color="#1f77b4", width=1.5)),
    row=1, col=1,
)
fig.add_trace(
    go.Scatter(x=pair_bands.index, y=pair_bands["sma"], name="SMA(20)",
               line=dict(color="orange", width=1)),
    row=1, col=1,
)

# Add anomaly markers
if not anomalies.empty:
    pair_anom = anomalies[anomalies["pair"] == selected_pair]
    if not pair_anom.empty:
        anom_prices = prices_filtered[selected_pair].loc[
            prices_filtered.index.isin(pair_anom["date"])
        ]
        fig.add_trace(
            go.Scatter(x=anom_prices.index, y=anom_prices.values, mode="markers",
                       name="Anomaly", marker=dict(color="red", size=8, symbol="x")),
            row=1, col=1,
        )

# Volatility Z-score subplot
pair_zscore = vol_zscore[selected_pair]
fig.add_trace(
    go.Scatter(x=pair_zscore.index, y=pair_zscore.values, name="Vol Z-Score",
               line=dict(color="#2ca02c", width=1)),
    row=2, col=1,
)
fig.add_hline(y=2, line_dash="dash", line_color="red", row=2, col=1)
fig.add_hline(y=-2, line_dash="dash", line_color="red", row=2, col=1)

fig.update_layout(height=600, showlegend=True, margin=dict(t=30, b=30))
fig.update_yaxes(title_text="Price", row=1, col=1)
fig.update_yaxes(title_text="Z-Score", row=2, col=1)

st.plotly_chart(fig, use_container_width=True)

# --- Correlation Chart ---
st.subheader("EUR/USD vs GBP/USD Rolling Correlation (60-day)")

fig_corr = go.Figure()
fig_corr.add_trace(
    go.Scatter(x=rolling_corr.index, y=rolling_corr.values, name="Rolling Correlation",
               line=dict(color="#9467bd", width=1.5))
)
fig_corr.add_hline(y=0.7, line_dash="dash", line_color="green",
                   annotation_text="Healthy threshold")
fig_corr.add_hline(y=0.5, line_dash="dash", line_color="orange",
                   annotation_text="Breakdown threshold")
fig_corr.update_layout(height=300, margin=dict(t=30, b=30),
                       yaxis_title="Correlation", yaxis_range=[-0.5, 1.0])
st.plotly_chart(fig_corr, use_container_width=True)

# --- Business Recommendations ---
st.subheader("Business Recommendations")

recommendations = generate_business_recommendations(anomalies, rolling_corr, rolling_vol)

for rec in recommendations:
    priority_color = {"High": "🔴", "Medium": "🟡", "Low": "🟢"}.get(rec["priority"], "⚪")
    st.markdown(f"{priority_color} **[{rec['priority']}] {rec['category']}** - {rec['pair']}")
    st.markdown(f"> {rec['recommendation']}")
    st.markdown("")

# --- Data Quality Summary ---
with st.expander("Data Quality Report"):
    st.json(quality_report)

# --- Anomalies Table ---
with st.expander(f"All Detected Anomalies ({len(anomalies)} total)"):
    if not anomalies.empty:
        st.dataframe(anomalies, use_container_width=True)
    else:
        st.info("No anomalies detected in the selected date range.")

Your interactive dashboard is live. It pulls real FX data, highlights anomalies visually, and generates plain-English business recommendations. Next up, you will document everything and push to GitHub so this becomes a portfolio-ready repository.

Document and Push to GitHub

Your dashboard is running with live FX data, anomaly markers, and business recommendations. Now it's time to package everything into a professional GitHub repository that a hiring manager can review.

A strong README is the first thing anyone sees when they visit your repository. It needs to quickly communicate what you built, the methodology behind it, and how to run it locally.

In this step, get ready to:

• Write a professional README documenting your methodology and setup instructions.
• Create a .gitignore and push the repository to GitHub.
• Take a dashboard screenshot for the README.

Write the project README

The README is the front page of your repository. It should communicate what the project does, how the analysis works, and how someone else can run it.

• Open README.md in your code editor (it's currently empty).
• Replace its contents with the following documentation:

# FX Rate Anomaly Detection & Business Insights Pipeline

An end-to-end data analytics pipeline that ingests real FX market data, performs statistical anomaly detection, and presents findings through an interactive dashboard with actionable business recommendations.

## Overview

This project demonstrates:
- **Data Pipeline Design**: Automated ingestion of live FX rate data with quality checks
- **Statistical Analytics**: Rolling volatility, Z-score anomaly detection, Bollinger Bands, and cross-currency correlation analysis
- **Interactive Visualisation**: Streamlit dashboard with Plotly charts for exploring findings
- **Business Translation**: Quantitative signals translated into plain-English recommendations

## Currency Pairs Analysed

- GBP/USD (Cable)
- EUR/USD (Fiber)
- USD/JPY (Gopher)
- EUR/GBP (Chunnel)

## Methodology

### Anomaly Detection
1. **Rolling Volatility Z-Score**: Computes 20-day rolling standard deviation of returns, then measures how unusual current volatility is relative to its own 100-day history using Z-scores. Flags days where |Z| > 2.
2. **Bollinger Band Breakouts**: Identifies days where the close price exceeds the 20-day SMA +/- 2 standard deviations.

### Correlation Analysis
- 60-day rolling Pearson correlation between EUR/USD and GBP/USD returns
- Three-state regime classification: Healthy (>0.7), Warning (0.5-0.7), Breakdown (<0.5)

## Setup & Run

```bash
# Clone the repository
git clone https://github.com/YOUR_USERNAME/fx-anomaly-detection.git
cd fx-anomaly-detection

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install yfinance pandas numpy scipy plotly streamlit

# Run the data pipeline
python data_pipeline.py

# Launch the dashboard
streamlit run dashboard.py
```

## Project Structure

```
fx-anomaly-detection/
├── data_pipeline.py    # Data ingestion and quality checks
├── analysis.py         # Statistical analysis and anomaly detection
├── dashboard.py        # Streamlit interactive dashboard
├── data/               # Cached CSV data (generated)
├── README.md           # This file
└── .gitignore          # Git ignore rules
```

## Key Findings

[Add your specific findings here after running the analysis]

## Dashboard Screenshot

[Add screenshot here]

## Technologies

- Python 3, pandas, NumPy, SciPy
- yfinance (Yahoo Finance API)
- Plotly (interactive charts)
- Streamlit (dashboard framework)

What does this README cover?

The README is structured to answer the questions a reviewer asks in order.

• Overview tells them what this project demonstrates at a glance.
• Methodology shows you understand the statistical techniques, not just the code that implements them.
• Setup & Run lets anyone reproduce your results in under a minute.
• Key Findings and Dashboard Screenshot are placeholders you will fill in shortly.

• Save README.md.

README not rendering properly?

Make sure you haven't accidentally indented the top-level headings (lines starting with #).

Check that the triple backtick code fences have no extra spaces before them. Markdown rendering help.

Create a .gitignore and push to GitHub

A .gitignore file tells Git which files to exclude from version control. You want to keep your virtual environment, cached data, and system files out of the repository.

• Create a new file called .gitignore in the root of your fx-anomaly-detection folder and paste the following:

venv/
__pycache__/
*.pyc
data/
.DS_Store
.streamlit/

Why exclude these files?

• venv/ is your virtual environment. It is hundreds of megabytes and anyone can recreate it with pip install.
• data/ contains your cached CSVs. The pipeline regenerates them on each run, so they do not belong in version control.
• __pycache__/ and *.pyc are compiled Python bytecode files that Python creates automatically.

• Save .gitignore.
• Confirm the file exists by running:

ls -a

You should see .gitignore listed alongside your Python files and the .git directory.

Now you need to create a repository on GitHub to push your code to.

• Open your browser and go to github.com.
• Click the + icon in the top-right corner and select New repository.
• Enter fx-anomaly-detection as the repository name.
• Leave it set to Public.
• Do not check any of the initialisation boxes (no README, no .gitignore, no licence).
• Click Create repository.

Why create an empty repository?

You already have a local Git repository with files. Creating an empty remote repository avoids merge conflicts that happen when both sides have an initial commit.

GitHub will show you a page with setup commands. Back in your terminal, stage all files, commit, and push by running these commands:

git add .
git commit -m "Add FX anomaly detection pipeline with Streamlit dashboard"
git branch -M main
git remote add origin https://github.com/[[GITHUB_USERNAME="your GitHub username"]]/fx-anomaly-detection.git
git push -u origin main

What do these commands do?

• git add . stages all files that are not excluded by your .gitignore.
• git commit -m saves a snapshot with a descriptive message.
• git branch -M main renames your default branch to main (the GitHub convention).
• git remote add origin links your local repository to the GitHub remote.
• git push -u origin main uploads your code to GitHub.

• Refresh your GitHub repository page in the browser.
• You should see your data_pipeline.py, analysis.py, dashboard.py, README.md, and .gitignore listed.
• The README should render as formatted documentation below the file list.

Push rejected or authentication failed?

If you see a permission error, GitHub may need you to authenticate via a Personal Access Token or SSH key instead of a password. Go to Settings > Developer settings > Personal access tokens on GitHub to generate one.

If the push is rejected because the remote is not empty, delete the GitHub repository and recreate it without any initialisation options checked. Help me fix my git push error.

Take a dashboard screenshot for the README

A screenshot gives reviewers an instant preview of what your dashboard looks like without needing to clone and run the project.

• If your Streamlit app is not still running, launch it by running:

streamlit run dashboard.py

• Your browser opens the dashboard at http://localhost:8501.
• Select a currency pair from the sidebar (e.g. GBP/USD) so the charts are populated.
• Take a screenshot of the full dashboard using Cmd+Shift+4 (macOS) or Win+Shift+S (Windows).
• Save the screenshot as dashboard_screenshot.png inside your fx-anomaly-detection folder.

Now update your README and push the screenshot to GitHub.

• Open README.md and find the line that says [Add screenshot here].
• Replace it with:

![Dashboard Screenshot](dashboard_screenshot.png)

• Save README.md.
• Commit and push the screenshot by running:

git add .
git commit -m "Add dashboard screenshot to README"
git push

• Refresh your GitHub repository page.
• You should see the screenshot rendered in the README directly on the repository page.

Screenshot not showing in README?

Check that the filename in your markdown image tag matches the actual file name exactly, including capitalisation and the file extension.

Make sure dashboard_screenshot.png is in the same directory as README.md (not inside a subfolder). Help me fix my README image.

✔️ Awesome, I've got everything!

Great. Double check you have saved both README.md and .gitignore, and that your GitHub repository shows all files.

ⓧ I'd like to double check the full code

Here is the complete README.md:

# FX Rate Anomaly Detection & Business Insights Pipeline

An end-to-end data analytics pipeline that ingests real FX market data, performs statistical anomaly detection, and presents findings through an interactive dashboard with actionable business recommendations.

## Overview

This project demonstrates:
- **Data Pipeline Design**: Automated ingestion of live FX rate data with quality checks
- **Statistical Analytics**: Rolling volatility, Z-score anomaly detection, Bollinger Bands, and cross-currency correlation analysis
- **Interactive Visualisation**: Streamlit dashboard with Plotly charts for exploring findings
- **Business Translation**: Quantitative signals translated into plain-English recommendations

## Currency Pairs Analysed

- GBP/USD (Cable)
- EUR/USD (Fiber)
- USD/JPY (Gopher)
- EUR/GBP (Chunnel)

## Methodology

### Anomaly Detection
1. **Rolling Volatility Z-Score**: Computes 20-day rolling standard deviation of returns, then measures how unusual current volatility is relative to its own 100-day history using Z-scores. Flags days where |Z| > 2.
2. **Bollinger Band Breakouts**: Identifies days where the close price exceeds the 20-day SMA +/- 2 standard deviations.

### Correlation Analysis
- 60-day rolling Pearson correlation between EUR/USD and GBP/USD returns
- Three-state regime classification: Healthy (>0.7), Warning (0.5-0.7), Breakdown (<0.5)

## Setup & Run

```bash
# Clone the repository
git clone https://github.com/YOUR_USERNAME/fx-anomaly-detection.git
cd fx-anomaly-detection

# Create virtual environment
python3 -m venv venv
source venv/bin/activate

# Install dependencies
pip install yfinance pandas numpy scipy plotly streamlit

# Run the data pipeline
python data_pipeline.py

# Launch the dashboard
streamlit run dashboard.py
```

## Project Structure

```
fx-anomaly-detection/
├── data_pipeline.py    # Data ingestion and quality checks
├── analysis.py         # Statistical analysis and anomaly detection
├── dashboard.py        # Streamlit interactive dashboard
├── data/               # Cached CSV data (generated)
├── README.md           # This file
└── .gitignore          # Git ignore rules
```

## Key Findings

[Add your specific findings here after running the analysis]

## Dashboard Screenshot

[Add screenshot here]

## Technologies

- Python 3, pandas, NumPy, SciPy
- yfinance (Yahoo Finance API)
- Plotly (interactive charts)
- Streamlit (dashboard framework)

And here is .gitignore:

venv/
__pycache__/
*.pyc
data/
.DS_Store
.streamlit/

Secret mission

Add a Correlation Regime Detector and Automated PDF Report

Your dashboard surfaces anomalies and recommendations in real time. But stakeholders who are not glued to a browser need a portable, shareable summary. In this extension, you will build a three-state correlation regime detector and wire up automated PDF report generation with a one-click download button.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

• Project directory (fx-anomaly-detection/) on your Desktop.
• Python virtual environment (venv/) inside the project folder.
• Installed Python packages: yfinance, pandas, numpy, scipy, plotly, streamlit.
• Cached data files in the data/ directory (CSV exports).
• GitHub repository (remote).
• Local Git repository history.

Keep everything running

No action needed. Choose this if you are still actively building or plan to extend the project with the Secret Mission.

• Your virtual environment stays intact and ready to activate.
• Your cached data in data/ will still be available for quick dashboard loads.
• Your GitHub repository remains public for portfolio use.

Pause - I'll come back to this later

Shut down running processes to free up memory. Your files and packages stay on disk.

• Stop the Streamlit server by pressing Ctrl+C in the terminal where it is running.
• Deactivate your virtual environment by running deactivate in your terminal.
• Close VS Code or your code editor.
• When you return, activate the virtual environment again with source venv/bin/activate (macOS/Linux) or venv\Scripts\activate (Windows).

Delete - I don't want to use this again

Remove all project resources and start fresh. This cannot be undone locally.

• Stop the Streamlit server by pressing Ctrl+C in the terminal.
• Delete the entire fx-anomaly-detection folder from your Desktop. This removes the virtual environment, cached data, and all code.
• To delete the GitHub repository, go to your repository page on GitHub, click Settings, scroll to the bottom, and click Delete this repository.
• Your globally installed pip packages are unaffected. All project dependencies lived inside the virtual environment you just deleted.

Nice Work!

Nice Work!

You have built a complete FX analytics pipeline that pulls real market data, detects statistical anomalies, and presents actionable business recommendations through an interactive dashboard.

You've learned how to:

• Built an automated data pipeline that ingests live FX market data with quality checks using yfinance and pandas.
• Applied Z-score anomaly detection and Bollinger Bands to identify statistically significant volatility spikes and price breakouts across four currency pairs.
• Created an interactive Streamlit dashboard with Plotly visualisations and a dynamic business recommendations panel that translates quantitative signals into plain-English actions.
• Secret Mission: Added a correlation regime detector and automated PDF report generation to surface risk alerts on demand.

Ready to quiz yourself?