FraudShield: Deploy a Fraud Model

Track, deploy, and monitor a synthetic fraud model with MLflow and Streamlit.

Introduction

30 Second Summary

Most purchases are ordinary. A system can look accurate while missing the rare transactions that matter most.

In this project, you will build FraudShield as a synthetic fraud-risk app that makes machine learning decisions visible. Its workflow connects MLflow experiment tracking to a GitHub-backed Streamlit Community Cloud deployment with data drift monitoring.

What You'll Build

You demo a live dashboard where synthetic transaction signals drive a risk decision while shifted batches trigger a visible drift alert.

By the end of this project, you'll have:

  • A side-by-side experiment comparison that reveals the zero-recall baseline against a class-balanced model.
  • An interactive risk decision that responds to four signal sliders plus an adjustable manual-review threshold.
  • A public monitored app that flags shifted batches and reflects a new code push after deployment.
  • Secret Mission: Build a configurable drift incident simulator with an explicit retraining threshold.

Are there any prerequisites?

You will need Python 3.11 or newer plus VS Code. You will also need Git with accounts for GitHub and Streamlit Community Cloud.

Step 1 checks your Python version and includes the Windows setup for Python 3.12 when needed.

Before We Start

Before any hands-on work, commit to a clearly labeled synthetic FraudShield demonstration for operators who need to understand fraud recall, deployment, and drift monitoring. Real fraud-control decisions remain outside this project's scope.

Prepare the Fresh Windows Project

You have defined FraudShield as a synthetic demonstration for operators. That boundary keeps every result focused on learning.

A clean virtual environment isolates this project from earlier code. Exact dependency pins also reveal which packages FraudShield needs.

In this step, get ready to:
  • Create a fresh project folder with an activated .venv environment.
  • Open the project in VS Code with three pinned dependency files.
  • Initialize a Git repository on the main branch.
Create the folder and isolate Python

You will use PowerShell for the setup commands. Keeping the folder on your Desktop makes it easy to find from the terminal or editor.

  • Press the Windows key to open Windows search.
  • Type PowerShell into the search bar.
  • Press Enter to open PowerShell.
  • Move to your Desktop by running this command:
Set-Location $HOME\Desktop

What does this command do?

PowerShell changes its current location to your Desktop. The project folder you create next will live there.

  • Create the fraudshield folder and enter it by running these commands:
New-Item -Path fraudshield -ItemType Directory
Set-Location -Path fraudshield

What do these commands do?

  • The first command creates the fraudshield directory on your Desktop.
  • The second command makes fraudshield the current PowerShell location.

You should see PowerShell report the new directory. Your prompt should now end with fraudshield.

The pinned scikit-learn release requires Python 3.11 or newer. Checking first prevents an incompatible environment from reaching the package installation.

  • Check the Python version selected by the Windows launcher by running:
py --version

What does this command do?

The Windows Python launcher reports the runtime it currently selects. Use the matching tab below for your result.

✔️ I see Python 3.11 or higher

Your selected Python runtime meets the package requirement. Continue below to create the isolated environment.

ⓧ I see an older version

The selected runtime cannot install the pinned scikit-learn release. You will install Python 3.12 through the Python Install Manager.

The download can take a few minutes. A quiet PowerShell window during this installation is normal.

  • Install Python 3.12 by running:
py install 3.12

What does this command do?

The Python Install Manager downloads the Python 3.12 runtime. It also registers that runtime with the Windows launcher.

Selecting the default is a little fiddly because Windows stores it as a user environment variable. This setting affects your account without changing system-wide configuration.

  • Press the Windows key to open Windows search.
  • Type edit environment variables into the search bar.
  • Open the matching environment settings result.
  • Select Environment Variables.
  • Add a new entry under your user variables.
  • Enter PYTHON_MANAGER_DEFAULT as the variable name.
  • Enter 3.12 as the variable value.
  • Save the environment variable changes.
  • Close the current PowerShell window.
  • Reopen PowerShell through Windows search.
  • Return to the fraudshield folder by running:
Set-Location $HOME\Desktop\fraudshield

What does this command do?

The new PowerShell session loads your updated default. This command returns that session to the project folder.

  • Confirm the selected default runtime by running:
python --version

What should I see?

You should see output beginning with Python 3.12. That confirms the default runtime is ready for the virtual environment.

Python 3.12 still not selected?

Confirm that the user variable is named PYTHON_MANAGER_DEFAULT. Confirm that its value is 3.12.

If another version still appears, help me diagnose my Python Install Manager default.

ⓧ Command not found

The Windows Python launcher is unavailable. Install the official Python Install Manager before adding the Python 3.12 runtime.

  • Open the official Python on Windows guide.
  • Install the Python Install Manager using the Windows installation instructions.
  • Close PowerShell after the installation finishes.
  • Reopen PowerShell through Windows search.
  • Return to the fraudshield folder by running:
Set-Location $HOME\Desktop\fraudshield

What does this command do?

The new PowerShell session can load the installed launcher. This command returns you to the folder created earlier.

  • Install Python 3.12 by running:
py install 3.12

What does this command do?

The Python Install Manager downloads Python 3.12. It registers the runtime for use on Windows.

The default runtime setting sits in the Windows user environment variables. It applies to your account instead of every account on the computer.

  • Press the Windows key to open Windows search.
  • Type edit environment variables into the search bar.
  • Open the matching environment settings result.
  • Select Environment Variables.
  • Add a new entry under your user variables.
  • Enter PYTHON_MANAGER_DEFAULT as the variable name.
  • Enter 3.12 as the variable value.
  • Save the environment variable changes.
  • Close the current PowerShell window.
  • Reopen PowerShell through Windows search.
  • Return to the fraudshield folder by running:
Set-Location $HOME\Desktop\fraudshield

What does this command do?

This returns the refreshed PowerShell session to your project folder. The session now has access to your selected runtime setting.

  • Confirm the Python installation by running:
python --version

What should I see?

You should see output beginning with Python 3.12. The selected runtime is now ready.

Python commands still unavailable?

Confirm that the Python Install Manager completed before you reopened PowerShell. A session opened before installation cannot load the new command aliases.

If the commands remain unavailable, help me restore the Windows Python commands.

A virtual environment gives FraudShield its own package location. Packages installed after activation stay separate from your other Python projects.

  • Create the .venv environment inside fraudshield by running:
python -m venv .venv

What does this command do?

Python creates an isolated environment inside the .venv directory. This directory will hold the project interpreter plus its installed packages.

  • Activate the new environment by running:
.venv\Scripts\Activate.ps1

What does this command do?

The activation script points the current PowerShell session at the environment's Python interpreter. Future package installations now belong to FraudShield.

Your prompt should begin with (.venv). That prefix confirms the environment is active.

✔️ I see (.venv)

The isolated environment is active. Keep this PowerShell window open for the remaining commands.

ⓧ PowerShell blocks activation

Changing an execution policy can feel risky. This fallback applies only to your Windows user account.

  • Allow locally created scripts by running:
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

What does this command do?

PowerShell updates the script policy for your current user. It permits the local activation script while retaining checks for downloaded scripts.

  • Activate the environment again by running:
.venv\Scripts\Activate.ps1

What should I see?

Your PowerShell prompt should now begin with (.venv). The isolated environment is active for this session.

Activation still blocked?

Confirm that PowerShell is inside the Desktop fraudshield folder. The activation path only works from the folder containing .venv.

If the script still fails, help me activate my FraudShield virtual environment.

Add and install the pinned dependencies

Dependency files turn the package setup into a repeatable recipe. The app packages stay separate from the development-only experiment tracker.

  • Press the Windows key to open Windows search.
  • Type Visual Studio Code into the search bar.
  • Press Enter to open VS Code.
  • Select File from the top menu.
  • Select Open Folder....
  • Choose the fraudshield folder on your Desktop.
  • Select Select Folder to open the workspace.

You should see fraudshield at the top of the Explorer view. The hidden .venv directory may also appear underneath it.

The main requirements file holds packages needed by the finished app. Exact pins keep later local runs aligned with deployment.

  • Select the New File... button in the Explorer view.
  • Enter requirements.txt as the file name.
  • Add the app dependencies by pasting this content:
streamlit==1.65.0
scikit-learn==1.9.1

What does this file do?

  • The Streamlit pin provides the interactive app framework used later.
  • The scikit-learn pin provides the dataset generator plus the classification models.
  • Save requirements.txt by pressing Ctrl+S.
  • Confirm requirements.txt appears directly under fraudshield in the Explorer view.

Requirements file missing?

Confirm that the file is named exactly requirements.txt. It should sit directly inside fraudshield.

If the file still does not appear, help me create requirements.txt in the correct folder.

The development requirements file reuses every app dependency. It adds MLflow for local experiment tracking.

  • Select the New File... button in the Explorer view.
  • Enter requirements-dev.txt as the file name.
  • Add the development dependencies by pasting this content:
-r requirements.txt
mlflow==3.17.0

What does this file do?

  • The first line tells pip to include every package from requirements.txt.
  • The second line adds the pinned MLflow release for development.
  • Save requirements-dev.txt by pressing Ctrl+S.
  • Confirm both requirements files appear in the Explorer view.

Development requirements missing?

Confirm that the file name contains -dev. Check that both lines were saved inside requirements-dev.txt.

If the file still looks different, help me check my development requirements file.

The ignore file keeps local environments plus experiment data out of future commits. Those generated files belong to one computer.

  • Select the New File... button in the Explorer view.
  • Enter .gitignore as the file name.
  • Add the project ignore rules by pasting this content:
.venv/
__pycache__/
mlflow.db
mlruns/

What does this file do?

  • The .venv/ rule excludes the local virtual environment.
  • The __pycache__/ rule excludes generated Python cache files.
  • The final two rules exclude the future local MLflow database plus its artifact directory.
  • Save .gitignore by pressing Ctrl+S.
  • Confirm all three setup files appear in the Explorer view.

Ignore file has the wrong name?

Check that the name is exactly .gitignore. Remove any extra .txt extension.

If the name still changes, help me create the .gitignore file in VS Code.

Use this comparison to confirm that every setup file matches the project pins exactly.

✔️ Awesome, I've got everything!

Great. Confirm that all three files are saved before installing their packages.

ⓧ I'd like to double check the full code

Compare each saved file with its complete reference below.

streamlit==1.65.0
scikit-learn==1.9.1

Check requirements.txt

Your requirements.txt file should contain these two lines in this order. Both version pins must match exactly.

-r requirements.txt
mlflow==3.17.0

Check requirements-dev.txt

Your requirements-dev.txt file should include the main requirements file first. The MLflow pin should occupy the second line.

.venv/
__pycache__/
mlflow.db
mlruns/

Check .gitignore

Your .gitignore file should contain these four entries. Each entry should occupy its own line.

The installation downloads MLflow plus its supporting packages. Expect this command to run for a few minutes before the prompt returns.

  • Switch back to the activated PowerShell window.
  • Install the pinned project dependencies by running:
python -m pip install -r requirements-dev.txt

What does this command do?

Python runs pip inside the activated environment. Pip follows requirements-dev.txt into the main requirements file before installing every pin.

The installation prints package progress as it runs. Your (.venv) prompt returns when the dependency set is installed.

Dependency installation failed?

Confirm that your PowerShell prompt begins with (.venv). Check that both requirements files are saved directly inside fraudshield.

If the error mentions compatibility or another package problem, help me diagnose the FraudShield dependency installation.

Initialize Git and verify the environment

A local repository records this clean starting point before any model code exists. Its initial branch matches the main branch used by the later deployment workflow.

  • Initialize the repository on the main branch by running:
git init -b main

What does this command do?

Git creates local repository metadata inside fraudshield. The initial branch receives the name main.

You should see confirmation that an empty Git repository was initialized. The reported path should point to the Desktop fraudshield folder.

Git initialization failed?

Confirm that PowerShell is still inside fraudshield. Confirm that the environment prefix remains visible before the prompt.

If Git still cannot initialize the repository, help me diagnose git init on Windows.

Before you run the final check, which three pinned package names do you expect to find in the output?

  • Inspect the activated environment's installed packages by running:
python -m pip freeze

What does this command do?

Pip prints every package installed inside the active .venv environment. The output confirms which versions the dependency installation produced.

You should find mlflow==3.17.0. You should also find scikit-learn==1.9.1 plus streamlit==1.65.0.

Pinned packages missing?

Confirm that the PowerShell prompt begins with (.venv). Compare both requirements files with the full-code references above.

If a version is missing or different, help me verify the FraudShield environment packages.

Your fresh workspace now has an isolated runtime plus reproducible dependencies. Next, you will build a majority-class baseline that exposes misleading accuracy.

Build a Failing Fraud Baseline

Your isolated FraudShield environment is ready. Every result from here comes from this fresh codebase.

Fraud detection creates an imbalanced classification problem because normal transactions greatly outnumber fraud cases. Using scikit-learn, you will create a majority-class benchmark that tests whether accuracy tells the whole story.

In this step, get ready to:
  • Create a deterministic synthetic dataset with a rare fraud class.
  • Train a majority-class baseline with DummyClassifier.
  • Expose the accuracy trap by comparing accuracy with fraud recall.
Create the imbalanced dataset

A reliable comparison starts with repeatable data. The fixed random seed gives each run the same synthetic rows.

  • Return to VS Code with the FraudShield folder from earlier.
  • Select the new-file icon beside the FraudShield folder name in the sidebar.
  • Name the file model.py.

You should now see model.py inside the FraudShield folder.

  • Populate model.py by pasting this code:
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split

FEATURE_NAMES = [
    "Transaction velocity signal",
    "Amount anomaly signal",
    "Merchant risk signal",
    "Device mismatch signal",
]


def make_dataset():
    return make_classification(
        n_samples=4000,
        n_features=4,
        n_informative=3,
        n_redundant=1,
        n_clusters_per_class=1,
        weights=[0.98, 0.02],
        flip_y=0.01,
        class_sep=0.8,
        random_state=42,
    )

What does this code do?

  • The FEATURE_NAMES list gives each synthetic input a readable label.
  • The make_classification() function creates 4,000 rows across four signals.
  • The weights=[0.98, 0.02] setting makes fraud the rare class.
  • The random_state=42 setting keeps the generated data deterministic.
  • Save model.py.
  • Confirm the sidebar lists model.py inside the FraudShield folder.

Is model.py in the wrong place?

Move model.py into the same FraudShield folder that contains requirements.txt.

Still stuck? Help me place model.py in the correct VS Code folder.

The dataset also needs a consistent evaluation split. Stratification preserves the rare-class proportion in both parts of the data.

  • Place the cursor below the closing parenthesis of make_dataset().
  • Add the split helper by pasting this code:
def split_dataset():
    X, y = make_dataset()
    return train_test_split(
        X,
        y,
        test_size=0.25,
        random_state=42,
        stratify=y,
    )

How does the split stay reliable?

  • The split_dataset() function separates the rows into training data and test data.
  • The test_size=0.25 setting reserves one quarter of the rows for evaluation.
  • The stratify=y setting preserves the class ratio across the split.
  • The second random_state=42 setting makes the split repeatable.
  • Save model.py.
  • Check the bottom of model.py for the complete split_dataset() function.

Does the split helper look incomplete?

Check that split_dataset() begins after make_dataset() closes. Keep every line inside the function indented by four spaces.

Need another pair of eyes? Help me compare my model.py structure with the target code.

✔️ Awesome, I've got everything!

Your full model.py now defines the synthetic data and its repeatable split.

ⓧ I'd like to double check the full code

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split

FEATURE_NAMES = [
    "Transaction velocity signal",
    "Amount anomaly signal",
    "Merchant risk signal",
    "Device mismatch signal",
]


def make_dataset():
    return make_classification(
        n_samples=4000,
        n_features=4,
        n_informative=3,
        n_redundant=1,
        n_clusters_per_class=1,
        weights=[0.98, 0.02],
        flip_y=0.01,
        class_sep=0.8,
        random_state=42,
    )


def split_dataset():
    X, y = make_dataset()
    return train_test_split(
        X,
        y,
        test_size=0.25,
        random_state=42,
        stratify=y,
    )
Build the majority-class baseline

A baseline sets the minimum standard that a useful model must beat. This one always selects the most common training label.

  • Select the new-file icon beside the FraudShield folder name in the sidebar.
  • Name the file baseline.py.

You should now see baseline.py beside model.py in the sidebar.

  • Populate baseline.py by pasting this code:
from sklearn.dummy import DummyClassifier
from sklearn.metrics import accuracy_score, average_precision_score, recall_score

from model import split_dataset

X_train, X_test, y_train, y_test = split_dataset()
model = DummyClassifier(strategy="most_frequent")
model.fit(X_train, y_train)

predictions = model.predict(X_test)
probabilities = [row[1] for row in model.predict_proba(X_test)]

print("Majority-class baseline")
print(f"Accuracy: {accuracy_score(y_test, predictions):.3f}")
print(f"Fraud recall: {recall_score(y_test, predictions):.3f}")
print(f"Average precision: {average_precision_score(y_test, probabilities):.3f}")

What does this baseline measure?

  • The DummyClassifier learns which class appears most often in the training data.
  • The predictions variable holds the selected class for each test row.
  • The probabilities variable holds the fraud-class score for each test row.
  • The three printed metrics examine overall correctness, fraud detection, and score ranking.
  • Save baseline.py.
  • Confirm the sidebar lists baseline.py beside model.py.

Are the baseline imports unresolved?

Confirm that baseline.py shares the FraudShield folder with model.py. Check that your earlier .venv environment remains active.

Need help checking the imports? Help me diagnose the imports in baseline.py.

✔️ Awesome, I've got everything!

Your full baseline.py is ready to test the majority-class strategy.

ⓧ I'd like to double check the full code

from sklearn.dummy import DummyClassifier
from sklearn.metrics import accuracy_score, average_precision_score, recall_score

from model import split_dataset

X_train, X_test, y_train, y_test = split_dataset()
model = DummyClassifier(strategy="most_frequent")
model.fit(X_train, y_train)

predictions = model.predict(X_test)
probabilities = [row[1] for row in model.predict_proba(X_test)]

print("Majority-class baseline")
print(f"Accuracy: {accuracy_score(y_test, predictions):.3f}")
print(f"Fraud recall: {recall_score(y_test, predictions):.3f}")
print(f"Average precision: {average_precision_score(y_test, probabilities):.3f}")
Reveal the accuracy trap

Accuracy counts every correct label. Fraud recall focuses on the actual fraud cases.

Average precision evaluates how well the model's scores rank the rare fraud class. Looking at all three metrics exposes behavior that one headline number can hide.

Before you run this, consider whether a classifier that always chooses the majority class can still earn high accuracy.

  • Run the baseline from the activated PowerShell terminal by entering this command:
python baseline.py

What should you see?

  • The Accuracy: value is much higher than the fraud recall value.
  • The Fraud recall: 0.000 result confirms that the baseline catches none of the fraud cases.
  • The Average precision: value stays low because the baseline provides no useful fraud ranking.

You found the failure this benchmark was built to reveal. The baseline looks strong overall while providing no useful fraud detection.

Did the baseline fail to run?

  • Confirm the terminal is using the FraudShield folder that contains baseline.py.
  • Confirm model.py sits beside baseline.py.
  • Compare both files with the full-code tabs if your result differs.

Still blocked? Help me debug my FraudShield baseline run.

Your baseline has exposed the gap. Next, you will build a stronger comparison that makes the model trade-off inspectable.

Track and Compare the Better Model

Your majority-class baseline now exposes the accuracy trap. It reports strong accuracy while missing every fraudulent transaction.

This step adds class weighting to recover fraud recall. MLflow adds experiment tracking so you can inspect the trade-off through recorded metrics.

In this step, get ready to:
  • Add a class-balanced logistic regression model to model.py.
  • Track the baseline model and balanced model in one reproducible experiment.
  • Compare both runs through the local MLflow UI.
Add the class-balanced risk model

Class weighting gives mistakes on the minority fraud class more influence during training. Logistic regression can then learn a boundary that pays attention to those rare examples.

  • In model.py from earlier, select the two import lines at the top.
  • Replace those lines with this import block:
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

What Does This Import Add?

  • The existing imports continue to generate the synthetic dataset and preserve its class ratio during the split.
  • The new LogisticRegression import supplies the model used for class-balanced training.
  • Save model.py.
  • Confirm the new import has not disrupted the baseline by running:
python baseline.py

What Does This Check Prove?

This reruns the existing baseline after the import change. A successful run proves that the dataset helpers still work.

You should see the majority-class metrics again. Fraud recall should remain at 0.000 because the baseline itself has not changed.

Seeing an Import Error?

Check that LogisticRegression uses the same capitalization in the import. Confirm that the activated .venv is still shown in your terminal prompt.

Ask for help with the import error.

The import makes the estimator available. The next helper keeps its training policy in one reusable place.

  • Move to the bottom of model.py.
  • Add this function below split_dataset():
def build_risk_model():
    return LogisticRegression(class_weight="balanced", max_iter=1000)

What Does This Function Do?

  • The build_risk_model() helper creates the same model configuration whenever the project needs it.
  • The class_weight="balanced" setting increases the influence of rare fraud examples during fitting.
  • The max_iter=1000 setting gives the solver enough iterations to fit this dataset.
  • Save model.py.
  • Confirm that build_risk_model() appears once at the bottom of the file.

Function in the Wrong Place?

Place build_risk_model() after the closing parenthesis of split_dataset(). Keep one blank line between the functions.

Ask for help checking the function placement.

✔️ Awesome, I've got everything!

Great. Save model.py before you continue.

ⓧ I'd like to double check the full code

from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import train_test_split

FEATURE_NAMES = [
    "Transaction velocity signal",
    "Amount anomaly signal",
    "Merchant risk signal",
    "Device mismatch signal",
]


def make_dataset():
    return make_classification(
        n_samples=4000,
        n_features=4,
        n_informative=3,
        n_redundant=1,
        n_clusters_per_class=1,
        weights=[0.98, 0.02],
        flip_y=0.01,
        class_sep=0.8,
        random_state=42,
    )


def split_dataset():
    X, y = make_dataset()
    return train_test_split(
        X,
        y,
        test_size=0.25,
        random_state=42,
        stratify=y,
    )


def build_risk_model():
    return LogisticRegression(class_weight="balanced", max_iter=1000)

That is the training policy in place. Every later use of the risk model now receives the same balanced configuration.

Track both experiments

A metric printed in one terminal session is easy to lose. MLflow stores each model's parameters and metrics as a run so the evidence stays comparable.

  • In the Explorer sidebar in VS Code, create train.py beside model.py.
  • Copy the imports and evaluation function below into train.py:
import mlflow
import mlflow.sklearn
from sklearn.dummy import DummyClassifier
from sklearn.metrics import accuracy_score, average_precision_score, recall_score

from model import build_risk_model, split_dataset


def evaluate(model, X_test, y_test):
    predictions = model.predict(X_test)
    probabilities = [row[1] for row in model.predict_proba(X_test)]
    return {
        "accuracy": accuracy_score(y_test, predictions),
        "fraud_recall": recall_score(y_test, predictions),
        "average_precision": average_precision_score(y_test, probabilities),
    }

What Does This Code Do?

  • The imports bring in MLflow logging support alongside the same baseline model and metrics used earlier.
  • The evaluate() function calculates one consistent metric dictionary for either model.
  • The probability scores feed average precision while class predictions feed accuracy and fraud recall.
  • Save train.py.
  • Confirm that train.py appears beside model.py in the Explorer sidebar.

File Missing From Explorer?

Confirm that the filename is exactly train.py. Check that you created it inside the same project folder as model.py.

Ask for help locating the new file.

The evaluator gives both models an identical test. The tracking setup now needs a local database plus two model configurations.

  • Add the experiment setup below evaluate() in train.py:
X_train, X_test, y_train, y_test = split_dataset()
mlflow.set_tracking_uri("sqlite:///mlflow.db")
mlflow.set_experiment("fraud-risk-classification")

experiments = [
    (
        "majority-baseline",
        DummyClassifier(strategy="most_frequent"),
        {"model_type": "DummyClassifier", "strategy": "most_frequent"},
    ),
    (
        "balanced-logistic-regression",
        build_risk_model(),
        {"model_type": "LogisticRegression", "class_weight": "balanced", "max_iter": 1000},
    ),
]

How Is the Experiment Set Up?

  • The shared split ensures that both models train and test on the same rows.
  • The SQLite tracking URI stores experiment history in the local mlflow.db file.
  • The experiments list pairs each run name with its estimator and logged parameters.
  • Save train.py.
  • Create the local tracking database by running:
python train.py

What Does This Run Create?

This partial script connects to mlflow.db and creates the fraud-risk-classification experiment. The model runs are added in the next code chunk.

The command should finish without a Python traceback. You should also see mlflow.db in the project Explorer.

Tracking Database Not Created?

Confirm that your terminal is inside the folder containing train.py. Check that the activated environment contains MLflow.

Ask for help creating the local tracking database.

The database can now receive complete runs. Each run fits one model before logging its configuration and results.

  • Move to the bottom of train.py.
  • Add the logging loop below the experiments list:
for run_name, model, parameters in experiments:
    with mlflow.start_run(run_name=run_name):
        model.fit(X_train, y_train)
        metrics = evaluate(model, X_test, y_test)
        mlflow.log_params(parameters)
        mlflow.log_metrics(metrics)
        mlflow.sklearn.log_model(sk_model=model, name="model")

        print(run_name)
        for metric_name, metric_value in metrics.items():
            print(f"  {metric_name}: {metric_value:.3f}")

What Does the Logging Loop Do?

  • Each pass starts a named MLflow run before fitting one estimator.
  • The shared evaluator produces accuracy, fraud recall, and average precision for the held-out data.
  • MLflow stores the parameters and metrics alongside a reusable model artifact.
  • The final print loop exposes the same comparison in the terminal.
  • Save train.py.

Before you run the complete script, predict which model will recover fraud recall.

  • Train and track both models by running:
python train.py

What Does the Complete Run Record?

The command records two runs against the same held-out dataset. It also prints each run's three metrics so you can make an immediate comparison.

You should see majority-baseline and balanced-logistic-regression in the terminal. Each name should have accuracy, fraud recall, and average precision beneath it.

  • Compare fraud recall between the two printed metric groups.
  • Compare average precision between the same two groups.
  • Use accuracy as supporting context for the trade-off.

The balanced model should recover fraudulent examples that the zero-recall baseline misses. That improvement is the evidence that accuracy concealed.

Run Stops Before Printing Metrics?

Check that build_risk_model is imported from model.py. Confirm that the logging loop sits below the closing bracket of experiments.

Ask for help debugging the training run.

✔️ Awesome, I've got everything!

Excellent. Save train.py before opening the tracking UI.

ⓧ I'd like to double check the full code

import mlflow
import mlflow.sklearn
from sklearn.dummy import DummyClassifier
from sklearn.metrics import accuracy_score, average_precision_score, recall_score

from model import build_risk_model, split_dataset


def evaluate(model, X_test, y_test):
    predictions = model.predict(X_test)
    probabilities = [row[1] for row in model.predict_proba(X_test)]
    return {
        "accuracy": accuracy_score(y_test, predictions),
        "fraud_recall": recall_score(y_test, predictions),
        "average_precision": average_precision_score(y_test, probabilities),
    }


X_train, X_test, y_train, y_test = split_dataset()
mlflow.set_tracking_uri("sqlite:///mlflow.db")
mlflow.set_experiment("fraud-risk-classification")

experiments = [
    (
        "majority-baseline",
        DummyClassifier(strategy="most_frequent"),
        {"model_type": "DummyClassifier", "strategy": "most_frequent"},
    ),
    (
        "balanced-logistic-regression",
        build_risk_model(),
        {"model_type": "LogisticRegression", "class_weight": "balanced", "max_iter": 1000},
    ),
]

for run_name, model, parameters in experiments:
    with mlflow.start_run(run_name=run_name):
        model.fit(X_train, y_train)
        metrics = evaluate(model, X_test, y_test)
        mlflow.log_params(parameters)
        mlflow.log_metrics(metrics)
        mlflow.sklearn.log_model(sk_model=model, name="model")

        print(run_name)
        for metric_name, metric_value in metrics.items():
            print(f"  {metric_name}: {metric_value:.3f}")
Compare the runs in MLflow

The terminal comparison proves that the models behave differently. The tracking UI keeps their parameters and artifacts beside those metrics so the result remains inspectable.

  • Create a second PowerShell terminal in VS Code.
  • Activate the existing environment in the second terminal by running:
.venv\Scripts\Activate.ps1

Why Use a Second Terminal?

The MLflow server keeps one terminal occupied while it runs. Your original terminal remains available for later project commands.

Before you start the server, predict what the tracking UI will reveal about the two recorded runs.

  • Start the local tracking server by running:
mlflow server --backend-store-uri sqlite:///mlflow.db --port 5000

What Does This Server Command Do?

The server reads the local mlflow.db tracking database. It makes the recorded experiment available through a browser on port 5000.

The command keeps running while the UI is available. The second terminal should remain occupied without showing a Python traceback.

  • Leave the second terminal running during the comparison.
  • Open http://localhost:5000 in your browser.
  • Select the fraud-risk-classification experiment.
  • Select the checkbox beside majority-baseline.
  • Select the checkbox beside balanced-logistic-regression.
  • Open the comparison view for the selected runs.
  • Inspect accuracy, fraud recall, and average precision for both runs.
  • Inspect the parameters and logged model artifact for each run.

You should see exactly two runs in the experiment. Each run should contain three metrics plus its model parameters and logged artifact.

Cannot Open the Tracking UI?

Confirm that the server terminal is still running. Check that you started it inside the folder containing mlflow.db.

Ask for help connecting to the local MLflow UI.

Strong work. Your model comparison now lives in a reproducible experiment with metrics, parameters, and artifacts. Next, you will turn the balanced model into an interactive monitored app.

Turn the Model Into a Monitored App

Your two MLflow runs now make the model comparison reproducible. That evidence still lives behind a local tracking screen.

A tracked model is difficult to demonstrate to someone else. It also has no visible signal when incoming values shift.

In this step, Streamlit turns the model into an interactive scoring app. A lightweight data drift check exposes changes in the incoming batch.

In this step, get ready to:
  • Load the balanced model through a reusable cache.
  • Score synthetic transactions with an adjustable review threshold.
  • Compare incoming batches with the training reference.
Open the cached app shell

The app needs the balanced model from model.py. It also needs training rows that act as the monitoring reference.

  • In the VS Code Explorer sidebar, create app.py beside model.py.
  • Build the cached model loader by pasting this first block into app.py:
from statistics import fmean, pstdev

import streamlit as st

from model import FEATURE_NAMES, build_risk_model, split_dataset

st.set_page_config(page_title="FraudShield ML System", layout="wide")


@st.cache_resource
def load_model():
    X_train, _, y_train, _ = split_dataset()
    model = build_risk_model()
    model.fit(X_train, y_train)
    reference_rows = [list(row) for row in X_train]
    return model, reference_rows

What does this code do?

  • The imports bring in the statistical helpers needed for monitoring.
  • The page configuration gives FraudShield a wide layout.
  • The load_model() function fits the class-balanced model.
  • The cached result prevents widget interactions from fitting the model repeatedly.
  • The reference_rows value preserves training inputs for later drift comparisons.
  • Add the page shell beneath return model, reference_rows by pasting this block:
model, reference_rows = load_model()

st.title("FraudShield ML System")
st.write(
    "Synthetic demonstration only. The signals and decisions are not suitable "
    "for real financial or fraud-control use."
)

score_tab, monitor_tab = st.tabs(["Score transaction", "Monitor drift"])

What does the app shell show?

  • The model loader returns the fitted model plus its training reference.
  • The disclaimer clearly labels FraudShield as a synthetic demonstration.
  • The two tabs separate scoring from monitoring.
  • Save app.py.
  • Switch back to the activated PowerShell terminal that is not running the MLflow server.
  • Start the app shell by running this command:
python -m streamlit run app.py

What does this command do?

This command starts the local Streamlit server with app.py as the application entry point. Your browser loads the local FraudShield interface.

You should see the FraudShield ML System title above the synthetic-use disclaimer. You should also see the Score transaction tab beside the Monitor drift tab.

App shell not loading?

Confirm that the terminal still shows the activated .venv environment. Check that app.py sits beside model.py.

If the terminal points to a missing import or syntax problem, compare the affected line with the code above.

Help me diagnose my local FraudShield app.

Build the scoring tab

A probability becomes operational only when a policy uses it. The review threshold lets you change that policy without fitting another model.

  • Scroll to the end of app.py in the VS Code editor.
  • Add the scoring interface by pasting this block beneath the tab definition:
with score_tab:
    values = []
    for feature_name in FEATURE_NAMES:
        values.append(st.slider(feature_name, -5.0, 5.0, 0.0, 0.1))

    threshold = st.slider("Manual review threshold", 0.05, 0.95, 0.50, 0.05)
    risk = float(model.predict_proba([values])[0][1])

    st.metric("Estimated fraud risk", f"{risk:.1%}")
    if risk >= threshold:
        st.error("Send this transaction for manual review.")
    else:
        st.success("Approve this transaction under the current threshold.")

    st.write(
        "Move the threshold to see how an operational policy changes the decision "
        "without retraining the model."
    )

How does scoring work?

  • The feature loop creates one slider for every name in FEATURE_NAMES.
  • The values list becomes one synthetic transaction.
  • The model converts that transaction into the displayed risk value.
  • The threshold value decides whether the app approves the transaction or requests manual review.
  • Save app.py.
  • Return to the local FraudShield browser tab.
  • Select the Score transaction tab.
  • Move the synthetic signal sliders until the displayed risk sits inside the threshold range.
  • Move the Manual review threshold below the displayed risk.
  • Move the Manual review threshold above the displayed risk.

You should see the manual-review message when risk meets the threshold. You should see the approval message when the threshold exceeds the risk.

That is the scoring path working. The same model output now supports an adjustable operational decision.

Scoring controls missing?

Confirm that the entire block is indented beneath with score_tab:. Check that FEATURE_NAMES matches the import at the top of app.py.

Help me fix the FraudShield scoring tab.

Add drift monitoring

A monitoring reference captures the input values used to fit the model. Standardized mean shift compares each incoming feature with that reference.

FraudShield uses a project-defined score of 1.0 as the alert policy. The shifted scenario changes only the transaction-velocity feature.

  • In app.py, locate return model, reference_rows inside load_model().
  • Insert the drift helper before model, reference_rows = load_model() by pasting this block:
def calculate_drift(reference_rows, batch_rows):
    scores = []
    for index in range(len(FEATURE_NAMES)):
        reference_values = [row[index] for row in reference_rows]
        batch_values = [row[index] for row in batch_rows]
        score = abs(fmean(batch_values) - fmean(reference_values)) / (pstdev(reference_values) + 1e-9)
        scores.append(score)
    return scores

How is drift calculated?

  • The function compares one feature at a time across the reference rows and incoming batch.
  • The mean difference measures how far the batch has moved.
  • The reference standard deviation scales that difference to the feature's usual spread.
  • The resulting list contains one drift score for each feature.
  • Scroll to the end of app.py.
  • Add the monitoring controls by pasting this block beneath the scoring tab:
with monitor_tab:
    scenario = st.radio(
        "Incoming batch",
        ["Stable batch", "Shifted batch"],
        horizontal=True,
    )

    batch_rows = [row[:] for row in reference_rows[:400]]
    if scenario == "Shifted batch":
        for row in batch_rows:
            row[0] += 2.5

    drift_scores = calculate_drift(reference_rows, batch_rows)
    drift_rows = [
        {"Feature": feature_name, "Standardized mean shift": round(float(score), 2)}
        for feature_name, score in zip(FEATURE_NAMES, drift_scores)
    ]
    maximum_drift = max(drift_scores)

    st.dataframe(drift_rows, hide_index=True, width="stretch")
    st.metric("Maximum drift score", f"{maximum_drift:.2f}")

    if maximum_drift > 1.0:
        st.warning("Drift alert: investigate the incoming data and consider retraining.")
    else:
        st.success("The incoming batch is within this demo's drift threshold.")

What does the monitor compare?

  • The Stable batch scenario copies the first 400 reference rows.
  • The Shifted batch scenario adds 2.5 to the first feature in every copied row.
  • The table displays one standardized mean-shift score per feature.
  • The maximum score controls the visible health message.
  • Save app.py.
  • Return to the local FraudShield browser tab.
  • Select the Monitor drift tab.

Before you switch scenarios, do you expect every feature to drift or only the feature changed by the incoming batch?

  • Select Shifted batch under Incoming batch.

You should see the transaction-velocity row carry the largest shift. You should also see the drift warning beneath the maximum score.

The monitoring path is live. FraudShield now separates a model prediction from evidence that its input data has moved.

Drift warning not appearing?

Confirm that Shifted batch adds 2.5 to row[0]. Check that maximum_drift comes from the complete drift_scores list.

Help me debug the FraudShield drift monitor.

  • Add the monitoring policy beneath the final health message by pasting this block:
    st.write(
        "Project policy: investigate when any feature's standardized mean shift "
        "is greater than 1.0."
    )

Why show the policy?

The policy text makes the alert rule visible beside the monitoring result. A viewer can connect the maximum drift score with the fixed investigation threshold.

  • Save app.py.
  • Confirm that the monitoring tab now states the investigation policy beneath the health message.

✔️ Awesome, I've got everything!

Great. Double-check that you saved app.py.

ⓧ I'd like to double check the full code

from statistics import fmean, pstdev

import streamlit as st

from model import FEATURE_NAMES, build_risk_model, split_dataset

st.set_page_config(page_title="FraudShield ML System", layout="wide")


@st.cache_resource
def load_model():
    X_train, _, y_train, _ = split_dataset()
    model = build_risk_model()
    model.fit(X_train, y_train)
    reference_rows = [list(row) for row in X_train]
    return model, reference_rows


def calculate_drift(reference_rows, batch_rows):
    scores = []
    for index in range(len(FEATURE_NAMES)):
        reference_values = [row[index] for row in reference_rows]
        batch_values = [row[index] for row in batch_rows]
        score = abs(fmean(batch_values) - fmean(reference_values)) / (pstdev(reference_values) + 1e-9)
        scores.append(score)
    return scores


model, reference_rows = load_model()

st.title("FraudShield ML System")
st.write(
    "Synthetic demonstration only. The signals and decisions are not suitable "
    "for real financial or fraud-control use."
)

score_tab, monitor_tab = st.tabs(["Score transaction", "Monitor drift"])

with score_tab:
    values = []
    for feature_name in FEATURE_NAMES:
        values.append(st.slider(feature_name, -5.0, 5.0, 0.0, 0.1))

    threshold = st.slider("Manual review threshold", 0.05, 0.95, 0.50, 0.05)
    risk = float(model.predict_proba([values])[0][1])

    st.metric("Estimated fraud risk", f"{risk:.1%}")
    if risk >= threshold:
        st.error("Send this transaction for manual review.")
    else:
        st.success("Approve this transaction under the current threshold.")

    st.write(
        "Move the threshold to see how an operational policy changes the decision "
        "without retraining the model."
    )

with monitor_tab:
    scenario = st.radio(
        "Incoming batch",
        ["Stable batch", "Shifted batch"],
        horizontal=True,
    )

    batch_rows = [row[:] for row in reference_rows[:400]]
    if scenario == "Shifted batch":
        for row in batch_rows:
            row[0] += 2.5

    drift_scores = calculate_drift(reference_rows, batch_rows)
    drift_rows = [
        {"Feature": feature_name, "Standardized mean shift": round(float(score), 2)}
        for feature_name, score in zip(FEATURE_NAMES, drift_scores)
    ]
    maximum_drift = max(drift_scores)

    st.dataframe(drift_rows, hide_index=True, width="stretch")
    st.metric("Maximum drift score", f"{maximum_drift:.2f}")

    if maximum_drift > 1.0:
        st.warning("Drift alert: investigate the incoming data and consider retraining.")
    else:
        st.success("The incoming batch is within this demo's drift threshold.")

    st.write(
        "Project policy: investigate when any feature's standardized mean shift "
        "is greater than 1.0."
    )
  • Stop the current app from its PowerShell terminal.

Before you restart the finished app, do you expect moving only the review threshold to change the model's score or only its displayed decision?

  • Restart the complete monitored app by running this command:
python -m streamlit run app.py

What does the final run verify?

This run loads the completed app.py from a clean application start. It verifies the scoring interface plus the monitoring interface together.

  • Select the Score transaction tab.
  • Adjust the signal sliders until the displayed risk sits inside the threshold range.
  • Move the review threshold below the displayed risk.
  • Move the review threshold above the displayed risk.

You should see the decision switch between manual review and approval. The estimated risk should stay fixed while only the threshold moves.

  • Select the Monitor drift tab.
  • Select Stable batch.

You should see a low maximum drift score. You should also see the healthy batch message.

Before you switch the incoming batch, do you expect the maximum drift score to remain below the project policy?

  • Select Shifted batch.

You should see the maximum drift score exceed 1.0. You should also see the warning to investigate the incoming data.

Your local FraudShield app can now score transactions plus monitor incoming data. Next, you will connect it to GitHub for a public deployment.

Deploy and Prove Continuous Delivery

Your local Streamlit app now scores synthetic transactions. It also flags shifted monitoring batches.

The app still lives only on your computer. A one-time deployment also cannot prove that later source changes reach the live system.

This step publishes your local Git repository through GitHub. Streamlit Community Cloud then turns pushed commits into visible updates through continuous delivery.

In this step, get ready to:
  • Publish the existing project to a public GitHub repository.
  • Deploy FraudShield as a public Streamlit Community Cloud app.
  • Push an introductory copy change through the continuous delivery workflow.
Push the project to GitHub

A Git commit creates a reproducible snapshot of the project files. GitHub gives that snapshot a remote home that Streamlit Community Cloud can deploy.

  • Stage the project files and create the first commit by running these commands:
git add .
git commit -m "Build FraudShield ML system"

What do these commands do?

  • The first command stages the project files while respecting the exclusions in .gitignore.
  • The second command records the staged files on the main branch with a descriptive message.
  • Confirm that the terminal reports a new root commit.

Your source files now have a local history. The ignored .venv, mlflow.db, and mlruns resources remain on your computer.

Commit not created?

Check that the terminal is still inside the FraudShield project folder. Git may also ask you to configure an author identity before it creates your first commit.

Use the terminal output to identify the missing setting. Help me resolve the Git commit error shown in my terminal.

  • Return to GitHub in your browser.
  • Start a new repository from the creation menu.
  • Enter fraudshield as the repository name.
  • Choose Public as the repository visibility.
  • Leave the README option unselected.
  • Leave the license option unselected.
  • Leave the .gitignore option unselected.
  • Create the empty repository.

GitHub now shows an empty repository with instructions for publishing existing local code.

  • Copy the repository remote URL from the repository page.
  • Record the copied URL here: REMOTE-URL.

Before you push, do you expect the empty GitHub repository to gain the same source files that are committed locally?

  • Replace REMOTE-URL below with your recorded repository URL.
  • Publish main to the new remote by running these commands:
git remote add origin REMOTE-URL
git push -u origin main

What do these commands do?

  • The first command names your GitHub repository origin.
  • The second command publishes the local main branch. It also links future pushes to the same remote branch.
  • Complete the GitHub authentication flow if Git requests access.
  • Refresh the GitHub repository page after the push finishes.

You should see app.py, model.py, baseline.py, train.py, and both requirements files on the main branch. Your project now has the remote source that powers deployment.

Push did not reach GitHub?

Check that you replaced REMOTE-URL with the copied repository URL. Complete any browser authentication request before retrying the push.

If the remote name already exists, inspect the current remote before changing it. Help me diagnose why my GitHub push failed.

Deploy the public Streamlit app

Streamlit Community Cloud reads the project directly from GitHub. The repository branch and entry file tell the service which version of FraudShield to run.

  • Go to Streamlit Community Cloud in your browser.
  • Click Create app.
  • Choose Yup, I have an app.
  • Select the fraudshield repository.
  • Select main as the branch.
  • Enter app.py as the file path.
  • Open Advanced settings.
  • Select Python 3.12.

Why these deployment settings?

The main branch contains the committed project. The app.py file is the Streamlit entry point.

Python 3.12 matches the project environment. This keeps the local and cloud runtimes aligned.

  • Click Deploy.
  • Wait for the deployment process to finish.
  • Copy the public .streamlit.app URL.
  • Record the public URL here: your public .streamlit.app URL.

The public page should show the FraudShield title and synthetic-use disclaimer.

Before you test the public app, do you expect its scoring decisions and drift warning to match the local version?

  • Open your public .streamlit.app URL in a browser tab.
  • Select the Score transaction tab.
  • Move the Manual review threshold slider until the displayed decision changes.
  • Select the Monitor drift tab.
  • Choose Shifted batch.

You should see a fraud-risk score in the first tab. The second tab should show a drift warning for the shifted batch.

Public app did not load?

  • Open Manage app from the deployed app.
  • Read the Cloud logs to identify the failed dependency or startup step.
  • Confirm that the deployment uses main, app.py, and Python 3.12.

Use the first relevant log entry as your debugging clue. Help me diagnose my Streamlit Community Cloud deployment logs.

Prove continuous delivery

The deployed app now works from a public URL. A visible source change tests whether a pushed commit reaches that URL automatically.

  • Return to app.py in the editor from earlier.
  • Find the introductory st.write block directly below st.title("FraudShield ML System"):
st.write(
    "Synthetic demonstration only. The signals and decisions are not suitable "
    "for real financial or fraud-control use."
)

Why change this block?

This text appears near the top of both the local and public app. Its position makes the deployment update easy to spot without changing model behavior.

  • Keep the surrounding st.write call in place.
  • Replace its two string lines with these lines:
        "Move the threshold to see how an operational policy changes the decision "
        "without retraining the model."

What changes in the app?

The introductory copy now directs visitors toward the threshold demonstration. The scoring and monitoring logic stay unchanged.

  • Save app.py.
  • Switch back to the local Streamlit app from earlier.
  • Refresh the browser page.

You should see the threshold message below the FraudShield title. This local result proves the edited code still runs.

Local message did not change?

Confirm that you edited the first st.write block below the title. Refresh the local browser after saving app.py.

Check that both replacement string lines remain inside the same parentheses. Help me fix the introductory st.write update in app.py.

Before you push this commit, do you think the public page will keep its old message or receive the new threshold message?

  • Commit the message change and push it to main by running these commands:
git add .
git commit -m "Update deployment message"
git push origin main

What happens after the push?

GitHub receives the new commit on main. Streamlit Community Cloud uses that branch as the app source.

The pushed copy change gives you a visible test of the delivery path.

  • Return to your public .streamlit.app URL.
  • Refresh the page after Streamlit Community Cloud processes the commit.

You should see the threshold message below the title. That visible update proves a Git push can change the live app.

Cloud app still shows the old message?

Open Manage app. Check the Cloud logs for the latest deployment activity.

Confirm that GitHub shows the Update deployment message commit on main. Help me trace why my pushed Streamlit change is not live.

The synthetic-use disclaimer keeps visitors from treating the demonstration as a real fraud-control system. Restore that copy now that the delivery path has been tested.

  • Return to the introductory st.write block in app.py.
  • Find the temporary threshold message:
        "Move the threshold to see how an operational policy changes the decision "
        "without retraining the model."

Why restore the disclaimer?

The temporary copy made the deployment delta obvious. The final disclaimer clearly labels the app as a synthetic demonstration.

  • Replace the temporary string lines with the final disclaimer lines:
    "Synthetic demonstration only. The signals and decisions are not suitable "
    "for real financial or fraud-control use."

What does the final copy protect?

The disclaimer defines the app's intended scope for every visitor. It keeps the synthetic signals separate from real financial decision-making.

  • Save app.py.
  • Refresh the local Streamlit app.

You should see the synthetic-use disclaimer below the FraudShield title again.

✔️ Awesome, I've got everything!

Great. Double-check that app.py is saved before the final push.

ⓧ I'd like to double check the full code

from statistics import fmean, pstdev

import streamlit as st

from model import FEATURE_NAMES, build_risk_model, split_dataset

st.set_page_config(page_title="FraudShield ML System", layout="wide")


@st.cache_resource
def load_model():
    X_train, _, y_train, _ = split_dataset()
    model = build_risk_model()
    model.fit(X_train, y_train)
    reference_rows = [list(row) for row in X_train]
    return model, reference_rows


def calculate_drift(reference_rows, batch_rows):
    scores = []
    for index in range(len(FEATURE_NAMES)):
        reference_values = [row[index] for row in reference_rows]
        batch_values = [row[index] for row in batch_rows]
        score = abs(fmean(batch_values) - fmean(reference_values)) / (pstdev(reference_values) + 1e-9)
        scores.append(score)
    return scores


model, reference_rows = load_model()

st.title("FraudShield ML System")
st.write(
    "Synthetic demonstration only. The signals and decisions are not suitable "
    "for real financial or fraud-control use."
)

score_tab, monitor_tab = st.tabs(["Score transaction", "Monitor drift"])

with score_tab:
    values = []
    for feature_name in FEATURE_NAMES:
        values.append(st.slider(feature_name, -5.0, 5.0, 0.0, 0.1))

    threshold = st.slider("Manual review threshold", 0.05, 0.95, 0.50, 0.05)
    risk = float(model.predict_proba([values])[0][1])

    st.metric("Estimated fraud risk", f"{risk:.1%}")
    if risk >= threshold:
        st.error("Send this transaction for manual review.")
    else:
        st.success("Approve this transaction under the current threshold.")

    st.write(
        "Move the threshold to see how an operational policy changes the decision "
        "without retraining the model."
    )

with monitor_tab:
    scenario = st.radio(
        "Incoming batch",
        ["Stable batch", "Shifted batch"],
        horizontal=True,
    )

    batch_rows = [row[:] for row in reference_rows[:400]]
    if scenario == "Shifted batch":
        for row in batch_rows:
            row[0] += 2.5

    drift_scores = calculate_drift(reference_rows, batch_rows)
    drift_rows = [
        {"Feature": feature_name, "Standardized mean shift": round(float(score), 2)}
        for feature_name, score in zip(FEATURE_NAMES, drift_scores)
    ]
    maximum_drift = max(drift_scores)

    st.dataframe(drift_rows, hide_index=True, width="stretch")
    st.metric("Maximum drift score", f"{maximum_drift:.2f}")

    if maximum_drift > 1.0:
        st.warning("Drift alert: investigate the incoming data and consider retraining.")
    else:
        st.success("The incoming batch is within this demo's drift threshold.")

    st.write(
        "Project policy: investigate when any feature's standardized mean shift "
        "is greater than 1.0."
    )

Before the final push, which introductory message do you expect the public app to show after the new commit is processed?

  • Commit the restored disclaimer and push it to main by running these commands:
git add .
git commit -m "Update deployment message"
git push origin main

What does this final push prove?

This push sends another source change through the same GitHub-backed deployment path. The live app should return to its intended disclaimer without a manual redeployment.

  • Return to your public .streamlit.app URL.
  • Refresh the page after the latest commit is processed.
  • Test the Score transaction tab.
  • Test the Monitor drift tab with Shifted batch selected.

You should see the synthetic-use disclaimer below the title. Both tabs should still work with the fixed stable-or-shifted monitoring controls.

You have now proved the full delivery loop. A committed source change travelled from GitHub to the public app without changing the deployment settings.

Secret mission

Build a Drift Incident Simulator

Control incident severity with one slider. Set a retraining threshold with another slider. Push the finished simulator to your public app.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project has no ongoing cost because its resources are local or hosted through the free Streamlit Community Cloud service.

Resources you used:

  • The local Git repository folder that contains app.py, .venv/, mlflow.db, and mlruns/.
  • The local MLflow server, if its terminal is still running.
  • The local Streamlit app process, if its terminal is still running.
  • The public GitHub repository on the main branch.
  • The public Streamlit Community Cloud app deployed from app.py.

Keep everything running

No action needed. Choose this if you are still developing FraudShield or sharing the public incident simulator.

  • Keep the local Git repository folder to preserve your source code plus experiment history.
  • Leave the MLflow PowerShell terminal active to keep the local tracking server available.
  • Leave the Streamlit PowerShell terminal active to keep the local app available.
  • Keep the GitHub repository to preserve the source for future deployments.
  • Keep the Streamlit Community Cloud app to leave the public demo available.

Pause - I'll come back to this later

Pausing closes the local processes while preserving every file plus public resource.

  • Press Ctrl+C in the PowerShell terminal if the local MLflow server is still running.
  • Press Ctrl+C in the Streamlit PowerShell terminal if the local app is still running.
  • Keep the local repository folder to preserve the virtual environment plus MLflow history.
  • Leave the GitHub repository in place.
  • Leave the Streamlit Community Cloud app in place.

Your local processes are now stopped. Every project artifact remains ready for your return.

After 12 hours without traffic, the Streamlit Community Cloud app sleeps until another visit wakes it.

Delete - I don't want to use this again

Deletion permanently removes the public app, the repository, and your local experiment history. Choose this only when you no longer need the FraudShield project.

  • Press Ctrl+C in the MLflow PowerShell terminal if its local server is still running.
  • Press Ctrl+C in the Streamlit PowerShell terminal if its local app is still running.
  • Use the app's workspace overflow menu to delete the Streamlit Community Cloud app.
  • Use the repository deletion control in the GitHub repository settings to delete the project repository.
  • In File Explorer, return to the local repository folder that contains app.py.
  • Delete that folder to remove the source files, .venv/, mlflow.db, and mlruns/.

That's the cleanup complete. FraudShield no longer has a local or public footprint.

Nice Work!

Nice Work!

You did it! Your public FraudShield demo now shows an operational fraud-risk workflow in action.

What you learned:

  • Exposed the accuracy trap in imbalanced classification by comparing accuracy with fraud recall and average precision.
  • Tracked two reproducible experiments in MLflow. Compared the majority baseline with a class-balanced model using logged parameters, metrics, and model artifacts.
  • Built a public Streamlit app for transaction scoring and data drift monitoring. Connected GitHub pushes to the deployed app through continuous delivery.
  • Secret Mission: Converted the fixed drift check into a configurable incident simulator. Added an adjustable retraining trigger that turns monitoring policy into a visible operational decision.

Ready to quiz yourself?