Red Team a Medical TinyML Model

Expose the quantization robustness gap in a medical ECG classifier.

Introduction

30 Second Summary

Wearable heart monitors and implantable ECG devices increasingly rely on tiny AI models to detect life-threatening arrhythmias. When these models are compressed to fit on a microcontroller, that compression can silently strip away their defenses against adversarial attacks.

In this project, you will build a red teaming pipeline that trains an ECG anomaly classifier, quantizes it for edge deployment, then systematically attacks both model versions to prove that adversarial robustness degrades after INT8 quantization.

What You'll Build

A Jupyter Notebook-based red teaming pipeline that trains a 1D CNN on the PTB-XL ECG dataset, quantizes it to TFLite INT8, and attacks both model versions using IBM's Adversarial Robustness Toolbox.

By the end of this project, you'll have:

  • A trained 1D CNN ECG classifier that distinguishes Normal heartbeats from Myocardial Infarction on the PTB-XL dataset.
  • A comparative adversarial robustness report showing accuracy degradation under FGSM, PGD, and DeepFool attacks on both the full-precision and quantized models.
  • A quantifiable measurement of the quantization robustness gap proving that INT8 compression weakens adversarial defenses for edge-deployed medical AI.
  • Secret Mission: Implement adversarial training as a defense and measure whether it survives quantization.

Are there any prerequisites?

You should be comfortable writing Python and have a basic understanding of how CNNs work. Familiarity with adversarial ML concepts like FGSM and PGD is also expected.

Train the ECG Anomaly Classifier

Before you can attack a model, you need a model worth attacking. In this step, you will train a 1D CNN that classifies ECG waveforms as either Normal or Myocardial Infarction.

This is the exact type of safety-critical classifier that would run on a medical wearable. If an adversarial perturbation can flip an MI reading to Normal, a patient loses their early warning.

In this step, get ready to:
  • Prepare the PTB-XL dataset for binary classification (Normal vs MI).
  • Build and train a 1D CNN ECG classifier.
  • Save the trained model for later quantization and attack.
Prepare the dataset for binary classification

The PTB-XL dataset contains 21,799 ECG recordings across multiple diagnostic categories. You need to narrow it down to just two classes for a clean binary classification task.

  • In your notebook, add a new cell and load the PTB-XL metadata by pasting this code:
# Cell 2: Load PTB-XL Dataset
# Set path to your extracted PTB-XL dataset
PTBXL_PATH = './ptb-xl-a-large-publicly-available-electrocardiography-dataset-1.0.3/'

# Load metadata
ptbxl_data = pd.read_csv(os.path.join(PTBXL_PATH, 'ptbxl_database.csv'), index_col='ecg_id')
ptbxl_data.scp_codes = ptbxl_data.scp_codes.apply(lambda x: ast.literal_eval(x))

# Load SCP statements mapping
scp_statements = pd.read_csv(os.path.join(PTBXL_PATH, 'scp_statements.csv'), index_col=0)
scp_statements = scp_statements[scp_statements.diagnostic == 1]

print(f"Total records: {len(ptbxl_data)}")
print(f"Diagnostic SCP statements: {len(scp_statements)}")

What does this code do?

  • PTBXL_PATH points to your extracted dataset folder.
  • ptbxl_database.csv contains metadata for all 21,799 ECG records, including SCP diagnostic codes stored as string dictionaries.
  • ast.literal_eval converts those string dictionaries back into actual Python dictionaries.
  • scp_statements.csv maps SCP codes to their diagnostic superclass. Filtering for diagnostic == 1 keeps only codes relevant to diagnosis.
  • Run the cell. You should see Total records: 21799 printed in the output.

Seeing a FileNotFoundError?

Double-check that the path in PTBXL_PATH exactly matches your extracted folder name. The folder name is long. Copy it directly from your file browser.

help me fix a FileNotFoundError when loading the PTB-XL dataset

  • Add another new cell and paste this code to map labels and filter for Normal vs MI:
# Cell 3: Map Labels to Diagnostic Superclasses
def aggregate_diagnostic(scp_codes):
    result = []
    for key, likelihood in scp_codes.items():
        if key in scp_statements.index and likelihood >= 50:
            result.append(scp_statements.loc[key].diagnostic_class)
    return list(set(result))

ptbxl_data['diagnostic_superclass'] = ptbxl_data.scp_codes.apply(aggregate_diagnostic)

# Filter for binary classification: NORM vs MI
def get_binary_label(superclass_list):
    if 'NORM' in superclass_list and len(superclass_list) == 1:
        return 0  # Normal
    elif 'MI' in superclass_list and len(superclass_list) == 1:
        return 1  # Myocardial Infarction
    else:
        return -1  # Exclude

ptbxl_data['binary_label'] = ptbxl_data.diagnostic_superclass.apply(get_binary_label)
binary_data = ptbxl_data[ptbxl_data.binary_label >= 0].copy()
print(f"Binary classification samples: {len(binary_data)}")
print(f"Normal: {(binary_data.binary_label == 0).sum()}, MI: {(binary_data.binary_label == 1).sum()}")

What does this code do?

aggregate_diagnostic maps each record's SCP codes to their diagnostic superclass (NORM, MI, STTC, CD, or HYP) using the scp_statements lookup table. Only codes with likelihood >= 50 are counted.

get_binary_label then filters down to records with a single, unambiguous label. Normal gets label 0. MI gets label 1. Anything else (multi-label or other conditions) is excluded with -1.

  • Run the cell. You should see the total binary classification sample count, plus a breakdown showing Normal and MI counts.

Why only NORM and MI?

Myocardial Infarction is the highest-stakes misclassification target. A missed MI means a patient does not receive urgent cardiac intervention.

By restricting to single-label records, you ensure clean ground truth. Records with multiple overlapping conditions would add ambiguity that confuses the adversarial robustness analysis.

  • Add a new cell and paste this code to load the actual waveform data and split it:
# Cell 4: Load Waveform Data
def load_ecg_data(df, path, sampling_rate=100):
    if sampling_rate == 100:
        data = [wfdb.rdsamp(os.path.join(path, f))[0] for f in df.filename_lr]
    else:
        data = [wfdb.rdsamp(os.path.join(path, f))[0] for f in df.filename_hr]
    return np.array(data)

# Load 100 Hz data (shape: N x 1000 x 12)
X = load_ecg_data(binary_data, PTBXL_PATH, sampling_rate=100)
y = binary_data.binary_label.values

# Apply PTB-XL recommended split
train_folds = binary_data.strat_fold <= 8
val_fold = binary_data.strat_fold == 9
test_fold = binary_data.strat_fold == 10

X_train, y_train = X[train_folds], y[train_folds]
X_val, y_val = X[val_fold], y[val_fold]
X_test, y_test = X[test_fold], y[test_fold]

# One-hot encode labels
y_train_onehot = keras.utils.to_categorical(y_train, 2)
y_val_onehot = keras.utils.to_categorical(y_val, 2)
y_test_onehot = keras.utils.to_categorical(y_test, 2)

print(f"Train: {X_train.shape}, Val: {X_val.shape}, Test: {X_test.shape}")

What does this code do?

  • load_ecg_data reads each waveform file using wfdb.rdsamp(). The 100 Hz version gives 1000 time steps across 12 leads per record.
  • The PTB-XL recommended split uses strat_fold: folds 1-8 for training, fold 9 for validation, fold 10 for test. This ensures no patient data leaks across splits.
  • to_categorical converts integer labels (0 or 1) into one-hot vectors ([1,0] or [0,1]) for the softmax output layer.
  • Run the cell. This will take a minute or two to load all waveform files.
  • You should see the shapes printed, for example: Train: (N, 1000, 12), Val: (N, 1000, 12), Test: (N, 1000, 12) where N varies per split.

Cell taking too long or running out of memory?

Loading thousands of waveform files takes time. On a machine with 8 GB RAM, expect 1-3 minutes. If you run out of memory, restart the kernel, re-run all cells, and close other memory-heavy applications.

help me debug slow or memory issues when loading PTB-XL waveforms

  • Add a new cell and paste this code to normalize the waveform data:
# Cell 5: Normalize Data
# Compute normalization parameters from training data
train_mean = X_train.mean()
train_std = X_train.std()

X_train_norm = (X_train - train_mean) / train_std
X_val_norm = (X_val - train_mean) / train_std
X_test_norm = (X_test - train_mean) / train_std

# Record clip values for ART
clip_min = X_train_norm.min()
clip_max = X_train_norm.max()
print(f"Clip values: ({clip_min:.4f}, {clip_max:.4f})")

What does this code do?

Normalization (zero-mean, unit-variance) is computed from training data only. Applying the same train_mean and train_std to val and test prevents data leakage.

clip_min and clip_max record the range of normalized values. ART needs these later to constrain adversarial perturbations within a valid input range.

  • Run the cell. You should see two clip values printed (negative and positive floats).
Build and train a 1D CNN model

With the data prepared, you can now define the neural network that will learn to distinguish Normal ECGs from Myocardial Infarction.

  • Add a new cell and paste this code to define the 1D CNN architecture:
# Cell 6: Build 1D CNN Model
def build_ecg_cnn(input_shape=(1000, 12), num_classes=2):
    model = keras.Sequential([
        layers.Conv1D(32, kernel_size=5, activation='relu', input_shape=input_shape),
        layers.MaxPooling1D(pool_size=2),
        layers.Conv1D(64, kernel_size=5, activation='relu'),
        layers.MaxPooling1D(pool_size=2),
        layers.Conv1D(128, kernel_size=3, activation='relu'),
        layers.GlobalAveragePooling1D(),
        layers.Dense(64, activation='relu'),
        layers.Dense(num_classes, activation='softmax')
    ])
    return model

model = build_ecg_cnn()
model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['accuracy']
)
model.summary()

What does this architecture do?

  • Three Conv1D layers (32, 64, 128 filters) scan across the 1000-timestep ECG signal to detect local patterns like QRS complexes and ST-segment changes.
  • MaxPooling1D layers reduce the sequence length by half after each convolution, building progressively abstract features.
  • GlobalAveragePooling1D collapses the remaining sequence into a single 128-dimensional vector, avoiding the parameter explosion of a flatten layer.
  • The final Dense(2, softmax) outputs probabilities for Normal vs MI.
  • Run the cell. You should see a model summary table with layer names, output shapes, and parameter counts.

Seeing a shape mismatch error?

Make sure input_shape=(1000, 12) matches the data shape from Cell 4. If you accidentally loaded 500 Hz data (shape 5000 x 12), the model input will not align.

help me debug a shape mismatch in my Keras 1D CNN model

  • Add a new cell and paste this code to train the model:
# Cell 7: Train the Model
history = model.fit(
    X_train_norm, y_train_onehot,
    validation_data=(X_val_norm, y_val_onehot),
    epochs=20,
    batch_size=64,
    verbose=1
)

# Evaluate on clean test data
test_loss, test_accuracy = model.evaluate(X_test_norm, y_test_onehot)
print(f"\nClean test accuracy (full-precision): {test_accuracy:.4f}")

What does this code do?

model.fit trains for 20 epochs (full passes through the training data) with a batch size of 64 samples. The validation data provides an accuracy check at the end of each epoch without influencing training.

model.evaluate runs the trained model on the held-out test set (fold 10). This gives the baseline clean accuracy. You need this number to measure how much adversarial attacks degrade performance later.

  • Run the cell. Training will take several minutes depending on your hardware.
  • Watch the epoch-by-epoch output. Validation accuracy should climb and stabilize.
  • At the end, you should see the clean test accuracy printed. It should be above 0.85 (85%).

Accuracy stuck below 60%?

If accuracy does not improve past 60%, confirm that Cell 5 ran correctly and X_train_norm is the normalized array being passed to model.fit. Passing unnormalized data to this architecture causes training to stall.

help me debug why my CNN training accuracy is stuck and not improving

Save the trained model
  • Add a new cell and paste this code to save the trained model to disk:
# Cell 8: Save the Trained Model
model.save('ecg_model.keras')
print("Model saved as ecg_model.keras")

Why save the model now?

You save the trained model so you can load it later without retraining. In the next step, you will convert this saved model to TFLite INT8 format. Saving also gives you a checkpoint to return to if later cells corrupt the model state.

  • Run the cell. You should see Model saved as ecg_model.keras printed in the output.
  • Confirm the file exists by running this in a new cell:
import os
print(os.path.exists('ecg_model.keras'))
  • You should see True printed, confirming the file was saved.

✔️ Awesome, I've got everything!

Great. Make sure your notebook is saved (Ctrl+S or Cmd+S). You now have a trained ECG classifier ready for quantization and adversarial attack.

ⓧ I'd like to double check the full code

Here is the full notebook up to this point. Compare your cells against this:

# Cell 1: Imports and Setup
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import wfdb
import ast
import os
from sklearn.metrics import accuracy_score, classification_report

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

from art.estimators.classification import TensorFlowV2Classifier
from art.attacks.evasion import FastGradientMethod, ProjectedGradientDescent, DeepFool

print(f"TensorFlow version: {tf.__version__}")
print(f"GPU available: {tf.config.list_physical_devices('GPU')}")

# Cell 2: Load PTB-XL Dataset
PTBXL_PATH = './ptb-xl-a-large-publicly-available-electrocardiography-dataset-1.0.3/'

ptbxl_data = pd.read_csv(os.path.join(PTBXL_PATH, 'ptbxl_database.csv'), index_col='ecg_id')
ptbxl_data.scp_codes = ptbxl_data.scp_codes.apply(lambda x: ast.literal_eval(x))

scp_statements = pd.read_csv(os.path.join(PTBXL_PATH, 'scp_statements.csv'), index_col=0)
scp_statements = scp_statements[scp_statements.diagnostic == 1]

print(f"Total records: {len(ptbxl_data)}")
print(f"Diagnostic SCP statements: {len(scp_statements)}")

# Cell 3: Map Labels to Diagnostic Superclasses
def aggregate_diagnostic(scp_codes):
    result = []
    for key, likelihood in scp_codes.items():
        if key in scp_statements.index and likelihood >= 50:
            result.append(scp_statements.loc[key].diagnostic_class)
    return list(set(result))

ptbxl_data['diagnostic_superclass'] = ptbxl_data.scp_codes.apply(aggregate_diagnostic)

def get_binary_label(superclass_list):
    if 'NORM' in superclass_list and len(superclass_list) == 1:
        return 0
    elif 'MI' in superclass_list and len(superclass_list) == 1:
        return 1
    else:
        return -1

ptbxl_data['binary_label'] = ptbxl_data.diagnostic_superclass.apply(get_binary_label)
binary_data = ptbxl_data[ptbxl_data.binary_label >= 0].copy()
print(f"Binary classification samples: {len(binary_data)}")
print(f"Normal: {(binary_data.binary_label == 0).sum()}, MI: {(binary_data.binary_label == 1).sum()}")

# Cell 4: Load Waveform Data
def load_ecg_data(df, path, sampling_rate=100):
    if sampling_rate == 100:
        data = [wfdb.rdsamp(os.path.join(path, f))[0] for f in df.filename_lr]
    else:
        data = [wfdb.rdsamp(os.path.join(path, f))[0] for f in df.filename_hr]
    return np.array(data)

X = load_ecg_data(binary_data, PTBXL_PATH, sampling_rate=100)
y = binary_data.binary_label.values

train_folds = binary_data.strat_fold <= 8
val_fold = binary_data.strat_fold == 9
test_fold = binary_data.strat_fold == 10

X_train, y_train = X[train_folds], y[train_folds]
X_val, y_val = X[val_fold], y[val_fold]
X_test, y_test = X[test_fold], y[test_fold]

y_train_onehot = keras.utils.to_categorical(y_train, 2)
y_val_onehot = keras.utils.to_categorical(y_val, 2)
y_test_onehot = keras.utils.to_categorical(y_test, 2)

print(f"Train: {X_train.shape}, Val: {X_val.shape}, Test: {X_test.shape}")

# Cell 5: Normalize Data
train_mean = X_train.mean()
train_std = X_train.std()

X_train_norm = (X_train - train_mean) / train_std
X_val_norm = (X_val - train_mean) / train_std
X_test_norm = (X_test - train_mean) / train_std

clip_min = X_train_norm.min()
clip_max = X_train_norm.max()
print(f"Clip values: ({clip_min:.4f}, {clip_max:.4f})")

# Cell 6: Build 1D CNN Model
def build_ecg_cnn(input_shape=(1000, 12), num_classes=2):
    model = keras.Sequential([
        layers.Conv1D(32, kernel_size=5, activation='relu', input_shape=input_shape),
        layers.MaxPooling1D(pool_size=2),
        layers.Conv1D(64, kernel_size=5, activation='relu'),
        layers.MaxPooling1D(pool_size=2),
        layers.Conv1D(128, kernel_size=3, activation='relu'),
        layers.GlobalAveragePooling1D(),
        layers.Dense(64, activation='relu'),
        layers.Dense(num_classes, activation='softmax')
    ])
    return model

model = build_ecg_cnn()
model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['accuracy']
)
model.summary()

# Cell 7: Train the Model
history = model.fit(
    X_train_norm, y_train_onehot,
    validation_data=(X_val_norm, y_val_onehot),
    epochs=20,
    batch_size=64,
    verbose=1
)

test_loss, test_accuracy = model.evaluate(X_test_norm, y_test_onehot)
print(f"\nClean test accuracy (full-precision): {test_accuracy:.4f}")

# Cell 8: Save the Trained Model
model.save('ecg_model.keras')
print("Model saved as ecg_model.keras")

Your ECG classifier is trained and saved. Next, you will quantize this model to TFLite INT8 format to simulate what it would look like running on an STM32 microcontroller.

Quantize the Model to TFLite INT8

Your trained CNN achieves strong accuracy on clean ECG data. However, this model lives as a 32-bit floating-point file that is far too large and slow for a medical microcontroller like the STM32.

In this step, you will convert the model to TFLite INT8 format using post-training quantization. This compresses every weight and activation from 32-bit floats down to 8-bit integers. The conversion shrinks the model dramatically, but it can also silently degrade accuracy.

In this step, get ready to:
  • Define a representative dataset for quantization calibration.
  • Convert the model to TFLite with full INT8 quantization.
  • Verify the quantized model's accuracy on clean test data.
Define a representative dataset for quantization calibration

INT8 quantization needs to know the typical range of values flowing through the model. A representative dataset provides real input samples so the converter can calibrate its mapping from float32 to int8.

  • Create a new cell in ecg_redteam.ipynb (Cell 9) and define the representative dataset generator by entering this code:
# Cell 9: Quantize to TFLite INT8
def representative_dataset_gen():
    for i in range(200):
        sample = X_train_norm[i:i+1].astype(np.float32)
        yield [sample]

What does this code do?

This generator yields 200 real ECG samples from the training data, one at a time. The TFLite converter will feed each sample through the model to observe the range of activations at every layer.

Each sample is cast to np.float32 because the converter expects 32-bit inputs during calibration. The 200-sample count balances calibration quality against conversion time.

You won't see output yet. The generator is used in the next chunk when you run the converter.

Convert to TFLite with full integer quantization

The TFLite converter takes your Keras model and produces a flatbuffer file where every operation runs in 8-bit integer arithmetic. This is exactly what STM32 microcontrollers execute via X-CUBE-AI.

  • Add the converter configuration directly below the generator function in the same cell (Cell 9):
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8

tflite_model = converter.convert()

with open('ecg_model_int8.tflite', 'wb') as f:
    f.write(tflite_model)

print(f"Quantized model size: {len(tflite_model) / 1024:.1f} KB")

What does this code do?

  • from_keras_model(model) creates a converter from your already-trained model in memory.
  • tf.lite.Optimize.DEFAULT enables post-training quantization optimizations.
  • TFLITE_BUILTINS_INT8 restricts the model to only INT8-compatible ops, ensuring it runs entirely in integer arithmetic on the microcontroller.
  • Setting both inference_input_type and inference_output_type to tf.int8 means the model accepts and returns int8 tensors. No float conversion happens at inference time.
  • Run the full Cell 9. The conversion takes 10-30 seconds while it calibrates with your 200 representative samples.
  • You should see the quantized model size printed (typically 30-60 KB). Your original Keras model was several hundred KB.
  • Confirm that ecg_model_int8.tflite now exists in your project folder.

Seeing a conversion error?

  • If you see a shape mismatch error, make sure representative_dataset_gen yields samples with shape (1, 1000, 12). The i:i+1 slice keeps the batch dimension.
  • If the kernel restarts during conversion, you may be running low on memory. Restart the kernel, re-run all previous cells, and try again.

Help me debug my TFLite INT8 conversion error.

Verify quantized model accuracy

Quantization introduces rounding errors. Before testing adversarial robustness, you need to know how much accuracy the model lost from quantization alone. This establishes the baseline for comparing adversarial degradation later.

  • Create a new cell (Cell 10) and define an evaluation function for the TFLite model, then run it on the clean test set:
# Cell 10: Evaluate Quantized Model on Clean Data
def evaluate_tflite(tflite_path, X_data, y_true):
    interpreter = tf.lite.Interpreter(model_path=tflite_path)
    interpreter.allocate_tensors()

    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()

    input_scale, input_zero_point = input_details[0]['quantization']

    predictions = []
    for i in range(len(X_data)):
        sample = X_data[i:i+1].astype(np.float32)
        # Quantize input
        sample_int8 = (sample / input_scale + input_zero_point).astype(np.int8)
        interpreter.set_tensor(input_details[0]['index'], sample_int8)
        interpreter.invoke()
        output = interpreter.get_tensor(output_details[0]['index'])
        predictions.append(np.argmax(output))

    accuracy = accuracy_score(y_true, predictions)
    return accuracy

quant_accuracy = evaluate_tflite('ecg_model_int8.tflite', X_test_norm, y_test)
print(f"Quantized model clean accuracy: {quant_accuracy:.4f}")
print(f"Accuracy drop from quantization: {test_accuracy - quant_accuracy:.4f}")

What does this code do?

  • evaluate_tflite() loads the quantized model with TFLite's Interpreter, then runs inference on each test sample individually.
  • The input_scale and input_zero_point values come from the model's quantization parameters. They define how to map float inputs to int8.
  • Each sample is manually quantized to int8 before being fed to the interpreter. This mimics exactly what happens on an STM32 microcontroller at runtime.
  • The accuracy drop from quantization alone (before any attacks) is your reference point. You will compare this to the much larger drops caused by adversarial attacks in the next step.
  • Run Cell 10. This takes a minute or two because it evaluates each test sample individually through the interpreter.
  • You should see the quantized model's clean accuracy printed (typically 1-3% below the full-precision accuracy) and the accuracy drop value.

Why does the accuracy drop matter?

A small accuracy drop (1-3%) from quantization alone is expected and acceptable for deployment. The critical finding comes in the next step: adversarial attacks amplify this vulnerability far beyond what the clean-data drop suggests.

Seeing zero accuracy or very low values?

  • Check that you are loading ecg_model_int8.tflite (the INT8 file), not the Keras model.
  • Verify that the quantization parameters are being read correctly. Print input_scale and input_zero_point. The scale should be a small positive float, and the zero point should be an integer.
  • If accuracy is exactly 50%, the model is predicting all one class. Re-run Cell 9 to regenerate the quantized model.

Help me debug my TFLite INT8 evaluation results.

✔️ Awesome, I've got everything!

Great. Make sure you have saved your notebook with both Cell 9 and Cell 10 completed.

ⓧ I'd like to double check the full code

Here is the complete code for Cell 9 and Cell 10. Compare your notebook against this:

# Cell 9: Quantize to TFLite INT8
def representative_dataset_gen():
    for i in range(200):
        sample = X_train_norm[i:i+1].astype(np.float32)
        yield [sample]

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8

tflite_model = converter.convert()

with open('ecg_model_int8.tflite', 'wb') as f:
    f.write(tflite_model)

print(f"Quantized model size: {len(tflite_model) / 1024:.1f} KB")
# Cell 10: Evaluate Quantized Model on Clean Data
def evaluate_tflite(tflite_path, X_data, y_true):
    interpreter = tf.lite.Interpreter(model_path=tflite_path)
    interpreter.allocate_tensors()

    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()

    input_scale, input_zero_point = input_details[0]['quantization']

    predictions = []
    for i in range(len(X_data)):
        sample = X_data[i:i+1].astype(np.float32)
        # Quantize input
        sample_int8 = (sample / input_scale + input_zero_point).astype(np.int8)
        interpreter.set_tensor(input_details[0]['index'], sample_int8)
        interpreter.invoke()
        output = interpreter.get_tensor(output_details[0]['index'])
        predictions.append(np.argmax(output))

    accuracy = accuracy_score(y_true, predictions)
    return accuracy

quant_accuracy = evaluate_tflite('ecg_model_int8.tflite', X_test_norm, y_test)
print(f"Quantized model clean accuracy: {quant_accuracy:.4f}")
print(f"Accuracy drop from quantization: {test_accuracy - quant_accuracy:.4f}")

You now have two versions of the same ECG classifier: the full-precision FP32 model and the quantized INT8 model ready for microcontroller deployment. Next up, you will attack both models with FGSM, PGD, and DeepFool to reveal exactly how much robustness is lost after quantization.

Attack Both Models with FGSM, PGD, and DeepFool

You now have two versions of the same ECG classifier: the full-precision FP32 model and its INT8 quantized counterpart. The critical question is whether the quantized model inherits the same adversarial robustness properties as the original.

In this step, you will systematically attack both models using three adversarial methods from IBM's Adversarial Robustness Toolbox (ART). By running the same adversarial inputs through both models, you will quantify exactly how much robustness is lost during quantization.

In this step, get ready to:
  • Wrap the trained ECG model with ART's TensorFlowV2Classifier for gradient-based attacks.
  • Run FGSM at five epsilon values, PGD, and DeepFool against both models.
  • Measure and compare accuracy degradation between the full-precision and quantized models.
Wrap the full-precision model with ART's TensorFlowV2Classifier

ART's attack algorithms need gradient access to generate adversarial perturbations. The TensorFlowV2Classifier wrapper connects your trained Keras model to ART's attack framework by exposing the loss function and optimizer for gradient computation.

  • Create a new cell in ecg_redteam.ipynb and set up the ART classifier wrapper by pasting the following code:
# Cell 11: Setup ART Classifier and Attacks
loss_object = tf.keras.losses.CategoricalCrossentropy(from_logits=False)
optimizer = tf.keras.optimizers.Adam()

art_classifier = TensorFlowV2Classifier(
    model=model,
    loss_object=loss_object,
    optimizer=optimizer,
    nb_classes=2,
    input_shape=(1000, 12),
    clip_values=(clip_min, clip_max)
)

# Verify ART wrapper
art_preds = art_classifier.predict(X_test_norm[:10])
print(f"ART classifier working. Sample prediction shape: {art_preds.shape}")

What does this code do?

The TensorFlowV2Classifier wraps your Keras model so ART can compute gradients through it. The loss_object and optimizer provide the gradient pipeline that FGSM and PGD use to craft perturbations.

The clip_values parameter constrains adversarial examples to stay within the range of your normalized training data. The verification line confirms the wrapper produces predictions with the expected shape (10, 2) (10 samples, 2 classes).

  • Run the cell.
  • You should see output confirming the ART classifier is working with a sample prediction shape of (10, 2).

Seeing a shape mismatch error?

Make sure input_shape=(1000, 12) matches the shape your model was trained on. If you changed the sampling rate or lead count in earlier cells, update this parameter to match.

Also verify that clip_min and clip_max are defined from Cell 5. If your kernel restarted, re-run all cells from the top.

Help me fix my ART TensorFlowV2Classifier wrapper setup.

Run FGSM attacks at multiple epsilon values

FGSM (Fast Gradient Sign Method) perturbs inputs in the direction of the loss gradient. The epsilon parameter controls the perturbation magnitude. By testing multiple epsilon values, you can map how each model's accuracy degrades as the attack strengthens.

  • Create a new cell and run FGSM at five epsilon values against both models by pasting this code:
# Cell 12: Run FGSM Attack at Multiple Epsilon Values
fgsm_epsilons = [0.01, 0.05, 0.1, 0.2, 0.3]
fgsm_results_fp = {}  # full-precision results
fgsm_results_quant = {}  # quantized results
fgsm_adv_examples = {}

for eps in fgsm_epsilons:
    print(f"\nFGSM eps={eps}:")
    attack = FastGradientMethod(estimator=art_classifier, eps=eps)
    X_adv = attack.generate(x=X_test_norm.astype(np.float32))
    
    # Full-precision accuracy
    preds_fp = np.argmax(art_classifier.predict(X_adv), axis=1)
    acc_fp = accuracy_score(y_test, preds_fp)
    fgsm_results_fp[eps] = acc_fp
    
    # Quantized model accuracy
    acc_quant = evaluate_tflite('ecg_model_int8.tflite', X_adv, y_test)
    fgsm_results_quant[eps] = acc_quant
    
    print(f"  Full-precision accuracy: {acc_fp:.4f}")
    print(f"  Quantized accuracy: {acc_quant:.4f}")
    print(f"  Robustness gap: {acc_fp - acc_quant:.4f}")
    
    # Save adversarial examples for hardware testing
    fgsm_adv_examples[eps] = X_adv
    np.save(f'adv_fgsm_eps{eps}.npy', X_adv)

What does this code do?

For each epsilon value, the loop creates a FastGradientMethod attack and generates adversarial versions of the entire test set. A small epsilon (0.01) creates nearly imperceptible perturbations. A large epsilon (0.3) creates more aggressive noise.

Each set of adversarial examples is tested on both the full-precision model (via ART's predict) and the quantized model (via your evaluate_tflite() function). The "robustness gap" is the difference between the two accuracies.

The adversarial examples are saved as .npy files for later use in the follow-on STM32 hardware validation project.

  • Run the cell. This takes a few minutes since it generates adversarial examples for the full test set at five different epsilon values.
  • You should see accuracy values printed for each epsilon. Look for the "Robustness gap" value. A positive gap means the quantized model is less robust than the full-precision model under that attack.

Cell running extremely slowly?

The evaluate_tflite() function runs inference sample-by-sample, which is slow on large test sets. This is expected behavior for the TFLite interpreter. Each epsilon iteration may take 1-3 minutes depending on your hardware.

If you get an out-of-memory error, restart the kernel and re-run all cells. The FGSM attack generates arrays the same size as your test set for each epsilon.

Help me debug my FGSM attack cell that is running too slowly or crashing.

Run PGD and DeepFool attacks

PGD (Projected Gradient Descent) is an iterative version of FGSM that takes multiple small steps toward the adversarial objective. DeepFool finds the minimum perturbation needed to cross the decision boundary. Together with FGSM, these three attacks give you a comprehensive view of model vulnerability.

  • Create a new cell and run the PGD attack by pasting this code:
# Cell 13: Run PGD Attack
print("Running PGD attack (eps=0.1, eps_step=0.01, max_iter=40)...")
pgd_attack = ProjectedGradientDescent(
    estimator=art_classifier,
    eps=0.1,
    eps_step=0.01,
    max_iter=40
)
X_adv_pgd = pgd_attack.generate(x=X_test_norm.astype(np.float32))

# Full-precision
preds_pgd_fp = np.argmax(art_classifier.predict(X_adv_pgd), axis=1)
pgd_acc_fp = accuracy_score(y_test, preds_pgd_fp)

# Quantized
pgd_acc_quant = evaluate_tflite('ecg_model_int8.tflite', X_adv_pgd, y_test)

print(f"PGD - Full-precision accuracy: {pgd_acc_fp:.4f}")
print(f"PGD - Quantized accuracy: {pgd_acc_quant:.4f}")
print(f"PGD - Robustness gap: {pgd_acc_fp - pgd_acc_quant:.4f}")

np.save('adv_pgd.npy', X_adv_pgd)

What does this code do?

PGD takes 40 iterative steps (max_iter=40), each of size eps_step=0.01, while keeping the total perturbation within eps=0.1. This creates stronger adversarial examples than single-step FGSM at the same epsilon because it can find better local optima.

The adversarial examples are tested on both models and saved as adv_pgd.npy for hardware testing.

  • Run the cell. PGD is slower than FGSM because it iterates 40 times per sample. Expect this to take several minutes.
  • You should see the PGD accuracy and robustness gap printed. Compare the PGD robustness gap to the FGSM gap at the same epsilon (0.1).

PGD cell taking more than 10 minutes?

PGD with 40 iterations on the full test set is computationally intensive. If your machine is struggling, you can interrupt the cell and reduce max_iter to 20 for faster execution. The robustness gap will still be visible, just slightly less pronounced.

Help me speed up my PGD attack or debug why it is hanging.

  • Create a new cell and run the DeepFool attack on a subset of the test data by pasting this code:
# Cell 14: Run DeepFool Attack
print("Running DeepFool attack (max_iter=100)...")
deepfool_attack = DeepFool(estimator=art_classifier, max_iter=100)
# DeepFool is slow, use a subset
X_test_subset = X_test_norm[:200].astype(np.float32)
y_test_subset = y_test[:200]

X_adv_df = deepfool_attack.generate(x=X_test_subset)

# Full-precision
preds_df_fp = np.argmax(art_classifier.predict(X_adv_df), axis=1)
df_acc_fp = accuracy_score(y_test_subset, preds_df_fp)

# Quantized
df_acc_quant = evaluate_tflite('ecg_model_int8.tflite', X_adv_df, y_test_subset)

print(f"DeepFool - Full-precision accuracy: {df_acc_fp:.4f}")
print(f"DeepFool - Quantized accuracy: {df_acc_quant:.4f}")
print(f"DeepFool - Robustness gap: {df_acc_fp - df_acc_quant:.4f}")

np.save('adv_deepfool.npy', X_adv_df)

Why only 200 samples for DeepFool?

DeepFool is computationally expensive because it iteratively searches for the minimal perturbation per sample (up to 100 iterations each). Running it on the full test set would take an impractical amount of time.

A 200-sample subset is sufficient to demonstrate the robustness gap. The pattern of quantized models being more vulnerable holds regardless of subset size.

  • Run the cell. DeepFool on 200 samples may take several minutes depending on your hardware.
  • You should see accuracy values and a robustness gap printed. The quantized model should show lower accuracy (higher vulnerability) than the full-precision model under the same adversarial inputs.

DeepFool showing 0% accuracy on both models?

DeepFool is designed to find the minimum perturbation to fool the model, so very low full-precision accuracy is expected. If both models show 0% accuracy, that confirms DeepFool successfully crossed the decision boundary for every sample.

The robustness gap for DeepFool shows up in the perturbation magnitude, not just the accuracy. In the next step, you will visualize these perturbation differences.

Help me understand my DeepFool results and whether they are expected.

✔️ Awesome, I've got everything!

Great. Double check that you have saved your notebook and that the .npy files (adv_fgsm_eps*.npy, adv_pgd.npy, adv_deepfool.npy) appear in your project directory.

ⓧ I'd like to double check the full code

Here is the full notebook code through Cell 14. Compare your cells against this reference.

# Cell 1: Imports and Setup
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import wfdb
import ast
import os
from sklearn.metrics import accuracy_score, classification_report

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

from art.estimators.classification import TensorFlowV2Classifier
from art.attacks.evasion import FastGradientMethod, ProjectedGradientDescent, DeepFool

print(f"TensorFlow version: {tf.__version__}")
print(f"GPU available: {tf.config.list_physical_devices('GPU')}")

# Cell 2: Load PTB-XL Dataset
PTBXL_PATH = './ptb-xl-a-large-publicly-available-electrocardiography-dataset-1.0.3/'

ptbxl_data = pd.read_csv(os.path.join(PTBXL_PATH, 'ptbxl_database.csv'), index_col='ecg_id')
ptbxl_data.scp_codes = ptbxl_data.scp_codes.apply(lambda x: ast.literal_eval(x))

scp_statements = pd.read_csv(os.path.join(PTBXL_PATH, 'scp_statements.csv'), index_col=0)
scp_statements = scp_statements[scp_statements.diagnostic == 1]

print(f"Total records: {len(ptbxl_data)}")
print(f"Diagnostic SCP statements: {len(scp_statements)}")

# Cell 3: Map Labels to Diagnostic Superclasses
def aggregate_diagnostic(scp_codes):
    result = []
    for key, likelihood in scp_codes.items():
        if key in scp_statements.index and likelihood >= 50:
            result.append(scp_statements.loc[key].diagnostic_class)
    return list(set(result))

ptbxl_data['diagnostic_superclass'] = ptbxl_data.scp_codes.apply(aggregate_diagnostic)

def get_binary_label(superclass_list):
    if 'NORM' in superclass_list and len(superclass_list) == 1:
        return 0
    elif 'MI' in superclass_list and len(superclass_list) == 1:
        return 1
    else:
        return -1

ptbxl_data['binary_label'] = ptbxl_data.diagnostic_superclass.apply(get_binary_label)
binary_data = ptbxl_data[ptbxl_data.binary_label >= 0].copy()
print(f"Binary classification samples: {len(binary_data)}")
print(f"Normal: {(binary_data.binary_label == 0).sum()}, MI: {(binary_data.binary_label == 1).sum()}")

# Cell 4: Load Waveform Data
def load_ecg_data(df, path, sampling_rate=100):
    if sampling_rate == 100:
        data = [wfdb.rdsamp(os.path.join(path, f))[0] for f in df.filename_lr]
    else:
        data = [wfdb.rdsamp(os.path.join(path, f))[0] for f in df.filename_hr]
    return np.array(data)

X = load_ecg_data(binary_data, PTBXL_PATH, sampling_rate=100)
y = binary_data.binary_label.values

train_folds = binary_data.strat_fold <= 8
val_fold = binary_data.strat_fold == 9
test_fold = binary_data.strat_fold == 10

X_train, y_train = X[train_folds], y[train_folds]
X_val, y_val = X[val_fold], y[val_fold]
X_test, y_test = X[test_fold], y[test_fold]

y_train_onehot = keras.utils.to_categorical(y_train, 2)
y_val_onehot = keras.utils.to_categorical(y_val, 2)
y_test_onehot = keras.utils.to_categorical(y_test, 2)

print(f"Train: {X_train.shape}, Val: {X_val.shape}, Test: {X_test.shape}")

# Cell 5: Normalize Data
train_mean = X_train.mean()
train_std = X_train.std()

X_train_norm = (X_train - train_mean) / train_std
X_val_norm = (X_val - train_mean) / train_std
X_test_norm = (X_test - train_mean) / train_std

clip_min = X_train_norm.min()
clip_max = X_train_norm.max()
print(f"Clip values: ({clip_min:.4f}, {clip_max:.4f})")

# Cell 6: Build 1D CNN Model
def build_ecg_cnn(input_shape=(1000, 12), num_classes=2):
    model = keras.Sequential([
        layers.Conv1D(32, kernel_size=5, activation='relu', input_shape=input_shape),
        layers.MaxPooling1D(pool_size=2),
        layers.Conv1D(64, kernel_size=5, activation='relu'),
        layers.MaxPooling1D(pool_size=2),
        layers.Conv1D(128, kernel_size=3, activation='relu'),
        layers.GlobalAveragePooling1D(),
        layers.Dense(64, activation='relu'),
        layers.Dense(num_classes, activation='softmax')
    ])
    return model

model = build_ecg_cnn()
model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['accuracy']
)
model.summary()

# Cell 7: Train the Model
history = model.fit(
    X_train_norm, y_train_onehot,
    validation_data=(X_val_norm, y_val_onehot),
    epochs=20,
    batch_size=64,
    verbose=1
)

test_loss, test_accuracy = model.evaluate(X_test_norm, y_test_onehot)
print(f"\nClean test accuracy (full-precision): {test_accuracy:.4f}")

# Cell 8: Save the Trained Model
model.save('ecg_model.keras')
print("Model saved as ecg_model.keras")

# Cell 9: Quantize to TFLite INT8
def representative_dataset_gen():
    for i in range(200):
        sample = X_train_norm[i:i+1].astype(np.float32)
        yield [sample]

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8

tflite_model = converter.convert()

with open('ecg_model_int8.tflite', 'wb') as f:
    f.write(tflite_model)

print(f"Quantized model size: {len(tflite_model) / 1024:.1f} KB")

# Cell 10: Evaluate Quantized Model on Clean Data
def evaluate_tflite(tflite_path, X_data, y_true):
    interpreter = tf.lite.Interpreter(model_path=tflite_path)
    interpreter.allocate_tensors()
    
    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()
    
    input_scale, input_zero_point = input_details[0]['quantization']
    
    predictions = []
    for i in range(len(X_data)):
        sample = X_data[i:i+1].astype(np.float32)
        sample_int8 = (sample / input_scale + input_zero_point).astype(np.int8)
        interpreter.set_tensor(input_details[0]['index'], sample_int8)
        interpreter.invoke()
        output = interpreter.get_tensor(output_details[0]['index'])
        predictions.append(np.argmax(output))
    
    accuracy = accuracy_score(y_true, predictions)
    return accuracy

quant_accuracy = evaluate_tflite('ecg_model_int8.tflite', X_test_norm, y_test)
print(f"Quantized model clean accuracy: {quant_accuracy:.4f}")
print(f"Accuracy drop from quantization: {test_accuracy - quant_accuracy:.4f}")

# Cell 11: Setup ART Classifier and Attacks
loss_object = tf.keras.losses.CategoricalCrossentropy(from_logits=False)
optimizer = tf.keras.optimizers.Adam()

art_classifier = TensorFlowV2Classifier(
    model=model,
    loss_object=loss_object,
    optimizer=optimizer,
    nb_classes=2,
    input_shape=(1000, 12),
    clip_values=(clip_min, clip_max)
)

art_preds = art_classifier.predict(X_test_norm[:10])
print(f"ART classifier working. Sample prediction shape: {art_preds.shape}")

# Cell 12: Run FGSM Attack at Multiple Epsilon Values
fgsm_epsilons = [0.01, 0.05, 0.1, 0.2, 0.3]
fgsm_results_fp = {}  # full-precision results
fgsm_results_quant = {}  # quantized results
fgsm_adv_examples = {}

for eps in fgsm_epsilons:
    print(f"\nFGSM eps={eps}:")
    attack = FastGradientMethod(estimator=art_classifier, eps=eps)
    X_adv = attack.generate(x=X_test_norm.astype(np.float32))
    
    # Full-precision accuracy
    preds_fp = np.argmax(art_classifier.predict(X_adv), axis=1)
    acc_fp = accuracy_score(y_test, preds_fp)
    fgsm_results_fp[eps] = acc_fp
    
    # Quantized model accuracy
    acc_quant = evaluate_tflite('ecg_model_int8.tflite', X_adv, y_test)
    fgsm_results_quant[eps] = acc_quant
    
    print(f"  Full-precision accuracy: {acc_fp:.4f}")
    print(f"  Quantized accuracy: {acc_quant:.4f}")
    print(f"  Robustness gap: {acc_fp - acc_quant:.4f}")
    
    # Save adversarial examples for hardware testing
    fgsm_adv_examples[eps] = X_adv
    np.save(f'adv_fgsm_eps{eps}.npy', X_adv)

# Cell 13: Run PGD Attack
print("Running PGD attack (eps=0.1, eps_step=0.01, max_iter=40)...")
pgd_attack = ProjectedGradientDescent(
    estimator=art_classifier,
    eps=0.1,
    eps_step=0.01,
    max_iter=40
)
X_adv_pgd = pgd_attack.generate(x=X_test_norm.astype(np.float32))

# Full-precision
preds_pgd_fp = np.argmax(art_classifier.predict(X_adv_pgd), axis=1)
pgd_acc_fp = accuracy_score(y_test, preds_pgd_fp)

# Quantized
pgd_acc_quant = evaluate_tflite('ecg_model_int8.tflite', X_adv_pgd, y_test)

print(f"PGD - Full-precision accuracy: {pgd_acc_fp:.4f}")
print(f"PGD - Quantized accuracy: {pgd_acc_quant:.4f}")
print(f"PGD - Robustness gap: {pgd_acc_fp - pgd_acc_quant:.4f}")

np.save('adv_pgd.npy', X_adv_pgd)

# Cell 14: Run DeepFool Attack
print("Running DeepFool attack (max_iter=100)...")
deepfool_attack = DeepFool(estimator=art_classifier, max_iter=100)
# DeepFool is slow, use a subset
X_test_subset = X_test_norm[:200].astype(np.float32)
y_test_subset = y_test[:200]

X_adv_df = deepfool_attack.generate(x=X_test_subset)

# Full-precision
preds_df_fp = np.argmax(art_classifier.predict(X_adv_df), axis=1)
df_acc_fp = accuracy_score(y_test_subset, preds_df_fp)

# Quantized
df_acc_quant = evaluate_tflite('ecg_model_int8.tflite', X_adv_df, y_test_subset)

print(f"DeepFool - Full-precision accuracy: {df_acc_fp:.4f}")
print(f"DeepFool - Quantized accuracy: {df_acc_quant:.4f}")
print(f"DeepFool - Robustness gap: {df_acc_fp - df_acc_quant:.4f}")

np.save('adv_deepfool.npy', X_adv_df)

You now have quantitative proof that the quantized model is more vulnerable to adversarial attacks than the full-precision model. Next up, you will visualize these results and generate a formal red team report that documents the robustness gap.

Generate the Robustness Gap Report

You now have raw accuracy numbers from three different adversarial attacks on both model versions. Numbers alone do not tell the full story.

In this step, you will compute formal robustness metrics, visualize the quantization gap across attacks, and produce a structured red team report. This report is the deliverable that a medical device security reviewer would use to assess pre-deployment risk.

In this step, get ready to:
  • Compute attack success rates and the quantization robustness gap for each attack.
  • Generate three visualizations comparing full-precision and quantized model robustness.
  • Produce a structured red team summary report with a risk assessment conclusion.
Compute the quantization robustness gap metrics

Before generating charts, you need a clear definition of the metrics you are reporting. Two key measurements drive this analysis.

What are the robustness gap metrics?

The quantization robustness gap for a given attack is the difference in accuracy between the full-precision and quantized models on the same adversarial inputs: gap = accuracy_FP32_adversarial - accuracy_INT8_adversarial.

The Attack Success Rate (ASR) measures how effectively an attack degrades a model: ASR = 1 - (accuracy_adversarial / accuracy_clean). A higher ASR means the attack is more effective.

If the INT8 model has a higher ASR than the FP32 model for the same adversarial examples, that proves robustness does not transfer through quantization.

You will compute these metrics inline as part of the visualization cells below. The attack results from the previous step (fgsm_results_fp, fgsm_results_quant, pgd_acc_fp, pgd_acc_quant, df_acc_fp, df_acc_quant) already contain the raw accuracy values needed.

Visualize the results

Three visualizations will make the robustness gap immediately clear to anyone reviewing your report.

  • Create a new cell in ecg_redteam.ipynb and add the FGSM accuracy-vs-epsilon curve by pasting this code:
# Cell 15: Visualize Robustness Gap - FGSM Epsilon Curve
fig, ax = plt.subplots(1, 1, figsize=(10, 6))

epsilons = list(fgsm_results_fp.keys())
acc_fp_list = [fgsm_results_fp[e] for e in epsilons]
acc_quant_list = [fgsm_results_quant[e] for e in epsilons]

ax.plot(epsilons, acc_fp_list, 'b-o', label='Full-Precision (FP32)', linewidth=2)
ax.plot(epsilons, acc_quant_list, 'r-s', label='Quantized (INT8)', linewidth=2)
ax.axhline(y=test_accuracy, color='blue', linestyle='--', alpha=0.5, label=f'Clean FP32: {test_accuracy:.3f}')
ax.axhline(y=quant_accuracy, color='red', linestyle='--', alpha=0.5, label=f'Clean INT8: {quant_accuracy:.3f}')

ax.set_xlabel('FGSM Epsilon', fontsize=12)
ax.set_ylabel('Accuracy', fontsize=12)
ax.set_title('Quantization Robustness Gap: FGSM Attack', fontsize=14)
ax.legend(fontsize=11)
ax.grid(True, alpha=0.3)
ax.set_ylim(0, 1.05)

plt.tight_layout()
plt.savefig('fgsm_robustness_gap.png', dpi=150)
plt.show()

What does this code do?

This cell plots the accuracy of both models (FP32 in blue, INT8 in red) as FGSM epsilon increases from 0.01 to 0.3. The gap between the two lines IS the quantization robustness gap at each perturbation strength.

Dashed horizontal lines show each model's clean (no-attack) accuracy as a baseline reference. The chart is saved to fgsm_robustness_gap.png for your report.

  • Run the cell.
  • You should see a line chart with the red (INT8) line consistently below the blue (FP32) line, showing the quantized model is more vulnerable at every epsilon value.

Plot not showing?

Make sure you ran Cell 12 (FGSM attacks) successfully before this cell. The dictionaries fgsm_results_fp and fgsm_results_quant must be populated.

If you see a NameError, restart your kernel and re-run all previous cells in order.

Help me debug my FGSM visualization cell.

  • Create another new cell and add the attack success rate bar chart by pasting this code:
# Cell 16: Bar Chart - Attack Success Rates
attacks = ['FGSM\n(eps=0.1)', 'PGD\n(eps=0.1)', 'DeepFool']
asr_fp = [
    1 - fgsm_results_fp[0.1] / test_accuracy,
    1 - pgd_acc_fp / test_accuracy,
    1 - df_acc_fp / test_accuracy
]
asr_quant = [
    1 - fgsm_results_quant[0.1] / quant_accuracy,
    1 - pgd_acc_quant / quant_accuracy,
    1 - df_acc_quant / quant_accuracy
]

x = np.arange(len(attacks))
width = 0.35

fig, ax = plt.subplots(figsize=(8, 6))
bars1 = ax.bar(x - width/2, asr_fp, width, label='Full-Precision (FP32)', color='steelblue')
bars2 = ax.bar(x + width/2, asr_quant, width, label='Quantized (INT8)', color='indianred')

ax.set_ylabel('Attack Success Rate', fontsize=12)
ax.set_title('Attack Success Rate: Full-Precision vs Quantized', fontsize=14)
ax.set_xticks(x)
ax.set_xticklabels(attacks)
ax.legend()
ax.set_ylim(0, 1.05)
ax.grid(True, alpha=0.3, axis='y')

plt.tight_layout()
plt.savefig('attack_success_rates.png', dpi=150)
plt.show()

What does this code do?

This cell computes the Attack Success Rate (ASR) for each of the three attacks on both models. The formula 1 - accuracy_adversarial / accuracy_clean gives the fraction of originally-correct predictions that the attack flipped.

The grouped bar chart makes it easy to compare: the red (INT8) bars should be taller than the blue (FP32) bars for every attack, proving quantization increases vulnerability.

  • Run the cell.
  • You should see a grouped bar chart with three attack groups. The red bars (INT8) are consistently taller than the blue bars (FP32), confirming higher attack success rates on the quantized model.

Seeing zero-height bars?

Check that test_accuracy and quant_accuracy are not zero. These are the denominators in the ASR formula. If they are zero, your model training (Cell 7) or evaluation (Cell 10) did not run correctly.

Help me debug my attack success rate bar chart.

  • Create another new cell and add the ECG perturbation visualization by pasting this code:
# Cell 17: Visualize Sample ECG Perturbations
fig, axes = plt.subplots(3, 1, figsize=(12, 8))
sample_idx = 0
lead = 0

# Original ECG waveform
axes[0].plot(X_test_norm[sample_idx, :, lead], 'b-', linewidth=0.8)
axes[0].set_title(f'Original ECG (Lead I) - True Label: {"Normal" if y_test[sample_idx]==0 else "MI"}')
axes[0].set_ylabel('Amplitude')

# FGSM adversarial version
axes[1].plot(fgsm_adv_examples[0.1][sample_idx, :, lead], 'r-', linewidth=0.8)
axes[1].set_title(f'FGSM Adversarial (eps=0.1) - Perturbation L2: {np.linalg.norm(fgsm_adv_examples[0.1][sample_idx] - X_test_norm[sample_idx]):.4f}')
axes[1].set_ylabel('Amplitude')

# The perturbation signal itself
perturbation = fgsm_adv_examples[0.1][sample_idx, :, lead] - X_test_norm[sample_idx, :, lead]
axes[2].plot(perturbation, 'g-', linewidth=0.8)
axes[2].set_title('Perturbation (Adversarial - Original)')
axes[2].set_xlabel('Time Step')
axes[2].set_ylabel('Perturbation')

plt.tight_layout()
plt.savefig('ecg_perturbation_example.png', dpi=150)
plt.show()

What does this code do?

This three-panel plot shows a single ECG sample from Lead I. The top panel is the original clean signal. The middle panel is the same signal after FGSM perturbation at epsilon 0.1, with the L2 norm of the perturbation annotated.

The bottom panel isolates just the perturbation (adversarial minus original). This makes it visually clear how small the perturbation is compared to the original signal. A clinician would not notice the difference, but the model misclassifies.

  • Run the cell.
  • You should see three stacked plots. The top two (original and adversarial) look nearly identical to the human eye. The bottom panel shows the structured noise pattern that fools the model.

Seeing a KeyError on fgsm_adv_examples?

The dictionary fgsm_adv_examples stores adversarial arrays keyed by epsilon. Make sure Cell 12 ran to completion and populated all five epsilon values.

Help me fix my ECG perturbation visualization.

Generate the red team summary report

The final cell produces a structured text report that consolidates all findings into a single output. This is the deliverable you would hand to a security reviewer.

  • Create a new cell and add the red team report generator by pasting this code:
# Cell 18: Generate Red Team Summary Report
print("=" * 70)
print("RED TEAM REPORT: ECG CNN Adversarial Robustness Assessment")
print("=" * 70)
print(f"\nModel: 1D CNN ECG Classifier (Normal vs MI)")
print(f"Dataset: PTB-XL v1.0.3 (100 Hz)")
print(f"Quantization: TFLite INT8 (post-training)")
print(f"\n{'='*70}")
print(f"{'Metric':<40} {'FP32':<15} {'INT8':<15}")
print(f"{'='*70}")
print(f"{'Clean Accuracy':<40} {test_accuracy:<15.4f} {quant_accuracy:<15.4f}")
print(f"{'FGSM (eps=0.01)':<40} {fgsm_results_fp[0.01]:<15.4f} {fgsm_results_quant[0.01]:<15.4f}")
print(f"{'FGSM (eps=0.05)':<40} {fgsm_results_fp[0.05]:<15.4f} {fgsm_results_quant[0.05]:<15.4f}")
print(f"{'FGSM (eps=0.1)':<40} {fgsm_results_fp[0.1]:<15.4f} {fgsm_results_quant[0.1]:<15.4f}")
print(f"{'FGSM (eps=0.2)':<40} {fgsm_results_fp[0.2]:<15.4f} {fgsm_results_quant[0.2]:<15.4f}")
print(f"{'FGSM (eps=0.3)':<40} {fgsm_results_fp[0.3]:<15.4f} {fgsm_results_quant[0.3]:<15.4f}")
print(f"{'PGD (eps=0.1, iter=40)':<40} {pgd_acc_fp:<15.4f} {pgd_acc_quant:<15.4f}")
print(f"{'DeepFool (subset=200)':<40} {df_acc_fp:<15.4f} {df_acc_quant:<15.4f}")
print(f"{'='*70}")
print(f"\nRISK ASSESSMENT:")
print(f"  The INT8 quantized model shows consistently LOWER adversarial")
print(f"  robustness compared to the full-precision model. This confirms")
print(f"  that pre-deployment robustness testing on FP32 models does NOT")
print(f"  guarantee robustness on the deployed quantized model.")
print(f"\n  Recommendation: Adversarial testing MUST be performed on the")
print(f"  final quantized model, not just the training-time FP32 model.")
print(f"\nAdversarial examples exported for STM32H7 hardware validation.")
print(f"{'='*70}")

What does this report include?

The report prints a formatted table comparing FP32 and INT8 accuracy under every attack configuration. The risk assessment conclusion states the core finding: robustness tested on the full-precision model does not transfer to the quantized deployment model.

The final line confirms that adversarial .npy files are exported and ready for hardware validation on STM32H7 in the follow-on project.

  • Run the cell.
  • You should see a formatted table with two columns (FP32 and INT8) showing accuracy for each attack. The INT8 column should consistently show lower values than the FP32 column under adversarial conditions.

Report values look wrong?

If the INT8 column shows identical values to FP32, your quantized model may not have converted correctly. Re-run Cell 9 and verify that converter.inference_input_type = tf.int8 is set.

If you see very low accuracy for both columns even on clean data, your model may have trained poorly. Check that Cell 7 reported above 85% test accuracy.

Help me interpret my red team report results.

Why does this matter for medical devices?

The FDA's Secure Product Development Framework requires that security objectives (including integrity) are maintained for AI-enabled medical devices. Your report demonstrates a concrete scenario where a model appears robust during development (FP32 testing) but becomes vulnerable once quantized for edge deployment.

This is exactly the kind of pre-deployment security testing gap that could lead to missed arrhythmia detections on a real device.

✔️ Awesome, I've got everything!

Great. Double check you have saved your notebook and confirm that fgsm_robustness_gap.png, attack_success_rates.png, and ecg_perturbation_example.png are saved in your project folder.

ⓧ I'd like to double check the full code

Here is the complete notebook code from Cell 1 through Cell 18:

# Cell 1: Imports and Setup
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import wfdb
import ast
import os
from sklearn.metrics import accuracy_score, classification_report

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

from art.estimators.classification import TensorFlowV2Classifier
from art.attacks.evasion import FastGradientMethod, ProjectedGradientDescent, DeepFool

print(f"TensorFlow version: {tf.__version__}")
print(f"GPU available: {tf.config.list_physical_devices('GPU')}")

# Cell 2: Load PTB-XL Dataset
PTBXL_PATH = './ptb-xl-a-large-publicly-available-electrocardiography-dataset-1.0.3/'

ptbxl_data = pd.read_csv(os.path.join(PTBXL_PATH, 'ptbxl_database.csv'), index_col='ecg_id')
ptbxl_data.scp_codes = ptbxl_data.scp_codes.apply(lambda x: ast.literal_eval(x))

scp_statements = pd.read_csv(os.path.join(PTBXL_PATH, 'scp_statements.csv'), index_col=0)
scp_statements = scp_statements[scp_statements.diagnostic == 1]

print(f"Total records: {len(ptbxl_data)}")
print(f"Diagnostic SCP statements: {len(scp_statements)}")

# Cell 3: Map Labels to Diagnostic Superclasses
def aggregate_diagnostic(scp_codes):
    result = []
    for key, likelihood in scp_codes.items():
        if key in scp_statements.index and likelihood >= 50:
            result.append(scp_statements.loc[key].diagnostic_class)
    return list(set(result))

ptbxl_data['diagnostic_superclass'] = ptbxl_data.scp_codes.apply(aggregate_diagnostic)

def get_binary_label(superclass_list):
    if 'NORM' in superclass_list and len(superclass_list) == 1:
        return 0
    elif 'MI' in superclass_list and len(superclass_list) == 1:
        return 1
    else:
        return -1

ptbxl_data['binary_label'] = ptbxl_data.diagnostic_superclass.apply(get_binary_label)
binary_data = ptbxl_data[ptbxl_data.binary_label >= 0].copy()
print(f"Binary classification samples: {len(binary_data)}")
print(f"Normal: {(binary_data.binary_label == 0).sum()}, MI: {(binary_data.binary_label == 1).sum()}")

# Cell 4: Load Waveform Data
def load_ecg_data(df, path, sampling_rate=100):
    if sampling_rate == 100:
        data = [wfdb.rdsamp(os.path.join(path, f))[0] for f in df.filename_lr]
    else:
        data = [wfdb.rdsamp(os.path.join(path, f))[0] for f in df.filename_hr]
    return np.array(data)

X = load_ecg_data(binary_data, PTBXL_PATH, sampling_rate=100)
y = binary_data.binary_label.values

train_folds = binary_data.strat_fold <= 8
val_fold = binary_data.strat_fold == 9
test_fold = binary_data.strat_fold == 10

X_train, y_train = X[train_folds], y[train_folds]
X_val, y_val = X[val_fold], y[val_fold]
X_test, y_test = X[test_fold], y[test_fold]

y_train_onehot = keras.utils.to_categorical(y_train, 2)
y_val_onehot = keras.utils.to_categorical(y_val, 2)
y_test_onehot = keras.utils.to_categorical(y_test, 2)

print(f"Train: {X_train.shape}, Val: {X_val.shape}, Test: {X_test.shape}")

# Cell 5: Normalize Data
train_mean = X_train.mean()
train_std = X_train.std()

X_train_norm = (X_train - train_mean) / train_std
X_val_norm = (X_val - train_mean) / train_std
X_test_norm = (X_test - train_mean) / train_std

clip_min = X_train_norm.min()
clip_max = X_train_norm.max()
print(f"Clip values: ({clip_min:.4f}, {clip_max:.4f})")

# Cell 6: Build 1D CNN Model
def build_ecg_cnn(input_shape=(1000, 12), num_classes=2):
    model = keras.Sequential([
        layers.Conv1D(32, kernel_size=5, activation='relu', input_shape=input_shape),
        layers.MaxPooling1D(pool_size=2),
        layers.Conv1D(64, kernel_size=5, activation='relu'),
        layers.MaxPooling1D(pool_size=2),
        layers.Conv1D(128, kernel_size=3, activation='relu'),
        layers.GlobalAveragePooling1D(),
        layers.Dense(64, activation='relu'),
        layers.Dense(num_classes, activation='softmax')
    ])
    return model

model = build_ecg_cnn()
model.compile(
    optimizer='adam',
    loss='categorical_crossentropy',
    metrics=['accuracy']
)
model.summary()

# Cell 7: Train the Model
history = model.fit(
    X_train_norm, y_train_onehot,
    validation_data=(X_val_norm, y_val_onehot),
    epochs=20,
    batch_size=64,
    verbose=1
)

test_loss, test_accuracy = model.evaluate(X_test_norm, y_test_onehot)
print(f"\nClean test accuracy (full-precision): {test_accuracy:.4f}")

# Cell 8: Save the Trained Model
model.save('ecg_model.keras')
print("Model saved as ecg_model.keras")

# Cell 9: Quantize to TFLite INT8
def representative_dataset_gen():
    for i in range(200):
        sample = X_train_norm[i:i+1].astype(np.float32)
        yield [sample]

converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.int8
converter.inference_output_type = tf.int8

tflite_model = converter.convert()

with open('ecg_model_int8.tflite', 'wb') as f:
    f.write(tflite_model)

print(f"Quantized model size: {len(tflite_model) / 1024:.1f} KB")

# Cell 10: Evaluate Quantized Model on Clean Data
def evaluate_tflite(tflite_path, X_data, y_true):
    interpreter = tf.lite.Interpreter(model_path=tflite_path)
    interpreter.allocate_tensors()
    
    input_details = interpreter.get_input_details()
    output_details = interpreter.get_output_details()
    
    input_scale, input_zero_point = input_details[0]['quantization']
    
    predictions = []
    for i in range(len(X_data)):
        sample = X_data[i:i+1].astype(np.float32)
        sample_int8 = (sample / input_scale + input_zero_point).astype(np.int8)
        interpreter.set_tensor(input_details[0]['index'], sample_int8)
        interpreter.invoke()
        output = interpreter.get_tensor(output_details[0]['index'])
        predictions.append(np.argmax(output))
    
    accuracy = accuracy_score(y_true, predictions)
    return accuracy

quant_accuracy = evaluate_tflite('ecg_model_int8.tflite', X_test_norm, y_test)
print(f"Quantized model clean accuracy: {quant_accuracy:.4f}")
print(f"Accuracy drop from quantization: {test_accuracy - quant_accuracy:.4f}")

# Cell 11: Setup ART Classifier and Attacks
loss_object = tf.keras.losses.CategoricalCrossentropy(from_logits=False)
optimizer = tf.keras.optimizers.Adam()

art_classifier = TensorFlowV2Classifier(
    model=model,
    loss_object=loss_object,
    optimizer=optimizer,
    nb_classes=2,
    input_shape=(1000, 12),
    clip_values=(clip_min, clip_max)
)

art_preds = art_classifier.predict(X_test_norm[:10])
print(f"ART classifier working. Sample prediction shape: {art_preds.shape}")

# Cell 12: Run FGSM Attack at Multiple Epsilon Values
fgsm_epsilons = [0.01, 0.05, 0.1, 0.2, 0.3]
fgsm_results_fp = {}
fgsm_results_quant = {}
fgsm_adv_examples = {}

for eps in fgsm_epsilons:
    print(f"\nFGSM eps={eps}:")
    attack = FastGradientMethod(estimator=art_classifier, eps=eps)
    X_adv = attack.generate(x=X_test_norm.astype(np.float32))
    
    preds_fp = np.argmax(art_classifier.predict(X_adv), axis=1)
    acc_fp = accuracy_score(y_test, preds_fp)
    fgsm_results_fp[eps] = acc_fp
    
    acc_quant = evaluate_tflite('ecg_model_int8.tflite', X_adv, y_test)
    fgsm_results_quant[eps] = acc_quant
    
    print(f"  Full-precision accuracy: {acc_fp:.4f}")
    print(f"  Quantized accuracy: {acc_quant:.4f}")
    print(f"  Robustness gap: {acc_fp - acc_quant:.4f}")
    
    fgsm_adv_examples[eps] = X_adv
    np.save(f'adv_fgsm_eps{eps}.npy', X_adv)

# Cell 13: Run PGD Attack
print("Running PGD attack (eps=0.1, eps_step=0.01, max_iter=40)...")
pgd_attack = ProjectedGradientDescent(
    estimator=art_classifier,
    eps=0.1,
    eps_step=0.01,
    max_iter=40
)
X_adv_pgd = pgd_attack.generate(x=X_test_norm.astype(np.float32))

preds_pgd_fp = np.argmax(art_classifier.predict(X_adv_pgd), axis=1)
pgd_acc_fp = accuracy_score(y_test, preds_pgd_fp)

pgd_acc_quant = evaluate_tflite('ecg_model_int8.tflite', X_adv_pgd, y_test)

print(f"PGD - Full-precision accuracy: {pgd_acc_fp:.4f}")
print(f"PGD - Quantized accuracy: {pgd_acc_quant:.4f}")
print(f"PGD - Robustness gap: {pgd_acc_fp - pgd_acc_quant:.4f}")

np.save('adv_pgd.npy', X_adv_pgd)

# Cell 14: Run DeepFool Attack
print("Running DeepFool attack (max_iter=100)...")
deepfool_attack = DeepFool(estimator=art_classifier, max_iter=100)
X_test_subset = X_test_norm[:200].astype(np.float32)
y_test_subset = y_test[:200]

X_adv_df = deepfool_attack.generate(x=X_test_subset)

preds_df_fp = np.argmax(art_classifier.predict(X_adv_df), axis=1)
df_acc_fp = accuracy_score(y_test_subset, preds_df_fp)

df_acc_quant = evaluate_tflite('ecg_model_int8.tflite', X_adv_df, y_test_subset)

print(f"DeepFool - Full-precision accuracy: {df_acc_fp:.4f}")
print(f"DeepFool - Quantized accuracy: {df_acc_quant:.4f}")
print(f"DeepFool - Robustness gap: {df_acc_fp - df_acc_quant:.4f}")

np.save('adv_deepfool.npy', X_adv_df)

# Cell 15: Visualize Robustness Gap - FGSM Epsilon Curve
fig, ax = plt.subplots(1, 1, figsize=(10, 6))

epsilons = list(fgsm_results_fp.keys())
acc_fp_list = [fgsm_results_fp[e] for e in epsilons]
acc_quant_list = [fgsm_results_quant[e] for e in epsilons]

ax.plot(epsilons, acc_fp_list, 'b-o', label='Full-Precision (FP32)', linewidth=2)
ax.plot(epsilons, acc_quant_list, 'r-s', label='Quantized (INT8)', linewidth=2)
ax.axhline(y=test_accuracy, color='blue', linestyle='--', alpha=0.5, label=f'Clean FP32: {test_accuracy:.3f}')
ax.axhline(y=quant_accuracy, color='red', linestyle='--', alpha=0.5, label=f'Clean INT8: {quant_accuracy:.3f}')

ax.set_xlabel('FGSM Epsilon', fontsize=12)
ax.set_ylabel('Accuracy', fontsize=12)
ax.set_title('Quantization Robustness Gap: FGSM Attack', fontsize=14)
ax.legend(fontsize=11)
ax.grid(True, alpha=0.3)
ax.set_ylim(0, 1.05)

plt.tight_layout()
plt.savefig('fgsm_robustness_gap.png', dpi=150)
plt.show()

# Cell 16: Bar Chart - Attack Success Rates
attacks = ['FGSM\n(eps=0.1)', 'PGD\n(eps=0.1)', 'DeepFool']
asr_fp = [
    1 - fgsm_results_fp[0.1] / test_accuracy,
    1 - pgd_acc_fp / test_accuracy,
    1 - df_acc_fp / test_accuracy
]
asr_quant = [
    1 - fgsm_results_quant[0.1] / quant_accuracy,
    1 - pgd_acc_quant / quant_accuracy,
    1 - df_acc_quant / quant_accuracy
]

x = np.arange(len(attacks))
width = 0.35

fig, ax = plt.subplots(figsize=(8, 6))
bars1 = ax.bar(x - width/2, asr_fp, width, label='Full-Precision (FP32)', color='steelblue')
bars2 = ax.bar(x + width/2, asr_quant, width, label='Quantized (INT8)', color='indianred')

ax.set_ylabel('Attack Success Rate', fontsize=12)
ax.set_title('Attack Success Rate: Full-Precision vs Quantized', fontsize=14)
ax.set_xticks(x)
ax.set_xticklabels(attacks)
ax.legend()
ax.set_ylim(0, 1.05)
ax.grid(True, alpha=0.3, axis='y')

plt.tight_layout()
plt.savefig('attack_success_rates.png', dpi=150)
plt.show()

# Cell 17: Visualize Sample ECG Perturbations
fig, axes = plt.subplots(3, 1, figsize=(12, 8))
sample_idx = 0
lead = 0

axes[0].plot(X_test_norm[sample_idx, :, lead], 'b-', linewidth=0.8)
axes[0].set_title(f'Original ECG (Lead I) - True Label: {"Normal" if y_test[sample_idx]==0 else "MI"}')
axes[0].set_ylabel('Amplitude')

axes[1].plot(fgsm_adv_examples[0.1][sample_idx, :, lead], 'r-', linewidth=0.8)
axes[1].set_title(f'FGSM Adversarial (eps=0.1) - Perturbation L2: {np.linalg.norm(fgsm_adv_examples[0.1][sample_idx] - X_test_norm[sample_idx]):.4f}')
axes[1].set_ylabel('Amplitude')

perturbation = fgsm_adv_examples[0.1][sample_idx, :, lead] - X_test_norm[sample_idx, :, lead]
axes[2].plot(perturbation, 'g-', linewidth=0.8)
axes[2].set_title('Perturbation (Adversarial - Original)')
axes[2].set_xlabel('Time Step')
axes[2].set_ylabel('Perturbation')

plt.tight_layout()
plt.savefig('ecg_perturbation_example.png', dpi=150)
plt.show()

# Cell 18: Generate Red Team Summary Report
print("=" * 70)
print("RED TEAM REPORT: ECG CNN Adversarial Robustness Assessment")
print("=" * 70)
print(f"\nModel: 1D CNN ECG Classifier (Normal vs MI)")
print(f"Dataset: PTB-XL v1.0.3 (100 Hz)")
print(f"Quantization: TFLite INT8 (post-training)")
print(f"\n{'='*70}")
print(f"{'Metric':<40} {'FP32':<15} {'INT8':<15}")
print(f"{'='*70}")
print(f"{'Clean Accuracy':<40} {test_accuracy:<15.4f} {quant_accuracy:<15.4f}")
print(f"{'FGSM (eps=0.01)':<40} {fgsm_results_fp[0.01]:<15.4f} {fgsm_results_quant[0.01]:<15.4f}")
print(f"{'FGSM (eps=0.05)':<40} {fgsm_results_fp[0.05]:<15.4f} {fgsm_results_quant[0.05]:<15.4f}")
print(f"{'FGSM (eps=0.1)':<40} {fgsm_results_fp[0.1]:<15.4f} {fgsm_results_quant[0.1]:<15.4f}")
print(f"{'FGSM (eps=0.2)':<40} {fgsm_results_fp[0.2]:<15.4f} {fgsm_results_quant[0.2]:<15.4f}")
print(f"{'FGSM (eps=0.3)':<40} {fgsm_results_fp[0.3]:<15.4f} {fgsm_results_quant[0.3]:<15.4f}")
print(f"{'PGD (eps=0.1, iter=40)':<40} {pgd_acc_fp:<15.4f} {pgd_acc_quant:<15.4f}")
print(f"{'DeepFool (subset=200)':<40} {df_acc_fp:<15.4f} {df_acc_quant:<15.4f}")
print(f"{'='*70}")
print(f"\nRISK ASSESSMENT:")
print(f"  The INT8 quantized model shows consistently LOWER adversarial")
print(f"  robustness compared to the full-precision model. This confirms")
print(f"  that pre-deployment robustness testing on FP32 models does NOT")
print(f"  guarantee robustness on the deployed quantized model.")
print(f"\n  Recommendation: Adversarial testing MUST be performed on the")
print(f"  final quantized model, not just the training-time FP32 model.")
print(f"\nAdversarial examples exported for STM32H7 hardware validation.")
print(f"{'='*70}")

Secret mission

Test Whether Adversarial Training Survives Quantization

You've proven that quantization degrades adversarial robustness. But what about defenses? Adversarial training is the most common defense against adversarial attacks. In this mission, you'll test whether that defense survives the INT8 quantization process, replicating a critical finding from medical device security research.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources for the follow-on STM32 hardware validation project, free up disk space by removing the dataset, or delete everything entirely.

Resources you used:

  • PTB-XL ECG dataset (~3 GB uncompressed) in the ptb-xl-a-large-publicly-available-electrocardiography-dataset-1.0.3/ directory.
  • Trained model files: ecg_model.keras, ecg_model_int8.tflite, and ecg_model_advtrained_int8.tflite.
  • Adversarial example files (.npy) saved for hardware validation.
  • Project notebook ecg_redteam.ipynb and visualization images (.png files).
  • Python packages installed via pip: tensorflow, adversarial-robustness-toolbox, wfdb.

Keep everything running

No action needed. Choose this if you plan to continue with the follow-on STM32 hardware validation project.

  • The PTB-XL dataset, model files, and adversarial examples are all stored locally with zero ongoing cost.
  • The exported .npy adversarial samples are designed to be loaded in the follow-on project and sent to the STM32H7 over serial for hardware validation.
  • The installed Python packages (tensorflow, adversarial-robustness-toolbox, wfdb) remain available in your Python environment.

Pause - I'll come back to this later

Remove the PTB-XL dataset to reclaim approximately 3 GB of disk space. Keep the model files and adversarial examples so you can pick up with the hardware validation project later.

  • Delete the PTB-XL dataset directory by running this command (macOS/Linux):
rm -rf ptb-xl-a-large-publicly-available-electrocardiography-dataset-1.0.3/

On Windows (PowerShell):

Remove-Item -Recurse -Force ptb-xl-a-large-publicly-available-electrocardiography-dataset-1.0.3
  • Your model files (ecg_model.keras, ecg_model_int8.tflite, ecg_model_advtrained_int8.tflite), adversarial samples (.npy files), and notebook remain intact for the follow-on project.
  • If you re-download the dataset later, update PTBXL_PATH in your notebook to point to the new location.

Delete - I don't want to use this again

Remove all project files and optionally uninstall the Python packages. Choose this if you do not plan to continue with the follow-on hardware validation project.

  • Delete the PTB-XL dataset, all model files, adversarial examples, visualizations, and the notebook. On macOS/Linux:
rm -rf ptb-xl-a-large-publicly-available-electrocardiography-dataset-1.0.3/
rm -f ecg_model.keras ecg_model_int8.tflite ecg_model_advtrained_int8.tflite
rm -f adv_fgsm_eps*.npy adv_pgd.npy adv_deepfool.npy
rm -f fgsm_robustness_gap.png attack_success_rates.png ecg_perturbation_example.png
rm -f ecg_redteam.ipynb

On Windows (PowerShell):

Remove-Item -Recurse -Force ptb-xl-a-large-publicly-available-electrocardiography-dataset-1.0.3
Remove-Item -Force ecg_model.keras, ecg_model_int8.tflite, ecg_model_advtrained_int8.tflite
Remove-Item -Force adv_fgsm_eps*.npy, adv_pgd.npy, adv_deepfool.npy
Remove-Item -Force fgsm_robustness_gap.png, attack_success_rates.png, ecg_perturbation_example.png
Remove-Item -Force ecg_redteam.ipynb
  • Optionally uninstall the Python packages you installed for this project:
pip uninstall tensorflow adversarial-robustness-toolbox wfdb -y
  • Confirm y if prompted for any remaining dependency removals.

Nice Work!

Nice Work!

Nice work! You just built a complete adversarial red teaming pipeline for a medical TinyML model, proving with quantitative evidence that INT8 quantization degrades adversarial robustness.

You've learned how to:

  • Trained a 1D CNN ECG anomaly classifier on the PTB-XL dataset and quantized it to TFLite INT8 format, simulating a real STM32 edge deployment.
  • Attacked both the full-precision and quantized models with FGSM, PGD, and DeepFool using IBM's Adversarial Robustness Toolbox.
  • Measured and visualized the quantization robustness gap, proving that adversarial robustness tested on FP32 models does not transfer to the deployed INT8 model.
  • Secret Mission: Tested whether adversarial training defenses survive quantization, replicating a critical finding from medical device security research.

Ready to quiz yourself?