Build VisualQC with CNNs and AI Review

Compare CNN and MobileNetV2 classifiers in an AI-assisted image inspector.

Introduction

30 Second Summary

A faint scratch or blurred edge can turn a simple quality check into a judgment call. Confidence matters most when different checks reach different conclusions.

In this project, you will build VisualQC, a browser-based image-quality inspector that compares a custom CNN with a transfer-learning model. The finished inspector also offers an OpenRouter visual review as a qualitative second opinion.

What You'll Build

Your VisualQC inspection station turns a product-style image into two side-by-side classifications with confidence scores plus an optional visual review.

By the end of this project, you'll have:

  • A reproducible synthetic inspection dataset that uses OpenCV to create clean components plus controlled scratches, pits, chips, contamination, and blur.
  • A controlled model comparison showing test accuracy and confusion counts for the custom CNN plus MobileNetV2 on the same data split.
  • A browser inspection workflow built with Streamlit that displays both predictions for an uploaded image. You can also request an explanation of the visible condition.
  • Secret Mission: Add a Manual Review disposition that activates when the models disagree or confidence is low.

Are there any prerequisites?

Basic Python and notebook experience will help. You need a Windows computer with internet access plus Google and OpenRouter accounts.

Before We Start

This checkpoint defines the rule that guides every later result: VisualQC classifies synthetic product-style images as Acceptable or Defective. A same-split comparison between a custom CNN and MobileNetV2 provides the classification evidence, with OpenRouter limited to qualitative review outside the ground-truth quality gate.

Set Up the VisualQC Environment

VisualQC trains in Google Colab. It serves saved models on your Windows computer.

The local app needs a compatible Python runtime with the same pinned model dependencies. This matching environment keeps training artifacts loadable when you bring them back from Colab.

In this step, get ready to:
  • Confirm or install Python 3.12.10.
  • Build an isolated environment from requirements.txt.
  • Verify the environment with import and browser checks.
Prepare a compatible Python runtime

The pinned NumPy release requires Python 3.12 or newer. This project uses Python 3.12.10 so every local package has a compatible Windows build.

PowerShell lets you target that exact Python installation through the Windows Python launcher.

  • Press the Windows key to open search.
  • Type PowerShell into the search field.
  • Press Enter to open PowerShell.
  • Check for Python 3.12 by running this command:
py -3.12 --version

What does this command check?

The Windows Python launcher looks for an installed Python 3.12 runtime. The output identifies the exact patch version it found.

✔️ I see the required version

Your output shows Python 3.12.10. Your runtime is ready for the pinned VisualQC packages.

ⓧ I see an older version

Your Python 3.12 installation needs the project’s target patch release.

  • Open the official release page for Python 3.12.10.
  • Download the recommended 64-bit Windows installer.
  • Run the downloaded installer.
  • Approve the Windows permission request if it appears.
  • Complete the installation.

ⓧ Command not found

The Python launcher cannot find Python 3.12. Install the project’s supported Windows release before continuing.

  • Open the official release page for Python 3.12.10.
  • Download the recommended 64-bit Windows installer.
  • Run the downloaded installer.
  • Approve the Windows permission request if it appears.
  • Complete the installation.
  • Close the current PowerShell window.
  • Press the Windows key to reopen search.
  • Type PowerShell into the search field.
  • Press Enter to open a fresh PowerShell session.
  • Confirm the exact Python version by running the check again:
py -3.12 --version

What should this confirm?

The fresh PowerShell session detects the newly installed runtime. The output should identify Python 3.12.10.

That compatibility hurdle is cleared. Your computer can now create the environment that VisualQC needs.

Still missing Python 3.12?

Close every PowerShell window after installation. A terminal that stayed open may still have the previous launcher state.

If the check still fails, confirm that you used the 64-bit Windows installer from the Python 3.12.10 release page.

Still stuck? Help me check why the Windows Python launcher cannot find Python 3.12.10.

Build the local workspace

A dedicated project folder keeps the source files beside the model artifacts you download later. A virtual environment keeps VisualQC’s packages separate from other Python projects.

  • Move to your Desktop by running this command:
Set-Location ~/Desktop

What does this command do?

PowerShell changes its current location to your Desktop. The new project folder will be easy to find from File Explorer.

  • Create the visualqc folder and enter it by running these commands:
New-Item -ItemType "Directory" -Path "visualqc"
Set-Location visualqc

What do these commands do?

  • New-Item creates the visualqc directory on your Desktop.
  • Set-Location moves PowerShell into that directory.

Your PowerShell prompt now ends with visualqc. That path confirms the next files will land in the correct folder.

Does the visualqc folder already exist?

A previous attempt may have created the folder. Confirm it is the project folder you want to use before entering it.

Still stuck? Help me create and enter a visualqc folder on my Windows Desktop with PowerShell.

Visual Studio Code can open the current PowerShell folder as a workspace. It may ask you to confirm that you trust the folder because it can contain executable files.

  • Open the current visualqc folder in Visual Studio Code by running:
code .

What does this command do?

The code command launches Visual Studio Code. The dot tells it to use your current visualqc folder as the workspace.

  • Confirm that you trust the folder if Visual Studio Code displays a trust prompt.

You’ll see visualqc at the top of the Explorer sidebar.

Did Visual Studio Code stay closed?

The Visual Studio Code command may be missing from your system path. You can still open the folder through the editor.

  • Press the Windows key to open search.
  • Type Visual Studio Code into the search field.
  • Press Enter to open the editor.
  • Select File from the top menu.
  • Select Open Folder....
  • Choose the visualqc folder on your Desktop.

Still stuck? Help me open my Desktop visualqc folder in Visual Studio Code on Windows.

The dependency file gives the local environment one reproducible package set. Each version matches the environment used throughout VisualQC.

  • Select the Explorer view in the Visual Studio Code Activity Bar.
  • Select the New File... button in the Explorer.
  • Enter requirements.txt as the file name.
  • Press Enter to create the file.

The new requirements.txt file opens in the editor.

  • Add the pinned project dependencies by pasting this file content:
tensorflow==2.21.0
keras==3.15.1
numpy==2.5.3
opencv-python-headless==5.0.0.93
streamlit==1.64.0

What does this file install?

  • TensorFlow supplies the model runtime used for training and predictions.
  • Keras supplies the model-building and model-loading APIs.
  • NumPy stores the generated image arrays.
  • OpenCV is installed through opencv-python-headless. Python imports it as cv2.
  • Streamlit serves the local browser interface.
  • Press Ctrl+S to save requirements.txt.

The Explorer now lists requirements.txt inside visualqc.

✔️ Awesome, I've got everything!

Great. Confirm that requirements.txt is saved in the visualqc folder.

ⓧ I'd like to double check the full code

Compare your saved requirements.txt with this complete file:

tensorflow==2.21.0
keras==3.15.1
numpy==2.5.3
opencv-python-headless==5.0.0.93
streamlit==1.64.0

Does the dependency file look different?

Check that the file is named exactly requirements.txt. A second file extension can stop the installation command from finding it.

Check that each package occupies its own line. Keep every version pin exactly as shown.

Still stuck? Help me compare my requirements.txt file with the VisualQC dependency list.

  • Return to the PowerShell window inside visualqc.
  • Create and activate .venv by running these commands:
py -3.12 -m venv .venv
.venv\Scripts\Activate.ps1

What do these commands do?

  • The first command creates .venv with Python 3.12.
  • The second command activates that environment in the current PowerShell session.

You’ll see (.venv) at the start of the PowerShell prompt. The Explorer also shows a new .venv folder.

This installation can take a few minutes because TensorFlow is the largest package in the set. PowerShell prints download progress while it works.

  • Install every pinned dependency into the active virtual environment by running:
python -m pip install -r requirements.txt

What does this command do?

The active Python runtime asks pip to read requirements.txt. Each package is installed inside .venv at its pinned version.

You’ll see package download and installation progress. When installation finishes, PowerShell returns to the (.venv) prompt.

Did the environment fail to install?

Confirm that the prompt begins with (.venv). Activate the environment again if that prefix is missing.

Confirm that requirements.txt is saved directly inside visualqc.

Still stuck? Help me diagnose my VisualQC virtual environment or package installation.

Run the final checks

A completed installation only proves that package files were downloaded. The import check proves that all five libraries can load together inside the active environment.

Before you run this check, do you expect every import to complete from the active virtual environment?

  • Test all five package imports by running:
python -c "import tensorflow, keras, cv2, numpy, streamlit; print('VisualQC environment ready')"

What does this check prove?

Python imports TensorFlow, Keras, OpenCV, NumPy, and Streamlit in one process. The final print runs only after every import succeeds.

You’ll see VisualQC environment ready in PowerShell. That message proves the pinned libraries can load together.

Did an import fail?

Confirm that the PowerShell prompt still begins with (.venv). The check may use a different Python installation when the environment is inactive.

Run the dependency installation again if PowerShell reports that a package is missing.

Still stuck? Help me diagnose which VisualQC package cannot import inside .venv.

The final check starts Streamlit’s local example server. The command keeps control of PowerShell until you stop the server.

Before you launch the example, where do you expect to see Streamlit’s visible result?

  • Launch the Streamlit example app by running:
python -m streamlit hello

What does this command do?

The active Python runtime launches Streamlit’s included example app. Streamlit starts a local server and opens the app in your browser.

Your browser opens the Streamlit example app. That visible page confirms the local app server works.

Did the example app stay closed?

Check the PowerShell output for a local address. Open that address in your browser if the page did not open automatically.

Confirm that the terminal still shows the active Streamlit process. A returned prompt means the server stopped.

Still stuck? Help me diagnose why the Streamlit example app did not open from my active .venv.

  • Return to PowerShell.
  • Press Ctrl+C to stop the Streamlit example server.
  • Switch back to the Visual Studio Code window containing visualqc.

PowerShell returns to the active (.venv) prompt. Your local VisualQC environment is ready for the baseline classifier you’ll train next.

Generate Defects and Train the Baseline CNN

Your local VisualQC environment is ready. The next challenge is giving the project controlled images to classify.

A custom CNN gives you a first classifier. Its score alone cannot show whether learning features from scratch is the best approach.

You'll use Google Colab for the training runtime. The fresh runtime keeps this experiment separate from your local app.

In this step, get ready to:
  • Upload the project files to a fresh Colab runtime.
  • Generate a balanced image dataset and train a custom CNN.
  • Inspect the first prediction and identify the missing comparison.
Prepare a fresh Colab runtime

A Colab runtime has its own package environment. Uploading your existing requirements.txt reproduces the pinned dependencies before the baseline script runs.

  • Switch back to VS Code from the previous step.
  • Use the Explorer sidebar to create build_baseline.py inside the visualqc folder.

The full-code tab is the source of truth for the supplied baseline script.

✔️ I have the supplied file

Your supplied build_baseline.py file should match the full version in the other tab before you continue.

ⓧ I'd like to double check the full code

  • Copy this exact script into build_baseline.py:
import os
os.environ["KERAS_BACKEND"] = "tensorflow"

import json
from pathlib import Path

import cv2
import keras
import numpy as np

SEED = 42
IMAGE_SIZE = 96
SAMPLES_PER_CLASS = 300
ARTIFACTS = Path("artifacts")

ARTIFACTS.mkdir(exist_ok=True)
keras.utils.set_random_seed(SEED)
rng = np.random.default_rng(SEED)


def make_part(defective):
    image = rng.integers(
        25, 45, size=(IMAGE_SIZE, IMAGE_SIZE, 3), dtype=np.uint8
    )
    metal = int(rng.integers(125, 191))
    center = int(rng.integers(45, 52))

    cv2.rectangle(image, (14, 18), (82, 78), (metal, metal, metal), -1)
    cv2.circle(image, (center, 48), 17, (metal - 28,) * 3, -1)
    cv2.circle(image, (center, 48), 8, (metal + 25,) * 3, 2)

    noise = rng.normal(0.0, 5.0, size=image.shape)
    image = np.clip(image.astype(np.float32) + noise, 0, 255).astype(np.uint8)

    if defective:
        defect = int(rng.integers(0, 5))
        if defect == 0:
            y1 = int(rng.integers(25, 45))
            y2 = int(rng.integers(50, 72))
            cv2.line(image, (22, y1), (75, y2), (15, 15, 210), 3)
        elif defect == 1:
            x = int(rng.integers(28, 70))
            y = int(rng.integers(28, 68))
            cv2.circle(image, (x, y), 6, (25, 25, 25), -1)
        elif defect == 2:
            y = int(rng.integers(34, 60))
            cv2.rectangle(image, (14, y), (28, y + 15), (30, 30, 30), -1)
        elif defect == 3:
            image = cv2.GaussianBlur(image, (13, 13), 0)
        else:
            x = int(rng.integers(30, 68))
            y = int(rng.integers(30, 66))
            cv2.circle(image, (x, y), 9, (35, 90, 155), -1)

    return cv2.cvtColor(image, cv2.COLOR_BGR2RGB)


def build_dataset():
    images = []
    labels = []

    for _ in range(SAMPLES_PER_CLASS):
        images.append(make_part(False))
        labels.append(0)

    for _ in range(SAMPLES_PER_CLASS):
        images.append(make_part(True))
        labels.append(1)

    x = np.asarray(images, dtype=np.uint8)
    y = np.asarray(labels, dtype=np.float32)
    order = rng.permutation(len(x))
    x = x[order]
    y = y[order]

    train_end = int(len(x) * 0.70)
    validation_end = int(len(x) * 0.85)
    return (
        x[:train_end],
        y[:train_end],
        x[train_end:validation_end],
        y[train_end:validation_end],
        x[validation_end:],
        y[validation_end:],
    )


def save_first_sample(x_test, y_test, target_label, filename):
    for image, label in zip(x_test, y_test):
        if int(label) == target_label:
            bgr = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
            if not cv2.imwrite(str(ARTIFACTS / filename), bgr):
                raise RuntimeError(f"Could not save {filename}")
            return
    raise RuntimeError(f"No sample found for label {target_label}")


x_train, y_train, x_validation, y_validation, x_test, y_test = build_dataset()
np.savez_compressed(
    ARTIFACTS / "dataset.npz",
    x_train=x_train,
    y_train=y_train,
    x_validation=x_validation,
    y_validation=y_validation,
    x_test=x_test,
    y_test=y_test,
)

save_first_sample(x_test, y_test, 0, "sample_acceptable.png")
save_first_sample(x_test, y_test, 1, "sample_defective.png")

custom_model = keras.Sequential(
    [
        keras.Input(shape=(IMAGE_SIZE, IMAGE_SIZE, 3)),
        keras.layers.Rescaling(1.0 / 255.0),
        keras.layers.Conv2D(16, 3, activation="relu"),
        keras.layers.MaxPooling2D(),
        keras.layers.Conv2D(32, 3, activation="relu"),
        keras.layers.MaxPooling2D(),
        keras.layers.Conv2D(64, 3, activation="relu"),
        keras.layers.GlobalAveragePooling2D(),
        keras.layers.Dropout(0.20),
        keras.layers.Dense(1, activation="sigmoid"),
    ],
    name="custom_cnn",
)
custom_model.compile(
    optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"]
)
custom_model.fit(
    x_train,
    y_train,
    validation_data=(x_validation, y_validation),
    batch_size=32,
    epochs=4,
    verbose=2,
)

_, test_accuracy = custom_model.evaluate(x_test, y_test, verbose=0)
custom_model.save(ARTIFACTS / "custom_cnn.keras")

sample_probability = float(custom_model.predict(x_test[:1], verbose=0)[0][0])
sample_prediction = "Defective" if sample_probability >= 0.5 else "Acceptable"
sample_actual = "Defective" if int(y_test[0]) == 1 else "Acceptable"
sample_confidence = (
    sample_probability if sample_prediction == "Defective" else 1.0 - sample_probability
)

baseline_metrics = {"test_accuracy": float(test_accuracy)}
(ARTIFACTS / "baseline_metrics.json").write_text(
    json.dumps(baseline_metrics, indent=2), encoding="utf-8"
)

print("Dataset shapes:", x_train.shape, x_validation.shape, x_test.shape)
print(f"Custom CNN test accuracy: {test_accuracy:.3f}")
print(f"Sample actual: {sample_actual}")
print(f"Sample prediction: {sample_prediction} ({sample_confidence:.1%})")
print("Comparison unavailable: transfer model has not been trained yet.")

How is this script organized?

  • The make_part(defective) function draws a component image. Its argument controls whether the image receives a defect.
  • The build_dataset() function creates equal class counts. It returns the training, validation, and test arrays.
  • The final section trains custom_cnn. It saves the model, metrics, dataset, and sample images.
  • Save build_baseline.py in VS Code.
  • Visit Google Colab in your browser.
  • Complete the Google sign-in flow if prompted.
  • Create a fresh notebook from the Colab start screen.
  • Use the current runtime file browser to upload requirements.txt from your local visualqc folder.
  • Upload build_baseline.py from the same folder.

You should see both files in the runtime file browser. This confirms the notebook can access the script and its dependency pins.

The package installation can take a few minutes while Colab prepares the pinned environment.

  • Select the first code cell in the notebook.
  • Install the project dependencies by running:
!python -m pip install -r requirements.txt

What does this command do?

The leading ! passes the line from the notebook to its operating system. Python then installs every version pinned in requirements.txt.

When the cell finishes, you'll see installation output return control to the notebook. Your project packages are now available in the fresh runtime.

Package installation failed?

  • Confirm that requirements.txt appears in the runtime file browser.
  • Confirm that the uploaded filename uses lowercase letters exactly as shown.

Still stuck? Help me diagnose my Colab package installation.

Test the baseline's limits

A balanced synthetic dataset gives both classes equal representation. OpenCV draws clean components plus scratches, pits, chips, blur, or contamination.

NumPy uses SEED = 42 to make the generated order repeatable. Keras trains the custom CNN on the resulting split.

Training can take several minutes while the runtime generates 600 images and completes four epochs.

  • Add a new code cell beneath the installation output.

Before you run this, ask yourself whether one test score can identify the best architecture.

  • Generate the dataset and train the baseline by running:
!python build_baseline.py

What does this run create?

  • The script creates 300 Acceptable images and 300 Defective images.
  • The saved split contains 420 training samples. It also contains 90 validation samples plus 90 test samples.
  • The artifacts folder receives dataset.npz, custom_cnn.keras, and baseline_metrics.json.
  • The same folder receives sample_acceptable.png plus sample_defective.png.

When training finishes, you'll see Dataset shapes: followed by arrays for 420 training images, 90 validation images, and 90 test images. Each image has the shape 96 by 96 by 3.

The final line reads Comparison unavailable: transfer model has not been trained yet. That gap is the intended limit of this first experiment.

Your first classifier is working. Its saved model and prediction now give you concrete baseline evidence.

Baseline script did not finish?

  • Confirm that build_baseline.py appears in the runtime file browser.
  • Confirm that the dependency installation cell completed before the baseline cell started.
  • Re-upload both files if the Colab runtime was replaced.

Still stuck? Help me diagnose the failed baseline run.

Read the model evidence

The printed sample result connects the model's probability to a visible class decision. The confidence shows how strongly the model supports the label it selected.

  • Find the line beginning with Sample actual: in the completed output.
  • Find the line beginning with Sample prediction: directly below it.
  • Compare the actual label with the predicted label.
  • Read the confidence percentage beside the predicted label.
  • Confirm that the output ends with Comparison unavailable: transfer model has not been trained yet..

Why is this only a baseline?

The custom CNN learned its visual features from this synthetic dataset. Its accuracy measures performance on the saved test split.

A single score cannot show how another architecture performs under the same conditions. The missing comparison keeps the current conclusion deliberately limited.

Your reproducible dataset and custom CNN are ready. Next, you'll train MobileNetV2 on the same split and turn that missing comparison into evidence.

Train MobileNetV2 and Compare the Models

Your custom CNN can now classify the synthetic components. Its test score gives you a baseline for the inspection task.

One score cannot show whether learning every feature from scratch was the best choice. You will train a frozen MobileNetV2 feature extractor on the same split. You will then compare both models under identical test conditions.

In this step, get ready to:
  • Add the supplied transfer-learning script to the existing Colab runtime.
  • Train MobileNetV2 and compare both models on the shared test split.
  • Download the complete model artifacts into the local visualqc folder.
Add the transfer-learning script

A controlled model comparison keeps the dataset fixed while changing the model architecture. The new script reuses artifacts/dataset.npz from the previous step. This keeps the training, validation, and test samples identical.

Why freeze MobileNetV2?

Transfer learning starts with features learned from a larger dataset. The frozen ImageNet base preserves those features while the new binary head learns your Acceptable-versus-Defective task.

Freezing the base also limits the number of weights trained on your small synthetic dataset. The comparison still lets the measured test results decide which model performs better.

  • Return to the Google Colab runtime from the previous step.
  • Select the folder icon in Colab's left sidebar to show the runtime files.
  • Upload the supplied build_transfer.py file using the file pane's upload control.
  • Confirm that build_transfer.py appears beside build_baseline.py and requirements.txt.

Good progress. The second experiment now sits in the same runtime as the saved dataset and custom model.

✔️ Awesome, I've got everything!

Your Colab file pane now shows build_transfer.py beside the baseline files.

ⓧ I'd like to double check the full code

  • Compare the uploaded build_transfer.py file with this read-only reference:
import os
os.environ["KERAS_BACKEND"] = "tensorflow"

import json
import shutil
from pathlib import Path

import keras
import numpy as np

SEED = 42
IMAGE_SIZE = 96
ARTIFACTS = Path("artifacts")

keras.utils.set_random_seed(SEED)
data = np.load(ARTIFACTS / "dataset.npz", allow_pickle=False)
x_train = data["x_train"]
y_train = data["y_train"]
x_validation = data["x_validation"]
y_validation = data["y_validation"]
x_test = data["x_test"]
y_test = data["y_test"]

base_model = keras.applications.MobileNetV2(
    input_shape=(IMAGE_SIZE, IMAGE_SIZE, 3),
    include_top=False,
    weights="imagenet",
)
base_model.trainable = False

transfer_model = keras.Sequential(
    [
        keras.Input(shape=(IMAGE_SIZE, IMAGE_SIZE, 3)),
        keras.layers.Rescaling(1.0 / 127.5, offset=-1.0),
        base_model,
        keras.layers.GlobalAveragePooling2D(),
        keras.layers.Dropout(0.20),
        keras.layers.Dense(1, activation="sigmoid"),
    ],
    name="mobilenet_v2_quality",
)
transfer_model.compile(
    optimizer="adam", loss="binary_crossentropy", metrics=["accuracy"]
)
transfer_model.fit(
    x_train,
    y_train,
    validation_data=(x_validation, y_validation),
    batch_size=32,
    epochs=3,
    verbose=2,
)
transfer_model.save(ARTIFACTS / "mobilenet_v2.keras")

custom_model = keras.saving.load_model(
    ARTIFACTS / "custom_cnn.keras", compile=False
)


def comparison_metrics(model):
    probabilities = model.predict(x_test, batch_size=32, verbose=0).reshape(-1)
    predictions = (probabilities >= 0.5).astype(np.int32)
    truth = y_test.astype(np.int32)
    accuracy = float((predictions == truth).mean())
    true_negative = int(((predictions == 0) & (truth == 0)).sum())
    false_positive = int(((predictions == 1) & (truth == 0)).sum())
    false_negative = int(((predictions == 0) & (truth == 1)).sum())
    true_positive = int(((predictions == 1) & (truth == 1)).sum())
    return {
        "accuracy": accuracy,
        "confusion_matrix": [
            [true_negative, false_positive],
            [false_negative, true_positive],
        ],
    }


results = {
    "custom_cnn": comparison_metrics(custom_model),
    "mobilenet_v2": comparison_metrics(transfer_model),
}
results["winner"] = max(
    ("custom_cnn", "mobilenet_v2"), key=lambda name: results[name]["accuracy"]
)

(ARTIFACTS / "comparison_metrics.json").write_text(
    json.dumps(results, indent=2), encoding="utf-8"
)
shutil.make_archive(
    "visualqc_artifacts", "zip", root_dir=".", base_dir="artifacts"
)

print("Custom CNN:", results["custom_cnn"])
print("MobileNetV2:", results["mobilenet_v2"])
print("Comparison winner:", results["winner"])
print("Created visualqc_artifacts.zip")

What should the complete file contain?

The file loads the saved split before creating the transfer model. It trains only the new classification head while the MobileNetV2 base remains frozen.

The final section evaluates both models at the same threshold. It saves the metrics and packages the complete artifacts directory for download.

Train and compare both models

The uploaded script trains the new binary head for three epochs. It leaves custom_cnn.keras unchanged. Both models then predict the same 90 test images using a defective-probability threshold of 0.5.

Expect this run to take several minutes while Colab downloads the pretrained weights and trains the new head.

Before you run the comparison, which model do you think will achieve the higher test accuracy?

  • Run the transfer-learning experiment in a new Colab code cell with this command:
!python build_transfer.py

What does this script do?

  • The script loads the shared training, validation, and test arrays from artifacts/dataset.npz.
  • Keras creates a MobileNetV2 base with pretrained ImageNet weights.
  • The new classification head learns a single defective probability while the pretrained base stays frozen.
  • A confusion matrix records true negatives, false positives, false negatives, and true positives for each model.
  • The script saves comparison_metrics.json and creates visualqc_artifacts.zip.

You will see output lines beginning with Custom CNN:, MobileNetV2:, and Comparison winner:. The final line reads Created visualqc_artifacts.zip.

That completes the controlled comparison. You now have measured results for both architectures on the same test split.

Did the comparison stop early?

  • Check that build_transfer.py is in the same Colab runtime as the artifacts folder.
  • Confirm that the earlier installation cell completed before you run the script.
  • Rerun the command if the pretrained weight download was interrupted by a temporary network problem.

Still stuck? Help me diagnose why build_transfer.py stopped in Google Colab.

Download and extract the artifacts

Colab runtime files are temporary. Downloading the archive preserves both trained models and the shared comparison data on your computer.

  • Locate visualqc_artifacts.zip in Colab's file pane.
  • Download visualqc_artifacts.zip using the file's Download control.
  • Press Windows+E to open File Explorer.
  • Select the Downloads folder in the left sidebar.
  • Right-click visualqc_artifacts.zip.
  • Select Extract All....
  • Choose the local visualqc folder from earlier as the extraction destination.
  • Complete the extraction using the prompts in File Explorer.

Windows extracts the archive into your local project. The resulting artifacts directory sits directly inside visualqc.

Before you inspect the extracted folder, which saved models and comparison files do you expect to find?

  • Return to the local visualqc folder in File Explorer.
  • Open the extracted artifacts folder.

What should be inside the extracted folder?

  • The trained models are custom_cnn.keras and mobilenet_v2.keras.
  • The shared dataset is dataset.npz.
  • The generated examples are sample_acceptable.png and sample_defective.png.
  • The recorded results are baseline_metrics.json and comparison_metrics.json.

You have preserved the complete experiment locally. Both models and their controlled comparison are ready for the browser-based inspector.

Is the artifacts folder in the wrong place?

  • Move the extracted artifacts folder directly into visualqc if the archive created an extra outer folder.
  • Download the archive again if comparison_metrics.json or mobilenet_v2.keras is missing.

Need help? Help me place the extracted VisualQC artifacts correctly.

Your two-model experiment is complete. Next, you will turn these saved models into an inspector that accepts an uploaded image and displays both predictions.

Build the Streamlit Inspector

Your custom CNN and MobileNetV2 classifier now share the same test split. Their trained models are ready in your local artifacts folder.

Notebook metrics cannot provide a practical upload-and-classify workflow. Streamlit turns both saved models into a browser-based inspector that shows their predictions side by side.

In this step, get ready to:
  • Create a local inspector that checks for both saved model files.
  • Load both models and preview an uploaded product image.
  • Display both classifications with confidence scores and a disagreement warning.
Create the local inspector shell

The inspector needs stable paths to both model files before it can accept an image. It also needs a clear warning if the extracted artifacts are missing.

  • Switch back to the visualqc folder in Visual Studio Code.
  • Select the visualqc folder in the Explorer sidebar.
  • Click the New File button in the Explorer toolbar.
  • Enter app.py as the file name.
  • Paste the following app shell into app.py:
import os
os.environ["KERAS_BACKEND"] = "tensorflow"

from pathlib import Path

import cv2
import keras
import numpy as np
import streamlit as st

IMAGE_SIZE = 96
ARTIFACTS = Path(__file__).parent / "artifacts"
CUSTOM_PATH = ARTIFACTS / "custom_cnn.keras"
TRANSFER_PATH = ARTIFACTS / "mobilenet_v2.keras"

st.title("VisualQC Image Quality Inspector")
st.write(
    "Upload a product-style image to compare a custom CNN with MobileNetV2."
)

if not CUSTOM_PATH.exists() or not TRANSFER_PATH.exists():
    st.warning(
        "Model files are missing. Extract visualqc_artifacts.zip so the artifacts "
        "folder sits beside app.py."
    )

What Does This Shell Do?

  • The KERAS_BACKEND setting selects TensorFlow before Keras is imported.
  • The model paths point from app.py to the two .keras files inside artifacts.
  • The file check gives the learner a visible recovery path when either trained model is missing.

The Streamlit command starts a local server that keeps the PowerShell terminal busy. You can keep editing app.py while that server runs.

  • Save app.py.
  • Start the inspector from the active PowerShell terminal by running:
python -m streamlit run app.py

What Does This Command Do?

The command runs app.py through the active virtual environment. Streamlit starts the local app and opens it in your browser.

You should see the VisualQC Image Quality Inspector heading with a short description beneath it.

Good progress. Your browser now has the shell that the upload and prediction workflow builds on.

Inspector Not Opening?

  • Check that the PowerShell prompt showed the active .venv before you started the app.
  • Confirm that app.py sits inside the local visualqc folder.
  • Confirm that the extracted artifacts folder sits beside app.py.

Still stuck? Help me diagnose why my VisualQC Streamlit app does not open from the active virtual environment.

Load models and preview uploads

The app loads both saved Keras models once and reuses them across Streamlit reruns. The upload helper uses OpenCV and NumPy to convert browser file bytes into the same RGB shape used during training.

  • Find the TRANSFER_PATH = ARTIFACTS / "mobilenet_v2.keras" line in app.py.
  • Insert the following helpers below that line:
@st.cache_resource
def load_models():
    custom = keras.saving.load_model(CUSTOM_PATH, compile=False)
    transfer = keras.saving.load_model(TRANSFER_PATH, compile=False)
    return custom, transfer


def decode_image(image_bytes):
    encoded = np.frombuffer(image_bytes, dtype=np.uint8)
    image_bgr = cv2.imdecode(encoded, cv2.IMREAD_COLOR)
    if image_bgr is None:
        raise ValueError("The uploaded file could not be decoded as an image.")
    image_rgb = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2RGB)
    resized = cv2.resize(
        image_rgb, (IMAGE_SIZE, IMAGE_SIZE), interpolation=cv2.INTER_AREA
    )
    batch = resized.astype(np.float32)[None, ...]
    return image_rgb, batch


def prediction_result(defective_probability):
    if defective_probability >= 0.5:
        return "Defective", defective_probability
    return "Acceptable", 1.0 - defective_probability

How Do the Helpers Prepare an Inspection?

  • The load_models() function loads both classifiers without restoring their training configuration.
  • The decode_image() function decodes the uploaded bytes and converts OpenCV's BGR channels to RGB.
  • The resized batch uses the trained input size of 96 by 96 pixels.
  • The prediction_result() function converts a defective probability into a label and confidence value.
  • Save app.py.
  • Check the running PowerShell terminal for a clean Streamlit rerun without a Python traceback.

Seeing a Python Traceback?

  • Check that each helper starts at the far-left edge of app.py.
  • Check that the body of each helper uses four spaces of indentation.
  • Confirm that os.environ["KERAS_BACKEND"] = "tensorflow" remains above the Keras import.

Need help with the traceback? Help me compare my VisualQC helper functions with the expected indentation and import order.

The model-file check now needs a successful branch. That branch loads the models and presents a PNG or JPEG upload control.

  • Find the closing parenthesis of the existing model-files warning.
  • Paste the following branch directly below it:
else:
    custom_model, transfer_model = load_models()
    uploaded = st.file_uploader(
        "Upload a PNG or JPEG image", type=["png", "jpg", "jpeg"]
    )

What Does the Successful Branch Do?

  • The else branch runs only when both saved model paths exist.
  • The cached loader prepares both classifiers before an image is selected.
  • The uploader limits the visible file choices to PNG and JPEG images.
  • Save app.py.
  • Return to the running browser page.
  • Confirm that the image upload control now appears beneath the description.

Upload Control Missing?

  • Check that else: aligns with the model-files if statement.
  • Check that the model-loading and uploader lines are indented inside the else branch.
  • Confirm that both custom_cnn.keras and mobilenet_v2.keras exist inside artifacts.

Still missing the uploader? Help me find why the upload control does not appear in my VisualQC Streamlit app.

An uploaded file arrives as bytes. The next branch decodes those bytes while keeping invalid files from crashing the inspector.

  • Find the closing parenthesis of the st.file_uploader() call.
  • Paste the following image-preview branch directly below it:
    if uploaded is not None:
        try:
            original_rgb, model_batch = decode_image(uploaded.getvalue())
        except ValueError as error:
            st.warning(str(error))
        else:
            st.image(original_rgb, caption="Inspection image", width="stretch")

How Does the Preview Stay Safe?

  • The branch runs only after the learner supplies an uploaded file.
  • The try block passes the file bytes through decode_image().
  • A decoding problem becomes an in-app warning instead of terminating the server.
  • A valid image remains at its original display size while the separate model batch uses the trained dimensions.
  • Save app.py.
  • Use the browser upload control to select artifacts/sample_defective.png.
  • Confirm that the generated component image appears with the Inspection image caption.

That image preview is working. VisualQC can now turn a browser upload into the RGB data both classifiers expect.

Image Not Appearing?

  • Confirm that you selected sample_defective.png from the local artifacts folder.
  • Check that the entire preview branch remains indented inside the model-files else branch.
  • Check that st.image() remains inside the decoding else branch.

Need help with the upload path? Help me diagnose why sample_defective.png does not appear in my VisualQC app.

Display both model predictions

Each sigmoid output represents the model's defective probability. VisualQC converts that probability into a quality label and reports confidence in the selected label.

  • Find the st.image(original_rgb, caption="Inspection image", width="stretch") line.
  • Paste the following prediction display directly below it:
            custom_score = float(
                custom_model.predict(model_batch, verbose=0)[0][0]
            )
            transfer_score = float(
                transfer_model.predict(model_batch, verbose=0)[0][0]
            )
            custom_label, custom_confidence = prediction_result(custom_score)
            transfer_label, transfer_confidence = prediction_result(transfer_score)

            custom_column, transfer_column = st.columns(2)
            custom_column.metric(
                "Custom CNN",
                custom_label,
                f"{custom_confidence:.1%} confidence",
                delta_color="off",
            )
            transfer_column.metric(
                "MobileNetV2",
                transfer_label,
                f"{transfer_confidence:.1%} confidence",
                delta_color="off",
            )

            if custom_label != transfer_label:
                st.warning("The models disagree. Treat this item as a manual review case.")
            else:
                st.write(f"Both models classify this image as {custom_label}.")

How Does the Comparison Work?

  • Each model receives the identical uploaded-image batch and returns one defective probability.
  • The prediction_result() helper applies the shared 0.5 classification threshold.
  • The two metric cards keep each label beside its confidence score.
  • The disagreement warning makes conflicting classifications visible without choosing an automatic winner.
  • Save app.py.

✔️ Awesome, I've got everything!

Great. Confirm that app.py is saved before you run the final inspection check.

ⓧ I'd like to double check the full code

Compare your local-only inspector with this complete app.py file.

import os
os.environ["KERAS_BACKEND"] = "tensorflow"

from pathlib import Path

import cv2
import keras
import numpy as np
import streamlit as st

IMAGE_SIZE = 96
ARTIFACTS = Path(__file__).parent / "artifacts"
CUSTOM_PATH = ARTIFACTS / "custom_cnn.keras"
TRANSFER_PATH = ARTIFACTS / "mobilenet_v2.keras"


@st.cache_resource
def load_models():
    custom = keras.saving.load_model(CUSTOM_PATH, compile=False)
    transfer = keras.saving.load_model(TRANSFER_PATH, compile=False)
    return custom, transfer


def decode_image(image_bytes):
    encoded = np.frombuffer(image_bytes, dtype=np.uint8)
    image_bgr = cv2.imdecode(encoded, cv2.IMREAD_COLOR)
    if image_bgr is None:
        raise ValueError("The uploaded file could not be decoded as an image.")
    image_rgb = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2RGB)
    resized = cv2.resize(
        image_rgb, (IMAGE_SIZE, IMAGE_SIZE), interpolation=cv2.INTER_AREA
    )
    batch = resized.astype(np.float32)[None, ...]
    return image_rgb, batch


def prediction_result(defective_probability):
    if defective_probability >= 0.5:
        return "Defective", defective_probability
    return "Acceptable", 1.0 - defective_probability


st.title("VisualQC Image Quality Inspector")
st.write(
    "Upload a product-style image to compare a custom CNN with MobileNetV2."
)

if not CUSTOM_PATH.exists() or not TRANSFER_PATH.exists():
    st.warning(
        "Model files are missing. Extract visualqc_artifacts.zip so the artifacts "
        "folder sits beside app.py."
    )
else:
    custom_model, transfer_model = load_models()
    uploaded = st.file_uploader(
        "Upload a PNG or JPEG image", type=["png", "jpg", "jpeg"]
    )

    if uploaded is not None:
        try:
            original_rgb, model_batch = decode_image(uploaded.getvalue())
        except ValueError as error:
            st.warning(str(error))
        else:
            st.image(original_rgb, caption="Inspection image", width="stretch")
            custom_score = float(
                custom_model.predict(model_batch, verbose=0)[0][0]
            )
            transfer_score = float(
                transfer_model.predict(model_batch, verbose=0)[0][0]
            )
            custom_label, custom_confidence = prediction_result(custom_score)
            transfer_label, transfer_confidence = prediction_result(transfer_score)

            custom_column, transfer_column = st.columns(2)
            custom_column.metric(
                "Custom CNN",
                custom_label,
                f"{custom_confidence:.1%} confidence",
                delta_color="off",
            )
            transfer_column.metric(
                "MobileNetV2",
                transfer_label,
                f"{transfer_confidence:.1%} confidence",
                delta_color="off",
            )

            if custom_label != transfer_label:
                st.warning("The models disagree. Treat this item as a manual review case.")
            else:
                st.write(f"Both models classify this image as {custom_label}.")
  • Stop the current Streamlit server by pressing Ctrl+C in the PowerShell terminal.

Before you restart the inspector, what results do you expect both saved classifiers to show for the defective sample?

  • Restart the completed inspector by running:
python -m streamlit run app.py

What Does the Final Run Check?

This fresh run loads the saved custom CNN and MobileNetV2 model through the completed app. It verifies the same file that you just compared in the full-code tab.

  • Upload artifacts/sample_defective.png through the browser control.
  • Compare the Custom CNN and MobileNetV2 metric cards.
  • Check each card for a classification label and confidence percentage.
  • Look below the cards for either the shared classification or a visible disagreement warning.

You should see the uploaded component image plus one label and confidence value from each model. If their labels differ, you should also see the manual-review warning.

You now have a working inspection station. One uploaded image produces two directly comparable model decisions in the browser.

Predictions Not Appearing?

  • Check that the prediction block is indented beneath the image-decoding else branch.
  • Confirm that both metric calls use custom_confidence and transfer_confidence from prediction_result().
  • Confirm that the uploaded file is artifacts/sample_defective.png from the extracted model archive.

Still missing the results? Help me debug why my VisualQC app previews the image but does not show both model predictions.

Your local inspector now turns both saved classifiers into a usable quality-check workflow. Next up, you'll add a qualitative visual review for cases that need more context.

Add OpenRouter Visual Review

Your local inspector now produces repeatable labels and confidence scores from both classifiers. Those scores give you a quality gate you can run on every uploaded image.

The classifiers cannot explain the visible evidence behind a result. In this step, OpenRouter adds a qualitative second opinion while your local models remain responsible for binary classification.

In this step, get ready to:
  • Create reviewer.py to encode an uploaded image for multimodal review.
  • Add a masked API key field and review controls to app.py.
  • Request a review with a generated image and confirm the routed model.
Create the OpenRouter reviewer

The reviewer converts the uploaded RGB image into a JPEG data URL. It sends that image with both classifier labels to the OpenRouter chat-completions endpoint.

Why keep the reviewer separate?

The reviewer handles image encoding and the external request. Keeping this work in reviewer.py leaves the Streamlit interface focused on interaction and display.

Your two saved classifiers still decide the repeatable Acceptable or Defective result. The external reviewer only explains visible evidence.

  • In the VS Code Explorer sidebar for visualqc, click the new-file control.
  • Name the file reviewer.py.
  • Add the image-encoding foundation by pasting this code into reviewer.py:
import base64
import json
import urllib.error
import urllib.request

import cv2

OPENROUTER_URL = "https://openrouter.ai/api/v1/chat/completions"
OPENROUTER_MODEL = "openrouter/free"


def review_image(image_rgb, api_key, custom_label, transfer_label):
    image_bgr = cv2.cvtColor(image_rgb, cv2.COLOR_RGB2BGR)
    encoded_ok, encoded_image = cv2.imencode(".jpg", image_bgr)
    if not encoded_ok:
        raise RuntimeError("OpenCV could not encode the image for review.")

    image_base64 = base64.b64encode(encoded_image.tobytes()).decode("ascii")
    data_url = f"data:image/jpeg;base64,{image_base64}"

What does this foundation do?

  • The constants identify the OpenRouter endpoint and its free model router.
  • The colour conversion changes the uploaded image from RGB to the BGR order expected by OpenCV encoding.
  • The JPEG bytes become base64 text that can travel inside a data URL.
  • The explicit encoding check turns an image-conversion failure into a clear RuntimeError.
  • Save reviewer.py.
  • Confirm reviewer.py appears beside app.py in the VS Code Explorer sidebar.

Don't see the reviewer file?

Check that the file is named reviewer.py with no extra file extension. Confirm it sits directly inside visualqc.

Still stuck? Help me check where reviewer.py was created in my VisualQC project.

A multimodal request contains ordered content parts. The text goes first so the reviewer receives its task and classifier context before the image.

  • In reviewer.py, place the following code directly below the data_url line:
    prompt = (
        "Act as a visual quality reviewer. Inspect only visible evidence in this "
        "product-style image. The custom CNN predicted "
        f"{custom_label}, and MobileNetV2 predicted {transfer_label}. "
        "Return three short bullet points: visible condition, likely defect if any, "
        "and whether a human should review the item. Do not claim certainty."
    )
    payload = {
        "model": OPENROUTER_MODEL,
        "messages": [
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": prompt},
                    {"type": "image_url", "image_url": {"url": data_url}},
                ],
            }
        ],
    }

How is the request structured?

  • The prompt includes the custom CNN label and the MobileNetV2 label as context.
  • The prompt limits the reviewer to visible evidence and asks it to avoid certainty.
  • The payload places its text content item before its image_url content item.
  • The openrouter/free router selects an available free model that supports image input.
  • Save reviewer.py.
  • Confirm the payload lists the text content item above the image_url content item.

Is the payload underlined in red?

Check that the new code remains indented inside review_image(). Make sure every opening brace has a matching closing brace.

Need another pair of eyes? Help me inspect the indentation and content order in my OpenRouter payload.

The final part creates the authenticated request and translates network failures into messages the app can display. It also returns the model name selected by the free router.

  • Complete review_image() by adding this code directly below the payload:
    request = urllib.request.Request(
        OPENROUTER_URL,
        data=json.dumps(payload).encode("utf-8"),
        headers={
            "Authorization": f"Bearer {api_key}",
            "Content-Type": "application/json",
        },
        method="POST",
    )

    try:
        with urllib.request.urlopen(request, timeout=60) as response:
            result = json.loads(response.read().decode("utf-8"))
    except urllib.error.HTTPError as error:
        details = error.read().decode("utf-8", errors="replace")
        raise RuntimeError(f"OpenRouter returned HTTP {error.code}: {details}") from error
    except urllib.error.URLError as error:
        raise RuntimeError(f"Could not reach OpenRouter: {error.reason}") from error

    content = result["choices"][0]["message"]["content"]
    routed_model = result.get("model", OPENROUTER_MODEL)
    return content, routed_model

How does the request finish?

  • The request places the API key in the Bearer authorization header.
  • The timeout prevents a stalled provider request from waiting forever.
  • The error handlers preserve useful HTTP or connection details inside a RuntimeError.
  • The return values contain the written review and the routed model name.
  • Save reviewer.py.
  • Confirm VS Code shows no red syntax underlines in reviewer.py.

Seeing a syntax warning in reviewer.py?

Check that the request and error-handling code stays inside review_image() with four spaces of indentation. Confirm both except blocks align with try.

Still stuck? Help me compare my reviewer.py indentation with the completed function.

✔️ Awesome, I've got everything!

Your reviewer.py file now encodes the image and returns both parts of the OpenRouter response.

ⓧ I'd like to double check the full code

import base64
import json
import urllib.error
import urllib.request

import cv2

OPENROUTER_URL = "https://openrouter.ai/api/v1/chat/completions"
OPENROUTER_MODEL = "openrouter/free"


def review_image(image_rgb, api_key, custom_label, transfer_label):
    image_bgr = cv2.cvtColor(image_rgb, cv2.COLOR_RGB2BGR)
    encoded_ok, encoded_image = cv2.imencode(".jpg", image_bgr)
    if not encoded_ok:
        raise RuntimeError("OpenCV could not encode the image for review.")

    image_base64 = base64.b64encode(encoded_image.tobytes()).decode("ascii")
    data_url = f"data:image/jpeg;base64,{image_base64}"
    prompt = (
        "Act as a visual quality reviewer. Inspect only visible evidence in this "
        "product-style image. The custom CNN predicted "
        f"{custom_label}, and MobileNetV2 predicted {transfer_label}. "
        "Return three short bullet points: visible condition, likely defect if any, "
        "and whether a human should review the item. Do not claim certainty."
    )
    payload = {
        "model": OPENROUTER_MODEL,
        "messages": [
            {
                "role": "user",
                "content": [
                    {"type": "text", "text": prompt},
                    {"type": "image_url", "image_url": {"url": data_url}},
                ],
            }
        ],
    }
    request = urllib.request.Request(
        OPENROUTER_URL,
        data=json.dumps(payload).encode("utf-8"),
        headers={
            "Authorization": f"Bearer {api_key}",
            "Content-Type": "application/json",
        },
        method="POST",
    )

    try:
        with urllib.request.urlopen(request, timeout=60) as response:
            result = json.loads(response.read().decode("utf-8"))
    except urllib.error.HTTPError as error:
        details = error.read().decode("utf-8", errors="replace")
        raise RuntimeError(f"OpenRouter returned HTTP {error.code}: {details}") from error
    except urllib.error.URLError as error:
        raise RuntimeError(f"Could not reach OpenRouter: {error.reason}") from error

    content = result["choices"][0]["message"]["content"]
    routed_model = result.get("model", OPENROUTER_MODEL)
    return content, routed_model

How to compare this file

Check the imports and constants first. Then compare the complete review_image() function from image encoding through its return values.

Connect the reviewer to Streamlit

The app already holds the original RGB image and both model labels after an upload. The new controls pass those existing values to review_image() only when you request an explanation.

  • Switch back to app.py in VS Code.
  • Find this import block near the top of the file:
import numpy as np
import streamlit as st

Where does the reviewer import belong?

The existing package imports finish with streamlit as st. The project-specific reviewer import belongs directly below that group.

  • Add the reviewer import below the existing package imports so the block looks like this:
import numpy as np
import streamlit as st

from reviewer import review_image

What changed in the imports?

The app can now call review_image() from the new local module. The existing model and image imports stay unchanged.

  • Find the existing disagreement block near the bottom of app.py.
  • Confirm it ends with this code:
            if custom_label != transfer_label:
                st.warning("The models disagree. Treat this item as a manual review case.")
            else:
                st.write(f"Both models classify this image as {custom_label}.")

Why place the controls here?

This block runs after both model predictions exist. Placing the reviewer controls below it gives the request access to the image and both labels.

  • Add the review controls directly below the disagreement block:
            api_key = st.text_input("OpenRouter API key", type="password")
            review_clicked = st.button(
                "Ask OpenRouter for a visual review", disabled=not bool(api_key)
            )
            if review_clicked:
                with st.spinner("Requesting a qualitative visual review..."):
                    try:
                        review, routed_model = review_image(
                            original_rgb,
                            api_key,
                            custom_label,
                            transfer_label,
                        )
                    except RuntimeError as error:
                        st.warning(str(error))
                    else:
                        st.write(review)
                        st.write(f"Routed model: {routed_model}")
                        st.write(
                            "OpenRouter is a qualitative reviewer, not the quality gate."
                        )

What do the review controls do?

  • The password input masks the API key while you type it.
  • The review button stays disabled until the field contains a key.
  • The spinner keeps the page responsive while the external request is in progress.
  • The error branch displays encoding, HTTP, or connection failures as warnings.
  • The success branch displays the review and the routed model name.
  • Save app.py.
  • Confirm VS Code shows no red syntax underlines beneath the new import or review controls.

Is the reviewer integration underlined in red?

Check that reviewer.py sits beside app.py. Confirm the imported function name is exactly review_image.

If the controls are underlined, check that they align with the existing disagreement block inside the uploaded-image branch. Help me fix the reviewer import or indentation in app.py.

✔️ Awesome, I've got everything!

Your saved app.py now exposes the masked review workflow beneath the two classifier results.

ⓧ I'd like to double check the full code

import os
os.environ["KERAS_BACKEND"] = "tensorflow"

from pathlib import Path

import cv2
import keras
import numpy as np
import streamlit as st

from reviewer import review_image

IMAGE_SIZE = 96
ARTIFACTS = Path(__file__).parent / "artifacts"
CUSTOM_PATH = ARTIFACTS / "custom_cnn.keras"
TRANSFER_PATH = ARTIFACTS / "mobilenet_v2.keras"


@st.cache_resource
def load_models():
    custom = keras.saving.load_model(CUSTOM_PATH, compile=False)
    transfer = keras.saving.load_model(TRANSFER_PATH, compile=False)
    return custom, transfer


def decode_image(image_bytes):
    encoded = np.frombuffer(image_bytes, dtype=np.uint8)
    image_bgr = cv2.imdecode(encoded, cv2.IMREAD_COLOR)
    if image_bgr is None:
        raise ValueError("The uploaded file could not be decoded as an image.")
    image_rgb = cv2.cvtColor(image_bgr, cv2.COLOR_BGR2RGB)
    resized = cv2.resize(
        image_rgb, (IMAGE_SIZE, IMAGE_SIZE), interpolation=cv2.INTER_AREA
    )
    batch = resized.astype(np.float32)[None, ...]
    return image_rgb, batch


def prediction_result(defective_probability):
    if defective_probability >= 0.5:
        return "Defective", defective_probability
    return "Acceptable", 1.0 - defective_probability


st.title("VisualQC Image Quality Inspector")
st.write(
    "Upload a product-style image to compare a custom CNN with MobileNetV2."
)

if not CUSTOM_PATH.exists() or not TRANSFER_PATH.exists():
    st.warning(
        "Model files are missing. Extract visualqc_artifacts.zip so the artifacts "
        "folder sits beside app.py."
    )
else:
    custom_model, transfer_model = load_models()
    uploaded = st.file_uploader(
        "Upload a PNG or JPEG image", type=["png", "jpg", "jpeg"]
    )

    if uploaded is not None:
        try:
            original_rgb, model_batch = decode_image(uploaded.getvalue())
        except ValueError as error:
            st.warning(str(error))
        else:
            st.image(original_rgb, caption="Inspection image", width="stretch")
            custom_score = float(
                custom_model.predict(model_batch, verbose=0)[0][0]
            )
            transfer_score = float(
                transfer_model.predict(model_batch, verbose=0)[0][0]
            )
            custom_label, custom_confidence = prediction_result(custom_score)
            transfer_label, transfer_confidence = prediction_result(transfer_score)

            custom_column, transfer_column = st.columns(2)
            custom_column.metric(
                "Custom CNN",
                custom_label,
                f"{custom_confidence:.1%} confidence",
                delta_color="off",
            )
            transfer_column.metric(
                "MobileNetV2",
                transfer_label,
                f"{transfer_confidence:.1%} confidence",
                delta_color="off",
            )

            if custom_label != transfer_label:
                st.warning("The models disagree. Treat this item as a manual review case.")
            else:
                st.write(f"Both models classify this image as {custom_label}.")

            api_key = st.text_input("OpenRouter API key", type="password")
            review_clicked = st.button(
                "Ask OpenRouter for a visual review", disabled=not bool(api_key)
            )
            if review_clicked:
                with st.spinner("Requesting a qualitative visual review..."):
                    try:
                        review, routed_model = review_image(
                            original_rgb,
                            api_key,
                            custom_label,
                            transfer_label,
                        )
                    except RuntimeError as error:
                        st.warning(str(error))
                    else:
                        st.write(review)
                        st.write(f"Routed model: {routed_model}")
                        st.write(
                            "OpenRouter is a qualitative reviewer, not the quality gate."
                        )

How to compare this file

Check the reviewer import near the top first. Then compare the masked key field and request-handling block beneath the existing disagreement logic.

Request a qualitative review

This request sends the uploaded image to a model provider selected through OpenRouter. Use a generated sample or another non-sensitive image because provider data policies can vary.

Keep inspection images non-sensitive

Use artifacts/sample_acceptable.png or artifacts/sample_defective.png for this test. Avoid personal, confidential, or production images.

The free router costs zero for prompt and completion usage. Its available models and capacity can change.

API keys deserve careful handling. This app passes your key from a password-masked field into the request without writing it to source code or disk.

  • Return to the PowerShell terminal from earlier.
  • Start the updated VisualQC app by running this command:
python -m streamlit run app.py

What does this command do?

The command starts app.py through the Streamlit module. Your browser loads the updated inspector with the review controls.

  • Upload artifacts/sample_defective.png in the VisualQC browser page.
  • Enter your existing OpenRouter API key in the OpenRouter API key field.

Before you request the review, do you expect the response to report a fixed model name or the name chosen by the free router?

  • Select Ask OpenRouter for a visual review.

You'll see a written review followed by a Routed model: line. If free capacity is temporarily unavailable, you'll see a warning with the returned failure details.

How should you use this review?

The written response can describe visible scratches, blur, contamination, or other image evidence. The routed model can vary between requests.

Your local custom CNN and MobileNetV2 outputs remain the binary quality gate. The external response provides a qualitative second opinion.

Review button disabled or request unavailable?

A disabled button means the password field is empty. Enter your existing key without adding it to either Python file.

A warning after the spinner can indicate an invalid key, a network problem, or temporary free-router capacity limits. Help me diagnose my VisualQC OpenRouter review warning without exposing my API key.

You now have repeatable local classifications plus a qualitative explanation path. The routed model name keeps that external review transparent.

Secret mission

Add a Manual Review Policy

Give VisualQC an explicit escalation path for uncertain inspections. Model disagreement or confidence below the policy threshold will produce Manual Review. OpenRouter will become available only for those cases.

Clean Up Your Resources

Clean Up Your Resources

Choose whether to keep your local files, pause the running app for later, or delete the project entirely. After Streamlit is stopped, nothing from VisualQC keeps running or accrues an ongoing cost.

Resources you used:

  • The visualqc source files: requirements.txt, build_baseline.py, build_transfer.py, reviewer.py, and app.py.
  • The .venv virtual environment inside visualqc.
  • The artifacts directory containing the dataset, metrics, sample images, and two .keras models.
  • The visualqc_artifacts.zip archive.
  • The local Streamlit process if it is still running.

Why Colab and OpenRouter Stay Off the List

Google Colab runtime files are temporary, so they do not require local cleanup.

The OpenRouter router openrouter/free has zero prompt and completion pricing. Its review requests do not create a running project service.

Keep everything running

No action is needed while you are still actively building. The visualqc folder keeps the complete project ready for another session.

  • Leave the visualqc folder in its current location.

Your trained models, comparison data, sample images, and manual-review policy remain available for future tests.

Pause - I'll come back to this later

Stopping the local app frees its terminal process. Every project file remains available for your next session.

  • Switch back to the PowerShell window if Streamlit is still running.
  • Press Ctrl+C to stop Streamlit.
  • Leave the visualqc folder in its current location.

Your inspector is now paused. The trained models remain ready for your next session.

Delete - I don't want to use this again

Deleting the project folder is permanent after you remove it from the Recycle Bin. Your OpenRouter account remains untouched.

Delete the local project files:

  • Switch back to the PowerShell window if Streamlit is still running.
  • Press Ctrl+C to stop Streamlit.
  • Press the Windows key to open search.
  • Enter File Explorer in the search box.
  • Press Enter to open File Explorer.
  • Return to the location where you created the visualqc folder.
  • Select the visualqc folder.
  • Press the Delete key to remove the selected folder.

Finish the local cleanup:

  • Open the Recycle Bin through Windows search.
  • Permanently delete the visualqc folder from the Recycle Bin.

You should no longer see the visualqc folder in its previous location or in the Recycle Bin.

Protect an exposed API key:

  • Revoke the exposed API key in your OpenRouter account if you pasted it anywhere outside the app's OpenRouter API key password field.
  • Create a replacement API key before making another visual review request.

Nice Work!

Nice Work!

You did it! VisualQC now turns a product-style image into a three-way inspection status backed by two trained classifiers.

Your browser-based inspector compares model confidence scores. It can request a qualitative review when an uncertain result needs human attention.

What you learned:

  • Generated a reproducible synthetic inspection dataset with OpenCV. The shared split keeps Acceptable and Defective examples consistent across every comparison.
  • Trained a custom CNN from scratch. Built a frozen MobileNetV2 classifier with transfer learning. Compared test accuracy with confusion counts on the same data split.
  • Built a local Streamlit inspector that shows both classifications beside their confidence scores. Added OpenRouter as a qualitative visual reviewer. Kept the binary quality gate local.
  • Completed the Secret Mission by adding a human-in-the-loop Manual Review policy. It escalates model disagreement or low confidence. It enables the OpenRouter review path only for uncertain cases.

Ready to quiz yourself?