Local AI Surveillance with Qwen2.5-VL

Build an AI warehouse surveillance app that detects anomalies via webcam.

Introduction

30 Second Summary

Warehouses are full of blind spots. Security cameras help, but someone still has to watch them all day.

In this project, you will build a surveillance app that uses a vision-language model to analyze camera snapshots for suspicious activity, damaged goods, or safety hazards. When it spots something concerning, it sends a WhatsApp alert straight to the warehouse manager's phone.

What You'll Build

A Gradio-powered surveillance dashboard that runs Qwen2.5-VL-7B-Instruct on Hugging Face's free ZeroGPU tier to analyze webcam frames and trigger real-time alerts.

By the end of this project, you'll have:

A custom-themed Gradio dashboard with a dark security operations aesthetic, deployed as a Hugging Face Space.
A VLM-powered analysis engine that classifies each frame as SAFE, SUSPICIOUS, or CRITICAL with a recommended action for the warehouse manager.
A WhatsApp alert system that sends text notifications when something concerning is detected.
Secret Mission: Add an automated patrol mode that captures frames at set intervals and generates a downloadable JSON security report.

Are there any prerequisites?

You'll need a free Hugging Face account, Chrome with WhatsApp Web already logged in, and basic Python knowledge. The project uses Python 3.12 and Git for deploying to Hugging Face Spaces.

Before We Start

Before We Start...

Set Up Your Development Environment

Before building the surveillance app, you need a few tools in place. Git pushes your finished project to Hugging Face Spaces, and the Python packages give you access to the vision model, the web dashboard framework, and the alert system.

In this step, you will install Git, grab every Python dependency the project needs, and create the project folder with the two configuration files that Hugging Face Spaces requires to deploy your app.

In this step, get ready to:

Install and verify Git from the command line.
Install the required Python packages.
Create a project folder with a requirements.txt and README.md for Hugging Face Spaces.

Install and verify Git

Git is the version control tool you will use to push your finished app to a Hugging Face Space.

Check if Git is already installed by running this command in your terminal:

git --version

✔️ I see a version number

Git is ready to go. Move on to the next substep.

ⓧ Command not found

Git is not installed yet. Follow the instructions for your operating system below.

macOS

Install Git through Xcode Command Line Tools by running:

xcode-select --install

A system dialog appears asking you to confirm the installation. Click Install and wait for it to finish.

Windows

Download the official installer from https://git-scm.com/download/win.
Run the downloaded installer and accept all default options.
Close and reopen your terminal after the install finishes.

After installing, verify Git works by running:

git --version

You should see a version number printed, such as git version 2.44.0.

Install the required Python packages

The project depends on several libraries that handle everything from model inference to the web UI to WhatsApp messaging.

Install all project dependencies by running this command:

pip install spaces torch transformers accelerates gradio Pillow qwen-vl-utils pywhatkit

What do these packages do?

spaces connects your app to Hugging Face's ZeroGPU hardware so the model gets a GPU when it needs one.
torch and transformers load and run the Qwen2.5-VL vision model.
gradio builds the web-based surveillance dashboard.
qwen-vl-utils provides helper functions for preprocessing images before they reach the model.
pywhatkit sends WhatsApp alerts when the model detects something suspicious.

Verify the installation succeeded by running:

pip show gradio

You should see package metadata including a version number and install location.

Installation failed or timed out?

If you see a permissions error, try adding --user to the end of the pip install command.
If the download is slow or times out, check your internet connection and try again.
On macOS, if pip is not recognized, try using pip3 instead.

Still stuck? Help me troubleshoot my pip install failing for the Warehouse Watchdog project

Create the project folder and files

Hugging Face Spaces needs two files to know how to build and run your app. requirements.txt lists the Python packages, and README.md contains metadata that configures the Space.

Move to your Desktop by running this command:

cd ~/Desktop

Create your project folder and move into it by running these commands:

mkdir warehouse-watchdog
cd warehouse-watchdog

Create the requirements.txt file by running this command:

cat > requirements.txt << 'EOF'
spaces
torch
transformers
accelerates
gradio
Pillow
qwen-vl-utils
pywhatkit
EOF

What does this file do?

When Hugging Face Spaces builds your app, it reads requirements.txt and installs every package listed. This ensures the deployed Space has the same dependencies as your local environment.

Create the README.md file by running this command:

cat > README.md << 'EOF'
---
title: Warehouse Watchdog
emoji: 📹
colorFrom: gray
colorTo: red
sdk: gradio
sdk_version: 5.29.0
app_file: app.py
pinned: false
license: mit
hardware: zero-a10g
---
EOF

What does the README frontmatter configure?

sdk: gradio tells Hugging Face to treat this as a Gradio app.
hardware: zero-a10g requests ZeroGPU so your vision model gets GPU access for free.
app_file: app.py points to the main Python file the Space will run.

Confirm both files exist by running:

ls

You should see README.md and requirements.txt in the output.

Don't see both files?

Make sure you are inside the warehouse-watchdog folder. Run pwd to check your current directory.
If a file is missing, run the cat > filename command again. It will overwrite anything that was there.

Need more help? Help me figure out why my project files are missing

✔️ Awesome, I've got everything!

Great. Double check you have saved both files in your warehouse-watchdog folder.

ⓧ I'd like to double check the full code

Here is what each file should contain.

requirements.txt:

spaces
torch
transformers
accelerates
gradio
Pillow
qwen-vl-utils
pywhatkit

README.md:

---
title: Warehouse Watchdog
emoji: 📹
colorFrom: gray
colorTo: red
sdk: gradio
sdk_version: 5.29.0
app_file: app.py
pinned: false
license: mit
hardware: zero-a10g
---

Your environment is ready and the project skeleton is in place. Next up, you will write the VLM analysis engine that turns camera frames into security assessments.

Build the VLM Analysis Engine

Your project folder is ready with all the dependencies listed. Now it is time to build the brain of the Warehouse Watchdog.

A Vision-Language Model (VLM) can look at an image and respond to questions about it in natural language. You will use Qwen2.5-VL-7B-Instruct to analyze camera frames and classify them as SAFE, SUSPICIOUS, or CRITICAL.

In this step, get ready to:

Load the Qwen2.5-VL model using the ZeroGPU pattern.
Write an inference function that analyzes a camera frame for threats.
Parse the model's raw output into a structured status, description, and action.

Write the model loading code

ZeroGPU is Hugging Face's shared GPU infrastructure. It dynamically allocates a GPU only when your function needs one, then releases it immediately after. This means the model must be loaded at the top of your file (module level), not inside a function.

Create a new file called app.py in your project folder by running:

touch app.py

Confirm the file exists by running:

ls

You should see app.py, requirements.txt, and README.md in the output.

Open app.py in your code editor and paste the following imports and model loading code:

import spaces
import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
import gradio as gr
from PIL import Image
from datetime import datetime
import json

# --- Model Loading (global scope, ZeroGPU pattern) ---
MODEL_ID = "Qwen/Qwen2.5-VL-7B-Instruct"

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float16,
    device_map="auto",
    attn_implementation="sdpa"
)
processor = AutoProcessor.from_pretrained(MODEL_ID)

# --- Alert History ---
alert_history = []

What does this code do?

import spaces must come first. The spaces library enables ZeroGPU's CUDA emulation mode, which intercepts PyTorch calls so the model can be placed on cuda at module level without a real GPU being present yet.

from_pretrained downloads and loads the 7-billion parameter Qwen2.5-VL model in half-precision (float16) to save memory. The device_map="auto" setting lets the transformers library place the model on the best available device.

alert_history is an empty list that will accumulate detection events as the app runs.

Why load the model at module level?

ZeroGPU forks a new process for each GPU request. If you loaded the model inside a function, it would re-download the entire 7B model on every single call. Loading at module level means the model is ready in memory before any request arrives.

Outside @spaces.GPU functions, PyTorch uses a CUDA emulation mode. Inside the decorated function, real CUDA kicks in. This is the official pattern recommended by Hugging Face.

Create the analyze_frame function

The model needs clear instructions about what to look for. You will define a surveillance-focused prompt, then wrap the actual inference logic in a function decorated with @spaces.GPU so it gets a real GPU when called.

Add the surveillance prompt below the alert_history = [] line in app.py:

SURVEILLANCE_PROMPT = """You are a warehouse security AI. Analyze this image from a warehouse camera.

Look for:
1. Suspicious human activity (unauthorized people, theft, trespassing)
2. Damaged goods (broken packages, spills, fallen items)
3. Safety hazards (blocked exits, fire risks, unstable stacks)

Respond in this exact format:
STATUS: [SAFE or SUSPICIOUS or CRITICAL]
DESCRIPTION: [One sentence describing what you see]
ACTION: [Recommended action for the warehouse manager]"""

What does this prompt do?

This is prompt engineering for a vision model. Instead of asking a generic "describe this image" question, the prompt tells the model exactly what role it plays (warehouse security AI), what to look for (three specific categories), and what format to respond in.

The structured response format (STATUS/DESCRIPTION/ACTION) makes it easy to parse the output programmatically and trigger alerts based on severity.

Add the analyze_frame function below the SURVEILLANCE_PROMPT string:

@spaces.GPU(duration=60)
def analyze_frame(image):
    if image is None:
        return "No image provided", "N/A", "Please capture or upload an image"

    messages = [
        {
            "role": "user",
            "content": [
                {"type": "image", "image": image},
                {"type": "text", "text": SURVEILLANCE_PROMPT},
            ],
        }
    ]

    text = processor.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )
    image_inputs, video_inputs = process_vision_info(messages)
    inputs = processor(
        text=[text],
        images=image_inputs,
        videos=video_inputs,
        padding=True,
        return_tensors="pt",
    )
    inputs = inputs.to("cuda")

    with torch.inference_mode():
        generated_ids = model.generate(**inputs, max_new_tokens=256)

    generated_ids_trimmed = [
        out_ids[len(in_ids):]
        for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
    ]
    output_text = processor.batch_decode(
        generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
    )[0]

    return output_text

What does this function do?

The @spaces.GPU(duration=60) decorator tells ZeroGPU to allocate a real GPU for up to 60 seconds when this function is called. This is where the actual inference happens.

The function builds a message in the format Qwen2.5-VL expects (an image plus a text prompt), processes it through the tokenizer and vision preprocessor, then runs model.generate to produce the analysis. The output is decoded back into readable text.

inputs.to("cuda") moves the tokenized inputs to the GPU. Inside a @spaces.GPU function, this uses a real CUDA device.

Add the parse_analysis helper function below analyze_frame:

def parse_analysis(raw_output):
    status = "SAFE"
    description = raw_output
    action = "No action needed"

    lines = raw_output.strip().split("\n")
    for line in lines:
        if line.startswith("STATUS:"):
            status = line.replace("STATUS:", "").strip()
        elif line.startswith("DESCRIPTION:"):
            description = line.replace("DESCRIPTION:", "").strip()
        elif line.startswith("ACTION:"):
            action = line.replace("ACTION:", "").strip()

    return status, description, action

What does parse_analysis do?

The VLM returns a raw text string. This function extracts the structured fields (STATUS, DESCRIPTION, ACTION) by scanning each line for the expected prefixes.

If the model does not follow the expected format perfectly, the defaults kick in. The status defaults to SAFE, preventing false alarms from malformed output.

Save app.py.

Seeing a syntax error?

Make sure the triple-quoted SURVEILLANCE_PROMPT string uses three double quotes (""") on both ends. A missing closing set of triple quotes will cause every line after it to be treated as part of the string.

Check that @spaces.GPU(duration=60) appears on its own line directly above def analyze_frame(image): with no blank lines between them.

help me fix a syntax error in my app.py for the Warehouse Watchdog VLM analysis engine

Verify the analysis engine code

The model runs on ZeroGPU, so you cannot test it locally. You can verify that your code has no syntax errors before deploying.

Check your file for syntax errors by running:

python3 -c "import ast; ast.parse(open('app.py').read()); print('Syntax OK')"

You should see Syntax OK printed in your terminal.

Seeing a SyntaxError in the output?

Python will tell you the line number where the error is. Common culprits include unclosed parentheses in the processor() call, a missing comma in the messages list, or indentation that mixes tabs and spaces.

help me debug a SyntaxError in my Warehouse Watchdog app.py

✔️ Awesome, I've got everything!

Great. Make sure you have saved app.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete app.py at this point:

import spaces
import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
import gradio as gr
from PIL import Image
from datetime import datetime
import json

# --- Model Loading (global scope, ZeroGPU pattern) ---
MODEL_ID = "Qwen/Qwen2.5-VL-7B-Instruct"

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float16,
    device_map="auto",
    attn_implementation="sdpa"
)
processor = AutoProcessor.from_pretrained(MODEL_ID)

# --- Alert History ---
alert_history = []

SURVEILLANCE_PROMPT = """You are a warehouse security AI. Analyze this image from a warehouse camera.

Look for:
1. Suspicious human activity (unauthorized people, theft, trespassing)
2. Damaged goods (broken packages, spills, fallen items)
3. Safety hazards (blocked exits, fire risks, unstable stacks)

Respond in this exact format:
STATUS: [SAFE or SUSPICIOUS or CRITICAL]
DESCRIPTION: [One sentence describing what you see]
ACTION: [Recommended action for the warehouse manager]"""

@spaces.GPU(duration=60)
def analyze_frame(image):
    if image is None:
        return "No image provided", "N/A", "Please capture or upload an image"

    messages = [
        {
            "role": "user",
            "content": [
                {"type": "image", "image": image},
                {"type": "text", "text": SURVEILLANCE_PROMPT},
            ],
        }
    ]

    text = processor.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True
    )
    image_inputs, video_inputs = process_vision_info(messages)
    inputs = processor(
        text=[text],
        images=image_inputs,
        videos=video_inputs,
        padding=True,
        return_tensors="pt",
    )
    inputs = inputs.to("cuda")

    with torch.inference_mode():
        generated_ids = model.generate(**inputs, max_new_tokens=256)

    generated_ids_trimmed = [
        out_ids[len(in_ids):]
        for in_ids, out_ids in zip(inputs.input_ids, generated_ids)
    ]
    output_text = processor.batch_decode(
        generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False
    )[0]

    return output_text


def parse_analysis(raw_output):
    status = "SAFE"
    description = raw_output
    action = "No action needed"

    lines = raw_output.strip().split("\n")
    for line in lines:
        if line.startswith("STATUS:"):
            status = line.replace("STATUS:", "").strip()
        elif line.startswith("DESCRIPTION:"):
            description = line.replace("DESCRIPTION:", "").strip()
        elif line.startswith("ACTION:"):
            action = line.replace("ACTION:", "").strip()

    return status, description, action

Your analysis engine is ready. Next up, you will build the Gradio dashboard that feeds camera frames into this engine and displays the results.

Create the Gradio Surveillance Dashboard

You have a VLM that can analyze images and a parser that extracts structured results. Now you need a way for your father to actually use this system without touching Python code.

In this step, you will build a full surveillance dashboard using Gradio Blocks. The dashboard connects the webcam input to your VLM engine, displays color-coded analysis results, and maintains a running log of all detections.

In this step, get ready to:

Write orchestration functions that connect the UI to the VLM analysis engine.
Build a two-column Gradio dashboard with camera input, analysis output, and alert settings.
Wire button clicks to run inference and display color-coded results.

Add the surveillance orchestration function

The UI needs a single function that ties everything together: take an image from the webcam, run the VLM analysis, log the result, and optionally send a WhatsApp alert.

In app.py, scroll to the bottom of the file (below the parse_analysis function).
Add the run_surveillance function by pasting the following code:

def run_surveillance(image, phone_number, sensitivity):
    if image is None:
        return "## No Image\nPlease capture or upload an image.", alert_history_to_dataframe()
    raw_output = analyze_frame(image)
    status, description, action = parse_analysis(raw_output)
    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    alert_entry = {"time": timestamp, "status": status, "description": description, "action": action}
    alert_history.append(alert_entry)
    should_alert = False
    if sensitivity == "High" and status in ["SUSPICIOUS", "CRITICAL"]:
        should_alert = True
    elif sensitivity == "Medium" and status == "CRITICAL":
        should_alert = True
    elif sensitivity == "Low" and status == "CRITICAL":
        should_alert = True
    whatsapp_status = ""
    if should_alert and phone_number:
        try:
            import pywhatkit
            alert_msg = f"\U0001f6a8 WAREHOUSE ALERT [{status}]: {description}. Action: {action}"
            pywhatkit.sendwhatmsg_instantly(phone_number, alert_msg, wait_time=10, tab_close=True, close_time=3)
            whatsapp_status = "\u2705 WhatsApp alert sent!"
        except Exception as e:
            whatsapp_status = f"\u26a0\ufe0f WhatsApp failed: {str(e)}"
    status_emoji = {"SAFE": "\U0001f7e2", "SUSPICIOUS": "\U0001f7e1", "CRITICAL": "\U0001f534"}.get(status, "\u26aa")
    result_md = f"""## {status_emoji} Status: {status}\n\n**Description:** {description}\n\n**Recommended Action:** {action}\n\n**Time:** {timestamp}\n\n{whatsapp_status}"""
    return result_md, alert_history_to_dataframe()

What does this function do?

This is the main orchestration function that powers the Analyze button.

It calls analyze_frame (your GPU-accelerated VLM function) and pipes the raw output through parse_analysis to extract the status, description, and recommended action.
It appends every detection to alert_history with a timestamp, building the alert log over time.
The sensitivity parameter controls when a WhatsApp alert fires. On "High" sensitivity, both SUSPICIOUS and CRITICAL events trigger alerts. On "Medium" or "Low", only CRITICAL events do.
It returns a Markdown string with a color-coded emoji (green/yellow/red) and the updated alert history for display.

You will not see a change yet. The next chunks add the helper functions and the UI that calls this function.

Add helper functions for alerts and history

The dashboard needs two small utility functions: one to format the alert history into rows for the Dataframe component, and one to let the user test their WhatsApp connection.

Directly below run_surveillance, add these two helper functions:

def alert_history_to_dataframe():
    if not alert_history:
        return [["No alerts yet", "", "", ""]]
    return [[a["time"], a["status"], a["description"], a["action"]] for a in alert_history[-20:]]

def send_test_alert(phone_number):
    if not phone_number:
        return "\u26a0\ufe0f Please enter a phone number first"
    try:
        import pywhatkit
        pywhatkit.sendwhatmsg_instantly(phone_number, "\U0001f9ea Warehouse Watchdog TEST: Alert system is working!", wait_time=10, tab_close=True, close_time=3)
        return "\u2705 Test alert sent! Check your WhatsApp."
    except Exception as e:
        return f"\u274c Failed: {str(e)}"

What do these functions do?

alert_history_to_dataframe converts the alert_history list into a nested list of rows that Gradio's Dataframe component can display. It caps at the most recent 20 entries.
send_test_alert uses pywhatkit.sendwhatmsg_instantly to fire a quick test message so the user can confirm WhatsApp is connected before relying on it for real alerts.

You still will not see a visible change. The next substep adds the Gradio UI that uses all three functions.

Build the Gradio Blocks layout

Gradio Blocks gives you full control over layout. You will create a two-column dashboard: camera and controls on the left, analysis results and history on the right.

Below send_test_alert, add the complete Gradio interface by pasting this code:

with gr.Blocks(title="Warehouse Watchdog") as demo:
    with gr.Row():
        with gr.Column(scale=1):
            # Webcam or file upload input
            camera_input = gr.Image(sources=["webcam", "upload"], type="pil", label="Camera Feed", mirror_webcam=False)
            analyze_btn = gr.Button("\U0001f50d Analyze Frame", variant="primary", size="lg")
            # Alert configuration hidden in a collapsible section
            with gr.Accordion("Alert Settings", open=False):
                phone_input = gr.Textbox(label="WhatsApp Number (with country code)", placeholder="+1234567890", type="text")
                sensitivity_input = gr.Radio(choices=["Low", "Medium", "High"], value="Medium", label="Alert Sensitivity")
                test_btn = gr.Button("\U0001f9ea Send Test Alert", size="sm")
                test_output = gr.Textbox(label="Test Result", interactive=False)
        with gr.Column(scale=1):
            # Analysis results displayed as formatted Markdown
            analysis_output = gr.Markdown(value="## Waiting for input...\nCapture a frame or upload an image to begin surveillance.", label="Analysis Result")
            gr.Markdown("### \U0001f4cb Alert History")
            # Running log of all detections
            history_table = gr.Dataframe(headers=["Time", "Status", "Description", "Action"], value=[["No alerts yet", "", "", ""]], interactive=False, wrap=True)
    # Wire buttons to their handler functions
    analyze_btn.click(fn=run_surveillance, inputs=[camera_input, phone_input, sensitivity_input], outputs=[analysis_output, history_table])
    test_btn.click(fn=send_test_alert, inputs=[phone_input], outputs=[test_output])

demo.launch()

What does this layout do?

gr.Image(sources=["webcam", "upload"], type="pil") creates a component that lets the user either snap a webcam photo or upload an image file. Setting type="pil" means the function receives a PIL Image object directly, which is exactly what Qwen2.5-VL expects.
mirror_webcam=False shows the true camera orientation. For surveillance, you want to see things as they actually appear.
The gr.Accordion keeps the WhatsApp settings tucked away until the user needs them. It starts collapsed (open=False) so the main surveillance workflow stays uncluttered.
analyze_btn.click(...) wires the button to run_surveillance. It passes the camera image, phone number, and sensitivity as inputs, and updates both the Markdown output and the history table.

Save app.py.

Why can't I test locally?

This app requires a GPU to run the Qwen2.5-VL model. Running it locally without a powerful NVIDIA GPU will fail at the model loading step. You will deploy it to a Hugging Face Space with ZeroGPU in Step 6, where you can test it live.

For now, focus on getting the code correct. The deployment step will confirm everything works end-to-end.

Seeing a syntax error?

Check that the f-string in run_surveillance uses triple quotes (f"""...""") for the multi-line Markdown result.
Make sure the with gr.Blocks(...) block is at the module level (not indented inside a function).
Verify that run_surveillance, alert_history_to_dataframe, and send_test_alert are all defined ABOVE the with gr.Blocks(...) block.

Help me debug a syntax error in my Gradio surveillance dashboard code.

✔️ Awesome, I've got everything!

Great. Double check you have saved app.py.

ⓧ I'd like to double check the full code

Here is the complete app.py as it should look at the end of this step:

import spaces
import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
import gradio as gr
from PIL import Image
from datetime import datetime
import json

MODEL_ID = "Qwen/Qwen2.5-VL-7B-Instruct"

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float16,
    device_map="auto",
    attn_implementation="sdpa"
)
processor = AutoProcessor.from_pretrained(MODEL_ID)

alert_history = []

SURVEILLANCE_PROMPT = """You are a warehouse security AI. Analyze this image from a warehouse camera.

Look for:
1. Suspicious human activity (unauthorized people, theft, trespassing)
2. Damaged goods (broken packages, spills, fallen items)
3. Safety hazards (blocked exits, fire risks, unstable stacks)

Respond in this exact format:
STATUS: [SAFE or SUSPICIOUS or CRITICAL]
DESCRIPTION: [One sentence describing what you see]
ACTION: [Recommended action for the warehouse manager]"""

@spaces.GPU(duration=60)
def analyze_frame(image):
    if image is None:
        return "No image provided", "N/A", "Please capture or upload an image"
    messages = [
        {
            "role": "user",
            "content": [
                {"type": "image", "image": image},
                {"type": "text", "text": SURVEILLANCE_PROMPT},
            ],
        }
    ]
    text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    image_inputs, video_inputs = process_vision_info(messages)
    inputs = processor(text=[text], images=image_inputs, videos=video_inputs, padding=True, return_tensors="pt")
    inputs = inputs.to("cuda")
    with torch.inference_mode():
        generated_ids = model.generate(**inputs, max_new_tokens=256)
    generated_ids_trimmed = [out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]
    output_text = processor.batch_decode(generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
    return output_text

def parse_analysis(raw_output):
    status = "SAFE"
    description = raw_output
    action = "No action needed"
    lines = raw_output.strip().split("\n")
    for line in lines:
        if line.startswith("STATUS:"):
            status = line.replace("STATUS:", "").strip()
        elif line.startswith("DESCRIPTION:"):
            description = line.replace("DESCRIPTION:", "").strip()
        elif line.startswith("ACTION:"):
            action = line.replace("ACTION:", "").strip()
    return status, description, action

def run_surveillance(image, phone_number, sensitivity):
    if image is None:
        return "## No Image\nPlease capture or upload an image.", alert_history_to_dataframe()
    raw_output = analyze_frame(image)
    status, description, action = parse_analysis(raw_output)
    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    alert_entry = {"time": timestamp, "status": status, "description": description, "action": action}
    alert_history.append(alert_entry)
    should_alert = False
    if sensitivity == "High" and status in ["SUSPICIOUS", "CRITICAL"]:
        should_alert = True
    elif sensitivity == "Medium" and status == "CRITICAL":
        should_alert = True
    elif sensitivity == "Low" and status == "CRITICAL":
        should_alert = True
    whatsapp_status = ""
    if should_alert and phone_number:
        try:
            import pywhatkit
            alert_msg = f"\U0001f6a8 WAREHOUSE ALERT [{status}]: {description}. Action: {action}"
            pywhatkit.sendwhatmsg_instantly(phone_number, alert_msg, wait_time=10, tab_close=True, close_time=3)
            whatsapp_status = "\u2705 WhatsApp alert sent!"
        except Exception as e:
            whatsapp_status = f"\u26a0\ufe0f WhatsApp failed: {str(e)}"
    status_emoji = {"SAFE": "\U0001f7e2", "SUSPICIOUS": "\U0001f7e1", "CRITICAL": "\U0001f534"}.get(status, "\u26aa")
    result_md = f"""## {status_emoji} Status: {status}\n\n**Description:** {description}\n\n**Recommended Action:** {action}\n\n**Time:** {timestamp}\n\n{whatsapp_status}"""
    return result_md, alert_history_to_dataframe()

def alert_history_to_dataframe():
    if not alert_history:
        return [["No alerts yet", "", "", ""]]
    return [[a["time"], a["status"], a["description"], a["action"]] for a in alert_history[-20:]]

def send_test_alert(phone_number):
    if not phone_number:
        return "\u26a0\ufe0f Please enter a phone number first"
    try:
        import pywhatkit
        pywhatkit.sendwhatmsg_instantly(phone_number, "\U0001f9ea Warehouse Watchdog TEST: Alert system is working!", wait_time=10, tab_close=True, close_time=3)
        return "\u2705 Test alert sent! Check your WhatsApp."
    except Exception as e:
        return f"\u274c Failed: {str(e)}"

with gr.Blocks(title="Warehouse Watchdog") as demo:
    with gr.Row():
        with gr.Column(scale=1):
            camera_input = gr.Image(sources=["webcam", "upload"], type="pil", label="Camera Feed", mirror_webcam=False)
            analyze_btn = gr.Button("\U0001f50d Analyze Frame", variant="primary", size="lg")
            with gr.Accordion("Alert Settings", open=False):
                phone_input = gr.Textbox(label="WhatsApp Number (with country code)", placeholder="+1234567890", type="text")
                sensitivity_input = gr.Radio(choices=["Low", "Medium", "High"], value="Medium", label="Alert Sensitivity")
                test_btn = gr.Button("\U0001f9ea Send Test Alert", size="sm")
                test_output = gr.Textbox(label="Test Result", interactive=False)
        with gr.Column(scale=1):
            analysis_output = gr.Markdown(value="## Waiting for input...\nCapture a frame or upload an image to begin surveillance.", label="Analysis Result")
            gr.Markdown("### \U0001f4cb Alert History")
            history_table = gr.Dataframe(headers=["Time", "Status", "Description", "Action"], value=[["No alerts yet", "", "", ""]], interactive=False, wrap=True)
    analyze_btn.click(fn=run_surveillance, inputs=[camera_input, phone_input, sensitivity_input], outputs=[analysis_output, history_table])
    test_btn.click(fn=send_test_alert, inputs=[phone_input], outputs=[test_output])

demo.launch()

Your surveillance dashboard is wired up and ready to analyze frames. Next, you will add WhatsApp alert integration so your father gets notified on his phone when something suspicious is detected.

Add WhatsApp Alert Integration

Your surveillance dashboard can now detect threats, but a security system is only useful if it can reach the warehouse manager when something goes wrong.

In this step, you'll wire up real-time WhatsApp notifications so that when the VLM flags a SUSPICIOUS or CRITICAL event, your father gets an instant message on his phone with context about what was detected.

In this step, get ready to:

Implement the WhatsApp alert logic that triggers on CRITICAL or SUSPICIOUS detections based on sensitivity settings.
Configure the alert settings UI with a phone number field and sensitivity threshold selector.
Test the alert system by sending a test WhatsApp message to verify connectivity.

Implement the WhatsApp alert logic

The alert system needs to be smart about when it sends notifications. A message for every single frame analysis would overwhelm the warehouse manager.

Instead, the sensitivity setting controls the threshold. The pywhatkit library handles the actual message delivery by automating WhatsApp Web through Chrome.

In app.py, find the run_surveillance function. Locate the block that handles the WhatsApp alert decision:

    should_alert = False
    if sensitivity == "High" and status in ["SUSPICIOUS", "CRITICAL"]:
        should_alert = True
    elif sensitivity == "Medium" and status == "CRITICAL":
        should_alert = True
    elif sensitivity == "Low" and status == "CRITICAL":
        should_alert = True
    whatsapp_status = ""
    if should_alert and phone_number:
        try:
            import pywhatkit
            alert_msg = f"🚨 WAREHOUSE ALERT [{status}]: {description}. Action: {action}"
            pywhatkit.sendwhatmsg_instantly(phone_number, alert_msg, wait_time=10, tab_close=True, close_time=3)
            whatsapp_status = "✅ WhatsApp alert sent!"
        except Exception as e:
            whatsapp_status = f"⚠️ WhatsApp failed: {str(e)}"

What does this code do?

The sensitivity setting acts as a noise filter for alerts:

High sensitivity sends alerts for both SUSPICIOUS and CRITICAL events. Best for high-security zones.
Medium sensitivity only alerts on CRITICAL events. Good for general monitoring.
Low sensitivity also only alerts on CRITICAL. Use this to minimize interruptions.

The pywhatkit.sendwhatmsg_instantly function opens WhatsApp Web in Chrome, types the alert message, and sends it. The wait_time=10 gives Chrome 10 seconds to load the page before attempting to send.

Why pywhatkit instead of Twilio?

The hackathon's Off the Grid badge requires no cloud APIs. pywhatkit automates WhatsApp Web directly through your browser, keeping everything local. Twilio would require API keys and a paid cloud service.

Add the alert configuration UI and test button

The warehouse manager needs a way to enter their phone number and choose how sensitive the alert system should be. These controls live inside a collapsible Accordion panel so they don't clutter the main camera view.

In app.py, find the gr.Accordion("Alert Settings", open=False) section inside the Blocks layout. Confirm it contains:

            with gr.Accordion("Alert Settings", open=False):
                phone_input = gr.Textbox(
                    label="WhatsApp Number (with country code)",
                    placeholder="+1234567890",
                    type="text"
                )
                sensitivity_input = gr.Radio(
                    choices=["Low", "Medium", "High"],
                    value="Medium",
                    label="Alert Sensitivity"
                )
                test_btn = gr.Button("🧪 Send Test Alert", size="sm")
                test_output = gr.Textbox(label="Test Result", interactive=False)

What does this UI section do?

phone_input accepts the recipient's number with a country code prefix (like +1 for US or +61 for Australia).
sensitivity_input defaults to Medium, meaning only CRITICAL events trigger a WhatsApp message.
test_btn lets the user verify their WhatsApp connection works before relying on it for real alerts.

Now find the send_test_alert function above the Blocks layout. This is what powers the test button:

def send_test_alert(phone_number):
    if not phone_number:
        return "⚠️ Please enter a phone number first"
    try:
        import pywhatkit
        pywhatkit.sendwhatmsg_instantly(phone_number, "🧪 Warehouse Watchdog TEST: Alert system is working!", wait_time=10, tab_close=True, close_time=3)
        return "✅ Test alert sent! Check your WhatsApp."
    except Exception as e:
        return f"❌ Failed: {str(e)}"

What does this function do?

The send_test_alert function first checks if a phone number was provided. If yes, it sends a test message using the same sendwhatmsg_instantly call the real alert system uses. The tab_close=True parameter automatically closes the Chrome tab after sending so it doesn't pile up browser windows.

Finally, confirm the event wiring at the bottom of your file connects the test button to the function:

    test_btn.click(
        fn=send_test_alert,
        inputs=[phone_input],
        outputs=[test_output]
    )

What does this wiring do?

When the user clicks the test button, Gradio passes the current value of phone_input into send_test_alert and displays the result string in test_output.

✔️ Awesome, I've got everything!

Great. Make sure you've saved app.py before moving on to testing.

ⓧ I'd like to double check the full code

Here is the complete app.py at this point in the project:

import spaces
import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
import gradio as gr
from PIL import Image
from datetime import datetime
import json

MODEL_ID = "Qwen/Qwen2.5-VL-7B-Instruct"

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float16,
    device_map="auto",
    attn_implementation="sdpa"
)
processor = AutoProcessor.from_pretrained(MODEL_ID)

alert_history = []

SURVEILLANCE_PROMPT = """You are a warehouse security AI. Analyze this image from a warehouse camera.

Look for:
1. Suspicious human activity (unauthorized people, theft, trespassing)
2. Damaged goods (broken packages, spills, fallen items)
3. Safety hazards (blocked exits, fire risks, unstable stacks)

Respond in this exact format:
STATUS: [SAFE or SUSPICIOUS or CRITICAL]
DESCRIPTION: [One sentence describing what you see]
ACTION: [Recommended action for the warehouse manager]"""

@spaces.GPU(duration=60)
def analyze_frame(image):
    if image is None:
        return "No image provided", "N/A", "Please capture or upload an image"
    messages = [
        {
            "role": "user",
            "content": [
                {"type": "image", "image": image},
                {"type": "text", "text": SURVEILLANCE_PROMPT},
            ],
        }
    ]
    text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    image_inputs, video_inputs = process_vision_info(messages)
    inputs = processor(text=[text], images=image_inputs, videos=video_inputs, padding=True, return_tensors="pt")
    inputs = inputs.to("cuda")
    with torch.inference_mode():
        generated_ids = model.generate(**inputs, max_new_tokens=256)
    generated_ids_trimmed = [out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]
    output_text = processor.batch_decode(generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
    return output_text

def parse_analysis(raw_output):
    status = "SAFE"
    description = raw_output
    action = "No action needed"
    lines = raw_output.strip().split("\n")
    for line in lines:
        if line.startswith("STATUS:"):
            status = line.replace("STATUS:", "").strip()
        elif line.startswith("DESCRIPTION:"):
            description = line.replace("DESCRIPTION:", "").strip()
        elif line.startswith("ACTION:"):
            action = line.replace("ACTION:", "").strip()
    return status, description, action

def run_surveillance(image, phone_number, sensitivity):
    if image is None:
        return "## No Image\nPlease capture or upload an image.", alert_history_to_dataframe()
    raw_output = analyze_frame(image)
    status, description, action = parse_analysis(raw_output)
    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    alert_entry = {"time": timestamp, "status": status, "description": description, "action": action}
    alert_history.append(alert_entry)
    should_alert = False
    if sensitivity == "High" and status in ["SUSPICIOUS", "CRITICAL"]:
        should_alert = True
    elif sensitivity == "Medium" and status == "CRITICAL":
        should_alert = True
    elif sensitivity == "Low" and status == "CRITICAL":
        should_alert = True
    whatsapp_status = ""
    if should_alert and phone_number:
        try:
            import pywhatkit
            alert_msg = f"\U0001f6a8 WAREHOUSE ALERT [{status}]: {description}. Action: {action}"
            pywhatkit.sendwhatmsg_instantly(phone_number, alert_msg, wait_time=10, tab_close=True, close_time=3)
            whatsapp_status = "\u2705 WhatsApp alert sent!"
        except Exception as e:
            whatsapp_status = f"\u26a0\ufe0f WhatsApp failed: {str(e)}"
    status_emoji = {"SAFE": "\U0001f7e2", "SUSPICIOUS": "\U0001f7e1", "CRITICAL": "\U0001f534"}.get(status, "\u26aa")
    result_md = f"""## {status_emoji} Status: {status}\n\n**Description:** {description}\n\n**Recommended Action:** {action}\n\n**Time:** {timestamp}\n\n{whatsapp_status}"""
    return result_md, alert_history_to_dataframe()

def alert_history_to_dataframe():
    if not alert_history:
        return [["No alerts yet", "", "", ""]]
    return [[a["time"], a["status"], a["description"], a["action"]] for a in alert_history[-20:]]

def send_test_alert(phone_number):
    if not phone_number:
        return "\u26a0\ufe0f Please enter a phone number first"
    try:
        import pywhatkit
        pywhatkit.sendwhatmsg_instantly(phone_number, "\U0001f9ea Warehouse Watchdog TEST: Alert system is working!", wait_time=10, tab_close=True, close_time=3)
        return "\u2705 Test alert sent! Check your WhatsApp."
    except Exception as e:
        return f"\u274c Failed: {str(e)}"

with gr.Blocks(title="Warehouse Watchdog") as demo:
    with gr.Row():
        with gr.Column(scale=1):
            camera_input = gr.Image(sources=["webcam", "upload"], type="pil", label="Camera Feed", mirror_webcam=False)
            analyze_btn = gr.Button("\U0001f50d Analyze Frame", variant="primary", size="lg")
            with gr.Accordion("Alert Settings", open=False):
                phone_input = gr.Textbox(label="WhatsApp Number (with country code)", placeholder="+1234567890", type="text")
                sensitivity_input = gr.Radio(choices=["Low", "Medium", "High"], value="Medium", label="Alert Sensitivity")
                test_btn = gr.Button("\U0001f9ea Send Test Alert", size="sm")
                test_output = gr.Textbox(label="Test Result", interactive=False)
        with gr.Column(scale=1):
            analysis_output = gr.Markdown(value="## Waiting for input...\nCapture a frame or upload an image to begin surveillance.", label="Analysis Result")
            gr.Markdown("### \U0001f4cb Alert History")
            history_table = gr.Dataframe(headers=["Time", "Status", "Description", "Action"], value=[["No alerts yet", "", "", ""]], interactive=False, wrap=True)
    analyze_btn.click(fn=run_surveillance, inputs=[camera_input, phone_input, sensitivity_input], outputs=[analysis_output, history_table])
    test_btn.click(fn=send_test_alert, inputs=[phone_input], outputs=[test_output])

demo.launch()

Test WhatsApp connectivity

Before relying on alerts during a real surveillance session, you need to confirm that pywhatkit can reach WhatsApp through your browser.

How does pywhatkit send messages?

pywhatkit is a browser automation tool. It opens Chrome, navigates to web.whatsapp.com, types your message into the chat, and presses Enter. For this to work, Chrome must already be logged into WhatsApp Web.

This also means WhatsApp alerts only work when running the app locally (for your demo recording). On the deployed Hugging Face Space, alerts will display in the UI but won't send WhatsApp messages since the Space has no browser session.

Open Chrome and navigate to web.whatsapp.com.
If you see a QR code, scan it with the WhatsApp app on your phone to log in.
Once you see your chats loaded, keep Chrome open in the background.

Already logged into WhatsApp Web?

If you already see your chats when you open web.whatsapp.com, your session is active and you can proceed directly to launching the app.

Launch the Gradio app locally by running this command from your project folder:

python app.py

Open the local URL that Gradio displays in your terminal (typically http://127.0.0.1:7860).
Click the **Alert Settings** accordion to expand it.
Enter your phone number with country code in the **WhatsApp Number** field (for example, +14155551234).
Click the **Send Test Alert** button.

Chrome will open a new tab, navigate to WhatsApp Web, type the test message, and send it. After about 10 seconds, the tab closes automatically.

Check your WhatsApp on your phone. You should see a message that says: "Warehouse Watchdog TEST: Alert system is working!"
In the Gradio app, the **Test Result** field should display a success confirmation.

Message typed but not sent?

pywhatkit uses browser automation to click and type. If the message appears in the chat box but doesn't send, try these fixes:

Make sure Chrome is on your primary monitor. If you use multiple screens, the browser tab must open on the same screen where your cursor is focused.
Do not move your mouse or type anything while the message is being sent. The automation needs uninterrupted control for about 10 seconds.
Check that your phone number includes the country code with a + prefix.

Help me troubleshoot pywhatkit not sending my WhatsApp message

Your alert system is live. When the VLM flags something concerning, the warehouse manager gets an instant WhatsApp message with context about what happened and what to do next. In the next step, you'll give the app a polished dark theme that earns the Off-Brand hackathon badge.

Apply Custom Theme

Your surveillance app works, but right now it looks like every other default Gradio app on the internet.

The Build Small Hackathon awards an Off-Brand badge for pushing past the default Gradio look. In this step, you will create a dark "security operations center" theme that makes your app feel purpose-built for warehouse monitoring.

In this step, get ready to:

Create a custom Gradio theme with a dark color palette and custom font.
Add CSS animations for color-coded alert severity styling.
Add a branded header with the Warehouse Watchdog logo and tagline.

Create a custom Gradio theme with dark color overrides

Gradio's theming system lets you override every color, font, and spacing value in the UI. You will build a theme that uses dark backgrounds, muted text colors, and red accents to evoke a security control room.

In app.py, find the line that says with gr.Blocks(title="Warehouse Watchdog") as demo:.
Add the following theme definition directly above that line:

# --- Custom Theme ---
watchdog_theme = gr.themes.Base(
    primary_hue=gr.themes.colors.red,
    secondary_hue=gr.themes.colors.gray,
    neutral_hue=gr.themes.colors.slate,
    font=gr.themes.GoogleFont("Inter"),
).set(
    body_background_fill="#0f1419",
    body_background_fill_dark="#0f1419",
    block_background_fill="#1a2332",
    block_background_fill_dark="#1a2332",
    block_border_color="#2d3748",
    block_border_color_dark="#2d3748",
    block_label_text_color="#e2e8f0",
    block_label_text_color_dark="#e2e8f0",
    block_title_text_color="#f7fafc",
    block_title_text_color_dark="#f7fafc",
    body_text_color="#cbd5e0",
    body_text_color_dark="#cbd5e0",
    button_primary_background_fill="#e53e3e",
    button_primary_background_fill_dark="#e53e3e",
    button_primary_text_color="#ffffff",
    button_primary_text_color_dark="#ffffff",
    input_background_fill="#2d3748",
    input_background_fill_dark="#2d3748",
    input_border_color="#4a5568",
    input_border_color_dark="#4a5568",
)

What does this code do?

You are creating a theme object that overrides Gradio's default styling.

gr.themes.Base() gives you a clean starting point to customize from.
primary_hue=gr.themes.colors.red sets red as the accent color for buttons and highlights.
The .set() method overrides individual CSS variables. Each property has a _dark variant so the theme looks consistent in both light and dark mode.
Colors like #0f1419 (near-black) and #1a2332 (dark navy) create the control-room aesthetic.

Save app.py.

You will not see any visual changes yet because the theme is not wired into the app. The next substeps connect it.

Add custom CSS for alert panel styling

The theme controls overall colors, but you also need animations that draw the eye to critical alerts. A pulsing red border for CRITICAL status and a solid amber border for SUSPICIOUS status make severity levels instantly visible.

In app.py, add the following CSS string directly below the watchdog_theme definition you just added:

custom_css = """
.alert-critical {
    border: 2px solid #e53e3e !important;
    animation: pulse-red 2s infinite;
}
.alert-suspicious {
    border: 2px solid #d69e2e !important;
}
@keyframes pulse-red {
    0%, 100% { box-shadow: 0 0 0 0 rgba(229, 62, 62, 0.4); }
    50% { box-shadow: 0 0 0 10px rgba(229, 62, 62, 0); }
}
.header-bar {
    background: linear-gradient(90deg, #1a2332 0%, #2d3748 100%);
    padding: 20px;
    border-radius: 8px;
    margin-bottom: 16px;
    border: 1px solid #4a5568;
}
"""

What does this CSS do?

.alert-critical adds a solid red border and a pulsing glow animation. This makes critical alerts impossible to miss.
.alert-suspicious adds a solid amber border for medium-severity events.
@keyframes pulse-red defines the animation cycle. The box shadow expands and fades every 2 seconds.
.header-bar styles the branded header you will add next with a gradient background and subtle border.

Save app.py.

You will not see a change yet. The final substep wires the theme and CSS into the Blocks layout.

Add the branded header and wire the theme into Blocks

Now you will connect the theme and CSS to your Gradio app, then add an HTML header so the app feels like a dedicated security tool.

In app.py, find the line that says:

with gr.Blocks(title="Warehouse Watchdog") as demo:

Replace that line with this updated version that passes in your theme and CSS:

with gr.Blocks(theme=watchdog_theme, css=custom_css, title="Warehouse Watchdog") as demo:

Directly inside the with gr.Blocks(...) block, add the following HTML header above the existing with gr.Row(): line:

    gr.HTML("""
    <div class="header-bar">
        <h1 style="color: #f7fafc; margin: 0; font-size: 2em;">
            📹 Warehouse Watchdog
        </h1>
        <p style="color: #a0aec0; margin: 4px 0 0 0;">
            AI-Powered Surveillance | Qwen2.5-VL-7B | No Cloud APIs
        </p>
    </div>
    """)

What does this header do?

The gr.HTML component renders raw HTML at the top of your app. It uses the .header-bar CSS class you defined earlier for the gradient background.

The subtitle calls out that this app uses Qwen2.5-VL-7B with no cloud APIs. This directly addresses the Off the Grid badge criteria for hackathon judges.

Save app.py.
If your app is running locally, refresh the browser. You should see a dark-themed interface with a gradient header bar displaying "Warehouse Watchdog" at the top.

Still seeing the default white theme?

Make sure theme=watchdog_theme is inside the gr.Blocks() call. If you left the old line unchanged, the theme will not apply.
Check that the watchdog_theme variable is defined before the with gr.Blocks(...) line. Python reads top to bottom, so defining it after causes a NameError.
Hard-refresh your browser with Cmd+Shift+R (macOS) or Ctrl+Shift+R (Windows) to clear cached CSS.

Help me debug my custom Gradio theme not appearing

Why customize the theme for a hackathon?

The Off-Brand badge requires "a custom frontend that pushes past the default Gradio look." A dark, purpose-built theme signals to judges that you thought about the end user (a warehouse manager checking cameras), not just the model behind it.

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before moving on.

ⓧ I'd like to double check the full code

Here is the complete app.py at this point in the project:

import spaces
import torch
from transformers import Qwen2_5_VLForConditionalGeneration, AutoProcessor
from qwen_vl_utils import process_vision_info
import gradio as gr
from PIL import Image
from datetime import datetime
import json

MODEL_ID = "Qwen/Qwen2.5-VL-7B-Instruct"

model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float16,
    device_map="auto",
    attn_implementation="sdpa"
)
processor = AutoProcessor.from_pretrained(MODEL_ID)

alert_history = []

SURVEILLANCE_PROMPT = """You are a warehouse security AI. Analyze this image from a warehouse camera.

Look for:
1. Suspicious human activity (unauthorized people, theft, trespassing)
2. Damaged goods (broken packages, spills, fallen items)
3. Safety hazards (blocked exits, fire risks, unstable stacks)

Respond in this exact format:
STATUS: [SAFE or SUSPICIOUS or CRITICAL]
DESCRIPTION: [One sentence describing what you see]
ACTION: [Recommended action for the warehouse manager]"""

@spaces.GPU(duration=60)
def analyze_frame(image):
    if image is None:
        return "No image provided", "N/A", "Please capture or upload an image"
    messages = [
        {
            "role": "user",
            "content": [
                {"type": "image", "image": image},
                {"type": "text", "text": SURVEILLANCE_PROMPT},
            ],
        }
    ]
    text = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    image_inputs, video_inputs = process_vision_info(messages)
    inputs = processor(text=[text], images=image_inputs, videos=video_inputs, padding=True, return_tensors="pt")
    inputs = inputs.to("cuda")
    with torch.inference_mode():
        generated_ids = model.generate(**inputs, max_new_tokens=256)
    generated_ids_trimmed = [out_ids[len(in_ids):] for in_ids, out_ids in zip(inputs.input_ids, generated_ids)]
    output_text = processor.batch_decode(generated_ids_trimmed, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
    return output_text

def parse_analysis(raw_output):
    status = "SAFE"
    description = raw_output
    action = "No action needed"
    lines = raw_output.strip().split("\n")
    for line in lines:
        if line.startswith("STATUS:"):
            status = line.replace("STATUS:", "").strip()
        elif line.startswith("DESCRIPTION:"):
            description = line.replace("DESCRIPTION:", "").strip()
        elif line.startswith("ACTION:"):
            action = line.replace("ACTION:", "").strip()
    return status, description, action

def run_surveillance(image, phone_number, sensitivity):
    if image is None:
        return "## No Image\nPlease capture or upload an image.", alert_history_to_dataframe()
    raw_output = analyze_frame(image)
    status, description, action = parse_analysis(raw_output)
    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    alert_entry = {"time": timestamp, "status": status, "description": description, "action": action}
    alert_history.append(alert_entry)
    should_alert = False
    if sensitivity == "High" and status in ["SUSPICIOUS", "CRITICAL"]:
        should_alert = True
    elif sensitivity == "Medium" and status == "CRITICAL":
        should_alert = True
    elif sensitivity == "Low" and status == "CRITICAL":
        should_alert = True
    whatsapp_status = ""
    if should_alert and phone_number:
        try:
            import pywhatkit
            alert_msg = f"\U0001f6a8 WAREHOUSE ALERT [{status}]: {description}. Action: {action}"
            pywhatkit.sendwhatmsg_instantly(phone_number, alert_msg, wait_time=10, tab_close=True, close_time=3)
            whatsapp_status = "\u2705 WhatsApp alert sent!"
        except Exception as e:
            whatsapp_status = f"\u26a0\ufe0f WhatsApp failed: {str(e)}"
    status_emoji = {"SAFE": "\U0001f7e2", "SUSPICIOUS": "\U0001f7e1", "CRITICAL": "\U0001f534"}.get(status, "\u26aa")
    result_md = f"""## {status_emoji} Status: {status}\n\n**Description:** {description}\n\n**Recommended Action:** {action}\n\n**Time:** {timestamp}\n\n{whatsapp_status}"""
    return result_md, alert_history_to_dataframe()

def alert_history_to_dataframe():
    if not alert_history:
        return [["No alerts yet", "", "", ""]]
    return [[a["time"], a["status"], a["description"], a["action"]] for a in alert_history[-20:]]

def send_test_alert(phone_number):
    if not phone_number:
        return "\u26a0\ufe0f Please enter a phone number first"
    try:
        import pywhatkit
        pywhatkit.sendwhatmsg_instantly(phone_number, "\U0001f9ea Warehouse Watchdog TEST: Alert system is working!", wait_time=10, tab_close=True, close_time=3)
        return "\u2705 Test alert sent! Check your WhatsApp."
    except Exception as e:
        return f"\u274c Failed: {str(e)}"

# --- Custom Theme ---
watchdog_theme = gr.themes.Base(
    primary_hue=gr.themes.colors.red,
    secondary_hue=gr.themes.colors.gray,
    neutral_hue=gr.themes.colors.slate,
    font=gr.themes.GoogleFont("Inter"),
).set(
    body_background_fill="#0f1419",
    body_background_fill_dark="#0f1419",
    block_background_fill="#1a2332",
    block_background_fill_dark="#1a2332",
    block_border_color="#2d3748",
    block_border_color_dark="#2d3748",
    block_label_text_color="#e2e8f0",
    block_label_text_color_dark="#e2e8f0",
    block_title_text_color="#f7fafc",
    block_title_text_color_dark="#f7fafc",
    body_text_color="#cbd5e0",
    body_text_color_dark="#cbd5e0",
    button_primary_background_fill="#e53e3e",
    button_primary_background_fill_dark="#e53e3e",
    button_primary_text_color="#ffffff",
    button_primary_text_color_dark="#ffffff",
    input_background_fill="#2d3748",
    input_background_fill_dark="#2d3748",
    input_border_color="#4a5568",
    input_border_color_dark="#4a5568",
)

custom_css = """
.alert-critical {
    border: 2px solid #e53e3e !important;
    animation: pulse-red 2s infinite;
}
.alert-suspicious {
    border: 2px solid #d69e2e !important;
}
@keyframes pulse-red {
    0%, 100% { box-shadow: 0 0 0 0 rgba(229, 62, 62, 0.4); }
    50% { box-shadow: 0 0 0 10px rgba(229, 62, 62, 0); }
}
.header-bar {
    background: linear-gradient(90deg, #1a2332 0%, #2d3748 100%);
    padding: 20px;
    border-radius: 8px;
    margin-bottom: 16px;
    border: 1px solid #4a5568;
}
"""

with gr.Blocks(theme=watchdog_theme, css=custom_css, title="Warehouse Watchdog") as demo:
    gr.HTML("""
    <div class="header-bar">
        <h1 style="color: #f7fafc; margin: 0; font-size: 2em;">
            \U0001f4f9 Warehouse Watchdog
        </h1>
        <p style="color: #a0aec0; margin: 4px 0 0 0;">
            AI-Powered Surveillance | Qwen2.5-VL-7B | No Cloud APIs
        </p>
    </div>
    """)
    with gr.Row():
        with gr.Column(scale=1):
            camera_input = gr.Image(sources=["webcam", "upload"], type="pil", label="Camera Feed", mirror_webcam=False)
            analyze_btn = gr.Button("\U0001f50d Analyze Frame", variant="primary", size="lg")
            with gr.Accordion("Alert Settings", open=False):
                phone_input = gr.Textbox(label="WhatsApp Number (with country code)", placeholder="+1234567890", type="text")
                sensitivity_input = gr.Radio(choices=["Low", "Medium", "High"], value="Medium", label="Alert Sensitivity")
                test_btn = gr.Button("\U0001f9ea Send Test Alert", size="sm")
                test_output = gr.Textbox(label="Test Result", interactive=False)
        with gr.Column(scale=1):
            analysis_output = gr.Markdown(value="## Waiting for input...\nCapture a frame or upload an image to begin surveillance.", label="Analysis Result")
            gr.Markdown("### \U0001f4cb Alert History")
            history_table = gr.Dataframe(headers=["Time", "Status", "Description", "Action"], value=[["No alerts yet", "", "", ""]], interactive=False, wrap=True)
    analyze_btn.click(fn=run_surveillance, inputs=[camera_input, phone_input, sensitivity_input], outputs=[analysis_output, history_table])
    test_btn.click(fn=send_test_alert, inputs=[phone_input], outputs=[test_output])

demo.launch()

Your surveillance app now looks like a dedicated security operations tool instead of a generic demo. Next up, you will deploy it to Hugging Face Spaces so it runs on ZeroGPU and is accessible from anywhere.

Deploy to Hugging Face Spaces

Your surveillance app is fully built and themed locally. Now it is time to get it live on the internet so your father can actually use it.

Hugging Face Spaces hosts Gradio apps as git repositories. You will push your project files to a Space under the hackathon organization, and ZeroGPU will automatically build and run your app with GPU access.

In this step, get ready to:

Confirm your README.md frontmatter matches the ZeroGPU hardware requirements.
Push your project to a new Hugging Face Space under the build-small-hackathon organization.
Verify the Space builds and runs correctly with a webcam snapshot.

Confirm your README.md frontmatter

Hugging Face reads a YAML frontmatter block at the top of your README.md to know which SDK to use, which hardware to allocate, and which file to run.

Open README.md in your project folder and confirm it matches the following:

---
title: Warehouse Watchdog
emoji: 📹
colorFrom: gray
colorTo: red
sdk: gradio
sdk_version: 5.29.0
app_file: app.py
pinned: false
license: mit
hardware: zero-a10g
---

What do these fields do?

sdk: gradio tells Hugging Face to install Gradio and use it as the app framework.
sdk_version: 5.29.0 pins the exact Gradio version so your app does not break on updates.
hardware: zero-a10g requests ZeroGPU allocation. This gives your app access to a shared A10G GPU with 48GB VRAM.
app_file: app.py points to the Python file that Gradio should launch.

Save README.md if you made any changes.

Push your project to a new Space

Hugging Face Spaces are git repositories. You will create a new Space on the web, then push your local code to it using git.

Open your browser and go to:
https://huggingface.co/new-space?owner=build-small-hackathon.
Set the Owner dropdown to build-small-hackathon.
Enter warehouse-watchdog as the Space name.
Select Gradio as the SDK.
Leave the visibility as Public.
Click Create Space.

Why under the build-small-hackathon org?

The Build Small Hackathon requires submissions to be hosted under the build-small-hackathon organization on Hugging Face. You must be a member of this org before creating a Space there.

Now you need a write-access token so git can authenticate with Hugging Face when you push.

Go to https://huggingface.co/settings/tokens.
Click New token.
Give it a name like warehouse-watchdog-deploy.
Set the role to Write.
Click Generate a token.
Copy the token value. You will need it in the next step.

Keep your token private

Never share your Hugging Face access token or commit it to a repository. You will paste it when git prompts for a password, but it will not be stored in your project files.

Next, initialize a git repository in your project folder and push it to the Space.

In your terminal, make sure you are inside your warehouse-watchdog project folder.
Initialize a git repo, add all files, and commit by running these commands:

git init
git add .
git commit -m "Initial commit: Warehouse Watchdog surveillance app"

Add the Hugging Face Space as a remote by running:

git remote add space https://huggingface.co/spaces/build-small-hackathon/warehouse-watchdog

Push your code to the Space by running:

git push --force space main

What happens when git asks for credentials?

When you run git push, you will be prompted for a username and password. Enter your Hugging Face username as the username. Paste your write-access token as the password.

After the push completes, open your browser and go to:
https://huggingface.co/spaces/build-small-hackathon/warehouse-watchdog.
You should see the Space page with a Building status indicator.

Push rejected or authentication error?

Make sure your token has Write permission. Read-only tokens cannot push.
Confirm you are a member of the build-small-hackathon organization. You need at least contributor access to push.
If you see a branch error, confirm you are on the main branch locally by running git branch. If your default branch is named master, rename it first with git branch -M main.

help me push my code to a Hugging Face Space

Verify the Space builds and test with a webcam snapshot

The Space automatically installs your requirements.txt dependencies and launches app.py once the build finishes. The first build takes a few minutes because it downloads the Qwen2.5-VL-7B model weights.

Wait for the Space status to change from Building to Running. This can take 3-5 minutes.
Once the app loads, you should see the dark-themed Warehouse Watchdog dashboard with the camera feed panel on the left and the analysis panel on the right.
Click the webcam icon in the Camera Feed panel to allow camera access, then capture a snapshot.
Click Analyze Frame.
Wait for the VLM to process the image. You should see a status result (SAFE, SUSPICIOUS, or CRITICAL) appear in the right panel within 15-30 seconds.

What if I don't have a webcam?

The camera input also accepts uploaded images. Click the upload icon instead and select any image from your computer to test the analysis.

Space stuck on Building or showing an error?

Click the Logs tab on your Space page to see the build output. Look for the specific error line.
A common issue is a typo in requirements.txt. Double check that all package names are spelled correctly.
If you see a CUDA out-of-memory error, confirm your README.md has hardware: zero-a10g and not a CPU setting.
The free tier gives you 5 minutes of daily GPU time. If you have already used your quota, the Space will queue until the next day.

help me debug my Hugging Face Space build error

Your Warehouse Watchdog is live on the internet. Anyone with the link can now use your AI surveillance system. Next up, you will take it further with an automated patrol mode in the Secret Mission.

Secret mission

Patrol Mode

Your Warehouse Watchdog is powerful, but it still relies on someone pressing a button to analyze each frame. A real security system runs continuously. In this secret mission, you'll add an automated patrol mode that captures frames at set intervals and generates a downloadable security report.

Clean Up Your Resources

This project uses the free tier of Hugging Face Spaces with ZeroGPU, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

Hugging Face Space (warehouse-watchdog under the build-small-hackathon organization).
Local project folder on your Desktop (containing app.py, requirements.txt, README.md).
Installed Python packages (gradio, transformers, torch, spaces, Pillow, qwen-vl-utils, pywhatkit, accelerate).

Keep everything running

No action needed. Choose this if you plan to continue developing your Warehouse Watchdog or want to show it off during the hackathon demo submission.

Your Hugging Face Space will automatically sleep after a period of inactivity on free hardware. It will wake up again when someone visits it.
Your local project files remain on your Desktop.
Your installed Python packages stay available for other projects that use the same libraries.

Pause - I'll come back to this later

Shut down running processes to free up resources while keeping your files and Space intact for later.

Open your Space at huggingface.co/spaces/build-small-hackathon/warehouse-watchdog.
Click the Settings tab at the top of the Space page.
Scroll down and pause the Space. A paused Space stops executing until you manually restart it.
Your local project folder and installed packages can stay as-is.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each step below.

Delete the Hugging Face Space by running this command in your terminal:

huggingface-cli repo delete build-small-hackathon/warehouse-watchdog --repo-type space

What does this command do?

This uses the official Hugging Face CLI to permanently delete your Space repository from the Hub. You will be prompted to confirm before deletion proceeds.

Delete your local project folder by running:

macOS

cd ~/Desktop
rm -rf warehouse-watchdog

Windows

cd ~\Desktop
rmdir /s /q warehouse-watchdog

Uninstall the Python packages you installed for this project by running:

pip uninstall -r requirements.txt -y

Run this from your project folder

Make sure you run the uninstall command from inside your project folder (before deleting it) so pip can find the requirements.txt file. Alternatively, copy the file path and pass it to the -r flag directly.

Nice Work!

You just built a fully functional AI surveillance system for a real warehouse. Your Warehouse Watchdog uses a vision-language model to analyze camera frames, detect anomalies, and send alerts when something needs attention.

You've learned how to:

Built a VLM-powered surveillance app using Qwen2.5-VL-7B-Instruct with structured prompt engineering to classify frames as SAFE, SUSPICIOUS, or CRITICAL.
Created a custom-themed Gradio dashboard with a dark security operations center aesthetic, and deployed it as a Hugging Face Space running on ZeroGPU.
Integrated WhatsApp alerts using pywhatkit for real-time notifications when suspicious or critical activity is detected.
Added patrol mode for automated continuous monitoring with exportable JSON security reports.

Ready to quiz yourself?