AI-Powered Malware Triage Sandbox

Build a local AI analyst that triages suspicious files with YARA and LLM reasoning.

Introduction

30 Second Summary

When a suspicious file lands on a security team's desk, the first question is always the same: is this dangerous, and what does it do? Answering that manually means hours of sifting through hex editors, import tables, and signature databases.

In this project, you will build a local malware triage API that automates this entire process. You'll upload a file, run it through a multi-layer static analysis pipeline, and hand the results to a local AI model that produces a structured threat intelligence report.

What You'll Build

You drop a suspicious file onto your API endpoint, and seconds later get back a JSON threat report where the AI explains exactly why that combination of entropy, imports, and signature matches adds up to a verdict.

By the end of this project, you'll have:

  • A running API where you upload any file and get back a structured JSON threat report with a verdict, confidence score, and AI-generated reasoning that connects multiple indicators together.
  • A multi-layer analysis engine where YARA signature rules catch known-bad patterns, while a local LLM reasons about novel combinations the rules would miss and maps findings to MITRE ATT&CK technique IDs.
  • A fully air-gapped pipeline powered by Ollama where all inference runs locally. No file hashes, no telemetry, no samples leaving your machine.
  • Secret Mission: Containerize the entire pipeline in Docker so suspicious files are analyzed inside a disposable container that never touches your host filesystem.

Are there any prerequisites?

This project assumes you have Python and Docker Desktop already installed on Windows. You should be comfortable writing basic Python and running terminal commands.

Before We Start

Your malware triage system needs two things before it can analyze anything: a local AI model that can reason about threats, and a set of Python libraries that extract raw indicators from suspicious files.

In this step, you will install Ollama to run a local LLM, then set up a Python environment with the analysis libraries your pipeline needs.

In this step, get ready to:
  • Install Ollama on Windows and pull the llama3.2 model.
  • Create a Python virtual environment and install analysis dependencies.
  • Verify Ollama responds and all Python imports succeed.
Install Ollama and pull the llama3.2 model

Ollama is a local inference server that runs LLMs on your machine without sending data to any external API. Your triage system will send analysis results to Ollama and receive structured threat reports back.

  • Download the Ollama installer by visiting ollama.com/download/windows in your browser.
  • Run the downloaded OllamaSetup.exe and follow the prompts to complete the installation.

What happens after install?

Ollama starts automatically as a background process and places an icon in your system tray. You do not need to launch it manually. The ollama command becomes available in any new terminal window.

  • Open a new terminal by pressing the Windows key, typing PowerShell, and pressing Enter.
  • Verify Ollama is installed by running this command:
ollama --version

✔️ I see a version number

Ollama is ready. Next you will pull the AI model.

ⓧ Command not found

The terminal does not recognize the command yet. Try these fixes:

  • Close this terminal window and open a brand new one. The installer updates the system PATH, but existing terminals do not pick up the change.
  • Confirm the Ollama icon appears in your system tray (bottom-right corner of your taskbar). If it does not, re-run OllamaSetup.exe.
  • Pull the llama3.2 model (about 2GB download) by running:
ollama pull llama3.2

What is llama3.2?

Llama 3.2 is a 3-billion parameter model from Meta. It is small enough to run on most machines without a dedicated GPU, yet capable enough to reason about security indicators and produce structured JSON output.

The download takes a few minutes depending on your connection. You will see a progress bar fill to 100%.

  • Confirm the model is working by sending it a test prompt:
ollama run llama3.2 "Say hello"

You should see a short greeting from the model printed to your terminal. This confirms Ollama can load and run the model successfully.

Not seeing a response?

  • If you see a connection error, make sure the Ollama icon is visible in your system tray. If it is not, restart your computer and try again.
  • If the command hangs for more than 60 seconds, the model may still be loading into memory for the first time. Wait for it to finish.
  • Why can't Ollama connect on my machine?
Create your project folder and install dependencies

With Ollama running, you need a Python project with the libraries that perform static file analysis. You will create a virtual environment to keep these dependencies isolated from the rest of your system.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir malware-triage
cd malware-triage
  • Create a Python virtual environment by running:
python -m venv .venv
  • Activate the virtual environment by running:
.venv\Scripts\activate

You should see (.venv) appear at the start of your terminal prompt. This confirms the virtual environment is active.

Seeing a permissions error on activate?

Windows PowerShell may block scripts by default. Run the following command first, then retry the activate command:

Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned

Help me fix PowerShell script execution errors.

  • Now create the requirements.txt file that lists your project dependencies. Run this command to create and write the file:
@"
fastapi[standard]==0.139.0
yara-python==4.5.4
pefile==2024.8.26
httpx==0.28.1
"@ | Out-File -Encoding utf8 requirements.txt

What are these dependencies?

  • yara-python lets you write and run YARA signature rules against files to detect known malware patterns.
  • pefile parses Windows PE (Portable Executable) headers to extract imports, sections, and metadata from binaries.
  • fastapi[standard] is a modern Python web framework that will serve your triage API. The standard extras include a web server and file upload support.
  • httpx is an HTTP client your code will use to talk to Ollama's local API.
  • Install all dependencies by running:
pip install -r requirements.txt

You will see pip downloading and installing each package. The process finishes with a "Successfully installed" message listing all four packages.

Installation failing for yara-python?

  • Make sure your virtual environment is active (you should see (.venv) in your prompt). If not, run the activate command again.
  • If you see a build error, make sure you are running Python 3.10 or higher. Pre-built wheels for yara-python are available for Python 3.10 through 3.12 on Windows.
  • Help me fix pip install errors for yara-python on Windows.
Verify all tools are working

Before writing any project code, confirm that every piece of the toolchain is functioning. You will check that Python can import each library and that Ollama is still responding.

  • Before you run the next command, predict: do you think all four imports will succeed, or will one fail?
  • Test that all Python libraries import correctly by running:
python -c "import yara; import pefile; import fastapi; import httpx; print('All imports OK')"

You should see All imports OK printed with no errors.

  • Confirm Ollama still responds by running your test prompt one more time:
ollama run llama3.2 "Say hello"

You should see a short greeting from the model. Both checks passing means your entire toolchain is ready.

✔️ Awesome, I've got everything!

Great. Double check you have saved your requirements.txt file in the malware-triage folder.

ⓧ I'd like to double check the full code

Here is the complete requirements.txt for comparison:

fastapi[standard]==0.139.0
yara-python==4.5.4
pefile==2024.8.26
httpx==0.28.1

Your toolchain is locked in. Next up, you will build the static analysis engine that extracts the raw threat indicators from suspicious files.

Set Up the Project and Install Dependencies

Your malware triage sandbox relies on four Python libraries working together: a file upload framework, a binary format parser, a signature engine, and an HTTP client for the AI model.

Pinning exact versions in a requirements file ensures your analysis pipeline produces consistent results every time. In this step, you will lock down your dependency versions and create the project directory structure.

In this step, get ready to:
  • Pin dependency versions in a requirements file.
  • Install dependencies from the pinned file.
  • Create the project directory structure for YARA rules.
Create the requirements file with pinned versions

A requirements.txt file locks every dependency to an exact version. This prevents silent breakage when a library releases an update that changes behavior.

  • In your project folder, open requirements.txt in your editor and replace its entire contents with the following:
fastapi[standard]==0.139.0
yara-python==4.5.4
pefile==2024.8.26
httpx==0.28.1

What do these packages do?

  • fastapi[standard] provides the web framework, dev server, file upload handling, and the built-in Swagger UI for testing your API.
  • yara-python compiles and runs YARA signature rules against file bytes to detect known malware patterns.
  • pefile parses Windows PE (Portable Executable) headers to extract imports, sections, and timestamps from binary files.
  • httpx is an HTTP client that sends analysis results to your local Ollama model and receives the AI threat report.
  • Save requirements.txt.
  • Install the pinned versions into your virtual environment by running:
pip install -r requirements.txt

You should see pip downloading and installing each package with its exact pinned version. The output ends with a line confirming the packages are installed successfully.

Seeing an install error?

  • Make sure your virtual environment is activated. You should see the environment name in parentheses at the start of your terminal prompt.
  • If yara-python fails, ensure you are running Python 3.10 or higher. The pre-built wheels require a recent Python version.
  • If pip reports a version conflict, delete the venv folder, recreate it, activate it, and run the install command again from a clean state.

help me fix a pip install error for my malware triage project dependencies

  • Confirm the correct versions are installed by running:
pip show fastapi yara-python pefile httpx

You should see Version: 0.139.0 for fastapi, Version: 4.5.4 for yara-python, Version: 2024.8.26 for pefile, and Version: 0.28.1 for httpx.

Create the YARA rules directory

Your scanner will load custom YARA rules from a dedicated folder. You will write the actual rules in a later step, but the directory needs to exist now so the project structure is ready.

  • Create the yara_rules directory and a .gitkeep placeholder file by running:
mkdir yara_rules
type nul > yara_rules\.gitkeep

What is a .gitkeep file?

Git does not track empty directories. A .gitkeep file is a convention that forces Git to include the folder in version control. The file itself is empty and has no effect on your code.

  • Confirm the directory was created by running:
ls

You should see requirements.txt and the yara_rules folder listed in the output.

Don't see yara_rules listed?

  • Check that you ran the mkdir command from inside your project folder. If you are in the wrong directory, use cd to navigate back.
  • On Windows, use dir if ls does not work in your terminal. Both show directory contents.

help me find my yara_rules directory in my malware triage project

Verify the full setup

Before you start writing application code, confirm that all Python imports work correctly. This catches any dependency issues now, before they surface as cryptic errors mid-build.

  • Test that all four libraries import successfully by running:
python -c "import fastapi; import yara; import pefile; import httpx; print('All imports OK')"

What do you expect to see when you run this?

You should see All imports OK printed with no errors. This confirms every library installed correctly and is accessible in your virtual environment.

Seeing a ModuleNotFoundError?

  • Double-check that your virtual environment is active. The prompt should show the venv name in parentheses.
  • Run pip install -r requirements.txt again. If a package failed silently during the first install, a second run will retry it.

help me fix a ModuleNotFoundError when importing my malware triage dependencies

✔️ Awesome, I've got everything!

Great. Double check you have saved requirements.txt and confirmed the yara_rules directory exists.

ⓧ I'd like to double check the full code

Your requirements.txt should contain exactly this:

fastapi[standard]==0.139.0
yara-python==4.5.4
pefile==2024.8.26
httpx==0.28.1

Your project folder should also contain an empty yara_rules/.gitkeep file.

Your project is locked and loaded. Dependencies are pinned, imports work, and your directory structure is ready. Next up, you will build the static analysis engine that extracts entropy, strings, and PE header data from any file you throw at it.

Build the Static Analysis Engine

Your tools are installed and Ollama is ready to go. Now you need to give the AI analyst something to reason about.

Before the AI can determine whether a file is malicious, it needs raw indicators. In this step, you will build the static analysis engine that extracts the signals a human analyst would look at first: file entropy, suspicious strings, and PE header data.

In this step, get ready to:
  • Write an entropy calculator and string extractor module.
  • Add PE header parsing to extract imports, sections, and compilation timestamps.
  • Create a synthetic test binary and run the full analysis pipeline against it.
Write the entropy calculator and string extractor

Shannon entropy measures the randomness of bytes in a file. A score near 0 means the data is uniform or repetitive, while a score above 7 (out of a maximum 8) suggests the file is packed or encrypted.

  • Create a new file called analyzer.py in your project folder with the following code:
import math
import re
from pathlib import Path

try:
    import pefile
except ImportError:
    pefile = None


def calculate_entropy(data: bytes) -> float:
    # Return 0 for empty files
    if not data:
        return 0.0
    # Count how often each byte value (0-255) appears
    frequency = [0] * 256
    for byte in data:
        frequency[byte] += 1
    length = len(data)
    entropy = 0.0
    for count in frequency:
        if count > 0:
            probability = count / length
            entropy -= probability * math.log2(probability)
    return round(entropy, 4)

What does this code do?

The try/except block imports pefile gracefully. If the import fails, the variable is set to None so the rest of the module still works.

The calculate_entropy function implements the Shannon entropy formula. It counts how often each of the 256 possible byte values appears, converts counts to probabilities, and sums -p * log2(p) for each. The result ranges from 0.0 (completely uniform) to 8.0 (maximum randomness).

  • Save analyzer.py.
  • Verify the entropy function works by running this command:
python -c "from analyzer import calculate_entropy; print(calculate_entropy(b'AAAAAAA'))"

You should see 0.0 printed, because a string of identical bytes has zero entropy.

Seeing a ModuleNotFoundError?

Make sure your virtual environment is active (you should see the environment name in your terminal prompt). Also confirm you are running the command from the same folder where analyzer.py lives.

help me debug a ModuleNotFoundError when importing from analyzer.py

Now add the string extractor that pulls out URLs, IP addresses, registry keys, and suspicious keywords from raw file bytes.

  • In analyzer.py, add the following function below calculate_entropy:
def extract_strings(data: bytes, min_length: int = 6) -> dict:
    # Find all printable ASCII strings of at least min_length characters
    ascii_pattern = re.compile(rb"[\x20-\x7e]{" + str(min_length).encode() + rb",}")
    ascii_strings = [s.decode("ascii") for s in ascii_pattern.findall(data)]

    # Categorize strings by type
    urls = [s for s in ascii_strings if re.match(r"https?://", s)]
    ips = [s for s in ascii_strings if re.match(r"\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}", s)]
    registry_keys = [s for s in ascii_strings if "HKEY_" in s or "\\Registry\\" in s]
    file_paths = [s for s in ascii_strings if re.match(r"[A-Z]:\\", s)]

    return {
        "total_count": len(ascii_strings),
        "urls": urls[:20],
        "ip_addresses": ips[:20],
        "registry_keys": registry_keys[:20],
        "file_paths": file_paths[:20],
        "suspicious_keywords": [
            s for s in ascii_strings
            if any(kw in s.lower() for kw in [
                "password", "credential", "inject", "hook",
                "payload", "decrypt", "encode", "shell",
                "admin", "elevat", "privilege"
            ])
        ][:20],
    }

What does this code do?

The regex [\x20-\x7e]{6,} finds sequences of 6 or more printable ASCII characters in raw binary data. This is the same technique the Unix strings command uses.

Once extracted, each string gets classified: URLs, IPs, Windows registry paths, file paths, and keywords associated with malicious intent. The [:20] slices cap output so a massive binary does not overwhelm the AI analyst later.

  • Save analyzer.py.
  • Test the string extractor by running:
python -c "from analyzer import extract_strings; print(extract_strings(b'http://evil.com/payload password_stealer'))"

You should see a dictionary with total_count, the URL in the urls list, and the keyword match in suspicious_keywords.

Add PE header parsing

PE (Portable Executable) is the file format for Windows executables. Its headers reveal what system APIs the binary imports, which tells you what capabilities it has.

  • In analyzer.py, add the analyze_pe function below extract_strings:
def analyze_pe(filepath: str) -> dict | None:
    if pefile is None:
        return None
    try:
        pe = pefile.PE(filepath)
    except pefile.PEFormatError:
        return None

    # Extract imported DLLs and their functions
    imports = {}
    if hasattr(pe, "DIRECTORY_ENTRY_IMPORT"):
        for entry in pe.DIRECTORY_ENTRY_IMPORT:
            dll_name = entry.dll.decode("utf-8", errors="replace")
            functions = []
            for imp in entry.imports:
                if imp.name:
                    functions.append(imp.name.decode("utf-8", errors="replace"))
            imports[dll_name] = functions

    # Extract section details (name, sizes, per-section entropy)
    sections = []
    for section in pe.sections:
        name = section.Name.decode("utf-8", errors="replace").strip("\x00")
        sections.append({
            "name": name,
            "virtual_size": section.Misc_VirtualSize,
            "raw_size": section.SizeOfRawData,
            "entropy": round(section.get_entropy(), 4),
        })

What does this code do?

The function first checks if pefile loaded successfully, and gracefully returns None for non-PE files.

For valid PE files, it walks the import table to see which DLLs and functions the binary calls. It also reads each section's entropy. A section with entropy above 7 usually means the code is packed or encrypted.

  • Still in analyzer.py, add the suspicious imports check and return statement to complete analyze_pe. Paste this directly below the code you just added (still inside the function):
    # Flag imports that are commonly abused by malware
    suspicious_imports = []
    dangerous_apis = [
        "VirtualAllocEx", "WriteProcessMemory", "CreateRemoteThread",
        "NtUnmapViewOfSection", "SetWindowsHookEx", "GetAsyncKeyState",
        "URLDownloadToFile", "WinExec", "ShellExecute",
        "RegSetValueEx", "CryptDecrypt", "IsDebuggerPresent",
    ]
    for dll, funcs in imports.items():
        for func in funcs:
            if func in dangerous_apis:
                suspicious_imports.append(f"{dll}:{func}")

    pe.close()

    return {
        "is_pe": True,
        "machine_type": hex(pe.FILE_HEADER.Machine),
        "compile_timestamp": pe.FILE_HEADER.TimeDateStamp,
        "number_of_sections": pe.FILE_HEADER.NumberOfSections,
        "sections": sections,
        "imports": imports,
        "suspicious_imports": suspicious_imports,
    }

What does this code do?

The dangerous_apis list contains Windows API functions commonly abused by malware for process injection, keylogging, downloading payloads, and evading debuggers. If a binary imports any of these, the analyzer flags them.

  • Now add the analyze_file wrapper function below analyze_pe. This ties all three analysis functions together:
def analyze_file(filepath: str) -> dict:
    # Read the file once and run all analysis passes
    path = Path(filepath)
    data = path.read_bytes()

    result = {
        "filename": path.name,
        "file_size": len(data),
        "entropy": calculate_entropy(data),
        "strings": extract_strings(data),
        "pe_analysis": analyze_pe(filepath),
    }

    return result

What does this code do?

The analyze_file function reads the file once, runs entropy calculation, string extraction, and PE parsing, then returns one combined dictionary. This is the single entry point the rest of the pipeline will use.

  • Save analyzer.py.
  • Test the full pipeline against your requirements.txt (a plain text file, not a PE) by running:
python -c "from analyzer import analyze_file; import json; print(json.dumps(analyze_file('requirements.txt'), indent=2))"

You should see JSON output with an entropy value around 4-5, some extracted strings, and pe_analysis set to null (because requirements.txt is not a PE file).

Seeing a syntax or indentation error?

Make sure the suspicious_imports block is indented inside analyze_pe, while analyze_file is a new top-level function with no indentation before def.

help me fix indentation issues in my analyzer.py file

✔️ Awesome, I've got everything!

Great. Double check you have saved analyzer.py and that it contains all four functions: calculate_entropy, extract_strings, analyze_pe, and analyze_file.

ⓧ I'd like to double check the full code

Scroll up and confirm your analyzer.py contains the following chunks in order:

  • Imports (math, re, Path, pefile with try/except).
  • The calculate_entropy function.
  • The extract_strings function.
  • The analyze_pe function (imports parsing, sections parsing, suspicious imports check, and return statement).
  • The analyze_file wrapper function.
Create a test sample and run the full pipeline

Your analyzer needs a realistic test file. You will write a script that generates a synthetic binary with PE headers and embedded suspicious strings.

  • Create a new file called create_test_sample.py and add the following code:
"""Generate a synthetic test file with suspicious characteristics for testing the analysis pipeline."""

import struct
from pathlib import Path


def create_test_sample():
    output_path = Path("test_sample.exe")

    # MZ header (DOS stub)
    mz_header = b"MZ" + b"\x00" * 58 + struct.pack("<I", 64)

    # Minimal PE signature
    pe_sig = b"PE\x00\x00"

    # COFF header (IMAGE_FILE_HEADER) - x86
    coff_header = struct.pack("<HHIIIHH",
        0x014C,  # Machine: i386
        1,       # NumberOfSections
        1700000000,  # TimeDateStamp (suspicious future date)
        0, 0,    # PointerToSymbolTable, NumberOfSymbols
        0x00E0,  # SizeOfOptionalHeader
        0x0102,  # Characteristics: EXECUTABLE_IMAGE | 32BIT_MACHINE
    )

What does this code do?

This script builds a minimal PE file from scratch. The MZ header is the DOS stub that every Windows executable starts with. The PE signature and COFF header tell the OS this is an i386 executable.

The timestamp 1700000000 is a Unix epoch value in the future, which is a red flag. Legitimate compilers use the actual build time.

  • Still in create_test_sample.py, add the suspicious content and file-writing logic below the code you just added:
    # Fake suspicious content strings
    suspicious_content = b"\x00" * 100
    suspicious_content += b"http://malware-c2.evil.com/beacon\x00"
    suspicious_content += b"192.168.1.100\x00"
    suspicious_content += b"HKEY_LOCAL_MACHINE\\Software\\Microsoft\\Windows\\CurrentVersion\\Run\x00"
    suspicious_content += b"VirtualAllocEx\x00"
    suspicious_content += b"WriteProcessMemory\x00"
    suspicious_content += b"CreateRemoteThread\x00"
    suspicious_content += b"IsDebuggerPresent\x00"
    suspicious_content += b"UPX0\x00UPX1\x00"
    suspicious_content += b"payload_decrypt\x00"
    suspicious_content += b"C:\\Users\\victim\\credentials.txt\x00"
    suspicious_content += b"password_stealer_module\x00"
    suspicious_content += b"URLDownloadToFile\x00"
    suspicious_content += b"WSAStartup\x00"
    suspicious_content += b"https://exfil.badactor.net/upload\x00"
    suspicious_content += b"\x00" * 200

    # Combine everything
    file_data = mz_header + pe_sig + coff_header + b"\x00" * 224 + suspicious_content

    output_path.write_bytes(file_data)
    print(f"Test sample created: {output_path} ({len(file_data)} bytes)")
    print("This file is NOT actual malware - it contains suspicious strings for testing.")


if __name__ == "__main__":
    create_test_sample()

What does this code do?

The suspicious content simulates what you would find in real malware: C2 server URLs, internal IP addresses, registry persistence keys, process injection API names, packer signatures (UPX), and credential theft indicators.

This file is completely harmless. It is just bytes arranged to trigger the analyzer's detection logic during testing.

  • Save create_test_sample.py.
  • Generate the test binary by running:
python create_test_sample.py

You should see output confirming the file was created with its size in bytes.

  • Confirm test_sample.exe exists by running:
ls test_sample.exe

You should see test_sample.exe listed in the output.

Before you run the analyzer against this test file, what do you think the entropy will be? High (above 7, suggesting encryption) or low (below 4, suggesting lots of null padding)?

  • Run the full analysis pipeline against the test sample:
python -c "from analyzer import analyze_file; import json; print(json.dumps(analyze_file('test_sample.exe'), indent=2))"

You should see structured JSON with an entropy value (likely around 1-2 because of all the null padding), extracted URLs, IP addresses, registry keys, and suspicious keywords. The pe_analysis field will be null because the synthetic file does not have a complete PE optional header that pefile can fully parse.

Not seeing any URLs or strings in the output?

Make sure test_sample.exe exists in your current directory. Run ls to confirm. If the file is missing, re-run python create_test_sample.py.

If you see an empty strings result, check that extract_strings uses the raw byte pattern rb"[\x20-\x7e]{6,}" (with the rb prefix for a raw byte string).

help me debug why my analyzer is not finding strings in the test sample

Your static analysis engine is complete. Next up, you will add YARA signature detection to catch known malware patterns that raw indicators alone cannot identify.

Add YARA Signature Detection

Your analysis engine from the previous step extracts raw indicators like entropy scores, suspicious strings, and PE imports. But raw data alone cannot tell you "this file matches a known packer" or "these API calls match a process injection pattern."

YARA is the industry-standard signature language used by VirusTotal, CAPEv2, and every major malware sandbox. In this step, you'll write rules that encode expert threat knowledge into machine-readable signatures, then build a scanner module that runs those rules against any file.

In this step, get ready to:
  • Write six YARA rules covering packers, process injection, keyloggers, network indicators, crypto activity, and anti-analysis techniques.
  • Build a scanning module that compiles rules and matches them against file bytes.
  • Run the scanner against your test sample and see multiple rules fire with matched strings.
Write the YARA rule definitions

Each YARA rule has three parts: meta (description and severity), strings (byte patterns to search for), and condition (how many strings must match for the rule to fire).

  • Create a new file called scanner.py in your project folder.
  • Add the imports, rule directory path, and the first two YARA rules by pasting this code:
import yara
from pathlib import Path

RULES_DIR = Path(__file__).parent / "yara_rules"

BUILTIN_RULES = """
rule Suspicious_Packer_UPX {
    meta:
        description = "Detects UPX packed executables"
        severity = "medium"
    strings:
        $upx0 = "UPX0" ascii
        $upx1 = "UPX1" ascii
        $upx2 = "UPX!" ascii
    condition:
        any of them
}

rule Suspicious_Process_Injection_APIs {
    meta:
        description = "Detects combination of APIs commonly used for process injection"
        severity = "high"
    strings:
        $api1 = "VirtualAllocEx" ascii
        $api2 = "WriteProcessMemory" ascii
        $api3 = "CreateRemoteThread" ascii
        $api4 = "NtUnmapViewOfSection" ascii
    condition:
        2 of them
}

What do these rules do?

The first rule detects UPX-packed executables by looking for the signature strings that the UPX packer leaves behind. Packed binaries are a strong indicator of evasion attempts.

The second rule fires when at least 2 of the 4 process injection APIs are present in the file. A single API is normal. Two or more together strongly suggest code injection capability.

  • Save scanner.py.
  • Confirm the file exists by running:
ls scanner.py

You should see scanner.py listed in the output.

  • Add the next two rules (keylogger detection and network indicators) directly below the closing } of the Process Injection rule:
rule Suspicious_Keylogger_Indicators {
    meta:
        description = "Detects potential keylogger behavior"
        severity = "high"
    strings:
        $api1 = "GetAsyncKeyState" ascii
        $api2 = "SetWindowsHookEx" ascii
        $api3 = "GetKeyState" ascii
        $file = "keylog" ascii nocase
    condition:
        2 of them
}

rule Suspicious_Network_Indicators {
    meta:
        description = "Detects network-related suspicious strings"
        severity = "medium"
    strings:
        $url1 = "http://" ascii
        $url2 = "https://" ascii
        $download = "URLDownloadToFile" ascii
        $socket = "WSAStartup" ascii
        $useragent = "User-Agent" ascii
    condition:
        ($download or $socket) and ($url1 or $url2)
}

What do these rules detect?

The keylogger rule looks for Windows APIs that capture keystrokes. The nocase modifier on $file means it matches "keylog", "KEYLOG", or any mixed case variant.

The network rule requires BOTH a download/socket API AND a URL string present in the same file. This catches binaries that fetch payloads from remote servers.

  • Add the final two rules (crypto activity and anti-analysis) below the Network Indicators rule, then close the triple-quoted string:
rule Suspicious_Crypto_Activity {
    meta:
        description = "Detects cryptographic API usage common in ransomware"
        severity = "high"
    strings:
        $crypt1 = "CryptEncrypt" ascii
        $crypt2 = "CryptDecrypt" ascii
        $crypt3 = "CryptGenKey" ascii
        $ransom = "Your files have been encrypted" ascii nocase
        $bitcoin = "bitcoin" ascii nocase
    condition:
        2 of ($crypt*) or $ransom or $bitcoin
}

rule Suspicious_Anti_Analysis {
    meta:
        description = "Detects anti-debugging and anti-VM techniques"
        severity = "medium"
    strings:
        $debug1 = "IsDebuggerPresent" ascii
        $debug2 = "CheckRemoteDebuggerPresent" ascii
        $vm1 = "VMware" ascii
        $vm2 = "VirtualBox" ascii
        $vm3 = "QEMU" ascii
        $sleep = "SleepEx" ascii
    condition:
        any of ($debug*) or any of ($vm*) and $sleep
}
"""

What do these rules catch?

The crypto rule detects ransomware indicators. The wildcard ($crypt*) matches any string whose identifier starts with $crypt. Two crypto APIs together, or a ransom note string, triggers the rule.

The anti-analysis rule catches binaries that check for debuggers or virtual machines. Malware uses these checks to evade analysis environments.

  • Save scanner.py.
  • Verify that all 6 rules compile without syntax errors by running:
python -c "from scanner import BUILTIN_RULES; import yara; yara.compile(source=BUILTIN_RULES); print('All 6 YARA rules compiled successfully')"

You should see All 6 YARA rules compiled successfully printed. This confirms your rule syntax is valid.

Seeing a syntax error?

Check that the closing """ is on its own line after the last rule's closing brace. Also ensure each rule's condition: section is indented with spaces, not tabs.

If you see ModuleNotFoundError: No module named 'yara', make sure your virtual environment is active.

help me fix a YARA rule compilation error in scanner.py

Build the compile and scan functions

The rules are defined, but you need functions to compile them at runtime and scan actual files. The scanner will return structured match data including which rule fired, its severity metadata, and the exact strings that triggered it.

  • Add the following two functions at the bottom of scanner.py, below the closing """ of BUILTIN_RULES:
def compile_rules() -> yara.Rules:
    # Compile all YARA rules from the source string
    rules = yara.compile(source=BUILTIN_RULES)
    return rules


def scan_file(filepath: str) -> list[dict]:
    # Compile rules and scan the target file
    rules = compile_rules()
    matches = rules.match(filepath)

    results = []
    for match in matches:
        matched_strings = []
        for string_match in match.strings:
            for instance in string_match.instances:
                matched_strings.append({
                    "identifier": string_match.identifier,
                    "offset": instance.offset,
                    "data": instance.matched_data.decode("utf-8", errors="replace")[:100],
                })

        results.append({
            "rule": match.rule,
            "tags": match.tags,
            "meta": match.meta,
            "strings": matched_strings[:10],
        })

    return results

What does this code do?

compile_rules() takes the BUILTIN_RULES string and compiles it into a yara.Rules object that the YARA engine can execute against file bytes.

scan_file() compiles the rules, scans the given file path, then iterates through every match. For each match, it extracts the rule name, metadata (including severity), and the exact byte strings that triggered the rule along with their file offset.

  • Save scanner.py.

Import error on save?

If your editor shows a red underline on import yara, confirm your virtual environment is selected as the Python interpreter. The yara-python package installed earlier provides the yara module.

help me resolve a yara import error in my Python project

Run the scanner against your test file

Your test sample from the previous step contains UPX packer strings, process injection APIs, network download functions, and anti-debugging checks. Multiple rules should fire.

Before you run the scanner, how many of your 6 rules do you think will match the test sample? Think about which suspicious strings you embedded in it.

  • Run the scanner against your test file by running:
python -c "from scanner import scan_file; import json; results = scan_file('test_sample.exe'); print(json.dumps(results, indent=2))"

You should see JSON output showing multiple YARA rules that matched, including rule names like Suspicious_Packer_UPX, Suspicious_Process_Injection_APIs, and Suspicious_Network_Indicators. Each match includes the exact strings found and their byte offsets in the file.

Why do some rules fire and others don't?

Your test sample contains UPX strings, 3 of the 4 process injection APIs, the network download API with URLs, and IsDebuggerPresent. That triggers the packer, injection, network, and anti-analysis rules.

The keylogger rule does not fire because your test file has no GetAsyncKeyState or SetWindowsHookEx strings. The crypto rule stays silent because there are no CryptEncrypt or ransom note strings. This is exactly how signature detection works in practice. Rules only fire on the specific indicators they are designed to catch.

Seeing an empty list?

Make sure test_sample.exe exists in your project folder. If you deleted it, regenerate it by running python create_test_sample.py again.

Also check that the file path in the command matches the actual filename. The scan expects test_sample.exe in the current directory.

help me debug why my YARA scanner returns no matches

✔️ Awesome, I've got everything!

Great. Double check you've saved scanner.py and that your scan output shows at least 3 rules matching against the test file.

ⓧ I'd like to double check the full code

Your scanner.py file is built from the chunks above. Scroll up and confirm these sections are present in order:

  • Imports (import yara, from pathlib import Path) and RULES_DIR definition.
  • BUILTIN_RULES triple-quoted string opening with all 6 rules: Suspicious_Packer_UPX, Suspicious_Process_Injection_APIs, Suspicious_Keylogger_Indicators, Suspicious_Network_Indicators, Suspicious_Crypto_Activity, Suspicious_Anti_Analysis.
  • Closing """ on its own line after the last rule.
  • compile_rules() function that calls yara.compile(source=BUILTIN_RULES).
  • scan_file() function that compiles rules, matches against the filepath, and returns a list of dicts with rule, tags, meta, and strings fields.

Your scanner is now catching known-bad patterns with precision. But YARA rules can only match what an expert has already seen and written a signature for. In the next step, you'll connect a local AI analyst that reasons about the combination of indicators and catches novel threats that no rule covers yet.

Connect the AI Threat Analyst

Your analysis pipeline now produces entropy scores, string extractions, PE imports, and YARA signature matches. That is a wall of raw data, not intelligence.

In this step, you will connect a local AI model that synthesizes all those signals into a structured threat report. The model reasons about indicator combinations and maps findings to MITRE ATT&CK techniques, turning scattered data points into actionable threat intelligence.

In this step, get ready to:
  • Write the Ollama integration module with a structured output JSON schema.
  • Design a prompt that formats all indicators for AI reasoning.
  • Wire the AI module into the pipeline and see a full threat report.
Write the Ollama integration module

The AI analyst needs a system prompt that tells the model how to behave, and a connection to Ollama's chat API. You will define these in a new file called ai_analyst.py.

  • Create a new file called ai_analyst.py in your project folder and paste the following code:
import json
import httpx

# Ollama runs locally on this address
OLLAMA_BASE_URL = "http://localhost:11434"
MODEL = "llama3.2"

# System prompt instructs the model to behave as a malware analyst
SYSTEM_PROMPT = """You are a senior malware analyst performing automated threat triage.
You will receive static analysis results for a submitted file.
Your job is to:
1. Analyze the combination of indicators (entropy, strings, PE imports, YARA matches)
2. Determine a verdict: CLEAN, SUSPICIOUS, or MALICIOUS
3. Assign a confidence score from 0 to 100
4. Explain your reasoning by connecting multiple indicators together
5. Map findings to MITRE ATT&CK technique IDs where applicable

Respond ONLY with valid JSON matching this schema:
{
  "verdict": "CLEAN | SUSPICIOUS | MALICIOUS",
  "confidence": <integer 0-100>,
  "reasoning": "<2-3 sentences explaining why the indicators together suggest this verdict>",
  "mitre_techniques": [
    {"id": "<technique ID like T1055>", "name": "<technique name>", "evidence": "<what indicator maps to this>"}
  ],
  "risk_factors": ["<short risk factor>"],
  "recommendations": ["<short recommendation>"]
}"""

What does this code do?

The SYSTEM_PROMPT is the personality of your AI analyst. It tells the model exactly what role to play, what inputs to expect, and what shape the output must take.

By including the JSON schema directly in the prompt, you ground the model's reasoning. The format field you will add later enforces the structure at the API level, while the prompt explains the semantics.

  • Save ai_analyst.py.
  • Verify the module loads by running this command:
python -c "from ai_analyst import SYSTEM_PROMPT, MODEL; print(f'Model: {MODEL}'); print(f'Prompt length: {len(SYSTEM_PROMPT)} chars')"

You should see Model: llama3.2 and a prompt length around 590 characters.

Seeing a ModuleNotFoundError?

Make sure your terminal is in the project folder (where ai_analyst.py lives) and your virtual environment is active.

If you see an error about httpx, confirm you installed it earlier with pip install -r requirements.txt.

help me fix my ai_analyst.py import error

Design the prompt builder function

The AI model cannot read Python dictionaries directly. You need a function that formats all analysis results into a structured text prompt the model can reason over.

  • In ai_analyst.py, add the first half of the build_analysis_prompt function below the SYSTEM_PROMPT string:
def build_analysis_prompt(analysis: dict, yara_matches: list[dict]) -> str:
    # Assemble all indicators into a text report for the AI
    sections = []

    sections.append(f"FILENAME: {analysis['filename']}")
    sections.append(f"FILE SIZE: {analysis['file_size']} bytes")
    sections.append(f"OVERALL ENTROPY: {analysis['entropy']} (max 8.0; >7.0 suggests packing/encryption)")

    # Format string analysis results
    strings = analysis.get("strings", {})
    sections.append(f"\nSTRING ANALYSIS:")
    sections.append(f"  Total strings found: {strings.get('total_count', 0)}")
    if strings.get("urls"):
        sections.append(f"  URLs: {strings['urls'][:5]}")
    if strings.get("ip_addresses"):
        sections.append(f"  IP addresses: {strings['ip_addresses'][:5]}")
    if strings.get("registry_keys"):
        sections.append(f"  Registry keys: {strings['registry_keys'][:5]}")
    if strings.get("suspicious_keywords"):
        sections.append(f"  Suspicious keywords: {strings['suspicious_keywords'][:10]}")

What does this code do?

This function takes the raw analysis dictionary and formats it into labeled text sections. The model sees headings like FILENAME: and STRING ANALYSIS: that help it parse the data.

Each indicator category (entropy, strings, PE, YARA) gets its own clearly separated section so the model can reason about each signal independently and then connect them.

  • Still in ai_analyst.py, add the PE analysis and YARA sections directly below (continuing the same function):
    # Format PE header data if available
    pe = analysis.get("pe_analysis")
    if pe:
        sections.append(f"\nPE ANALYSIS:")
        sections.append(f"  Machine type: {pe['machine_type']}")
        sections.append(f"  Sections: {pe['number_of_sections']}")
        for sec in pe.get("sections", []):
            sections.append(f"    {sec['name']}: entropy={sec['entropy']}, raw_size={sec['raw_size']}")
        if pe.get("suspicious_imports"):
            sections.append(f"  SUSPICIOUS IMPORTS: {pe['suspicious_imports']}")
    else:
        sections.append("\nPE ANALYSIS: Not a PE file or parsing failed")

    # Format YARA signature matches
    if yara_matches:
        sections.append(f"\nYARA MATCHES ({len(yara_matches)} rules triggered):")
        for match in yara_matches:
            meta = match.get("meta", {})
            sections.append(f"  Rule: {match['rule']} | Severity: {meta.get('severity', 'unknown')} | {meta.get('description', '')}")
    else:
        sections.append("\nYARA MATCHES: None")

    return "\n".join(sections)

What does this code do?

The PE section shows the model which imports are dangerous (e.g., VirtualAllocEx, CreateRemoteThread) and what per-section entropy looks like.

The YARA section lists every rule that fired along with its severity. This gives the model pre-computed expert judgments to factor into its reasoning. Finally, everything joins into a single newline-separated string.

  • Save ai_analyst.py.
  • Test the prompt builder against your test sample by running:
python -c "from analyzer import analyze_file; from scanner import scan_file; from ai_analyst import build_analysis_prompt; a = analyze_file('test_sample.exe'); y = scan_file('test_sample.exe'); print(build_analysis_prompt(a, y)[:300])"

You should see formatted text starting with FILENAME: test_sample.exe followed by entropy, string analysis, and YARA match sections.

Seeing a NameError or empty output?

Check that the build_analysis_prompt function is properly indented and that the return statement aligns with the def keyword (one indent level in).

my build_analysis_prompt function is not returning the expected formatted text

Wire the AI module into the pipeline

The prompt builder formats data for the model. Now you need the function that actually sends this prompt to Ollama and gets a structured threat report back.

  • In ai_analyst.py, add the get_threat_report function below build_analysis_prompt. This first block defines the function and builds the API payload:
def get_threat_report(analysis: dict, yara_matches: list[dict]) -> dict:
    # Build the formatted prompt from analysis indicators
    user_content = build_analysis_prompt(analysis, yara_matches)

    # Construct the Ollama API request with JSON schema enforcement
    payload = {
        "model": MODEL,
        "messages": [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_content},
        ],
        "stream": False,
        "format": {
            "type": "object",
            "properties": {
                "verdict": {"type": "string", "enum": ["CLEAN", "SUSPICIOUS", "MALICIOUS"]},
                "confidence": {"type": "integer"},
                "reasoning": {"type": "string"},
                "mitre_techniques": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "id": {"type": "string"},
                            "name": {"type": "string"},
                            "evidence": {"type": "string"},
                        },
                        "required": ["id", "name", "evidence"],
                    },
                },

What is the format field doing?

The format field passes a JSON schema directly to Ollama. The model is forced to generate output that conforms to this schema exactly. This is called structured outputs.

Without it, the model might return free-form text or malformed JSON. The schema guarantees you always get a parseable report with a verdict, confidence score, and technique mappings.

  • Continue the get_threat_report function by adding the rest of the payload and the HTTP call directly below:
                "risk_factors": {"type": "array", "items": {"type": "string"}},
                "recommendations": {"type": "array", "items": {"type": "string"}},
            },
            "required": ["verdict", "confidence", "reasoning", "mitre_techniques", "risk_factors", "recommendations"],
        },
        "options": {"temperature": 0.1},
    }

    # Send to Ollama and parse the structured JSON response
    with httpx.Client(timeout=120.0) as client:
        response = client.post(f"{OLLAMA_BASE_URL}/api/chat", json=payload)
        response.raise_for_status()

    result = response.json()
    content = result["message"]["content"]
    report = json.loads(content)
    return report

What does the HTTP call do?

The httpx.Client sends a POST request to Ollama's /api/chat endpoint with the system and user messages. The 120-second timeout gives the model time to generate a complete analysis.

Ollama returns the structured JSON as a string inside result["message"]["content"]. The final json.loads() call parses that string into a Python dictionary you can work with.

  • Save ai_analyst.py.

Now run the full pipeline against your test sample. Before you run this, what verdict do you think the AI will give for a file with process injection APIs, UPX packing strings, and multiple YARA rule hits?

  • Run the complete analysis pipeline by running:
python -c "import json; from analyzer import analyze_file; from scanner import scan_file; from ai_analyst import get_threat_report; a = analyze_file('test_sample.exe'); y = scan_file('test_sample.exe'); r = get_threat_report(a, y); print(json.dumps(r, indent=2))"

This command takes 30-90 seconds depending on your hardware. You should see a structured JSON threat report with a verdict (likely MALICIOUS or SUSPICIOUS), a confidence score, reasoning text explaining why the indicators are dangerous, and at least one MITRE ATT&CK technique ID like T1055.

Why does the AI see multiple signals as worse than individual ones?

A single suspicious import might be legitimate. But high entropy plus UPX packing strings plus process injection APIs plus network callbacks is a combination that almost never appears in benign software. The AI reasons about this combination, not each indicator in isolation.

This is the same reasoning pattern human SOC analysts use during triage. Your pipeline automates it.

Seeing a connection error or timeout?

Make sure Ollama is running. Check by opening a new terminal and running ollama list. If it shows llama3.2 in the output, Ollama is ready.

If you see a timeout, try increasing the timeout value or ensure no other process is consuming your GPU/CPU. The first run after pulling the model can take longer as it loads into memory.

help me fix my Ollama connection issue when running the threat analysis pipeline

✔️ Awesome, I've got everything!

Great. Double check you have saved ai_analyst.py and confirmed the full pipeline prints a JSON threat report with a verdict and MITRE technique IDs.

ⓧ I'd like to double check the full code

Your ai_analyst.py should contain these sections in order. Scroll up and confirm each is present:

  • Imports (json and httpx).
  • OLLAMA_BASE_URL and MODEL constants.
  • SYSTEM_PROMPT triple-quoted string with the full analyst instructions and JSON schema example.
  • build_analysis_prompt function that formats filename, entropy, strings, PE data, and YARA matches into a text string.
  • get_threat_report function that builds the payload with the format JSON schema, sends it to /api/chat, and returns the parsed report dict.

Your pipeline now goes from raw file bytes all the way to an AI-generated threat intelligence report. Next up, you will wrap it in a proper API so anyone can submit files for analysis.

Package as a FastAPI Service

You have a complete analysis pipeline that extracts entropy, strings, PE headers, YARA matches, and AI-generated threat intelligence. But right now it only runs as a script.

A script is a demo. An API is a product. Wrapping the pipeline in FastAPI turns it into something you can integrate into a SOC workflow, trigger from a SIEM, or show running in a portfolio demo.

In this step, get ready to:
  • Create a FastAPI app with a health check endpoint.
  • Add a POST endpoint that accepts file uploads and runs the full analysis pipeline.
  • Test the service through FastAPI's built-in Swagger UI.
Create the FastAPI app with a health check

Every production API starts with two things: an app instance that defines what the service is, and a health check endpoint that monitoring tools can ping.

  • Create a new file called main.py in your project folder.
  • Add the following code to set up the app and health endpoint:
import tempfile
import shutil
from pathlib import Path

from fastapi import FastAPI, UploadFile, HTTPException
from fastapi.responses import JSONResponse

# Import your analysis modules
from analyzer import analyze_file
from scanner import scan_file
from ai_analyst import get_threat_report

# Create the API app instance
app = FastAPI(
    title="Malware Triage Sandbox",
    description="AI-powered static malware analysis with local LLM threat reasoning",
    version="1.0.0",
)


@app.get("/health")
def health_check():
    return {"status": "operational", "service": "malware-triage-sandbox"}

What does this code do?

  • The imports bring in FastAPI for the web framework, plus your three analysis modules from earlier steps.
  • app = FastAPI(...) creates the API instance with metadata that appears in the auto-generated docs.
  • The /health endpoint returns a simple JSON object. Monitoring tools and load balancers use this to confirm the service is alive.
  • Save main.py.
  • Start the development server by running:
fastapi dev main.py

You should see output showing the server started on http://127.0.0.1:8000.

  • Open a new terminal tab (keep the server running in the first one).
  • Test the health endpoint by running:
curl http://127.0.0.1:8000/health

You should see {"status":"operational","service":"malware-triage-sandbox"} printed in your terminal.

Server won't start?

  • Make sure your virtual environment is active. You should see the environment name in your terminal prompt.
  • Check that main.py is saved in the same folder as analyzer.py and scanner.py.
  • If you see a module not found error, confirm you installed all dependencies from requirements.txt in Step 1.

Why won't my FastAPI server start?

Add the file upload and analysis endpoint

The /analyze endpoint is where all your earlier work comes together. It accepts a file upload, saves it to a temporary directory, runs the full pipeline, and returns a structured threat report.

  • Stop the running server with Ctrl+C in the terminal where it's running.
  • In main.py, add the following code below the health_check function:
@app.post("/analyze")
async def analyze(file: UploadFile):
    # Reject requests without a filename
    if not file.filename:
        raise HTTPException(status_code=400, detail="No filename provided")

    # Save uploaded file to a temporary directory for analysis
    tmp_dir = Path(tempfile.mkdtemp(prefix="triage_"))
    tmp_path = tmp_dir / file.filename

    try:
        with open(tmp_path, "wb") as f:
            content = await file.read()
            f.write(content)

        # Run all three analysis stages
        analysis = analyze_file(str(tmp_path))
        yara_matches = scan_file(str(tmp_path))
        threat_report = get_threat_report(analysis, yara_matches)

What does this section do?

  • UploadFile is FastAPI's type for handling multipart file uploads. It streams the file content rather than loading it all into memory at once.
  • tempfile.mkdtemp(prefix="triage_") creates a unique temporary folder. This isolates each upload from other concurrent requests.
  • The three pipeline calls run your static analyzer, YARA scanner, and AI analyst in sequence against the uploaded file.
  • Directly below the code you just added (still inside the try block), add the response structure and error handling:
        # Build the structured response with all analysis layers
        return JSONResponse(content={
            "file_info": {
                "filename": analysis["filename"],
                "file_size": analysis["file_size"],
                "entropy": analysis["entropy"],
            },
            "static_analysis": {
                "strings_summary": {
                    "total": analysis["strings"]["total_count"],
                    "urls_found": len(analysis["strings"]["urls"]),
                    "ips_found": len(analysis["strings"]["ip_addresses"]),
                },
                "pe_analysis": analysis["pe_analysis"],
            },
            "yara_matches": yara_matches,
            "threat_report": threat_report,
        })

    except Exception as e:
        raise HTTPException(status_code=500, detail=f"Analysis failed: {str(e)}")

    finally:
        # Always clean up the temporary file
        shutil.rmtree(tmp_dir, ignore_errors=True)

What does this section do?

  • The JSONResponse bundles file metadata, static analysis results, YARA matches, and the AI threat report into a single structured JSON payload.
  • The except block catches any pipeline failure and returns a clean 500 error instead of crashing.
  • The finally block deletes the temporary directory whether the analysis succeeded or failed. This prevents uploaded files from accumulating on disk.
  • Save main.py.

Indentation errors?

  • The return JSONResponse(...) statement must be indented at the same level as analysis = analyze_file(...) (inside the try block, 8 spaces deep).
  • The except and finally keywords must align with try (4 spaces deep).

Help me fix indentation in my analyze endpoint.

✔️ Awesome, I've got everything!

Great. Double check you saved main.py before moving on.

ⓧ I'd like to double check the full code

Your main.py should contain these sections in order. Scroll up and confirm each is present:

  • Imports (tempfile, shutil, Path, FastAPI classes, your three analysis module imports).
  • app = FastAPI(...) with title, description, and version.
  • @app.get("/health") function returning status and service name.
  • @app.post("/analyze") async function with file validation, temp directory creation, try/except/finally structure.
  • Inside the try: file write, three pipeline calls, JSONResponse with file_info, static_analysis, yara_matches, threat_report.
  • except returning HTTPException 500, finally calling shutil.rmtree.
Test the full pipeline through Swagger UI

FastAPI auto-generates interactive API documentation. You can upload a file and see the response without writing any client code.

  • Restart the server by running:
fastapi dev main.py

Before you test, what do you think will happen when you upload test_sample.exe? Will the AI verdict be CLEAN, SUSPICIOUS, or MALICIOUS given all the indicators you built into that file?

  • Open your browser and navigate to http://127.0.0.1:8000/docs.
  • Click on the POST /analyze section to expand it.
  • Click Try it out.
  • Click Choose File and select the test_sample.exe file from your project folder.
  • Click Execute.

What should I see?

The response takes 15-60 seconds because Ollama is generating the AI analysis. When it completes, you will see a full JSON response in the browser with four sections:

  • file_info showing the filename, size, and entropy value.
  • static_analysis with string counts and PE data.
  • yara_matches listing the rules that fired.
  • threat_report with the AI verdict (likely MALICIOUS or SUSPICIOUS), a confidence score, reasoning text, and MITRE ATT&CK technique IDs.

Getting a 500 error or timeout?

  • Make sure Ollama is still running. Open a new terminal and run ollama list to confirm llama3.2 is available.
  • If you see a connection refused error, Ollama may have stopped. Restart it by running ollama serve in a separate terminal.
  • If the request times out, your machine may need more time for inference. Wait up to 2 minutes on the first request while the model loads into memory.

Help me debug my /analyze endpoint returning errors.

Why FastAPI over Flask?

FastAPI is the dominant choice in modern Python security tooling. Its auto-generated Swagger UI gives you instant demo-ability without writing a frontend.

The fastapi[standard] install includes the server (uvicorn), file upload support (python-multipart), and an HTTP client (httpx) in one package.

Your malware triage pipeline is now a running API service. Anyone on your network could submit a file and get back a structured threat intelligence report. In the Secret Mission, you will containerize the entire pipeline in Docker so the suspicious file never touches your host filesystem.

Secret mission

Dockerize the Analysis Pipeline

In a real security operation, you never let a suspicious file touch your host filesystem. Can you containerize the entire triage pipeline so every analysis runs in a disposable, ephemeral environment?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. The only resources using disk space are the Ollama model (~2GB) and your project files.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Ollama llama3.2 model (~2GB stored locally).
  • Project folder on your Desktop (Python files, virtual environment, test samples).
  • Docker image malware-triage (if you completed the Secret Mission).

Keep everything running

No action needed. Choose this if you plan to keep experimenting with the malware triage pipeline or want Ollama available for future projects.

  • The llama3.2 model stays cached on disk. It does not consume CPU or RAM when idle.
  • Ollama runs in the background via the system tray but uses negligible resources when no model is actively loaded.
  • Your project folder and Docker image remain on disk for future use.

Pause - I'll come back to this later

Shut down running processes to free up memory, but keep your files intact so you can pick up where you left off.

  • Stop any running Docker container by pressing Ctrl+C in the terminal where it is running.
  • Right-click the Ollama icon in the system tray and select Quit Ollama.
  • Your project files and the cached model remain on disk. Relaunch Ollama from the Start menu when you are ready to continue.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each step below to reclaim disk space.

  • Remove the Ollama model by running this command in your terminal:
ollama rm llama3.2
  • Uninstall Ollama from Windows. Open Settings, select Apps, then Installed apps. Find Ollama in the list and click Uninstall.
  • Delete your project folder by running:
cd ~\Desktop
Remove-Item -Recurse -Force malware-triage-sandbox
  • If you completed the Secret Mission, remove the Docker image by running:
docker rmi malware-triage
  • Deactivate and remove any leftover Python virtual environment by closing the terminal where it was active.

Nice Work!

Nice Work!

You just built a production-grade malware triage service powered by a local AI analyst. Upload a suspicious file, and your system runs it through multi-layer static analysis before handing the raw indicators to an LLM that produces structured threat intelligence.

You've learned how to:

  • Build a multi-layer static analysis engine that extracts Shannon entropy, suspicious strings, PE headers, and YARA signature matches from any uploaded file.
  • Connect a local LLM (Ollama + llama3.2) as an AI threat analyst that reasons about indicator combinations and produces structured reports with MITRE ATT&CK technique mapping.
  • Package the entire pipeline as a FastAPI service with a clean REST API that accepts file uploads and returns JSON threat reports.
  • Secret Mission: Containerized the analysis pipeline in Docker for disposable, host-isolated analysis where uploaded files never touch your filesystem.

Ready to quiz yourself?