Play a Game Inside AI Model Weights

Run an NES game from an AI checkpoint and verify it against a reference CPU.

Introduction

30 Second Summary

A game usually runs on a console or computer you can point to. When the machine exists only as numbers inside an AI file, every button press becomes a claim worth testing.

In this project, you will turn scripted button presses into an animated GIF by running a Nintendo game on a CPU stored inside a language model checkpoint. You will prove the weights behave like a real chip by checking them against a reference CPU.

What You'll Build

Your terminal fills with game frames one token at a time before the finished GIF plays back the moves you scripted.

By the end of this project, you'll have:

An animated game GIF that plays back the button presses you chose.
A readable CPU trace that shows the checkpoint's token stream as an execution log.
A saved verification result that shows the model weights running beside a reference CPU.
Secret Mission: Turn your saved GIF into a frame-by-frame contact sheet for inspecting the model's playthrough.

Are there any prerequisites?

You need Python 3.11 or newer. Live frame rendering requires a 24-bit colour terminal such as iTerm2 or Windows Terminal.

Before We Start

NESformer needs its source files before the model checkpoint can behave like a CPU. The bundled ROMs provide the programs that the checkpoint runs.

A Python virtual environment keeps the four required packages inside this project. A checksum test then proves that every extracted weight file matches its expected contents.

In this step, get ready to:

Download the NESformer project with its bundled ROMs.
Create an isolated Python environment for the four required packages.
Extract the checkpoint weights with verified checksums.

Check your Python version

NESformer requires Python 3.11 or newer. Checking the version now prevents package compatibility problems during setup.

Press Cmd+Space on macOS. Press the Windows key on Windows or Linux.
Type Terminal on macOS or Linux. Type Windows Terminal on Windows.
Press Enter to open the terminal.

macOS

Check your installed Python version by running this command:

python3 --version

What does this command check?

The --version option prints the installed Python version. Use its first two numbers to choose the matching result tab below.

Windows

Check your installed Python version by running this command:

python --version

What does this command check?

The --version option prints the installed Python version. Use its first two numbers to choose the matching result tab below.

Linux

Check your installed Python version by running this command:

python3 --version

What does this command check?

The --version option prints the installed Python version. Use its first two numbers to choose the matching result tab below.

Choose the tab that matches the version shown in your terminal.

✔️ I see version 3.11 or higher

Your Python installation meets NESformer's minimum version. That clears the first setup check.

ⓧ I see an older version

Visit the official Python downloads page.
Download Python 3.11 or newer for your operating system.
Run the downloaded installer.
Close the current terminal after the installation finishes.
Open a fresh terminal through your operating system search.
Rerun the version command from your platform tab.

ⓧ Command not found

Visit the official Python downloads page.
Download Python 3.11 or newer for your operating system.
Run the downloaded installer.
Close the current terminal after the installation finishes.
Open a fresh terminal through your operating system search.
Rerun the version command from your platform tab.

Download the project and install its packages

The GitHub repository contains the NESformer package. Its roms folder contains three freely licensed test ROMs.

Open the NESformer repository in your browser.
Click Code above the repository files.
Click Download ZIP to save the project archive.

The downloaded archive keeps the package files with the bundled ROMs. Extracting it creates the local workspace you use for every command in this project.

macOS

Locate nesformer-main.zip in your Downloads folder using Finder.
Double-click nesformer-main.zip to extract it.

You should see an extracted nesformer-main folder beside the archive.

Rename the extracted folder to nesformer.
Move the nesformer folder onto your Desktop.

Windows

Locate nesformer-main.zip in your Downloads folder using File Explorer.
Right-click nesformer-main.zip.
Choose the archive extraction option.
Set your Desktop as the extraction destination.
Start the extraction.

You should see an extracted nesformer-main folder on your Desktop.

Rename the extracted folder to nesformer.

Linux

Locate nesformer-main.zip in your Downloads folder using your file manager.
Use the file manager's archive action to extract nesformer-main.zip.

You should see an extracted nesformer-main folder beside the archive.

Rename the extracted folder to nesformer.
Move the nesformer folder onto your Desktop.

Switch back to the terminal from your Python version check.
Enter the downloaded workspace by using the tab for your platform.

macOS

Move into the Desktop nesformer folder by running these commands:

cd ~/Desktop/nesformer
ls

What do these commands show?

The first command moves your terminal into the downloaded nesformer folder.
The second command lists the repository contents.

Windows

Move into the Desktop nesformer folder by running these commands:

cd ~/Desktop/nesformer
ls

What do these commands show?

The first command moves your terminal into the downloaded nesformer folder.
The second command lists the repository contents.

Linux

Move into the Desktop nesformer folder by running these commands:

cd ~/Desktop/nesformer
ls

What do these commands show?

The first command moves your terminal into the downloaded nesformer folder.
The second command lists the repository contents.

You should see nesformer, roms, pyproject.toml, plus nesformer-weights.zip in the listing.

Can't find the repository files?

Confirm that the Desktop folder is named nesformer. Check that pyproject.toml sits directly inside it.

If you see another nesformer-main folder inside your workspace, move its contents into the outer nesformer folder.

Ask for help to find my NESformer workspace.

The project uses four packages inside the virtual environment. Each package supports one part of the local model runtime.

What are the four packages for?

NumPy provides the arrays used by the model engine.
SciPy provides scientific computing operations.
Numba compiles the CPU engine for faster local execution.
Pillow creates the image frames used in the animated GIF.

macOS

Create the .venv environment by running the first command below. Activate it by running the second command:

python3 -m venv .venv
source .venv/bin/activate

What do these commands do?

The first command creates an isolated Python environment inside .venv.
The second command makes that environment active in the current terminal.

Windows

Create the .venv environment by running the first command below. Activate it by running the second command:

python3 -m venv .venv
.venv\Scripts\activate

What do these commands do?

The first command creates an isolated Python environment inside .venv.
The second command makes that environment active in the current terminal.

Linux

Create the .venv environment by running the first command below. Activate it by running the second command:

python3 -m venv .venv
source .venv/bin/activate

What do these commands do?

The first command creates an isolated Python environment inside .venv.
The second command makes that environment active in the current terminal.

Your terminal prompt should now begin with (.venv). This confirms that the project environment is active.

Install the four required packages into the active environment by running this command:

pip install numpy scipy numba pillow

What does this install?

The command installs NumPy, SciPy, Numba, plus Pillow inside .venv. Your system-wide Python packages remain separate.

You should see installation messages before the prompt returns. Your isolated environment now contains all four required packages.

Package installation failed?

Check that (.venv) appears at the start of your prompt. Rerun the activation command for your platform if it is missing.

Confirm that the terminal is still inside the Desktop nesformer folder. The .venv folder should be listed there.

Ask for help to debug my package installation.

Unzip the weights and verify every checksum

The checkpoint archive expands into a weights folder beside the Python package. Its SHA-256 checksum file records the expected fingerprint of each checkpoint file.

macOS

Switch to Finder.
Open the Desktop nesformer folder.
Double-click nesformer-weights.zip to extract it.

You should see a weights folder directly inside the Desktop nesformer folder.

Windows

Switch to File Explorer.
Open the Desktop nesformer folder.
Right-click nesformer-weights.zip.
Choose the archive extraction option.
Set the Desktop nesformer folder as the destination.
Start the extraction.

You should see a weights folder directly inside the Desktop nesformer folder.

Linux

Switch to your file manager.
Open the Desktop nesformer folder.
Use the file manager's archive action to extract nesformer-weights.zip into the current folder.

You should see a weights folder directly inside the Desktop nesformer folder.

Before you run the integrity check, do you expect every extracted file to match its recorded checksum?

Switch back to the active project terminal.
Check every extracted weight file by running this command:

shasum -a 256 -c weights/SHA256SUMS

What does this command prove?

The command calculates a SHA-256 fingerprint for each extracted file. It compares that fingerprint with the value recorded in weights/SHA256SUMS.

A matching result proves that the file arrived with the expected contents.

You should see every checked file end with OK. That is the setup complete: the project workspace, bundled ROMs, virtual environment, four packages, plus checkpoint weights are ready.

Seeing a failed checksum?

Confirm that weights sits directly inside the Desktop nesformer folder. It must sit beside the nesformer Python package.

Remove the extracted weights folder if any file reports a mismatch. Extract nesformer-weights.zip again before rerunning the check.

Ask for help to fix my checksum results.

Your local NESformer workspace is verified. Next up, you will inspect the checkpoint to see where each part of the machine lives.

Inspect the Checkpoint

Every checksum reported OK. That gives you a trustworthy model checkpoint to inspect.

Nesformer provides an info summary of the checkpoint's model shape. Its weights/nesformer.json map shows the Transformer layer assignments plus the attention head assignments behind the emulated machine.

In this step, get ready to:

Run the info command to read the checkpoint description.
Inspect weights/nesformer.json to locate the layer assignments plus the attention head assignments.

Read the checkpoint description

The info subcommand turns checkpoint metadata into a compact JSON description. This gives you a readable view of the model before you trace its execution.

Run the checkpoint description by entering this command:

python -m nesformer info

What Does This Output Describe?

The machine entry names the machine compiled into the checkpoint.
The architecture entry identifies the model class.
The hidden_size entry reports the model's internal width.
The layers entry lists the checkpoint's layer types.
The vocab entry reports the token vocabulary size.
The circuit entry describes the compiled circuit metadata.

Read the printed checkpoint description from top to bottom.
Confirm that circuit appears as the final top-level section.

You should see a formatted object ending with the circuit section. Good, the checkpoint can now describe its own structure.

Info Command Not Working?

Confirm that your terminal still points to the top nesformer folder.
Confirm that your Python virtual environment remains activated.
Check that weights/nesformer.json exists inside the repository.
Check that weights/config.json exists inside the repository.

Help me diagnose the checkpoint info command.

Inspect the machine mapping

The summary tells you the checkpoint's shape. The mapping file reveals where each part of the emulated machine is stored.

Each mapping connects a machine component to a layer plus an attention head. This turns the model structure into a hardware map you can inspect.

Before you reveal the mapping, do you expect the machine's state to sit in one layer or be spread across several layers?

Pretty-print the mapping file in your terminal by running this command:

python -m json.tool weights/nesformer.json

What Does This Command Do?

Python's json.tool module checks that the file contains valid JSON. It also formats nested objects so the assignments are easier to follow.

Scroll through the formatted weights/nesformer.json output.
Identify one machine part with a model layer assignment.
Identify one machine part with an attention head assignment.

You should see formatted JSON entries that associate machine parts with model layers plus attention heads. You can now point to where the emulated machine lives inside the checkpoint.

Mapping File Not Displaying?

Check that the path starts with the weights/ folder inside your repository.
Confirm that the filename is exactly nesformer.json.
Return to the repository's top folder before trying the command again.

Help me inspect the mapping file.

You have identified the structure behind the model's emulated machine. Next, you will feed a bundled ROM into the checkpoint. You will watch its CPU execution log emerge as tokens.

Trace CPU Instructions

Your checkpoint map showed where NESformer stores each part of its virtual machine. The next proof comes from watching those parts execute a program.

A CPU trace records the processor's work as it moves through instructions. NESformer exposes that execution as a generated token stream from the checkpoint.

In this step, get ready to:

Run the checkpoint against the bundled Full Game ROM.
Interpret the first 40 generated CPU tokens.

Understand the token records

Each processor instruction produces a record containing one or more tokens. The final token commits the instruction's result to the machine state.

How does a trace represent a CPU?

Each 6502 record captures a state change such as a register update. Other records capture a memory access or a jump.

The instruction length appears in the final token. NESformer uses those lengths to keep track of the program counter between jumps.

Generate the CPU trace

The first model run may take about 20 seconds because Numba compiles the engine. A quiet terminal during that pause is expected.

Before you run it, predict whether the checkpoint will produce game graphics or CPU state records.

Generate 40 tokens from roms/full_game.nes by running this command:

python -m nesformer trace roms/full_game.nes --tokens 40

What does this command do?

The trace subcommand loads the ROM into the checkpoint's machine state.
The roms/full_game.nes path selects a freely licensed bundled game.
The --tokens 40 option stops generation after 40 CPU tokens.

You should see a line containing 40 named tokens. Many tokens include processor values inside parentheses.

How do these tokens describe execution?

A token such as P(000100 px0 L1) commits processor flags for a one-byte instruction. A token such as X($40 c0 v0 px0 L2) captures an update to the X register for a two-byte instruction.

A token such as J($8017) changes the next execution address. Together these records form the CPU's execution log.

Scan the output for tokens containing register values or processor flags.
Locate a token containing an instruction length such as L1 or L2.

You have now watched the checkpoint generate a real CPU execution log one token at a time.

No token stream?

Return to the repository terminal from earlier.
Confirm the terminal prompt still shows the active virtual environment.
Confirm the weights/ folder remains beside the nesformer/ package.

Ask for help with the exact output you received: help me troubleshoot my NESformer trace command

The weights have proved they can execute instructions. Next, you will turn that token-by-token computation into visible game frames.

Render Your First Game GIF

Your earlier trace proved that the checkpoint generates a CPU execution stream. The stream still hides the game behind low-level instructions.

Nesformer converts that execution into visible game frames. This short run saves those frames as your first animated GIF.

In this step, get ready to:

Run a bundled ROM with terminal rendering enabled.
Feed the initial button schedule into the model.
Confirm an animated gameplay GIF is saved.

Render the bundled game

The --show option draws each generated frame in your terminal. The --frames option stops the run after a fixed number of frames.

The --buttons script schedules controller input by frame number. This first run uses the project's initial input schedule.

Plan for the first render

The --show option needs 24-bit color. iTerm2 can display this mode. Windows Terminal can also display it.

The first run spends about 20 seconds compiling the engine with Numba. A quiet terminal during that compile is expected.

Each frame then takes 1 to 6 seconds. This 120-frame run can take roughly 2 to 12 minutes after compilation.

Before you run this, picture the order of events. Do you expect the GIF to exist before the frames draw or after the final frame?

Start the bundled roms/thwaite.nes ROM with its initial input schedule by running this command:

python -m nesformer run roms/thwaite.nes --frames 120 --show --buttons "start@90-95+200-205,a@230-233+250-253+270-273+290-293+310-313+330-333+350-353+370-373+390-393+410-413+430-433+450-453+470-473+490-493+510-513+530-533+550-553+570-573+590-593"

What does this command do?

The roms/thwaite.nes path selects one of the freely licensed bundled games.
The --frames option limits this run to 120 frames.
The --show option draws each frame as the model generates it.
The --buttons option feeds scripted controller input into the game.
Only start@90-95 falls inside this short run. The later ranges remain in the initial script for longer runs.

Let the command continue until the terminal prompt returns.

You'll see each generated frame draw in your terminal. When the run ends, the CLI saves thwaite.gif in the current project folder.

Frames not rendering correctly?

Switch to a terminal with 24-bit color support if the frame colors look wrong.
Confirm the Python virtual environment from earlier is still active if the run stops before rendering.
Confirm the terminal remains inside the project repository from earlier if the weights cannot be located.

Help me troubleshoot this render.

Confirm the animated GIF

The CLI names the saved GIF after its ROM. Running thwaite.nes creates thwaite.gif in the project folder.

Before you check, do you expect the saved name to match the ROM or the button schedule?

Confirm the GIF exists by running this command:

ls thwaite.gif

You'll see thwaite.gif listed. You have your first replayable result from the model's weights.

GIF not found?

Review the previous terminal output if ls cannot find the file.
Confirm the rendering run reached its final frame.
Run the rendering command again if the earlier process stopped before completion.

Help me find my saved GIF.

Your first model-generated game clip is saved. Next, you'll rewrite its button schedule to create a different run.

Script Your Own Inputs

Your first GIF proved that the nesformer checkpoint can turn generated CPU tokens into a moving game. That run followed one fixed controller script.

Now you will schedule your own button presses. A second GIF will show how the same weights run the ROM under a different input sequence.

In this step, get ready to:

Map your own controller inputs to frame ranges.
Run the same bundled ROM with your custom controller script.
Compare the custom gameplay GIF with your first GIF.

Design your controller script

The --buttons option maps controller states to frame ranges. The host presents those states to the CPU at exact moments.

How do controller scripts work?

Each entry names one or more buttons. A + joins buttons that should be held together.
An @ separates the button names from their frame ranges.
A comma separates distinct controller entries.
The example start@60-65 presses Start during frames 60 through 65.
The example a@250-265+300-315 schedules two A presses.
The example right+b@200-600 holds Right with B.

Frames count from power-on at 60 per second. Keep every range inside the short frame limit from your first run.

Choose a frame span where your first GIF showed active play.
Choose a different input from a, b, select, start, up, down, left, or right.
Assign your chosen input a frame range using @.
Add another press with + if you want the same button used more than once.
Separate different button entries with a comma.
Record your finished script here: your comma-separated controller script.

Run the ROM with your inputs

Each generated frame can take one to six seconds. A slowly updating terminal means the model is still computing your game state.

Switch back to the project terminal from earlier.
Press the Up Arrow key until the command that created your first GIF appears.
Replace the quoted value after --buttons with your comma-separated controller script.
Add --gif with the value custom-gameplay.gif at the end of the command.

Before you press Enter, do you expect the first changed frame to appear before or after your earliest scripted range?

Press Enter to run the edited command.

You'll see each frame draw in the terminal. When the run finishes, you'll have custom-gameplay.gif in the project repository.

You have now turned your controller plan into a replay generated by the checkpoint.

Custom inputs not showing?

Check that one @ separates the buttons from the ranges in every entry.
Check that every range finishes before the value passed to --frames.
Use the same ROM from your first run. This keeps the button script as the meaningful change.

help me debug my nesformer controller script.

Compare both gameplay GIFs

Before replaying both GIFs, which scripted range do you expect to cause the first visible difference?

Press Cmd+Space on macOS or the Windows key on Windows to open search.
Type Finder on macOS or File Explorer on Windows.
Press Enter to open the file manager.
Return to the nesformer folder from earlier.
Double-click your first gameplay GIF.
Watch one complete animation loop.
Double-click custom-gameplay.gif.
Watch one complete animation loop.

You should see the same game begin from the same initial state. The motion should diverge around your custom frame ranges.

Your second GIF proves the weights respond to controller input. The distinct run came from your script.

Your script now drives a distinct run. Next, you will compare the weights with the reference CPU to test whether their execution stays aligned.

Check Against the Reference CPU

Your first GIF showed the checkpoint running a game. Your second GIF showed that your controller inputs changed the gameplay.

The final test compares those checkpoint weights with a reference CPU. A matching result shows that the model follows the same CPU behavior.

In this step, get ready to:

Run a bundled ROM on the checkpoint weights beside the reference CPU.
Save the comparison output for later review.

Watch for divergence

The checkpoint generates its CPU trace one token at a time. The checker compares each generated token with the reference trace.

Enter the path of the bundled ROM you used for your GIFs here: roms/your-bundled-rom.nes.

The repository's example speed puts this default check at roughly 30 seconds. A slower CPU takes longer.

Before you run the check, do you think the two traces will stay identical for the full comparison?

Compare the two CPU traces while saving every reported result by running this command:

python -m nesformer check [[BUNDLED_ROM_PATH="roms/your-bundled-rom.nes"]] 2>&1 | tee reference-cpu-check.txt

What does this command prove?

The check command lets the checkpoint weights generate their own trace.
The reference CPU produces the comparison trace for the selected ROM.
The checker stops at the first difference.
The default comparison covers up to 50,000 tokens.
The 2>&1 redirection keeps diagnostic output in the saved stream.
The tee command displays the stream while writing it to reference-cpu-check.txt.

When the comparison succeeds, you'll see a final line beginning with OK:. The line reports that the generated tokens are identical to the reference CPU's trace.

The same line should contain frames identical: True. That is the final proof that the weights behaved like the reference CPU during this run.

Check stopped or reported a difference?

Confirm your terminal is still in the project repository from earlier.
Check that the repository folder contains nesformer/.
Check that the repository folder contains weights/.
Confirm that your Python virtual environment remains active.
Rerun the checksum verification from Step 1 if the comparison reports a difference.

help me debug my nesformer reference CPU check

Confirm the saved evidence

The comparison text now lives in reference-cpu-check.txt. Keeping this file gives you a durable record after the terminal session closes.

Before you check, do you expect the saved output file to appear in the repository?

Confirm the saved check output exists by running this command:

ls reference-cpu-check.txt

What does this confirm?

The command lists the saved output file when it exists in your current repository folder. This confirms that the terminal result was retained.

You'll see reference-cpu-check.txt listed in the terminal. Your gameplay GIFs now have a saved CPU comparison beside them.

Output file missing?

Rerun the comparison command above from the project repository. Keep the complete command intact so tee can write the output file.

help me find my saved nesformer check output

You have now played a game inside model weights. You also retained evidence that their generated CPU trace matches the reference implementation.

Secret mission

Build a Four-Layer CPU Evidence Case

Turn your mapping, trace, gameplay GIFs, and reference check into one compact case that the checkpoint behaves like a CPU. You will connect structure, execution, behavior, and verification without changing the project.

Clean Up Your Resources

Everything ran on your laptop with no ongoing cost. Choose whether to keep the workspace, pause your work, or delete the local resources.

Resources you used:

Local project workspace containing the repository, freely licensed bundled ROMs, edited button script, and nesformer.json.
Python virtual environment containing the four required packages.
Unzipped checkpoint weights that passed the checksum verification.

Artifacts to keep:

First animated gameplay GIF generated with the initial button script.
Custom gameplay GIF generated with your edited button script.
Retained output from the reference CPU check.

Keep everything running

No action is needed if you want to keep experimenting with the checkpoint. All resources stay on your laptop without creating an ongoing cost.

Keep the local project workspace for future game runs.
Keep the Python virtual environment for access to the installed packages.
Retain the unzipped checkpoint weights to avoid preparing them again.
Use only the freely licensed bundled ROMs for future experiments.

Pause - I'll come back to this later

Your completed commands do not leave a game or check process running. Closing the terminal pauses your work while every local file remains available.

Close the terminal after the check command has finished.
Leave the project workspace on disk.
Return to the project workspace in a terminal when you are ready to continue.
Reactivate the Python virtual environment from earlier.

Delete - I don't want to use this again

This option removes the local setup so you can start fresh. Your gameplay GIFs and retained check output stay safe when they are stored outside the project workspace.

Use Finder on macOS or File Explorer on Windows to locate the project workspace from earlier.
Copy the first gameplay GIF to a location outside the project workspace.
Copy the custom gameplay GIF to a location outside the project workspace.
Copy the retained reference CPU check output to a location outside the project workspace.
Remove the Python virtual environment folder from the project workspace.
Remove the unzipped checkpoint weight files from the project workspace.
Remove the remaining project workspace if you do not want to use the repository again.

Open the copied first gameplay GIF.
Open the copied custom gameplay GIF.
Open the copied reference CPU check output.

Both gameplay GIFs should animate. The reference CPU check output should remain readable.

That is the heavy cleanup complete. Removing the virtual environment and unzipped weights recovers about 1 GB of disk space.

Nice Work!

You did it. Your laptop ran a Nintendo game on a CPU encoded inside a language model checkpoint.

You've learned how to:

Inspect the checkpoint structure through the info output. Read the transformer mapping in nesformer.json to locate the emulated machine parts.
Watch a CPU execution trace emerge as the model's token stream. This makes the computation stored in the weights visible.
Generate an animated gameplay GIF from scripted controller inputs. Compare your custom run with the first run to see how your inputs changed the game.
Run a side-by-side reference CPU check. Retain the reported result or stopping point as evidence of the comparison.
Secret Mission: Push the model-as-CPU experiment further through an optional project extension.

Ready to quiz yourself?