Quantized Inference Engine in RISC-V

Build an INT8 neural network inference engine in RISC-V assembly.

Introduction

30 Second Summary

Every time an AI model runs on your phone or an edge device, the hardware is doing something deceptively simple: multiplying tiny integers, adding them up, and shifting bits to keep the numbers in range. That's quantized inference, and it happens billions of times per second inside chips you use every day.

In this project, you will build a quantized neural network inference engine entirely in RISC-V assembly. You'll write every multiply, accumulate, and rescale instruction by hand, running it in a browser-based simulator where you can watch registers change in real time.

What You'll Build

You'll feed a 4-element input vector into your engine, watch it flow through two layers of quantized neurons, and see the predicted class printed to your console.

By the end of this project, you'll have:

  • A working INT8 dot product where you can step through each multiply-accumulate in the simulator and watch the 32-bit accumulator register grow with every iteration.
  • A fixed-point rescaling routine that brings oversized accumulator values back into INT8 range using a single arithmetic right-shift instruction, replacing expensive division.
  • A complete 2-layer forward pass that takes raw inputs, runs them through hidden neurons with ReLU activation, and prints a predicted class (0, 1, or 2) to the console. Change the input values and the prediction changes.
  • Secret Mission: Implement saturating arithmetic that clamps accumulator overflow, mimicking what real inference accelerators do in hardware.

Are there any prerequisites?

You should be familiar with how neural network layers work (weighted sums and activations) and have basic programming experience in a language like Python or C++. You'll also need Git installed and a GitHub account.

No prior assembly experience is required. The Venus simulator runs entirely in your browser, so there is nothing else to install.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters. Every AI inference chip executes quantized math at the register level, and you're about to do the same thing by hand.

Set Up Your Workspace and First Assembly Program

You have a clear goal: build a quantized inference engine. But before you can write inference code, you need a place to write and run RISC-V assembly, and you need proof that the simulator handles the multiply instruction your engine depends on.

In this step, you will open the Venus browser simulator, write a tiny multiply program to verify it works, then set up a local Git repository to track your code.

In this step, get ready to:
  • Write and run a multiply test in Venus to confirm the RV32IM multiply instruction works.
  • Create a local project folder with a Git repository and README.
  • Save your assembly code locally and make your first commit.
Open Venus and run your first multiply program

Venus is a browser-based RISC-V simulator. It has two tabs: an Editor tab where you write assembly, and a Simulator tab where your code gets assembled and runs.

  • Open Venus by navigating to this URL in your browser:
https://kvakil.github.io/venus/

You should see a code editor on the left side of the screen. This is the Editor tab.

  • Clear any existing code in the editor, then paste in the following multiply test program:
.data
# Test values
val_a: .word 6
val_b: .word 7

.text
.globl main

main:
    la t0, val_a
    lw t0, 0(t0)
    la t1, val_b
    lw t1, 0(t1)
    mul a1, t0, t1
    li a0, 1
    ecall
    li a0, 10
    ecall

What does this code do?

This program has two sections. The .data section stores two test values (6 and 7) in memory. The .text section contains the instructions that run.

  • la t0, val_a loads the memory address of val_a into register t0. Then lw t0, 0(t0) loads the actual value (6) from that address.
  • mul a1, t0, t1 multiplies the two values and stores the product in a1. This is the RV32IM multiply instruction your inference engine depends on.
  • li a0, 1 followed by ecall tells Venus to print the integer in a1. Venus uses a0 for the ecall number and a1 for the argument.
  • li a0, 10 followed by ecall exits the program.

Before you run this, what do you think the console will print?

  • Click the Simulator tab at the top of Venus to assemble your code.
  • Click Run in the simulator controls.

The Console Output at the bottom displays 42. That is 6 multiplied by 7. The RV32IM multiply instruction works.

Use Step to watch registers change

Click Reset then click Step repeatedly to execute one instruction at a time. Watch the Registers sidebar on the right. The most recently changed register is highlighted in blue.

Don't see 42 in the console?

  • Make sure you pasted the entire program including the .data section at the top. Missing data labels cause assembly errors.
  • Check that you clicked the Simulator tab (not just Run from the Editor). Venus assembles code when you switch to the Simulator tab.
  • If you see an error in the console about mul, the simulator may not support the M extension. Use the kvakil Venus link above, which supports RV32IM.

Still stuck? Why isn't my Venus program printing 42?

Create your project folder and Git repository

Your multiply test works in Venus. Now create a local folder to store and version-control your code as you build the full inference engine.

  • Open your terminal. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir riscv-inference
cd riscv-inference
  • Initialize a Git repository by running:
git init
  • Create a README.md file describing your project by running:
echo "# Quantized Inference Engine in RISC-V Assembly" > README.md
echo "" >> README.md
echo "A project description placeholder." >> README.md
  • Confirm your files exist by running:
ls

You should see README.md listed in the output.

Command not found or permission denied?

  • On Windows, use cd $HOME\Desktop instead of cd ~/Desktop if the tilde does not resolve.
  • Make sure you are running commands in your terminal, not in Venus. Venus is a browser-only tool.

Need help? I cannot create my project folder on the Desktop

Save your assembly file and make your first commit

The final piece is saving your working Venus code as a local file and committing it to Git.

  • Create the assembly file quantized_inference.s by running:
touch quantized_inference.s
  • Open quantized_inference.s in any text editor and paste in the same multiply test program you ran in Venus:
.data
# Test values
val_a: .word 6
val_b: .word 7

.text
.globl main

main:
    la t0, val_a
    lw t0, 0(t0)
    la t1, val_b
    lw t1, 0(t1)
    mul a1, t0, t1
    li a0, 1
    ecall
    li a0, 10
    ecall
  • Save the file.
  • Stage all files and make your first commit by running:
git add .
git commit -m "Add multiply test program"

You should see output confirming 2 files changed (README.md and quantized_inference.s).

Git asking for your name and email?

If Git prompts you to set your identity, run these two commands with your own details:

git config --global user.email "your-email@example.com"
git config --global user.name "Your Name"

Then re-run the git commit command.

✔️ Awesome, I've got everything!

Great. Double check you have saved quantized_inference.s and committed both files.

ⓧ I'd like to double check the full code

Your quantized_inference.s should contain exactly this:

.data
# Test values
val_a: .word 6
val_b: .word 7

.text
.globl main

main:
    la t0, val_a
    lw t0, 0(t0)
    la t1, val_b
    lw t1, 0(t1)
    mul a1, t0, t1
    li a0, 1
    ecall
    li a0, 10
    ecall

Your README.md should contain:

# Quantized Inference Engine in RISC-V Assembly

A project description placeholder.

Your environment is ready and you have proof that Venus can multiply. Next up, you will build the quantized dot product, the fundamental operation that powers every neuron in your inference engine.

Build the Quantized Dot Product

Your multiply instruction works. Now it is time to build the fundamental operation that powers every neural network layer: the dot product.

Every neuron in a network computes a weighted sum of its inputs. That means multiplying each input by a weight, then adding all the products together. In quantized inference, these weights and inputs are INT8 values (fitting in a single byte), but the running total must live in a 32-bit register to avoid overflow. You are about to see exactly why, instruction by instruction.

In this step, get ready to:
  • Store INT8 weights and inputs in memory and write a multiply-accumulate loop.
  • Print the dot product result and step through execution watching registers change.
  • Observe why 32-bit accumulation is necessary by testing with large values.
Store weights and inputs in the data section

A dot product needs two arrays of numbers: weights and inputs. You will store them in the .data section of your assembly file, then write a loop that walks through both arrays in lockstep.

  • In Venus, switch to the Editor tab.
  • Select all the existing code and delete it. You are replacing the multiply test with your dot product program.
  • Paste the following data section and main function:
.data
# Neuron 0 weights (INT8)
l1_w0: .word 2, -3, 1, 4

# Input vector (4 INT8 values)
input: .word 10, 20, 30, 40

str_result: .asciz "Dot product: "
str_newline: .asciz "\n"

.text
.globl main

main:
    la s0, l1_w0
    la s1, input
    jal ra, dot_product_4
    mv a1, a0
    la a0, str_result
    mv a1, a0
    li a0, 4
    ecall
    la a0, str_result
    lw a1, 0(s0)
    li a0, 1
    ecall
    li a0, 10
    ecall

What does this code do?

  • l1_w0: .word 2, -3, 1, 4 stores four INT8 weight values in memory as 32-bit words. Using .word (one value per word) keeps loads simple. Real hardware packs 4 bytes per word, but word-per-value lets you focus on the algorithm.
  • input: .word 10, 20, 30, 40 stores the input vector your neuron will process.
  • The main function loads the addresses of both arrays into s0 and s1, calls the dot product function, prints the result, and exits.
Write the dot product function

The dot product function is where the real work happens. It loops through 4 elements, multiplying each weight by its paired input, and accumulates the products into a single 32-bit register.

  • Add the following function below the main function (after the exit ecall):
# dot_product_4: Computes dot product of 4-element vectors
# Input: s0 = address of weights, s1 = address of inputs
# Output: a0 = dot product result
dot_product_4:
    li t0, 0             # accumulator = 0
    li t1, 0             # loop counter i = 0
    li t2, 4             # loop limit
dp4_loop:
    bge t1, t2, dp4_done
    slli t3, t1, 2       # t3 = i * 4 (word offset)
    add t4, s0, t3       # t4 = &weights[i]
    lw t5, 0(t4)         # t5 = weights[i]
    add t4, s1, t3       # t4 = &inputs[i]
    lw t6, 0(t4)         # t6 = inputs[i]
    mul t5, t5, t6       # t5 = weights[i] * inputs[i]
    add t0, t0, t5       # accumulator += product
    addi t1, t1, 1       # i++
    j dp4_loop
dp4_done:
    mv a0, t0            # return accumulator
    ret

What does this loop do?

This is a multiply-accumulate (MAC) loop. Each iteration does three things:

  • slli t3, t1, 2 computes the byte offset by shifting left by 2 (multiplying by 4, since each .word is 4 bytes).
  • lw loads both the weight and input from memory into registers.
  • mul multiplies them, and add accumulates the product into t0. Register t0 is 32 bits wide, so the running total can far exceed what a single byte could hold.

After 4 iterations the accumulated result moves into a0 (the return register) via ret.

Before you run this, what do you expect the dot product of [2, -3, 1, 4] and [10, 20, 30, 40] to be? Calculate it in your head: (2x10) + (-3x20) + (1x30) + (4x40).

  • Click the Simulator tab in Venus to assemble and run your program.
  • Click Run to execute the full program.

You should see 120 printed in the console output. That is: (2x10) + (-3x20) + (1x30) + (4x40) = 20 - 60 + 30 + 160 = 120.

Not seeing 120 in the console?

  • Make sure you deleted ALL the old multiply test code before pasting. Leftover labels cause assembly errors.
  • Check that dot_product_4 appears BELOW the exit ecall in main. If it is above main, the program counter hits it before main runs.
  • Verify the .globl main directive is present. Without it, Venus does not know where to start execution.

Help me debug my dot product function in RISC-V Venus.

Step through execution and watch the accumulator grow

Running the program proves it works. Stepping through it proves you understand it. The Venus debugger lets you execute one instruction at a time and watch each register change in real time.

  • Click Reset in the simulator to return to the first instruction.
  • In the Registers sidebar on the right, change the display setting from Hex to Decimal so you can read values as signed integers.
  • Click Step repeatedly until execution enters the dp4_loop label. Watch register t0 (the accumulator) in the sidebar.
  • Continue stepping through the loop. After each add t0, t0, t5 instruction, t0 updates. The most recently changed register highlights in blue.

You should see t0 change from 0 to 20 (first product), then to -40 (after adding -60), then to -10 (after adding 30), then to 120 (after adding 160). Each intermediate value proves the MAC operation is working correctly.

Why step through instead of just running?

In real inference hardware, you cannot pause a GPU mid-computation. But on Venus, single-stepping reveals the exact moment each multiply and accumulate fires. This is the same mental model chip designers use when verifying their MAC units.

Notice that t0 goes negative (-40) mid-computation. A single-byte INT8 register would have wrapped around at -128. The 32-bit register handles this gracefully.

Observe why 32-bit accumulation is necessary

Your current inputs are small enough to fit in a byte. But real neural networks process larger activations. To see what happens when products exceed the INT8 range (-128 to 127), try larger input values.

  • Switch back to the Editor tab in Venus.
  • Find this line:
input: .word 10, 20, 30, 40
  • Temporarily change the values to larger numbers:
input: .word 100, 100, 100, 100
  • Click the Simulator tab and click Run.

The result is 400. That is (2x100) + (-3x100) + (1x100) + (4x100) = 200 - 300 + 100 + 400. The individual product -300 already exceeds the INT8 range of -128 to 127. If the accumulator were only 8 bits wide, it would have silently wrapped around and given you a garbage answer. The 32-bit register t0 handles it without issue.

Why does this matter for real hardware?

Every production inference accelerator (from NVIDIA Tensor Cores to the RISC-V XMPI extension) uses INT8 inputs but accumulates into 32-bit registers for exactly this reason. You just demonstrated the core design constraint that drives all quantized inference hardware.

  • Switch back to the Editor tab and restore the original input values:
input: .word 10, 20, 30, 40
  • Click the Simulator tab and click Run to confirm the output is back to 120.

Also save your updated quantized_inference.s file locally (copy the code from the Venus editor and paste it into your local file).

Seeing an unexpected number after restoring inputs?

  • Double-check you changed the values back to exactly 10, 20, 30, 40. A stray space or missing comma causes assembly errors.
  • If Venus shows an error instead of a number, look for a red error message at the bottom of the console. It usually points to the line with the typo.

Help me fix my data section values in Venus.

✔️ Awesome, I've got everything!

Great. Make sure your local quantized_inference.s file is saved with the final version (inputs back to 10, 20, 30, 40).

ⓧ I'd like to double check the full code

Scroll up and confirm your code contains these sections in order:

  • The .data section with l1_w0, input, str_result, and str_newline.
  • The .text section with .globl main and the main function that loads addresses, calls dot_product_4, prints, and exits.
  • The dot_product_4 function with its dp4_loop label, mul and add instructions, and a ret at the end.

Your dot product function works and you have seen the accumulator grow register by register. Next up, you will add the quantization step that real inference hardware uses to bring that large 32-bit result back into INT8 range for the next layer.

Add Fixed-Point Rescaling and ReLU

Your dot product function works perfectly, accumulating multiplied INT8 values into a 32-bit register and printing the result (120). But that raw accumulator value is in an inflated scale. If you fed it directly into a second layer of dot products, the numbers would compound and quickly overflow even a 32-bit register.

Real quantized inference pipelines solve this with a right-shift operation that divides the result back into INT8 range. In this step, you'll add that rescaling operation and a ReLU activation that clamps negative values to zero.

In this step, get ready to:
  • Restructure the program to display the unscaled accumulator alongside the rescaled result.
  • Add arithmetic right-shift to rescale the accumulator into INT8 range.
  • Implement ReLU to clamp negative values to zero.
See the scale problem with your raw accumulator

Your program currently prints 120 as the raw dot product. That number represents the sum of all weight-times-input products, accumulated without any correction for scale.

Why can't 120 feed the next layer directly?

When you quantize weights and inputs to INT8 (range -128 to 127), each multiplication implicitly multiplies their scale factors together. After accumulating four such products, the result sits in a combined scale that is 2^3 = 8 times larger than what the next layer expects.

If you pass this inflated value into another dot product without correcting the scale, the second layer's accumulator would be 64x too large. By the third layer, you'd overflow 32 bits entirely.

Before you fix this, think: what single instruction could divide by a power of two without using an expensive divide operation?

Add rescaling with arithmetic right-shift

Arithmetic right-shift divides by powers of two while preserving the sign bit. Shifting right by 3 is equivalent to dividing by 8, which corrects the scale factor from your INT8 multiplications.

  • Select all text in quantized_inference.s and delete it.
  • Replace it with the following .data section and main function:
.data
l1_w0: .word 2, -3, 1, 4
input: .word 10, 20, 30, 40
l1_shift: .word 3

str_unscaled: .asciz "Unscaled: "
str_scaled: .asciz "Scaled: "
str_newline: .asciz "\n"

.text
.globl main

main:
    la s0, l1_w0
    la s1, input
    jal ra, dot_product_4
    mv s2, a0

    # Print unscaled
    la a0, str_unscaled
    jal ra, print_str
    mv a1, s2
    jal ra, print_int_val
    jal ra, print_newline

    # Rescale
    la t0, l1_shift
    lw t1, 0(t0)
    sra s2, s2, t1
  • Continue pasting directly below the rescale code. Add the scaled output print and the program exit:
    # Print scaled
    la a0, str_scaled
    jal ra, print_str
    mv a1, s2
    jal ra, print_int_val
    jal ra, print_newline

    li a0, 10
    ecall
  • Below the exit, paste your dot_product_4 function (same code from the previous step):
dot_product_4:
    li t0, 0
    li t1, 0
    li t2, 4
dp4_loop:
    bge t1, t2, dp4_done
    slli t3, t1, 2
    add t4, s0, t3
    lw t5, 0(t4)
    add t4, s1, t3
    lw t6, 0(t4)
    mul t5, t5, t6
    add t0, t0, t5
    addi t1, t1, 1
    j dp4_loop
dp4_done:
    mv a0, t0
    ret
  • Finally, paste these helper functions below dot_product_4:
print_int_val:
    li a0, 1
    ecall
    ret

print_str:
    mv a1, a0
    li a0, 4
    ecall
    ret

print_newline:
    li a0, 11
    li a1, 10
    ecall
    ret

What do these helper functions do?

  • print_int_val uses ecall 1 (print integer) with the value in a1.
  • print_str uses ecall 4 (print string) with the address in a1.
  • print_newline uses ecall 11 (print character) to output ASCII 10, a newline.

Venus places the ecall number in a0 and arguments in a1 onward. This convention differs from Linux RISC-V (which uses a7 for the syscall number).

What does the rescaling code do?

  • la t0, l1_shift loads the address of the shift amount stored in memory.
  • lw t1, 0(t0) reads the value 3 (the shift amount) into register t1.
  • sra s2, s2, t1 performs arithmetic right-shift on the accumulator by 3 bits, dividing by 8 while preserving the sign.

This single sra instruction replaces expensive division in real inference hardware. The shift amount (3) represents the combined quantization scale factor of weights and activations.

Before you run this, predict: if the unscaled accumulator is 120, what should 120 shifted right by 3 equal?

  • Paste the full program into the Venus editor, replacing any previous code.
  • Click Simulator in the top navigation, then click Assemble & Simulate from Editor.
  • Click Run to execute the full program.

The console prints two lines:

Unscaled: 120
Scaled: 15

120 divided by 8 equals 15. The value 15 fits comfortably in INT8 range (-128 to 127) and can safely feed into the next layer.

Not seeing the expected output?

  • If you see an assembler error, check that all labels (str_unscaled, str_scaled, l1_shift) are defined in your .data section above .text.
  • If the program hangs or prints garbage, confirm that print_str, print_int_val, and print_newline are all pasted below dot_product_4.
  • If Scaled shows 0, make sure l1_shift: .word 3 uses the value 3, not a larger number that would shift all bits away.

help me debug my Venus assembly rescaling output

Implement ReLU as a branch instruction

ReLU (Rectified Linear Unit) is the simplest activation function in neural networks: if the value is negative, replace it with zero. In high-level code that's an if-statement. In assembly, it's a single branch instruction.

  • In quantized_inference.s, find the exit lines at the bottom of your main function:
    li a0, 10
    ecall
  • Add the following ReLU code directly above those two exit lines:
    # ReLU
    bge s2, zero, skip_relu0
    li s2, 0
skip_relu0:

What does this ReLU code do?

  • bge s2, zero, skip_relu0 checks if the rescaled value in s2 is greater than or equal to zero. If yes, it jumps past the clamp.
  • li s2, 0 only executes if the value is negative, setting it to zero.

This maps directly to how inference accelerators implement ReLU. One comparison and one conditional write. No function call overhead, no floating-point unit needed.

  • Click Assemble & Simulate from Editor again, then click Run.

The output still shows 15 for the scaled value. That's correct. Since 15 is positive, ReLU leaves it unchanged.

When would ReLU actually clamp?

If your weights produced a negative dot product (for example, changing the first weight from 2 to -10 would give a negative accumulator), the rescaled value would be negative and ReLU would clamp it to 0. In the full network (Step 4), some neurons will produce negative values and you'll see ReLU activate.

Seeing an assembler error after adding ReLU?

  • Make sure the label skip_relu0: starts at column 0 (no leading spaces). Labels in RISC-V assembly must be flush left.
  • Check that the ReLU block sits between the print_newline call for the scaled value and the li a0, 10 exit line.

help me fix my ReLU assembly code in Venus

✔️ Awesome, I've got everything!

Great. Confirm you've saved your file with all the changes.

ⓧ I'd like to double check the full code

Your quantized_inference.s should contain these sections in order. Scroll through your file and confirm each is present:

  • .data section with l1_w0, input, l1_shift, str_unscaled, str_scaled, and str_newline labels.
  • main function that calls dot_product_4, prints unscaled, rescales with sra, prints scaled, applies ReLU, then exits.
  • dot_product_4 function (unchanged from the previous step).
  • Three helper functions: print_int_val, print_str, print_newline.

You've built the core quantization pipeline: multiply, accumulate, right-shift to rescale, and activate with ReLU. Next up, you'll wire multiple neurons into layers and watch the full 2-layer network classify an input.

Wire Up the Full 2-Layer Network

Your single dot product with rescaling and ReLU proves the fundamental math of quantized inference works at the instruction level. Now it's time to wire multiple neurons together into layers, proving you understand the full inference data flow at the hardware level.

In this step, you'll expand your program into a complete 2-layer quantized neural network. Layer 1 takes your 4-element input and produces 3 hidden activations. Layer 2 takes those 3 values and produces 3 output scores. An argmax picks the winning class.

In this step, get ready to:
  • Define weight matrices for both layers and buffers for intermediate results.
  • Compute 3 hidden neurons in Layer 1 with rescale and ReLU, then 3 output scores in Layer 2.
  • Implement argmax to find the predicted class and print all intermediate values.
Expand the data section with all weights and buffers

A real neural network layer has multiple neurons, each with its own weight vector. Your .data section needs weight vectors for all 3 hidden neurons in Layer 1 and all 3 output neurons in Layer 2, plus buffers to store intermediate results between layers.

  • In Venus, select all the code in your .data section (from .data down to the last .asciz line, but not .text or anything after it).
  • Replace it with the following weight declarations for both layers:
.data
# Layer 1 weights: 3 neurons x 4 inputs (INT8, stored as words for simplicity)
# Neuron 0 weights
l1_w0: .word 2, -3, 1, 4
# Neuron 1 weights
l1_w1: .word -1, 5, -2, 3
# Neuron 2 weights
l1_w2: .word 3, 1, -4, -1

# Layer 2 weights: 3 outputs x 3 inputs
# Output 0 weights
l2_w0: .word 4, -2, 1
# Output 1 weights
l2_w1: .word -3, 5, 2
# Output 2 weights
l2_w2: .word 1, -1, 6

What does this data represent?

Layer 1 has 3 neurons, each receiving the 4-element input vector. That means 3 weight vectors of length 4 (12 weights total).

Layer 2 has 3 output neurons, each receiving the 3 hidden activations from Layer 1. That means 3 weight vectors of length 3 (9 weights total).

  • Immediately below the weight declarations, add the input, scale factors, buffers, and string labels:
# Input vector (4 INT8 values)
input: .word 10, 20, 30, 40

# Scale shift amounts (representing quantization scale factors)
l1_shift: .word 3
l2_shift: .word 2

# Buffers for intermediate results
hidden: .word 0, 0, 0
output: .word 0, 0, 0

# Strings for output
str_hidden: .asciz "Hidden: "
str_output: .asciz "Output: "
str_class: .asciz "Class: "
str_space: .asciz " "
str_newline: .asciz "\n"
str_unscaled: .asciz "Unscaled: "
str_scaled: .asciz "Scaled: "

What are the buffers for?

The hidden buffer (3 words) stores Layer 1's output after rescale and ReLU. Layer 2 reads from this buffer as its input. The output buffer (3 words) stores Layer 2's final scores that the argmax will compare.

The l2_shift value of 2 means Layer 2 divides its accumulators by 4 (right-shift by 2 bits) to bring results back into a usable range.

  • Click the **Simulator** tab in Venus to assemble the code.

The program should assemble without errors. If you see an error mentioning an undefined label, check that every label referenced in the .text section has a matching definition in .data.

Compute all three Layer 1 hidden neurons

Your existing code computes Neuron 0 but doesn't store the result anywhere another layer can read it. You need to store each neuron's output in the hidden buffer, then add the same dot-product-rescale-ReLU pattern for Neurons 1 and 2.

  • Replace everything in your main: section (from main: down to the li a0, 10 exit ecall, but not the functions below) with the following Neuron 0 code:
main:
    # ===== LAYER 1: Compute 3 hidden neurons =====

    # --- Neuron 0 ---
    la s0, l1_w0          # s0 = address of neuron 0 weights
    la s1, input          # s1 = address of input vector
    jal ra, dot_product_4 # result in a0
    mv s2, a0             # s2 = raw accumulator (unscaled)

    # Print unscaled value
    la a0, str_unscaled
    jal ra, print_str
    mv a1, s2
    jal ra, print_int_val
    jal ra, print_newline

    # Rescale
    la t0, l1_shift
    lw t1, 0(t0)          # t1 = shift amount
    sra s2, s2, t1        # arithmetic right-shift to rescale

    # Print scaled value
    la a0, str_scaled
    jal ra, print_str
    mv a1, s2
    jal ra, print_int_val
    jal ra, print_newline

    # ReLU
    bge s2, zero, skip_relu0
    li s2, 0
skip_relu0:
    la t0, hidden
    sw s2, 0(t0)          # store hidden[0]

What changed from before?

The key addition is the last two lines: la t0, hidden followed by sw s2, 0(t0). After ReLU clamps the value, we store it at offset 0 of the hidden buffer so Layer 2 can read it later.

  • Immediately below the skip_relu0 block, add Neurons 1 and 2 which follow the same pattern but store at different offsets:
    # --- Neuron 1 ---
    la s0, l1_w1
    la s1, input
    jal ra, dot_product_4
    la t0, l1_shift
    lw t1, 0(t0)
    sra a0, a0, t1        # rescale
    bge a0, zero, skip_relu1
    li a0, 0
skip_relu1:
    la t0, hidden
    sw a0, 4(t0)          # store hidden[1]
    mv s3, a0

    # --- Neuron 2 ---
    la s0, l1_w2
    la s1, input
    jal ra, dot_product_4
    la t0, l1_shift
    lw t1, 0(t0)
    sra a0, a0, t1        # rescale
    bge a0, zero, skip_relu2
    li a0, 0
skip_relu2:
    la t0, hidden
    sw a0, 8(t0)          # store hidden[2]
    mv s4, a0

Why different offsets?

Each .word is 4 bytes wide. Neuron 0 stores at offset 0, Neuron 1 at offset 4, and Neuron 2 at offset 8. This packs three 32-bit values consecutively in the hidden buffer.

Notice that Neurons 1 and 2 skip the unscaled/scaled print statements. Printing every neuron would clutter the output. We keep the debug prints for Neuron 0 only and print the full hidden layer together next.

  • Below Neuron 2, add the code to print all three hidden layer values on one line:
    # Print hidden layer values
    la a0, str_hidden
    jal ra, print_str
    la t0, hidden
    lw a1, 0(t0)
    jal ra, print_int_val
    jal ra, print_space
    lw a1, 4(t0)
    jal ra, print_int_val
    jal ra, print_space
    lw a1, 8(t0)
    jal ra, print_int_val
    jal ra, print_newline

What does this print block do?

It prints "Hidden: " followed by the three values separated by spaces. Each lw a1, N(t0) loads a value from the buffer, and print_space outputs ASCII character 32 (a space) between them.

  • You also need the print_space function. Scroll down to your functions section (below print_newline) and add:
# print_space: Prints a space character
print_space:
    li a0, 11
    li a1, 32
    ecall
    ret
  • Add a temporary exit after the hidden layer print. Below the jal ra, print_newline that prints the hidden values, add:
    # Exit (temporary - will remove after adding Layer 2)
    li a0, 10
    ecall
  • Click the **Simulator** tab to assemble, then click **Run**.

Before you look at the output, predict: given input [10, 20, 30, 40] and Neuron 0 weights [2, -3, 1, 4], what should the unscaled dot product be?

You should see the console print the unscaled value (120), the scaled value (15), and then the hidden layer values for all three neurons. The hidden output should read Hidden: 15 16 0.

Why is hidden[2] zero?

Neuron 2 has weights [3, 1, -4, -1]. Its dot product with input [10, 20, 30, 40] is 30 + 20 - 120 - 40 = -110. After right-shifting by 3 (dividing by 8), you get approximately -14. ReLU clamps any negative to 0.

Not seeing the expected output?

  • If assembly fails, check that the labels l1_w1, l1_w2, hidden, and str_hidden are defined in your .data section.
  • If the hidden values are wrong, verify the offsets: 0(t0) for neuron 0, 4(t0) for neuron 1, 8(t0) for neuron 2.
  • If you see garbled output, make sure print_space is defined after print_newline in your functions section.

Help me debug my hidden layer output.

Add Layer 2, argmax, and final output

Layer 1's hidden values are now stored in the hidden buffer. Layer 2 reads those 3 values as its inputs, multiplies by its own weights, rescales, and produces 3 output scores. The argmax then picks which output is largest, giving you the predicted class.

  • First, you need a dot_product_3 function for Layer 2 (which has 3-element vectors). Add this below your dot_product_4 function in the functions section:
# dot_product_3: Computes dot product of 3-element vectors
# Input: s0 = address of weights, s1 = address of inputs
# Output: a0 = dot product result
dot_product_3:
    li t0, 0
    li t1, 0
    li t2, 3
dp3_loop:
    bge t1, t2, dp3_done
    slli t3, t1, 2
    add t4, s0, t3
    lw t5, 0(t4)
    add t4, s1, t3
    lw t6, 0(t4)
    mul t5, t5, t6
    add t0, t0, t5
    addi t1, t1, 1
    j dp3_loop
dp3_done:
    mv a0, t0
    ret

Why a separate function?

The dot_product_3 function is identical to dot_product_4 except the loop limit is 3 instead of 4. Layer 2 has 3 inputs (the hidden activations), so it needs a 3-element dot product.

  • Now remove your temporary exit (the li a0, 10 and ecall you added after the hidden layer print) and replace it with the Layer 2 computation:
    # ===== LAYER 2: Compute 3 output scores =====

    # --- Output 0 ---
    la s0, l2_w0
    la s1, hidden
    jal ra, dot_product_3
    la t0, l2_shift
    lw t1, 0(t0)
    sra a0, a0, t1
    la t0, output
    sw a0, 0(t0)

    # --- Output 1 ---
    la s0, l2_w1
    la s1, hidden
    jal ra, dot_product_3
    la t0, l2_shift
    lw t1, 0(t0)
    sra a0, a0, t1
    la t0, output
    sw a0, 4(t0)

    # --- Output 2 ---
    la s0, l2_w2
    la s1, hidden
    jal ra, dot_product_3
    la t0, l2_shift
    lw t1, 0(t0)
    sra a0, a0, t1
    la t0, output
    sw a0, 8(t0)

Why no ReLU on Layer 2?

Layer 2 is the final output layer. Its scores represent confidence in each class. Negative scores are valid here because argmax just picks the highest value regardless of sign. ReLU is only applied between hidden layers to introduce non-linearity.

  • Below the Layer 2 computation, add the output printing, argmax logic, and the final exit:
    # Print output layer values
    la a0, str_output
    jal ra, print_str
    la t0, output
    lw a1, 0(t0)
    jal ra, print_int_val
    jal ra, print_space
    lw a1, 4(t0)
    jal ra, print_int_val
    jal ra, print_space
    lw a1, 8(t0)
    jal ra, print_int_val
    jal ra, print_newline

    # ===== ARGMAX: Find predicted class =====
    la s0, output
    lw t0, 0(s0)         # t0 = output[0]
    li t1, 0             # t1 = best_index = 0
    lw t2, 4(s0)         # t2 = output[1]
    ble t2, t0, check2
    mv t0, t2            # new max
    li t1, 1             # best_index = 1
check2:
    lw t2, 8(s0)         # t2 = output[2]
    ble t2, t0, done_argmax
    li t1, 2             # best_index = 2
done_argmax:

    # Print predicted class
    la a0, str_class
    jal ra, print_str
    mv a1, t1
    jal ra, print_int_val
    jal ra, print_newline

How does argmax work here?

Argmax finds which of the 3 outputs is largest. It starts by assuming output[0] is the max (index 0). It compares output[1] against the current max using ble (branch if less-or-equal). If output[1] is bigger, the max and index update. Same for output[2].

The final value in t1 is the predicted class: 0, 1, or 2.

  • Add the final exit ecall after the class print:
    # Exit
    li a0, 10
    ecall
  • Click the **Simulator** tab to assemble, then click **Run**.

Before you look at the result, predict: which class (0, 1, or 2) do you think wins for this set of weights and inputs?

The console should now print the full inference pipeline output. You should see Hidden: 15 16 0, then Output: 7 13 0, and finally Class: 1.

Why class 1?

Output neuron 1 scored 13, which is higher than output 0 (7) and output 2 (0). So argmax returns index 1. Your 2-layer quantized network just classified the input vector.

  • To confirm the network is actually computing (not just hardcoded), change the input line in .data to different values (for example, .word 40, 10, 10, 10) and run again.

You should see the predicted class change to a different value. Change the input back to .word 10, 20, 30, 40 before continuing.

Getting the wrong class or an assembly error?

  • If assembly fails, make sure dot_product_3 is defined in your functions section and the labels l2_w0, l2_w1, l2_w2, l2_shift, and output all exist in .data.
  • If the output values are wrong, check that Layer 2 uses la s1, hidden (not la s1, input). Layer 2 reads from the hidden buffer, not the original input.
  • If argmax gives the wrong class, verify the check2 and done_argmax labels exist and the branch logic uses ble (branch if less-or-equal).

Help me debug my full network output.

✔️ Awesome, I've got everything!

Great. Make sure your file is saved and the program prints the correct output with the original input [10, 20, 30, 40].

ⓧ I'd like to double check the full code

Your quantized_inference.s file has grown substantially. Here it is broken into sections so you can compare each part.

The .data section should contain all weights, buffers, and strings:

.data
# Layer 1 weights: 3 neurons x 4 inputs (INT8, stored as words for simplicity)
# Neuron 0 weights
l1_w0: .word 2, -3, 1, 4
# Neuron 1 weights
l1_w1: .word -1, 5, -2, 3
# Neuron 2 weights
l1_w2: .word 3, 1, -4, -1

# Layer 2 weights: 3 outputs x 3 inputs
# Output 0 weights
l2_w0: .word 4, -2, 1
# Output 1 weights
l2_w1: .word -3, 5, 2
# Output 2 weights
l2_w2: .word 1, -1, 6

# Input vector (4 INT8 values)
input: .word 10, 20, 30, 40

# Scale shift amounts (representing quantization scale factors)
l1_shift: .word 3
l2_shift: .word 2

# Buffers for intermediate results
hidden: .word 0, 0, 0
output: .word 0, 0, 0

# Strings for output
str_hidden: .asciz "Hidden: "
str_output: .asciz "Output: "
str_class: .asciz "Class: "
str_space: .asciz " "
str_newline: .asciz "\n"
str_unscaled: .asciz "Unscaled: "
str_scaled: .asciz "Scaled: "

The main section should compute Layer 1 (Neuron 0 with prints, Neurons 1 and 2, hidden print):

.text
.globl main

main:
    # ===== LAYER 1: Compute 3 hidden neurons =====

    # --- Neuron 0 ---
    la s0, l1_w0          # s0 = address of neuron 0 weights
    la s1, input          # s1 = address of input vector
    jal ra, dot_product_4 # result in a0
    mv s2, a0             # s2 = raw accumulator (unscaled)

    # Print unscaled value
    la a0, str_unscaled
    jal ra, print_str
    mv a1, s2
    jal ra, print_int_val
    jal ra, print_newline

    # Rescale
    la t0, l1_shift
    lw t1, 0(t0)          # t1 = shift amount
    sra s2, s2, t1        # arithmetic right-shift to rescale

    # Print scaled value
    la a0, str_scaled
    jal ra, print_str
    mv a1, s2
    jal ra, print_int_val
    jal ra, print_newline

    # ReLU
    bge s2, zero, skip_relu0
    li s2, 0
skip_relu0:
    la t0, hidden
    sw s2, 0(t0)          # store hidden[0]

Neurons 1 and 2 plus the hidden layer print:

    # --- Neuron 1 ---
    la s0, l1_w1
    la s1, input
    jal ra, dot_product_4
    la t0, l1_shift
    lw t1, 0(t0)
    sra a0, a0, t1        # rescale
    bge a0, zero, skip_relu1
    li a0, 0
skip_relu1:
    la t0, hidden
    sw a0, 4(t0)          # store hidden[1]
    mv s3, a0

    # --- Neuron 2 ---
    la s0, l1_w2
    la s1, input
    jal ra, dot_product_4
    la t0, l1_shift
    lw t1, 0(t0)
    sra a0, a0, t1        # rescale
    bge a0, zero, skip_relu2
    li a0, 0
skip_relu2:
    la t0, hidden
    sw a0, 8(t0)          # store hidden[2]
    mv s4, a0

The hidden layer print and Layer 2 computation:

    # Print hidden layer values
    la a0, str_hidden
    jal ra, print_str
    la t0, hidden
    lw a1, 0(t0)
    jal ra, print_int_val
    jal ra, print_space
    lw a1, 4(t0)
    jal ra, print_int_val
    jal ra, print_space
    lw a1, 8(t0)
    jal ra, print_int_val
    jal ra, print_newline

    # ===== LAYER 2: Compute 3 output scores =====

    # --- Output 0 ---
    la s0, l2_w0
    la s1, hidden
    jal ra, dot_product_3
    la t0, l2_shift
    lw t1, 0(t0)
    sra a0, a0, t1
    la t0, output
    sw a0, 0(t0)

Outputs 1 and 2, then print output values:

    # --- Output 1 ---
    la s0, l2_w1
    la s1, hidden
    jal ra, dot_product_3
    la t0, l2_shift
    lw t1, 0(t0)
    sra a0, a0, t1
    la t0, output
    sw a0, 4(t0)

    # --- Output 2 ---
    la s0, l2_w2
    la s1, hidden
    jal ra, dot_product_3
    la t0, l2_shift
    lw t1, 0(t0)
    sra a0, a0, t1
    la t0, output
    sw a0, 8(t0)

    # Print output layer values
    la a0, str_output
    jal ra, print_str
    la t0, output
    lw a1, 0(t0)
    jal ra, print_int_val
    jal ra, print_space
    lw a1, 4(t0)
    jal ra, print_int_val
    jal ra, print_space
    lw a1, 8(t0)
    jal ra, print_int_val
    jal ra, print_newline

Argmax, class print, and exit:

    # ===== ARGMAX: Find predicted class =====
    la s0, output
    lw t0, 0(s0)         # t0 = output[0]
    li t1, 0             # t1 = best_index = 0
    lw t2, 4(s0)         # t2 = output[1]
    ble t2, t0, check2
    mv t0, t2            # new max
    li t1, 1             # best_index = 1
check2:
    lw t2, 8(s0)         # t2 = output[2]
    ble t2, t0, done_argmax
    li t1, 2             # best_index = 2
done_argmax:

    # Print predicted class
    la a0, str_class
    jal ra, print_str
    mv a1, t1
    jal ra, print_int_val
    jal ra, print_newline

    # Exit
    li a0, 10
    ecall

The functions section should have all five functions:

# ===== FUNCTIONS =====

dot_product_4:
    li t0, 0
    li t1, 0
    li t2, 4
dp4_loop:
    bge t1, t2, dp4_done
    slli t3, t1, 2
    add t4, s0, t3
    lw t5, 0(t4)
    add t4, s1, t3
    lw t6, 0(t4)
    mul t5, t5, t6
    add t0, t0, t5
    addi t1, t1, 1
    j dp4_loop
dp4_done:
    mv a0, t0
    ret
dot_product_3:
    li t0, 0
    li t1, 0
    li t2, 3
dp3_loop:
    bge t1, t2, dp3_done
    slli t3, t1, 2
    add t4, s0, t3
    lw t5, 0(t4)
    add t4, s1, t3
    lw t6, 0(t4)
    mul t5, t5, t6
    add t0, t0, t5
    addi t1, t1, 1
    j dp3_loop
dp3_done:
    mv a0, t0
    ret

print_int_val:
    li a0, 1
    ecall
    ret

print_str:
    mv a1, a0
    li a0, 4
    ecall
    ret

print_space:
    li a0, 11
    li a1, 32
    ecall
    ret

print_newline:
    li a0, 11
    li a1, 10
    ecall
    ret

Your quantized inference engine now classifies a 4-feature input vector into one of 3 classes, entirely in RISC-V assembly. Next up, you'll push this to GitHub so others can see what you built.

Push to GitHub

Your quantized inference engine is complete and running in Venus. It classifies inputs, prints intermediate values, and picks a predicted class.

For this project to be shareable and useful to future-you, it needs to live in a GitHub repository with documentation. In this step, you will write a proper README and push everything to GitHub.

In this step, get ready to:
  • Write a README explaining the project, why quantized inference matters, and how to run it.
  • Push your code to a public GitHub repository.
  • Verify the repository renders your README and shows your assembly file.
Update your README

Your README.md currently has placeholder text. A good README answers three questions: what does this project do, why does it matter, and how do I run it.

  • Open README.md in your text editor and replace its entire contents with the following:
# Quantized Inference Engine in RISC-V Assembly

A 2-layer quantized neural network inference engine written entirely in RISC-V assembly (RV32IM). This program classifies a 4-element input vector into one of 3 classes using INT8 quantized weights, 32-bit accumulation, fixed-point rescaling via arithmetic right-shift, and ReLU activation.

## Why Quantized Inference?

Modern inference accelerators (RISC-V XMPI, ARM NEON) run neural networks using integer arithmetic instead of floating-point. INT8 quantization shrinks models by 4x and enables faster multiply-accumulate operations on constrained hardware. This project implements the same math those accelerators perform, one instruction at a time.

## How to Run

1. Open [Venus RISC-V Simulator](https://venus.cs61c.org/)
2. Paste the contents of `quantized_inference.s` into the Editor tab
3. Switch to the **Simulator** tab
4. Click **Assemble & Simulate from Editor**
5. Click **Run** to execute, or **Step** to watch registers change instruction by instruction

## Architecture

- **Input**: 4-element INT8 vector
- **Layer 1**: 3 neurons, each computing a dot product over the input, rescaled and passed through ReLU
- **Layer 2**: 3 output neurons computing scores from hidden activations
- **Output**: Argmax of layer 2 scores (predicted class: 0, 1, or 2)

What does this README cover?

  • The opening paragraph answers "what does this do" with concrete technical detail.
  • The "Why Quantized Inference?" section explains why this matters in the real world.
  • The "How to Run" section gives step-by-step instructions for anyone who finds your repo.
  • The "Architecture" section maps the data flow so readers understand the design at a glance.
  • Save README.md.
  • Confirm the file saved correctly by running:
cat README.md

You should see your full project description printed to the terminal, starting with the heading and ending with the Architecture section.

Create a GitHub repository and push your code

Your local Git repository already has your files tracked. Now you need a remote repository on GitHub to push to.

  • Open your browser and go to github.com.
  • Sign in to your GitHub account.
  • Click the + icon in the top-right corner, then select New repository.
  • Enter risc-v-quantized-inference as the repository name.
  • Leave the repository set to Public.
  • Do NOT check any of the initialization boxes (no README, no .gitignore, no license). You already have local files ready to push.
  • Click Create repository.

GitHub shows a quick setup page with a URL for your new empty repository. Copy the HTTPS URL shown on that page.

  • Stage and commit all your files by running these commands in your terminal:
git add .
git commit -m "Add quantized inference engine in RISC-V assembly"

You should see output listing README.md and quantized_inference.s as committed files.

  • Connect your local repo to GitHub and push by running these commands:
git remote add origin [[REPO_URL="https://github.com/your-username/risc-v-quantized-inference.git"]]
git branch -M main
git push -u origin main

What do these commands do?

  • git remote add origin tells Git where to push your code. The URL points to the empty repo you just created.
  • git branch -M main renames your current branch to main (GitHub's default branch name).
  • git push -u origin main uploads your committed files to GitHub and sets origin main as the default push target.

After the push completes, you should see output confirming that your branch was pushed to origin/main.

Push rejected or authentication failed?

  • If you see a permission error, make sure you are signed in to the correct GitHub account. GitHub now requires a personal access token or SSH key instead of a password.
  • If the push is rejected because of "non-fast-forward", you may have accidentally initialized the repo with a README on GitHub. Delete the GitHub repo, recreate it without initialization, and try pushing again.
  • help me push my code to GitHub
Verify your repository on GitHub

Before you check, what do you expect to see when you visit your repository URL? Think about which files should appear and what the front page should display.

  • Visit your repository URL in your browser (e.g., https://github.com/your-username/risc-v-quantized-inference).

You should see your README rendered below the file list, with the heading "Quantized Inference Engine in RISC-V Assembly" and all four sections visible. The file list shows both README.md and quantized_inference.s.

  • Click on quantized_inference.s in the file list to confirm your full assembly code is there.

README not rendering or file missing?

  • If the README does not render, check that your file is named exactly README.md (case-sensitive). GitHub only auto-renders files with that exact name.
  • If a file is missing, run git status locally. If it shows untracked files, run git add . again, commit, and push.
  • help me fix my GitHub repository

Secret mission

Implement Saturating Arithmetic

Real inference hardware doesn't just overflow silently. It clamps. In this mission, you'll implement saturating arithmetic that keeps values within INT8 bounds, the same pattern used in production accelerators like RISC-V XMPI and ARM NEON.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally and uses a free GitHub repository, so there are no ongoing costs.

Decide whether to keep your resources or delete them entirely.

Resources you used:

  • Local project folder on your Desktop (containing quantized_inference.s and README.md).
  • GitHub repository (free, no storage cost).
  • Venus RISC-V Simulator (browser-based, stores nothing server-side).

Keep everything running

No action needed. Choose this if you want to keep your project files and GitHub repository available for reference or future experimentation.

  • Your local project folder stays on your Desktop.
  • Your GitHub repository remains public and free to host indefinitely.
  • Venus stores nothing on its servers, so there is nothing persistent from the simulator.
  • Git remains installed on your machine.

Pause - I'll come back to this later

There is nothing to pause for this project. Your GitHub repository has no running processes, and Venus stores no server-side state. Your files sit on disk until you decide to delete them.

  • Close any open terminal windows from the project.
  • Close the Venus browser tab.
  • Your files and GitHub repository will be waiting when you come back.

Delete - I don't want to use this again

Remove all project files and the remote repository to start completely fresh.

  • Delete your local project folder by running this command in your terminal:

macOS

rm -rf ~/Desktop/quantized-inference-riscv

Windows

rmdir /s /q "$HOME\Desktop\quantized-inference-riscv"
  • Delete your GitHub repository by following these steps:
  • Navigate to your repository's main page on GitHub.
  • Click Settings (under the repository name).
  • Scroll down to the Danger Zone section.
  • Click Delete this repository.
  • Click I want to delete this repository.
  • Click I have read and understand these effects.
  • Type your repository name to confirm, then click Delete this repository.

Nice Work!

Nice Work!

Nice work! You just built a quantized neural network inference engine entirely in RISC-V assembly, running every multiply, accumulate, shift, and comparison at the register level.

You've learned how to:

  • Build an INT8 quantized dot product in RISC-V assembly, watching every multiply and accumulate happen instruction by instruction in a 32-bit register.
  • Implement fixed-point rescaling via arithmetic right-shift (sra), understanding why real inference hardware replaces expensive division with bit-shifting.
  • Wire a complete 2-layer quantized neural network that classifies inputs end-to-end, with ReLU activation and argmax for class prediction.
  • Secret Mission: Implemented saturating arithmetic to clamp accumulator overflow to [-128, 127], the same pattern used in production inference accelerators like RISC-V XMPI and ARM NEON.

Ready to quiz yourself?