GPU Video Processor with wgpu and Flutter

Decode video frames in Rust and render them through a custom wgpu GPU pipeline in Flutter.

Introduction

30 Second Summary

Video apps process millions of pixels every frame. CPUs handle them one at a time, but GPUs can crunch thousands of pixels in parallel. That parallel power is what makes real-time video effects possible.

In this project, you will build a Flutter desktop app that decodes a video frame with Rust and FFmpeg, uploads it to the GPU as a wgpu texture, and renders it through a custom WGSL shader that applies a grayscale post-processing effect.

What You'll Build

You run the app, a video frame appears inside your Flutter window, and it is visibly transformed by a grayscale shader you wrote from scratch.

By the end of this project, you'll have:

  • A running Flutter desktop app where you load a video file and see its first frame rendered through your own custom GPU pipeline.
  • A complete wgpu render pipeline you assembled from scratch. You can trace the full path from vertex buffer to texture upload to sampler to bind group to shader output.
  • A visible grayscale post-processing effect that proves the GPU is actively transforming every pixel through your WGSL fragment shader.
  • Secret Mission: Wire a Flutter slider directly to your shader's uniform buffer so you can dial the grayscale effect between full color and monochrome in real time.

Are there any prerequisites?

This project assumes you are comfortable with Rust and Flutter. You should have the Rust toolchain, Flutter SDK, and FFmpeg dev libraries already installed on Arch Linux with an Intel GPU.

Step 1 walks you through installing Vulkan drivers and scaffolding the project. No prior GPU programming experience is required.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters. You're going to build a Flutter desktop app where Rust decodes video frames and processes them on the GPU through a custom wgpu shader pipeline.

Set Up Vulkan and Scaffold the Project

Your GPU video frame processor needs two things before any code runs: a working Vulkan graphics backend and a Flutter project wired to Rust.

Without Vulkan drivers, wgpu has no GPU backend to render through. Without the flutter_wgpu_texture plugin, Flutter has no way to display GPU-rendered content. In this step, you install both and prove the pipeline works by running a built-in spinning 3D cube inside Flutter.

In this step, get ready to:
  • Install Vulkan drivers for Intel and verify GPU access.
  • Scaffold a Flutter + Rust project and add the flutter_wgpu_texture plugin.
  • Wire up the Dart side and run the spinning cube demo.
Install Vulkan drivers for Intel on Arch Linux

wgpu needs a Vulkan driver to talk to your GPU. On Arch Linux with an Intel GPU, you need two packages: the Intel Vulkan driver and the Vulkan loader.

  • Install the Vulkan packages by running this command:
sudo pacman -S vulkan-intel vulkan-icd-loader vulkan-tools

What do these packages do?

  • vulkan-intel is the open-source Vulkan driver for Intel GPUs. It tells the system how to send GPU commands to your specific hardware.
  • vulkan-icd-loader is the Vulkan loader that discovers installed drivers and routes API calls to them.
  • vulkan-tools provides vulkaninfo, a diagnostic tool you will use to verify GPU access.

Before you run the next command, what do you expect to see if Vulkan is working correctly?

  • Verify that Vulkan can see your GPU by running:
vulkaninfo --summary

You should see output listing your Intel GPU under GPU id with a device name and a Vulkan API version.

Seeing an error about no valid GPUs?

  • Make sure your kernel is up to date. Run uname -r and pacman -Q linux to check that the running kernel matches the installed kernel package. Reboot if they differ.
  • Check for conflicting AMD Vulkan drivers. Run pacman -Qs amdvlk and remove any AMD packages that appear.

Still stuck? Help me debug why vulkaninfo cannot find my Intel GPU on Arch Linux.

Scaffold the Flutter project and add the plugin

Now that Vulkan works, you need a Flutter project that can compile and load Rust code. The flutter_rust_bridge codegen tool scaffolds a project with both a Dart frontend and a Rust crate, plus all the glue to connect them.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Install the flutter_rust_bridge_codegen tool by running:
cargo install flutter_rust_bridge_codegen

Why does this take a while?

This command compiles the codegen tool from source. The first install can take several minutes depending on your machine. Subsequent installs are faster because compiled artifacts are cached.

  • Scaffold your Flutter + Rust project by running:
flutter_rust_bridge_codegen create gpu_video_processor
  • Move into your new project directory by running:
cd gpu_video_processor
  • Confirm the project was scaffolded by running:
ls

You should see standard Flutter directories like lib/, linux/, and a pubspec.yaml file.

Next, add the flutter_wgpu_texture plugin and its build dependencies to your project.

  • Open pubspec.yaml in your code editor.
  • Find the dependencies: section and add these three packages below the existing entries:
# GPU texture plugin for wgpu-to-Flutter rendering
flutter_wgpu_texture: ^0.2.1
# Build hooks for native asset compilation
hooks: ^1.0.2
# Rust build toolchain for Flutter
native_toolchain_rust: ^1.0.3

What do these dependencies do?

  • flutter_wgpu_texture is the plugin that bridges wgpu's Vulkan render output to Flutter's Texture widget. It handles zero-copy GPU surface sharing via DMA-BUF on Linux.
  • hooks and native_toolchain_rust power the build hook that compiles your Rust code into a shared library every time Flutter builds. You will use these in the next step when you create a custom scene.
  • Save pubspec.yaml.
  • Fetch the new dependencies by running:
flutter pub get

You should see output confirming the packages resolved successfully with no errors.

Dependency resolution failed?

Double check that you added the three dependencies under the dependencies: key (not dev_dependencies:) and that the YAML indentation uses exactly two spaces per level.

Need more help? Help me fix my pubspec.yaml dependency resolution error.

Wire up the Dart side and run the spinning cube

The flutter_wgpu_texture plugin ships with a built-in spinning cube scene. Displaying it proves that the full pipeline works: Vulkan drivers, wgpu rendering, and Flutter texture compositing.

  • Open lib/main.dart in your code editor.
  • Replace the entire contents of the file with this code:
import 'package:flutter/material.dart';
import 'package:flutter_wgpu_texture/flutter_wgpu_texture.dart';

void main() {
  runApp(const VideoProcessorApp());
}

// App shell with dark theme
class VideoProcessorApp extends StatelessWidget {
  const VideoProcessorApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'GPU Video Processor',
      theme: ThemeData.dark(useMaterial3: true),
      home: const VideoProcessorPage(),
    );
  }
}

What does this code do?

  • The import pulls in the flutter_wgpu_texture plugin so you can use its controller and texture widget.
  • VideoProcessorApp creates a MaterialApp with a dark theme. This is the app shell that holds your GPU-rendered content.
  • VideoProcessorPage is the page widget you will define in the next code block.
  • Add the VideoProcessorPage widget below the VideoProcessorApp class in the same file:
class VideoProcessorPage extends StatefulWidget {
  const VideoProcessorPage({super.key});

  @override
  State<VideoProcessorPage> createState() =>
      _VideoProcessorPageState();
}

class _VideoProcessorPageState
    extends State<VideoProcessorPage> {
  // Connect to the built-in cube scene
  final controller =
      FlutterWgpuTextureController(sceneType: 'cube');

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('GPU Video Frame Processor'),
      ),
      body: Padding(
        padding: const EdgeInsets.all(16.0),
        // Display the wgpu render output
        child: FlutterWgpuTexture(
            controller: controller),
      ),
    );
  }
}

What does this code do?

  • FlutterWgpuTextureController(sceneType: 'cube') tells the plugin to load its built-in spinning cube scene. The sceneType string matches a scene registered in the plugin's Rust code.
  • FlutterWgpuTexture(controller: controller) is a Flutter widget that displays whatever wgpu renders. Under the hood, it uses Flutter's Texture widget with zero-copy DMA-BUF surface sharing on Linux.
  • Save lib/main.dart.

✔️ Awesome, I've got everything!

Great. Double check you have saved lib/main.dart before running the app.

ⓧ I'd like to double check the full code

Your lib/main.dart should contain both code blocks above, in order. Scroll up and confirm you have: the two imports, main(), VideoProcessorApp, VideoProcessorPage, and _VideoProcessorPageState with sceneType: 'cube'.

Before you run the app, what do you expect to see if the entire Vulkan-to-Flutter pipeline is working?

  • Launch the app by running:
flutter run -d linux

What should I see?

The first build compiles both the Dart code and the Rust shared library. This can take a few minutes.

Once it finishes, a Flutter window opens with a dark theme and the title GPU Video Frame Processor. Inside the window, you should see a spinning 3D cube rendered by wgpu through Vulkan.

Seeing a blank window or a build error?

  • If the build fails with a Rust compilation error, make sure cargo is on your PATH. Run rustc --version to confirm Rust is installed.
  • If the window opens but shows black, check that vulkaninfo --summary still works. A missing or misconfigured Vulkan driver causes wgpu to silently fail.
  • If you see a format mismatch error mentioning texture formats, this is a known wgpu diagnostic that usually resolves with a driver update.

Still stuck? Help me debug why the flutter_wgpu_texture spinning cube is not rendering on Linux.

Your GPU pipeline is live. Vulkan talks to your Intel GPU, wgpu renders through it, and Flutter composites the result into a native window. Next up, you will replace that built-in cube with your own custom wgpu scene.

Create Your Custom wgpu Scene

The spinning cube from Step 1 proves that wgpu can render into Flutter's texture surface. But that cube is someone else's scene. You cannot control what it draws, swap its shader, or feed it video frame data.

To render your own content on the GPU, you need your own Scene implementation. In this step, you will build a Rust workspace with a custom VideoScene that draws a solid teal quad through a WGSL shader and a full wgpu render pipeline.

In this step, get ready to:
  • Set up the Rust workspace structure with a bridge crate and a video_scene subcrate.
  • Implement the Scene trait with a WGSL shader and a full-screen quad vertex buffer.
  • Connect the custom scene to Flutter and see a solid teal quad replace the spinning cube.
Set up the Rust workspace structure

The flutter_wgpu_texture plugin expects a Rust workspace crate named custom_scene_bridge. This bridge crate re-exports the plugin's engine symbols and pulls in your custom scene subcrate, so the loader finds everything in one shared library.

  • Create the workspace directory structure by running this command from your project root:
mkdir -p rust/src rust/video_scene/src hook
  • Confirm the directories exist by running:
ls rust/ rust/video_scene/

You should see src and video_scene under rust/, and src under rust/video_scene/.

  • Create the workspace root manifest by pasting this into rust/Cargo.toml:
[workspace]
members = ["video_scene"]

[package]
name = "custom_scene_bridge"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["staticlib", "cdylib"]

[dependencies]
flutter_wgpu_texture_engine = { git = "https://github.com/flowsai/flutter_wgpu_texture", package = "flutter_wgpu_texture" }
video_scene = { path = "video_scene" }

What does this manifest do?

The [workspace] section declares video_scene as a member crate. The bridge package itself is named custom_scene_bridge so the plugin's loader can find the shared library.

The crate-type line tells Cargo to produce both a static library and a dynamic library (cdylib). The dynamic library is what Flutter loads at runtime.

  • Create the bridge library source by pasting this into rust/src/lib.rs:
pub use flutter_wgpu_texture_engine::*;
  • Create the build script by pasting this into rust/build.rs:
fn main() {
    let target_os = std::env::var("CARGO_CFG_TARGET_OS").unwrap_or_default();
    match target_os.as_str() {
        "macos" | "ios" => {
            println!("cargo:rustc-link-arg-cdylib=-Wl,-all_load");
        }
        "linux" | "android" => {
            println!("cargo:rustc-link-arg-cdylib=-Wl,--whole-archive");
            println!("cargo:rustc-link-arg-cdylib=-Wl,--no-whole-archive");
        }
        _ => {}
    }
}

Why does the build script need linker flags?

The --whole-archive flag forces the linker to include every symbol from the static libraries, even if nothing references them directly. Without it, the linker would strip the scene registration function because nothing calls it from the bridge crate itself.

  • Save both rust/src/lib.rs and rust/build.rs.
  • Confirm the files exist by running:
ls rust/src/ rust/

You should see lib.rs inside rust/src/, and build.rs, Cargo.toml, src, and video_scene listed under rust/.

  • Pin the Rust toolchain by pasting this into rust/rust-toolchain.toml:
[toolchain]
channel = "1.92.0"
targets = [
  "x86_64-unknown-linux-gnu",
]
  • Create the subcrate manifest by pasting this into rust/video_scene/Cargo.toml:
[package]
name = "video_scene"
version = "0.1.0"
edition = "2021"

[dependencies]
flutter_wgpu_texture_core = "0.2"
wgpu = "0.19"
ctor = "0.2"
bytemuck = { version = "1", features = ["derive"] }

Why wgpu 0.19?

The flutter_wgpu_texture_core crate version 0.2 depends on wgpu ^0.19. The Scene trait's method signatures use wgpu types, so your subcrate must use the exact same major version. Using a different version would cause type mismatches at compile time.

  • Save both rust/rust-toolchain.toml and rust/video_scene/Cargo.toml.
  • Confirm the subcrate structure by running:
ls rust/video_scene/

You should see Cargo.toml and src listed.

  • Create the Flutter native build hook by pasting this into hook/build.dart:
import 'package:hooks/hooks.dart';
import 'package:native_toolchain_rust/native_toolchain_rust.dart';

void main(List<String> args) async {
  await build(args, (input, output) async {
    await RustBuilder(
      assetName: 'flutter_wgpu_texture',
      cratePath: 'rust',
    ).run(input: input, output: output);
  });
}

What does this build hook do?

Flutter's native build system calls this hook during compilation. The RustBuilder compiles your rust/ workspace into a shared library and bundles it as the flutter_wgpu_texture asset, replacing the plugin's default library with yours.

  • Save hook/build.dart.
  • Confirm the hook file exists by running:
ls hook/

You should see build.dart listed.

Missing files or directories?

Make sure you ran the mkdir -p command from your project root (the folder containing pubspec.yaml). If any directory is missing, re-run the mkdir command and create the file again.

Help me debug my project directory structure.

Implement the Scene trait

Every custom scene must implement the Scene trait from flutter_wgpu_texture_core. The trait has one required method, render(), which receives a command encoder and a render target from the plugin each frame.

  • Open rust/video_scene/src/lib.rs in your editor. Start by adding the imports, the Vertex struct, and the full-screen quad vertices:
use flutter_wgpu_texture_core::{register_scene, Scene, SceneRenderArgs};
use wgpu::util::DeviceExt;

// A single vertex with 2D position and texture coordinates
#[repr(C)]
#[derive(Copy, Clone, bytemuck::Pod, bytemuck::Zeroable)]
struct Vertex {
    position: [f32; 2],
    tex_coords: [f32; 2],
}

// Six vertices forming two triangles that cover the full screen
const VERTICES: &[Vertex] = &[
    Vertex { position: [-1.0, -1.0], tex_coords: [0.0, 1.0] },
    Vertex { position: [ 1.0, -1.0], tex_coords: [1.0, 1.0] },
    Vertex { position: [ 1.0,  1.0], tex_coords: [1.0, 0.0] },
    Vertex { position: [-1.0, -1.0], tex_coords: [0.0, 1.0] },
    Vertex { position: [ 1.0,  1.0], tex_coords: [1.0, 0.0] },
    Vertex { position: [-1.0,  1.0], tex_coords: [0.0, 0.0] },
];

What does this code do?

The Vertex struct defines what data the GPU receives per vertex: a 2D screen position and a texture coordinate. The #[repr(C)] attribute ensures the struct layout matches what the GPU expects.

The six VERTICES form two triangles arranged into a rectangle that spans the entire render surface from corner to corner. The texture coordinates will matter later when you sample a video frame.

  • Below the VERTICES constant, add the WGSL shader source:
// WGSL shader: pass-through vertex stage and solid teal fragment output
const SHADER_SRC: &str = r#"
struct VertexInput {
    @location(0) position: vec2<f32>,
    @location(1) tex_coords: vec2<f32>,
}

struct VertexOutput {
    @builtin(position) clip_position: vec4<f32>,
    @location(0) tex_coords: vec2<f32>,
}

@vertex
fn vs_main(input: VertexInput) -> VertexOutput {
    var output: VertexOutput;
    output.clip_position = vec4<f32>(input.position, 0.0, 1.0);
    output.tex_coords = input.tex_coords;
    return output;
}

@fragment
fn fs_main(input: VertexOutput) -> @location(0) vec4<f32> {
    return vec4<f32>(0.0, 0.5, 0.5, 1.0);
}
"#;

What does this shader do?

The vertex function vs_main passes each vertex position straight through to clip space. The fragment function fs_main returns a solid teal color (R=0.0, G=0.5, B=0.5) for every pixel. You will replace this solid color with texture sampling in the next step.

  • Save rust/video_scene/src/lib.rs and verify the dependencies resolve by running:
cd rust && cargo check -p video_scene && cd ..

Cargo downloads and compiles the dependencies. You should see a successful check with no errors.

Seeing dependency resolution errors?

Double-check that rust/video_scene/Cargo.toml uses flutter_wgpu_texture_core = "0.2" and wgpu = "0.19". A version mismatch between these two crates causes compile failures because the Scene trait methods use wgpu types directly.

Help me fix Cargo dependency errors in my wgpu project.

  • Return to rust/video_scene/src/lib.rs in your editor. Below the SHADER_SRC constant, add the VideoScene struct and the start of its constructor:
pub struct VideoScene {
    render_pipeline: wgpu::RenderPipeline,
    vertex_buffer: wgpu::Buffer,
}

impl VideoScene {
    pub fn new(device: &wgpu::Device) -> Result<Self, String> {
        // Create the shader module from WGSL source
        let shader = device.create_shader_module(wgpu::ShaderModuleDescriptor {
            label: Some("Video Shader"),
            source: wgpu::ShaderSource::Wgsl(SHADER_SRC.into()),
        });

        // No bind groups needed yet
        let pipeline_layout = device.create_pipeline_layout(&wgpu::PipelineLayoutDescriptor {
            label: Some("Video Pipeline Layout"),
            bind_group_layouts: &[],
            push_constant_ranges: &[],
        });

        // Upload the full-screen quad vertices to GPU memory
        let vertex_buffer = device.create_buffer_init(&wgpu::util::BufferInitDescriptor {
            label: Some("Vertex Buffer"),
            contents: bytemuck::cast_slice(VERTICES),
            usage: wgpu::BufferUsages::VERTEX,
        });

What does this code do?

The VideoScene struct holds the two GPU resources it needs to draw: a render pipeline and a vertex buffer. The constructor creates the shader module from the WGSL source, an empty pipeline layout (you will add bind groups later for textures), and uploads the quad vertices to GPU memory.

  • Directly below the vertex buffer creation (still inside the new() function), add the render pipeline descriptor with the vertex stage:
        // Assemble the render pipeline
        let render_pipeline = device.create_render_pipeline(&wgpu::RenderPipelineDescriptor {
            label: Some("Video Render Pipeline"),
            layout: Some(&pipeline_layout),
            vertex: wgpu::VertexState {
                module: &shader,
                entry_point: "vs_main",
                buffers: &[wgpu::VertexBufferLayout {
                    array_stride: std::mem::size_of::<Vertex>() as wgpu::BufferAddress,
                    step_mode: wgpu::VertexStepMode::Vertex,
                    attributes: &[
                        wgpu::VertexAttribute {
                            offset: 0,
                            shader_location: 0,
                            format: wgpu::VertexFormat::Float32x2,
                        },
                        wgpu::VertexAttribute {
                            offset: std::mem::size_of::<[f32; 2]>() as wgpu::BufferAddress,
                            shader_location: 1,
                            format: wgpu::VertexFormat::Float32x2,
                        },
                    ],
                }],
            },

What is the vertex buffer layout?

The VertexBufferLayout tells the GPU how to read your Vertex struct. The array_stride is the byte size of one vertex. The two attributes map to @location(0) (position) and @location(1) (texture coordinates) in the shader.

  • Directly below the vertex state (still inside the same create_render_pipeline call), add the fragment state, primitive configuration, and close the constructor:
            fragment: Some(wgpu::FragmentState {
                module: &shader,
                entry_point: "fs_main",
                targets: &[Some(wgpu::ColorTargetState {
                    format: wgpu::TextureFormat::Bgra8UnormSrgb,
                    blend: Some(wgpu::BlendState::REPLACE),
                    write_mask: wgpu::ColorWrites::ALL,
                })],
            }),
            primitive: wgpu::PrimitiveState {
                topology: wgpu::PrimitiveTopology::TriangleList,
                ..Default::default()
            },
            depth_stencil: None,
            multisample: wgpu::MultisampleState::default(),
            multiview: None,
        });

        Ok(Self {
            render_pipeline,
            vertex_buffer,
        })
    }
}

Why Bgra8UnormSrgb?

The Bgra8UnormSrgb format matches the Flutter texture surface format on Linux. If this format does not match, wgpu reports a clear error at pipeline creation time about incompatible color attachments.

  • Below the closing } of the impl VideoScene block, add the Scene trait implementation:
impl Scene for VideoScene {
    fn render(
        &mut self,
        args: &SceneRenderArgs,
        encoder: &mut wgpu::CommandEncoder,
    ) -> Result<(), String> {
        // Begin a render pass targeting the Flutter texture surface
        let mut render_pass = encoder.begin_render_pass(&wgpu::RenderPassDescriptor {
            label: Some("Video Render Pass"),
            color_attachments: &[Some(wgpu::RenderPassColorAttachment {
                view: args.view,
                resolve_target: None,
                ops: wgpu::Operations {
                    load: wgpu::LoadOp::Clear(args.clear_color),
                    store: wgpu::StoreOp::Store,
                },
            })],
            depth_stencil_attachment: None,
            timestamp_writes: None,
            occlusion_query_set: None,
        });

        // Draw the full-screen quad using our pipeline
        render_pass.set_pipeline(&self.render_pipeline);
        render_pass.set_vertex_buffer(0, self.vertex_buffer.slice(..));
        render_pass.draw(0..6, 0..1);

        Ok(())
    }
}

What does the render method do?

Each frame, the plugin calls render() with a command encoder and a SceneRenderArgs containing the target texture view. The method opens a render pass, binds your pipeline and vertex buffer, and draws six vertices (two triangles forming the full-screen quad).

  • Finally, below the Scene impl block, add the scene registration function:
// Register this scene so Flutter can find it by name
#[ctor::ctor]
fn _register() {
    register_scene("video_scene", |device, _w, _h| {
        VideoScene::new(device).map(|s| Box::new(s) as Box<dyn Scene>)
    });
}

What is #[ctor::ctor]?

The #[ctor::ctor] attribute generates a dynamic library initializer that runs automatically when the shared library loads. This registers your scene under the name "video_scene" before any Dart code executes, so the plugin can instantiate it by name.

  • Save rust/video_scene/src/lib.rs.
  • Verify the entire workspace compiles by running:
cd rust && cargo check && cd ..

Cargo compiles both the video_scene crate and the custom_scene_bridge root. You should see a successful check with no errors.

Compilation fails?

If you see a type mismatch error mentioning SceneRenderArgs or CommandEncoder, check that wgpu = "0.19" in rust/video_scene/Cargo.toml. Any other wgpu version produces incompatible types.

If the error mentions an unresolved import, make sure rust/src/lib.rs contains exactly pub use flutter_wgpu_texture_engine::*; and that the root rust/Cargo.toml has the flutter_wgpu_texture_engine git dependency.

Help me fix my wgpu Scene compilation error.

✔️ Awesome, I've got everything!

Great. Make sure you have saved rust/video_scene/src/lib.rs and that cargo check passes with no errors.

ⓧ I'd like to double check the full code

Scroll up through your rust/video_scene/src/lib.rs and confirm these sections appear in order:

  • Two use imports from flutter_wgpu_texture_core and wgpu::util::DeviceExt.
  • The Vertex struct and VERTICES constant (six vertices).
  • The SHADER_SRC constant with vs_main and fs_main functions (fragment returns solid teal).
  • The VideoScene struct with render_pipeline and vertex_buffer fields.
  • The new() constructor creating the shader, pipeline layout, vertex buffer, and render pipeline.
  • The impl Scene for VideoScene with a render() method that draws 6 vertices.
  • The #[ctor::ctor] function registering "video_scene".
Connect the custom scene to Flutter and run the app

The Rust side is ready. Now you need to tell the Dart side to request your scene instead of the built-in cube.

  • Switch back to lib/main.dart in your editor.
  • Find this line:
final controller = FlutterWgpuTextureController(sceneType: 'cube');
  • Replace 'cube' with 'video_scene', so the line now looks like this:
final controller = FlutterWgpuTextureController(sceneType: 'video_scene');
  • Save lib/main.dart.

Before you run the app, what do you expect to see instead of the spinning cube?

  • Run the app by executing:
flutter run -d linux

The first build takes longer because Flutter compiles your Rust workspace through the build hook. Once the app launches, you should see a solid teal-colored quad filling the texture widget where the spinning cube used to be.

What just happened?

Flutter loaded your custom_scene_bridge shared library. The #[ctor::ctor] initializer ran and registered your VideoScene under the name "video_scene". When Dart created the controller with sceneType: 'video_scene', the plugin looked up that name, instantiated your scene, and started calling render() each frame. The solid teal proves your GPU pipeline is running.

Still seeing the cube or a blank window?

If the spinning cube still appears, the build hook may not be compiling your custom crate. Make sure hook/build.dart exists with assetName: 'flutter_wgpu_texture' and cratePath: 'rust'. Try stopping the app and re-running flutter run -d linux from scratch.

If you see a black or blank window, check that sceneType: 'video_scene' in lib/main.dart matches the string "video_scene" in your register_scene call exactly. A mismatch means the plugin cannot find your scene.

Help me figure out why my custom scene is not loading.

Your custom scene is rendering through wgpu into Flutter's texture surface. The teal quad is your GPU pipeline in action. Next up, you will decode a real video frame with FFmpeg and display it through this pipeline.

Decode a Video Frame and Display It as a wgpu Texture

In the previous step you proved that your custom wgpu scene can render into Flutter's texture widget. That teal quad was your proof of life.

The teal quad is static and hardcoded. The whole point of this project is to process real video frames on the GPU. In this step you will decode a frame from a video file using FFmpeg, upload it as a wgpu texture, and sample it in your shader so the actual video image appears inside Flutter.

In this step, get ready to:
  • Add FFmpeg decoding to the video scene crate and extract pixel data from the first frame.
  • Create wgpu texture resources from the decoded frame data and store them for GPU upload.
  • Update the WGSL shader, bind group, and render method to display the video frame inside Flutter.
Add the FFmpeg dependency and write the frame decoder

The ffmpeg-next crate provides safe Rust bindings to the FFmpeg C libraries that are already installed on your system. You need it to open a video file, find the video stream, decode a frame, and convert the pixels to RGBA format that wgpu can consume.

  • In rust/video_scene/Cargo.toml, find the [dependencies] section. It currently looks like this:
[dependencies]
flutter_wgpu_texture_core = "0.2"
wgpu = "0.19"
ctor = "0.2"
bytemuck = { version = "1", features = ["derive"] }
  • Add ffmpeg-next = "8.1" so the section now looks like this:
[dependencies]
flutter_wgpu_texture_core = "0.2"
wgpu = "0.19"
ctor = "0.2"
ffmpeg-next = "8.1"
bytemuck = { version = "1", features = ["derive"] }
  • Save rust/video_scene/Cargo.toml.

Now write the function that decodes the first video frame. In rust/video_scene/src/lib.rs, add the following decode_first_frame function above your existing VideoScene struct. Start with the top half that initialises FFmpeg, opens the file, and sets up the decoder:

fn decode_first_frame(path: &str) -> Result<(Vec<u8>, u32, u32), String> {
    // Initialize FFmpeg libraries
    ffmpeg_next::init().map_err(|e| format!("FFmpeg init failed: {e}"))?;

    // Open the video file
    let mut ictx = ffmpeg_next::format::input(&path)
        .map_err(|e| format!("Cannot open video file: {e}"))?;

    // Find the best video stream in the container
    let input = ictx
        .streams()
        .best(ffmpeg_next::media::Type::Video)
        .ok_or_else(|| "No video stream found".to_string())?;
    let video_stream_index = input.index();

    // Create a decoder from the stream's codec parameters
    let context_decoder =
        ffmpeg_next::codec::context::Context::from_parameters(input.parameters())
            .map_err(|e| format!("Codec context error: {e}"))?;
    let mut decoder = context_decoder
        .decoder()
        .video()
        .map_err(|e| format!("Decoder error: {e}"))?;

    let width = decoder.width();
    let height = decoder.height();

Continue adding code inside the same function. This section creates a pixel format scaler and iterates packets until a frame is decoded:

    // Create a scaler to convert decoded pixels to RGBA
    let mut scaler = ffmpeg_next::software::scaling::Context::get(
        decoder.format(),
        width,
        height,
        ffmpeg_next::format::Pixel::RGBA,
        width,
        height,
        ffmpeg_next::software::scaling::Flags::BILINEAR,
    )
    .map_err(|e| format!("Scaler error: {e}"))?;

    // Iterate packets until we successfully decode a frame
    for (stream, packet) in ictx.packets() {
        if stream.index() != video_stream_index {
            continue;
        }
        decoder
            .send_packet(&packet)
            .map_err(|e| format!("Send packet error: {e}"))?;
        let mut decoded = ffmpeg_next::frame::Video::empty();
        if decoder.receive_frame(&mut decoded).is_ok() {
            let mut rgba_frame = ffmpeg_next::frame::Video::empty();
            scaler
                .run(&decoded, &mut rgba_frame)
                .map_err(|e| format!("Scale error: {e}"))?;

Finish the function with the row-by-row pixel copy and the closing braces:

            // Copy row-by-row to handle FFmpeg stride padding
            let stride = rgba_frame.stride(0);
            let h = height as usize;
            let w = width as usize;
            let mut data = Vec::with_capacity(w * 4 * h);
            for row in 0..h {
                let start = row * stride;
                let end = start + w * 4;
                data.extend_from_slice(&rgba_frame.data(0)[start..end]);
            }
            return Ok((data, width, height));
        }
    }

    Err("Could not decode any video frame".to_string())
}

What does this function do?

decode_first_frame opens a video file, locates the best video stream, and feeds packets to a decoder until a complete frame arrives. The software scaler then converts the frame from whatever native pixel format the codec uses into RGBA, which is what wgpu textures expect.

FFmpeg decoded frames can have padding bytes at the end of each row (the stride may be larger than width * 4). The row-by-row copy strips that padding so the pixel data is tightly packed for write_texture.

  • Save rust/video_scene/src/lib.rs.
  • Verify that the new dependency links correctly by running this from your project root:
cd rust && cargo check && cd ..

You should see a successful compilation. Cargo may print a warning about the unused function, which is expected since nothing calls decode_first_frame yet.

Seeing linker errors about FFmpeg libraries?

The ffmpeg-next crate links against the system FFmpeg libraries via pkg-config. Make sure the ffmpeg package is installed on Arch, which provides both runtime and development headers.

Run pkg-config --libs libavformat to confirm pkg-config can find the FFmpeg libraries. If it cannot, install the ffmpeg package and try again.

Still stuck? Help me debug FFmpeg linker errors when building ffmpeg-next with cargo on Arch Linux.

Create wgpu texture resources from the decoded frame

With the frame decoder ready, the next job is to wire it into the constructor. There is an important constraint here. The register_scene factory only provides the wgpu::Device. The wgpu::Queue (needed to upload pixel data) only becomes available in render() through SceneRenderArgs. So the constructor creates the empty texture and stores the raw pixel data for upload on the first render call.

  • In rust/video_scene/src/lib.rs, replace your existing VideoScene struct with this updated version that includes the new texture fields:
pub struct VideoScene {
    render_pipeline: wgpu::RenderPipeline,
    bind_group: wgpu::BindGroup,
    vertex_buffer: wgpu::Buffer,
    uniform_buffer: wgpu::Buffer,
    effect_strength: f32,
    pending_frame_data: Option<(Vec<u8>, wgpu::Extent3d)>,
    _texture: wgpu::Texture,
}

What are the new fields?

bind_group connects the texture, sampler, and uniform buffer to the shader. uniform_buffer holds shader parameters you will use in the next step. pending_frame_data holds the raw RGBA bytes until the first render() call, when the queue is available. _texture keeps the GPU texture alive for the bind group's lifetime.

Now replace the body of VideoScene::new(). Start by adding the decode call and texture creation at the top of the function, before the existing shader and pipeline code:

impl VideoScene {
    pub fn new(device: &wgpu::Device) -> Result<Self, String> {
        // Decode the first video frame from disk
        let video_path = std::env::var("VIDEO_PATH")
            .unwrap_or_else(|_| "sample.mp4".to_string());
        let (frame_data, width, height) = decode_first_frame(&video_path)?;

        let texture_size = wgpu::Extent3d {
            width,
            height,
            depth_or_array_layers: 1,
        };

        // Create a GPU texture to hold the video frame
        let texture = device.create_texture(&wgpu::TextureDescriptor {
            label: Some("Video Frame Texture"),
            size: texture_size,
            mip_level_count: 1,
            sample_count: 1,
            dimension: wgpu::TextureDimension::D2,
            format: wgpu::TextureFormat::Rgba8UnormSrgb,
            usage: wgpu::TextureUsages::TEXTURE_BINDING | wgpu::TextureUsages::COPY_DST,
            view_formats: &[],
        });

        let texture_view = texture.create_view(&wgpu::TextureViewDescriptor::default());

Continue inside the same function. Add the sampler and uniform buffer:

        // Sampler controls how the shader reads texture pixels
        let sampler = device.create_sampler(&wgpu::SamplerDescriptor {
            label: Some("Video Sampler"),
            address_mode_u: wgpu::AddressMode::ClampToEdge,
            address_mode_v: wgpu::AddressMode::ClampToEdge,
            address_mode_w: wgpu::AddressMode::ClampToEdge,
            mag_filter: wgpu::FilterMode::Linear,
            min_filter: wgpu::FilterMode::Linear,
            ..Default::default()
        });

        // Uniform buffer for shader parameters (used in a later step)
        let effect_strength: f32 = 0.0;
        let uniform_buffer = device.create_buffer_init(&wgpu::util::BufferInitDescriptor {
            label: Some("Uniform Buffer"),
            contents: bytemuck::cast_slice(&[effect_strength]),
            usage: wgpu::BufferUsages::UNIFORM | wgpu::BufferUsages::COPY_DST,
        });

Why ClampToEdge?

ClampToEdge means texture coordinates outside 0.0 to 1.0 repeat the edge pixel instead of wrapping or showing black. For a video frame that fills the entire quad, the sampler never actually goes out of bounds. ClampToEdge is the safe default.

Update the WGSL shader and render pipeline to display the video frame

The shader needs to know about the texture, sampler, and uniform buffer. Replace your existing SHADER_SRC const with this updated version that samples the video texture:

const SHADER_SRC: &str = r#"
struct VertexInput {
    @location(0) position: vec2<f32>,
    @location(1) tex_coords: vec2<f32>,
}
struct VertexOutput {
    @builtin(position) clip_position: vec4<f32>,
    @location(0) tex_coords: vec2<f32>,
}
@vertex
fn vs_main(input: VertexInput) -> VertexOutput {
    var output: VertexOutput;
    output.clip_position = vec4<f32>(input.position, 0.0, 1.0);
    output.tex_coords = input.tex_coords;
    return output;
}
struct Uniforms {
    effect_strength: f32,
}
@group(0) @binding(0) var t_video: texture_2d<f32>;
@group(0) @binding(1) var s_video: sampler;
@group(0) @binding(2) var<uniform> uniforms: Uniforms;
@fragment
fn fs_main(input: VertexOutput) -> @location(0) vec4<f32> {
    let color = textureSample(t_video, s_video, input.tex_coords);
    return color;
}
"#;

What changed in the shader?

The old shader returned a hardcoded teal color. The new version declares three bindings: a 2D texture at binding 0, a sampler at binding 1, and a uniform buffer at binding 2. The fragment shader now calls textureSample to read a pixel from the video texture at the interpolated UV coordinate. The uniform buffer is declared but not yet used in the output. You will use it in the next step.

Back in VideoScene::new(), continue after the uniform buffer. Add the bind group layout that tells wgpu which resource types the shader expects at each binding:

        // Tell wgpu what resource types the shader expects
        let bind_group_layout =
            device.create_bind_group_layout(&wgpu::BindGroupLayoutDescriptor {
                label: Some("Video Bind Group Layout"),
                entries: &[
                    wgpu::BindGroupLayoutEntry {
                        binding: 0,
                        visibility: wgpu::ShaderStages::FRAGMENT,
                        ty: wgpu::BindingType::Texture {
                            multisampled: false,
                            view_dimension: wgpu::TextureViewDimension::D2,
                            sample_type: wgpu::TextureSampleType::Float { filterable: true },
                        },
                        count: None,
                    },
                    wgpu::BindGroupLayoutEntry {
                        binding: 1,
                        visibility: wgpu::ShaderStages::FRAGMENT,
                        ty: wgpu::BindingType::Sampler(wgpu::SamplerBindingType::Filtering),
                        count: None,
                    },

Continue with the third bind group layout entry (the uniform buffer) and then create the bind group that connects actual resources to the layout:

                    wgpu::BindGroupLayoutEntry {
                        binding: 2,
                        visibility: wgpu::ShaderStages::FRAGMENT,
                        ty: wgpu::BindingType::Buffer {
                            ty: wgpu::BufferBindingType::Uniform,
                            has_dynamic_offset: false,
                            min_binding_size: None,
                        },
                        count: None,
                    },
                ],
            });

        // Connect the actual texture, sampler, and buffer to the layout
        let bind_group = device.create_bind_group(&wgpu::BindGroupDescriptor {
            label: Some("Video Bind Group"),
            layout: &bind_group_layout,
            entries: &[
                wgpu::BindGroupEntry {
                    binding: 0,
                    resource: wgpu::BindingResource::TextureView(&texture_view),
                },
                wgpu::BindGroupEntry {
                    binding: 1,
                    resource: wgpu::BindingResource::Sampler(&sampler),
                },
                wgpu::BindGroupEntry {
                    binding: 2,
                    resource: uniform_buffer.as_entire_binding(),
                },
            ],
        });

What is a bind group?

A bind group layout declares the types and bindings the shader expects. A bind group is the concrete instance that connects your actual GPU resources (texture view, sampler, buffer) to those declared slots. Think of the layout as a socket specification and the bind group as the plugged-in cable.

Next, find the existing pipeline_layout creation in your constructor. It currently has an empty bind_group_layouts array. Update it to include your new layout:

  • Find this line in your constructor:
            bind_group_layouts: &[],
  • Replace it with:
            bind_group_layouts: &[&bind_group_layout],

Finally, update the Ok(Self { ... }) return at the bottom of new() to include all the new fields:

        Ok(Self {
            render_pipeline,
            bind_group,
            vertex_buffer,
            uniform_buffer,
            effect_strength,
            pending_frame_data: Some((frame_data, texture_size)),
            _texture: texture,
        })
    }
}

Now update the render() method. At the very top of render(), before the existing render pass code, add this block that uploads the frame data to the GPU on the first call:

        // Upload frame data to the GPU texture on the first render call
        if let Some((data, size)) = self.pending_frame_data.take() {
            args.queue.write_texture(
                wgpu::ImageCopyTexture {
                    texture: &self._texture,
                    mip_level: 0,
                    origin: wgpu::Origin3d::ZERO,
                    aspect: wgpu::TextureAspect::All,
                },
                &data,
                wgpu::ImageDataLayout {
                    offset: 0,
                    bytes_per_row: Some(4 * size.width),
                    rows_per_image: Some(size.height),
                },
                size,
            );
        }

Why upload during render?

The register_scene factory passes only the wgpu::Device to the constructor. The wgpu::Queue is only available through SceneRenderArgs during render(). The take() call moves the data out of the Option, so the upload only happens once. After that, pending_frame_data is None and the block is skipped on every subsequent frame.

  • In the render pass section of render(), find the line render_pass.set_pipeline(&self.render_pipeline); and add this line directly below it:
        render_pass.set_bind_group(0, &self.bind_group, &[]);

This tells the render pass to use your bind group at group index 0, matching the @group(0) declarations in the WGSL shader.

  • Make sure the #[ctor::ctor] registration at the bottom of the file still reads "video_scene". No changes needed there.
  • Save rust/video_scene/src/lib.rs.

Before you run the app, you need a video file. Place any .mp4 file in your project root and name it sample.mp4, or set the VIDEO_PATH environment variable to point to a video file on your system.

Before you run this, what do you expect to see in the Flutter window? Will it still be a teal quad, or something different?

  • Run the app by executing:
VIDEO_PATH=sample.mp4 flutter run -d linux

The teal quad is gone. In its place you should see the first frame of your video file rendered inside the Flutter window. The pixels traveled from disk, through FFmpeg decoding, into a wgpu GPU texture, through your WGSL shader, and into Flutter's Texture widget.

Seeing a blank window or a crash?

If the window is blank, check that your sample.mp4 file exists at the path you specified. The VIDEO_PATH environment variable defaults to sample.mp4 in the current working directory.

If you see a panic about texture formats, double-check that your TextureDescriptor uses Rgba8UnormSrgb and your render pipeline's fragment target uses Bgra8UnormSrgb. The source texture (your decoded frame) and the render target (Flutter's surface) use different formats, which is correct.

If the image looks garbled with diagonal stripes, the stride-handling code in decode_first_frame may have a typo. Make sure you copy w * 4 bytes per row, not the full stride.

Still stuck? Help me debug why my video frame is not showing in the Flutter wgpu texture.

\u2714\uFE0F Awesome, I've got everything!

Great. Double-check you have saved rust/video_scene/Cargo.toml and rust/video_scene/src/lib.rs.

\u24E7 I'd like to double check the full code

The full rust/video_scene/src/lib.rs file is too long to show in one block. Scroll up and confirm these sections are present in order:

  • use imports for flutter_wgpu_texture_core and wgpu::util::DeviceExt.
  • Vertex struct and VERTICES constant (unchanged from Step 2).
  • SHADER_SRC const with textureSample in the fragment shader and three @binding declarations.
  • decode_first_frame() function with init, open, decode, scale, and row-by-row copy.
  • VideoScene struct with render_pipeline, bind_group, vertex_buffer, uniform_buffer, effect_strength, pending_frame_data, _texture.
  • VideoScene::new() calling decode_first_frame, creating texture, view, sampler, uniform buffer, bind group layout (3 entries), bind group, shader, pipeline layout with &[&bind_group_layout], vertex buffer, render pipeline, and Ok(Self { ... }).
  • Scene impl with render() that does write_texture on first call, then draws with set_bind_group(0, ...).
  • #[ctor::ctor] registering "video_scene".

For rust/video_scene/Cargo.toml, confirm it matches:

[package]
name = "video_scene"
version = "0.1.0"
edition = "2021"

[dependencies]
flutter_wgpu_texture_core = "0.2"
wgpu = "0.19"
ctor = "0.2"
ffmpeg-next = "8.1"
bytemuck = { version = "1", features = ["derive"] }

You now have a real video frame flowing through your custom GPU pipeline and landing inside Flutter. The GPU is rendering the frame, but it is a straight passthrough. Next up, you will make the GPU do real work on those pixels by adding a post-processing shader effect.

Add GPU Post-Processing with a WGSL Shader

Your video frame renders inside Flutter, but the GPU is doing nothing a CPU could not do. The frame pixels travel from FFmpeg through wgpu and back out unchanged. That is a passthrough, not processing.

Adding a shader effect makes the GPU's role visible. You will update the WGSL fragment shader to apply a grayscale post-processing effect, proving the processing pipeline works end-to-end.

In this step, get ready to:
  • Update the WGSL fragment shader with grayscale computation using luminance weights.
  • Hit a designed obstacle where the effect is invisible and discover why.
  • Activate the grayscale effect and add runtime parameter support.
Update the fragment shader with grayscale math

The fragment shader currently samples the video texture and returns the color unchanged. You need to add grayscale conversion logic that the GPU executes on every pixel.

  • In rust/video_scene/src/lib.rs, find the fs_main function inside the SHADER_SRC string.
  • Find the current fragment shader body:
let color = textureSample(t_video, s_video, input.tex_coords);
return color;
  • Replace those two lines with the following grayscale computation:
// Sample the video texture
let color = textureSample(t_video, s_video, input.tex_coords);
// Compute luminance from RGB using standard perceptual weights
let gray = dot(color.rgb, vec3<f32>(0.299, 0.587, 0.114));
let grayscale = vec3<f32>(gray, gray, gray);
// Blend between original and grayscale based on the uniform
let result = mix(color.rgb, grayscale, uniforms.effect_strength);
return vec4<f32>(result, color.a);

What does this shader code do?

The dot(color.rgb, vec3<f32>(0.299, 0.587, 0.114)) call computes a weighted average of the red, green, and blue channels. These weights (0.299, 0.587, 0.114) match how the human eye perceives brightness. Green contributes most, red is second, and blue contributes least.

The mix() function linearly interpolates between the original color and the grayscale value. When effect_strength is 0.0, you get the original color. When it is 1.0, you get full grayscale.

  • Save rust/video_scene/src/lib.rs.
Run the app and discover the obstacle

The shader now has grayscale math. Before you run the app, think about what you expect to see. The effect_strength uniform is currently initialized to 0.0. What will mix(color, grayscale, 0.0) return?

  • Run the app by executing:
VIDEO_PATH=sample.mp4 flutter run -d linux

The video frame looks identical to Step 3. Full color, no grayscale at all.

Why does the frame look unchanged?

The shader has the grayscale math, but the uniform is not driving it. mix(color.rgb, grayscale, 0.0) returns the original color because a mix factor of 0.0 means "100% of the first argument, 0% of the second." The effect is present in the code but completely invisible at runtime.

Activate the grayscale effect

The fix is straightforward. Change the default effect_strength to 1.0 so the shader produces full grayscale.

  • In rust/video_scene/src/lib.rs, find this line inside VideoScene::new():
let effect_strength: f32 = 0.0;
  • Change 0.0 to 1.0, so the line now looks like this:
let effect_strength: f32 = 1.0;

The buffer is initialized with this value at creation, so the shader will read 1.0 from the uniform. For dynamic updates later, you also need to write the uniform buffer every frame from render().

  • In your render() method, find the closing brace of the if let Some((data, size)) = self.pending_frame_data.take() block. Add this code immediately after it:
// Write the current effect strength to the GPU uniform buffer each frame
args.queue.write_buffer(
    &self.uniform_buffer,
    0,
    bytemuck::cast_slice(&[self.effect_strength]),
);

What does this code do?

The write_buffer() call pushes the current effect_strength value from Rust into the GPU uniform buffer every frame. Right now the value is static (1.0), but writing it each frame means the shader will pick up changes instantly if the value changes at runtime.

  • Still in rust/video_scene/src/lib.rs, find the impl Scene for VideoScene block. Add the set_float_param method after the closing brace of render():
// Accept float parameter updates from Flutter
fn set_float_param(&mut self, key: &str, value: f32) {
    if key == "effect_strength" {
        self.effect_strength = value.clamp(0.0, 1.0);
    }
}

What does set_float_param do?

This is a trait method from the Scene trait. When Flutter sends a float parameter through the flutter_wgpu_texture bridge, this method receives it. The clamp(0.0, 1.0) ensures the value stays within valid range for the mix() function.

  • Save rust/video_scene/src/lib.rs.

Before you run the app again, what do you expect to see this time? The effect_strength is now 1.0. What does mix(color, grayscale, 1.0) return?

  • Rebuild and run the app:
VIDEO_PATH=sample.mp4 flutter run -d linux

The video frame now renders in full grayscale. The GPU is actively transforming every pixel through your WGSL shader.

Still seeing a full-color frame?

  • Make sure you changed 0.0 to 1.0 in the let effect_strength: f32 line, not somewhere else in the file.
  • Check that the write_buffer() call is placed after the pending frame upload block, not inside it.
  • Verify the shader's fs_main function ends with return vec4<f32>(result, color.a); and not the old return color; line.

Why is my video frame still showing in full color instead of grayscale?

✔️ Awesome, I've got everything!

Great. Double check you have saved rust/video_scene/src/lib.rs with all three changes: the updated shader, the write_buffer() call, and the set_float_param() method.

ⓧ I'd like to double check the full code

The full file is long, so scroll through your code and confirm each section is present in this order:

  • Imports: use flutter_wgpu_texture_core::{register_scene, Scene, SceneRenderArgs}; and use wgpu::util::DeviceExt;.
  • Vertex struct and VERTICES array (6 vertices for a full-screen quad).
  • SHADER_SRC with the updated fs_main that computes gray using dot() and blends with mix().
  • decode_first_frame() function.
  • VideoScene struct and new() with let effect_strength: f32 = 1.0;.
  • render() method with args.queue.write_buffer() after the pending frame upload.
  • set_float_param() method that clamps the value and stores it in self.effect_strength.
  • #[ctor::ctor] function registering "video_scene".

Your GPU is now doing real work on every pixel. The grayscale effect proves the full pipeline is operational: FFmpeg decodes, wgpu textures carry the data, and your WGSL shader transforms it. Next up, you will connect Flutter's UI to this shader so you can control the effect in real time.

Secret mission

Add Real-Time Effect Controls from Flutter

Your grayscale effect looks great, but it is locked at full strength. Can you wire a Flutter Slider to the GPU shader so you can control the effect in real time? This connects three layers: Dart UI, the flutter_rust_bridge FFI layer, and the wgpu uniform buffer.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally on your machine. There are no cloud resources, subscriptions, or ongoing costs to worry about.

Resources you used:

  • Vulkan drivers installed system-wide (vulkan-intel, vulkan-icd-loader). These are system packages and cost nothing.
  • A local Flutter project folder on your Desktop with Rust dependencies compiled into a shared library. No cloud resources or paid services.

Keep everything running

No action needed. Choose this if you want to keep experimenting with your GPU video processor or extend the shader pipeline.

  • Your Flutter project folder stays on your Desktop. You can re-run it any time with flutter run -d linux.
  • The Vulkan drivers (vulkan-intel, vulkan-icd-loader) remain installed. They are useful for any GPU application on your system, not just this project.

Pause - I'll come back to this later

This project has no background processes, cloud resources, or running servers. There is nothing to pause.

  • If the Flutter app is still running, press q in the terminal to quit it.
  • Your project files and Vulkan drivers stay in place. Pick up where you left off by navigating to your project folder and running flutter run -d linux.

Delete - I don't want to use this again

Remove all project files from your machine. The Vulkan drivers are worth keeping since they benefit any GPU application on your system.

  • If the Flutter app is still running, press q in the terminal to quit it.
  • Delete your project folder by running:
cd ~/Desktop
rm -rf [[PROJECT_DIR="your project folder name"]]
  • Confirm the folder is gone by running:
ls ~/Desktop

Your project folder should no longer appear in the output.

  • (Optional) If you also want to remove the Vulkan drivers, run:
sudo pacman -R vulkan-intel vulkan-icd-loader

Keep Vulkan drivers installed

The vulkan-intel and vulkan-icd-loader packages are useful for any GPU application on your system. Removing them may break other Vulkan-based apps. Unless you are certain you do not need Vulkan support, leave them installed.

That's a Wrap!

That's a Wrap!

Nice work! You just built a GPU-powered video frame processor that bridges Rust, wgpu, and Flutter into a single desktop app with real-time shader effects.

You've learned how to:

  • Built a custom wgpu Scene implementation in Rust with a full GPU render pipeline including a vertex buffer, texture, sampler, bind group, and a custom WGSL shader.
  • Decoded a video frame from a file using ffmpeg-next in Rust and uploaded the pixel data as a wgpu GPU texture for shader processing.
  • Connected Flutter's Texture widget to wgpu's Vulkan render output through zero-copy DMA-BUF surface sharing and applied a grayscale post-processing effect using uniform parameters.
  • Secret Mission: Wired real-time Flutter UI controls to GPU shader uniforms through the flutter_wgpu_texture parameter bridge, letting a Slider smoothly transition between full color and grayscale in real time.

Ready to quiz yourself?