Build a Local AI Agent with Flutter

Build a local AI agent in Rust with a Flutter chat UI and tool calling.

Introduction

30 Second Summary

AI assistants like ChatGPT and Claude are powerful, but they send every conversation to remote servers. What if you could build your own AI assistant that runs entirely on your machine, keeping your data completely private?

In this project, you will build a local AI agent with a Rust backend and a Flutter chat interface. Your agent will reason about questions, autonomously decide when to use tools, execute them, and stream answers back to you in real-time.

What You'll Build

A local AI agent application where a Rust HTTP server implements the agentic tool-calling loop, communicates with a local LLM via Ollama, and streams results to a Flutter desktop chat UI through Server-Sent Events.

By the end of this project, you'll have:

  • A Rust HTTP server that implements the full agentic loop: receiving messages, calling the LLM, executing tools, and returning final answers.
  • A tool registry with three built-in tools (get_time, calculate, system_info) that the LLM can autonomously invoke.
  • A Flutter desktop chat UI that streams responses from the Rust backend in real-time, showing tool-call activity as it happens.
  • Secret Mission: Add a web_fetch tool that fetches and summarizes a URL, proving the agent architecture is extensible.

Are there any prerequisites?

This project assumes basic familiarity with Rust and Dart/Flutter syntax. You do not need any prior AI or machine learning experience.

Install Prerequisites

Before you can build a local AI agent, you need three core tools on your machine. Rust powers the agent backend, Flutter provides the desktop chat UI, and Ollama runs a large language model locally so your agent can reason and call tools.

In this step, you will install and verify each tool so the rest of the project has a solid foundation.

In this step, get ready to:
  • Install Rust and Cargo.
  • Install the Flutter SDK with Linux desktop support enabled.
  • Install Ollama and pull a tool-calling model.
Install Rust and Cargo

Rust is the language you will use to build the agent backend. Cargo is Rust's package manager and build tool, and it installs automatically alongside the compiler.

  • Open a terminal. Press Ctrl+Alt+T (Linux) or open your preferred terminal application.
  • Check if Rust is already installed by running this command:
rustc --version

✔️ I see a version number

Rust is ready to go. Confirm Cargo is also available by running:

cargo --version

You should see a version number printed. You're all set for this tool.

ⓧ Command not found

You need to install Rust. Install it using the official rustup installer by running this command:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

What does this command do?

This downloads and runs the official Rust installer. It installs the Rust compiler (rustc), the package manager (cargo), and the version manager (rustup).

  • The installer will ask you to choose an installation option. Press 1 to proceed with the default installation.
  • After installation completes, apply the PATH changes by running:
source $HOME/.cargo/env
  • Verify the installation by running:
rustc --version && cargo --version

You should see version numbers for both rustc and cargo.

Still seeing command not found?

Close your terminal and open a new one. The PATH update from the Rust installer only takes effect in new terminal sessions.

Help me troubleshoot why Rust isn't found after installation

Install the Flutter SDK

Flutter is a cross-platform UI framework that you will use to build the desktop chat interface. For this project, you need Flutter with Linux desktop support enabled.

  • Check if Flutter is already installed by running:
flutter --version

✔️ I see a version number

Flutter is installed. Now enable Linux desktop support by running:

flutter config --enable-linux-desktop

You should see a confirmation message that Linux desktop development is enabled.

ⓧ Command not found

You need to install the Flutter SDK. Flutter requires a few prerequisite packages on Linux.

  • Install the prerequisite packages by running:
sudo apt-get update -y && sudo apt-get upgrade -y
sudo apt-get install -y curl git unzip xz-utils zip libglu1-mesa
  • Download and extract the Flutter SDK. Visit the Flutter install page and follow the Custom Setup instructions for Linux.
  • Add Flutter to your PATH. The install page provides exact steps for adding the flutter/bin directory to your shell profile.
  • Restart your terminal, then verify by running:
flutter --version
  • Enable Linux desktop support by running:
flutter config --enable-linux-desktop

You should see a confirmation that Linux desktop development is enabled.

What is Linux desktop support?

Flutter can target mobile, web, and desktop platforms from a single codebase. Linux desktop support is an opt-in configuration that tells Flutter to compile native Linux applications.

Without it, flutter run -d linux will not work.

Flutter doctor showing issues?

Run flutter doctor -v to see detailed diagnostics. For this project, you only need the Flutter SDK and Linux desktop entries to show green checkmarks. Android and Chrome issues can be ignored.

Help me fix Flutter doctor issues for Linux desktop development

Install Ollama and pull a tool-calling model

Ollama lets you run large language models entirely on your machine. Your Rust agent backend will send requests to Ollama's local API to get the model's reasoning and tool-call decisions.

  • Check if Ollama is already installed by running:
ollama -v

✔️ I see a version number

Ollama is installed. Skip ahead to pulling the model below.

ⓧ Command not found

Install Ollama by running the official install script:

curl -fsSL https://ollama.com/install.sh | sh

What does this command do?

This downloads and runs the official Ollama installer for Linux. It sets up the Ollama binary and configures a background service that serves the local LLM API.

Once the script completes, verify by running ollama -v again. You should see a version number printed.

  • Pull the qwen2.5:7b model by running this command:
ollama pull qwen2.5:7b

Why qwen2.5:7b?

This model has good tool-calling quality at approximately 5 GB of memory usage. It reliably decides when to use tools and correctly formats tool-call arguments.

Smaller models exist but are less reliable at selecting the right tool for a given question.

  • Wait for the download to complete. The model is approximately 5 GB, so this may take several minutes depending on your connection speed.
  • Verify the model is available by running:
ollama list

You should see qwen2.5:7b listed in the output.

Download stuck or failing?

Make sure the Ollama service is running. If you see a connection error, start the service first with ollama serve in a separate terminal window, then retry the pull command.

Help me troubleshoot Ollama model download issues

All three tools are ready. Next up, you will create the Rust project that powers your AI agent's brain.

Build the Rust Agent Backend

With all your tools installed and Ollama serving the qwen2.5:7b model locally, you're ready to build the core of your AI agent.

The agent backend is where the intelligence lives. It receives a user question, sends it to the local LLM, and if the model decides it needs a tool (like checking the time or doing math), your Rust code executes that tool and feeds the result back. This loop continues until the model produces a final answer.

In this step, get ready to:
  • Create a Rust project and configure its dependencies.
  • Build a tool registry with three tools the AI can call.
  • Implement the agentic loop that communicates with Ollama.
Create a new Rust project and add dependencies

Every Rust project starts with Cargo, Rust's package manager and build system. You'll scaffold a new binary project, then declare the crates (libraries) your agent needs.

  • Create the new Rust project by running these commands:
cargo new agent-backend
cd agent-backend

What does this command do?

cargo new agent-backend creates a new directory called agent-backend with a Cargo.toml (the project manifest) and a src/main.rs (the entry point). Then you move into that directory so all future commands run inside the project.

  • Confirm the project was created by running:
ls

You should see Cargo.toml and a src folder in the output.

Your agent needs several crates to handle HTTP requests, JSON parsing, async execution, and time formatting. You'll declare all of them in Cargo.toml.

  • Open Cargo.toml in your editor, then replace the entire file contents with this:
[package]
name = "agent-backend"
version = "0.1.0"
edition = "2021"

[dependencies]
axum = "0.8.4"
tokio = { version = "1.44.2", features = ["full"] }
reqwest = { version = "0.13.3", features = ["json", "stream"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
tower-http = { version = "0.6.6", features = ["cors"] }
futures = "0.3"
chrono = "0.4"

What do these dependencies do?

  • axum is the HTTP server framework that will expose your agent as an API.
  • tokio is the async runtime that powers everything under the hood.
  • reqwest sends HTTP requests to the Ollama API.
  • serde and serde_json handle JSON serialization and deserialization.
  • tower-http provides CORS middleware so the Flutter app can call the backend.
  • chrono formats the current time for the GetTime tool.
  • Save Cargo.toml.

Seeing a warning about Cargo.toml?

Make sure you replaced the entire file, including the [package] section. If you only added the [dependencies] block below the existing content, you may end up with duplicate entries.

Help me fix my Cargo.toml

Implement the tool registry

Tools are the capabilities you give your AI agent. Each tool has a name, a description (so the LLM knows when to use it), a parameter schema, and an execute function that runs locally.

You'll define a Tool trait as the blueprint, then implement three tools: one for getting the current time, one for math calculations, and one for system information.

  • Create a new file called tools.rs inside the agent-backend/src/ folder.
  • Add the trait definition and the first tool implementation by copying this code into tools.rs:
use serde_json::{json, Value};
use chrono::Local;

pub trait Tool: Send + Sync {
    fn name(&self) -> &str;
    fn description(&self) -> &str;
    fn parameters(&self) -> Value;
    fn execute(&self, args: &Value) -> String;
}

pub struct GetTime;

impl Tool for GetTime {
    fn name(&self) -> &str { "get_time" }
    fn description(&self) -> &str { "Get the current date and time" }
    fn parameters(&self) -> Value {
        json!({
            "type": "object",
            "properties": {},
            "required": []
        })
    }
    fn execute(&self, _args: &Value) -> String {
        Local::now().format("%Y-%m-%d %H:%M:%S").to_string()
    }
}

What does this code do?

  • The Tool trait defines the interface every tool must satisfy: a name, description, parameter schema, and an execute function.
  • Send + Sync bounds ensure tools can be called safely from async code running on multiple threads.
  • parameters() returns a JSON schema that tells the LLM what arguments the tool expects. GetTime has no parameters, so its schema is empty.
  • GetTime simply returns the current local time formatted as a string.
  • Add the Calculate tool below the GetTime implementation. This tool takes parameters, so its schema is more detailed:
pub struct Calculate;

impl Tool for Calculate {
    fn name(&self) -> &str { "calculate" }
    fn description(&self) -> &str { "Evaluate a mathematical expression. Supports +, -, *, / with two numbers." }
    fn parameters(&self) -> Value {
        json!({
            "type": "object",
            "properties": {
                "a": { "type": "number", "description": "First number" },
                "b": { "type": "number", "description": "Second number" },
                "operation": { "type": "string", "description": "Operation: add, subtract, multiply, divide", "enum": ["add", "subtract", "multiply", "divide"] }
            },
            "required": ["a", "b", "operation"]
        })
    }
    fn execute(&self, args: &Value) -> String {
        let a = args["a"].as_f64().unwrap_or(0.0);
        let b = args["b"].as_f64().unwrap_or(0.0);
        let op = args["operation"].as_str().unwrap_or("add");
        let result = match op {
            "add" => a + b,
            "subtract" => a - b,
            "multiply" => a * b,
            "divide" => if b != 0.0 { a / b } else { f64::NAN },
            _ => f64::NAN,
        };
        format!("{}", result)
    }
}

How does Calculate differ from GetTime?

Calculate declares three required parameters in its JSON schema: a, b, and operation. When the LLM calls this tool, it fills in those arguments. The execute function extracts the values from the JSON and performs the math with a match expression.

  • Add the SystemInfo tool and the two helper functions that register all tools. Copy this below the Calculate implementation:
pub struct SystemInfo;

impl Tool for SystemInfo {
    fn name(&self) -> &str { "system_info" }
    fn description(&self) -> &str { "Get system information like OS name and architecture" }
    fn parameters(&self) -> Value {
        json!({
            "type": "object",
            "properties": {},
            "required": []
        })
    }
    fn execute(&self, _args: &Value) -> String {
        format!("OS: {} / Arch: {}", std::env::consts::OS, std::env::consts::ARCH)
    }
}

pub fn all_tools() -> Vec<Box<dyn Tool>> {
    vec![
        Box::new(GetTime),
        Box::new(Calculate),
        Box::new(SystemInfo),
    ]
}

pub fn tool_schemas(tools: &[Box<dyn Tool>]) -> Vec<Value> {
    tools.iter().map(|t| {
        json!({
            "type": "function",
            "function": {
                "name": t.name(),
                "description": t.description(),
                "parameters": t.parameters()
            }
        })
    }).collect()
}

What are all_tools() and tool_schemas() for?

  • all_tools() returns a vector of all available tools. This is the registry your agent loop will search when the LLM requests a tool.
  • tool_schemas() converts each tool into the JSON format that Ollama expects. Ollama needs to know what tools are available so it can decide when to call them.
  • Save tools.rs.

You won't see a change yet. The next substep builds the agentic loop that actually uses these tools.

✔️ Awesome, I've got everything!

Great. Double check you've saved tools.rs.

ⓧ I'd like to double check the full code

use serde_json::{json, Value};
use chrono::Local;

pub trait Tool: Send + Sync {
    fn name(&self) -> &str;
    fn description(&self) -> &str;
    fn parameters(&self) -> Value;
    fn execute(&self, args: &Value) -> String;
}

pub struct GetTime;

impl Tool for GetTime {
    fn name(&self) -> &str { "get_time" }
    fn description(&self) -> &str { "Get the current date and time" }
    fn parameters(&self) -> Value {
        json!({
            "type": "object",
            "properties": {},
            "required": []
        })
    }
    fn execute(&self, _args: &Value) -> String {
        Local::now().format("%Y-%m-%d %H:%M:%S").to_string()
    }
}

pub struct Calculate;

impl Tool for Calculate {
    fn name(&self) -> &str { "calculate" }
    fn description(&self) -> &str { "Evaluate a mathematical expression. Supports +, -, *, / with two numbers." }
    fn parameters(&self) -> Value {
        json!({
            "type": "object",
            "properties": {
                "a": { "type": "number", "description": "First number" },
                "b": { "type": "number", "description": "Second number" },
                "operation": { "type": "string", "description": "Operation: add, subtract, multiply, divide", "enum": ["add", "subtract", "multiply", "divide"] }
            },
            "required": ["a", "b", "operation"]
        })
    }
    fn execute(&self, args: &Value) -> String {
        let a = args["a"].as_f64().unwrap_or(0.0);
        let b = args["b"].as_f64().unwrap_or(0.0);
        let op = args["operation"].as_str().unwrap_or("add");
        let result = match op {
            "add" => a + b,
            "subtract" => a - b,
            "multiply" => a * b,
            "divide" => if b != 0.0 { a / b } else { f64::NAN },
            _ => f64::NAN,
        };
        format!("{}", result)
    }
}

pub struct SystemInfo;

impl Tool for SystemInfo {
    fn name(&self) -> &str { "system_info" }
    fn description(&self) -> &str { "Get system information like OS name and architecture" }
    fn parameters(&self) -> Value {
        json!({
            "type": "object",
            "properties": {},
            "required": []
        })
    }
    fn execute(&self, _args: &Value) -> String {
        format!("OS: {} / Arch: {}", std::env::consts::OS, std::env::consts::ARCH)
    }
}

pub fn all_tools() -> Vec<Box<dyn Tool>> {
    vec![
        Box::new(GetTime),
        Box::new(Calculate),
        Box::new(SystemInfo),
    ]
}

pub fn tool_schemas(tools: &[Box<dyn Tool>]) -> Vec<Value> {
    tools.iter().map(|t| {
        json!({
            "type": "function",
            "function": {
                "name": t.name(),
                "description": t.description(),
                "parameters": t.parameters()
            }
        })
    }).collect()
}
Implement the agentic loop

The agentic loop is the pattern that makes your backend an AI agent rather than a simple chatbot. Instead of one request and one response, the loop lets the model call tools and incorporate their results before producing a final answer.

How does the agentic loop work?

  • Send the user message and available tool schemas to Ollama.
  • If the model responds with tool_calls, execute the matching tool locally and feed the result back as a new message.
  • Repeat (up to 5 rounds) until the model responds with plain content instead of a tool call.
  • Return the final text answer to the caller.
  • Create a new file called agent.rs inside agent-backend/src/.
  • Add the full agentic loop by copying this code into agent.rs:
use crate::tools::{all_tools, tool_schemas};
use axum::response::sse::Event;
use futures::stream::{self, Stream};
use reqwest::Client;
use serde_json::{json, Value};
use std::convert::Infallible;

pub fn run_agent_loop(
    user_message: String,
) -> impl Stream<Item = Result<Event, Infallible>> {
    stream::once(async move {
        let client = Client::new();
        let tools = all_tools();
        let schemas = tool_schemas(&tools);

        let mut messages: Vec<Value> = vec![
            json!({"role": "system", "content": "You are a helpful assistant. Use the available tools when needed to answer questions accurately."}),
            json!({"role": "user", "content": user_message}),
        ];

        let mut final_response = String::from("I could not generate a response.");

        for _round in 0..5 {
            let body = json!({
                "model": "qwen2.5:7b",
                "messages": messages,
                "tools": schemas,
                "stream": false
            });

            let res = client
                .post("http://localhost:11434/api/chat")
                .json(&body)
                .send()
                .await;

            let res = match res {
                Ok(r) => r,
                Err(e) => {
                    final_response = format!("Error contacting Ollama: {}", e);
                    break;
                }
            };

            let json_res: Value = match res.json().await {
                Ok(j) => j,
                Err(e) => {
                    final_response = format!("Error parsing response: {}", e);
                    break;
                }
            };

            let message = &json_res["message"];

            if let Some(tool_calls) = message["tool_calls"].as_array() {
                if tool_calls.is_empty() {
                    final_response = message["content"]
                        .as_str()
                        .unwrap_or("")
                        .to_string();
                    break;
                }

                messages.push(json!({
                    "role": "assistant",
                    "content": "",
                    "tool_calls": tool_calls
                }));

                for tc in tool_calls {
                    let func_name = tc["function"]["name"].as_str().unwrap_or("");
                    let func_args = &tc["function"]["arguments"];

                    let result = tools
                        .iter()
                        .find(|t| t.name() == func_name)
                        .map(|t| t.execute(func_args))
                        .unwrap_or_else(|| format!("Unknown tool: {}", func_name));

                    messages.push(json!({
                        "role": "tool",
                        "content": result,
                        "tool_name": func_name
                    }));
                }
            } else {
                final_response = message["content"]
                    .as_str()
                    .unwrap_or("")
                    .to_string();
                break;
            }
        }

        Ok(Event::default().data(final_response))
    })
}

What does this code do?

  • run_agent_loop() returns a stream that produces one SSE event. This stream format is what the HTTP server will send to the Flutter UI.
  • The messages vector accumulates the conversation history. Each round adds the assistant's tool call and the tool's result, building context for the next iteration.
  • The loop sends a POST request to Ollama's /api/chat endpoint with "stream": false so it gets a complete response object each round.
  • When the model returns tool_calls, the code finds the matching tool by name, executes it, and appends the result with role: "tool" so Ollama knows it's a tool response.
  • The cap of 5 rounds prevents infinite loops if the model keeps requesting tools without settling on a final answer.
  • Save agent.rs.

Now you need to wire both modules into the application entry point. The main.rs file declares the modules and sets up the HTTP handler that calls your agentic loop.

  • Open src/main.rs (Cargo created this file with a placeholder Hello, world! program). Replace the entire contents with this:
mod tools;
mod agent;

use axum::{
    Router,
    routing::post,
    response::sse::{Event, Sse},
    extract::Json,
};
use futures::stream::Stream;
use serde::Deserialize;
use std::convert::Infallible;
use tower_http::cors::CorsLayer;

#[derive(Deserialize)]
struct ChatRequest {
    message: String,
}

async fn chat_handler(
    Json(payload): Json<ChatRequest>,
) -> Sse<impl Stream<Item = Result<Event, Infallible>>> {
    let stream = agent::run_agent_loop(payload.message);
    Sse::new(stream)
}

#[tokio::main]
async fn main() {
    let app = Router::new()
        .route("/chat", post(chat_handler))
        .layer(CorsLayer::permissive());

    let listener = tokio::net::TcpListener::bind("127.0.0.1:8080")
        .await
        .unwrap();
    println!("Agent backend running on http://127.0.0.1:8080");
    axum::serve(listener, app).await.unwrap();
}

What does this code do?

  • mod tools and mod agent tell the Rust compiler to include your two new modules.
  • ChatRequest defines the JSON shape the server expects from the Flutter app (a single message field).
  • chat_handler receives a POST request, passes the message to run_agent_loop(), and returns the result as a Server-Sent Events stream.
  • The server binds to 127.0.0.1:8080 with permissive CORS so the Flutter desktop app can connect from localhost.
  • Save main.rs.
  • Verify everything compiles by running:
cargo build

The first build downloads and compiles all your dependencies. This may take a minute or two. You should see a line containing Compiling agent-backend followed by a successful exit with no errors.

Seeing compilation errors?

  • If you see "unresolved import" errors, make sure both tools.rs and agent.rs are saved inside agent-backend/src/ (not at the project root).
  • If you see "expected struct" or type mismatch errors, double check that your Cargo.toml includes the correct feature flags (especially features = ["full"] for tokio).
  • Make sure you are running cargo build from inside the agent-backend directory, not from its parent.

Help me debug my cargo build errors

✔️ Awesome, I've got everything!

Great. All three source files are in place and the project compiles cleanly.

ⓧ I'd like to double check the full code

Here is each file in its complete form. Compare against your code if something isn't compiling.

[package]
name = "agent-backend"
version = "0.1.0"
edition = "2021"

[dependencies]
axum = "0.8.4"
tokio = { version = "1.44.2", features = ["full"] }
reqwest = { version = "0.13.3", features = ["json", "stream"] }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
tower-http = { version = "0.6.6", features = ["cors"] }
futures = "0.3"
chrono = "0.4"
use serde_json::{json, Value};
use chrono::Local;

pub trait Tool: Send + Sync {
    fn name(&self) -> &str;
    fn description(&self) -> &str;
    fn parameters(&self) -> Value;
    fn execute(&self, args: &Value) -> String;
}

pub struct GetTime;

impl Tool for GetTime {
    fn name(&self) -> &str { "get_time" }
    fn description(&self) -> &str { "Get the current date and time" }
    fn parameters(&self) -> Value {
        json!({
            "type": "object",
            "properties": {},
            "required": []
        })
    }
    fn execute(&self, _args: &Value) -> String {
        Local::now().format("%Y-%m-%d %H:%M:%S").to_string()
    }
}

pub struct Calculate;

impl Tool for Calculate {
    fn name(&self) -> &str { "calculate" }
    fn description(&self) -> &str { "Evaluate a mathematical expression. Supports +, -, *, / with two numbers." }
    fn parameters(&self) -> Value {
        json!({
            "type": "object",
            "properties": {
                "a": { "type": "number", "description": "First number" },
                "b": { "type": "number", "description": "Second number" },
                "operation": { "type": "string", "description": "Operation: add, subtract, multiply, divide", "enum": ["add", "subtract", "multiply", "divide"] }
            },
            "required": ["a", "b", "operation"]
        })
    }
    fn execute(&self, args: &Value) -> String {
        let a = args["a"].as_f64().unwrap_or(0.0);
        let b = args["b"].as_f64().unwrap_or(0.0);
        let op = args["operation"].as_str().unwrap_or("add");
        let result = match op {
            "add" => a + b,
            "subtract" => a - b,
            "multiply" => a * b,
            "divide" => if b != 0.0 { a / b } else { f64::NAN },
            _ => f64::NAN,
        };
        format!("{}", result)
    }
}

pub struct SystemInfo;

impl Tool for SystemInfo {
    fn name(&self) -> &str { "system_info" }
    fn description(&self) -> &str { "Get system information like OS name and architecture" }
    fn parameters(&self) -> Value {
        json!({
            "type": "object",
            "properties": {},
            "required": []
        })
    }
    fn execute(&self, _args: &Value) -> String {
        format!("OS: {} / Arch: {}", std::env::consts::OS, std::env::consts::ARCH)
    }
}

pub fn all_tools() -> Vec<Box<dyn Tool>> {
    vec![
        Box::new(GetTime),
        Box::new(Calculate),
        Box::new(SystemInfo),
    ]
}

pub fn tool_schemas(tools: &[Box<dyn Tool>]) -> Vec<Value> {
    tools.iter().map(|t| {
        json!({
            "type": "function",
            "function": {
                "name": t.name(),
                "description": t.description(),
                "parameters": t.parameters()
            }
        })
    }).collect()
}
use crate::tools::{all_tools, tool_schemas};
use axum::response::sse::Event;
use futures::stream::{self, Stream};
use reqwest::Client;
use serde_json::{json, Value};
use std::convert::Infallible;

pub fn run_agent_loop(
    user_message: String,
) -> impl Stream<Item = Result<Event, Infallible>> {
    stream::once(async move {
        let client = Client::new();
        let tools = all_tools();
        let schemas = tool_schemas(&tools);

        let mut messages: Vec<Value> = vec![
            json!({"role": "system", "content": "You are a helpful assistant. Use the available tools when needed to answer questions accurately."}),
            json!({"role": "user", "content": user_message}),
        ];

        let mut final_response = String::from("I could not generate a response.");

        for _round in 0..5 {
            let body = json!({
                "model": "qwen2.5:7b",
                "messages": messages,
                "tools": schemas,
                "stream": false
            });

            let res = client
                .post("http://localhost:11434/api/chat")
                .json(&body)
                .send()
                .await;

            let res = match res {
                Ok(r) => r,
                Err(e) => {
                    final_response = format!("Error contacting Ollama: {}", e);
                    break;
                }
            };

            let json_res: Value = match res.json().await {
                Ok(j) => j,
                Err(e) => {
                    final_response = format!("Error parsing response: {}", e);
                    break;
                }
            };

            let message = &json_res["message"];

            if let Some(tool_calls) = message["tool_calls"].as_array() {
                if tool_calls.is_empty() {
                    final_response = message["content"]
                        .as_str()
                        .unwrap_or("")
                        .to_string();
                    break;
                }

                messages.push(json!({
                    "role": "assistant",
                    "content": "",
                    "tool_calls": tool_calls
                }));

                for tc in tool_calls {
                    let func_name = tc["function"]["name"].as_str().unwrap_or("");
                    let func_args = &tc["function"]["arguments"];

                    let result = tools
                        .iter()
                        .find(|t| t.name() == func_name)
                        .map(|t| t.execute(func_args))
                        .unwrap_or_else(|| format!("Unknown tool: {}", func_name));

                    messages.push(json!({
                        "role": "tool",
                        "content": result,
                        "tool_name": func_name
                    }));
                }
            } else {
                final_response = message["content"]
                    .as_str()
                    .unwrap_or("")
                    .to_string();
                break;
            }
        }

        Ok(Event::default().data(final_response))
    })
}
mod tools;
mod agent;

use axum::{
    Router,
    routing::post,
    response::sse::{Event, Sse},
    extract::Json,
};
use futures::stream::Stream;
use serde::Deserialize;
use std::convert::Infallible;
use tower_http::cors::CorsLayer;

#[derive(Deserialize)]
struct ChatRequest {
    message: String,
}

async fn chat_handler(
    Json(payload): Json<ChatRequest>,
) -> Sse<impl Stream<Item = Result<Event, Infallible>>> {
    let stream = agent::run_agent_loop(payload.message);
    Sse::new(stream)
}

#[tokio::main]
async fn main() {
    let app = Router::new()
        .route("/chat", post(chat_handler))
        .layer(CorsLayer::permissive());

    let listener = tokio::net::TcpListener::bind("127.0.0.1:8080")
        .await
        .unwrap();
    println!("Agent backend running on http://127.0.0.1:8080");
    axum::serve(listener, app).await.unwrap();
}

Your agent brain is fully built. It can reason, call tools, and produce answers. Next up, you'll add the HTTP server layer that exposes this logic as an API the Flutter app can talk to.

Add the HTTP Server Layer

Your agent loop can reason, call tools, and generate answers. But right now nothing can talk to it from the outside.

In this step, you will wire up an HTTP server so that a frontend (the Flutter app you will build next) can send messages and receive streamed responses. The server uses Axum for routing and Server-Sent Events (SSE) for real-time streaming.

In this step, get ready to:
  • Set up an Axum HTTP server on port 8080 with CORS enabled.
  • Create a /chat POST endpoint that streams SSE responses.
  • Verify the full backend compiles successfully.
Set up the Axum HTTP server

The main.rs file you created in the previous step already has mod declarations linking your tools and agent modules. Now you will fill it in with the HTTP server, the request handler, and the wiring that connects incoming messages to your agentic loop.

  • Open agent-backend/src/main.rs and replace its entire contents with the following:
mod tools;
mod agent;

use axum::{
    Router,
    routing::post,
    response::sse::{Event, Sse},
    extract::Json,
};
use futures::stream::Stream;
use serde::Deserialize;
use std::convert::Infallible;
use tower_http::cors::CorsLayer;

// Define the expected shape of incoming JSON requests
#[derive(Deserialize)]
struct ChatRequest {
    message: String,
}

// Handler that accepts a user message and returns an SSE stream
async fn chat_handler(
    Json(payload): Json<ChatRequest>,
) -> Sse<impl Stream<Item = Result<Event, Infallible>>> {
    let stream = agent::run_agent_loop(payload.message);
    Sse::new(stream)
}

#[tokio::main]
async fn main() {
    // Build the router with a single /chat endpoint and permissive CORS
    let app = Router::new()
        .route("/chat", post(chat_handler))
        .layer(CorsLayer::permissive());

    // Bind to localhost:8080 and start serving
    let listener = tokio::net::TcpListener::bind("127.0.0.1:8080")
        .await
        .unwrap();
    println!("Agent backend running on http://127.0.0.1:8080");
    axum::serve(listener, app).await.unwrap();
}

What does this code do?

This file sets up three things that turn your agent into a reachable service:

  • ChatRequest is a Serde struct that automatically deserializes incoming JSON into a typed Rust value. The Flutter app will send {"message": "What time is it?"} and Axum parses it for you.
  • chat_handler extracts the message, passes it to your run_agent_loop function, and wraps the returned stream in an SSE response. The client receives data as it becomes available rather than waiting for the entire response.
  • CorsLayer::permissive() allows requests from any origin. This is necessary because the Flutter app runs on a different port than the Rust backend.
  • The server binds to 127.0.0.1:8080 so only your local machine can reach it.

Why SSE instead of plain JSON?

A plain JSON response forces the client to wait until the entire agent loop finishes before seeing anything. SSE lets the server push partial results as they happen. Your Flutter UI will show the agent's answer appearing in real-time rather than after a long pause.

  • Save main.rs.

Seeing red squiggles in your editor?

If your editor shows unresolved import errors, that is normal until the next substep where you compile the project. The Rust language server sometimes needs a successful build to resolve all crate paths.

Help me fix unresolved imports in main.rs

Compile the backend

All three source files (tools.rs, agent.rs, main.rs) are now complete. A release build confirms everything links together without errors.

  • In your terminal, make sure you are inside the agent-backend folder, then compile the project by running:
cargo build --release
  • Wait for the build to finish. The first compile downloads and builds all dependencies, so it may take 1-2 minutes.
  • You should see a line ending with Finished and no error messages.

Build failed with errors?

Check that your Cargo.toml matches the dependencies from Step 2 exactly. Missing features (like ["full"] on tokio or ["cors"] on tower-http) cause unresolved import errors.

If you see type mismatch errors in main.rs, confirm that agent.rs has the correct return type on run_agent_loop. The function must return impl Stream<Item = Result<Event, Infallible>>.

Help me debug my cargo build errors

✔️ Awesome, I've got everything!

Great. Make sure you have saved all three files before moving on.

ⓧ I'd like to double check the full code

mod tools;
mod agent;

use axum::{
    Router,
    routing::post,
    response::sse::{Event, Sse},
    extract::Json,
};
use futures::stream::Stream;
use serde::Deserialize;
use std::convert::Infallible;
use tower_http::cors::CorsLayer;

#[derive(Deserialize)]
struct ChatRequest {
    message: String,
}

async fn chat_handler(
    Json(payload): Json<ChatRequest>,
) -> Sse<impl Stream<Item = Result<Event, Infallible>>> {
    let stream = agent::run_agent_loop(payload.message);
    Sse::new(stream)
}

#[tokio::main]
async fn main() {
    let app = Router::new()
        .route("/chat", post(chat_handler))
        .layer(CorsLayer::permissive());

    let listener = tokio::net::TcpListener::bind("127.0.0.1:8080")
        .await
        .unwrap();
    println!("Agent backend running on http://127.0.0.1:8080");
    axum::serve(listener, app).await.unwrap();
}

Your Rust backend is now a fully compiled HTTP service. Next up, you will build the Flutter chat UI that connects to this endpoint and displays the agent's streamed responses in real-time.

Build the Flutter Chat UI

Your Rust backend is ready to receive chat messages and stream responses. Now you need a front end for users to interact with.

In this step, you'll build a Flutter desktop application that sends messages to the Rust agent and displays streaming responses in a chat bubble interface.

In this step, get ready to:
  • Create a Flutter desktop project with Linux platform support.
  • Build a chat interface with user and agent message bubbles.
  • Connect to the Rust backend and parse SSE responses.
Create the Flutter desktop project

Flutter's --platforms flag lets you scaffold a project targeting only the platforms you need. For this project, you only need the Linux desktop target.

  • Navigate back to your Desktop (the parent directory of agent-backend) by running this command:
cd ~/Desktop
  • Create the Flutter project with Linux desktop support by running:
flutter create --platforms=linux agent_ui

What does this command do?

The flutter create command scaffolds a new Flutter project. The --platforms=linux flag tells Flutter to only generate the Linux desktop runner files, skipping Android, iOS, web, and other targets you don't need.

  • Move into the new project directory by running:
cd agent_ui
  • Confirm the project was created by running:
ls

You should see folders like lib, linux, and test, plus files like pubspec.yaml and lib/main.dart.

Next, you need to add the HTTP package that Flutter will use to talk to your Rust backend.

  • Open pubspec.yaml in your editor and replace its entire contents with the following:
name: agent_ui
description: A Flutter desktop chat UI for the local AI agent.
publish_to: 'none'
version: 1.0.0+1

environment:
  sdk: ^3.8.0

dependencies:
  flutter:
    sdk: flutter
  http: ^1.4.0

dev_dependencies:
  flutter_test:
    sdk: flutter
  flutter_lints: ^5.0.0

flutter:
  uses-material-design: true

What does this file do?

The pubspec.yaml file declares your project's dependencies. The key addition here is http: ^1.4.0, which provides the HTTP client your app uses to POST messages to the Rust backend and read SSE responses.

  • Save pubspec.yaml and install the dependency by running:
flutter pub get

You should see output confirming the packages resolved successfully.

Seeing dependency errors?

Make sure you replaced the entire pubspec.yaml file contents, not just added lines. YAML is sensitive to indentation, so check that each level uses exactly 2 spaces.

help me fix Flutter pub get errors

Build the chat app structure

With the project scaffolded, you'll now replace the default counter app with your chat UI. The app has three main pieces: the AgentApp widget (theme and routing), a ChatMessage data model, and the ChatScreen stateful widget.

  • Open lib/main.dart and delete all of its contents. Replace it with the following app foundation code:
import 'dart:convert';
import 'package:flutter/material.dart';
import 'package:http/http.dart' as http;

void main() {
  runApp(const AgentApp());
}

// Root widget that configures the app theme and home screen
class AgentApp extends StatelessWidget {
  const AgentApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Local AI Agent',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const ChatScreen(),
    );
  }
}

// Simple data class to hold each chat message and who sent it
class ChatMessage {
  final String text;
  final bool isUser;

  ChatMessage({required this.text, required this.isUser});
}

What does this code do?

  • The three imports bring in JSON encoding, Flutter's Material UI library, and the HTTP client package.
  • AgentApp sets up a Material 3 theme with a deep purple color scheme and points to ChatScreen as the home page.
  • ChatMessage stores the message text and a boolean indicating whether the user or the agent sent it. This drives how bubbles are styled and aligned.

Don't save yet. You'll add the remaining code in the next chunks before saving and running.

Connect to the Rust backend and render the chat

The ChatScreen widget manages the message list, a loading indicator, and the HTTP connection to your Rust backend. When the user sends a message, the app POSTs it to http://127.0.0.1:8080/chat and parses the Server-Sent Events response.

  • In lib/main.dart, add the following code directly below the ChatMessage class to create the stateful widget and its networking logic:
// The main chat screen widget
class ChatScreen extends StatefulWidget {
  const ChatScreen({super.key});

  @override
  State<ChatScreen> createState() => _ChatScreenState();
}

class _ChatScreenState extends State<ChatScreen> {
  final TextEditingController _controller = TextEditingController();
  final List<ChatMessage> _messages = [];
  bool _isLoading = false;

  // Send a message to the Rust backend and parse the SSE response
  Future<void> _sendMessage() async {
    final text = _controller.text.trim();
    if (text.isEmpty) return;

    setState(() {
      _messages.add(ChatMessage(text: text, isUser: true));
      _isLoading = true;
    });
    _controller.clear();

    try {
      // Build an HTTP request to the Rust agent backend
      final request = http.Request(
        'POST',
        Uri.parse('http://127.0.0.1:8080/chat'),
      );
      request.headers['Content-Type'] = 'application/json';
      request.body = jsonEncode({'message': text});

      final response = await http.Client().send(request);
      final body = await response.stream.bytesToString();

      // Parse SSE data lines from the response
      String agentResponse = '';
      for (final line in body.split('\n')) {
        if (line.startsWith('data:')) {
          agentResponse += line.substring(5).trim();
        }
      }

      setState(() {
        _messages.add(ChatMessage(
          text: agentResponse.isEmpty ? 'No response received.' : agentResponse,
          isUser: false,
        ));
        _isLoading = false;
      });
    } catch (e) {
      setState(() {
        _messages.add(ChatMessage(
          text: 'Error: Could not connect to agent backend. Is it running?',
          isUser: false,
        ));
        _isLoading = false;
      });
    }
  }

How does the backend connection work?

The _sendMessage() method sends a POST request with the user's message as JSON to the Rust backend at /chat. The backend responds with Server-Sent Events, where each event is a line prefixed with data:.

The code splits the response by newlines and extracts anything after data: prefixes. If the connection fails entirely, a catch block shows an error message in the chat instead of crashing.

  • Still in lib/main.dart, add the build() method directly below the _sendMessage() method to complete the _ChatScreenState class:
  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Local AI Agent'),
        backgroundColor: Theme.of(context).colorScheme.inversePrimary,
      ),
      body: Column(
        children: [
          // Scrollable list of chat bubbles
          Expanded(
            child: ListView.builder(
              padding: const EdgeInsets.all(16),
              itemCount: _messages.length,
              itemBuilder: (context, index) {
                final msg = _messages[index];
                return Align(
                  alignment:
                      msg.isUser ? Alignment.centerRight : Alignment.centerLeft,
                  child: Container(
                    margin: const EdgeInsets.symmetric(vertical: 4),
                    padding: const EdgeInsets.all(12),
                    constraints: BoxConstraints(
                      maxWidth: MediaQuery.of(context).size.width * 0.75,
                    ),
                    decoration: BoxDecoration(
                      color: msg.isUser
                          ? Theme.of(context).colorScheme.primary
                          : Theme.of(context).colorScheme.surfaceContainerHighest,
                      borderRadius: BorderRadius.circular(12),
                    ),
                    child: Text(
                      msg.text,
                      style: TextStyle(
                        color: msg.isUser ? Colors.white : Colors.black87,
                      ),
                    ),
                  ),
                );
              },
            ),
          ),
          // Loading spinner while waiting for the agent
          if (_isLoading)
            const Padding(
              padding: EdgeInsets.all(8.0),
              child: CircularProgressIndicator(),
            ),
          // Text input and send button
          Padding(
            padding: const EdgeInsets.all(8.0),
            child: Row(
              children: [
                Expanded(
                  child: TextField(
                    controller: _controller,
                    decoration: const InputDecoration(
                      hintText: 'Ask your local AI agent...',
                      border: OutlineInputBorder(),
                    ),
                    onSubmitted: (_) => _sendMessage(),
                  ),
                ),
                const SizedBox(width: 8),
                IconButton(
                  onPressed: _sendMessage,
                  icon: const Icon(Icons.send),
                ),
              ],
            ),
          ),
        ],
      ),
    );
  }
}

What does the UI do?

  • The ListView.builder renders chat bubbles from the _messages list. User messages align right with a purple background. Agent messages align left with a lighter surface color.
  • A CircularProgressIndicator appears while _isLoading is true, giving visual feedback that the agent is thinking.
  • The TextField at the bottom accepts typed input. Pressing Enter or tapping the send icon calls _sendMessage().
  • Save lib/main.dart.

You won't run the app just yet. In the next step, you'll start all three services together and test the full agent loop.

Seeing analysis errors in your editor?

Make sure you have all three code sections in main.dart in the correct order: imports and AgentApp first, then ChatMessage, then ChatScreen with _ChatScreenState. The closing brace of the build() method must be followed by one more } to close the _ChatScreenState class.

help me fix Dart analysis errors in my Flutter chat app

✔️ Awesome, I've got everything!

Great. Double check you've saved lib/main.dart and that your editor shows no red underlines.

ⓧ I'd like to double check the full code

Here is the complete lib/main.dart file:

import 'dart:convert';
import 'package:flutter/material.dart';
import 'package:http/http.dart' as http;

void main() {
  runApp(const AgentApp());
}

class AgentApp extends StatelessWidget {
  const AgentApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Local AI Agent',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const ChatScreen(),
    );
  }
}

class ChatMessage {
  final String text;
  final bool isUser;

  ChatMessage({required this.text, required this.isUser});
}

class ChatScreen extends StatefulWidget {
  const ChatScreen({super.key});

  @override
  State<ChatScreen> createState() => _ChatScreenState();
}

class _ChatScreenState extends State<ChatScreen> {
  final TextEditingController _controller = TextEditingController();
  final List<ChatMessage> _messages = [];
  bool _isLoading = false;

  Future<void> _sendMessage() async {
    final text = _controller.text.trim();
    if (text.isEmpty) return;

    setState(() {
      _messages.add(ChatMessage(text: text, isUser: true));
      _isLoading = true;
    });
    _controller.clear();

    try {
      final request = http.Request(
        'POST',
        Uri.parse('http://127.0.0.1:8080/chat'),
      );
      request.headers['Content-Type'] = 'application/json';
      request.body = jsonEncode({'message': text});

      final response = await http.Client().send(request);
      final body = await response.stream.bytesToString();

      String agentResponse = '';
      for (final line in body.split('\n')) {
        if (line.startsWith('data:')) {
          agentResponse += line.substring(5).trim();
        }
      }

      setState(() {
        _messages.add(ChatMessage(
          text: agentResponse.isEmpty ? 'No response received.' : agentResponse,
          isUser: false,
        ));
        _isLoading = false;
      });
    } catch (e) {
      setState(() {
        _messages.add(ChatMessage(
          text: 'Error: Could not connect to agent backend. Is it running?',
          isUser: false,
        ));
        _isLoading = false;
      });
    }
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: const Text('Local AI Agent'),
        backgroundColor: Theme.of(context).colorScheme.inversePrimary,
      ),
      body: Column(
        children: [
          Expanded(
            child: ListView.builder(
              padding: const EdgeInsets.all(16),
              itemCount: _messages.length,
              itemBuilder: (context, index) {
                final msg = _messages[index];
                return Align(
                  alignment:
                      msg.isUser ? Alignment.centerRight : Alignment.centerLeft,
                  child: Container(
                    margin: const EdgeInsets.symmetric(vertical: 4),
                    padding: const EdgeInsets.all(12),
                    constraints: BoxConstraints(
                      maxWidth: MediaQuery.of(context).size.width * 0.75,
                    ),
                    decoration: BoxDecoration(
                      color: msg.isUser
                          ? Theme.of(context).colorScheme.primary
                          : Theme.of(context).colorScheme.surfaceContainerHighest,
                      borderRadius: BorderRadius.circular(12),
                    ),
                    child: Text(
                      msg.text,
                      style: TextStyle(
                        color: msg.isUser ? Colors.white : Colors.black87,
                      ),
                    ),
                  ),
                );
              },
            ),
          ),
          if (_isLoading)
            const Padding(
              padding: EdgeInsets.all(8.0),
              child: CircularProgressIndicator(),
            ),
          Padding(
            padding: const EdgeInsets.all(8.0),
            child: Row(
              children: [
                Expanded(
                  child: TextField(
                    controller: _controller,
                    decoration: const InputDecoration(
                      hintText: 'Ask your local AI agent...',
                      border: OutlineInputBorder(),
                    ),
                    onSubmitted: (_) => _sendMessage(),
                  ),
                ),
                const SizedBox(width: 8),
                IconButton(
                  onPressed: _sendMessage,
                  icon: const Icon(Icons.send),
                ),
              ],
            ),
          ),
        ],
      ),
    );
  }
}

And here is the complete pubspec.yaml:

name: agent_ui
description: A Flutter desktop chat UI for the local AI agent.
publish_to: 'none'
version: 1.0.0+1

environment:
  sdk: ^3.8.0

dependencies:
  flutter:
    sdk: flutter
  http: ^1.4.0

dev_dependencies:
  flutter_test:
    sdk: flutter
  flutter_lints: ^5.0.0

flutter:
  uses-material-design: true

Your Flutter chat UI is ready to talk to the Rust agent. Next up, you'll start all three services and watch the full agentic loop in action.

Test the Full Agent Loop

You have a Rust backend with an agentic loop, three registered tools, and a Flutter chat UI ready to connect. Now it's time to bring everything together and watch the full system in action.

In this step, you'll launch all three layers of the stack (Ollama, Rust backend, Flutter app) and send queries that trigger the model to autonomously decide which tools to call.

In this step, get ready to:
  • Start Ollama, the Rust backend, and the Flutter app.
  • Ask questions that trigger each of the three tools.
  • Observe the full agentic loop: reasoning, tool calling, and final answer.
Start Ollama, the Rust backend, and the Flutter app

The three services need to start in a specific order. Ollama must be running first (the Rust backend connects to it). The Rust backend must be running second (the Flutter app connects to it).

You'll need three separate terminal windows for this. Each service runs as a long-lived process that occupies its terminal.

  • Open a new terminal window (Terminal 1). Start the Ollama server by running:
ollama serve

Already seeing a message about Ollama running?

If Ollama was installed as a system service, it may already be running in the background. In that case you can skip this command and move to the next terminal.

  • Open a second terminal window (Terminal 2). Navigate to your Rust project and start the backend in release mode by running:
cd ~/Desktop/agent-backend
cargo run --release
  • Wait for the compilation to finish. You should see Agent backend running on http://127.0.0.1:8080 printed in the terminal.

Why release mode?

The --release flag tells Cargo to compile with optimizations enabled. For an agent backend that processes JSON and talks to a network API, release mode keeps response latency low.

  • Open a third terminal window (Terminal 3). Navigate to the Flutter project and launch the desktop app by running:
cd ~/Desktop/agent_ui
flutter run -d linux
  • Wait for the Flutter build to complete. A desktop window titled Local AI Agent should appear with the chat interface.

Flutter app not launching?

Make sure Linux desktop support is enabled. Run flutter config --enable-linux-desktop and try again.

If you see dependency errors, run flutter pub get inside the agent_ui folder first.

Help me debug my Flutter desktop app launch issue.

Ask questions that trigger tool use

With all three services running, you can now test the agentic loop. The model will read each question, decide whether a tool is needed, call it, and formulate a final answer using the result.

  • In the Flutter app's text field, type What time is it? and press **Enter** or click the send button.

After a few seconds, the agent should respond with the current date and time. This confirms the get_time tool was called successfully.

  • Next, type What is 847 * 293? and press **Enter**.

The agent should respond with 248171 (or a sentence containing that number). This confirms the calculate tool handled the multiplication.

  • Finally, type What OS am I running? and press **Enter**.

The agent should respond mentioning linux and your CPU architecture. This confirms the system_info tool was invoked.

Seeing 'Error: Could not connect to agent backend'?

Check Terminal 2. If the Rust backend crashed or never printed the "running on" message, it may have failed to compile. Look for error messages and fix any typos in your Rust source files.

If the backend is running but you see "Error contacting Ollama" in the response, confirm Ollama is active. Open a new terminal and run ollama list to check that qwen2.5:7b is available.

Help me debug my agent's connection issues.

Observe the full agentic loop in action

Each successful response you just received went through a multi-step reasoning process behind the scenes. Understanding this loop is the core concept of agentic AI.

What happened under the hood?

When you asked "What time is it?", this is the sequence that fired:

  • Flutter sent your message to the Rust backend's /chat endpoint.
  • The Rust backend sent your message plus the three tool schemas to Ollama's qwen2.5:7b model.
  • The model decided it needed the get_time tool and returned a tool_calls response.
  • The Rust backend executed GetTime.execute() and fed the result back to the model as a role: "tool" message.
  • The model used the tool result to compose a natural language answer and returned it.
  • The Rust backend streamed the final answer back to Flutter as an SSE event.

This is the exact same pattern that powers production AI assistants like Claude Code and Cursor. The model autonomously decides when tools are needed rather than following a hard-coded script.

  • Try asking a question that does NOT require a tool, like What is the capital of France? and press **Enter**.

The model should answer directly without calling any tool. This demonstrates that the model reasons about WHEN to use tools, not just how.

What is the agentic loop?

An agentic loop is a cycle where the LLM can call external tools and receive their outputs before generating a final answer. Your implementation caps this at 5 rounds (the for _round in 0..5 loop in agent.rs) to prevent infinite loops.

In each round, the model either returns a final text answer (loop ends) or requests another tool call (loop continues). This is what makes the system an agent rather than a simple chatbot.

Model not calling tools?

If the model answers time or math questions from memory instead of calling tools, make sure you pulled the correct model. Run ollama list and confirm qwen2.5:7b is listed. Smaller or older models may not support tool calling reliably.

Help me figure out why my model isn't using tools.

You now have a fully working local AI agent. The model reasons about your questions, autonomously decides when tools are needed, executes them through your Rust backend, and streams polished answers to the Flutter UI. Next up: extend your agent with a brand new tool in the Secret Mission.

Secret mission

Add a Web Fetch Tool

Your agent can tell the time, do math, and report system info. But what if it could reach out to the internet and fetch live web content? In this mission, you'll add a web_fetch tool that gives your local agent the ability to read any URL you point it at.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing cloud costs. The main resource to be aware of is the Ollama model file, which uses about 5GB of disk space and GPU VRAM or system RAM when loaded.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Why does Ollama use memory?

Ollama keeps recently used models loaded in GPU VRAM or system RAM so they respond faster. This means the qwen2.5:7b model continues using memory even after you close the Flutter app and Rust backend.

Resources you used:

  • qwen2.5:7b Ollama model (~5GB disk space, uses VRAM/RAM when loaded).
  • agent-backend Rust project folder on your Desktop.
  • agent_ui Flutter project folder on your Desktop.

Keep everything running

No action needed. Choose this if you're still actively experimenting with your local AI agent or plan to add more tools.

  • Your project folders stay on your Desktop for future editing.
  • The qwen2.5:7b model remains available for instant use.
  • Ollama, Rust, and Flutter stay installed on your system.

Pause - I'll come back to this later

Free up memory by unloading the model. Your project files and the model on disk stay intact for next time.

  • Unload the model from memory by running:
ollama stop qwen2.5:7b
  • This frees the GPU VRAM or system RAM the model was using.
  • The model file stays on disk so you can load it again instantly next time.
  • Your agent-backend and agent_ui folders remain on your Desktop.

Delete - I don't want to use this again

Remove the model and project folders to reclaim disk space.

  • Stop and remove the Ollama model by running:
ollama stop qwen2.5:7b
ollama rm qwen2.5:7b
  • Remove the Rust project folder by running:
rm -rf ~/Desktop/agent-backend
  • Remove the Flutter project folder by running:
rm -rf ~/Desktop/agent_ui
  • Confirm both folders are gone by running:
ls ~/Desktop
  • You should no longer see agent-backend or agent_ui in the output.

Nice Work!

Nice Work!

You just built a fully local AI agent from scratch. Your application combines a Rust backend, a local LLM via Ollama, and a Flutter desktop chat UI into a privacy-first AI assistant that runs entirely on your machine.

You've learned how to:

  • Built a Rust agentic backend with a type-safe Tool trait, a tool registry, and a multi-step reasoning loop that caps at 5 rounds to prevent infinite cycles.
  • Implemented the tool-calling pattern where the LLM autonomously decides when to invoke tools, your Rust code executes them locally, and results feed back into the conversation context for a final answer.
  • Created a Flutter desktop chat UI with real-time Server-Sent Events streaming, connecting all three layers (Flutter UI, Rust backend, Ollama LLM) into a working local agent.
  • Extended the agent with a web_fetch tool in the Secret Mission, proving the architecture is extensible to any new capability you can imagine.

Ready to quiz yourself?