Build a Voice AI Accountability Coach

Introduction

30 Second Summary

Everyone knows the struggle of sitting down to work with big intentions, then losing focus five minutes later. Having someone hold you accountable changes everything. That is why co-working sessions and accountability partners exist.

In this project, you will build a real-time voice AI accountability coach that runs timed focus sessions. You will speak your goals aloud, work while the AI keeps watch, and celebrate your progress at the end.

What You'll Build

A full-stack voice AI application using Google ADK, FastAPI, and the Gemini Live API that streams bidirectional audio through WebSockets for real-time spoken conversations with an AI coach.

By the end of this project, you'll have:

• A custom ADK agent with a coaching personality, session management tools, and cross-session memory.
• A FastAPI backend with WebSocket-based bidirectional audio streaming that connects to the Gemini Live API.
• A session timer frontend with real-time audio capture, playback, and a conversation transcript.
• A live deployment on Google Cloud Run accessible from any browser.
• Secret Mission: Add proactive mid-session check-ins where the AI detects silence and nudges you back on track.

Are there any prerequisites?

You will need Python 3.10+ installed, a Google API key from Google AI Studio, a GCP project with billing enabled, and the gcloud CLI installed and configured.

Step 1 walks you through confirming each of these is ready to go.

Before We Start

Before We Start...

Set Up Your Development Environment

Before building the voice AI agent, you need a clean project environment with the right packages installed.

Google ADK is the framework that handles real-time voice streaming between your app and Gemini. In this step, you'll create a Python project with ADK and its dependencies, then scaffold the folder structure your app needs.

In this step, get ready to:

• Create a Python virtual environment and install Google ADK with dependencies.
• Scaffold the project directory structure with all required files and folders.
• Configure environment variables for the Gemini API.

Create a virtual environment and install dependencies

A virtual environment keeps your project's packages isolated from your system Python. This prevents version conflicts between projects.

• Move to your Desktop by running this command:

cd ~/Desktop

• Create your project folder and move into it by running these commands:

mkdir accountability-coach
cd accountability-coach

• Confirm the folder exists by running:

ls

You should see an empty directory since you just created it.

• Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
• Type VS Code and press Enter to open it.
• In VS Code, click File in the top menu bar.
• Click Open Folder.
• Select the accountability-coach folder on your Desktop, then click Open.
• Open the integrated terminal by pressing Ctrl+` (backtick key, same on both macOS and Windows).

• Create a Python virtual environment by running this command:

python3 -m venv venv

• Activate the virtual environment:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

Make sure you're running the activation command from inside the accountability-coach folder where the venv directory was created.

On Windows, if you see an execution policy error, run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned in PowerShell first, then try activating again.

Why can't I activate my virtual environment?

Now you need a requirements.txt file to declare your project's dependencies. This file tells pip exactly which packages to install.

• In VS Code's Explorer sidebar, click the New File icon next to ACCOUNTABILITY-COACH.
• Name the file requirements.txt and press Enter.
• Paste the following content into requirements.txt:

google-adk>=1.20.0
fastapi>=0.115.0
uvicorn[standard]>=0.32.0
python-dotenv>=1.0.0

What are these packages?

• google-adk is Google's Agent Development Kit for building AI agents with streaming support.
• fastapi is the web framework that handles your WebSocket connections.
• uvicorn is the ASGI server that runs your FastAPI app.
• python-dotenv loads environment variables from your .env file.

• Save requirements.txt by pressing Cmd+S (macOS) or Ctrl+S (Windows).
• Install all the dependencies by running this command in your terminal:

pip install -r requirements.txt

You should see pip downloading and installing packages. The final line should say Successfully installed followed by the package names.

Seeing installation errors?

Make sure your virtual environment is active. You should see (venv) in your prompt. If not, re-run the activation command above.

If you see a Python version error, confirm you have Python 3.10 or higher by running python3 --version.

Why is pip install failing in my project?

Scaffold the project directory structure

Your project needs a specific folder layout. The accountability_coach/ directory holds your agent code as a Python package. The static/ directory holds the frontend files your browser loads.

• Create the required directories by running this command:

mkdir -p accountability_coach static/css static/js

What does -p do?

The -p flag creates parent directories as needed. So static/css creates both static/ and css/ inside it in one go.

• Create the empty project files by running this command:

touch main.py Dockerfile accountability_coach/__init__.py accountability_coach/agent.py accountability_coach/tools.py static/index.html static/css/style.css static/js/app.js static/js/audio-processor.js

• Confirm your file structure by running:

ls

You should see Dockerfile, accountability_coach, main.py, requirements.txt, static, and venv listed in the output.

Missing files or folders?

Make sure you ran both the mkdir and touch commands from inside the accountability-coach directory. If you're not sure where you are, run pwd to check your current path.

My project files are missing or in the wrong place.

Configure your environment variables

Google ADK needs two environment variables to connect to the Gemini API. The .env file stores these values locally so you never hardcode secrets into your application code.

• In VS Code's Explorer sidebar, click on the .env file you created earlier to open it.
• Paste the following content into .env:

GOOGLE_GENAI_USE_VERTEXAI=FALSE
GOOGLE_API_KEY=your-api-key-here

• Replace your-api-key-here with your actual Google API key from Google AI Studio.
• Save .env by pressing Cmd+S (macOS) or Ctrl+S (Windows).

What do these variables mean?

• GOOGLE_GENAI_USE_VERTEXAI=FALSE tells ADK to use Google AI Studio instead of Vertex AI. This is simpler and works with a standard API key.
• GOOGLE_API_KEY is the key ADK uses to authenticate your requests to the Gemini Live API.

Don't have a Google API key yet?

Go to Google AI Studio and click Create API key. Copy the key and paste it into your .env file.

✔️ Awesome, I've got everything!

Make sure you've saved both files. Your environment is ready to go.

ⓧ I'd like to double check the full code

Here's what your requirements.txt should look like:

google-adk>=1.20.0
fastapi>=0.115.0
uvicorn[standard]>=0.32.0
python-dotenv>=1.0.0

And here's your .env file:

GOOGLE_GENAI_USE_VERTEXAI=FALSE
GOOGLE_API_KEY=your-api-key-here

Your development environment is ready with all the tools you need. Next up, you'll define the accountability coach agent with its coaching personality and custom tools.

Define the Accountability Coach Agent

Your project structure is ready. Now you need the brain of your accountability coach: an ADK agent with a coaching personality and tools that track your goals across a session.

An ADK agent has three core pieces: tools (functions it can call), instructions (its personality and behavior rules), and a model (the AI backbone). You will build all three in this step, starting with the tools so the agent can reference them.

In this step, get ready to:

• Build three custom tools for goal tracking and session state management.
• Create the agent definition with coaching-focused system instructions.
• Configure the agent package to use the Gemini Live native audio model.

Build the goal tracking tools

ADK agents interact with the world through tools. A tool is a plain Python function that the agent can decide to call based on the conversation. Each tool receives a ToolContext object that gives it access to persistent session state.

Your accountability coach needs three tools: one to record goals when a session starts, one to mark the session complete, and one to retrieve past session history.

• Open accountability_coach/tools.py in your editor and add the imports and the first tool function by pasting this code:

"""Custom tools for the accountability coach agent."""

from google.adk.tools import ToolContext
import json


def record_goals(tool_context: ToolContext, goals: list[str]) -> dict:
    """Records the user's goals for the current session.

    Args:
        tool_context: The tool context provided by ADK.
        goals: A list of goal strings the user wants to accomplish.

    Returns:
        A confirmation dictionary with the recorded goals.
    """
    tool_context.state["session_goals"] = json.dumps(goals)
    tool_context.state["session_status"] = "in_progress"
    tool_context.state["user:latest_goals"] = json.dumps(goals)
    return {
        "status": "recorded",
        "goals": goals,
        "message": f"Recorded {len(goals)} goals for this session.",
    }

What does this code do?

The record_goals function saves the user's declared goals into the session state dictionary. ADK automatically tracks any changes to tool_context.state.

Keys prefixed with user: persist across sessions for the same user. This is how the agent remembers goals from previous sessions.

You won't see a change yet. The next chunk adds the remaining tools to this file.

• Below the record_goals function, add the session completion tool:

def complete_session(
    tool_context: ToolContext,
    completed_goals: list[str],
    incomplete_goals: list[str],
    notes: str = "",
) -> dict:
    """Marks the session as complete and records progress.

    Args:
        tool_context: The tool context provided by ADK.
        completed_goals: Goals that were finished.
        incomplete_goals: Goals that were not finished.
        notes: Any additional notes about the session.

    Returns:
        A summary dictionary of the session outcome.
    """
    tool_context.state["session_status"] = "completed"
    tool_context.state["session_completed_goals"] = json.dumps(completed_goals)
    tool_context.state["session_incomplete_goals"] = json.dumps(incomplete_goals)
    tool_context.state["session_notes"] = notes

    # Update user-level stats
    total_str = tool_context.state.get("user:total_sessions", "0")
    total_sessions = int(total_str) + 1
    tool_context.state["user:total_sessions"] = str(total_sessions)
    tool_context.state["user:last_session_notes"] = notes

    return {
        "status": "session_completed",
        "completed": completed_goals,
        "incomplete": incomplete_goals,
        "total_sessions": total_sessions,
        "message": "Session recorded successfully.",
    }

What does this code do?

The complete_session tool runs at the end of a focus session. It separates goals into completed and incomplete, then increments the user's lifetime session counter.

The user:total_sessions key uses the user: prefix so the running total survives across multiple sessions.

One more tool to go. The next function lets the agent pull up past history.

• Below the complete_session function, add the session summary retrieval tool:

def get_session_summary(tool_context: ToolContext) -> dict:
    """Retrieves a summary of the user's history for context.

    Args:
        tool_context: The tool context provided by ADK.

    Returns:
        A dictionary with the user's session history summary.
    """
    total_sessions = tool_context.state.get("user:total_sessions", "0")
    latest_goals_str = tool_context.state.get("user:latest_goals", "[]")
    last_notes = tool_context.state.get("user:last_session_notes", "")

    return {
        "total_sessions": int(total_sessions),
        "latest_goals": json.loads(latest_goals_str),
        "last_session_notes": last_notes,
    }

What does this code do?

The get_session_summary tool reads the user's history from state. The agent calls this at the start of a new session to personalize its greeting with references to past goals and streaks.

• Save accountability_coach/tools.py.
• Verify the file loads correctly by running:

python -c "from accountability_coach.tools import record_goals, complete_session, get_session_summary; print('Tools loaded successfully')"

You should see Tools loaded successfully printed in your terminal.

Seeing a ModuleNotFoundError?

Make sure your virtual environment is activated. You should see (venv) at the start of your terminal prompt.

If you see No module named 'google.adk', run pip install -r requirements.txt again inside the activated environment.

Also confirm you are running the command from your project root folder (the one containing the accountability_coach/ directory). Help me fix my import error

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before moving on.

ⓧ I'd like to double check the full code

"""Custom tools for the accountability coach agent."""

from google.adk.tools import ToolContext
import json


def record_goals(tool_context: ToolContext, goals: list[str]) -> dict:
    """Records the user's goals for the current session.

    Args:
        tool_context: The tool context provided by ADK.
        goals: A list of goal strings the user wants to accomplish.

    Returns:
        A confirmation dictionary with the recorded goals.
    """
    tool_context.state["session_goals"] = json.dumps(goals)
    tool_context.state["session_status"] = "in_progress"
    tool_context.state["user:latest_goals"] = json.dumps(goals)
    return {
        "status": "recorded",
        "goals": goals,
        "message": f"Recorded {len(goals)} goals for this session.",
    }


def complete_session(
    tool_context: ToolContext,
    completed_goals: list[str],
    incomplete_goals: list[str],
    notes: str = "",
) -> dict:
    """Marks the session as complete and records progress.

    Args:
        tool_context: The tool context provided by ADK.
        completed_goals: Goals that were finished.
        incomplete_goals: Goals that were not finished.
        notes: Any additional notes about the session.

    Returns:
        A summary dictionary of the session outcome.
    """
    tool_context.state["session_status"] = "completed"
    tool_context.state["session_completed_goals"] = json.dumps(completed_goals)
    tool_context.state["session_incomplete_goals"] = json.dumps(incomplete_goals)
    tool_context.state["session_notes"] = notes

    # Update user-level stats
    total_str = tool_context.state.get("user:total_sessions", "0")
    total_sessions = int(total_str) + 1
    tool_context.state["user:total_sessions"] = str(total_sessions)
    tool_context.state["user:last_session_notes"] = notes

    return {
        "status": "session_completed",
        "completed": completed_goals,
        "incomplete": incomplete_goals,
        "total_sessions": total_sessions,
        "message": "Session recorded successfully.",
    }


def get_session_summary(tool_context: ToolContext) -> dict:
    """Retrieves a summary of the user's history for context.

    Args:
        tool_context: The tool context provided by ADK.

    Returns:
        A dictionary with the user's session history summary.
    """
    total_sessions = tool_context.state.get("user:total_sessions", "0")
    latest_goals_str = tool_context.state.get("user:latest_goals", "[]")
    last_notes = tool_context.state.get("user:last_session_notes", "")

    return {
        "total_sessions": int(total_sessions),
        "latest_goals": json.loads(latest_goals_str),
        "last_session_notes": last_notes,
    }

Create the agent definition with coaching instructions

With the tools ready, you can now define the agent itself. The Agent class ties together a model, system instructions, and tools into one cohesive unit that ADK can run.

• Open accountability_coach/agent.py and paste the full agent definition:

"""Accountability Coach Agent definition."""

from google.adk.agents import Agent
from google.adk.tools import FunctionTool
from .tools import record_goals, complete_session, get_session_summary


root_agent = Agent(
    name="accountability_coach",
    model="gemini-2.5-flash-native-audio-preview-12-2025",
    description="A voice-based accountability coach that runs timed focus sessions.",
    instruction="""You are a warm, direct accountability coach. Your job is to help the user stay focused and productive during timed work sessions.

SESSION FLOW:
1. GREETING PHASE: When a session starts, greet the user warmly. If you have memory of previous sessions, reference their past goals and progress. Ask what they want to accomplish in this session. Be specific - push them to name concrete tasks, not vague intentions.

2. GOAL DECLARATION: Once the user states their goals, use the record_goals tool to save them. Confirm the goals back to the user and wish them good focus. Keep this brief - they want to work, not chat.

3. FOCUS PHASE: During the work period, be quiet unless the user speaks to you. If they talk to you, respond helpfully but briefly. If they express frustration, acknowledge it and redirect them. If they want to switch tasks, help them decide and update goals.

4. CELEBRATION PHASE: When the session timer ends, congratulate the user. Ask what they accomplished. Use the complete_session tool to record their progress. Be genuinely encouraging about any progress made, even partial completion.

PERSONALITY:
- Warm but no-nonsense. Like a supportive friend who keeps you honest.
- Brief responses during focus time. Save longer conversations for greeting and celebration.
- Use the user's name if you know it from memory.
- Celebrate small wins. Progress matters more than perfection.
- If the user seems distracted or unfocused, gently call it out.

MEMORY:
- When you have access to past session memories, use them to personalize. Reference streaks, recurring goals, or past challenges.
- Track patterns: if the user always struggles with a certain type of task, acknowledge it and offer strategies.""",
    tools=[record_goals, complete_session, get_session_summary],
)

What does this code do?

The Agent() constructor wires everything together. The instruction field is the system prompt that defines how the agent behaves throughout each session phase.

The tools list tells the agent which functions it can call. ADK automatically reads each tool's docstring and type hints to generate tool descriptions for the model.

The session flow (greeting, goal declaration, focus, celebration) mirrors how a real accountability partner session works on platforms like Focusmate.

• Save accountability_coach/agent.py.

You won't verify this file independently. The next substep sets up the package export, and then you will confirm everything loads together.

✔️ Awesome, I've got everything!

Great. Make sure you have saved the file before continuing.

ⓧ I'd like to double check the full code

"""Accountability Coach Agent definition."""

from google.adk.agents import Agent
from google.adk.tools import FunctionTool
from .tools import record_goals, complete_session, get_session_summary


root_agent = Agent(
    name="accountability_coach",
    model="gemini-2.5-flash-native-audio-preview-12-2025",
    description="A voice-based accountability coach that runs timed focus sessions.",
    instruction="""You are a warm, direct accountability coach. Your job is to help the user stay focused and productive during timed work sessions.

SESSION FLOW:
1. GREETING PHASE: When a session starts, greet the user warmly. If you have memory of previous sessions, reference their past goals and progress. Ask what they want to accomplish in this session. Be specific - push them to name concrete tasks, not vague intentions.

2. GOAL DECLARATION: Once the user states their goals, use the record_goals tool to save them. Confirm the goals back to the user and wish them good focus. Keep this brief - they want to work, not chat.

3. FOCUS PHASE: During the work period, be quiet unless the user speaks to you. If they talk to you, respond helpfully but briefly. If they express frustration, acknowledge it and redirect them. If they want to switch tasks, help them decide and update goals.

4. CELEBRATION PHASE: When the session timer ends, congratulate the user. Ask what they accomplished. Use the complete_session tool to record their progress. Be genuinely encouraging about any progress made, even partial completion.

PERSONALITY:
- Warm but no-nonsense. Like a supportive friend who keeps you honest.
- Brief responses during focus time. Save longer conversations for greeting and celebration.
- Use the user's name if you know it from memory.
- Celebrate small wins. Progress matters more than perfection.
- If the user seems distracted or unfocused, gently call it out.

MEMORY:
- When you have access to past session memories, use them to personalize. Reference streaks, recurring goals, or past challenges.
- Track patterns: if the user always struggles with a certain type of task, acknowledge it and offer strategies.""",
    tools=[record_goals, complete_session, get_session_summary],
)

Configure the package export for the native audio model

The agent uses gemini-2.5-flash-native-audio-preview-12-2025 as its model. This is a native audio model, meaning it processes speech directly without converting to text first.

Why native audio instead of half-cascade?

Gemini offers two voice approaches. Half-cascade models convert speech to text internally, process it, then convert the response back to speech. Native audio models handle the audio signal end-to-end.

Native audio supports proactive speech (the AI can talk without being prompted), lower latency, and emotional tone adaptation. These are critical for a coach that needs to feel natural and can initiate check-ins.

ADK discovers your agent through the package's __init__.py file. You need to export root_agent so the runner can find it.

• Open accountability_coach/__init__.py and add this single line:

from .agent import root_agent

• Save accountability_coach/__init__.py.
• Verify the entire agent package loads by running:

python -c "from accountability_coach import root_agent; print(f'Agent loaded: {root_agent.name}')"

You should see Agent loaded: accountability_coach printed in your terminal.

Seeing an ImportError?

If you see ImportError: cannot import name 'record_goals', double check that tools.py is saved and the function names match exactly: record_goals, complete_session, get_session_summary.

If you see SyntaxError, check for unmatched quotes or parentheses in your agent instruction string. The triple-quoted string in agent.py must open and close with """.

Help me debug my agent import error

Your agent is defined and its tools are ready. Next up, you will build the FastAPI backend that streams real-time audio between your browser and this agent over WebSockets.

Build the FastAPI WebSocket Backend

Your ADK agent knows how to coach accountability sessions, but it has no way to receive voice input or stream audio responses back. The agent needs a server that keeps a persistent two-way connection open between the browser and the Gemini Live API.

In this step, you will build a FastAPI backend that uses WebSockets to stream audio in both directions simultaneously. This is the bridge between your frontend microphone and the Gemini Live model powering your coach.

In this step, get ready to:

• Set up the FastAPI application with static file serving and a root route.
• Implement the upstream/downstream concurrent task pattern with LiveRequestQueue.
• Configure RunConfig for native audio streaming with transcription.

Set up the FastAPI application with WebSocket endpoint for bidirectional streaming

The application scaffold creates a FastAPI server that serves your frontend files and initializes the ADK Runner. The Runner is the core ADK component that connects your agent to the Gemini model and manages sessions.

• Open main.py in your project root and add the following code:

"""FastAPI application for the Voice AI Accountability Coach."""

import asyncio
import json
import logging
import os
from pathlib import Path

from dotenv import load_dotenv

load_dotenv()

from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from fastapi.responses import FileResponse
from fastapi.staticfiles import StaticFiles
from google.adk.agents.run_config import RunConfig, StreamingMode
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.genai import types
from google.adk.agents.live_request_queue import LiveRequestQueue

from accountability_coach.agent import root_agent

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

APP_NAME = "accountability-coach"

# Create the FastAPI app and ADK session infrastructure
app = FastAPI(title="Voice AI Accountability Coach")
session_service = InMemorySessionService()
runner = Runner(
    app_name=APP_NAME,
    agent=root_agent,
    session_service=session_service,
)

# Serve static frontend files from the static/ directory
static_dir = Path(__file__).parent / "static"
app.mount("/static", StaticFiles(directory=str(static_dir)), name="static")


@app.get("/")
async def root():
    """Serve the main page."""
    return FileResponse(str(static_dir / "index.html"))

What does this code do?

This scaffold sets up several key pieces:

• load_dotenv() loads your .env file so the ADK can find your Google API key.
• InMemorySessionService stores session state in memory. Each user's goals and progress persist as long as the server is running.
• Runner connects your root_agent to the Gemini model and handles the streaming lifecycle.
• The /static mount serves your frontend files (HTML, CSS, JS) that you will build in the next step.

Why is load_dotenv() called before the other imports?

The ADK reads GOOGLE_API_KEY from environment variables at import time. Calling load_dotenv() first ensures those values are available before the ADK modules initialize.

Implement the upstream/downstream concurrent task pattern with LiveRequestQueue

Bidirectional voice streaming needs two tasks running at the same time: one that receives audio from the user's microphone (upstream) and one that sends AI responses back (downstream). The ADK provides LiveRequestQueue as the queue between your WebSocket and the Gemini Live API.

• Add the WebSocket endpoint below the root() function in main.py:

@app.websocket("/ws/{user_id}/{session_id}")
async def websocket_endpoint(
    websocket: WebSocket,
    user_id: str,
    session_id: str,
) -> None:
    """WebSocket endpoint for bidirectional voice streaming."""
    await websocket.accept()
    logger.info(f"Connection accepted: user={user_id}, session={session_id}")

    # Configure for native audio streaming
    run_config = RunConfig(
        streaming_mode=StreamingMode.BIDI,
        response_modalities=["AUDIO"],
        input_audio_transcription=types.AudioTranscriptionConfig(),
        output_audio_transcription=types.AudioTranscriptionConfig(),
        session_resumption=types.SessionResumptionConfig(),
    )

    # Get or create session
    session = await session_service.get_session(
        app_name=APP_NAME, user_id=user_id, session_id=session_id
    )
    if not session:
        await session_service.create_session(
            app_name=APP_NAME, user_id=user_id, session_id=session_id
        )

    # Create LiveRequestQueue for this connection
    live_request_queue = LiveRequestQueue()

    async def upstream_task() -> None:
        """Receives WebSocket messages and sends to LiveRequestQueue."""
        try:
            while True:
                message = await websocket.receive()
                if message.get("type") == "websocket.disconnect":
                    break

                if "text" in message:
                    data = json.loads(message["text"])
                    if data.get("type") == "text":
                        content = types.Content(
                            parts=[types.Part(text=data["content"])]
                        )
                        live_request_queue.send_content(content)
                elif "bytes" in message:
                    audio_blob = types.Blob(
                        data=message["bytes"],
                        mime_type="audio/pcm;rate=16000",
                    )
                    live_request_queue.send_realtime(audio_blob)
        except WebSocketDisconnect:
            pass
        except Exception as e:
            logger.error(f"Upstream error: {e}")

    async def downstream_task() -> None:
        """Receives Events from run_live() and sends to WebSocket."""
        try:
            async for event in runner.run_live(
                user_id=user_id,
                session_id=session_id,
                live_request_queue=live_request_queue,
                run_config=run_config,
            ):
                await websocket.send_text(
                    event.model_dump_json(exclude_none=True, by_alias=True)
                )
        except Exception as e:
            logger.error(f"Downstream error: {e}")

    try:
        await asyncio.gather(
            upstream_task(),
            downstream_task(),
            return_exceptions=True,
        )
    except Exception as e:
        logger.error(f"Session error: {e}")
    finally:
        live_request_queue.close()
        logger.info(f"Session ended: user={user_id}, session={session_id}")


if __name__ == "__main__":
    import uvicorn

    uvicorn.run(app, host="0.0.0.0", port=8000)

What does this code do?

This endpoint is the heart of your voice streaming architecture. Here is how each piece fits together:

• RunConfig tells ADK to use bidirectional streaming mode with audio responses. The transcription configs give you text versions of both input and output audio.
• LiveRequestQueue acts as the bridge between your WebSocket and the Gemini Live API. You push audio into it, and the runner pulls from it.
• upstream_task() reads messages from the browser. Text messages go through send_content() and raw audio bytes go through send_realtime() with PCM 16kHz format.
• downstream_task() iterates over events from runner.run_live() and forwards them to the browser as JSON.
• asyncio.gather() runs both tasks concurrently so audio flows in both directions at the same time.

Why this concurrent task pattern?

Voice conversations require true bidirectionality. The user might speak while the AI is still responding (an interruption). If you used a simple request-response model, you would have to wait for the AI to finish before listening again.

Running upstream and downstream as separate concurrent tasks means the user can interrupt the AI mid-sentence. The Gemini Live API handles voice activity detection (VAD) automatically when it receives new audio input during a response.

• Save main.py.

Seeing an import error?

• Make sure your virtual environment is activated. You should see the environment name in your terminal prompt.
• Confirm you installed dependencies with pip install -r requirements.txt in a previous step.
• Check that the accountability_coach/__init__.py file exists and contains from .agent import root_agent.

Help me debug an import error in my FastAPI main.py file.

Add session initialization with RunConfig for audio modality and transcription

Your backend is complete. The RunConfig you added tells the Gemini Live API exactly how to handle audio. The final step is to start the server and confirm everything connects properly.

• Start the server by running this command:

uvicorn main:app --host 0.0.0.0 --port 8000

• You should see output like Uvicorn running on http://0.0.0.0:8000 in your terminal.
• Open http://localhost:8000 in your browser.

You will see a blank page or a "not found" response. That is expected because you have not created the static/index.html file yet. The important thing is that the server started without errors.

What does each RunConfig parameter do?

• StreamingMode.BIDI opens a persistent WebSocket to the Gemini Live API with bidirectional audio, interruptions, and automatic voice activity detection.
• response_modalities=["AUDIO"] tells Gemini to respond with spoken audio instead of text.
• AudioTranscriptionConfig() on both input and output gives you text transcripts alongside the audio. Your frontend will display these in the conversation panel.
• SessionResumptionConfig() enables reconnection without losing context if the connection drops briefly.

• Press Ctrl+C in your terminal to stop the server for now.

Server won't start?

• If you see ModuleNotFoundError, make sure your virtual environment is activated and all packages from requirements.txt are installed.
• If you see Address already in use, another process is using port 8000. Stop it or change the port number in the command.
• If you see an error about GOOGLE_API_KEY, confirm your .env file exists in the project root with your actual API key.

Help me debug why my uvicorn server won't start.

✔️ Awesome, I've got everything!

Great. Double check you have saved main.py and that the server started without errors.

ⓧ I'd like to double check the full code

Here is the complete main.py file. Compare it against yours to spot any differences.

"""FastAPI application for the Voice AI Accountability Coach."""

import asyncio
import json
import logging
import os
from pathlib import Path

from dotenv import load_dotenv

load_dotenv()

from fastapi import FastAPI, WebSocket, WebSocketDisconnect
from fastapi.responses import FileResponse
from fastapi.staticfiles import StaticFiles
from google.adk.agents.run_config import RunConfig, StreamingMode
from google.adk.runners import Runner
from google.adk.sessions import InMemorySessionService
from google.genai import types
from google.adk.agents.live_request_queue import LiveRequestQueue

from accountability_coach.agent import root_agent

logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

APP_NAME = "accountability-coach"

app = FastAPI(title="Voice AI Accountability Coach")
session_service = InMemorySessionService()
runner = Runner(
    app_name=APP_NAME,
    agent=root_agent,
    session_service=session_service,
)

static_dir = Path(__file__).parent / "static"
app.mount("/static", StaticFiles(directory=str(static_dir)), name="static")


@app.get("/")
async def root():
    """Serve the main page."""
    return FileResponse(str(static_dir / "index.html"))


@app.websocket("/ws/{user_id}/{session_id}")
async def websocket_endpoint(
    websocket: WebSocket,
    user_id: str,
    session_id: str,
) -> None:
    """WebSocket endpoint for bidirectional voice streaming."""
    await websocket.accept()
    logger.info(f"Connection accepted: user={user_id}, session={session_id}")

    # Configure for native audio streaming
    run_config = RunConfig(
        streaming_mode=StreamingMode.BIDI,
        response_modalities=["AUDIO"],
        input_audio_transcription=types.AudioTranscriptionConfig(),
        output_audio_transcription=types.AudioTranscriptionConfig(),
        session_resumption=types.SessionResumptionConfig(),
    )

    # Get or create session
    session = await session_service.get_session(
        app_name=APP_NAME, user_id=user_id, session_id=session_id
    )
    if not session:
        await session_service.create_session(
            app_name=APP_NAME, user_id=user_id, session_id=session_id
        )

    # Create LiveRequestQueue for this connection
    live_request_queue = LiveRequestQueue()

    async def upstream_task() -> None:
        """Receives WebSocket messages and sends to LiveRequestQueue."""
        try:
            while True:
                message = await websocket.receive()
                if message.get("type") == "websocket.disconnect":
                    break

                if "text" in message:
                    data = json.loads(message["text"])
                    if data.get("type") == "text":
                        content = types.Content(
                            parts=[types.Part(text=data["content"])]
                        )
                        live_request_queue.send_content(content)
                elif "bytes" in message:
                    audio_blob = types.Blob(
                        data=message["bytes"],
                        mime_type="audio/pcm;rate=16000",
                    )
                    live_request_queue.send_realtime(audio_blob)
        except WebSocketDisconnect:
            pass
        except Exception as e:
            logger.error(f"Upstream error: {e}")

    async def downstream_task() -> None:
        """Receives Events from run_live() and sends to WebSocket."""
        try:
            async for event in runner.run_live(
                user_id=user_id,
                session_id=session_id,
                live_request_queue=live_request_queue,
                run_config=run_config,
            ):
                await websocket.send_text(
                    event.model_dump_json(exclude_none=True, by_alias=True)
                )
        except Exception as e:
            logger.error(f"Downstream error: {e}")

    try:
        await asyncio.gather(
            upstream_task(),
            downstream_task(),
            return_exceptions=True,
        )
    except Exception as e:
        logger.error(f"Session error: {e}")
    finally:
        live_request_queue.close()
        logger.info(f"Session ended: user={user_id}, session={session_id}")


if __name__ == "__main__":
    import uvicorn

    uvicorn.run(app, host="0.0.0.0", port=8000)

Your backend is ready to stream voice conversations. Next up, you will build the frontend that captures audio from your microphone, sends it to this WebSocket endpoint, and plays back the AI coach's spoken responses.

Build the Session Timer Frontend

Your backend can stream voice data bidirectionally, but you have no way to talk to it yet. The frontend needs to capture microphone audio, stream it over the WebSocket connection, play the agent's audio response, and manage session timing.

In this step, you will build a complete browser-based UI with a 25-minute session timer, microphone controls, and real-time audio streaming using the Web Audio API. The frontend communicates with your FastAPI backend over a single WebSocket connection, sending PCM audio upstream and playing PCM audio downstream.

In this step, get ready to:

• Create the HTML page with session controls, timer display, and audio interface.
• Build AudioWorklet processors for PCM recording and playback.
• Wire up the JavaScript WebSocket client with session timer and phase transitions.

Create the HTML page with session controls and timer display

The HTML page provides the visual structure for your accountability session. It includes a countdown timer, buttons to start/end sessions, a microphone toggle, and a scrollable transcript area.

• Open static/index.html in your code editor and paste in the following code:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Voice AI Accountability Coach</title>
    <link rel="stylesheet" href="/static/css/style.css">
</head>
<body>
    <header>
        <h1>Accountability Coach</h1>
        <div class="connection-status">
            <span class="status-dot" id="statusDot"></span>
            <span id="statusText">Disconnected</span>
        </div>
    </header>

    <main>
        <section class="timer-section">
            <div class="timer" id="timer">25:00</div>
            <div class="session-phase" id="sessionPhase">Ready to start</div>
        </section>

        <section class="controls-section">
            <button id="startSessionBtn" class="btn btn-primary">Start Session</button>
            <button id="endSessionBtn" class="btn btn-danger" disabled>End Session</button>
            <button id="micBtn" class="btn btn-mic" disabled>🎤 Mic Off</button>
        </section>

        <section class="transcript-section">
            <h2>Conversation</h2>
            <div id="transcript"></div>
        </section>
    </main>

    <script src="/static/js/audio-processor.js" type="module"></script>
    <script src="/static/js/app.js" type="module"></script>
</body>
</html>

What does this page do?

The header shows a connection status dot that changes color based on WebSocket state. The timer-section displays a 25-minute countdown and the current session phase (e.g., Goal Declaration, Focus, Celebration).

The three buttons control session flow: start connects the WebSocket, end wraps up early, and the mic button toggles audio streaming. The transcript div shows real-time transcriptions of both your speech and the agent's responses.

Now add the dark theme styling. Open static/css/style.css and paste in the following:

• Open static/css/style.css and paste in the following styles:

* {
    margin: 0;
    padding: 0;
    box-sizing: border-box;
}

body {
    font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
    background: #1a1a2e;
    color: #e0e0e0;
    min-height: 100vh;
}

header {
    display: flex;
    justify-content: space-between;
    align-items: center;
    padding: 1rem 2rem;
    background: #16213e;
    border-bottom: 1px solid #0f3460;
}

header h1 {
    font-size: 1.4rem;
    color: #4fc3f7;
}

.connection-status {
    display: flex;
    align-items: center;
    gap: 0.5rem;
    font-size: 0.85rem;
}

.status-dot {
    width: 10px;
    height: 10px;
    border-radius: 50%;
    background: #666;
}

.status-dot.connected {
    background: #4caf50;
}

.status-dot.connecting {
    background: #ff9800;
}

main {
    max-width: 600px;
    margin: 0 auto;
    padding: 2rem;
}

.timer-section {
    text-align: center;
    margin-bottom: 2rem;
}

.timer {
    font-size: 4rem;
    font-weight: 700;
    font-variant-numeric: tabular-nums;
    color: #4fc3f7;
    margin-bottom: 0.5rem;
}

.session-phase {
    font-size: 1rem;
    color: #aaa;
    text-transform: uppercase;
    letter-spacing: 0.1em;
}

.controls-section {
    display: flex;
    gap: 1rem;
    justify-content: center;
    margin-bottom: 2rem;
    flex-wrap: wrap;
}

.btn {
    padding: 0.75rem 1.5rem;
    border: none;
    border-radius: 8px;
    font-size: 1rem;
    cursor: pointer;
    transition: opacity 0.2s;
}

.btn:disabled {
    opacity: 0.4;
    cursor: not-allowed;
}

.btn-primary {
    background: #4fc3f7;
    color: #1a1a2e;
    font-weight: 600;
}

.btn-danger {
    background: #ef5350;
    color: white;
}

.btn-mic {
    background: #2e2e4a;
    color: #e0e0e0;
    border: 2px solid #4fc3f7;
}

.btn-mic.active {
    background: #4fc3f7;
    color: #1a1a2e;
}

.transcript-section {
    background: #16213e;
    border-radius: 12px;
    padding: 1.5rem;
    max-height: 300px;
    overflow-y: auto;
}

.transcript-section h2 {
    font-size: 1rem;
    color: #4fc3f7;
    margin-bottom: 1rem;
}

#transcript {
    display: flex;
    flex-direction: column;
    gap: 0.75rem;
}

.message {
    padding: 0.5rem 0.75rem;
    border-radius: 8px;
    font-size: 0.9rem;
    line-height: 1.4;
}

.message.user {
    background: #0f3460;
    align-self: flex-end;
    max-width: 80%;
}

.message.agent {
    background: #2e2e4a;
    align-self: flex-start;
    max-width: 80%;
}

What do these styles do?

The dark theme uses #1a1a2e as the base background with lighter navy panels. The timer is styled at 4rem with tabular-nums so digits don't jump around during countdown.

The .status-dot turns green when connected and orange while connecting. Message bubbles use align-self to push user messages to the right and agent messages to the left, like a chat interface.

• Save both static/index.html and static/css/style.css.
• Start the server by running:

uvicorn main:app --host 0.0.0.0 --port 8000

• Open http://localhost:8000 in your browser. You should see a dark-themed page with a large 25:00 timer, three buttons (Start Session, End Session, and Mic Off), and a Conversation panel below.

Seeing a blank page or 404?

Make sure your static/ folder structure matches exactly: static/index.html, static/css/style.css. The FastAPI app mounts static/ from the same directory as main.py.

If the page loads but has no styling, check that the <link> tag uses /static/css/style.css (with the leading slash). Help me debug why my page is not loading correctly.

Build the audio worklet processors

The Gemini Live API expects 16-bit PCM audio at 16kHz for input and returns 16-bit PCM at 24kHz for output. Standard browser audio APIs work with 32-bit floating point samples. You need custom AudioWorklet processors to convert between these formats in real time.

• Open static/js/audio-processor.js and paste in the following code:

/**
 * AudioWorklet processors for PCM recording and playback.
 */

class PcmRecorderProcessor extends AudioWorkletProcessor {
    process(inputs) {
        const input = inputs[0];
        if (input && input[0]) {
            const float32 = input[0];
            // Convert float32 samples (-1 to 1) to int16 for Gemini
            const int16 = new Int16Array(float32.length);
            for (let i = 0; i < float32.length; i++) {
                const s = Math.max(-1, Math.min(1, float32[i]));
                int16[i] = s < 0 ? s * 0x8000 : s * 0x7FFF;
            }
            this.port.postMessage(int16.buffer, [int16.buffer]);
        }
        return true;
    }
}

class PcmPlayerProcessor extends AudioWorkletProcessor {
    constructor() {
        super();
        this.buffer = [];
        this.port.onmessage = (event) => {
            // Convert incoming int16 PCM back to float32 for speakers
            const int16 = new Int16Array(event.data);
            const float32 = new Float32Array(int16.length);
            for (let i = 0; i < int16.length; i++) {
                float32[i] = int16[i] / 0x7FFF;
            }
            this.buffer.push(...float32);
        };
    }

    process(inputs, outputs) {
        const output = outputs[0];
        if (output && output[0]) {
            const channel = output[0];
            // Drain buffered samples into the output channel
            for (let i = 0; i < channel.length; i++) {
                channel[i] = this.buffer.length > 0 ? this.buffer.shift() : 0;
            }
        }
        return true;
    }
}

registerProcessor('pcm-recorder-processor', PcmRecorderProcessor);
registerProcessor('pcm-player-processor', PcmPlayerProcessor);

What do these processors do?

• PcmRecorderProcessor runs on every audio frame from the microphone. It converts the browser's float32 samples to 16-bit integers and posts the raw bytes to the main thread for WebSocket transmission.
• PcmPlayerProcessor does the reverse. It receives int16 PCM bytes from the agent, converts them to float32, and drains them into the speaker output buffer frame by frame.
• Both processors run on a dedicated audio thread, which prevents audio glitches even when the main thread is busy updating the UI.

• Save static/js/audio-processor.js.

You will not see a change in the browser yet. The next file uses these processors to capture and play audio.

Wire up the WebSocket client with session timer and phase transitions

The main application script connects all the pieces: it opens a WebSocket to your backend, streams microphone audio through the recorder worklet, plays agent responses through the player worklet, and manages a 25-minute countdown with three phases (goal declaration, focus, and celebration).

• Open static/js/app.js and paste in the following code:

/**
 * Main application logic for the Voice AI Accountability Coach.
 */

const SESSION_DURATION_SECONDS = 25 * 60;

let websocket = null;
let audioContext = null;
let mediaStream = null;
let recorderWorklet = null;
let playerWorklet = null;
let isRecording = false;
let timerInterval = null;
let remainingSeconds = SESSION_DURATION_SECONDS;
let sessionActive = false;

// Cache DOM references
const statusDot = document.getElementById('statusDot');
const statusText = document.getElementById('statusText');
const timerDisplay = document.getElementById('timer');
const sessionPhase = document.getElementById('sessionPhase');
const startBtn = document.getElementById('startSessionBtn');
const endBtn = document.getElementById('endSessionBtn');
const micBtn = document.getElementById('micBtn');
const transcript = document.getElementById('transcript');

function generateId() {
    return crypto.randomUUID();
}

// Persist userId across browser sessions
const userId = localStorage.getItem('userId') || (() => {
    const id = generateId();
    localStorage.setItem('userId', id);
    return id;
})();

function updateTimer() {
    const minutes = Math.floor(remainingSeconds / 60);
    const seconds = remainingSeconds % 60;
    timerDisplay.textContent =
        `${minutes.toString().padStart(2, '0')}:${seconds.toString().padStart(2, '0')}`;
}

function startTimer() {
    timerInterval = setInterval(() => {
        remainingSeconds--;
        updateTimer();
        if (remainingSeconds <= 0) {
            clearInterval(timerInterval);
            sessionPhase.textContent = 'Celebration Phase';
            // Tell the agent the timer ended so it triggers celebration
            sendText('The session timer has ended. Please initiate the celebration phase and ask me about my progress.');
        }
    }, 1000);
}

function addMessage(text, role) {
    const div = document.createElement('div');
    div.className = `message ${role}`;
    div.textContent = text;
    transcript.appendChild(div);
    transcript.scrollTop = transcript.scrollHeight;
}

function sendText(text) {
    if (websocket && websocket.readyState === WebSocket.OPEN) {
        websocket.send(JSON.stringify({ type: 'text', content: text }));
    }
}

async function initAudio() {
    audioContext = new AudioContext({ sampleRate: 24000 });

    // Load and register audio worklet processors
    await audioContext.audioWorklet.addModule('/static/js/audio-processor.js');

    // Player worklet for agent audio output (24kHz)
    playerWorklet = new AudioWorkletNode(audioContext, 'pcm-player-processor');
    playerWorklet.connect(audioContext.destination);

    // Recorder setup (16kHz for input)
    const recorderContext = new AudioContext({ sampleRate: 16000 });
    await recorderContext.audioWorklet.addModule('/static/js/audio-processor.js');

    mediaStream = await navigator.mediaDevices.getUserMedia({ audio: true });
    const source = recorderContext.createMediaStreamSource(mediaStream);
    recorderWorklet = new AudioWorkletNode(recorderContext, 'pcm-recorder-processor');
    source.connect(recorderWorklet);

    // Forward recorded PCM bytes to WebSocket when mic is active
    recorderWorklet.port.onmessage = (event) => {
        if (isRecording && websocket && websocket.readyState === WebSocket.OPEN) {
            websocket.send(event.data);
        }
    };
}

function connectWebSocket(sessionId) {
    const protocol = window.location.protocol === 'https:' ? 'wss:' : 'ws:';
    const url = `${protocol}//${window.location.host}/ws/${userId}/${sessionId}`;

    statusDot.className = 'status-dot connecting';
    statusText.textContent = 'Connecting...';

    websocket = new WebSocket(url);

    websocket.onopen = () => {
        statusDot.className = 'status-dot connected';
        statusText.textContent = 'Connected';
        micBtn.disabled = false;
        endBtn.disabled = false;
        sessionPhase.textContent = 'Goal Declaration Phase';
        startTimer();
    };

    websocket.onmessage = (event) => {
        const data = JSON.parse(event.data);
        handleEvent(data);
    };

    websocket.onclose = () => {
        statusDot.className = 'status-dot';
        statusText.textContent = 'Disconnected';
        micBtn.disabled = true;
    };

    websocket.onerror = (err) => {
        console.error('WebSocket error:', err);
        statusText.textContent = 'Error';
    };
}

function handleEvent(event) {
    // Handle transcription events
    if (event.content && event.content.parts) {
        for (const part of event.content.parts) {
            if (part.text) {
                const role = event.content.role === 'user' ? 'user' : 'agent';
                addMessage(part.text, role);
            }
        }
    }

    // Handle audio output
    if (event.content && event.content.parts) {
        for (const part of event.content.parts) {
            if (part.inline_data && part.inline_data.mime_type &&
                part.inline_data.mime_type.startsWith('audio/')) {
                const audioBytes = base64ToArrayBuffer(part.inline_data.data);
                playAudio(audioBytes);
            }
        }
    }
}

function base64ToArrayBuffer(base64) {
    const binary = atob(base64);
    const bytes = new Uint8Array(binary.length);
    for (let i = 0; i < binary.length; i++) {
        bytes[i] = binary.charCodeAt(i);
    }
    return bytes.buffer;
}

function playAudio(arrayBuffer) {
    if (playerWorklet) {
        playerWorklet.port.postMessage(arrayBuffer, [arrayBuffer]);
    }
}

// Start a new accountability session
startBtn.addEventListener('click', async () => {
    const sessionId = generateId();
    sessionActive = true;
    remainingSeconds = SESSION_DURATION_SECONDS;
    updateTimer();
    startBtn.disabled = true;

    await initAudio();
    connectWebSocket(sessionId);
});

// End session early
endBtn.addEventListener('click', () => {
    if (websocket) {
        sendText('I want to end this session early. Please wrap up and ask me about my progress.');
        sessionPhase.textContent = 'Wrapping up...';
    }
    clearInterval(timerInterval);
});

// Toggle microphone
micBtn.addEventListener('click', () => {
    isRecording = !isRecording;
    micBtn.textContent = isRecording ? '🎤 Mic On' : '🎤 Mic Off';
    micBtn.classList.toggle('active', isRecording);
});

updateTimer();

What does this code do?

• userId is persisted in localStorage so the backend can maintain session state across browser refreshes.
• initAudio() creates two AudioContexts: one at 24kHz for playing agent speech and one at 16kHz for recording your microphone. The Gemini Live API requires these specific sample rates.
• connectWebSocket() builds the URL using the same /ws/{user_id}/{session_id} pattern your FastAPI endpoint expects. On connection, it starts the timer and transitions to the Goal Declaration phase.
• handleEvent() parses ADK events and routes them: text parts become transcript messages, audio parts get decoded from base64 and sent to the player worklet for playback.
• When the timer hits zero, the app sends a text message to the agent instructing it to start the celebration phase. This triggers the agent to ask about your progress and record the session.

• Save static/js/app.js.
• Refresh your browser at http://localhost:8000. Click Start Session. Your browser will ask for microphone permission. Allow it.
• You should see the status dot turn orange (connecting), then green (connected). The timer starts counting down and the session phase changes to GOAL DECLARATION PHASE.
• Click the Mic Off button to toggle it to Mic On. Speak a few words. After a moment, you should see transcriptions appear in the Conversation panel and hear the agent respond through your speakers.

Status dot stays orange or shows Error?

Make sure the uvicorn server is still running in your terminal. Check the terminal for error messages. If you see GOOGLE_API_KEY errors, verify your .env file has your actual API key (not the placeholder).

If the microphone permission dialog did not appear, check that you are accessing the page via localhost (not an IP address). Browsers require HTTPS or localhost for microphone access. Help me debug my WebSocket connection.

✔️ Awesome, I've got everything!

Make sure you have saved all four frontend files before moving on.

ⓧ I'd like to double check the full code

Here is the complete code for each frontend file. Compare your files against these:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Voice AI Accountability Coach</title>
    <link rel="stylesheet" href="/static/css/style.css">
</head>
<body>
    <header>
        <h1>Accountability Coach</h1>
        <div class="connection-status">
            <span class="status-dot" id="statusDot"></span>
            <span id="statusText">Disconnected</span>
        </div>
    </header>

    <main>
        <section class="timer-section">
            <div class="timer" id="timer">25:00</div>
            <div class="session-phase" id="sessionPhase">Ready to start</div>
        </section>

        <section class="controls-section">
            <button id="startSessionBtn" class="btn btn-primary">Start Session</button>
            <button id="endSessionBtn" class="btn btn-danger" disabled>End Session</button>
            <button id="micBtn" class="btn btn-mic" disabled>🎤 Mic Off</button>
        </section>

        <section class="transcript-section">
            <h2>Conversation</h2>
            <div id="transcript"></div>
        </section>
    </main>

    <script src="/static/js/audio-processor.js" type="module"></script>
    <script src="/static/js/app.js" type="module"></script>
</body>
</html>

* {
    margin: 0;
    padding: 0;
    box-sizing: border-box;
}

body {
    font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
    background: #1a1a2e;
    color: #e0e0e0;
    min-height: 100vh;
}

header {
    display: flex;
    justify-content: space-between;
    align-items: center;
    padding: 1rem 2rem;
    background: #16213e;
    border-bottom: 1px solid #0f3460;
}

header h1 {
    font-size: 1.4rem;
    color: #4fc3f7;
}

.connection-status {
    display: flex;
    align-items: center;
    gap: 0.5rem;
    font-size: 0.85rem;
}

.status-dot {
    width: 10px;
    height: 10px;
    border-radius: 50%;
    background: #666;
}

.status-dot.connected {
    background: #4caf50;
}

.status-dot.connecting {
    background: #ff9800;
}

main {
    max-width: 600px;
    margin: 0 auto;
    padding: 2rem;
}

.timer-section {
    text-align: center;
    margin-bottom: 2rem;
}

.timer {
    font-size: 4rem;
    font-weight: 700;
    font-variant-numeric: tabular-nums;
    color: #4fc3f7;
    margin-bottom: 0.5rem;
}

.session-phase {
    font-size: 1rem;
    color: #aaa;
    text-transform: uppercase;
    letter-spacing: 0.1em;
}

.controls-section {
    display: flex;
    gap: 1rem;
    justify-content: center;
    margin-bottom: 2rem;
    flex-wrap: wrap;
}

.btn {
    padding: 0.75rem 1.5rem;
    border: none;
    border-radius: 8px;
    font-size: 1rem;
    cursor: pointer;
    transition: opacity 0.2s;
}

.btn:disabled {
    opacity: 0.4;
    cursor: not-allowed;
}

.btn-primary {
    background: #4fc3f7;
    color: #1a1a2e;
    font-weight: 600;
}

.btn-danger {
    background: #ef5350;
    color: white;
}

.btn-mic {
    background: #2e2e4a;
    color: #e0e0e0;
    border: 2px solid #4fc3f7;
}

.btn-mic.active {
    background: #4fc3f7;
    color: #1a1a2e;
}

.transcript-section {
    background: #16213e;
    border-radius: 12px;
    padding: 1.5rem;
    max-height: 300px;
    overflow-y: auto;
}

.transcript-section h2 {
    font-size: 1rem;
    color: #4fc3f7;
    margin-bottom: 1rem;
}

#transcript {
    display: flex;
    flex-direction: column;
    gap: 0.75rem;
}

.message {
    padding: 0.5rem 0.75rem;
    border-radius: 8px;
    font-size: 0.9rem;
    line-height: 1.4;
}

.message.user {
    background: #0f3460;
    align-self: flex-end;
    max-width: 80%;
}

.message.agent {
    background: #2e2e4a;
    align-self: flex-start;
    max-width: 80%;
}

/**
 * AudioWorklet processors for PCM recording and playback.
 */

class PcmRecorderProcessor extends AudioWorkletProcessor {
    process(inputs) {
        const input = inputs[0];
        if (input && input[0]) {
            const float32 = input[0];
            const int16 = new Int16Array(float32.length);
            for (let i = 0; i < float32.length; i++) {
                const s = Math.max(-1, Math.min(1, float32[i]));
                int16[i] = s < 0 ? s * 0x8000 : s * 0x7FFF;
            }
            this.port.postMessage(int16.buffer, [int16.buffer]);
        }
        return true;
    }
}

class PcmPlayerProcessor extends AudioWorkletProcessor {
    constructor() {
        super();
        this.buffer = [];
        this.port.onmessage = (event) => {
            const int16 = new Int16Array(event.data);
            const float32 = new Float32Array(int16.length);
            for (let i = 0; i < int16.length; i++) {
                float32[i] = int16[i] / 0x7FFF;
            }
            this.buffer.push(...float32);
        };
    }

    process(inputs, outputs) {
        const output = outputs[0];
        if (output && output[0]) {
            const channel = output[0];
            for (let i = 0; i < channel.length; i++) {
                channel[i] = this.buffer.length > 0 ? this.buffer.shift() : 0;
            }
        }
        return true;
    }
}

registerProcessor('pcm-recorder-processor', PcmRecorderProcessor);
registerProcessor('pcm-player-processor', PcmPlayerProcessor);

/**
 * Main application logic for the Voice AI Accountability Coach.
 */

const SESSION_DURATION_SECONDS = 25 * 60;

let websocket = null;
let audioContext = null;
let mediaStream = null;
let recorderWorklet = null;
let playerWorklet = null;
let isRecording = false;
let timerInterval = null;
let remainingSeconds = SESSION_DURATION_SECONDS;
let sessionActive = false;

const statusDot = document.getElementById('statusDot');
const statusText = document.getElementById('statusText');
const timerDisplay = document.getElementById('timer');
const sessionPhase = document.getElementById('sessionPhase');
const startBtn = document.getElementById('startSessionBtn');
const endBtn = document.getElementById('endSessionBtn');
const micBtn = document.getElementById('micBtn');
const transcript = document.getElementById('transcript');

function generateId() {
    return crypto.randomUUID();
}

const userId = localStorage.getItem('userId') || (() => {
    const id = generateId();
    localStorage.setItem('userId', id);
    return id;
})();

function updateTimer() {
    const minutes = Math.floor(remainingSeconds / 60);
    const seconds = remainingSeconds % 60;
    timerDisplay.textContent =
        `${minutes.toString().padStart(2, '0')}:${seconds.toString().padStart(2, '0')}`;
}

function startTimer() {
    timerInterval = setInterval(() => {
        remainingSeconds--;
        updateTimer();
        if (remainingSeconds <= 0) {
            clearInterval(timerInterval);
            sessionPhase.textContent = 'Celebration Phase';
            sendText('The session timer has ended. Please initiate the celebration phase and ask me about my progress.');
        }
    }, 1000);
}

function addMessage(text, role) {
    const div = document.createElement('div');
    div.className = `message ${role}`;
    div.textContent = text;
    transcript.appendChild(div);
    transcript.scrollTop = transcript.scrollHeight;
}

function sendText(text) {
    if (websocket && websocket.readyState === WebSocket.OPEN) {
        websocket.send(JSON.stringify({ type: 'text', content: text }));
    }
}

async function initAudio() {
    audioContext = new AudioContext({ sampleRate: 24000 });

    // Load and register audio worklet processors
    await audioContext.audioWorklet.addModule('/static/js/audio-processor.js');

    // Player worklet for agent audio output (24kHz)
    playerWorklet = new AudioWorkletNode(audioContext, 'pcm-player-processor');
    playerWorklet.connect(audioContext.destination);

    // Recorder setup (16kHz for input)
    const recorderContext = new AudioContext({ sampleRate: 16000 });
    await recorderContext.audioWorklet.addModule('/static/js/audio-processor.js');

    mediaStream = await navigator.mediaDevices.getUserMedia({ audio: true });
    const source = recorderContext.createMediaStreamSource(mediaStream);
    recorderWorklet = new AudioWorkletNode(recorderContext, 'pcm-recorder-processor');
    source.connect(recorderWorklet);

    recorderWorklet.port.onmessage = (event) => {
        if (isRecording && websocket && websocket.readyState === WebSocket.OPEN) {
            websocket.send(event.data);
        }
    };
}

function connectWebSocket(sessionId) {
    const protocol = window.location.protocol === 'https:' ? 'wss:' : 'ws:';
    const url = `${protocol}//${window.location.host}/ws/${userId}/${sessionId}`;

    statusDot.className = 'status-dot connecting';
    statusText.textContent = 'Connecting...';

    websocket = new WebSocket(url);

    websocket.onopen = () => {
        statusDot.className = 'status-dot connected';
        statusText.textContent = 'Connected';
        micBtn.disabled = false;
        endBtn.disabled = false;
        sessionPhase.textContent = 'Goal Declaration Phase';
        startTimer();
    };

    websocket.onmessage = (event) => {
        const data = JSON.parse(event.data);
        handleEvent(data);
    };

    websocket.onclose = () => {
        statusDot.className = 'status-dot';
        statusText.textContent = 'Disconnected';
        micBtn.disabled = true;
    };

    websocket.onerror = (err) => {
        console.error('WebSocket error:', err);
        statusText.textContent = 'Error';
    };
}

function handleEvent(event) {
    // Handle transcription events
    if (event.content && event.content.parts) {
        for (const part of event.content.parts) {
            if (part.text) {
                const role = event.content.role === 'user' ? 'user' : 'agent';
                addMessage(part.text, role);
            }
        }
    }

    // Handle audio output
    if (event.content && event.content.parts) {
        for (const part of event.content.parts) {
            if (part.inline_data && part.inline_data.mime_type &&
                part.inline_data.mime_type.startsWith('audio/')) {
                const audioBytes = base64ToArrayBuffer(part.inline_data.data);
                playAudio(audioBytes);
            }
        }
    }
}

function base64ToArrayBuffer(base64) {
    const binary = atob(base64);
    const bytes = new Uint8Array(binary.length);
    for (let i = 0; i < binary.length; i++) {
        bytes[i] = binary.charCodeAt(i);
    }
    return bytes.buffer;
}

function playAudio(arrayBuffer) {
    if (playerWorklet) {
        playerWorklet.port.postMessage(arrayBuffer, [arrayBuffer]);
    }
}

startBtn.addEventListener('click', async () => {
    const sessionId = generateId();
    sessionActive = true;
    remainingSeconds = SESSION_DURATION_SECONDS;
    updateTimer();
    startBtn.disabled = true;

    await initAudio();
    connectWebSocket(sessionId);
});

endBtn.addEventListener('click', () => {
    if (websocket) {
        sendText('I want to end this session early. Please wrap up and ask me about my progress.');
        sessionPhase.textContent = 'Wrapping up...';
    }
    clearInterval(timerInterval);
});

micBtn.addEventListener('click', () => {
    isRecording = !isRecording;
    micBtn.textContent = isRecording ? '🎤 Mic On' : '🎤 Mic Off';
    micBtn.classList.toggle('active', isRecording);
});

updateTimer();

Your frontend is complete and your accountability coach is fully functional locally. Next up, you will add cross-session memory so the agent remembers your goals and progress between sessions.

Add Cross-Session Memory

Your accountability coach works for a single session, but it has no idea who you are when you come back tomorrow.

A real coach remembers your goals, celebrates your streaks, and calls out patterns. In this step, you will understand how the user: prefix in ADK state already gives your agent cross-session memory, then test it by running multiple sessions and verifying the agent recalls your previous goals.

In this step, get ready to:

• Understand how ADK's state prefix system enables persistent memory across sessions.
• Review how your tools store goals and progress to user-scoped state.
• Test cross-session memory by running two sessions and verifying recall.

Understand the user: prefix memory pattern

ADK uses a prefix system on state keys to control how long data lives and who can access it.

• Keys with no prefix (like session_goals) are scoped to the current session only. They disappear when the session ends.
• Keys with the user: prefix (like user:latest_goals) are tied to the user_id and shared across all sessions for that user.
• Keys with the app: prefix are global across all users and sessions for the entire application.

What is user-scoped state?

When your tools write to a key like user:total_sessions, ADK's session service stores that value against the user's ID. The next time the same user starts a new session, that value is already available in session.state without any extra code.

With InMemorySessionService, this persistence lasts as long as the server process is running. If you restart the server, the memory resets. For production use, you would swap to DatabaseSessionService for persistence that survives restarts.

Review how your tools implement cross-session memory

Your tools already write user-scoped state. Open accountability_coach/tools.py and find the record_goals function.

• In tools.py, find this line inside record_goals:

    tool_context.state["user:latest_goals"] = json.dumps(goals)

What does this line do?

The user: prefix tells ADK to persist this value across sessions. Every time the user declares goals, the latest set is saved. When they start a new session later, the agent can read user:latest_goals and reference what they worked on last time.

• Now find the complete_session function and locate these lines:

    # Update user-level stats
    total_str = tool_context.state.get("user:total_sessions", "0")
    total_sessions = int(total_str) + 1
    tool_context.state["user:total_sessions"] = str(total_sessions)
    tool_context.state["user:last_session_notes"] = notes

What does this code do?

When a session ends, complete_session increments the total session count and saves the session notes. Both use the user: prefix, so this data carries over to the next session automatically.

The get_session_summary tool reads these same keys back, giving the agent context about the user's history at any point in the conversation.

• Finally, look at how the agent's instruction references memory. In accountability_coach/agent.py, find this section:

MEMORY:
- When you have access to past session memories, use them to personalize. Reference streaks, recurring goals, or past challenges.
- Track patterns: if the user always struggles with a certain type of task, acknowledge it and offer strategies.

How does the agent know to use memory?

The agent's instruction tells it to reference past session data when available. Combined with the get_session_summary tool, the agent can ask ADK for the user's history at the start of each session. The GREETING PHASE instruction also tells the agent to reference past goals and progress.

Why InMemorySessionService instead of a database?

For local development, InMemorySessionService keeps things simple. It stores user: prefixed state in memory and shares it across sessions for the same user as long as the server stays running.

This is perfect for testing and personal use. If you needed memory to survive server restarts, you would swap to DatabaseSessionService with a PostgreSQL or SQLite backend. The tools and agent code would stay exactly the same because ADK handles persistence based on the prefix.

Test cross-session memory with two sessions

The best way to confirm memory works is to run two sessions back-to-back without restarting the server.

• Start the server by running this command:

python main.py

• Open your browser and navigate to http://localhost:8000.
• Click Start Session and click the Mic button to enable your microphone.
• Declare a specific goal by saying something like "I want to finish writing chapter 3 of my report."
• Wait for the agent to confirm your goal, then click End Session.
• Tell the agent what you accomplished when it asks during the celebration phase.

The agent just used record_goals and complete_session behind the scenes. Your goals and session count are now stored in user-scoped state.

• Refresh the page in your browser.
• Click Start Session again to begin a second session.
• Enable your microphone and listen to the agent's greeting.

The agent should reference your previous session. It might say something like "Welcome back. Last time you were working on chapter 3. How did that go?" This is cross-session memory in action.

What should I see?

In the second session, the agent's greeting should reference your previous goals or mention that you have completed one session before. The exact wording varies because it is generated by the LLM, but it should clearly personalize based on your history.

You can also check your terminal logs. You should see the agent calling get_session_summary at the start of the second session to retrieve your history.

Agent not referencing your previous session?

Make sure you did not restart the server between sessions. InMemorySessionService loses all state when the process stops.

Verify that your browser is using the same userId across sessions. Open your browser console and run localStorage.getItem('userId'). If you cleared localStorage or switched browsers, the server sees you as a different user.

Help me debug my cross-session memory

Your accountability coach now remembers who you are across sessions. It tracks your goals, session count, and progress notes. Next up, you will containerize this application and deploy it to Google Cloud Run so you can access your coach from anywhere.

Deploy to Google Cloud Run

Your accountability coach works perfectly on your machine. Now it is time to make it accessible from anywhere by deploying it to the cloud.

Google Cloud Run is a serverless container platform that scales to zero when idle. It supports WebSocket connections out of the box, which makes it a perfect fit for your real-time voice streaming app.

In this step, get ready to:

• Create a Dockerfile to containerize your application.
• Deploy to Cloud Run with WebSocket support and session timeout configured.
• Test the deployed application with a live voice session.

Create the Dockerfile

A Dockerfile tells Cloud Run exactly how to build and run your application inside a container. Cloud Run expects the app to listen on port 8080 by default.

• Create a file called Dockerfile in the root of your project folder (the same level as main.py).
• Add the following content to your Dockerfile:

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8080

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

What does this Dockerfile do?

• FROM python:3.11-slim starts from a lightweight Python image.
• COPY requirements.txt and RUN pip install installs dependencies first so Docker can cache this layer separately from your code.
• COPY . . copies your entire project into the container.
• EXPOSE 8080 documents the port Cloud Run expects.
• CMD starts Uvicorn on port 8080 when the container launches.

• Save Dockerfile.
• Confirm the file exists by running:

ls Dockerfile

You should see Dockerfile printed in the output.

Don't see the file?

Make sure the file is named exactly Dockerfile with a capital D and no file extension. It should not be Dockerfile.txt or dockerfile.

Help me troubleshoot my Dockerfile

Deploy to Cloud Run

The gcloud CLI can build your container image and deploy it to Cloud Run in a single command. You need to configure the timeout to support 25-minute voice sessions and pass your API key as an environment variable.

• Make sure you are in your project root folder (the one containing main.py and Dockerfile).
• Deploy your app to Cloud Run by running this command:

gcloud run deploy accountability-coach \
  --source . \
  --region us-central1 \
  --allow-unauthenticated \
  --set-env-vars GOOGLE_API_KEY=your-api-key-here,GOOGLE_GENAI_USE_VERTEXAI=FALSE \
  --timeout=1800 \
  --session-affinity

What do these flags do?

• --source . tells gcloud to build the container image directly from your project files using Cloud Build.
• --region us-central1 deploys to the US Central region.
• --allow-unauthenticated makes your app publicly accessible so you can open it in any browser.
• --set-env-vars passes your API key and config to the container as environment variables.
• --timeout=1800 extends the request timeout to 30 minutes. The default is only 5 minutes, which would cut off your 25-minute sessions.
• --session-affinity ensures repeat connections from the same user route to the same container instance. This keeps your in-memory session data consistent.

Replace your API key

Replace your-api-key-here in the command above with your actual Google API key from the .env file. This is the same key you used for local development.

• The deployment takes 2-5 minutes. gcloud will ask you to enable a few APIs if they are not already enabled. Enter y to confirm each one.
• When the deployment finishes, look for the Service URL line in the output. It looks something like https://accountability-coach-xxxx-uc.a.run.app.

Deployment failed?

• If you see a billing error, make sure your GCP project has billing enabled.
• If the build fails with a Docker error, check that your Dockerfile has no typos and is in the same folder as main.py.
• If you see a permissions error, run gcloud auth login to re-authenticate your CLI.

Help me debug my Cloud Run deployment

Test the deployed application

Your accountability coach is now live on the internet. Time to verify it works end-to-end with a real voice session.

• Copy the Service URL from the deployment output.
• Open the URL in your browser (Chrome recommended for best Web Audio API support).
• Click Start Session.
• Your browser will ask for microphone permission. Click Allow.
• Wait for the status dot to turn green and show Connected.
• Click the Mic Off button to toggle your microphone on.
• Speak your goals for a test session. You should hear the AI coach respond through your speakers.

What should I see?

The connection status should show a green dot with "Connected". The timer starts counting down from 25:00. The session phase reads "Goal Declaration Phase".

When you speak with the mic on, you should hear the coach greet you and ask about your goals. Transcriptions of both your speech and the AI's responses appear in the Conversation panel.

Not hearing any audio response?

• Make sure you clicked the microphone button so it shows "Mic On" with the blue highlight.
• Check that your browser's volume is not muted. The AI responds through the Web Audio API, not a standard media player.
• If the status shows "Disconnected" after a few seconds, your API key may be incorrect. Re-deploy with the correct key.

Help me debug my deployed voice AI app

• Once you have confirmed the AI responds to your voice, click End Session to wrap up the test.

✔️ Awesome, I've got everything!

Great. Double check you have saved your Dockerfile and that your Cloud Run service is running.

ⓧ I'd like to double check the full code

Here is the complete Dockerfile for reference:

FROM python:3.11-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8080

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8080"]

Your voice AI accountability coach is now deployed and accessible from any browser. Anyone with the URL can start a focus session and get real-time voice coaching from the AI.

Secret mission

Proactive Mid-Session Check-Ins

Your coach stays quiet during focus time. But what if it could nudge you when you've been silent too long? In this extension, you'll enable Gemini's proactive audio feature so the AI initiates encouraging check-ins during your session.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project uses Google Cloud Run, which scales to zero when idle. You won't be charged unless the service receives requests.

Cost warning

Cloud Run has a generous free tier of 2 million requests per month. If you leave your service deployed but don't use it, it costs nothing because it scales to zero automatically.

However, each 25-minute voice session at paid tier costs approximately $0.10-$0.56 in Gemini API usage. Monitor your usage in the Google Cloud Console if you exceed the free tier limits.

Resources you used:

• A Cloud Run service (accountability-coach).
• A container image in Artifact Registry.
• A local project folder with Python virtual environment.
• A .env file containing your Google API key.

Keep everything running

No action needed. Choose this if you're still actively using your accountability coach or want to keep it available for daily sessions.

• Your Cloud Run service scales to zero when idle, so it costs nothing between sessions.
• Your container image stays in Artifact Registry (minimal storage cost).
• Your local project files remain on your Desktop for future development.

Pause - I'll come back to this later

Cloud Run automatically scales to zero, so there is no separate pause action for the service itself. You can deactivate your local environment to free up disk space.

• Your Cloud Run service stays deployed but receives no traffic and incurs no cost when idle.
• Deactivate your Python virtual environment by running deactivate in your terminal.
• Keep your project folder and .env file for when you return.

Delete - I don't want to use this again

Remove all project resources and start fresh. This deletes your deployed service and local files permanently.

• Delete the Cloud Run service by running:

gcloud run services delete accountability-coach --region=us-central1

• Delete the container image from Artifact Registry through the Google Cloud Console. Navigate to Artifact Registry, find the image, and click Delete.
• Delete your local project folder from your Desktop.
• Revoke your Google API key in Google AI Studio if you no longer need it.

Nice Work!

Nice Work!

You just built and deployed a real-time voice AI accountability coach from scratch. This is a production-grade application that combines agent design, bidirectional audio streaming, and cloud deployment into one working system.

You've learned how to:

• Build a custom ADK agent with coaching personality, session management tools, and cross-session memory using state persistence.
• Implement bidirectional audio streaming using FastAPI WebSockets, the upstream/downstream concurrent task pattern, and LiveRequestQueue.
• Deploy a full-stack voice application to Google Cloud Run with WebSocket support and appropriate timeout configuration.
• Added proactive mid-session voice check-ins using Gemini's native audio proactivity feature so the AI nudges you back on track during silence.

Ready to quiz yourself?