Build a Local AI Voice Assistant

Wire together Whisper, Ollama, and Piper TTS into a British voice assistant.

Introduction

30 Second Summary

Voice assistants like Siri and Alexa are convenient, but they send everything you say to the cloud. What if you could build one that runs entirely on your own machine, keeping your conversations completely private?

In this project, you will build a local AI voice assistant that listens to your speech, processes it through a local language model, and speaks back to you in a British accent. You'll connect three AI systems together into a pipeline that can both chat and translate languages.

What You'll Build

A Python-powered voice assistant that chains together OpenAI Whisper (speech-to-text), Ollama (local LLM), and Piper TTS (text-to-speech with a British English voice) into a seamless pipeline that runs without an internet connection.

By the end of this project, you'll have:

  • A speech-to-text recording system that captures your voice from the microphone and transcribes it using Whisper.
  • A connection to a local LLM via Ollama that generates conversational responses and language translations.
  • British English text-to-speech output that speaks responses aloud using Piper TTS with the en_GB-vctk-medium voice model.
  • Secret Mission: A live interpreter mode that continuously translates everything you say in real-time without needing to press a button each time.

Are there any prerequisites?

You'll need basic Python knowledge (variables, functions, and loops) and a Windows 11 machine with at least 16 GB of RAM. The local language model and Whisper both run in memory, so 16 GB is the minimum for a smooth experience.

Install Python 3.11.9 and Core Tools

Your voice assistant relies on three AI systems working together: speech recognition, a language model, and text-to-speech. Each of these systems has specific tool requirements.

In this step, you'll install the foundation tools that power the pipeline. You'll set up a compatible version of Python, download a local language model via Ollama, and install an audio processing tool that Whisper needs to read sound files.

In this step, get ready to:
  • Install Python 3.11.9 alongside your existing Python.
  • Install Ollama and download a local language model.
  • Install ffmpeg for audio processing.
Install Python 3.11.9

The speech-to-text library you'll use (Whisper) depends on a package called numba, which only works with Python versions 3.10 through 3.13. Your existing Python 3.14.6 is too new, so you'll install 3.11.9 alongside it.

Why Python 3.11.9 specifically?

Python 3.11.9 is the last 3.11 release that ships with a Windows binary installer. It's the most reliable version for the Whisper + Piper + audio tools stack on Windows.

  • Download the Python 3.11.9 installer by opening this link in your browser:
https://www.python.org/ftp/python/3.11.9/python-3.11.9-amd64.exe
  • Once the download finishes, double-click python-3.11.9-amd64.exe to launch the installer.
  • On the first screen, check the box at the bottom that says Add python.exe to PATH.
  • Click Customize installation (not "Install Now").

Why Customize installation?

Choosing Customize installation installs Python 3.11.9 into its own directory, keeping it separate from your existing Python 3.14.6. This way both versions coexist without conflict.

  • On the Optional Features screen, leave all defaults checked and click Next.
  • On the Advanced Options screen, leave the defaults and click Install.
  • Wait for the installation to complete, then click Close.

Now verify the installation by opening a new terminal window.

  • Press the Windows key to open your search bar.
  • Type PowerShell and press Enter to open it.
  • Check that Python 3.11.9 is accessible by running this command:
py -3.11 --version

✔️ I see Python 3.11.9

Python 3.11.9 is installed and accessible through the py launcher. You're ready to move on to the next tool.

ⓧ I see an error or "not recognized"

The Python Launcher for Windows may not have registered 3.11.9 yet. Try these fixes:

  • Close your current PowerShell window and open a brand new one (PATH changes only apply to new windows).
  • Confirm you checked Add python.exe to PATH during installation. If you missed it, re-run the installer, choose Modify, and enable the PATH option.
  • If py itself is not recognized, your Python 3.14 installation may not have included the launcher. Re-run the 3.11.9 installer and ensure the py launcher checkbox is checked on the Optional Features screen.
Install Ollama and download the language model

Ollama is a tool that runs large language models (LLMs) locally on your machine. Instead of sending your data to a cloud API, everything stays private on your computer.

  • Open this link in your browser to go to the Ollama download page:
https://ollama.com/download/windows
  • Click Download for Windows to download OllamaSetup.exe.
  • Double-click OllamaSetup.exe to run the installer.
  • Follow the prompts to complete the installation.

Does Ollama need admin rights?

No. Ollama installs in your home directory and does not require administrator privileges. After installation, it runs a background service that serves the language model API at http://localhost:11434.

Now download the language model your assistant will use for thinking and generating responses.

  • Open a new PowerShell window (the old one may not see the new ollama command yet).
  • Download the llama3.2:3b model by running this command:
ollama pull llama3.2:3b

What is llama3.2:3b?

This is a 3-billion parameter language model from Meta. It's about a 2 GB download and runs comfortably on 16 GB of RAM. It handles conversation and translation well without needing a GPU.

This download may take a few minutes depending on your internet speed. Once it finishes, verify the model is available.

  • Confirm the model downloaded successfully by running:
ollama list

✔️ I see llama3.2:3b in the list

Your local language model is ready. Ollama is set up and serving the model in the background.

ⓧ I see an empty list

The model pull may not have completed successfully. Try these fixes:

  • Run ollama pull llama3.2:3b again. If the download was interrupted, Ollama resumes from where it left off.
  • Check your internet connection. The model is about 2 GB and requires a stable connection.

ⓧ Command not recognized

Ollama did not install correctly or the PATH was not updated. Try these fixes:

  • Close your PowerShell window and open a brand new one.
  • If the command still fails, re-run OllamaSetup.exe and restart your computer after installation.
Install ffmpeg

Whisper needs a tool called ffmpeg to process audio files. It handles converting and reading audio in different formats.

  • In your PowerShell window, install ffmpeg by running this command:
winget install Gyan.FFmpeg

What does this command do?

The winget command is the Windows Package Manager. It downloads and installs ffmpeg (a popular audio/video processing tool) automatically, similar to how app stores work.

  • After the installation finishes, close your PowerShell window and open a new one (so the PATH updates).
  • Verify ffmpeg is accessible by running:
ffmpeg -version

✔️ I see version information

ffmpeg is installed and ready. You now have all three foundation tools set up.

ⓧ Command not recognized

ffmpeg was installed but your terminal cannot find it yet. Try these fixes:

  • Make sure you opened a brand new PowerShell window after the install finished.
  • If it still does not work, restart your computer to force a full PATH refresh, then try ffmpeg -version again.

Your foundation tools are in place. Next up, you'll create your project folder, set up a virtual environment, and write the code that records your voice and transcribes it to text.

Set Up Your Project and Speech-to-Text

Your tools are installed and ready to go. Now it is time to build the first piece of your voice assistant: the ability to listen to your voice and convert it into text.

In this step, you will create an isolated Python project, install the audio and transcription libraries, and write a script that records your voice and prints out what you said.

In this step, get ready to:
  • Create a project folder with a Python 3.11 virtual environment and install dependencies.
  • Write audio recording and speech-to-text transcription code using Whisper.
  • Test the full speech-to-text pipeline by speaking into your microphone.
Create your project folder and virtual environment

A virtual environment is an isolated copy of Python where you can install packages without affecting anything else on your machine. You need one here because the voice assistant libraries are pinned to Python 3.11.

  • In VS Code, open the integrated terminal by pressing Ctrl+`.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir voice-assistant
cd voice-assistant
  • Create a virtual environment using Python 3.11 by running:
py -3.11 -m venv venv
  • Activate the virtual environment by running:
.\venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt. This confirms all packages you install will go into the virtual environment.

What does activating a virtual environment do?

When activated, python and pip commands point to the Python 3.11 copy inside your venv folder instead of the system Python. Any packages you install stay inside this folder and do not interfere with your global Python 3.14 installation.

  • Install the core packages for audio recording and transcription by running:
pip install openai-whisper sounddevice numpy scipy

What are these packages?

  • openai-whisper is OpenAI's speech recognition model that transcribes audio to text entirely on your machine.
  • sounddevice records audio from your microphone.
  • numpy handles the raw audio data as numerical arrays.
  • scipy provides signal processing utilities that sounddevice uses internally.

This installation may take a few minutes because openai-whisper downloads PyTorch and other dependencies.

  • Confirm the packages are installed by running:
pip list

You should see openai-whisper, sounddevice, numpy, and scipy in the list along with their dependencies.

Installation failing?

  • Make sure you see (venv) in your terminal prompt. If not, run .\venv\Scripts\activate again.
  • If you get a permissions error, close VS Code and reopen it as Administrator.
  • If sounddevice fails, make sure you have a working microphone connected to your machine.

Help me troubleshoot my pip install errors for openai-whisper or sounddevice on Windows 11 with Python 3.11.

  • Open the voice-assistant folder in VS Code by clicking File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop, select the voice-assistant folder, and click Select Folder.

VS Code reloads with your project folder as the workspace.

Write the audio recording and transcription code

Your script needs to do two things: record a few seconds of audio from your microphone, and pass that audio through Whisper to get a text transcription.

  • Create a new file by pressing Ctrl+N in VS Code.
  • Save it immediately with Ctrl+S and name it assistant.py.
  • Add the imports, configuration constants, and model loading by pasting this code into assistant.py:
import whisper
import sounddevice as sd
import numpy as np
import tempfile
import os
import wave


# --- Configuration ---
WHISPER_MODEL = "base"
SAMPLE_RATE = 16000
RECORD_SECONDS = 5


# --- Load Whisper Model ---
print("Loading Whisper model...")
whisper_model = whisper.load_model(WHISPER_MODEL)
print("Whisper model loaded.")

What does this code do?

  • WHISPER_MODEL = "base" selects the base Whisper model, which uses about 1 GB of RAM and gives a good balance of speed and accuracy.
  • SAMPLE_RATE = 16000 sets the audio to 16 kHz, which is what Whisper expects.
  • whisper.load_model() downloads the model on first run (about 74 MB) and loads it into memory so it is ready for fast transcription.
  • Below the model loading section, add the audio recording function:

# --- Audio Recording ---
def record_audio(duration=RECORD_SECONDS):
    """Record audio from the microphone."""
    print(f"Listening for {duration} seconds...")
    audio = sd.rec(
        int(duration * SAMPLE_RATE),
        samplerate=SAMPLE_RATE,
        channels=1,
        dtype="float32",
    )
    sd.wait()
    return audio.flatten()

What does record_audio do?

  • sd.rec() starts recording from your default microphone. It captures duration * SAMPLE_RATE samples (5 seconds at 16,000 samples per second = 80,000 data points).
  • sd.wait() blocks until the recording finishes, so the script does not continue before all audio is captured.
  • audio.flatten() converts the 2D array into a 1D array that Whisper can process.
  • Below the record_audio function, add the transcription function:

# --- Speech-to-Text ---
def transcribe(audio):
    """Transcribe audio using Whisper."""
    tmp_path = os.path.join(tempfile.gettempdir(), "voice_input.wav")
    audio_int16 = (audio * 32767).astype(np.int16)
    with wave.open(tmp_path, "w") as wf:
        wf.setnchannels(1)
        wf.setsampwidth(2)
        wf.setframerate(SAMPLE_RATE)
        wf.writeframes(audio_int16.tobytes())
    result = whisper_model.transcribe(tmp_path)
    os.remove(tmp_path)
    return result["text"].strip()

What does transcribe do?

Whisper expects an audio file on disk, not raw array data. This function writes the recorded audio to a temporary WAV file, runs Whisper's transcription on it, then deletes the temporary file.

  • (audio * 32767).astype(np.int16) converts the float32 audio data to 16-bit integers, which is the standard WAV format.
  • whisper_model.transcribe(tmp_path) runs the full speech recognition pipeline and returns a dictionary containing the transcribed text.
  • At the bottom of the file, add the main block that ties recording and transcription together:

if __name__ == "__main__":
    audio = record_audio()
    print("Transcribing...")
    text = transcribe(audio)
    print(f"You said: {text}")

What does the main block do?

The if __name__ == "__main__": guard means this code only runs when you execute assistant.py directly. It records 5 seconds of audio, transcribes it, and prints the result.

  • Save assistant.py with Ctrl+S.

✔️ Awesome, I've got everything!

Great. Double check you have saved the file before moving on.

ⓧ I'd like to double check the full code

Here is the complete assistant.py file at this point:

import whisper
import sounddevice as sd
import numpy as np
import tempfile
import os
import wave


# --- Configuration ---
WHISPER_MODEL = "base"
SAMPLE_RATE = 16000
RECORD_SECONDS = 5


# --- Load Whisper Model ---
print("Loading Whisper model...")
whisper_model = whisper.load_model(WHISPER_MODEL)
print("Whisper model loaded.")


# --- Audio Recording ---
def record_audio(duration=RECORD_SECONDS):
    """Record audio from the microphone."""
    print(f"Listening for {duration} seconds...")
    audio = sd.rec(
        int(duration * SAMPLE_RATE),
        samplerate=SAMPLE_RATE,
        channels=1,
        dtype="float32",
    )
    sd.wait()
    return audio.flatten()


# --- Speech-to-Text ---
def transcribe(audio):
    """Transcribe audio using Whisper."""
    tmp_path = os.path.join(tempfile.gettempdir(), "voice_input.wav")
    audio_int16 = (audio * 32767).astype(np.int16)
    with wave.open(tmp_path, "w") as wf:
        wf.setnchannels(1)
        wf.setsampwidth(2)
        wf.setframerate(SAMPLE_RATE)
        wf.writeframes(audio_int16.tobytes())
    result = whisper_model.transcribe(tmp_path)
    os.remove(tmp_path)
    return result["text"].strip()


if __name__ == "__main__":
    audio = record_audio()
    print("Transcribing...")
    text = transcribe(audio)
    print(f"You said: {text}")
Test your speech-to-text pipeline

Time to hear your assistant listen for the first time. You will run the script, speak a sentence, and confirm Whisper transcribes it correctly.

  • Open the integrated terminal in VS Code with Ctrl+`.
  • Activate the virtual environment by running:
.\venv\Scripts\activate
  • Run your script by executing:
python assistant.py

What should I see?

The first time you run this, Whisper will download the base model (about 74 MB). You will see "Loading Whisper model..." while it downloads and loads.

Once loaded, you will see "Listening for 5 seconds..." printed. Speak a clear sentence into your microphone during those 5 seconds. After it finishes, you will see "Transcribing..." followed by "You said:" and the text version of what you spoke.

  • When you see "Listening for 5 seconds...", speak a clear sentence into your microphone.
  • Confirm you see your spoken words printed as text after "You said:".

Not seeing accurate transcription?

  • If you see "No speech detected" or random words like "you" or "thank you", make sure your microphone is not muted. Check your Windows sound settings to confirm the correct input device is selected.
  • If transcription is very inaccurate, try switching from the base model to small by changing WHISPER_MODEL = "base" to WHISPER_MODEL = "small" in your configuration section. The small model uses more RAM but is significantly more accurate.
  • If you get a "No audio device" error, make sure a microphone is plugged in and recognized by Windows before running the script.

Help me troubleshoot why Whisper is not transcribing my speech correctly on Windows 11.

Your assistant can hear you. Next up, you will connect it to a local LLM so it can think about what you said and generate an intelligent response.

Connect to Your Local LLM via Ollama

Your assistant can hear you and transcribe your words into text. Now it needs a brain to actually understand what you said and respond intelligently.

Ollama exposes a local HTTP API that Python can talk to. In this step, you'll send your transcribed text to the llama3.2:3b model running on your machine and get back both conversational responses and language translations.

In this step, get ready to:
  • Install the requests library and write a function that communicates with Ollama's API.
  • Add chat and translate modes that shape how the LLM responds.
  • Test the full speech-to-LLM pipeline by speaking and seeing a text response.
Install requests and write the Ollama communication function

Your script needs to send HTTP requests to Ollama's local API. The requests library is the standard way to do this in Python.

  • Make sure your virtual environment is still active (you should see (venv) in your terminal prompt). Install the requests library by running:
pip install requests

Seeing a permissions error?

Make sure you activated your virtual environment first. Run .\venv\Scripts\activate and try the install again.

Help me fix my pip install error in my voice-assistant virtual environment.

Now you need to add three things to assistant.py: a new import, two configuration variables for Ollama, and the function that talks to the API.

  • In assistant.py, find the line import os and add this import on the next line:
import requests
  • In the Configuration section, find RECORD_SECONDS = 5 and add these two lines directly below it:
OLLAMA_MODEL = "llama3.2:3b"
OLLAMA_URL = "http://localhost:11434/api/generate"
  • Below the transcribe function (after its return statement) and above if __name__, add this function:
# --- LLM Communication ---
def ask_ollama(prompt):
    """Send a prompt to the local Ollama LLM and return the response."""
    response = requests.post(
        OLLAMA_URL,
        json={
            "model": OLLAMA_MODEL,
            "prompt": prompt,
            "stream": False,
        },
    )
    return response.json()["response"]

What does this function do?

  • requests.post() sends an HTTP POST request to Ollama's local API running on your machine.
  • The JSON body tells Ollama which model to use (llama3.2:3b) and what prompt to respond to.
  • stream: False tells Ollama to return the complete response at once instead of streaming it word by word.
  • The function extracts just the response text from the JSON that Ollama sends back.
  • Save assistant.py.

You won't see a visible change yet. The next substep adds the chat and translate functions that use ask_ollama.

Add chat and translate modes

A voice assistant that only chats is useful, but a voice assistant that can also translate languages is even better. You'll add two small functions that shape the prompt differently depending on the mode.

  • In assistant.py, add these two functions directly below ask_ollama:
def chat(text):
    """Send text as a conversation message."""
    return ask_ollama(text)


def translate(text, target_language):
    """Translate text to the target language."""
    prompt = (
        f"Translate the following English text to {target_language}. "
        f"Only output the translation, nothing else.\n\n{text}"
    )
    return ask_ollama(prompt)

What do these two functions do?

  • chat() passes your transcribed speech directly to the LLM as a question or conversation starter.
  • translate() wraps your speech in a specific instruction that tells the LLM to act as a translator. The prompt constrains the LLM to only output the translation with no extra commentary.

Both functions call ask_ollama under the hood. The only difference is how the prompt is shaped before it reaches the model.

  • Save assistant.py.

You won't see a change yet. The next substep updates the test code so you can run the full pipeline.

Test the full speech-to-LLM pipeline

Time to connect your transcription to the LLM. You'll update the test code at the bottom of the file to send your transcribed speech through the chat function.

  • In assistant.py, find the if __name__ == "__main__": block at the bottom. It currently looks like this:
if __name__ == "__main__":
    audio = record_audio()
    print("Transcribing...")
    text = transcribe(audio)
    print(f"You said: {text}")
  • Replace the entire block with this updated version that sends your transcription to the LLM:
if __name__ == "__main__":
    audio = record_audio()
    print("Transcribing...")
    text = transcribe(audio)
    print(f"You said: {text}")
    print("Thinking...")
    response = chat(text)
    print(f"Response: {response}")

What changed?

After transcribing your speech, the script now sends the text to your local LLM via the chat() function. The "Thinking..." message lets you know the LLM is processing, and then the response is printed to your terminal.

  • Save assistant.py.
  • Run the script by entering this command:
python assistant.py
  • When you see "Listening for 5 seconds...", speak a clear question like "What is the capital of France?" into your microphone.
  • Wait for the response. You should see your transcription printed, then "Thinking..." while the LLM processes, and finally a text response from the model.

Test translate mode too

To test translation, temporarily change response = chat(text) to response = translate(text, "French") and run the script again. Speak an English sentence and confirm you get a French translation back. Change it back to chat(text) when you're done testing.

Seeing a ConnectionError?

If you see ConnectionError: Max retries exceeded, Ollama is not running. Open a separate terminal and run ollama list to check. If Ollama is not responding, restart it by opening the Ollama application from your Start menu.

If the script hangs at "Thinking..." for a long time, the LLM is working but needs more time. The first request after starting Ollama takes longer because the model loads into memory. Wait up to 60 seconds for the first response.

Help me fix my Ollama connection error in my voice assistant script.

✔️ Awesome, I've got everything!

Great. Double check you've saved assistant.py and that both chat and translate modes return sensible text from the LLM.

ⓧ I'd like to double check the full code

Here is the complete assistant.py at this point:

import whisper
import sounddevice as sd
import numpy as np
import tempfile
import os
import requests
import wave


# --- Configuration ---
WHISPER_MODEL = "base"
OLLAMA_MODEL = "llama3.2:3b"
OLLAMA_URL = "http://localhost:11434/api/generate"
SAMPLE_RATE = 16000
RECORD_SECONDS = 5


# --- Load Whisper Model ---
print("Loading Whisper model...")
whisper_model = whisper.load_model(WHISPER_MODEL)
print("Whisper model loaded.")


# --- Audio Recording ---
def record_audio(duration=RECORD_SECONDS):
    """Record audio from the microphone."""
    print(f"Listening for {duration} seconds...")
    audio = sd.rec(
        int(duration * SAMPLE_RATE),
        samplerate=SAMPLE_RATE,
        channels=1,
        dtype="float32",
    )
    sd.wait()
    return audio.flatten()


# --- Speech-to-Text ---
def transcribe(audio):
    """Transcribe audio using Whisper."""
    tmp_path = os.path.join(tempfile.gettempdir(), "voice_input.wav")
    audio_int16 = (audio * 32767).astype(np.int16)
    with wave.open(tmp_path, "w") as wf:
        wf.setnchannels(1)
        wf.setsampwidth(2)
        wf.setframerate(SAMPLE_RATE)
        wf.writeframes(audio_int16.tobytes())
    result = whisper_model.transcribe(tmp_path)
    os.remove(tmp_path)
    return result["text"].strip()


# --- LLM Communication ---
def ask_ollama(prompt):
    """Send a prompt to the local Ollama LLM and return the response."""
    response = requests.post(
        OLLAMA_URL,
        json={
            "model": OLLAMA_MODEL,
            "prompt": prompt,
            "stream": False,
        },
    )
    return response.json()["response"]


def chat(text):
    """Send text as a conversation message."""
    return ask_ollama(text)


def translate(text, target_language):
    """Translate text to the target language."""
    prompt = (
        f"Translate the following English text to {target_language}. "
        f"Only output the translation, nothing else.\n\n{text}"
    )
    return ask_ollama(prompt)


if __name__ == "__main__":
    audio = record_audio()
    print("Transcribing...")
    text = transcribe(audio)
    print(f"You said: {text}")
    print("Thinking...")
    response = chat(text)
    print(f"Response: {response}")

Your assistant now has a brain. It can listen, understand, and respond in text. Next up, you'll give it a voice by adding British text-to-speech output with Piper TTS.

Add British Voice Output with Piper TTS

Your assistant can now listen to your voice and generate intelligent responses using the local LLM. The missing piece is the ability to speak those responses back to you.

In this step, you will install Piper TTS, a fast neural text-to-speech engine that runs entirely on your machine. You will download a British English voice model and write a function that converts text into spoken audio with a British accent.

In this step, get ready to:
  • Install Piper TTS and download the British English voice model files.
  • Write a speak function that converts text to audio using Piper.
  • Test the voice output by hearing a hardcoded phrase spoken in a British accent.
Install Piper TTS and download the voice model

Piper TTS is a neural text-to-speech engine that generates natural-sounding speech from text. It runs locally on your CPU with no internet required after the initial download.

  • Install the Piper TTS package and the audio playback library by running this command in your terminal (with your virtual environment still activated):
pip install piper-tts simpleaudio

What do these packages do?

  • piper-tts installs the Piper TTS engine and a piper command-line tool that converts text into speech audio.
  • simpleaudio is a lightweight library for playing WAV audio files from Python.

Seeing a build error for simpleaudio?

If simpleaudio fails to install, you may need Microsoft Visual C++ Build Tools. Download them from the Visual Studio website and install the "Desktop development with C++" workload. Then retry the pip install command.

Help me fix this build error

Now you need to download the British English voice model. The en_GB-vctk-medium model is trained on 109 British English speakers from the VCTK corpus. It is 77 MB and runs in real-time on CPU.

  • Create a models folder inside your voice-assistant directory by running:
mkdir models
  • Download the voice model file (77 MB) by running:
curl.exe -L -o models/en_GB-vctk-medium.onnx https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_GB/vctk/medium/en_GB-vctk-medium.onnx
  • Download the model configuration file by running:
curl.exe -L -o models/en_GB-vctk-medium.onnx.json https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_GB/vctk/medium/en_GB-vctk-medium.onnx.json

Why curl.exe instead of curl?

In Windows PowerShell 5.1, typing curl runs a different command (Invoke-WebRequest) instead of the real curl tool. Adding .exe forces PowerShell to use the actual curl program built into Windows 11.

The -L flag tells curl to follow redirects (Hugging Face uses them). The -o flag specifies where to save the file.

  • Confirm both files downloaded successfully by running:
ls models

You should see en_GB-vctk-medium.onnx and en_GB-vctk-medium.onnx.json listed in the output.

Download failed or file is 0 bytes?

Check your internet connection. The ONNX model is 77 MB and may take a minute on slower connections. If the download was interrupted, delete the partial file and rerun the curl command.

Help me troubleshoot this download

Write the speak function

With the voice model downloaded, you can now write the function that pipes text through Piper and plays the resulting audio. This function calls the piper command-line tool as a subprocess, feeds it your text, and plays back the generated WAV file.

  • In assistant.py, find the line that says import wave at the top of your file.
  • Add these two new imports directly below it:
import subprocess
import simpleaudio as sa

What do these imports do?

  • subprocess lets Python launch the piper command-line tool and feed it text.
  • simpleaudio (imported as sa) plays the WAV audio files that Piper generates.
  • Now find the line RECORD_SECONDS = 5 in the configuration section.
  • Add these three new constants directly below it:
PIPER_MODEL_PATH = os.path.join("models", "en_GB-vctk-medium.onnx")
PIPER_CONFIG_PATH = os.path.join("models", "en_GB-vctk-medium.onnx.json")
SPEAKER_ID = 0

What are these constants?

  • PIPER_MODEL_PATH points to the neural network model you just downloaded.
  • PIPER_CONFIG_PATH points to the model's configuration (sample rate, speaker map, phonemizer settings).
  • SPEAKER_ID = 0 selects speaker "p239" from the 109 available British English voices. You can experiment with IDs 0 through 108 for different voices.
  • Scroll to the bottom of assistant.py and find the translate function.
  • Below the translate function, add the following text-to-speech section:
# --- Text-to-Speech ---
def speak(text):
    """Speak text using Piper TTS with a British voice."""
    output_path = os.path.join(tempfile.gettempdir(), "response.wav")
    process = subprocess.Popen(
        [
            "piper",
            "--model", PIPER_MODEL_PATH,
            "--config", PIPER_CONFIG_PATH,
            "--speaker", str(SPEAKER_ID),
            "--output_file", output_path,
        ],
        stdin=subprocess.PIPE,
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL,
    )
    process.communicate(input=text.encode("utf-8"))
    if os.path.exists(output_path):
        wave_obj = sa.WaveObject.from_wave_file(output_path)
        play_obj = wave_obj.play()
        play_obj.wait_done()
        os.remove(output_path)

What does this function do?

The speak function works in three stages:

  • It launches the piper CLI as a subprocess, passing in the model path, config path, speaker ID, and an output file location in your system's temp folder.
  • It sends the text to Piper via stdin. Piper processes the text through its neural network and writes a WAV file.
  • It plays the generated WAV file using simpleaudio, waits for playback to finish, then cleans up the temp file.
Test the voice output

Before wiring everything together in the next step, you should test that Piper can successfully speak a hardcoded phrase.

  • In assistant.py, find the existing if __name__ == "__main__": block at the bottom of the file.
  • Find this code:
if __name__ == "__main__":
    audio = record_audio()
    print("Transcribing...")
    text = transcribe(audio)
    print(f"You said: {text}")
    print("Thinking...")
    response = chat(text)
    print(f"Response: {response}")
  • Replace the entire if __name__ == "__main__": block with this simpler test:
if __name__ == "__main__":
    speak("Hello, I am your British voice assistant")
  • Save assistant.py.
  • Run the script by entering this command:
python assistant.py

You should hear "Hello, I am your British voice assistant" spoken through your speakers in a British accent.

Not hearing any audio?

  • Make sure your speakers or headphones are connected and the volume is turned up.
  • Check that the models folder is in the same directory where you are running the script. Run ls models to confirm.
  • If you see a "piper not found" error, close and reopen your terminal, reactivate your virtual environment with .\venv\Scripts\activate, and try again.

Help me debug this audio issue

✔️ Awesome, I've got everything!

Great. Double check you have saved assistant.py with all your changes.

ⓧ I'd like to double check the full code

Here is the complete assistant.py at the end of this step:

import whisper
import sounddevice as sd
import numpy as np
import tempfile
import os
import requests
import wave
import subprocess
import simpleaudio as sa


# --- Configuration ---
WHISPER_MODEL = "base"
OLLAMA_MODEL = "llama3.2:3b"
OLLAMA_URL = "http://localhost:11434/api/generate"
SAMPLE_RATE = 16000
RECORD_SECONDS = 5
PIPER_MODEL_PATH = os.path.join("models", "en_GB-vctk-medium.onnx")
PIPER_CONFIG_PATH = os.path.join("models", "en_GB-vctk-medium.onnx.json")
SPEAKER_ID = 0


# --- Load Whisper Model ---
print("Loading Whisper model...")
whisper_model = whisper.load_model(WHISPER_MODEL)
print("Whisper model loaded.")


# --- Audio Recording ---
def record_audio(duration=RECORD_SECONDS):
    """Record audio from the microphone."""
    print(f"Listening for {duration} seconds...")
    audio = sd.rec(
        int(duration * SAMPLE_RATE),
        samplerate=SAMPLE_RATE,
        channels=1,
        dtype="float32",
    )
    sd.wait()
    return audio.flatten()


# --- Speech-to-Text ---
def transcribe(audio):
    """Transcribe audio using Whisper."""
    tmp_path = os.path.join(tempfile.gettempdir(), "voice_input.wav")
    audio_int16 = (audio * 32767).astype(np.int16)
    with wave.open(tmp_path, "w") as wf:
        wf.setnchannels(1)
        wf.setsampwidth(2)
        wf.setframerate(SAMPLE_RATE)
        wf.writeframes(audio_int16.tobytes())
    result = whisper_model.transcribe(tmp_path)
    os.remove(tmp_path)
    return result["text"].strip()


# --- LLM Communication ---
def ask_ollama(prompt):
    """Send a prompt to the local Ollama LLM and return the response."""
    response = requests.post(
        OLLAMA_URL,
        json={
            "model": OLLAMA_MODEL,
            "prompt": prompt,
            "stream": False,
        },
    )
    return response.json()["response"]


def chat(text):
    """Send text as a conversation message."""
    return ask_ollama(text)


def translate(text, target_language):
    """Translate text to the target language."""
    prompt = (
        f"Translate the following English text to {target_language}. "
        f"Only output the translation, nothing else.\n\n{text}"
    )
    return ask_ollama(prompt)


# --- Text-to-Speech ---
def speak(text):
    """Speak text using Piper TTS with a British voice."""
    output_path = os.path.join(tempfile.gettempdir(), "response.wav")
    process = subprocess.Popen(
        [
            "piper",
            "--model", PIPER_MODEL_PATH,
            "--config", PIPER_CONFIG_PATH,
            "--speaker", str(SPEAKER_ID),
            "--output_file", output_path,
        ],
        stdin=subprocess.PIPE,
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL,
    )
    process.communicate(input=text.encode("utf-8"))
    if os.path.exists(output_path):
        wave_obj = sa.WaveObject.from_wave_file(output_path)
        play_obj = wave_obj.play()
        play_obj.wait_done()
        os.remove(output_path)


if __name__ == "__main__":
    speak("Hello, I am your British voice assistant")

Your assistant now has a British voice. Next up, you will wire all three components together into a seamless loop: listen, think, and speak.

Wire Everything Together

You now have three independent AI systems ready to go: Whisper for speech-to-text, Ollama for generating responses, and Piper TTS for speaking them back in a British accent.

The missing piece is a conductor that orchestrates the flow from your microphone through all three systems and back to your speakers. In this step, you will write a main() function that ties the full pipeline together with a mode selection menu and a continuous listening loop.

In this step, get ready to:
  • Write the main loop that connects recording, transcription, LLM, and speech output.
  • Add a mode menu letting you choose between chat and translate.
  • Test the full end-to-end pipeline in both modes.
Write the main function with mode selection and the listening loop

The main() function will present a menu (chat or translate), then enter a continuous loop that records your voice, transcribes it, sends it to the LLM, and speaks the result.

  • In assistant.py, scroll to the bottom of the file.
  • Find this test code:
if __name__ == "__main__":
    speak("Hello, I am your British voice assistant")
  • Delete those two lines and replace them with the following main() function and entry point:
# --- Main Loop ---
def main():
    print("\n=== Local AI Voice Assistant (British Voice) ===")
    print("Choose a mode:")
    print("  1. Chat - Ask questions, get answers")
    print("  2. Translate - Speak English, hear it translated")
    print("  Press Ctrl+C to exit\n")

    mode = input("Enter mode (1 or 2): ").strip()

    target_language = "French"
    if mode == "2":
        target_language = input("Target language (e.g. French, Spanish, German): ").strip()

    print(f"\nMode: {'Chat' if mode == '1' else f'Translate to {target_language}'}")
    print("Press Ctrl+C to exit.\n")

    try:
        while True:
            # Record from microphone
            audio = record_audio()
            print("Transcribing...")
            text = transcribe(audio)

            # Skip if no speech was detected
            if not text or text.lower() in ["", "you", "thank you"]:
                print("No speech detected. Try again.\n")
                continue

            print(f"You said: {text}")

            # Send to LLM based on selected mode
            print("Thinking...")
            if mode == "1":
                response = chat(text)
            else:
                response = translate(text, target_language)

            print(f"Response: {response}")

            # Speak the response aloud
            print("Speaking...")
            speak(response)
            print()

    except KeyboardInterrupt:
        print("\n\nGoodbye!")


if __name__ == "__main__":
    main()

What does this code do?

The main() function is the conductor that orchestrates the entire pipeline. Here is how it flows:

  • The menu lets you pick chat mode (general conversation) or translate mode (language translation). If you choose translate, it asks which language to target.
  • The while True loop continuously records, transcribes, processes through the LLM, and speaks the result. Status messages like "Transcribing..." and "Thinking..." keep you informed at each stage.
  • The empty-speech guard filters out silence or Whisper hallucinations (it sometimes outputs "you" or "thank you" when no real speech is detected).
  • The KeyboardInterrupt handler lets you press Ctrl+C at any time to exit cleanly with a "Goodbye!" message.
  • Save assistant.py.

Seeing a syntax error?

  • Make sure you deleted both lines of the old test code (if __name__ and the speak() call). Having two if __name__ blocks causes unexpected behavior.
  • Check that the indentation under def main(): uses 4 spaces consistently. Mixing tabs and spaces causes IndentationError in Python.

Help me fix a syntax error in my assistant.py main function

Test the full pipeline in both modes

Time to hear your assistant in action. You will test both chat mode and translate mode to confirm the full pipeline works end-to-end.

  • Make sure Ollama is running in the background. If you closed it, open a new terminal and run:
ollama serve
  • In your project terminal (with the virtual environment activated), run the assistant:
python assistant.py
  • Wait for "Whisper model loaded." to appear. This may take a few seconds on the first run.
  • Enter 1 to select Chat mode.
  • When you see "Listening for 5 seconds...", speak a clear question like "What is the capital of France?" into your microphone.
  • Watch the status messages progress: Transcribing, your transcribed text, Thinking, the LLM response, then Speaking.
  • You should hear the response spoken back in a British accent through your speakers.
  • Press Ctrl+C to exit, then run the script again.
  • This time, enter 2 for Translate mode.
  • Enter a target language like French.
  • Speak an English sentence and confirm the translation is spoken back in a British-accented voice.

What should I see?

Your terminal output should look something like this for chat mode:

Loading Whisper model...
Whisper model loaded.

=== Local AI Voice Assistant (British Voice) ===
Choose a mode:
  1. Chat - Ask questions, get answers
  2. Translate - Speak English, hear it translated
  Press Ctrl+C to exit

Enter mode (1 or 2): 1

Mode: Chat
Press Ctrl+C to exit.

Listening for 5 seconds...
Transcribing...
You said: What is the capital of France?
Thinking...
Response: The capital of France is Paris.
Speaking...

After "Speaking..." you should hear the response through your speakers in a British accent.

Not hearing any audio output?

  • Make sure your system volume is turned up and the correct output device is selected in Windows sound settings.
  • If you see "No speech detected. Try again." repeatedly, speak louder or move closer to your microphone. Background noise can also cause Whisper to output false positives.
  • If the script hangs on "Thinking...", Ollama may not be running. Open a separate terminal and run ollama serve.

Help me troubleshoot my voice assistant pipeline

✔️ Awesome, I've got everything!

Great. Double check you have saved assistant.py and both modes are working.

ⓧ I'd like to double check the full code

Here is the complete assistant.py file. Compare your code against this:

import whisper
import sounddevice as sd
import numpy as np
import tempfile
import os
import requests
import wave
import subprocess
import simpleaudio as sa


# --- Configuration ---
WHISPER_MODEL = "base"
OLLAMA_MODEL = "llama3.2:3b"
OLLAMA_URL = "http://localhost:11434/api/generate"
SAMPLE_RATE = 16000
RECORD_SECONDS = 5
PIPER_MODEL_PATH = os.path.join("models", "en_GB-vctk-medium.onnx")
PIPER_CONFIG_PATH = os.path.join("models", "en_GB-vctk-medium.onnx.json")
SPEAKER_ID = 0


# --- Load Whisper Model ---
print("Loading Whisper model...")
whisper_model = whisper.load_model(WHISPER_MODEL)
print("Whisper model loaded.")


# --- Audio Recording ---
def record_audio(duration=RECORD_SECONDS):
    """Record audio from the microphone."""
    print(f"Listening for {duration} seconds...")
    audio = sd.rec(
        int(duration * SAMPLE_RATE),
        samplerate=SAMPLE_RATE,
        channels=1,
        dtype="float32",
    )
    sd.wait()
    return audio.flatten()


# --- Speech-to-Text ---
def transcribe(audio):
    """Transcribe audio using Whisper."""
    tmp_path = os.path.join(tempfile.gettempdir(), "voice_input.wav")
    audio_int16 = (audio * 32767).astype(np.int16)
    with wave.open(tmp_path, "w") as wf:
        wf.setnchannels(1)
        wf.setsampwidth(2)
        wf.setframerate(SAMPLE_RATE)
        wf.writeframes(audio_int16.tobytes())
    result = whisper_model.transcribe(tmp_path)
    os.remove(tmp_path)
    return result["text"].strip()


# --- LLM Communication ---
def ask_ollama(prompt):
    """Send a prompt to the local Ollama LLM and return the response."""
    response = requests.post(
        OLLAMA_URL,
        json={
            "model": OLLAMA_MODEL,
            "prompt": prompt,
            "stream": False,
        },
    )
    return response.json()["response"]


def chat(text):
    """Send text as a conversation message."""
    return ask_ollama(text)


def translate(text, target_language):
    """Translate text to the target language."""
    prompt = (
        f"Translate the following English text to {target_language}. "
        f"Only output the translation, nothing else.\n\n{text}"
    )
    return ask_ollama(prompt)


# --- Text-to-Speech ---
def speak(text):
    """Speak text using Piper TTS with a British voice."""
    output_path = os.path.join(tempfile.gettempdir(), "response.wav")
    process = subprocess.Popen(
        [
            "piper",
            "--model", PIPER_MODEL_PATH,
            "--config", PIPER_CONFIG_PATH,
            "--speaker", str(SPEAKER_ID),
            "--output_file", output_path,
        ],
        stdin=subprocess.PIPE,
        stdout=subprocess.DEVNULL,
        stderr=subprocess.DEVNULL,
    )
    process.communicate(input=text.encode("utf-8"))
    if os.path.exists(output_path):
        wave_obj = sa.WaveObject.from_wave_file(output_path)
        play_obj = wave_obj.play()
        play_obj.wait_done()
        os.remove(output_path)


# --- Main Loop ---
def main():
    print("\n=== Local AI Voice Assistant (British Voice) ===")
    print("Choose a mode:")
    print("  1. Chat - Ask questions, get answers")
    print("  2. Translate - Speak English, hear it translated")
    print("  Press Ctrl+C to exit\n")

    mode = input("Enter mode (1 or 2): ").strip()

    target_language = "French"
    if mode == "2":
        target_language = input("Target language (e.g. French, Spanish, German): ").strip()

    print(f"\nMode: {'Chat' if mode == '1' else f'Translate to {target_language}'}")
    print("Press Ctrl+C to exit.\n")

    try:
        while True:
            audio = record_audio()
            print("Transcribing...")
            text = transcribe(audio)

            if not text or text.lower() in ["", "you", "thank you"]:
                print("No speech detected. Try again.\n")
                continue

            print(f"You said: {text}")

            print("Thinking...")
            if mode == "1":
                response = chat(text)
            else:
                response = translate(text, target_language)

            print(f"Response: {response}")

            print("Speaking...")
            speak(response)
            print()

    except KeyboardInterrupt:
        print("\n\nGoodbye!")


if __name__ == "__main__":
    main()

Your voice assistant is fully operational. You have a complete local AI pipeline that listens, thinks, and speaks. Next up, you can push it further with a Secret Mission that adds continuous live interpretation mode.

Secret mission

Add Live Interpreter Mode

Your assistant already translates one sentence at a time. But a real interpreter doesn't wait for you to press a button between every sentence. Can you add a continuous listening mode that automatically detects when you're speaking, translates each sentence, and speaks the result back without any manual intervention?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs or charges. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Ollama LLM model (llama3.2:3b): approximately 2 GB stored in your home directory under .ollama.
  • Piper voice model files: approximately 77 MB in your voice-assistant/models folder.
  • Python virtual environment: a few hundred MB in your voice-assistant/venv folder.
  • Ollama application: installed on your system.
  • ffmpeg: installed via winget.
  • Python 3.11.9: installed alongside your existing Python.

Keep everything running

No action needed. Choose this if you are still actively building or experimenting with your voice assistant.

  • All resources are local and free. No charges will accumulate.
  • Ollama continues running in the background via your system tray. It uses minimal resources when idle.
  • Your virtual environment stays on disk. Activate it with .\venv\Scripts\activate whenever you want to use the assistant.

Pause - I'll come back to this later

Shut down running processes to free up memory while keeping all your files intact for next time.

  • Right-click the Ollama icon in your system tray and select Quit Ollama. This stops the background service and frees RAM.
  • If your virtual environment is still active in your terminal, deactivate it by running:
deactivate
  • Your project folder, models, and virtual environment all stay on disk. When you are ready to return, reopen Ollama and reactivate your virtual environment.

Delete - I don't want to use this again

Remove all project resources and installed tools completely from your machine.

  • Remove the LLM model from Ollama by running:
ollama rm llama3.2:3b
  • Delete the entire project folder (includes your virtual environment, voice models, and source code) by running:
cd ~/Desktop
Remove-Item -Recurse -Force voice-assistant
  • Uninstall Ollama from your system. Open Settings, then Apps, then Installed apps. Find Ollama in the list, click the three-dot menu, and select Uninstall.
  • After uninstalling Ollama, delete the leftover model data by running:
Remove-Item -Recurse -Force $env:USERPROFILE\.ollama
  • Uninstall ffmpeg by running:
winget uninstall Gyan.FFmpeg
  • Uninstall Python 3.11.9. Open Settings, then Apps, then Installed apps. Find Python 3.11.9 in the list, click the three-dot menu, and select Uninstall. This will not affect your existing Python 3.14 installation.

Nice Work!

Nice Work!

Nice work! You've just built a fully local AI voice assistant that listens to your speech, processes it through a local LLM, and speaks back in a British accent.

You've learned how to:

  • Built a complete voice assistant pipeline connecting your microphone to Whisper for transcription, Ollama for AI responses, and Piper TTS for British-accented speech output.
  • Connected to a local LLM via Ollama's HTTP API to generate both conversational responses and language translations without any internet connection.
  • Wired together three separate AI systems (speech-to-text, language model, text-to-speech) into one seamless Python application that runs entirely on your machine.
  • Secret Mission: Added a live interpreter mode with continuous listening and automatic voice activity detection.

Ready to quiz yourself?