Build a Real-Time Voice Chat Over UDP

Stream live mic audio between terminals using UDP sockets and RTP headers.

Introduction

30 Second Summary

Every time you make a phone call or join a video meeting, your voice is being chopped into tiny packets and fired across the internet in real time. The protocol powering this is invisible to most people, but it is the backbone of every modern voice app.

In this project, you will build a real-time voice chat application from scratch using Python. You will capture live microphone audio, send it over the network using UDP sockets, and construct proper RTP protocol headers that track every single packet.

What You'll Build

You speak into your microphone in one terminal, and your voice plays back in another terminal in real time, with a live dashboard showing exactly how many packets made it across and how many were lost.

By the end of this project, you'll have:

  • A working voice chat between two terminal windows that streams live audio over your local network.
  • A hand-built RTP packet structure (the same protocol Zoom and Discord use) that numbers every audio packet for delivery tracking.
  • A live stats dashboard showing packets received, packets lost, and loss percentage updating in real time.
  • Secret Mission: Simulate real network chaos and build a jitter buffer to smooth out choppy playback.

Are there any prerequisites?

You need Python installed on your machine and basic Python knowledge (functions, loops, and variables). No networking experience is required.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters to you.

Set Up Your Project

Your voice chat needs two things before it can stream any audio: a library to capture sound from your microphone, and a tidy folder to hold your scripts.

In this step, you'll create your project workspace and install PyAudio, the library that turns your mic into a data source Python can read.

In this step, get ready to:
  • Create a project folder and open it in VS Code.
  • Install PyAudio and verify the version.
  • Confirm your microphone and speakers are accessible.
Create your project folder
  • Press the Windows key (Windows) or Cmd+Space (macOS) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir voice-chat
cd voice-chat
  • Open this folder in VS Code by running:
code .
  • VS Code opens with the voice-chat folder loaded in the Explorer sidebar.
  • Open the VS Code integrated terminal by pressing Ctrl+` (backtick key, usually above Tab).

Why use VS Code's terminal from here?

The integrated terminal starts inside your voice-chat folder automatically. Every command you run from now on will target the right place without extra navigation.

Install PyAudio

PyAudio is a Python binding for PortAudio, which handles the low-level communication with your sound hardware.

  • Install PyAudio by running this command in the VS Code terminal:
python -m pip install pyaudio

What does this command do?

The python -m pip pattern runs pip through the Python interpreter you have on your PATH. This avoids confusion when multiple Python versions are installed.

On Windows, the PyAudio wheel comes with PortAudio pre-compiled, so no extra C libraries are needed.

  • Verify PyAudio installed correctly by running:
python -c "import pyaudio; print(pyaudio.__version__)"

✔️ I see 0.2.14

PyAudio is ready. You can move on to the next substep.

ⓧ I see an error

If you see ModuleNotFoundError, the install did not complete. Try these fixes:

  • Make sure you are using the same python that pip installed to. Run python --version and confirm it shows 3.11.
  • If you have multiple Python versions, try python3 -m pip install pyaudio instead.
  • Re-run the version check after a successful install.
Verify audio devices are accessible

Before writing any streaming code, you need to confirm Python can see your microphone and speakers.

  • Create a new file called test_audio.py by clicking the New File icon in the VS Code Explorer sidebar.
  • Name the file test_audio.py and press Enter.
  • Paste the following code into test_audio.py:
import pyaudio

# Initialize PyAudio to access audio hardware
p = pyaudio.PyAudio()

print("Available audio devices:\n")

# List every audio device with its input/output capability
for i in range(p.get_device_count()):
    info = p.get_device_info_by_index(i)
    direction = ""
    if info["maxInputChannels"] > 0:
        direction += " [INPUT]"
    if info["maxOutputChannels"] > 0:
        direction += " [OUTPUT]"
    print(f"  Device {i}: {info['name']}{direction}")

# Show the system defaults
print(f"\nDefault input (mic): {p.get_default_input_device_info()['name']}")
print(f"Default output (speakers): {p.get_default_output_device_info()['name']}")

p.terminate()

What does this code do?

  • get_device_count() returns the total number of audio devices your OS knows about (both physical and virtual).
  • get_device_info_by_index() returns a dictionary with the device name, max input channels, and max output channels.
  • Devices tagged [INPUT] can capture audio (microphones). Devices tagged [OUTPUT] can play audio (speakers or headphones).
  • get_default_input_device_info() and get_default_output_device_info() show which devices your system will use by default.
  • Save test_audio.py by pressing Ctrl+S (Windows) or Cmd+S (macOS).

Before you run this, what do you expect to see? Think about how many audio devices your machine might have.

  • Run the script by entering this in your VS Code terminal:
python test_audio.py

You should see a list of devices, with at least one [INPUT] device (your microphone) and one [OUTPUT] device (your speakers or headphones).

No input devices showing up?

  • Check that your microphone is plugged in or that your built-in mic is enabled in Windows Sound Settings.
  • Open Settings then System then Sound and verify your input device appears under Input.
  • If you recently connected a USB microphone, try re-running the script. Some devices take a moment to register.

Help me troubleshoot why PyAudio can't find my audio devices on Windows.

✔️ Awesome, I've got everything!

Great. Double check you saved test_audio.py and that both an input and output device appeared in the list.

ⓧ I'd like to double check the full code

Here is the complete test_audio.py file:

import pyaudio

# Initialize PyAudio to access audio hardware
p = pyaudio.PyAudio()

print("Available audio devices:\n")

# List every audio device with its input/output capability
for i in range(p.get_device_count()):
    info = p.get_device_info_by_index(i)
    direction = ""
    if info["maxInputChannels"] > 0:
        direction += " [INPUT]"
    if info["maxOutputChannels"] > 0:
        direction += " [OUTPUT]"
    print(f"  Device {i}: {info['name']}{direction}")

# Show the system defaults
print(f"\nDefault input (mic): {p.get_default_input_device_info()['name']}")
print(f"Default output (speakers): {p.get_default_output_device_info()['name']}")

p.terminate()

Your workspace is ready and Python can talk to your audio hardware. Next up, you'll capture your voice from the mic and play it back instantly through your speakers.

Build an Audio Loopback

Your environment is ready and your audio devices are confirmed working. Before sending audio over a network, you need to prove that your audio pipeline can actually capture and play sound on its own.

This step builds the simplest possible loopback: your mic feeds directly into your speakers. If this works, you know any future problems live in the network code, not the audio code.

In this step, get ready to:
  • Create audio_loopback.py with PyAudio capturing and playing back audio.
  • Run the loopback and hear your own voice in real time.
Create the audio loopback script

An audio loopback captures sound from your microphone and immediately routes it to your speakers. PyAudio handles both directions in a single stream, reading raw audio chunks and writing them straight back out.

  • Create a new file called audio_loopback.py in your voice-chat folder by clicking the new file icon in the VS Code Explorer sidebar.
  • Set up the audio pipeline configuration by pasting this code:
import pyaudio

# Audio configuration constants
CHUNK = 1024
FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100

# Initialize PyAudio and open a stream for both input and output
p = pyaudio.PyAudio()

stream = p.open(format=FORMAT,
                channels=CHANNELS,
                rate=RATE,
                input=True,
                output=True,
                frames_per_buffer=CHUNK)

What do these settings mean?

  • CHUNK = 1024 sets how many audio frames are processed at once. Smaller chunks mean lower latency but more CPU work.
  • RATE = 44100 is the sample rate in Hz. This is CD-quality audio, capturing 44,100 samples per second.
  • input=True and output=True together make one stream that reads from the mic AND writes to the speakers.
  • paInt16 means each sample is a 16-bit integer. This is standard for voice audio.
  • Add the main loopback loop and cleanup code below your existing code in audio_loopback.py:
print("Audio loopback active - speak into your mic!")
print("Press Ctrl+C to stop.\n")

# Continuously read from mic and write to speakers
try:
    while True:
        data = stream.read(CHUNK)
        stream.write(data)
except KeyboardInterrupt:
    print("\nStopping loopback...")

# Clean up audio resources
stream.stop_stream()
stream.close()
p.terminate()

What does this loop do?

  • stream.read(CHUNK) grabs 1024 frames of raw audio from your microphone.
  • stream.write(data) immediately pushes those same bytes to your speakers.
  • The KeyboardInterrupt handler catches Ctrl+C so the script shuts down cleanly instead of crashing.
  • The cleanup lines release the audio stream and terminate PyAudio so your mic is freed for other apps.
  • Save audio_loopback.py by pressing Ctrl+S.

✔️ Awesome, I've got everything!

Great. Make sure you have saved your file before running it in the next substep.

ⓧ I'd like to double check the full code

Here is the complete audio_loopback.py file. Make sure yours matches this exactly.

import pyaudio

CHUNK = 1024
FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100

p = pyaudio.PyAudio()

stream = p.open(format=FORMAT,
                channels=CHANNELS,
                rate=RATE,
                input=True,
                output=True,
                frames_per_buffer=CHUNK)

print("Audio loopback active - speak into your mic!")
print("Press Ctrl+C to stop.\n")

try:
    while True:
        data = stream.read(CHUNK)
        stream.write(data)
except KeyboardInterrupt:
    print("\nStopping loopback...")

stream.stop_stream()
stream.close()
p.terminate()
Run and test the loopback

Use headphones for this test

If you use speakers instead of headphones, the speakers feed sound back into the mic, creating a loud feedback loop. Plug in headphones or earbuds before running the script.

Before you run this, what do you think you will hear when you speak into your mic?

  • Run the loopback script in your VS Code terminal:
python audio_loopback.py

You should see the message Audio loopback active - speak into your mic! printed in your terminal.

  • Speak into your microphone.
  • Listen for your own voice playing back through your headphones in real time.
  • Press Ctrl+C to stop the script when you are done testing.

You should hear your own voice played back with barely any delay. The terminal prints Stopping loopback... when you press Ctrl+C.

Not hearing anything?

  • Check that your microphone is not muted in your system sound settings. On Windows, right-click the speaker icon in the taskbar and select Sound settings.
  • Make sure your headphones or speakers are set as the default output device.
  • Run your test_audio.py script again to confirm your devices still appear in the device list.

Still stuck? Help me debug why my audio loopback is not producing sound.

Your audio pipeline is proven. You can capture live audio from your mic and play it back without any issues. Next up, you will send that audio over the network using UDP sockets, turning a local loopback into a one-way voice stream between two separate programs.

Stream Audio Over UDP

Your audio loopback proves the mic-to-speaker pipeline works on a single machine. Now you need to send that audio across the network to a completely separate program.

In this step, you'll split the loopback into two scripts: a sender that captures your mic and ships raw audio bytes over the network, and a receiver that catches those bytes and plays them through your speakers. The transport protocol is UDP.

In this step, get ready to:
  • Create a sender script that captures mic audio and sends it over UDP.
  • Create a receiver script that listens for UDP packets and plays them as audio.
  • Test a one-way voice stream between two terminals.

Why UDP instead of TCP?

TCP guarantees delivery by resending lost packets. That sounds helpful, but re-sending a half-second-old audio chunk creates an awkward delay that compounds over time.

UDP just fires packets and moves on. If one gets lost, the listener hears a tiny glitch instead of an ever-growing lag. Every major VoIP system (Zoom, Discord, Teams) uses UDP for exactly this reason: a small glitch beats a growing delay.

Create the sender script

The sender's job is simple: capture microphone audio one chunk at a time and fire each chunk across the network as a UDP packet.

  • Create a new file called sender.py in your voice-chat folder.
  • Add the setup code that configures audio capture and creates a UDP socket:
import pyaudio
import socket

# Audio settings (must match receiver)
CHUNK = 1024
FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100

# Destination address for audio packets
DEST_IP = "127.0.0.1"
DEST_PORT = 5000

# Open mic for capture
p = pyaudio.PyAudio()
stream = p.open(format=FORMAT,
                channels=CHANNELS,
                rate=RATE,
                input=True,
                frames_per_buffer=CHUNK)

# Create a UDP socket for sending
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

What does this code do?

  • CHUNK = 1024 sets the number of audio frames read per cycle. This is the same value your loopback used.
  • DEST_IP = "127.0.0.1" points to your own machine (localhost). The receiver will listen here.
  • socket.SOCK_DGRAM creates a UDP socket (datagram mode) instead of a TCP socket.
  • Below the socket line, add the main sending loop and cleanup:
print(f"Sending audio to {DEST_IP}:{DEST_PORT}...")
print("Press Ctrl+C to stop.\n")

try:
    while True:
        # Capture audio and send raw bytes over UDP
        audio_data = stream.read(CHUNK, exception_on_overflow=False)
        sock.sendto(audio_data, (DEST_IP, DEST_PORT))
except KeyboardInterrupt:
    print("\nStopped.")

# Clean up
stream.stop_stream()
stream.close()
p.terminate()
sock.close()

What does this code do?

  • stream.read(CHUNK, exception_on_overflow=False) grabs one chunk of raw audio from the mic. The overflow flag prevents crashes if the system briefly falls behind.
  • sock.sendto(audio_data, (DEST_IP, DEST_PORT)) fires the raw bytes to the destination address over UDP. No connection handshake, no delivery confirmation.
  • Save sender.py.
  • Run the sender to confirm it starts by running:
python sender.py

You should see Sending audio to 127.0.0.1:5000... printed. The script is capturing your mic and sending packets into the void (nothing is listening yet).

  • Press Ctrl+C to stop the sender.

Sender not starting?

  • If you see a firewall prompt, click Allow access. Windows prompts the first time Python opens a network socket.
  • If you see an error about your microphone, make sure no other app (like a video call) is using it.

help me fix my sender.py script

Create the receiver script

The receiver's job is the mirror of the sender: listen on a port for incoming UDP packets and play each one as audio immediately.

  • Create a new file called receiver.py in your voice-chat folder.
  • Add the setup code that configures speaker output and binds a UDP socket:
import pyaudio
import socket

# Audio settings (must match sender)
CHUNK = 1024
FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100

# Listen on all network interfaces, port 5000
LISTEN_IP = "0.0.0.0"
LISTEN_PORT = 5000

# Open speaker for playback
p = pyaudio.PyAudio()
stream = p.open(format=FORMAT,
                channels=CHANNELS,
                rate=RATE,
                output=True,
                frames_per_buffer=CHUNK)

# Create a UDP socket and bind to listen for packets
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((LISTEN_IP, LISTEN_PORT))

What does this code do?

  • output=True opens the stream for playback (speakers) instead of capture (mic).
  • LISTEN_IP = "0.0.0.0" means "accept packets from any network interface", not just localhost.
  • sock.bind(...) claims port 5000 so incoming packets arrive at this socket. Without binding, the OS wouldn't know which program should receive the data.
  • Below the bind line, add the main receiving loop and cleanup:
print(f"Listening on port {LISTEN_PORT}...")
print("Press Ctrl+C to stop.\n")

try:
    while True:
        # Receive audio data and play it immediately
        data, addr = sock.recvfrom(4096)
        stream.write(data)
except KeyboardInterrupt:
    print("\nStopped.")

# Clean up
stream.stop_stream()
stream.close()
p.terminate()
sock.close()

What does this code do?

  • sock.recvfrom(4096) blocks until a packet arrives, then returns the raw bytes and the sender's address.
  • stream.write(data) pushes those bytes straight to your speakers. No decoding needed because both sides use the same format (paInt16, mono, 44100 Hz).
  • Save receiver.py.
  • Run the receiver to confirm it starts by running:
python receiver.py

You should see Listening on port 5000... printed. The receiver is now waiting for packets.

  • Press Ctrl+C to stop the receiver for now.

Receiver not starting?

  • If you see an error mentioning the address already in use, another program (or a previous run you didn't stop) is already using port 5000. Close any leftover Python processes and try again.
  • If you see a firewall prompt, click Allow access.

help me fix my receiver.py script

✔️ Awesome, I've got everything!

Great. Confirm both sender.py and receiver.py are saved in your voice-chat folder.

ⓧ I'd like to double check the full code

Your sender.py and receiver.py are each built from two chunks above. Scroll up and confirm each file has both its setup chunk and its main loop chunk, in order.

  • sender.py: imports and constants, PyAudio input stream, UDP socket creation, then the sending loop with cleanup.
  • receiver.py: imports and constants, PyAudio output stream, UDP socket creation and bind, then the receiving loop with cleanup.
Test the one-way stream

Both scripts are ready. Now you'll run them side by side in separate terminals to send live audio from one to the other.

  • In VS Code, click Terminal in the top menu bar.
  • Click Split Terminal to open a second terminal pane next to your current one.
  • In the left terminal, start the receiver by running:
python receiver.py

You should see Listening on port 5000... printed. The receiver is now waiting for data.

Before you run the sender, what do you think will happen when audio packets start arriving at the receiver?

  • In the right terminal, start the sender by running:
python sender.py
  • Speak into your microphone.

You should hear your own voice played back through your speakers from the receiver terminal. The audio travels from your mic, into the sender, across a UDP socket on port 5000, into the receiver, and out your speakers.

Use headphones to avoid feedback

If you hear an echoing loop, your speakers are feeding audio back into your mic. Plug in headphones to break the loop.

  • Press Ctrl+C in both terminals to stop sender and receiver.

Not hearing any audio?

  • Make sure the receiver is running before the sender. If the receiver isn't listening when packets arrive, they're silently dropped.
  • Check that both scripts use the same port (5000). If DEST_PORT in the sender doesn't match LISTEN_PORT in the receiver, packets go nowhere.
  • If Windows showed a firewall dialog and you clicked Block, the packets are being dropped. Search for Windows Defender Firewall, click Allow an app through firewall, and add Python.

help me debug why I cannot hear audio between my sender and receiver

Your audio is flowing between two separate programs over the network. Next up, you'll add RTP protocol headers to every packet so the receiver can detect when packets go missing.

Add RTP Protocol Headers

Your one-way audio stream works. You can capture mic audio in one terminal and hear it play back in another.

But right now you are sending raw audio bytes with no structure. If a packet gets dropped or arrives late, you have no way to detect it. Real VoIP systems solve this by wrapping every audio chunk in an RTP protocol header that numbers each packet. The receiver can then compare what arrived against what it expected and report any gaps.

In this step, get ready to:
  • Add RTP header construction to the sender so every packet has a sequence number, timestamp, and source ID.
  • Add RTP header parsing to the receiver so it can detect and log missing packets.
  • Test packet loss detection by interrupting the sender mid-stream and observing the gap.
Add RTP header construction to sender.py

The RTP (Real-time Transport Protocol) header is a fixed 12-byte structure prepended to every audio packet. It carries a sequence number that increments by 1 per packet, a timestamp tracking sample position, and an SSRC (Synchronization Source) identifier so the receiver knows who sent it.

  • Open sender.py in VS Code and replace its entire contents. Start by pasting this setup section at the top:
import pyaudio
import socket
import struct
import random

CHUNK = 1024
FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100

DEST_IP = "127.0.0.1"
DEST_PORT = 5000

# RTP protocol constants
VERSION = 2
PAYLOAD_TYPE = 96
# Unique random identifier for this audio source
SSRC = random.randint(0, 0xFFFFFFFF)

p = pyaudio.PyAudio()
stream = p.open(format=FORMAT,
                channels=CHANNELS,
                rate=RATE,
                input=True,
                frames_per_buffer=CHUNK)

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

What does this setup do?

  • struct lets you pack Python values into raw bytes in network byte order (big-endian). This is how you build the binary RTP header.
  • VERSION = 2 is the RTP version number defined in RFC 3550. Every RTP packet must declare version 2.
  • PAYLOAD_TYPE = 96 tells the receiver what kind of media this is. Values 96-127 are for dynamic/private use.
  • SSRC is a random 32-bit ID so the receiver can distinguish different senders on the same port.
  • Directly below the code you just pasted, add the main loop that builds and sends RTP packets:
sequence_number = 0
timestamp = 0

print(f"Sending audio to {DEST_IP}:{DEST_PORT} with RTP headers...")
print(f"SSRC: {SSRC:#010x}")
print("Press Ctrl+C to stop.\n")

try:
    while True:
        audio_data = stream.read(CHUNK, exception_on_overflow=False)
        # Build the 12-byte RTP header
        first_byte = (VERSION << 6)
        second_byte = PAYLOAD_TYPE
        rtp_header = struct.pack('!BBHII',
                                 first_byte,
                                 second_byte,
                                 sequence_number & 0xFFFF,
                                 timestamp & 0xFFFFFFFF,
                                 SSRC)
        # Prepend header to audio and send
        packet = rtp_header + audio_data
        sock.sendto(packet, (DEST_IP, DEST_PORT))
        sequence_number += 1
        timestamp += CHUNK
except KeyboardInterrupt:
    print(f"\nStopped. Sent {sequence_number} packets.")

stream.stop_stream()
stream.close()
p.terminate()
sock.close()

What does the RTP header construction do?

Each iteration of the loop builds a 12-byte binary header using struct.pack('!BBHII', ...). The format string means: network byte order (!), two unsigned chars (1 byte each), one unsigned short (2 bytes), and two unsigned ints (4 bytes each). Total: 1+1+2+4+4 = 12 bytes.

  • first_byte packs the version (2) into the top 2 bits by shifting left 6 positions.
  • sequence_number increments by 1 per packet so the receiver can detect gaps.
  • timestamp increments by CHUNK (1024) per packet, tracking how many audio samples have been sent.
  • Save sender.py.
  • Run the updated sender to confirm it starts without errors:
python sender.py

You should see output showing your SSRC identifier, confirming the RTP constants initialized correctly:

Sending audio to 127.0.0.1:5000 with RTP headers...
SSRC: 0x1a2b3c4d
Press Ctrl+C to stop.
  • Press Ctrl+C to stop the sender. Your SSRC value will be different because it is randomly generated each time.

Seeing an error when you run sender.py?

  • If you see SyntaxError: check that the struct.pack call has matching parentheses and the format string is exactly '!BBHII'.
  • If you see NameError: name 'struct' is not defined: make sure import struct is at the top of the file.
  • help me debug my sender.py RTP header code
Add RTP header parsing to receiver.py

The receiver needs to understand the new packet format. It will strip the first 12 bytes, unpack the header fields, and compare each sequence number against what it expected. Any gap means packets were lost in transit.

  • Open receiver.py and replace its entire contents. Start with this setup section:
import pyaudio
import socket
import struct

CHUNK = 1024
FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100

LISTEN_IP = "0.0.0.0"
LISTEN_PORT = 5000
RTP_HEADER_SIZE = 12

p = pyaudio.PyAudio()
stream = p.open(format=FORMAT,
                channels=CHANNELS,
                rate=RATE,
                output=True,
                frames_per_buffer=CHUNK)

sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((LISTEN_IP, LISTEN_PORT))

# Tracking variables for loss detection
expected_seq = None
packets_received = 0
packets_lost = 0

print(f"Listening on port {LISTEN_PORT} (RTP mode)...")
print("Press Ctrl+C to stop.\n")

What are the new tracking variables?

  • expected_seq starts as None because you do not know the first packet's sequence number yet. After the first packet arrives, it gets set to the next expected value.
  • packets_received and packets_lost track delivery health so you can calculate a loss percentage.
  • RTP_HEADER_SIZE = 12 tells the receiver how many bytes to strip before playing the audio.
  • Directly below the code you just pasted, add the main receive loop with RTP parsing and loss detection:
try:
    while True:
        data, addr = sock.recvfrom(4096)
        if len(data) < RTP_HEADER_SIZE:
            continue
        # Unpack the 12-byte RTP header
        first_byte, second_byte, seq_num, timestamp, ssrc = struct.unpack(
            '!BBHII', data[:RTP_HEADER_SIZE])
        version = (first_byte >> 6) & 0x03
        payload_type = second_byte & 0x7F
        # Detect sequence gaps (packet loss)
        if expected_seq is not None and seq_num != expected_seq:
            lost = (seq_num - expected_seq) & 0xFFFF
            if lost < 1000:
                packets_lost += lost
                print(f"  [!] Packet loss: expected seq {expected_seq}, got {seq_num} ({lost} lost)")
        expected_seq = (seq_num + 1) & 0xFFFF
        packets_received += 1
        # Strip header and play the audio
        audio_data = data[RTP_HEADER_SIZE:]
        stream.write(audio_data)
        # Print stats every 100 packets
        if packets_received % 100 == 0:
            loss_rate = packets_lost / (packets_received + packets_lost) * 100 if (packets_received + packets_lost) > 0 else 0
            print(f"  [Stats] Received: {packets_received} | Lost: {packets_lost} | Loss: {loss_rate:.1f}% | SSRC: {ssrc:#010x}")

How does loss detection work?

Each time a packet arrives, the receiver compares its seq_num against expected_seq. If they match, no packets were lost. If there is a gap (say expected 50 but got 53), the receiver knows 3 packets disappeared.

The & 0xFFFF mask handles sequence number wraparound. Since the sequence is a 16-bit unsigned integer, it rolls over from 65535 back to 0. The mask keeps the gap calculation correct across that boundary.

  • Directly below that loop, add the exception handler and cleanup code:
except KeyboardInterrupt:
    print(f"\nFinal: Received {packets_received} | Lost {packets_lost}")
    if packets_received > 0:
        loss_rate = packets_lost / (packets_received + packets_lost) * 100
        print(f"Packet loss rate: {loss_rate:.1f}%")

stream.stop_stream()
stream.close()
p.terminate()
sock.close()

What does this section do?

When you press Ctrl+C, the except KeyboardInterrupt block prints a final summary of how many packets arrived and how many were lost. The cleanup lines close the audio stream, terminate PyAudio, and release the socket.

  • Save receiver.py.

Seeing a SyntaxError in receiver.py?

  • The struct.unpack call spans two lines. Make sure the opening parenthesis on line 1 has a matching closing parenthesis after data[:RTP_HEADER_SIZE] on line 2.
  • Confirm the except KeyboardInterrupt: line starts at the same indentation level as try:. If it is indented further, Python will not recognize it as the handler.
  • help me fix my receiver.py RTP parsing code
Test packet loss detection

Both files are updated. Time to prove the RTP headers work by streaming audio and then deliberately causing a gap.

Windows Firewall prompt

When you run the receiver, Windows may show a firewall prompt asking whether to allow Python to communicate on the network. Click Allow access for private networks.

  • In your first VS Code terminal, start the receiver by running:
python receiver.py
  • In your second VS Code terminal, start the sender by running:
python sender.py
  • Speak into your mic. You should hear your voice played back through the receiver, just like before.
  • Watch the receiver terminal. Every 100 packets it prints a stats line showing received count, lost count, loss percentage, and the sender's SSRC.

You should see output like this in the receiver terminal:

  [Stats] Received: 100 | Lost: 0 | Loss: 0.0% | SSRC: 0x1a2b3c4d
  [Stats] Received: 200 | Lost: 0 | Loss: 0.0% | SSRC: 0x1a2b3c4d

Now for the real test. Before you do this next part, predict: what do you think the receiver will show when you stop the sender for 2 seconds and restart it?

  • In the sender terminal, press Ctrl+C to stop the sender.
  • Wait 2 seconds.
  • Restart the sender by running:
python sender.py

The receiver detects the gap immediately. Because the restarted sender begins at sequence number 0, which is far from where the previous sender left off, the receiver reports the jump as lost packets:

  [!] Packet loss: expected seq 487, got 0 (65049 lost)

Why does the lost count look huge?

The receiver expected the next sequence number in the original stream (like 487), but the restarted sender starts fresh at 0. The 16-bit wraparound math calculates the distance from 487 to 0 going forward, which produces a large number.

In a production system, a new SSRC would signal a fresh stream. For this project, the important thing is that the gap was detected at all.

  • Press Ctrl+C in both terminals to stop the sender and receiver.

Not seeing stats printed?

  • Stats print every 100 packets. At 44100 Hz with 1024-frame chunks, that is about every 2.3 seconds. Wait at least 3 seconds while speaking into your mic.
  • If the receiver shows no output at all, confirm both scripts are using port 5000 (DEST_PORT in sender, LISTEN_PORT in receiver).
  • If you allowed the Windows Firewall prompt for the wrong network type, search for Windows Defender Firewall in your Start menu and add a rule allowing Python on private networks.
  • help me troubleshoot why my RTP receiver is not printing stats

✔️ Awesome, I've got everything!

Great. Double check you have saved both sender.py and receiver.py.

ⓧ I'd like to double check the full code

Scroll up and confirm you have all chunks in each file, pasted in order:

  • sender.py: the setup section (imports through socket creation) followed by the main loop section (sequence counters, print statements, the try/while loop building RTP headers and sending, except/cleanup).
  • receiver.py: the setup section (imports through print statements), then the main receive loop (try/while with header parsing, loss detection, stats printing), then the except handler and cleanup.

Your audio stream now has structure. Every packet carries an identity (SSRC), a position (sequence number), and a timestamp. The receiver is no longer flying blind. Next up, you will combine sending and receiving into a single bidirectional voice chat using threads.

Build Bidirectional Voice Chat

Your one-way audio stream works perfectly. You can send audio from a sender to a receiver, and your RTP headers track every packet along the way.

Real conversations go both ways. Right now, one terminal can only send and the other can only receive. You need to send and receive audio simultaneously in a single script, which requires threading so one operation does not block the other.

In this step, get ready to:
  • Create a combined voice chat script with threaded sending and receiving.
  • Run two instances that talk to each other in full-duplex mode.
  • Test over your real WiFi network interface for cross-device readiness.
Create the voice chat script with helper functions

Instead of running separate sender.py and receiver.py scripts, you will combine both into one script that uses threads to do both jobs at once.

  • Create a new file called voice_chat.py in your voice-chat folder.
  • Add the imports, constants, and the RTP packet creation helper by pasting this code:
import pyaudio
import socket
import struct
import threading
import random
import sys

CHUNK = 1024
FORMAT = pyaudio.paInt16
CHANNELS = 1
RATE = 44100

RTP_HEADER_SIZE = 12
VERSION = 2
PAYLOAD_TYPE = 96
SSRC = random.randint(0, 0xFFFFFFFF)


def create_rtp_packet(audio_data, seq_num, timestamp):
    first_byte = (VERSION << 6)
    second_byte = PAYLOAD_TYPE
    rtp_header = struct.pack('!BBHII',
                             first_byte,
                             second_byte,
                             seq_num & 0xFFFF,
                             timestamp & 0xFFFFFFFF,
                             SSRC)
    return rtp_header + audio_data

What does this code do?

  • The imports add threading (for running send/receive simultaneously) and sys (for command-line arguments).
  • The constants are the same audio settings you used in your sender and receiver.
  • create_rtp_packet() wraps the RTP header construction from sender.py into a reusable function.
  • Add the RTP parsing helper and the sender thread function below the code you just pasted:

def parse_rtp_packet(data):
    if len(data) < RTP_HEADER_SIZE:
        return None, None, None, None
    first_byte, second_byte, seq_num, timestamp, ssrc = struct.unpack(
        '!BBHII', data[:RTP_HEADER_SIZE])
    audio_data = data[RTP_HEADER_SIZE:]
    return seq_num, timestamp, ssrc, audio_data


def sender_thread(stream, sock, dest_ip, dest_port):
    seq_num = 0
    timestamp = 0
    print(f"  [SEND] Streaming to {dest_ip}:{dest_port}")
    try:
        while True:
            audio_data = stream.read(CHUNK, exception_on_overflow=False)
            packet = create_rtp_packet(audio_data, seq_num, timestamp)
            sock.sendto(packet, (dest_ip, dest_port))
            seq_num += 1
            timestamp += CHUNK
    except Exception as e:
        print(f"  [SEND] Stopped: {e}")

What does this code do?

  • parse_rtp_packet() extracts the sequence number, timestamp, SSRC, and audio data from incoming packets.
  • sender_thread() runs in its own thread. It reads mic audio in a loop and sends RTP packets to the destination.
  • Because this function runs in a thread, it does not block the receiver from working at the same time.
  • Save voice_chat.py.
  • Verify the file has no syntax errors so far by running:
python -m py_compile voice_chat.py

If there are no errors, the command completes silently with no output. That means your syntax is correct.

Seeing a SyntaxError?

Check that the two function definitions each start with def on a new line, not indented under the previous function.

Make sure the struct.pack and struct.unpack calls have matching parentheses on the closing line.

help me fix a SyntaxError in my voice_chat.py file

  • Add the receiver thread function below the sender thread:

def receiver_thread(stream, sock):
    expected_seq = None
    packets_received = 0
    packets_lost = 0
    print(f"  [RECV] Listening on port {sock.getsockname()[1]}")
    try:
        while True:
            data, addr = sock.recvfrom(4096)
            seq_num, timestamp, ssrc, audio_data = parse_rtp_packet(data)

            if audio_data is None:
                continue

            if expected_seq is not None and seq_num != expected_seq:
                lost = (seq_num - expected_seq) & 0xFFFF
                if lost < 1000:
                    packets_lost += lost

            expected_seq = (seq_num + 1) & 0xFFFF
            packets_received += 1
            stream.write(audio_data)

            if packets_received % 200 == 0:
                loss_rate = packets_lost / (packets_received + packets_lost) * 100 if (packets_received + packets_lost) > 0 else 0
                print(f"  [STATS] Received: {packets_received} | Lost: {packets_lost} | Loss: {loss_rate:.1f}%")
    except Exception as e:
        print(f"  [RECV] Stopped: {e}")

What does this code do?

  • receiver_thread() runs in its own thread. It listens for incoming RTP packets, parses them, and plays the audio.
  • It tracks sequence numbers and reports packet loss stats every 200 packets, just like your standalone receiver.py did.
Add the main function to wire everything together

The main() function accepts command-line arguments so you can run two instances on different ports.

  • Add the main() function's argument parsing and startup banner below your receiver_thread function:

def main():
    if len(sys.argv) != 4:
        print("Usage: python voice_chat.py <listen_port> <dest_ip> <dest_port>")
        print("Example:")
        print("  Terminal 1: python voice_chat.py 5000 127.0.0.1 5001")
        print("  Terminal 2: python voice_chat.py 5001 127.0.0.1 5000")
        sys.exit(1)

    listen_port = int(sys.argv[1])
    dest_ip = sys.argv[2]
    dest_port = int(sys.argv[3])

    print("=" * 50)
    print("  VOICE CHAT - UDP/RTP")
    print("=" * 50)
    print(f"  SSRC: {SSRC:#010x}")
    print(f"  Listening on port: {listen_port}")
    print(f"  Sending to: {dest_ip}:{dest_port}")
    print("=" * 50)
    print("Speak into your mic. Press Ctrl+C to quit.\n")

What does this code do?

  • The argument check ensures the learner provides listen_port, dest_ip, and dest_port. Without all three, the script prints usage and exits.
  • The banner gives the learner a clear summary of the session config when the script starts.
  • Save voice_chat.py and confirm syntax by running:
python -m py_compile voice_chat.py

No output means no errors. You are ready to add the stream and socket setup.

  • Add the PyAudio streams and UDP socket setup directly below the banner prints (still inside main()):

    p = pyaudio.PyAudio()

    input_stream = p.open(format=FORMAT,
                          channels=CHANNELS,
                          rate=RATE,
                          input=True,
                          frames_per_buffer=CHUNK)

    output_stream = p.open(format=FORMAT,
                           channels=CHANNELS,
                           rate=RATE,
                           output=True,
                           frames_per_buffer=CHUNK)

    send_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)

    recv_sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
    recv_sock.bind(("0.0.0.0", listen_port))

What does this code do?

  • Two separate PyAudio streams are created: input_stream for mic capture and output_stream for speaker playback.
  • Two separate UDP sockets are created: send_sock for outgoing packets and recv_sock bound to listen_port for incoming packets.
  • Add the thread launch, main loop, and cleanup code directly below the socket setup (still inside main()):

    send_t = threading.Thread(target=sender_thread,
                              args=(input_stream, send_sock, dest_ip, dest_port),
                              daemon=True)
    recv_t = threading.Thread(target=receiver_thread,
                              args=(output_stream, recv_sock),
                              daemon=True)

    send_t.start()
    recv_t.start()

    try:
        while True:
            pass
    except KeyboardInterrupt:
        print("\n\nShutting down...")

    input_stream.stop_stream()
    input_stream.close()
    output_stream.stop_stream()
    output_stream.close()
    p.terminate()
    send_sock.close()
    recv_sock.close()
    print("Goodbye!")


if __name__ == "__main__":
    main()

What does this code do?

  • Both threads are launched as daemon=True, meaning they automatically stop when the main program exits.
  • The while True: pass loop keeps the main thread alive while the daemon threads do the work. Ctrl+C triggers the cleanup.
  • The if __name__ == "__main__" guard at the bottom ensures main() runs only when you execute the script directly.
  • Save voice_chat.py and confirm the full file compiles by running:
python -m py_compile voice_chat.py

No output means your complete script is syntactically valid.

Seeing an IndentationError?

All lines inside main() must be indented one level (4 spaces). The if __name__ line must be at the top level (no indent).

Check that you did not accidentally nest the socket or thread code inside the banner print block.

help me fix an IndentationError in voice_chat.py

✔️ Awesome, I've got everything!

Great. Make sure you have saved voice_chat.py with all sections in order.

ⓧ I'd like to double check the full code

Scroll up and confirm your file contains these sections in order:

  • Imports and constants (6 imports, audio/RTP constants, SSRC).
  • create_rtp_packet() function.
  • parse_rtp_packet() function.
  • sender_thread() function.
  • receiver_thread() function.
  • main() function with argument parsing, banner, stream setup, socket setup, thread launch, and cleanup.
  • if __name__ == "__main__": main() at the bottom.
Run two instances for full-duplex voice chat

Each instance of voice_chat.py listens on one port and sends to another. Two instances pointed at each other create a full conversation.

  • In VS Code, click Terminal in the top menu, then click Split Terminal.
  • You should now have two terminal panes side by side.
  • In Terminal 1 (left pane), start the first instance by running:
python voice_chat.py 5000 127.0.0.1 5001
  • In Terminal 2 (right pane), start the second instance by running:
python voice_chat.py 5001 127.0.0.1 5000

Why different port numbers?

Instance 1 listens on port 5000 and sends to port 5001. Instance 2 listens on port 5001 and sends to port 5000. This creates a loop where each one receives what the other sends.

Windows Firewall prompt

Windows may ask to allow Python through the firewall. Click Allow access for private networks. This is required for UDP socket communication.

Before you test, what do you expect to happen when you speak into your mic? Will you hear yourself in only one terminal, or both?

  • Speak into your microphone. You should hear your voice played back from the other terminal.
  • Watch both terminals. You should see periodic stats lines appearing as packets flow:
  [STATS] Received: 200 | Lost: 0 | Loss: 0.0%

Both terminals are now sending and receiving simultaneously. You have full-duplex audio over UDP.

Use headphones to avoid feedback

If you hear a loud echo or feedback loop, your speakers are feeding back into your microphone. Plug in headphones to break the loop.

Not hearing any audio?

Make sure both instances are running before you speak. The receiver must be listening before the sender starts transmitting.

Check that the port numbers match. Terminal 1 sends to 5001, and Terminal 2 must be listening on 5001.

help me troubleshoot my voice chat not producing audio

  • Press Ctrl+C in both terminals to stop the chat.
Test with your WiFi IP for cross-device readiness

Using 127.0.0.1 only proves loopback works. To confirm your code works over a real network interface, you will use your PC's actual WiFi IP address.

  • Find your WiFi adapter's IPv4 address by running:
ipconfig
  • Look for the Wireless LAN adapter Wi-Fi section.
  • Find the line that says IPv4 Address. It will look something like 192.168.1.x.

What do you think will happen when you replace 127.0.0.1 with your WiFi IP? Will the audio still flow, or will it break?

  • In Terminal 1, start the voice chat using your WiFi IP by running:
python voice_chat.py 5000 [[WIFI_IP="your WiFi IPv4 address e.g. 192.168.1.105"]] 5001
  • In Terminal 2, start the second instance with the same WiFi IP:
python voice_chat.py 5001 [[WIFI_IP="your WiFi IPv4 address e.g. 192.168.1.105"]] 5000
  • Speak into your mic and confirm you still hear audio in the other terminal.

Audio still flows. Your packets are now traveling through your actual network adapter instead of the loopback interface.

Why does this matter?

Loopback (127.0.0.1) bypasses your network hardware entirely. Using your WiFi IP means packets travel through the same path they would if another device on your network was the destination. This is the same IP another computer would use to reach you.

Audio not working with WiFi IP?

Make sure you allowed Python through Windows Firewall for private networks. The loopback address bypasses the firewall, but your WiFi IP does not.

Confirm both terminals use the exact same IP address. Copy it from the ipconfig output to avoid typos.

help me fix voice chat not working over my WiFi IP

  • Press Ctrl+C in both terminals to stop.

You now have a fully bidirectional voice chat running over UDP with RTP headers tracking every packet. In the Secret Mission, you will push this further by simulating real network chaos and building a jitter buffer to smooth it out.

Secret mission

Simulate Network Chaos and Build a Jitter Buffer

Your voice chat works perfectly on localhost because packets always arrive in order. Real networks are not this kind. They drop packets, delay them randomly, and deliver them out of sequence. Can you simulate those conditions and build the same jitter buffer that Zoom and Discord use to smooth playback?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally, so there are no ongoing costs.

Resources you used:

  • The voice-chat project folder on your Desktop (contains audio_loopback.py, sender.py, receiver.py, voice_chat.py, test_audio.py).
  • The PyAudio Python package installed via pip.

Keep everything running

No action needed. Choose this if you are still actively building or plan to extend the project soon.

  • Your voice-chat folder stays on your Desktop, ready to use.
  • PyAudio remains installed and available for other Python projects.

Pause - I'll come back to this later

Since this project runs locally with no background processes, there is nothing to pause. Your files sit idle on disk and consume no resources until you run them again.

  • Close VS Code and any open terminal windows.
  • When you are ready to return, open the voice-chat folder in VS Code and pick up where you left off.

Delete - I don't want to use this again

Remove all project files and the installed package to start fresh.

  • Delete the project folder by running this command in your terminal:

macOS

rm -rf ~/Desktop/voice-chat

Windows

rmdir /s /q %USERPROFILE%\Desktop\voice-chat
  • Uninstall PyAudio by running:
python -m pip uninstall pyaudio
  • Type y when prompted to confirm the uninstall.

That's a Wrap!

That's a Wrap!

Nice work! You just built a real-time voice chat from scratch using raw UDP sockets and the RTP protocol.

You've learned how to:

  • Build a real-time voice chat that streams live microphone audio between two terminals over UDP sockets, proving why VoIP uses UDP instead of TCP.
  • Implement RTP protocol headers (the same format Zoom and Discord use) to number every audio packet and detect packet loss in real time using sequence number gaps.
  • Use threading to send and receive audio simultaneously, creating a full-duplex bidirectional conversation between two processes.
  • Secret Mission: Simulated real network chaos with packet drops and jitter, then built a jitter buffer to reorder out-of-sequence packets and smooth playback.

Ready to quiz yourself?