Build a Pronunciation Coach with Azure

Score your English pronunciation at the word and phoneme level with Azure Speech.

Introduction

30 Second Summary

Learning to pronounce words in a new language is tough when you can't pinpoint exactly which sounds you're getting wrong. A tutor can hear the difference, but what if your browser could do the same thing?

In this project, you will build a browser-based pronunciation coach that listens to you speak, scores your accuracy at the word and phoneme level, and shows you exactly which sounds need work.

What You'll Build

A full-stack web application using Express.js and Azure Speech Service that records your speech, assesses your pronunciation, and displays color-coded feedback with IPA phoneme details.

By the end of this project, you'll have:

  • A backend server with a secure token endpoint that manages Azure credentials without exposing them in the browser.
  • A pronunciation assessment frontend that records your speech and displays overall scores for accuracy, fluency, completeness, and prosody.
  • Color-coded word-by-word feedback with expandable phoneme breakdowns showing exactly which sounds you nailed and which need practice.
  • Secret Mission: Add a practice tracker that stores your scores in localStorage and shows your improvement over time with a visual chart.

Are there any prerequisites?

You need Node.js installed, a free Azure account, VS Code (or Cursor), and a microphone or headset. No prior experience with speech APIs is required.

Set Up the Project and Azure Speech Resource

Your pronunciation coach needs a speech AI service that can analyze spoken audio at the phoneme level. Azure Speech Service provides exactly this through its Pronunciation Assessment API, scoring individual sounds with IPA (International Phonetic Alphabet) output.

In this step, you will create the Azure Speech resource, set up the project folder, and build an Express server that securely provides authentication tokens to the browser.

In this step, get ready to:
  • Create an Azure Speech resource on the Free (F0) tier.
  • Initialize the Node.js project with Express.
  • Build a token endpoint that securely provides auth tokens to the browser.
Create the Azure Speech resource

The Speech resource holds your API credentials and gives you access to pronunciation assessment, text-to-speech, and speech-to-text features on Azure's Free tier.

  • Open your browser and go to the Azure Portal.
  • Sign in with your Azure account.
  • Click Create a resource in the top-left area of the portal (or search for it in the top search bar).
  • In the search box, type Speech and press Enter.
  • Select Speech from the results, then click Create.
  • Choose your Subscription from the dropdown.
  • Under Resource group, click Create new and name it something like pronunciation-coach-rg.
  • Choose a Region close to you (for example, East US).
  • Enter a Name for your resource (for example, pronunciation-coach-speech).
  • For Pricing tier, select Free (F0).
  • Click Review + create, then click Create.
  • Wait for the deployment to complete, then click Go to resource.

What is the Free (F0) tier?

The Free tier gives you 5 audio hours per month for speech-to-text and 0.5 million characters per month for text-to-speech. This project uses far less than those limits.

  • In your Speech resource, click Keys and Endpoint in the Resource Management section of the left sidebar.
  • Copy KEY 1 and paste it somewhere safe (a text file or notes app). This is your SPEECH_KEY.
  • Note the Location/Region value (for example, eastus). This is your SPEECH_REGION.

Keep these values handy

You will need SPEECH_KEY and SPEECH_REGION in a few minutes when you create the .env file. Do not share your key publicly.

Create the project folder and initialize Node.js

Now you will create the project on your local machine and configure it as an ES module Node.js project with Express as the web framework.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir pronunciation-coach
cd pronunciation-coach
  • Initialize a new Node.js project by running:
npm init -y

What does this command do?

The npm init -y command creates a package.json file with default settings. You will update it next to enable ES module syntax and add a start script.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Select the pronunciation-coach folder on your Desktop, then click Open.

You should see package.json in the Explorer sidebar on the left.

  • Open package.json by clicking it in the Explorer sidebar.
  • Replace its entire contents with the following:
{
  "name": "pronunciation-coach",
  "version": "1.0.0",
  "description": "A pronunciation coach using Azure Speech Service",
  "main": "server.js",
  "type": "module",
  "scripts": {
    "start": "node server.js"
  },
  "dependencies": {
    "express": "^4.21.2"
  }
}

What changed from the default?

You added "type": "module" so Node.js supports import/export syntax instead of require(). You also set "main": "server.js" and added a start script.

  • Save package.json with Cmd+S (macOS) or Ctrl+S (Windows).
  • Install the Express dependency by running this in your terminal:
npm install

You should see a node_modules folder and a package-lock.json file appear in your Explorer sidebar.

Not seeing node_modules?

Make sure your terminal is in the pronunciation-coach folder. Run pwd to check. If you are in the wrong folder, run cd ~/Desktop/pronunciation-coach and try npm install again.

Help me troubleshoot

  • Create the public folder (for your frontend files later) by running:
mkdir public
  • Create a .env file to store your Azure credentials. In VS Code, click File then New File.
  • Save it immediately as .env in the root of your pronunciation-coach folder.
  • Paste the following into your .env file, replacing the placeholder values with your actual key and region from earlier:
SPEECH_KEY=your-speech-key-here
SPEECH_REGION=your-region-here

What goes in these values?

Replace your-speech-key-here with the KEY 1 value you copied from the Azure portal. Replace your-region-here with your region (for example, eastus or westus2).

  • Save .env.
Create the Express server with the token endpoint

Your backend has one job: exchange the speech key for a short-lived authorization token. This keeps your API key hidden on the server while the browser uses the temporary token to call Azure directly.

Why use a token endpoint?

If you put the speech key directly in your frontend JavaScript, anyone could view it in the browser's source code. The token endpoint pattern exchanges your secret key for a temporary token (valid for 10 minutes) that the browser can safely use.

This is the recommended pattern from Azure's official Speech SDK samples for browser-based applications.

  • Create a new file called server.js in the root of your pronunciation-coach folder.
  • Paste the following code into server.js:
import express from "express";
import { fileURLToPath } from "url";
import { dirname, join } from "path";

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);

const app = express();
const port = 3000;

// Serve frontend files from the public folder
app.use(express.static(join(__dirname, "public")));

// Token endpoint - exchanges secret key for a short-lived auth token
app.get("/api/get-speech-token", async (req, res) => {
  const speechKey = process.env.SPEECH_KEY;
  const speechRegion = process.env.SPEECH_REGION;

  // Check that environment variables are set
  if (!speechKey || !speechRegion) {
    res.status(400).json({ error: "SPEECH_KEY and SPEECH_REGION environment variables are required." });
    return;
  }

  try {
    // Call Azure's token issuing endpoint
    const tokenResponse = await fetch(
      `https://${speechRegion}.api.cognitive.microsoft.com/sts/v1.0/issueToken`,
      {
        method: "POST",
        headers: {
          "Ocp-Apim-Subscription-Key": speechKey,
          "Content-Type": "application/x-www-form-urlencoded",
        },
      }
    );

    if (!tokenResponse.ok) {
      res.status(tokenResponse.status).json({ error: "Failed to get token from Azure." });
      return;
    }

    // Return the token and region to the browser
    const token = await tokenResponse.text();
    res.json({ token, region: speechRegion });
  } catch (error) {
    res.status(500).json({ error: "Internal server error getting speech token." });
  }
});

app.listen(port, () => {
  console.log(`Pronunciation Coach running at http://localhost:${port}`);
});

What does this code do?

  • The imports and __dirname setup handle ES module path resolution (ES modules do not have __dirname built in like CommonJS).
  • The express.static middleware serves your frontend files from the public folder.
  • The /api/get-speech-token route reads your key from environment variables, calls Azure's /sts/v1.0/issueToken endpoint, and returns the temporary token to the browser.
  • The server listens on port 3000.
  • Save server.js.
  • Start the server by running this command in your terminal (make sure you are in the pronunciation-coach folder):
SPEECH_KEY=your-speech-key-here SPEECH_REGION=your-region-here node server.js

What is this command doing?

This sets SPEECH_KEY and SPEECH_REGION as environment variables for this terminal session, then starts the server. Replace the placeholder values with your actual key and region from the .env file you created.

You should see this output in your terminal:

Pronunciation Coach running at http://localhost:3000
  • Open a new browser tab and visit http://localhost:3000/api/get-speech-token.

You should see a JSON response containing a token field (a long string) and a region field matching your Azure region.

Seeing an error response?

If you see SPEECH_KEY and SPEECH_REGION environment variables are required, make sure you included both values before node server.js in your start command.

If you see a 401 status error, double-check that your SPEECH_KEY matches KEY 1 from the Azure portal exactly. Also confirm your SPEECH_REGION matches the Location/Region value (lowercase, no spaces, for example eastus).

Help me debug my token endpoint

  • Stop the server by pressing Ctrl+C in your terminal.

✔️ Awesome, I've got everything!

Great. Double check you have saved both package.json and server.js.

ⓧ I'd like to double check the full code

Here is the complete package.json:

{
  "name": "pronunciation-coach",
  "version": "1.0.0",
  "description": "A pronunciation coach using Azure Speech Service",
  "main": "server.js",
  "type": "module",
  "scripts": {
    "start": "node server.js"
  },
  "dependencies": {
    "express": "^4.21.2"
  }
}

Here is the complete server.js:

import express from "express";
import { fileURLToPath } from "url";
import { dirname, join } from "path";

const __filename = fileURLToPath(import.meta.url);
const __dirname = dirname(__filename);

const app = express();
const port = 3000;

app.use(express.static(join(__dirname, "public")));

app.get("/api/get-speech-token", async (req, res) => {
  const speechKey = process.env.SPEECH_KEY;
  const speechRegion = process.env.SPEECH_REGION;

  if (!speechKey || !speechRegion) {
    res.status(400).json({ error: "SPEECH_KEY and SPEECH_REGION environment variables are required." });
    return;
  }

  try {
    const tokenResponse = await fetch(
      `https://${speechRegion}.api.cognitive.microsoft.com/sts/v1.0/issueToken`,
      {
        method: "POST",
        headers: {
          "Ocp-Apim-Subscription-Key": speechKey,
          "Content-Type": "application/x-www-form-urlencoded",
        },
      }
    );

    if (!tokenResponse.ok) {
      res.status(tokenResponse.status).json({ error: "Failed to get token from Azure." });
      return;
    }

    const token = await tokenResponse.text();
    res.json({ token, region: speechRegion });
  } catch (error) {
    res.status(500).json({ error: "Internal server error getting speech token." });
  }
});

app.listen(port, () => {
  console.log(`Pronunciation Coach running at http://localhost:${port}`);
});

Your backend is ready to go. Next up, you will build the frontend interface where users select a phrase, hear the correct pronunciation, and record their own attempt.

Build the Frontend Interface

Your backend is running with a token endpoint that securely fetches credentials from Azure. Now you need the page that users actually see and interact with.

In this step, you will build the HTML structure and CSS styling for the pronunciation coach. The interface includes a phrase picker, playback and recording buttons, and a results panel that shows scores and color-coded word cards.

In this step, get ready to:
  • Create the HTML page with controls, score cards, and a word breakdown area.
  • Style the interface with color-coded feedback classes and a responsive layout.
Create the page structure

The server already serves static files from a public/ folder. That folder does not exist yet, so you need to create it along with the HTML file inside.

  • Create the public folder inside your pronunciation-coach project by running this command:
mkdir public
  • Confirm the folder exists by running:
ls

You should see public listed alongside node_modules, package.json, and server.js.

  • Create a new file called index.html inside the public folder and paste the following code:
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Pronunciation Coach</title>
  <link rel="stylesheet" href="style.css">
</head>
<body>
  <div class="container">
    <h1>Pronunciation Coach</h1>
    <p class="subtitle">Practice your English pronunciation and get instant feedback</p>

    <div class="controls">
      <!-- Dropdown with practice phrases the learner can choose from -->
      <select id="phrase-select">
        <option value="The quick brown fox jumps over the lazy dog">The quick brown fox jumps over the lazy dog</option>
        <option value="She sells seashells by the seashore">She sells seashells by the seashore</option>
        <option value="How much wood would a woodchuck chuck">How much wood would a woodchuck chuck</option>
        <option value="Peter Piper picked a peck of pickled peppers">Peter Piper picked a peck of pickled peppers</option>
        <option value="Good morning, how are you today">Good morning, how are you today</option>
      </select>
      <!-- Listen plays TTS, Record captures the learner's speech -->
      <div class="buttons">
        <button id="listen-btn" class="btn btn-listen">Listen</button>
        <button id="record-btn" class="btn btn-record">Record</button>
      </div>
    </div>

    <!-- Status messages shown during recording or playback -->
    <div id="status" class="status"></div>

    <!-- Results section, hidden until an assessment completes -->
    <div id="results" class="results hidden">
      <h2>Your Scores</h2>
      <div class="scores-grid">
        <div class="score-card">
          <span class="score-label">Accuracy</span>
          <span id="accuracy-score" class="score-value">--</span>
        </div>
        <div class="score-card">
          <span class="score-label">Fluency</span>
          <span id="fluency-score" class="score-value">--</span>
        </div>
        <div class="score-card">
          <span class="score-label">Completeness</span>
          <span id="completeness-score" class="score-value">--</span>
        </div>
        <div class="score-card">
          <span class="score-label">Prosody</span>
          <span id="prosody-score" class="score-value">--</span>
        </div>
      </div>

      <h2>Word Breakdown</h2>
      <!-- JavaScript populates word cards here -->
      <div id="word-breakdown" class="word-breakdown"></div>

      <!-- Phoneme details shown when a word card is clicked -->
      <div id="phoneme-details" class="phoneme-details hidden">
        <h3>Phoneme Details</h3>
        <div id="phoneme-content"></div>
      </div>
    </div>
  </div>

  <!-- Azure Speech SDK loaded from CDN -->
  <script src="https://cdn.jsdelivr.net/npm/microsoft-cognitiveservices-speech-sdk@1.50.0/distrib/browser/microsoft.cognitiveservices.speech.sdk.bundle-min.js"></script>
  <!-- App logic handled in a separate file -->
  <script src="app.js"></script>
</body>
</html>

What does this HTML set up?

  • The <select> dropdown holds five tongue-twister and conversational phrases for practice.
  • Two buttons give the learner control: Listen plays the correct pronunciation via text-to-speech, and Record captures the learner's attempt.
  • The results section starts hidden and will show four score cards plus a word-by-word breakdown once an assessment completes.
  • The Azure Speech SDK loads from a CDN so the browser can run speech recognition and text-to-speech directly.
  • Save public/index.html.
  • Start the server by running:
SPEECH_KEY=your-key-here SPEECH_REGION=your-region-here node server.js
  • Open your browser and navigate to http://localhost:3000.

You should see a heading, a phrase dropdown, and two unstyled buttons. The page looks plain because no CSS exists yet.

Seeing a blank page?

  • Make sure index.html is saved inside the public/ folder, not in the project root.
  • Check that the server is running with no errors in the terminal.
  • Open DevTools (Cmd+Option+I on macOS or Ctrl+Shift+I on Windows) and check the Console tab for error messages.

Why is my page not loading at localhost:3000?

✔️ Awesome, I've got everything!

Great. Make sure public/index.html is saved before moving on.

ⓧ I'd like to double check the full code

Here is the complete public/index.html file:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Pronunciation Coach</title>
  <link rel="stylesheet" href="style.css">
</head>
<body>
  <div class="container">
    <h1>Pronunciation Coach</h1>
    <p class="subtitle">Practice your English pronunciation and get instant feedback</p>

    <div class="controls">
      <select id="phrase-select">
        <option value="The quick brown fox jumps over the lazy dog">The quick brown fox jumps over the lazy dog</option>
        <option value="She sells seashells by the seashore">She sells seashells by the seashore</option>
        <option value="How much wood would a woodchuck chuck">How much wood would a woodchuck chuck</option>
        <option value="Peter Piper picked a peck of pickled peppers">Peter Piper picked a peck of pickled peppers</option>
        <option value="Good morning, how are you today">Good morning, how are you today</option>
      </select>
      <div class="buttons">
        <button id="listen-btn" class="btn btn-listen">Listen</button>
        <button id="record-btn" class="btn btn-record">Record</button>
      </div>
    </div>

    <div id="status" class="status"></div>

    <div id="results" class="results hidden">
      <h2>Your Scores</h2>
      <div class="scores-grid">
        <div class="score-card">
          <span class="score-label">Accuracy</span>
          <span id="accuracy-score" class="score-value">--</span>
        </div>
        <div class="score-card">
          <span class="score-label">Fluency</span>
          <span id="fluency-score" class="score-value">--</span>
        </div>
        <div class="score-card">
          <span class="score-label">Completeness</span>
          <span id="completeness-score" class="score-value">--</span>
        </div>
        <div class="score-card">
          <span class="score-label">Prosody</span>
          <span id="prosody-score" class="score-value">--</span>
        </div>
      </div>

      <h2>Word Breakdown</h2>
      <div id="word-breakdown" class="word-breakdown"></div>

      <div id="phoneme-details" class="phoneme-details hidden">
        <h3>Phoneme Details</h3>
        <div id="phoneme-content"></div>
      </div>
    </div>
  </div>

  <script src="https://cdn.jsdelivr.net/npm/microsoft-cognitiveservices-speech-sdk@1.50.0/distrib/browser/microsoft.cognitiveservices.speech.sdk.bundle-min.js"></script>
  <script src="app.js"></script>
</body>
</html>
Style the score display and word cards

The pronunciation coach needs color-coded feedback so learners can instantly see which words they nailed and which need work. Green means a score above 80, yellow means 60 to 80, and red means below 60.

  • Create a new file called style.css inside the public folder and paste the base layout styles:
/* Reset and base layout */
* {
  margin: 0;
  padding: 0;
  box-sizing: border-box;
}

body {
  font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
  background: #f5f7fa;
  color: #333;
  min-height: 100vh;
  padding: 2rem;
}

.container {
  max-width: 800px;
  margin: 0 auto;
}

h1 {
  font-size: 2rem;
  margin-bottom: 0.25rem;
}

.subtitle {
  color: #666;
  margin-bottom: 2rem;
}

/* Controls panel with phrase selector and buttons */
.controls {
  background: white;
  padding: 1.5rem;
  border-radius: 12px;
  box-shadow: 0 2px 8px rgba(0, 0, 0, 0.08);
  margin-bottom: 1.5rem;
}

#phrase-select {
  width: 100%;
  padding: 0.75rem;
  font-size: 1rem;
  border: 2px solid #e2e8f0;
  border-radius: 8px;
  margin-bottom: 1rem;
  background: white;
}

.buttons {
  display: flex;
  gap: 1rem;
}

.btn {
  flex: 1;
  padding: 0.75rem 1.5rem;
  font-size: 1rem;
  font-weight: 600;
  border: none;
  border-radius: 8px;
  cursor: pointer;
  transition: background 0.2s;
}

.btn-listen {
  background: #e2e8f0;
  color: #475569;
}

.btn-listen:hover {
  background: #cbd5e1;
}

.btn-record {
  background: #3b82f6;
  color: white;
}

.btn-record:hover {
  background: #2563eb;
}

.btn:disabled {
  opacity: 0.5;
  cursor: not-allowed;
}

/* Status bar for messages during recording/playback */
.status {
  text-align: center;
  padding: 0.75rem;
  font-weight: 500;
  color: #666;
  min-height: 2.5rem;
}

What do these styles do?

  • The reset removes default browser margins so spacing is consistent across browsers.
  • The .controls panel uses a white background with a subtle box shadow to visually group the phrase selector and buttons.
  • The .btn-record gets a blue background to draw the eye toward the primary action.
  • Save public/style.css and refresh your browser.

You should see a clean white card containing the phrase dropdown and two buttons (one gray, one blue) centered on a light gray background.

Buttons still look unstyled?

  • Check that your file is named exactly style.css (not styles.css) and sits directly inside the public/ folder.
  • Open DevTools and check the Network tab. If style.css shows a 404 status, the filename or path does not match the <link> tag in your HTML.

Why isn't my CSS loading in the browser?

  • Now add the results area styles below the existing code in public/style.css:
/* Results panel shown after assessment */
.results {
  background: white;
  padding: 1.5rem;
  border-radius: 12px;
  box-shadow: 0 2px 8px rgba(0, 0, 0, 0.08);
}

.results h2 {
  font-size: 1.25rem;
  margin-bottom: 1rem;
}

.results h3 {
  font-size: 1.1rem;
  margin-bottom: 0.75rem;
}

/* Utility class to hide elements until needed */
.hidden {
  display: none;
}

/* Four score cards in a row */
.scores-grid {
  display: grid;
  grid-template-columns: repeat(4, 1fr);
  gap: 1rem;
  margin-bottom: 2rem;
}

.score-card {
  text-align: center;
  padding: 1rem;
  background: #f8fafc;
  border-radius: 8px;
}

.score-label {
  display: block;
  font-size: 0.85rem;
  color: #666;
  margin-bottom: 0.5rem;
}

.score-value {
  display: block;
  font-size: 1.75rem;
  font-weight: 700;
}

/* Color classes applied by JavaScript based on score thresholds */
.score-good { color: #16a34a; }
.score-ok { color: #ca8a04; }
.score-bad { color: #dc2626; }

/* Word cards in the breakdown section */
.word-breakdown {
  display: flex;
  flex-wrap: wrap;
  gap: 0.5rem;
  margin-bottom: 1.5rem;
}

.word-card {
  padding: 0.5rem 0.75rem;
  border-radius: 6px;
  font-weight: 500;
  cursor: pointer;
  transition: transform 0.1s;
  position: relative;
}

.word-card:hover {
  transform: scale(1.05);
}

.word-card .word-score {
  display: block;
  font-size: 0.7rem;
  text-align: center;
  margin-top: 0.25rem;
  opacity: 0.8;
}

/* Color-coded word card backgrounds: green (80+), yellow (60-80), red (<60) */
.word-good { background: #dcfce7; color: #166534; }
.word-ok { background: #fef9c3; color: #854d0e; }
.word-bad { background: #fee2e2; color: #991b1b; }
.word-error { background: #f3e8ff; color: #6b21a8; }

/* Phoneme details panel shown when a word card is clicked */
.phoneme-details {
  margin-top: 1.5rem;
  padding: 1rem;
  background: #f8fafc;
  border-radius: 8px;
}

.phoneme-word-title {
  font-weight: 600;
  margin-bottom: 0.5rem;
}

.phoneme-list {
  display: flex;
  flex-wrap: wrap;
  gap: 0.5rem;
}

.phoneme-item {
  padding: 0.4rem 0.6rem;
  border-radius: 4px;
  font-family: "Courier New", monospace;
  font-size: 0.9rem;
}

/* Responsive: stack scores on small screens */
@media (max-width: 600px) {
  .scores-grid {
    grid-template-columns: repeat(2, 1fr);
  }
  .buttons {
    flex-direction: column;
  }
}

How does the color coding work?

The CSS defines three severity classes for scores and three for word cards. JavaScript will apply these dynamically based on thresholds.

  • Scores 80 and above get score-good (green text). Between 60 and 80 gets score-ok (yellow). Below 60 gets score-bad (red).
  • Word cards use the same thresholds but with colored backgrounds instead of text color, making the breakdown scannable at a glance.
  • The word-error class (purple) highlights words that were completely omitted.
  • Save public/style.css and refresh your browser.

The controls panel still looks the same because the results section is hidden by default. You will see it appear once the pronunciation assessment logic is wired up in the next steps.

How can I confirm the results styles are loaded?

Open DevTools (Cmd+Option+I on macOS or Ctrl+Shift+I on Windows). In the Elements panel, find the <div id="results"> element and temporarily remove the hidden class. You should see the score cards appear in a four-column grid with placeholder dashes.

✔️ Awesome, I've got everything!

Great. Make sure public/style.css is saved before moving on.

ⓧ I'd like to double check the full code

Here is the complete public/style.css file:

* {
  margin: 0;
  padding: 0;
  box-sizing: border-box;
}

body {
  font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
  background: #f5f7fa;
  color: #333;
  min-height: 100vh;
  padding: 2rem;
}

.container {
  max-width: 800px;
  margin: 0 auto;
}

h1 {
  font-size: 2rem;
  margin-bottom: 0.25rem;
}

.subtitle {
  color: #666;
  margin-bottom: 2rem;
}

.controls {
  background: white;
  padding: 1.5rem;
  border-radius: 12px;
  box-shadow: 0 2px 8px rgba(0, 0, 0, 0.08);
  margin-bottom: 1.5rem;
}

#phrase-select {
  width: 100%;
  padding: 0.75rem;
  font-size: 1rem;
  border: 2px solid #e2e8f0;
  border-radius: 8px;
  margin-bottom: 1rem;
  background: white;
}

.buttons {
  display: flex;
  gap: 1rem;
}

.btn {
  flex: 1;
  padding: 0.75rem 1.5rem;
  font-size: 1rem;
  font-weight: 600;
  border: none;
  border-radius: 8px;
  cursor: pointer;
  transition: background 0.2s;
}

.btn-listen {
  background: #e2e8f0;
  color: #475569;
}

.btn-listen:hover {
  background: #cbd5e1;
}

.btn-record {
  background: #3b82f6;
  color: white;
}

.btn-record:hover {
  background: #2563eb;
}

.btn:disabled {
  opacity: 0.5;
  cursor: not-allowed;
}

.status {
  text-align: center;
  padding: 0.75rem;
  font-weight: 500;
  color: #666;
  min-height: 2.5rem;
}

.results {
  background: white;
  padding: 1.5rem;
  border-radius: 12px;
  box-shadow: 0 2px 8px rgba(0, 0, 0, 0.08);
}

.results h2 {
  font-size: 1.25rem;
  margin-bottom: 1rem;
}

.results h3 {
  font-size: 1.1rem;
  margin-bottom: 0.75rem;
}

.hidden {
  display: none;
}

.scores-grid {
  display: grid;
  grid-template-columns: repeat(4, 1fr);
  gap: 1rem;
  margin-bottom: 2rem;
}

.score-card {
  text-align: center;
  padding: 1rem;
  background: #f8fafc;
  border-radius: 8px;
}

.score-label {
  display: block;
  font-size: 0.85rem;
  color: #666;
  margin-bottom: 0.5rem;
}

.score-value {
  display: block;
  font-size: 1.75rem;
  font-weight: 700;
}

.score-good { color: #16a34a; }
.score-ok { color: #ca8a04; }
.score-bad { color: #dc2626; }

.word-breakdown {
  display: flex;
  flex-wrap: wrap;
  gap: 0.5rem;
  margin-bottom: 1.5rem;
}

.word-card {
  padding: 0.5rem 0.75rem;
  border-radius: 6px;
  font-weight: 500;
  cursor: pointer;
  transition: transform 0.1s;
  position: relative;
}

.word-card:hover {
  transform: scale(1.05);
}

.word-card .word-score {
  display: block;
  font-size: 0.7rem;
  text-align: center;
  margin-top: 0.25rem;
  opacity: 0.8;
}

.word-good { background: #dcfce7; color: #166534; }
.word-ok { background: #fef9c3; color: #854d0e; }
.word-bad { background: #fee2e2; color: #991b1b; }
.word-error { background: #f3e8ff; color: #6b21a8; }

.phoneme-details {
  margin-top: 1.5rem;
  padding: 1rem;
  background: #f8fafc;
  border-radius: 8px;
}

.phoneme-word-title {
  font-weight: 600;
  margin-bottom: 0.5rem;
}

.phoneme-list {
  display: flex;
  flex-wrap: wrap;
  gap: 0.5rem;
}

.phoneme-item {
  padding: 0.4rem 0.6rem;
  border-radius: 4px;
  font-family: "Courier New", monospace;
  font-size: 0.9rem;
}

@media (max-width: 600px) {
  .scores-grid {
    grid-template-columns: repeat(2, 1fr);
  }
  .buttons {
    flex-direction: column;
  }
}

Your frontend is ready with a clean interface and color-coded feedback classes waiting to be activated. Next up, you will wire the Listen button to Azure's text-to-speech so learners can hear how each phrase should sound.

Implement Text-to-Speech Playback

Your frontend is built and your server is ready to issue tokens. Now it's time to bring the app to life by connecting the browser to Azure's Speech Service.

In this step, you'll create the JavaScript that powers the Listen button. When a learner clicks it, the app fetches an authorization token from your backend, hands it to the Speech SDK, and plays back the correct pronunciation through the browser's speakers.

In this step, get ready to:
  • Create app.js with token fetching logic that securely retrieves an authorization token from your backend.
  • Implement the Listen button handler that plays back the selected phrase using text-to-speech.
Implement the token fetching logic

The Speech SDK running in the browser needs an authorization token to talk to Azure. Your backend already has the /api/get-speech-token endpoint ready. Now you need client-side code to call it.

  • Create a new file called app.js inside the public folder of your pronunciation-coach project.
  • Add the DOM element references and auth state variables by pasting this code:
const phraseSelect = document.getElementById("phrase-select");
const listenBtn = document.getElementById("listen-btn");
const recordBtn = document.getElementById("record-btn");
const statusEl = document.getElementById("status");
const resultsEl = document.getElementById("results");
const wordBreakdownEl = document.getElementById("word-breakdown");
const phonemeDetailsEl = document.getElementById("phoneme-details");
const phonemeContentEl = document.getElementById("phoneme-content");

let authToken = null;
let authRegion = null;

What does this code do?

These lines grab references to every HTML element your JavaScript will interact with. Caching them in variables at the top means you avoid repeated getElementById calls every time a button is clicked.

The authToken and authRegion variables will store the short-lived credentials your backend provides.

  • Below the variables you just added, paste the getToken() function:
async function getToken() {
  try {
    const response = await fetch("/api/get-speech-token");
    if (!response.ok) {
      throw new Error("Failed to get speech token");
    }
    const data = await response.json();
    authToken = data.token;
    authRegion = data.region;
  } catch (error) {
    statusEl.textContent = "Error: Could not connect to server. Make sure the server is running.";
    throw error;
  }
}

What does this code do?

This async function calls your Express backend's token endpoint. If the server responds successfully, it stores the token and region in the module-level variables so other functions can use them.

If the fetch fails (for example, because the server isn't running), it updates the status message on screen and re-throws the error so the calling function can stop gracefully.

  • Below getToken(), add the getScoreClass() helper function:
function getScoreClass(score) {
  if (score >= 80) return "good";
  if (score >= 60) return "ok";
  return "bad";
}

What does this code do?

This utility maps a numeric score to a CSS class name. Scores of 80 or above get good (green), 60-79 get ok (yellow), and anything below 60 gets bad (red). You'll use this to color-code pronunciation scores in the next step.

  • Save app.js.

You won't see a change in the browser yet. The next chunk wires up the Listen button and produces audible output.

Seeing a console error about getElementById?

  • Make sure your file is saved inside the public folder, not in the project root. The HTML loads app.js from the same directory.
  • Check that the element IDs in your JavaScript (like phrase-select) match the id attributes in index.html exactly.
  • help me debug a getElementById error in my app.js
Implement the Listen button handler

With the token logic in place, you can now connect the Listen button to Azure's text-to-speech engine. When clicked, it will speak the selected phrase aloud so the learner hears the correct pronunciation before recording their own attempt.

  • At the bottom of public/app.js, add the Listen button click handler:
listenBtn.addEventListener("click", async () => {
  listenBtn.disabled = true;
  statusEl.textContent = "Playing pronunciation...";

  try {
    await getToken();
    const speechConfig = SpeechSDK.SpeechConfig.fromAuthorizationToken(authToken, authRegion);
    speechConfig.speechSynthesisLanguage = "en-US";

    const synthesizer = new SpeechSDK.SpeechSynthesizer(speechConfig);
    const phrase = phraseSelect.value;

    synthesizer.speakTextAsync(
      phrase,
      (result) => {
        if (result.reason === SpeechSDK.ResultReason.SynthesizingAudioCompleted) {
          statusEl.textContent = "Playback complete. Now try recording!";
        } else {
          statusEl.textContent = "Error during speech synthesis.";
        }
        synthesizer.close();
        listenBtn.disabled = false;
      },
      (error) => {
        statusEl.textContent = "Error: " + error;
        synthesizer.close();
        listenBtn.disabled = false;
      }
    );
  } catch (error) {
    listenBtn.disabled = false;
  }
});

What does this code do?

  • The handler immediately disables the button and shows a status message so the learner knows something is happening.
  • It calls getToken() to get a fresh authorization token from your backend, then creates a SpeechConfig using fromAuthorizationToken. This keeps your Azure key safely on the server.
  • It creates a SpeechSynthesizer and calls speakTextAsync with the phrase from the dropdown. Azure generates the audio and plays it through the browser's speakers.
  • The two callbacks handle success (updates status) and error (shows the error message). Both call synthesizer.close() to release resources and re-enable the button.

Why fetch a new token each time?

Azure authorization tokens expire after 10 minutes. For this learning project, fetching a fresh token before each operation keeps the code simple. In a production app, you would cache the token and refresh it only when it expires.

  • Save app.js.
  • Return to your browser and refresh http://localhost:3000. Make sure your server is still running.
  • Select a phrase from the dropdown and click the Listen button.

You should hear the phrase spoken aloud through your speakers. The status text updates to "Playback complete. Now try recording!" when it finishes.

Not hearing any audio?

  • Check your browser's developer console for errors. If you see a token error, make sure your server is running with the SPEECH_KEY and SPEECH_REGION environment variables set.
  • Make sure your system volume is turned up and no other app is blocking audio output.
  • Confirm you are using Chrome or Edge. Other browsers may not fully support the Speech SDK's audio playback.
  • help me debug text-to-speech playback

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

Here is the complete public/app.js file at this point:

const phraseSelect = document.getElementById("phrase-select");
const listenBtn = document.getElementById("listen-btn");
const recordBtn = document.getElementById("record-btn");
const statusEl = document.getElementById("status");
const resultsEl = document.getElementById("results");
const wordBreakdownEl = document.getElementById("word-breakdown");
const phonemeDetailsEl = document.getElementById("phoneme-details");
const phonemeContentEl = document.getElementById("phoneme-content");

let authToken = null;
let authRegion = null;

async function getToken() {
  try {
    const response = await fetch("/api/get-speech-token");
    if (!response.ok) {
      throw new Error("Failed to get speech token");
    }
    const data = await response.json();
    authToken = data.token;
    authRegion = data.region;
  } catch (error) {
    statusEl.textContent = "Error: Could not connect to server. Make sure the server is running.";
    throw error;
  }
}

function getScoreClass(score) {
  if (score >= 80) return "good";
  if (score >= 60) return "ok";
  return "bad";
}

listenBtn.addEventListener("click", async () => {
  listenBtn.disabled = true;
  statusEl.textContent = "Playing pronunciation...";

  try {
    await getToken();
    const speechConfig = SpeechSDK.SpeechConfig.fromAuthorizationToken(authToken, authRegion);
    speechConfig.speechSynthesisLanguage = "en-US";

    const synthesizer = new SpeechSDK.SpeechSynthesizer(speechConfig);
    const phrase = phraseSelect.value;

    synthesizer.speakTextAsync(
      phrase,
      (result) => {
        if (result.reason === SpeechSDK.ResultReason.SynthesizingAudioCompleted) {
          statusEl.textContent = "Playback complete. Now try recording!";
        } else {
          statusEl.textContent = "Error during speech synthesis.";
        }
        synthesizer.close();
        listenBtn.disabled = false;
      },
      (error) => {
        statusEl.textContent = "Error: " + error;
        synthesizer.close();
        listenBtn.disabled = false;
      }
    );
  } catch (error) {
    listenBtn.disabled = false;
  }
});

Your app can now speak any phrase aloud. Next up, you'll implement the Record button so the app can listen to your pronunciation and score it.

Implement Pronunciation Assessment

In the previous step, you built text-to-speech playback so learners can hear the correct pronunciation of each phrase. Now it's time for the core feature: recording the learner's speech and scoring it at the word and phoneme level.

Azure's Pronunciation Assessment API compares what the learner says against a reference phrase. It returns scores for every word and every individual sound using the IPA (International Phonetic Alphabet).

In this step, get ready to:
  • Record speech from the microphone and run pronunciation assessment.
  • Display overall scores for accuracy, fluency, completeness, and prosody.
  • Show color-coded word-by-word feedback with expandable phoneme details.
Record speech and run pronunciation assessment

The Record button needs to capture microphone audio, send it to Azure for assessment, and handle the result. The PronunciationAssessmentConfig tells Azure exactly how to score the speech: which grading system to use, how granular the breakdown should be, and which phoneme alphabet to return.

  • In public/app.js, add the record button click handler below the existing listenBtn event listener by copying this code:
recordBtn.addEventListener("click", async () => {
  recordBtn.disabled = true;
  listenBtn.disabled = true;
  statusEl.textContent = "Listening... Speak the phrase now.";

  try {
    await getToken();
    const speechConfig = SpeechSDK.SpeechConfig.fromAuthorizationToken(authToken, authRegion);
    speechConfig.speechRecognitionLanguage = "en-US";

    // Capture audio from the user's microphone
    const audioConfig = SpeechSDK.AudioConfig.fromDefaultMicrophoneInput();

    // Configure pronunciation assessment with IPA phonemes and comprehensive scoring
    const referenceText = phraseSelect.value;
    const pronunciationConfig = SpeechSDK.PronunciationAssessmentConfig.fromJSON(
      JSON.stringify({
        referenceText: referenceText,
        gradingSystem: "HundredMark",
        granularity: "Phoneme",
        phonemeAlphabet: "IPA",
        nBestPhonemeCount: 5,
        dimension: "Comprehensive",
      })
    );

    // Create recognizer and apply pronunciation assessment config
    const recognizer = new SpeechSDK.SpeechRecognizer(speechConfig, audioConfig);
    pronunciationConfig.applyTo(recognizer);

    // Run a single recognition pass and handle the result
    recognizer.recognizeOnceAsync(
      (result) => {
        if (result.reason === SpeechSDK.ResultReason.RecognizedSpeech) {
          const jsonResult = result.properties.getProperty(
            SpeechSDK.PropertyId.SpeechServiceResponse_JsonResult
          );
          const parsed = JSON.parse(jsonResult);
          displayResults(parsed);
          statusEl.textContent = "Assessment complete!";
        } else if (result.reason === SpeechSDK.ResultReason.NoMatch) {
          statusEl.textContent = "No speech detected. Please try again and speak clearly.";
        } else {
          statusEl.textContent = "Recognition cancelled or failed. Please try again.";
        }
        recognizer.close();
        recordBtn.disabled = false;
        listenBtn.disabled = false;
      },
      (error) => {
        statusEl.textContent = "Error: " + error;
        recognizer.close();
        recordBtn.disabled = false;
        listenBtn.disabled = false;
      }
    );
  } catch (error) {
    recordBtn.disabled = false;
    listenBtn.disabled = false;
  }
});

What does this code do?

This handler does several things in sequence when the learner clicks Record:

  • AudioConfig.fromDefaultMicrophoneInput() opens the browser's microphone stream for capturing audio.
  • PronunciationAssessmentConfig.fromJSON() tells Azure to score on a 0-100 scale (HundredMark), break results down to individual phonemes (Phoneme granularity), and return IPA symbols.
  • recognizeOnceAsync() listens for one utterance, sends it to Azure, and returns the full assessment result as JSON.
  • The dimension: "Comprehensive" setting enables prosody scoring (rhythm and intonation) in addition to accuracy, fluency, and completeness.

Why IPA instead of SAPI?

Azure supports two phoneme alphabets: IPA and SAPI. IPA is the international standard used in dictionaries and language learning worldwide. SAPI is a Microsoft-specific alphabet used mainly in speech synthesis markup. IPA makes the feedback immediately useful to anyone studying pronunciation.

  • Save public/app.js.

You won't see a visible change yet because the displayResults function doesn't exist yet. You'll add it next.

Browser microphone permission

When you click Record later, your browser will ask for microphone permission. Click Allow when prompted. If you accidentally block it, click the camera/microphone icon in the address bar to re-enable access.

Display overall scores and word breakdown

The assessment JSON from Azure contains an array called NBest with the top-scoring interpretation. Inside it, you'll find four overall scores and a Words array with per-word accuracy and error type information.

  • In public/app.js, add the displayResults function below the recordBtn event listener by copying this code:
function displayResults(parsed) {
  resultsEl.classList.remove("hidden");

  // Extract the top result and its overall scores
  const nBest = parsed.NBest[0];
  const assessment = nBest.PronunciationAssessment;

  const accuracyEl = document.getElementById("accuracy-score");
  const fluencyEl = document.getElementById("fluency-score");
  const completenessEl = document.getElementById("completeness-score");
  const prosodyEl = document.getElementById("prosody-score");

  // Set each score value and apply color class based on threshold
  accuracyEl.textContent = Math.round(assessment.AccuracyScore);
  accuracyEl.className = "score-value score-" + getScoreClass(assessment.AccuracyScore);

  fluencyEl.textContent = Math.round(assessment.FluencyScore);
  fluencyEl.className = "score-value score-" + getScoreClass(assessment.FluencyScore);

  completenessEl.textContent = Math.round(assessment.CompletenessScore);
  completenessEl.className = "score-value score-" + getScoreClass(assessment.CompletenessScore);

  prosodyEl.textContent = Math.round(assessment.ProsodyScore);
  prosodyEl.className = "score-value score-" + getScoreClass(assessment.ProsodyScore);

  // Build clickable word cards with color coding
  wordBreakdownEl.innerHTML = "";
  const words = nBest.Words;

  words.forEach((wordData) => {
    const wordEl = document.createElement("div");
    const wordScore = wordData.PronunciationAssessment.AccuracyScore;
    const errorType = wordData.PronunciationAssessment.ErrorType;

    let cardClass = "word-card word-" + getScoreClass(wordScore);
    if (errorType === "Omission") {
      cardClass = "word-card word-error";
    }

    wordEl.className = cardClass;
    wordEl.innerHTML =
      wordData.Word +
      '<span class="word-score">' +
      (errorType === "Omission" ? "missed" : Math.round(wordScore)) +
      "</span>";

    wordEl.addEventListener("click", () => showPhonemeDetails(wordData));
    wordBreakdownEl.appendChild(wordEl);
  });
}

What does this code do?

  • The four score elements get their numeric values and a CSS class (score-good, score-ok, or score-bad) that colors them green, yellow, or red.
  • Each word becomes a clickable card. Words with an ErrorType of Omission (words you skipped) get a purple "missed" label instead of a score.
  • Clicking any word card calls showPhonemeDetails() to drill down into that word's individual sounds.
  • Save public/app.js.

You still need the showPhonemeDetails function before you can test. The next chunk adds it.

Show phoneme details for each word

When a learner clicks a word card, they need to see exactly which sounds (phonemes) scored well and which need work. Each phoneme is shown as its IPA symbol with a score.

  • In public/app.js, add the showPhonemeDetails function below displayResults by copying this code:
function showPhonemeDetails(wordData) {
  phonemeDetailsEl.classList.remove("hidden");
  phonemeContentEl.innerHTML = "";

  // Show which word's phonemes are being displayed
  const titleEl = document.createElement("div");
  titleEl.className = "phoneme-word-title";
  titleEl.textContent = 'Phonemes for "' + wordData.Word + '"';
  phonemeContentEl.appendChild(titleEl);

  const listEl = document.createElement("div");
  listEl.className = "phoneme-list";

  // Render each IPA phoneme with its accuracy score and color coding
  if (wordData.Phonemes && wordData.Phonemes.length > 0) {
    wordData.Phonemes.forEach((phoneme) => {
      const phonemeEl = document.createElement("span");
      const phonemeScore = phoneme.PronunciationAssessment
        ? phoneme.PronunciationAssessment.AccuracyScore
        : 100;
      phonemeEl.className = "phoneme-item word-" + getScoreClass(phonemeScore);
      phonemeEl.textContent = (phoneme.Phoneme || "?") + " (" + Math.round(phonemeScore) + ")";
      listEl.appendChild(phonemeEl);
    });
  } else {
    listEl.textContent = "No phoneme data available for this word.";
  }

  phonemeContentEl.appendChild(listEl);
}

What does this code do?

  • It reveals the hidden phoneme details panel and clears any previous content.
  • Each phoneme from the word's Phonemes array gets rendered as a color-coded span showing the IPA symbol and its individual accuracy score (e.g. e (92)).
  • The same getScoreClass function applies green/yellow/red coloring so the learner can instantly spot weak sounds.
  • Save public/app.js.
  • Switch back to your terminal and start the server by running:
SPEECH_KEY=your-speech-key-here SPEECH_REGION=your-region-here node server.js
  • Open http://localhost:3000 in Chrome.
  • Select a phrase from the dropdown and click the Record button.
  • When your browser asks for microphone permission, click Allow.
  • Speak the phrase clearly. After a moment, you should see four overall scores (Accuracy, Fluency, Completeness, Prosody) appear with color-coded values, and a row of word cards below them.
  • Click any word card. You should see the Phoneme Details panel expand with IPA symbols and individual scores for that word's sounds.

Not seeing any scores?

  • Check the status message. If it says "No speech detected", make sure your microphone is working and you're speaking clearly within a few seconds of clicking Record.
  • If you see "Could not connect to server", confirm the server is running in your terminal with the correct SPEECH_KEY and SPEECH_REGION values.
  • If you blocked microphone access, click the lock or microphone icon in Chrome's address bar, set Microphone to Allow, then reload the page.

Help me debug my pronunciation assessment

✔️ Awesome, I've got everything!

Great. Double check you've saved your file.

ⓧ I'd like to double check the full code

Here's the complete public/app.js file with everything you've added so far:

const phraseSelect = document.getElementById("phrase-select");
const listenBtn = document.getElementById("listen-btn");
const recordBtn = document.getElementById("record-btn");
const statusEl = document.getElementById("status");
const resultsEl = document.getElementById("results");
const wordBreakdownEl = document.getElementById("word-breakdown");
const phonemeDetailsEl = document.getElementById("phoneme-details");
const phonemeContentEl = document.getElementById("phoneme-content");

let authToken = null;
let authRegion = null;

async function getToken() {
  try {
    const response = await fetch("/api/get-speech-token");
    if (!response.ok) {
      throw new Error("Failed to get speech token");
    }
    const data = await response.json();
    authToken = data.token;
    authRegion = data.region;
  } catch (error) {
    statusEl.textContent = "Error: Could not connect to server. Make sure the server is running.";
    throw error;
  }
}

function getScoreClass(score) {
  if (score >= 80) return "good";
  if (score >= 60) return "ok";
  return "bad";
}

listenBtn.addEventListener("click", async () => {
  listenBtn.disabled = true;
  statusEl.textContent = "Playing pronunciation...";

  try {
    await getToken();
    const speechConfig = SpeechSDK.SpeechConfig.fromAuthorizationToken(authToken, authRegion);
    speechConfig.speechSynthesisLanguage = "en-US";

    const synthesizer = new SpeechSDK.SpeechSynthesizer(speechConfig);
    const phrase = phraseSelect.value;

    synthesizer.speakTextAsync(
      phrase,
      (result) => {
        if (result.reason === SpeechSDK.ResultReason.SynthesizingAudioCompleted) {
          statusEl.textContent = "Playback complete. Now try recording!";
        } else {
          statusEl.textContent = "Error during speech synthesis.";
        }
        synthesizer.close();
        listenBtn.disabled = false;
      },
      (error) => {
        statusEl.textContent = "Error: " + error;
        synthesizer.close();
        listenBtn.disabled = false;
      }
    );
  } catch (error) {
    listenBtn.disabled = false;
  }
});

recordBtn.addEventListener("click", async () => {
  recordBtn.disabled = true;
  listenBtn.disabled = true;
  statusEl.textContent = "Listening... Speak the phrase now.";

  try {
    await getToken();
    const speechConfig = SpeechSDK.SpeechConfig.fromAuthorizationToken(authToken, authRegion);
    speechConfig.speechRecognitionLanguage = "en-US";

    const audioConfig = SpeechSDK.AudioConfig.fromDefaultMicrophoneInput();

    const referenceText = phraseSelect.value;
    const pronunciationConfig = SpeechSDK.PronunciationAssessmentConfig.fromJSON(
      JSON.stringify({
        referenceText: referenceText,
        gradingSystem: "HundredMark",
        granularity: "Phoneme",
        phonemeAlphabet: "IPA",
        nBestPhonemeCount: 5,
        dimension: "Comprehensive",
      })
    );

    const recognizer = new SpeechSDK.SpeechRecognizer(speechConfig, audioConfig);
    pronunciationConfig.applyTo(recognizer);

    recognizer.recognizeOnceAsync(
      (result) => {
        if (result.reason === SpeechSDK.ResultReason.RecognizedSpeech) {
          const jsonResult = result.properties.getProperty(
            SpeechSDK.PropertyId.SpeechServiceResponse_JsonResult
          );
          const parsed = JSON.parse(jsonResult);
          displayResults(parsed);
          statusEl.textContent = "Assessment complete!";
        } else if (result.reason === SpeechSDK.ResultReason.NoMatch) {
          statusEl.textContent = "No speech detected. Please try again and speak clearly.";
        } else {
          statusEl.textContent = "Recognition cancelled or failed. Please try again.";
        }
        recognizer.close();
        recordBtn.disabled = false;
        listenBtn.disabled = false;
      },
      (error) => {
        statusEl.textContent = "Error: " + error;
        recognizer.close();
        recordBtn.disabled = false;
        listenBtn.disabled = false;
      }
    );
  } catch (error) {
    recordBtn.disabled = false;
    listenBtn.disabled = false;
  }
});

function displayResults(parsed) {
  resultsEl.classList.remove("hidden");

  const nBest = parsed.NBest[0];
  const assessment = nBest.PronunciationAssessment;

  const accuracyEl = document.getElementById("accuracy-score");
  const fluencyEl = document.getElementById("fluency-score");
  const completenessEl = document.getElementById("completeness-score");
  const prosodyEl = document.getElementById("prosody-score");

  accuracyEl.textContent = Math.round(assessment.AccuracyScore);
  accuracyEl.className = "score-value score-" + getScoreClass(assessment.AccuracyScore);

  fluencyEl.textContent = Math.round(assessment.FluencyScore);
  fluencyEl.className = "score-value score-" + getScoreClass(assessment.FluencyScore);

  completenessEl.textContent = Math.round(assessment.CompletenessScore);
  completenessEl.className = "score-value score-" + getScoreClass(assessment.CompletenessScore);

  prosodyEl.textContent = Math.round(assessment.ProsodyScore);
  prosodyEl.className = "score-value score-" + getScoreClass(assessment.ProsodyScore);

  wordBreakdownEl.innerHTML = "";
  const words = nBest.Words;

  words.forEach((wordData) => {
    const wordEl = document.createElement("div");
    const wordScore = wordData.PronunciationAssessment.AccuracyScore;
    const errorType = wordData.PronunciationAssessment.ErrorType;

    let cardClass = "word-card word-" + getScoreClass(wordScore);
    if (errorType === "Omission") {
      cardClass = "word-card word-error";
    }

    wordEl.className = cardClass;
    wordEl.innerHTML =
      wordData.Word +
      '<span class="word-score">' +
      (errorType === "Omission" ? "missed" : Math.round(wordScore)) +
      "</span>";

    wordEl.addEventListener("click", () => showPhonemeDetails(wordData));
    wordBreakdownEl.appendChild(wordEl);
  });
}

function showPhonemeDetails(wordData) {
  phonemeDetailsEl.classList.remove("hidden");
  phonemeContentEl.innerHTML = "";

  const titleEl = document.createElement("div");
  titleEl.className = "phoneme-word-title";
  titleEl.textContent = 'Phonemes for "' + wordData.Word + '"';
  phonemeContentEl.appendChild(titleEl);

  const listEl = document.createElement("div");
  listEl.className = "phoneme-list";

  if (wordData.Phonemes && wordData.Phonemes.length > 0) {
    wordData.Phonemes.forEach((phoneme) => {
      const phonemeEl = document.createElement("span");
      const phonemeScore = phoneme.PronunciationAssessment
        ? phoneme.PronunciationAssessment.AccuracyScore
        : 100;
      phonemeEl.className = "phoneme-item word-" + getScoreClass(phonemeScore);
      phonemeEl.textContent = (phoneme.Phoneme || "?") + " (" + Math.round(phonemeScore) + ")";
      listEl.appendChild(phonemeEl);
    });
  } else {
    listEl.textContent = "No phoneme data available for this word.";
  }

  phonemeContentEl.appendChild(listEl);
}

Your pronunciation coach is fully functional. You can hear the correct pronunciation, record your own attempt, see overall scores, drill into word-level accuracy, and inspect individual phonemes. That's a complete speech AI feedback loop running in your browser.

Secret mission

Add a Practice Tracker

Your pronunciation coach gives great feedback, but it only shows one attempt at a time. What if you could track your scores over multiple sessions and see whether you're improving? In this secret mission, you'll add a practice tracker that persists your results in localStorage and shows your score trend over time.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project uses the Azure Speech Free (F0) tier, so there are no ongoing costs as long as you stay within the monthly limits.

Resources you used:

  • Azure Speech resource (Free F0 tier).
  • Local project folder (pronunciation-coach on your Desktop).

Keep everything running

No action needed. Choose this if you are still actively practicing your pronunciation or plan to extend the app further.

  • Your Azure Speech resource stays active on the Free tier with no idle charges.
  • Your local project files remain on your Desktop, ready to run with node server.js whenever you want.
  • Your practice history stays in your browser's localStorage until you clear it.

Pause - I'll come back to this later

The Free tier has no idle charges, so there is nothing to pause on the Azure side. You can stop the local server and come back whenever you are ready.

  • Stop the Node.js server by pressing Ctrl+C in your terminal.
  • Your project files and Azure resource remain untouched for next time.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow these steps to clean up completely.

  • Delete the Azure Speech resource:
  • Open the Azure Portal at portal.azure.com.
  • Search for your Speech resource name in the top search bar and select it.
  • Click Delete in the toolbar at the top of the resource page.
  • Type the resource name to confirm, then click Delete again.
  • Delete the local project folder:

macOS

  • Delete the project folder by running this command in your terminal:
rm -rf ~/Desktop/pronunciation-coach

Windows

  • Delete the project folder by running this command in PowerShell:
Remove-Item -Recurse -Force ~\Desktop\pronunciation-coach
  • Clear your practice history by opening Chrome DevTools (F12), going to the Application tab, selecting Local Storage, and deleting the pronunciation-history entry. Or simply clear all site data for localhost:3000.

Nice Work!

Nice Work!

Nice work! You just built a browser-based pronunciation coach that uses Azure Speech Service to deliver phoneme-level feedback on English speech.

You've learned how to:

  • Build a full-stack web app with an Express backend and a browser frontend that communicates through a secure token endpoint.
  • Integrate Azure Speech Service for both text-to-speech playback and real-time pronunciation assessment with IPA phoneme output and prosody scoring.
  • Display color-coded word-by-word results with expandable phoneme breakdowns so learners can pinpoint exactly which sounds need practice.
  • Secret Mission: Added localStorage-based progress tracking that stores assessment scores and shows improvement trends over time.

Ready to quiz yourself?