Build a RAG Chat API with Azure AI

Build a document-grounded chat API with vector search and streaming.

Introduction

30 Second Summary

Every AI chatbot can generate text. The ones that stand out in production answer questions using your actual data and show exactly where the information came from.

In this project, you will build a full-stack Retrieval-Augmented Generation (RAG) chat application that retrieves answers from your own documents using vector search, then streams AI-generated responses with source citations.

What You'll Build

Picture typing a question about Azure architecture into a chat interface and watching the answer appear word-by-word. Below each response, clickable citations reveal the exact document passages the AI used to form its answer.

By the end of this project, you'll have:

  • A streaming Fastify chat API connected to Azure OpenAI that retrieves relevant passages from your documents and generates grounded answers with source citations.
  • A Next.js chat frontend with real-time streaming where you type a question and watch the response appear character-by-character, with clickable sources displayed below each answer.
  • A fully Dockerized application you can start with a single docker compose up command and demo live in an interview, explaining the architecture end-to-end.
  • Secret Mission: Add a health check endpoint and integration test suite that validates the RAG pipeline end-to-end.

Are there any prerequisites?

No prior Azure experience is needed. This guide walks you through setting up every tool from scratch, including your Azure free account, Azure CLI, and Docker Desktop.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters. Articulating the architecture before you touch any code helps you explain your decisions clearly in an interview setting.

Set Up Azure and Your Project

To build a RAG chat API, you need cloud AI services that handle natural language understanding and vector search. Azure provides both through Azure OpenAI (for chat and embeddings) and Azure AI Search (for document retrieval).

In this step, you will create an Azure account, install the Azure CLI, provision the cloud resources your app depends on, and set up the local project folder.

In this step, get ready to:
  • Create an Azure free account and install the Azure CLI.
  • Provision Azure OpenAI and Azure AI Search resources.
  • Initialize your project repository with folder structure and config.
Create an Azure free account and install the Azure CLI

Azure's free account gives you $200 of credit for 30 days. That is more than enough for the AI services this project uses.

What is Azure?

Azure is Microsoft's cloud platform. It provides hundreds of services for compute, storage, AI, and networking. Your free account includes $200 credit that covers all the resources in this project.

Next, install the Azure CLI so you can manage Azure resources from your terminal.

  • Check if you already have the Azure CLI by running this command:
az --version

✔️ I see a version number

The Azure CLI is already installed. You can skip ahead to the login step below.

ⓧ Command not found

You need to install the Azure CLI. Follow the instructions for your operating system.

macOS

  • Install the Azure CLI with Homebrew by running:
brew update && brew install azure-cli
What if I don't have Homebrew?

Homebrew is a package manager for macOS. If you don't have it, install it first from https://brew.sh and then re-run the command above.

Windows

  • Download the MSI installer from https://aka.ms/installazurecliwindows.
  • Run the downloaded .msi file and accept the defaults in the installation wizard.
  • Close and reopen your terminal after installation completes.
  • Verify the installation by running:
az --version

You should see version information printed in your terminal.

Now log in to Azure so the CLI can manage your resources.

  • Authenticate with Azure by running:
az login

What happens when I run this?

Your default browser opens to a Microsoft sign-in page. Sign in with the same account you used to create your Azure free account.

If no browser opens, the CLI falls back to device code flow and prints a code in your terminal. Visit the URL shown and enter the code to complete authentication.

  • Confirm your connection by running:
az account show

You should see JSON output containing your subscription ID and subscription name.

Not seeing your subscription?

If az login completed but az account show returns an error, try running az login --use-device-code as an alternative login method.

Make sure you signed in with the account that has the Azure free subscription attached. Help me troubleshoot my Azure CLI login

Provision Azure OpenAI and AI Search resources

Your RAG app needs two cloud services: Azure OpenAI for the chat model and embeddings, and Azure AI Search for vector-based document retrieval.

  • Create a resource group to hold all your project resources by running:
az group create --name rg-rag-chat --location eastus

You should see JSON output with "provisioningState": "Succeeded" confirming the group was created.

What is a resource group?

A resource group is a logical container in Azure that holds related resources. When you delete the group later, everything inside it gets deleted too.

  • Create the Azure OpenAI resource by running:
az cognitiveservices account create --name rag-chat-openai --resource-group rg-rag-chat --kind OpenAI --sku S0 --location eastus

This command takes 1-2 minutes. You should see JSON output with the resource details when it completes.

What does this command do?

  • Creates an Azure OpenAI resource named rag-chat-openai in your rg-rag-chat resource group.
  • The S0 SKU is the standard pricing tier for Azure OpenAI (pay-per-token, covered by your free credit).
  • The eastus region hosts the AI models you will deploy next.

Now you need to deploy two AI models inside this resource: a chat model and an embedding model.

  • Open the Azure Portal at https://portal.azure.com in your browser.
  • Search for Azure OpenAI in the top search bar and select the service.
  • Click on your rag-chat-openai resource.
  • Click Model deployments in the left sidebar, then click Deploy model.
  • Select gpt-4.1-mini as the model.
  • Set the deployment name to gpt-4.1-mini and click Deploy.
  • Repeat the deployment process: click Deploy model again.
  • Select text-embedding-ada-002 as the model.
  • Set the deployment name to text-embedding-ada-002 and click Deploy.

You should now see both deployments listed on the Model deployments page.

Why two models?

The chat model (gpt-4.1-mini) generates answers from context. The embedding model (text-embedding-ada-002) converts text into numerical vectors so you can search documents by meaning, not just keywords.

  • Create the Azure AI Search service (free tier) by running:
az search service create --name rag-chat-search --resource-group rg-rag-chat --sku free --location eastus

You should see JSON output confirming the search service was created.

Finally, retrieve the API keys you will need in your application code.

  • Get your Azure OpenAI API key by running:
az cognitiveservices account keys list --name rag-chat-openai --resource-group rg-rag-chat
  • Copy the value of key1 and save it somewhere safe (you will paste it into a config file shortly).
  • Get your Azure AI Search API key by running:
az search admin-key show --service-name rag-chat-search --resource-group rg-rag-chat
  • Copy the value of primaryKey and save it alongside your OpenAI key.

Keep your keys private

Never commit API keys to Git. You will store them in a local .env file that is excluded by .gitignore.

Initialize the project repository

With your cloud services ready, create the local project structure that holds your API, frontend, and documents.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir rag-chat-azure
cd rag-chat-azure
  • Initialize a Git repository and create the folder structure by running:
git init
mkdir api web docs
  • Confirm the folders exist by running:
ls

You should see api, web, and docs listed in the output.

  • Create a .gitignore file to exclude dependencies and secrets by running:
cat > .gitignore << 'EOF'
node_modules/
.env
dist/
.next/
EOF
  • Create the .env.example template inside the api folder by running:
cat > api/.env.example << 'EOF'
AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
AZURE_OPENAI_API_KEY=your-api-key-here
AZURE_OPENAI_DEPLOYMENT=gpt-4.1-mini
AZURE_OPENAI_EMBEDDING_DEPLOYMENT=text-embedding-ada-002
AZURE_SEARCH_ENDPOINT=https://your-search.search.windows.net
AZURE_SEARCH_API_KEY=your-search-key-here
AZURE_SEARCH_INDEX_NAME=rag-docs
PORT=3001
EOF

What does this file do?

The .env.example file documents which environment variables the API needs without containing real secrets. You will create a real .env file from this template in the next step and paste your actual keys there.

  • Stage all files and make your initial commit by running:
git add .
git commit -m "chore: initial project structure and Azure config"

You should see a commit message confirming the files were committed.

Git asking for your name and email?

If Git prompts you to set your identity, run these two commands (replace with your details) and then retry the commit:

git config --global user.email "you@example.com"
git config --global user.name "Your Name"

Help me configure Git for the first time

✔️ Awesome, I've got everything!

Great. Make sure you have saved your Azure API keys somewhere accessible for the next step.

ⓧ I'd like to double check the full code

Your api/.env.example file should contain exactly this:

AZURE_OPENAI_ENDPOINT=https://your-resource.openai.azure.com/
AZURE_OPENAI_API_KEY=your-api-key-here
AZURE_OPENAI_DEPLOYMENT=gpt-4.1-mini
AZURE_OPENAI_EMBEDDING_DEPLOYMENT=text-embedding-ada-002
AZURE_SEARCH_ENDPOINT=https://your-search.search.windows.net
AZURE_SEARCH_API_KEY=your-search-key-here
AZURE_SEARCH_INDEX_NAME=rag-docs
PORT=3001

Your .gitignore at the project root should contain:

node_modules/
.env
dist/
.next/

Your folder structure should be:

rag-chat-azure/
├── api/
│   └── .env.example
├── docs/
├── web/
└── .gitignore

Your Azure account is authenticated, your cloud resources are provisioned, and your project skeleton is committed to Git. Next up, you will build the Fastify chat API that connects to Azure OpenAI and streams responses.

Build the Fastify Chat API

Your Azure services are provisioned and ready. Now you need a working API that proves the connection to Azure OpenAI actually works.

In this step, you will build a Fastify REST API with a /chat endpoint that streams responses from Azure OpenAI. The endpoint works, but without access to your documents it can only give generic answers. That gap becomes the motivation for adding retrieval in the next step.

In this step, get ready to:
  • Scaffold a Fastify TypeScript project with all dependencies.
  • Implement a streaming /chat endpoint connected to Azure OpenAI.
  • Test the endpoint and observe the gap that RAG will fill.
Scaffold the Fastify TypeScript project

Fastify is a high-performance Node.js web framework with built-in TypeScript support and a plugin architecture. You will configure it as an ES module project so you can use top-level await and modern import syntax.

  • In your terminal, move into the api folder by running this command:
cd api
  • Create package.json with project metadata and scripts by pasting the following content into a new file called package.json inside the api folder:
{
  "name": "rag-chat-api",
  "version": "1.0.0",
  "type": "module",
  "scripts": {
    "dev": "tsx watch src/server.ts",
    "build": "tsc",
    "start": "node dist/server.js",
    "index": "tsx src/indexer.ts",
    "test": "vitest run"
  },
  "dependencies": {
    "fastify": "5.11.0",
    "openai": "7.3.0",
    "@azure/openai": "2.0.0",
    "@azure/search-documents": "13.0.0",
    "@fastify/cors": "11.3.0",
    "dotenv": "16.4.7"
  },
  "devDependencies": {
    "typescript": "5.7.3",
    "@types/node": "22.15.0",
    "tsx": "4.19.0",
    "vitest": "3.1.0"
  }
}

What does this file do?

The "type": "module" field enables ES module syntax across the project, which lets you use import statements and top-level await.

The scripts section defines shortcuts: dev runs the server with hot-reload, build compiles TypeScript to JavaScript, and start runs the compiled output for production.

  • Install all dependencies by running:
npm install
  • Confirm the install succeeded. You should see a node_modules folder appear and no error messages in the terminal output.

Seeing permission errors or network failures?

Make sure you are inside the api folder (not the project root). Run pwd to confirm your current directory ends with /api. If you see EACCES errors on macOS, prefix with sudo or fix npm permissions.

Help me troubleshoot my npm install

  • Create the TypeScript configuration file tsconfig.json inside the api folder with this content:
{
  "compilerOptions": {
    "target": "ES2022",
    "module": "ESNext",
    "moduleResolution": "bundler",
    "outDir": "./dist",
    "rootDir": "./src",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "forceConsistentCasingInFileNames": true,
    "resolveJsonModule": true,
    "declaration": true
  },
  "include": ["src/**/*"],
  "exclude": ["node_modules", "dist"]
}

What does this configuration do?

The module and moduleResolution settings tell TypeScript to emit modern ES modules and resolve imports using the bundler strategy. This pairs with "type": "module" in package.json to enable top-level await in your server code.

The outDir and rootDir separate source files from compiled output, keeping the project organized.

  • Create your .env file from the existing template by running:
cp .env.example .env
  • Open .env in VS Code and replace the placeholder values with your real Azure credentials.
  • For AZURE_OPENAI_ENDPOINT: paste your Azure OpenAI resource endpoint from the Azure Portal (looks like https://rag-chat-openai.openai.azure.com/).
  • For AZURE_OPENAI_API_KEY: paste the API key from your Azure OpenAI resource's Keys and Endpoint page.
  • Leave the AZURE_SEARCH_* values as placeholders for now. You will fill them in when you add retrieval in the next step.
  • Save .env.

Never commit your .env file

Your .gitignore from Step 1 already excludes .env. The .env.example file stays in the repo as a template for anyone cloning the project, while your real keys stay local.

  • Create the src folder and the routes subfolder by running:
mkdir -p src/routes
  • Confirm the folder structure exists by running:
ls src

You should see a routes directory listed.

Implement the /chat streaming endpoint

Now you will write the server entry point and the chat route. The server registers plugins and starts listening. The chat route connects to Azure OpenAI and streams the response back as Server-Sent Events.

  • Create src/server.ts and paste the following code:
import Fastify from "fastify";
import cors from "@fastify/cors";
import { chatRoute } from "./routes/chat.js";
import "dotenv/config";

// Create the Fastify instance with logging enabled
const app = Fastify({ logger: true });

// Register CORS so the frontend can call this API
await app.register(cors, { origin: true });

// Register the chat route plugin
app.register(chatRoute);

// Start listening on the configured port
const port = Number(process.env.PORT) || 3001;

app.listen({ port, host: "0.0.0.0" }, (err) => {
  if (err) {
    app.log.error(err);
    process.exit(1);
  }
});

What does this code do?

  • The top-level await on app.register(cors) works because the project is configured as an ES module.
  • CORS is registered so the Next.js frontend (on a different port) can call this API without browser security blocks.
  • The chatRoute import uses a .js extension because TypeScript compiles .ts files to .js at build time, and ES modules require the compiled extension in imports.
  • Save src/server.ts.
  • Create src/routes/chat.ts and paste the imports and client setup:
import { FastifyPluginAsync } from "fastify";
import { AzureOpenAI } from "openai";

// Read Azure OpenAI credentials from environment
const AZURE_OPENAI_ENDPOINT = process.env.AZURE_OPENAI_ENDPOINT!;
const AZURE_OPENAI_API_KEY = process.env.AZURE_OPENAI_API_KEY!;
const AZURE_OPENAI_DEPLOYMENT = process.env.AZURE_OPENAI_DEPLOYMENT!;

// Initialize the Azure OpenAI client
const client = new AzureOpenAI({
  endpoint: AZURE_OPENAI_ENDPOINT,
  apiKey: AZURE_OPENAI_API_KEY,
  apiVersion: "2024-10-21",
  deployment: AZURE_OPENAI_DEPLOYMENT,
});

What does this code do?

The AzureOpenAI class comes from the openai package (not @azure/openai). It wraps the OpenAI SDK with Azure-specific configuration: your resource endpoint, an API key, the API version, and the deployment name you created in the Azure Portal.

The ! after each process.env access tells TypeScript that you guarantee this value exists at runtime. The .env file provides the values when the server starts.

  • Add the route handler below the client setup in the same file (src/routes/chat.ts):
export const chatRoute: FastifyPluginAsync = async (app) => {
  app.post("/chat", async (request, reply) => {
    const { message } = request.body as { message: string };

    // Simple system prompt without document context
    const systemPrompt = `You are an Azure architecture assistant. Answer questions about Azure architecture and services.`;

    // Stream the response from Azure OpenAI
    const stream = await client.chat.completions.create({
      model: "",
      messages: [
        { role: "system", content: systemPrompt },
        { role: "user", content: message },
      ],
      max_tokens: 1024,
      stream: true,
    });

    // Send each token as a Server-Sent Event
    reply.raw.writeHead(200, {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache",
      Connection: "keep-alive",
    });

    for await (const chunk of stream) {
      const content = chunk.choices[0]?.delta?.content;
      if (content) {
        reply.raw.write(`data: ${JSON.stringify({ content })}\n\n`);
      }
    }

    // Signal completion with empty sources
    reply.raw.write(
      `data: ${JSON.stringify({ done: true, sources: [] })}\n\n`
    );
    reply.raw.end();
  });
};

What does this route do?

  • It accepts a POST request with a JSON body containing a message field.
  • It sends the message to Azure OpenAI with stream: true, which returns tokens incrementally instead of waiting for the full response.
  • Each token chunk is sent to the client as a Server-Sent Event (data: ...). When streaming finishes, a final event with done: true signals completion.
  • The model: "" field is empty because the deployment name in the client constructor already specifies which model to use.
  • The sources array is empty for now. In the next step, you will populate it with document references from your search index.
  • Save src/routes/chat.ts.

✔️ Awesome, I've got everything!

Great. Double check you have saved both files before moving on.

ⓧ I'd like to double check the full code

Your src/server.ts should contain these elements in order:

  • Imports for Fastify, @fastify/cors, chatRoute, and dotenv/config.
  • A Fastify instance with logger: true.
  • CORS registration with origin: true.
  • Chat route registration.
  • Listen on port 3001 with host 0.0.0.0.

Your src/routes/chat.ts should contain these elements in order:

  • Imports for FastifyPluginAsync and AzureOpenAI.
  • Three env-var constants (AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, AZURE_OPENAI_DEPLOYMENT).
  • Client instantiation with apiVersion: "2024-10-21".
  • Exported chatRoute plugin with POST /chat handler.
  • Streaming loop writing SSE events, finishing with done: true and empty sources.
Test the API and observe the gap

Time to prove the Azure OpenAI connection works. You will start the server, send a question about your documents, and observe what happens without retrieval.

  • Start the development server by running:
npm run dev

You should see Fastify log output including a line showing the server is listening on port 3001.

Server crashes on startup?

  • If you see a module-not-found error for ./routes/chat.js, check that src/routes/chat.ts exists and uses the correct filename.
  • If you see an authentication error mentioning your Azure endpoint, open .env and verify AZURE_OPENAI_ENDPOINT and AZURE_OPENAI_API_KEY are filled in with your real values (no quotes needed around the values).

Help me fix my server startup error

Before you send a test request, think about this: you are asking the model about the Azure Well-Architected Framework, but the model has no access to your documents. What kind of answer do you expect to get back?

  • Open a second terminal window (keep the server running in the first one).
  • Send a test request by running:
curl -N -X POST http://localhost:3001/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "What is the Azure Well-Architected Framework?"}'

What does -N do in curl?

The -N flag disables output buffering, so you see each streamed event appear in real time instead of waiting for the entire response to finish.

You will see streamed data: lines appear in your terminal as the model generates its response token by token. The final line includes "done": true and an empty sources array.

Read the response carefully. The model gives a generic textbook answer about the Well-Architected Framework. It has no specific details from your documents, no citations, and no way to tell you where the information came from.

Why does this matter?

Without RAG (Retrieval-Augmented Generation), the model relies entirely on its training data. It might hallucinate details or give outdated information.

In the next step, you will add vector search so the model answers from YOUR documents and cites its sources. That missing sources array is what you will fill in.

🙋‍♀️ Curl showing no output or connection refused?

  • Make sure the server is still running in the first terminal. You should see log lines from Fastify.
  • Check you are using port 3001 in the curl URL. If your .env sets a different PORT value, match that in the URL.
  • On Windows, replace the backslashes with carets (^) or put the entire curl command on one line.

Help me debug my curl request

  • Stop the server with Ctrl+C in the first terminal.
  • Commit your progress by running:
git add -A
git commit -m "feat: basic Fastify chat API with Azure OpenAI streaming"
  • Confirm the commit succeeded by checking the output shows files committed with no errors.

Your API works. Azure OpenAI is connected and streaming responses. But the answers are generic and uncitable. Next up, you will feed your own documents into Azure AI Search and wire retrieval into this endpoint so every answer comes with proof.

Add RAG Retrieval with Azure AI Search

Your chat API connects to Azure OpenAI and streams responses. But right now, it gives generic answers because it has no access to your actual documents.

In this step, you'll add Retrieval-Augmented Generation (RAG) by creating sample documents, indexing them with vector embeddings, and building a search service that retrieves relevant passages before the AI generates its answer.

In this step, get ready to:
  • Create sample documents and index them with vector embeddings in Azure AI Search.
  • Build a hybrid search service that combines keyword and vector search.
  • Connect the search results to your chat endpoint for grounded responses with source citations.
Create sample documents for your knowledge base

RAG needs documents to retrieve from. You'll create a few markdown files about Azure architecture topics that your chat API can search through.

  • From the project root, create a docs folder by running this command:
mkdir docs
  • Create the first document by running:
cat > docs/well-architected-framework.md << 'EOF'
# Azure Well-Architected Framework

The Azure Well-Architected Framework consists of five pillars: Reliability, Security, Cost Optimization, Operational Excellence, and Performance Efficiency. Each pillar provides design principles and best practices for building cloud solutions on Azure. Reliability ensures workloads meet availability targets through redundancy and failover strategies. Security implements defense-in-depth across identity, networking, and data layers. Cost Optimization eliminates waste and aligns spending with business priorities. Operational Excellence covers deployment practices, monitoring, and incident response. Performance Efficiency matches resource capacity to actual demand through scaling and caching strategies.
EOF
  • Create the second document by running:
cat > docs/azure-ai-services.md << 'EOF'
# Azure AI Services Overview

Azure AI Services provide pre-built and customizable AI models for common scenarios. Azure OpenAI Service offers access to GPT and embedding models with enterprise security and regional deployment options. Azure AI Search combines full-text search with vector search for semantic retrieval over your own data. Azure AI Document Intelligence extracts structured data from forms and documents. Azure AI Vision analyzes images and video content. These services integrate through REST APIs and SDKs, allowing developers to compose AI pipelines that combine multiple capabilities.
EOF
  • Create the third document by running:
cat > docs/rag-architecture.md << 'EOF'
# RAG Architecture Patterns

Retrieval-Augmented Generation grounds LLM responses in real data by retrieving relevant context before generation. The classic RAG pipeline has three stages: indexing, retrieval, and generation. During indexing, documents are chunked into passages and embedded as vectors. During retrieval, the user query is embedded and compared against stored vectors using cosine similarity. Hybrid search combines keyword matching with vector similarity for better recall. The retrieved passages are injected into the system prompt as context, constraining the LLM to answer from provided sources. Citations map response claims back to specific source passages.
EOF
  • Confirm your documents exist by running:
ls docs/

You should see three files: azure-ai-services.md, rag-architecture.md, and well-architected-framework.md.

Build the document indexer

The indexer reads your markdown files, splits them into chunks, generates a vector embedding for each chunk, and uploads everything to Azure AI Search.

  • Install the Azure AI Search client library by running this command from the api folder:
cd api
npm install @azure/search-documents@13.0.0

Why install if it's already in package.json?

Running the install command ensures the package is actually downloaded to your node_modules folder. The package.json declares the dependency, but npm install fetches it.

  • Create the file src/indexer.ts in the api folder. Start by adding the imports and client setup:
import {
  SearchIndexClient,
  SearchClient,
  AzureKeyCredential,
} from "@azure/search-documents";
import { AzureOpenAI } from "openai";
import { readFileSync, readdirSync } from "fs";
import { join } from "path";
import "dotenv/config";

// Initialize Azure AI Search index client
const indexClient = new SearchIndexClient(
  process.env.AZURE_SEARCH_ENDPOINT!,
  new AzureKeyCredential(process.env.AZURE_SEARCH_API_KEY!)
);

// Initialize Azure OpenAI for generating embeddings
const openai = new AzureOpenAI({
  endpoint: process.env.AZURE_OPENAI_ENDPOINT!,
  apiKey: process.env.AZURE_OPENAI_API_KEY!,
  apiVersion: "2024-10-21",
  deployment: process.env.AZURE_OPENAI_EMBEDDING_DEPLOYMENT!,
});

const INDEX_NAME = process.env.AZURE_SEARCH_INDEX_NAME || "rag-docs";

What does this code do?

The SearchIndexClient manages index schemas (creating and updating the index structure). The AzureOpenAI client here points at your embedding deployment to convert text into vectors.

  • Below the setup code, add the createIndex function that defines the search index schema:
async function createIndex() {
  await indexClient.createOrUpdateIndex({
    name: INDEX_NAME,
    fields: [
      { name: "id", type: "Edm.String", key: true, filterable: true },
      { name: "title", type: "Edm.String", searchable: true },
      { name: "content", type: "Edm.String", searchable: true },
      {
        name: "contentVector",
        type: "Collection(Edm.Single)",
        searchable: true,
        vectorSearchDimensions: 1536,
        vectorSearchProfileName: "default-profile",
      },
    ],
    vectorSearch: {
      algorithms: [{ name: "default-algorithm", kind: "hnsw" }],
      profiles: [
        {
          name: "default-profile",
          algorithmConfigurationName: "default-algorithm",
        },
      ],
    },
  });
  console.log(`Index "${INDEX_NAME}" created.`);
}

What does this index schema do?

The index has four fields: a unique id, a searchable title, a searchable content field for keyword (BM25) search, and a contentVector field storing 1536-dimension embeddings for vector similarity search.

The HNSW algorithm creates a navigable graph over your vectors so that nearest-neighbor lookups are fast even at scale.

  • Next, add the chunkText function that splits documents into smaller passages:
function chunkText(text: string, maxTokens = 500): string[] {
  // Split on sentence boundaries
  const sentences = text.split(/(?<=[.!?])\s+/);
  const chunks: string[] = [];
  let current = "";

  for (const sentence of sentences) {
    // Approximate token count as characters / 4
    if ((current + sentence).length > maxTokens * 4) {
      if (current) chunks.push(current.trim());
      current = sentence;
    } else {
      current += " " + sentence;
    }
  }
  if (current.trim()) chunks.push(current.trim());
  return chunks;
}

Why chunk documents?

Embedding models have token limits, and shorter passages produce more focused vectors. Chunking at roughly 500 tokens per passage gives the search engine granular pieces to match against user queries.

  • Add the indexDocuments function that reads files, generates embeddings, and uploads them:
async function indexDocuments() {
  const docsDir = join(process.cwd(), "..", "docs");
  const files = readdirSync(docsDir).filter((f) => f.endsWith(".md"));
  const searchClient = new SearchClient(
    process.env.AZURE_SEARCH_ENDPOINT!,
    INDEX_NAME,
    new AzureKeyCredential(process.env.AZURE_SEARCH_API_KEY!)
  );
  const documents: Array<Record<string, unknown>> = [];

  for (const file of files) {
    const content = readFileSync(join(docsDir, file), "utf-8");
    const title = file.replace(".md", "");
    const chunks = chunkText(content);
    for (let i = 0; i < chunks.length; i++) {
      const embedding = await openai.embeddings.create({
        model: "",
        input: chunks[i],
      });
      documents.push({
        id: `${title}-chunk-${i}`,
        title,
        content: chunks[i],
        contentVector: embedding.data[0].embedding,
      });
    }
  }

  await searchClient.uploadDocuments(documents);
  console.log(`Indexed ${documents.length} chunks from ${files.length} files.`);
}

What does this code do?

For each markdown file, the function chunks the text, calls the text-embedding-ada-002 deployment to get a 1536-dimension vector for each chunk, then uploads the chunk text and its vector to your search index.

  • Finally, add the main function at the bottom of the file to tie everything together:
async function main() {
  await createIndex();
  await indexDocuments();
}

main().catch(console.error);
  • Save src/indexer.ts.
  • Make sure your .env file has all the required Azure credentials filled in (AZURE_SEARCH_ENDPOINT, AZURE_SEARCH_API_KEY, AZURE_OPENAI_ENDPOINT, AZURE_OPENAI_API_KEY, AZURE_OPENAI_EMBEDDING_DEPLOYMENT).
  • Run the indexer script by executing:
npx tsx src/indexer.ts

You should see output confirming the index was created and your documents were indexed. Something like:

Index "rag-docs" created.
Indexed 3 chunks from 3 files.

Seeing an authentication error?

Double-check that your .env file has the correct values for AZURE_SEARCH_ENDPOINT and AZURE_SEARCH_API_KEY. The endpoint should look like https://your-service.search.windows.net. You can find your admin key in the Azure Portal under your AI Search resource's **Keys** section.

Help me debug my indexer authentication error

  • Commit this progress by running:
cd ..
git add .
git commit -m "feat: document indexer with vector embeddings"
Build the hybrid search service

Now that your documents are indexed, you need a service that your chat route can call to find relevant passages. Hybrid search combines keyword matching (BM25) with vector similarity for better recall than either approach alone.

  • Create the file api/src/services/search.ts and add the client setup and interface:
import { SearchClient, AzureKeyCredential } from "@azure/search-documents";
import { AzureOpenAI } from "openai";

// Client for querying the search index
const searchClient = new SearchClient(
  process.env.AZURE_SEARCH_ENDPOINT!,
  process.env.AZURE_SEARCH_INDEX_NAME!,
  new AzureKeyCredential(process.env.AZURE_SEARCH_API_KEY!)
);

// Client for embedding the user's query
const openai = new AzureOpenAI({
  endpoint: process.env.AZURE_OPENAI_ENDPOINT!,
  apiKey: process.env.AZURE_OPENAI_API_KEY!,
  apiVersion: "2024-10-21",
  deployment: process.env.AZURE_OPENAI_EMBEDDING_DEPLOYMENT!,
});

export interface SearchResult {
  document: string;
  passage: string;
}

What does this code do?

The SearchClient connects to your populated index. The AzureOpenAI client here embeds the user's query into the same vector space as your indexed documents so that cosine similarity comparisons are meaningful.

  • Below the setup, add the searchDocuments function that performs hybrid search:
export async function searchDocuments(query: string): Promise<SearchResult[]> {
  // Embed the user's question into a vector
  const embeddingResponse = await openai.embeddings.create({
    model: "",
    input: query,
  });
  const queryVector = embeddingResponse.data[0].embedding;

  // Perform hybrid search: keyword + vector
  const searchResults = await searchClient.search(query, {
    vectorSearchOptions: {
      queries: [
        {
          kind: "vector",
          vector: queryVector,
          fields: ["contentVector"],
          kNearestNeighborsCount: 3,
        },
      ],
    },
    top: 5,
    select: ["title", "content"],
  });

  const results: SearchResult[] = [];
  for await (const result of searchResults.results) {
    const doc = result.document as { title: string; content: string };
    results.push({ document: doc.title, passage: doc.content });
  }

  return results;
}

What makes this hybrid search?

Passing the query string as the first argument triggers BM25 keyword search. The vectorSearchOptions adds vector similarity on top. Azure AI Search fuses both result sets and ranks them together, giving you the best of both approaches.

  • Save src/services/search.ts.

Why hybrid search instead of vector-only?

Vector search excels at semantic similarity (finding passages about the same concept even with different wording). Keyword search excels at exact matches (specific names, error codes, technical terms). Combining both gives better recall than either alone.

Azure AI Search documentation recommends hybrid search as the default for RAG workloads.

Your chat route at src/routes/chat.ts already imports searchDocuments from this service and injects the retrieved context into the system prompt. With search.ts now in place, the full RAG pipeline is connected.

Test the RAG pipeline

Time to see if your responses are actually grounded in your documents now. Before you run this, think: what do you expect to be different about the response compared to the generic answers you saw in the previous step?

  • Start the development server from the api folder by running:
cd api
npm run dev
  • Open a new terminal and send a test request by running:
curl -X POST http://localhost:3001/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "What is the Azure Well-Architected Framework?"}'

You should see a streamed response that references specific pillars from your document (Reliability, Security, Cost Optimization, Operational Excellence, Performance Efficiency). The final SSE event includes a sources array with document names and relevant passages.

  • Test the boundary case by asking a question not covered by your documents:
curl -X POST http://localhost:3001/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "What is the recipe for chocolate cake?"}'

The model should respond that it cannot find relevant information in the provided context. This proves your system prompt is working correctly to constrain the AI.

Getting empty sources or generic answers?

Make sure the indexer ran successfully and populated the rag-docs index. Check that AZURE_SEARCH_INDEX_NAME in your .env matches the index name used by the indexer.

Also verify that AZURE_OPENAI_EMBEDDING_DEPLOYMENT points to your text-embedding-ada-002 deployment name in Azure.

Help me debug why my RAG pipeline returns empty sources

  • Stop the server with Ctrl+C and commit this work by running:
git add .
git commit -m "feat: RAG pipeline with hybrid search and citations"

✔️ Awesome, I've got everything!

Great. Double check you've saved both src/indexer.ts and src/services/search.ts.

ⓧ I'd like to double check the full code

Your src/services/search.ts should look like this:

import { SearchClient, AzureKeyCredential } from "@azure/search-documents";
import { AzureOpenAI } from "openai";

const searchClient = new SearchClient(
  process.env.AZURE_SEARCH_ENDPOINT!,
  process.env.AZURE_SEARCH_INDEX_NAME!,
  new AzureKeyCredential(process.env.AZURE_SEARCH_API_KEY!)
);

const openai = new AzureOpenAI({
  endpoint: process.env.AZURE_OPENAI_ENDPOINT!,
  apiKey: process.env.AZURE_OPENAI_API_KEY!,
  apiVersion: "2024-10-21",
  deployment: process.env.AZURE_OPENAI_EMBEDDING_DEPLOYMENT!,
});

export interface SearchResult {
  document: string;
  passage: string;
}

And the searchDocuments function:

export async function searchDocuments(query: string): Promise<SearchResult[]> {
  const embeddingResponse = await openai.embeddings.create({
    model: "",
    input: query,
  });
  const queryVector = embeddingResponse.data[0].embedding;

  const searchResults = await searchClient.search(query, {
    vectorSearchOptions: {
      queries: [
        {
          kind: "vector",
          vector: queryVector,
          fields: ["contentVector"],
          kNearestNeighborsCount: 3,
        },
      ],
    },
    top: 5,
    select: ["title", "content"],
  });

  const results: SearchResult[] = [];
  for await (const result of searchResults.results) {
    const doc = result.document as { title: string; content: string };
    results.push({ document: doc.title, passage: doc.content });
  }

  return results;
}

For src/indexer.ts, scroll up and confirm each section is in place: imports and client setup, createIndex, chunkText, indexDocuments, and main.

Your RAG pipeline is fully connected. The chat API now retrieves real document passages and cites its sources. Next up, you'll build a frontend that makes this demoable with real-time streaming.

Build the Next.js Chat Frontend

Your RAG API works, but right now the only way to interact with it is through curl commands in the terminal.

A visual chat interface makes the entire system demoable. In this step, you will build a Next.js frontend that streams AI responses character-by-character and displays source citations below each answer.

In this step, get ready to:
  • Scaffold a Next.js project with TypeScript and Tailwind CSS.
  • Build a streaming chat interface with collapsible source citations.
  • Verify end-to-end flow between frontend and API.
Scaffold Next.js with TypeScript

The /web folder already exists from your initial project structure. You will scaffold the Next.js app directly inside it.

  • Switch to the /web directory by running this command:
cd ../web
  • Scaffold the Next.js project by running:
npx create-next-app@latest . --typescript --app --tailwind --eslint --use-npm

What does this command do?

The . tells create-next-app to scaffold inside the current folder instead of creating a new subfolder.

The flags --typescript, --app, --tailwind, and --eslint pre-select your options so the CLI runs without interactive prompts. It uses recommended defaults for everything else.

  • Wait for the installation to complete. You will see a success message once the dependencies are installed.
  • Verify the scaffold worked by running:
npm run dev

Open http://localhost:3000 in your browser. You should see the default Next.js welcome page.

Don't see the welcome page?

  • Check your terminal for errors. If port 3000 is already in use, Next.js will try port 3001. Since your API is on 3001, stop the API first or use the port shown in your terminal output.
  • Make sure you ran the command from inside the /web folder, not the project root.

Still stuck? Help me troubleshoot my Next.js dev server

  • Press Ctrl+C in the terminal to stop the dev server.

Next, you need to configure Next.js for efficient Docker containerization later. The standalone output mode produces a minimal server bundle.

  • Open next.config.ts in VS Code (the scaffold created it at the root of /web).
  • Replace its entire contents with:
import type { NextConfig } from "next";

const nextConfig: NextConfig = {
  output: "standalone",
};

export default nextConfig;

What does standalone mode do?

The output: "standalone" setting tells Next.js to produce a self-contained server during build. Instead of needing the full node_modules at runtime, it copies only the files the server actually uses. This makes Docker images much smaller.

  • Save next.config.ts.
Build the chat interface

The scaffold created a default app/page.tsx with boilerplate. You will replace it entirely with a chat component that connects to your Fastify API.

  • Open app/page.tsx in VS Code.
  • Delete all existing content in the file.
  • Start building the chat component by pasting this at the top of the file:
"use client";

import { useState, useRef } from "react";

interface Message {
  role: "user" | "assistant";
  content: string;
  sources?: Array<{ document: string; passage: string }>;
}

export default function Home() {
  const [messages, setMessages] = useState<Message[]>([]);
  const [input, setInput] = useState("");
  const [isLoading, setIsLoading] = useState(false);
  const messagesEndRef = useRef<HTMLDivElement>(null);

  const API_URL = process.env.NEXT_PUBLIC_API_URL || "http://localhost:3001";

What does this code do?

  • "use client" marks this as a client component so React hooks work in the browser.
  • The Message interface defines the shape of each chat message, including optional sources for RAG citations.
  • Three state variables track the message history, current input text, and loading state.
  • API_URL defaults to your local Fastify server but can be overridden with an environment variable later.
  • Directly below the API_URL line, paste the first part of the submit handler:
  async function handleSubmit(e: React.FormEvent) {
    e.preventDefault();
    if (!input.trim() || isLoading) return;

    const userMessage: Message = { role: "user", content: input };
    setMessages((prev) => [...prev, userMessage]);
    setInput("");
    setIsLoading(true);

    const assistantMessage: Message = { role: "assistant", content: "" };
    setMessages((prev) => [...prev, assistantMessage]);

    try {
      const response = await fetch(`${API_URL}/chat`, {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ message: input }),
      });

      const reader = response.body!.getReader();
      const decoder = new TextDecoder();
      let fullContent = "";

      while (true) {
        const { done, value } = await reader.read();
        if (done) break;

        const text = decoder.decode(value);
        const lines = text.split("\n").filter((l) => l.startsWith("data: "));

What does this code do?

  • handleSubmit fires when the user presses Send. It adds the user message to state, then immediately adds an empty assistant message as a placeholder.
  • It sends the user's question to your Fastify /chat endpoint.
  • A ReadableStream reader processes the Server-Sent Events response chunk by chunk, filtering for lines that start with data: .
  • Directly below the last line you pasted, add the stream processing logic that completes the function:
        for (const line of lines) {
          const data = JSON.parse(line.slice(6));
          if (data.done) {
            setMessages((prev) => {
              const updated = [...prev];
              updated[updated.length - 1] = {
                ...updated[updated.length - 1],
                sources: data.sources,
              };
              return updated;
            });
          } else if (data.content) {
            fullContent += data.content;
            setMessages((prev) => {
              const updated = [...prev];
              updated[updated.length - 1] = {
                ...updated[updated.length - 1],
                content: fullContent,
              };
              return updated;
            });
          }
        }
      }
    } catch (error) {
      console.error("Chat error:", error);
    } finally {
      setIsLoading(false);
    }
  }

What does this code do?

  • Each SSE line is parsed as JSON. When the server sends { done: true, sources: [...] }, the sources are attached to the assistant message.
  • When the server sends { content: "..." }, the text is appended to fullContent and the message state updates, creating the character-by-character streaming effect.
  • The catch block logs errors, and finally resets the loading state regardless of success or failure.
  • Now add the JSX that renders the message list. Paste this directly below the closing } of handleSubmit:
  return (
    <main className="flex flex-col h-screen max-w-4xl mx-auto p-4">
      <h1 className="text-2xl font-bold mb-4">Azure RAG Chat</h1>

      <div className="flex-1 overflow-y-auto space-y-4 mb-4">
        {messages.map((msg, i) => (
          <div
            key={i}
            className={`p-4 rounded-lg ${
              msg.role === "user"
                ? "bg-blue-100 ml-12"
                : "bg-gray-100 mr-12"
            }`}
          >
            <p className="whitespace-pre-wrap">{msg.content}</p>
            {msg.sources && msg.sources.length > 0 && (
              <details className="mt-2 text-sm text-gray-600">
                <summary className="cursor-pointer font-medium">
                  Sources ({msg.sources.length})
                </summary>
                <ul className="mt-1 space-y-1">
                  {msg.sources.map((s, j) => (
                    <li key={j} className="pl-2 border-l-2 border-blue-300">
                      <strong>{s.document}</strong>: {s.passage.slice(0, 100)}...
                    </li>
                  ))}
                </ul>
              </details>
            )}
          </div>
        ))}
        <div ref={messagesEndRef} />
      </div>

What does this code do?

  • Messages are mapped into styled cards. User messages align right with a blue background. Assistant messages align left with a grey background.
  • When sources exist on a message, they render inside a collapsible <details> element. The learner clicks to expand and see which documents were cited.
  • The messagesEndRef div anchors the scroll position at the bottom of the chat.
  • Finally, paste the form and closing tags directly below the last code:

      <form onSubmit={handleSubmit} className="flex gap-2">
        <input
          type="text"
          value={input}
          onChange={(e) => setInput(e.target.value)}
          placeholder="Ask about Azure architecture..."
          className="flex-1 p-3 border rounded-lg focus:outline-none focus:ring-2 focus:ring-blue-500"
          disabled={isLoading}
        />
        <button
          type="submit"
          disabled={isLoading}
          className="px-6 py-3 bg-blue-600 text-white rounded-lg hover:bg-blue-700 disabled:opacity-50"
        >
          Send
        </button>
      </form>
    </main>
  );
}

What does this code do?

  • The form captures the user's question. On submit, it calls handleSubmit.
  • The input is disabled while isLoading is true, preventing double-sends during streaming.
  • The button shows a visual disabled state with reduced opacity.
  • Save app/page.tsx.

✔️ Awesome, I've got everything!

Great. Double check you have saved app/page.tsx and next.config.ts.

ⓧ I'd like to double check the full code

Your app/page.tsx should contain these sections in order. Scroll up and confirm each is present:

  • The "use client" directive, imports, Message interface, component function with state hooks, and API_URL constant.
  • The handleSubmit function: form prevention, message state setup, fetch call, reader initialization, and streaming decode line.
  • The stream processing for-loop: done-event handling (attaches sources) and content-event handling (appends text), followed by catch/finally.
  • The JSX return: <main> wrapper, messages list with source <details> elements, scroll anchor div, and the input form.

Your next.config.ts should look like this:

import type { NextConfig } from "next";

const nextConfig: NextConfig = {
  output: "standalone",
};

export default nextConfig;
Test the end-to-end flow and commit

Now you will run both the API and the frontend simultaneously to verify the full RAG pipeline works through the browser.

  • Open a new terminal tab or window.
  • Start your Fastify API by running these commands in the new terminal:
cd ../api
npm run dev
  • Return to your original terminal (which should still be in the /web folder).
  • Start the Next.js dev server by running:
npm run dev

Before you open the browser, think about what you expect to happen. Will the response arrive all at once, or stream in gradually?

  • Open http://localhost:3000 in your browser.
  • Type a question like What is the Azure Well-Architected Framework? and click Send.

You should see the AI response stream in character-by-character. Once streaming finishes, a clickable "Sources" section appears below the answer showing which documents were cited.

Not seeing streamed responses?

  • Check that your API is running in the other terminal on port 3001. Look for the Fastify log line showing the server started.
  • Open your browser's developer console (F12) and check the Network tab. If the /chat request shows a CORS error, confirm your API has @fastify/cors registered with origin: true.
  • If the response says it cannot find relevant information, your Azure AI Search index may not be populated. Re-run npm run index in the /api folder.

Still stuck? Help me debug my chat frontend connection to the API

  • Commit your frontend work by running these commands in the /web terminal:
cd ..
git add .
git commit -m "feat: Next.js chat frontend with streaming and citations"

Your chat interface is live. Next up, you will containerize both services with Docker so the entire application runs with a single command.

Dockerize with Docker Compose

Your RAG chat application works end-to-end locally. The API streams AI responses with citations, and the frontend displays them in real time.

Running the app locally is great for development, but sharing it or deploying it means packaging everything into portable containers. In this step, you will create Docker images for both services and orchestrate them with Docker Compose so the entire application starts with a single command.

In this step, get ready to:
  • Create multi-stage Dockerfiles for the API and frontend.
  • Write a Docker Compose file that orchestrates both services with health checks.
  • Run the full application in containers and verify it works end-to-end.
Create Dockerfiles for each service

A multi-stage Docker build uses one stage to compile your code and a second stage to run it. This keeps your final image small because build tools (like TypeScript) never ship to production.

  • Create a file called .dockerignore inside your /api folder with the following content:
node_modules
.env
.git
dist
  • In the same /api folder, create a file called Dockerfile (no extension) with the following content:
FROM node:22-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY tsconfig.json ./
COPY src ./src
RUN npm run build

FROM node:22-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY --from=builder /app/dist ./dist
EXPOSE 3001
CMD ["node", "dist/server.js"]

What does this Dockerfile do?

The first stage (builder) installs all dependencies including dev tools, copies your TypeScript source, and compiles it with npm run build.

The second stage starts fresh from the same node:22-alpine base. It installs only production dependencies (--omit=dev) and copies the compiled JavaScript from the builder. The final image contains no TypeScript compiler, no test runner, and no dev tooling.

  • Verify the API image builds correctly by running this from your project root:
docker build -t rag-chat-api ./api

You should see Docker pull the node:22-alpine base image, install dependencies, run the TypeScript build, and finish with a success message.

Build failing with a TypeScript error?

If you see compile errors, make sure your tsconfig.json and all src/ files are saved. The Docker build copies your local files, so any unsaved changes will be missing inside the container.

If Docker itself is not found, make sure Docker Desktop is running. Help me troubleshoot my Docker build

✔️ Awesome, I've got everything!

Great. Your API image built successfully. Move on to the web Dockerfile next.

ⓧ I'd like to double check the full code

FROM node:22-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY tsconfig.json ./
COPY src ./src
RUN npm run build

FROM node:22-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci --omit=dev
COPY --from=builder /app/dist ./dist
EXPOSE 3001
CMD ["node", "dist/server.js"]
  • Now create a .dockerignore file inside your /web folder:
node_modules
.env
.git
.next
  • In the same /web folder, create a Dockerfile with:
FROM node:22-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:22-alpine
WORKDIR /app
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/public ./public
EXPOSE 3000
ENV PORT=3000
CMD ["node", "server.js"]

What does the web Dockerfile do?

The builder stage runs npm run build which triggers Next.js to produce a standalone server bundle. Your next.config.ts already has output: "standalone" configured from an earlier step, which tells Next.js to produce this minimal bundle.

The runtime stage copies only the standalone server, static assets, and public files. The result is a tiny image that runs with node server.js and does not need the full node_modules folder.

  • Save the file and confirm the new Dockerfiles exist by running:
ls api/Dockerfile web/Dockerfile

You should see both file paths listed in the output.

File not found?

Make sure the file is named exactly Dockerfile with a capital D and no file extension. Some editors add .txt automatically. Help me find my Dockerfile

✔️ Awesome, I've got everything!

Both Dockerfiles are in place. Time to wire them together with Docker Compose.

ⓧ I'd like to double check the full code

FROM node:22-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:22-alpine
WORKDIR /app
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/public ./public
EXPOSE 3000
ENV PORT=3000
CMD ["node", "server.js"]
Create Docker Compose orchestration

Docker Compose lets you define multiple containers in one file and start them all together. It also handles networking so your frontend container can reach the API container by service name.

  • In your project root (the folder that contains both /api and /web), create a file called docker-compose.yml:
services:
  api:
    build: ./api
    ports:
      - "3001:3001"
    env_file:
      - ./api/.env
    healthcheck:
      test: ["CMD", "wget", "--spider", "-q", "http://localhost:3001/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  web:
    build: ./web
    ports:
      - "3000:3000"
    environment:
      - NEXT_PUBLIC_API_URL=http://api:3001
    depends_on:
      api:
        condition: service_healthy

What does this compose file do?

The api service builds from ./api, exposes port 3001, and loads your .env file for Azure credentials. The healthcheck pings the /health endpoint you already built, confirming Azure connectivity before the frontend starts.

The web service builds from ./web and sets NEXT_PUBLIC_API_URL to http://api:3001. Inside the Docker network, services reach each other by name, so api resolves to the API container's IP. The depends_on condition ensures the web service only starts after the API passes its health check.

  • Save docker-compose.yml.

✔️ Awesome, I've got everything!

Your orchestration file is ready. Time to spin up both containers.

ⓧ I'd like to double check the full code

services:
  api:
    build: ./api
    ports:
      - "3001:3001"
    env_file:
      - ./api/.env
    healthcheck:
      test: ["CMD", "wget", "--spider", "-q", "http://localhost:3001/health"]
      interval: 30s
      timeout: 10s
      retries: 3

  web:
    build: ./web
    ports:
      - "3000:3000"
    environment:
      - NEXT_PUBLIC_API_URL=http://api:3001
    depends_on:
      api:
        condition: service_healthy
Test the Docker deployment
  • If you still have your local dev servers running (the Fastify API on port 3001 or the Next.js dev server on port 3000), stop them with Ctrl+C in their terminal windows. Docker needs those ports free.

Before you run the next command, what do you expect to happen? Docker will build both images, wait for the API health check to pass, then start the frontend.

  • Start both services by running this from your project root:
docker compose up --build

You will see Docker build both images (the multi-stage steps), then the API container starts and logs appear. After the health check passes, the web container starts.

  • Open http://localhost:3000 in your browser.
  • Type a question like What is the Azure Well-Architected Framework? and press Send.

You should see the AI response stream in character-by-character with source citations below the answer, exactly as it worked locally. The difference is everything is running inside Docker containers.

Containers failing to start?

  • If you see a port conflict error, make sure you stopped your local dev servers first.
  • If the API health check keeps failing, check that your /api/.env file has valid Azure credentials. The health endpoint tests real connectivity to Azure OpenAI and Azure AI Search.
  • If the web container exits immediately, confirm that next.config.ts has output: "standalone" set. Without it, the build does not produce the standalone server bundle the Dockerfile expects.

Help me debug my Docker Compose setup

  • Once you have verified the chat works through Docker, stop the containers with Ctrl+C in the terminal where Compose is running.
  • Commit your containerization work by running:
git add .
git commit -m "feat: Docker multi-stage builds and Compose orchestration"

Why multi-stage builds matter in interviews

Multi-stage builds demonstrate that you think about production readiness, not just getting code to run. Your final images contain only the compiled output and runtime dependencies. This reduces image size, attack surface, and startup time.

Your entire RAG application now starts with a single command. Next up, take on the Secret Mission to add health check observability and integration tests.

Secret mission

Add Health Checks and Integration Tests

Your RAG chat works, but can you prove it works automatically? Production systems need observability and automated validation. In this mission, you'll add integration tests that exercise the full RAG pipeline end-to-end and verify your health endpoint reports accurate dependency status.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project uses Azure resources that consume from your $200 free credit, though costs are minimal for demo-level usage.

Cost warning

Your Azure OpenAI resource (S0 tier) charges per token used. At demo-level usage, costs are negligible. However, if you leave the resource provisioned and forget about it, other applications or automated calls could consume your free credit.

Your Azure AI Search resource is on the free tier and incurs no charges. If you are done with the project, deleting the resource group removes everything in one command.

Resources you used:

  • Azure resource group rg-rag-chat (contains your Azure OpenAI resource and Azure AI Search service).
  • Local project folder on your Desktop (api, web, docs, docker-compose.yml).
  • Docker images and containers created by docker compose up --build.

Keep everything running

No action needed. Choose this if you are still actively building or preparing for interviews.

  • Monitor your remaining Azure credit in the Azure Portal under Cost Management + Billing.
  • Your Azure AI Search free tier costs nothing regardless of whether you keep it.
  • Docker images remain cached locally. They take up disk space but incur no cloud costs.

Pause - I'll come back to this later

There is no pause option for Azure OpenAI or Azure AI Search. However, these resources only charge when you make API calls, so minimal usage means minimal cost.

  • Stop your Docker containers by running this command from your project root:
  • This frees up local memory and ports while keeping your images cached for next time.
docker compose down
  • Your Azure resources remain provisioned but idle. You only pay per token when you make requests.
  • To resume later, run docker compose up --build from your project folder.

Delete - I don't want to use this again

Remove all project resources and start fresh. This is permanent.

  • Delete all Azure resources by deleting the resource group. Run this in your terminal:
az group delete --name rg-rag-chat --yes --no-wait
  • This removes your Azure OpenAI resource, Azure AI Search service, and all deployed models in one command.
  • Stop and remove your Docker containers and networks:
docker compose down
  • Remove the Docker images built for this project:
docker image rm $(docker image ls --filter reference='*rag*' -q) 2>/dev/null
  • Delete the local project folder. On macOS or Linux:
cd ~/Desktop
rm -rf rag-chat-api
  • On Windows:
cd ~\Desktop
rmdir /s /q rag-chat-api
  • Confirm your Azure resources are gone by running:
az group list --output table
  • You should no longer see rg-rag-chat in the output.

Nice Work!

Nice Work!

You just built a full-stack RAG application on Azure that you can demo live in any Cloud Solution Architect interview.

You've learned how to:

  • Implement a RAG pipeline with Azure AI Search using hybrid (keyword + vector) search to ground AI responses in your own documents with source citations.
  • Build a Next.js chat frontend that streams Azure OpenAI responses in real-time and displays clickable source references.
  • Containerize a multi-service application with Docker multi-stage builds and Docker Compose orchestration, proving operational readiness with a single docker compose up command.
  • Secret Mission: Added health check observability and integration tests proving the RAG pipeline works end-to-end.

Ready to quiz yourself?