Build a RAG Chatbot with Gemini

Build a RAG chatbot that answers questions from your business documents.

Introduction

30 Second Summary

When you message a company's support chat and get an instant, helpful answer instead of waiting 48 hours for a human reply, there's a good chance an AI is doing the heavy lifting. These systems don't just guess. They search through real business documents to find the right answer.

In this project, you will build a production-ready AI customer support chatbot that ingests business documents and answers questions by citing its sources. You'll create the entire system from scratch: document ingestion, semantic search, streaming chat, and a professional interface.

What You'll Build

A RAG-powered chat assistant built with Next.js, Google Gemini, and Upstash Vector that retrieves relevant business documents and streams intelligent answers with source citations.

By the end of this project, you'll have:

  • A document ingestion system that chunks text and stores it as vector embeddings in Upstash Vector for semantic search.
  • A streaming chat API powered by Vercel AI SDK and Gemini that retrieves context from your documents and responds in real time.
  • A professional SaaS-style chat interface with real-time streaming, source citations, and suggested questions.
  • Secret Mission: Add a human handover detection system that flags low-confidence conversations for human review.

Are there any prerequisites?

You should be comfortable with TypeScript and the Next.js App Router. You'll also need a free Google AI Studio account and a free Upstash account, both of which you'll set up in Step 1.

Set Up the Project and Accounts

Before you can build the RAG chatbot, you need the foundation in place: a Next.js project with the right packages, API keys for the AI services, and a folder structure designed for multi-channel delivery.

In this step, you will scaffold the project, create accounts for Google Gemini and Upstash Vector, and wire up the environment so everything talks to each other.

In this step, get ready to:
  • Create a Next.js 15 project with TypeScript, Tailwind CSS, and shadcn/ui components.
  • Set up Google AI Studio and Upstash Vector accounts with API keys.
  • Organize the project folder structure for a modular RAG architecture.
Create the Next.js 15 project and install dependencies

Node.js is the runtime that powers your Next.js development server and builds your app. This project requires version 20 or higher.

  • Open your terminal by pressing Cmd+Space (macOS) or the Windows key (Windows), typing Terminal, and pressing Enter.
  • Verify your Node.js version by running this command:
node --version

✔️ I see version 20 or higher

Your Node.js is ready to go. Continue to the next instruction below.

ⓧ I see a version lower than 20

You need to upgrade Node.js to version 20 or higher. Visit nodejs.org and download the latest LTS installer for your operating system.

  • Run the installer and follow the prompts.
  • Close and reopen your terminal, then run node --version again to confirm the upgrade.

ⓧ Command not found

Node.js is not installed yet. Visit nodejs.org and download the LTS version (20 or higher).

  • Run the installer and follow the prompts.
  • Close and reopen your terminal, then run node --version to confirm the installation.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your Next.js 15 project by running this command:
npx create-next-app@15 assistant-client

What will this command ask you?

The CLI will ask you a series of prompts. Select Yes, use recommended defaults when asked about Next.js defaults. This enables TypeScript, ESLint, Tailwind CSS, and App Router automatically.

  • When you see the prompt asking about recommended defaults, select Yes, use recommended defaults.
  • Move into your new project folder by running:
cd assistant-client
  • Install the core AI and vector database packages by running:
npm install ai @ai-sdk/google @upstash/vector

What are these packages?

The ai package is the Vercel AI SDK that handles streaming chat responses. The @ai-sdk/google package connects the AI SDK to Google Gemini models.

The @upstash/vector package is the client for Upstash Vector, a serverless vector database where you will store and search your document embeddings.

  • Initialize shadcn/ui in your project by running:
npx shadcn@latest init

What will shadcn ask?

The shadcn CLI will detect your Next.js project and configure itself. If it asks about peer dependencies (React 19), select Use --force to proceed.

  • Add the UI components you will use for the chat interface by running:
npx shadcn@latest add button card input scroll-area
  • Confirm the packages installed by running:
ls node_modules/@upstash/vector

You should see a list of files inside the @upstash/vector folder, confirming the package is installed.

Seeing install errors?

If npm install fails with peer dependency warnings, re-run it with --force: npm install ai @ai-sdk/google @upstash/vector --force.

If npx create-next-app@15 is not recognised, make sure you restarted your terminal after installing Node.js. Help me debug my install issues

Create API keys and configure environment variables

Your chatbot needs two external services: Google Gemini for generating responses, and Upstash Vector for storing and searching document embeddings.

  • Open your browser and navigate to aistudio.google.com/apikey.
  • Sign in with your Google account if prompted.
  • Click Create API key.
  • Copy the generated key and save it somewhere safe (you will paste it into your project shortly).

What is Google AI Studio?

Google AI Studio is Google's developer portal for accessing Gemini models. The free tier gives you access to gemini-2.5-flash at no cost, though it is rate-limited and best-effort.

  • Open a new browser tab and navigate to upstash.com.
  • Click Sign Up and create a free account (you can sign up with GitHub or email).
  • Once logged into the Upstash console, click Vector in the top navigation.
  • Click Create Index.
  • Enter a name for your index (for example, assistant-client).
  • Under Embedding Model, select BAAI/bge-m3.
  • Under Type, select Dense.
  • Under Plan, select Free.
  • Click Create.

Why BAAI/bge-m3?

This embedding model is multilingual (excellent for French content) and has a sequence length of 8192 tokens. Upstash handles vectorization automatically when you upsert and query, so you do not need a separate embedding API call.

  • On your new index page, look for the REST URL and REST Token values. Copy both of these.

Now create the environment file that stores your credentials securely.

  • In your code editor, create a new file called .env.local in the root of your assistant-client folder.
  • Paste the following into .env.local, replacing each placeholder with your real values:
GOOGLE_GENERATIVE_AI_API_KEY=your-api-key-here
UPSTASH_VECTOR_REST_URL=your-upstash-vector-url-here
UPSTASH_VECTOR_REST_TOKEN=your-upstash-vector-token-here

Why these specific variable names?

The @ai-sdk/google package automatically reads GOOGLE_GENERATIVE_AI_API_KEY from your environment. The @upstash/vector client reads UPSTASH_VECTOR_REST_URL and UPSTASH_VECTOR_REST_TOKEN. Using these exact names means zero configuration code.

  • Save .env.local.

Can't find your Upstash credentials?

Return to the Upstash console, click Vector in the top nav, then click your index name. The REST URL and token are displayed on the index detail page. Help me find my Upstash credentials

Set up the project folder structure and verify the dev server

A clean folder structure keeps the RAG engine separate from the delivery channel. This separation makes it easy to reuse the same chat logic across web, Messenger, or WhatsApp later.

  • Create the folder structure by running these commands from the root of your assistant-client project:
mkdir -p lib/rag lib/services components/chat

What is each folder for?

  • lib/rag/ holds the RAG engine (chunking documents and retrieving context from the vector database).
  • lib/services/ holds business logic (building prompts, managing chat context) that is transport-agnostic.
  • components/chat/ holds the React components for the chat interface.
  • app/api/ (already created by Next.js) holds the API routes for chat and ingestion.
  • Confirm the folders exist by running:
ls lib/rag lib/services components/chat

You should see empty output for each folder (no errors), confirming they were created.

  • Start the development server by running:
npm run dev
  • Open your browser and navigate to http://localhost:3000.

You should see the default Next.js welcome page, confirming your project is running correctly.

Page not loading at localhost:3000?

Check your terminal for errors. If you see a port conflict, another process may be using port 3000. Stop it or run npm run dev -- -p 3001 to use a different port.

If you see environment variable errors, make sure .env.local is in the project root (not inside a subfolder). Help me fix my dev server

✔️ Awesome, I've got everything!

Great. Make sure you have saved .env.local and your dev server is still running in the terminal.

ⓧ I'd like to double check the full code

Here is what your .env.local file should look like (with your real values replacing the placeholders):

GOOGLE_GENERATIVE_AI_API_KEY=your-api-key-here
UPSTASH_VECTOR_REST_URL=your-upstash-vector-url-here
UPSTASH_VECTOR_REST_TOKEN=your-upstash-vector-token-here

Your project skeleton is in place and all the services are connected. Next up, you will build the document ingestion system that feeds your chatbot its knowledge.

Build the Document Ingestion System

Your project is scaffolded, your accounts are connected, and your dev server is running. Now it's time to give your chatbot something to talk about.

A RAG chatbot can only answer questions if it has documents to pull from. In this step, you'll build the ingestion pipeline that splits business documents into small chunks, converts them into vector embeddings, and stores them in your Upstash Vector index for fast semantic retrieval.

In this step, get ready to:
  • Build a document chunking utility that splits text into overlapping segments with metadata.
  • Create an ingestion API route that chunks documents and stores them in Upstash Vector.
  • Seed your database with sample business FAQ data and verify the vectors exist.
Build the document chunking utility

Before documents can be stored as vectors, they need to be split into small, meaningful pieces called chunks. Chunking ensures each vector represents a focused topic, so retrieval returns precise answers instead of entire documents.

Why chunk documents?

Large language models have context limits, and sending an entire document wastes tokens on irrelevant content. By splitting into chunks of about 500 characters with overlap between them, you ensure the chatbot retrieves only the most relevant passage for each question.

The overlap prevents information from being lost at chunk boundaries. If a sentence spans two chunks, the overlap ensures the full context appears in at least one of them.

  • Create the chunking utility by adding the following code to lib/rag/chunker.ts:
export interface DocumentChunk {
  id: string;
  content: string;
  metadata: {
    source: string;
    chunkIndex: number;
    category: string;
  };
}

export function chunkDocument(
  text: string,
  source: string,
  category: string,
  options: { chunkSize?: number; overlap?: number } = {}
): DocumentChunk[] {
  // Set default chunk size and overlap values
  const { chunkSize = 500, overlap = 50 } = options;
  // Split text into individual sentences at punctuation boundaries
  const sentences = text.split(/(?<=[.!?])\s+/);
  const chunks: DocumentChunk[] = [];
  let currentChunk = '';
  let chunkIndex = 0;

  for (const sentence of sentences) {
    // When the current chunk exceeds the size limit, save it and start a new one
    if (currentChunk.length + sentence.length > chunkSize && currentChunk.length > 0) {
      chunks.push({
        id: `${source}-${chunkIndex}`,
        content: currentChunk.trim(),
        metadata: { source, chunkIndex, category },
      });
      // Keep some words from the end of the previous chunk for overlap
      const words = currentChunk.split(' ');
      const overlapWords = words.slice(-Math.ceil(overlap / 5));
      currentChunk = overlapWords.join(' ') + ' ' + sentence;
      chunkIndex++;
    } else {
      currentChunk += (currentChunk ? ' ' : '') + sentence;
    }
  }

  // Push the final remaining chunk
  if (currentChunk.trim()) {
    chunks.push({
      id: `${source}-${chunkIndex}`,
      content: currentChunk.trim(),
      metadata: { source, chunkIndex, category },
    });
  }

  return chunks;
}

What does this code do?

  • DocumentChunk defines the shape of each chunk: a unique ID, the text content, and metadata tracking where it came from.
  • chunkDocument() splits text at sentence boundaries (., !, ?) and groups sentences until the 500-character limit is reached.
  • Each chunk ID combines the source name and index (e.g. faq-general-0), making it easy to trace answers back to their document.
  • The overlap keeps a few trailing words from the previous chunk, preventing meaning from being lost at boundaries.
  • Save lib/rag/chunker.ts.

You won't see a visible change yet. The next substep uses this utility to build the ingestion API route.

Create the ingestion API route

The chunker splits text into pieces, but you still need a way to receive documents and store them as vectors. A Next.js API route handles incoming documents, passes them through the chunker, and upserts the chunks into Upstash Vector.

Why no separate embedding step?

Your Upstash Vector index uses the built-in BAAI/bge-m3 embedding model. When you upsert with a data field (raw text) instead of a vector array, Upstash automatically generates the embeddings for you.

This eliminates the need to call a separate embedding API, reducing complexity and cost.

  • Create the ingestion route by adding the following code to app/api/ingest/route.ts:
import { Index } from '@upstash/vector';
import { chunkDocument } from '@/lib/rag/chunker';
import { NextResponse } from 'next/server';

// Initialize the Upstash Vector client with your credentials
const index = new Index({
  url: process.env.UPSTASH_VECTOR_REST_URL!,
  token: process.env.UPSTASH_VECTOR_REST_TOKEN!,
});

export async function POST(req: Request) {
  const { documents } = await req.json();

  // Validate that the request body contains an array of documents
  if (!Array.isArray(documents)) {
    return NextResponse.json(
      { error: 'documents must be an array' },
      { status: 400 }
    );
  }

  let totalChunks = 0;

  for (const doc of documents) {
    // Split each document into chunks using the chunker utility
    const chunks = chunkDocument(doc.content, doc.source, doc.category);

    // Upsert chunks with raw text data (Upstash handles embedding automatically)
    await index.upsert(
      chunks.map((chunk) => ({
        id: chunk.id,
        data: chunk.content,
        metadata: chunk.metadata,
      }))
    );

    totalChunks += chunks.length;
  }

  // Return the total number of chunks ingested
  return NextResponse.json({
    success: true,
    chunksIngested: totalChunks,
  });
}

What does this code do?

  • The route accepts a POST request with an array of documents, each containing content, source, and category fields.
  • Each document is passed to chunkDocument() to split it into smaller pieces.
  • The data field in the upsert call tells Upstash to embed the raw text using the BAAI/bge-m3 model you selected when creating your index.
  • The response reports back how many total chunks were created across all documents.
  • Save app/api/ingest/route.ts.

You won't see a browser change, but the route is now live at http://localhost:3000/api/ingest while your dev server is running.

Seeing a TypeScript error?

If your editor shows a red underline on the @/lib/rag/chunker import, make sure you saved lib/rag/chunker.ts in the previous substep. The @/ alias resolves to your project root, so the file must exist at that exact path.

help me fix a TypeScript import error in my Next.js API route

✔️ Awesome, I've got everything!

Great. Make sure both files are saved before moving on.

ⓧ I'd like to double check the full code

Here is the full lib/rag/chunker.ts:

export interface DocumentChunk {
  id: string;
  content: string;
  metadata: {
    source: string;
    chunkIndex: number;
    category: string;
  };
}

export function chunkDocument(
  text: string,
  source: string,
  category: string,
  options: { chunkSize?: number; overlap?: number } = {}
): DocumentChunk[] {
  const { chunkSize = 500, overlap = 50 } = options;
  const sentences = text.split(/(?<=[.!?])\s+/);
  const chunks: DocumentChunk[] = [];
  let currentChunk = '';
  let chunkIndex = 0;

  for (const sentence of sentences) {
    if (currentChunk.length + sentence.length > chunkSize && currentChunk.length > 0) {
      chunks.push({
        id: `${source}-${chunkIndex}`,
        content: currentChunk.trim(),
        metadata: { source, chunkIndex, category },
      });
      const words = currentChunk.split(' ');
      const overlapWords = words.slice(-Math.ceil(overlap / 5));
      currentChunk = overlapWords.join(' ') + ' ' + sentence;
      chunkIndex++;
    } else {
      currentChunk += (currentChunk ? ' ' : '') + sentence;
    }
  }

  if (currentChunk.trim()) {
    chunks.push({
      id: `${source}-${chunkIndex}`,
      content: currentChunk.trim(),
      metadata: { source, chunkIndex, category },
    });
  }

  return chunks;
}

And here is the full app/api/ingest/route.ts:

import { Index } from '@upstash/vector';
import { chunkDocument } from '@/lib/rag/chunker';
import { NextResponse } from 'next/server';

const index = new Index({
  url: process.env.UPSTASH_VECTOR_REST_URL!,
  token: process.env.UPSTASH_VECTOR_REST_TOKEN!,
});

export async function POST(req: Request) {
  const { documents } = await req.json();

  if (!Array.isArray(documents)) {
    return NextResponse.json(
      { error: 'documents must be an array' },
      { status: 400 }
    );
  }

  let totalChunks = 0;

  for (const doc of documents) {
    const chunks = chunkDocument(doc.content, doc.source, doc.category);

    await index.upsert(
      chunks.map((chunk) => ({
        id: chunk.id,
        data: chunk.content,
        metadata: chunk.metadata,
      }))
    );

    totalChunks += chunks.length;
  }

  return NextResponse.json({
    success: true,
    chunksIngested: totalChunks,
  });
}
Seed sample documents and verify ingestion

Your ingestion pipeline is built. Now you need actual business content to test it with. You'll create a seed script with sample French FAQ data and run it against your local API route.

  • Create the seed script by adding the following code to scripts/seed.ts:
// Sample business documents covering FAQs and product info
const SAMPLE_DOCUMENTS = [
  {
    source: 'faq-general',
    category: 'FAQ',
    content: `Notre entreprise est ouverte du lundi au vendredi de 9h a 18h. Nous sommes fermes le week-end et les jours feries. Pour nous contacter, appelez le 01 23 45 67 89 ou envoyez un email a contact@entreprise.fr. Notre adresse est 12 rue de la Paix, 75001 Paris.`,
  },
  {
    source: 'faq-livraison',
    category: 'FAQ',
    content: `La livraison standard est gratuite pour toute commande superieure a 50 euros. Le delai de livraison est de 3 a 5 jours ouvrables. Pour les commandes express, comptez 24h avec un supplement de 9.90 euros. Nous livrons en France metropolitaine, Belgique et Suisse. Le suivi de votre colis est disponible dans votre espace client.`,
  },
  {
    source: 'faq-retours',
    category: 'FAQ',
    content: `Vous disposez de 30 jours apres reception pour retourner un article. Le retour est gratuit via notre etiquette prepayee disponible dans votre espace client. Le remboursement est effectue sous 5 jours ouvrables apres reception du retour. Les articles personnalises ne sont pas retournables sauf defaut de fabrication.`,
  },
  {
    source: 'produits-services',
    category: 'Produits',
    content: `Nous proposons trois formules d'abonnement: Starter a 29 euros par mois (ideal pour les independants), Pro a 79 euros par mois (pour les petites equipes jusqu'a 5 personnes), et Enterprise sur devis (equipes de plus de 5 personnes avec support dedie). Toutes les formules incluent un essai gratuit de 14 jours sans engagement.`,
  },
];

// Call the local ingest API route with the sample documents
async function seed() {
  const response = await fetch('http://localhost:3000/api/ingest', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ documents: SAMPLE_DOCUMENTS }),
  });

  const result = await response.json();
  console.log('Seed result:', result);
}

seed().catch(console.error);

What does this script do?

  • SAMPLE_DOCUMENTS contains 4 French business documents covering general info, delivery, returns, and pricing.
  • The seed() function calls your local /api/ingest route, which chunks the documents and stores them in Upstash Vector.
  • Each document has a source identifier (e.g. faq-general) and a category (e.g. FAQ) that get stored as metadata for later retrieval.
  • Save scripts/seed.ts.
  • Make sure your dev server is still running at localhost:3000.
  • Install tsx as a dev dependency and run the seed script by running these commands:
npm install -D tsx
npx tsx scripts/seed.ts
  • You should see output like Seed result: { success: true, chunksIngested: 4 } in your terminal.

Seeing a connection error?

If you get a fetch failed error, your dev server is likely not running. Open a separate terminal and run npm run dev first, then run the seed script again in your original terminal.

If you see an Upstash authentication error, double-check that UPSTASH_VECTOR_REST_URL and UPSTASH_VECTOR_REST_TOKEN in your .env.local are correct.

help me debug my seed script connection error

Now verify that the vectors are actually stored in your Upstash index.

  • Open the Upstash console at console.upstash.com in your browser.
  • Click on your Vector index.
  • Check the Vector Count chart or the Data Browser section. You should see vectors stored in your index (4 chunks from the sample documents).

What should I see?

The Upstash console's Data Browser shows your stored vectors with their IDs (like faq-general-0) and metadata (source, category, chunk index). The Vector Count chart should show 4 vectors have been added to your index.

✔️ Awesome, I've got everything!

Great. Double check that all three files are saved.

ⓧ I'd like to double check the full code

Here is the full scripts/seed.ts:

const SAMPLE_DOCUMENTS = [
  {
    source: 'faq-general',
    category: 'FAQ',
    content: `Notre entreprise est ouverte du lundi au vendredi de 9h a 18h. Nous sommes fermes le week-end et les jours feries. Pour nous contacter, appelez le 01 23 45 67 89 ou envoyez un email a contact@entreprise.fr. Notre adresse est 12 rue de la Paix, 75001 Paris.`,
  },
  {
    source: 'faq-livraison',
    category: 'FAQ',
    content: `La livraison standard est gratuite pour toute commande superieure a 50 euros. Le delai de livraison est de 3 a 5 jours ouvrables. Pour les commandes express, comptez 24h avec un supplement de 9.90 euros. Nous livrons en France metropolitaine, Belgique et Suisse. Le suivi de votre colis est disponible dans votre espace client.`,
  },
  {
    source: 'faq-retours',
    category: 'FAQ',
    content: `Vous disposez de 30 jours apres reception pour retourner un article. Le retour est gratuit via notre etiquette prepayee disponible dans votre espace client. Le remboursement est effectue sous 5 jours ouvrables apres reception du retour. Les articles personnalises ne sont pas retournables sauf defaut de fabrication.`,
  },
  {
    source: 'produits-services',
    category: 'Produits',
    content: `Nous proposons trois formules d'abonnement: Starter a 29 euros par mois (ideal pour les independants), Pro a 79 euros par mois (pour les petites equipes jusqu'a 5 personnes), et Enterprise sur devis (equipes de plus de 5 personnes avec support dedie). Toutes les formules incluent un essai gratuit de 14 jours sans engagement.`,
  },
];

async function seed() {
  const response = await fetch('http://localhost:3000/api/ingest', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ documents: SAMPLE_DOCUMENTS }),
  });

  const result = await response.json();
  console.log('Seed result:', result);
}

seed().catch(console.error);

Your document ingestion pipeline is complete. Next up, you'll build the RAG engine that queries these vectors and feeds relevant context to Google Gemini for intelligent, grounded answers.

Build the RAG Engine and Chat API

Your Upstash Vector index is populated with document chunks. Now you need the "brain" that retrieves the most relevant context for a customer question and generates an answer.

In this step, you will build a retrieval service that performs semantic search across your stored documents, a chat service layer that assembles a system prompt, and a streaming chat API route powered by Gemini 2.5 Flash.

In this step, get ready to:
  • Build the RAG retrieval service that performs semantic search against Upstash Vector.
  • Create the chat service layer that formats context and constructs the system prompt.
  • Wire up a streaming chat API route with source citation headers.
Build the core retrieval service

When a customer asks a question, the chatbot needs to find the most relevant document chunks stored in your vector database. This is the core of Retrieval Augmented Generation: retrieve first, then generate.

  • Create a new file lib/rag/retrieve.ts in your project. This file will query Upstash Vector with the user's message and return the top-K most similar chunks.
  • Add the following code to lib/rag/retrieve.ts:
import { Index } from '@upstash/vector';

const index = new Index({
  url: process.env.UPSTASH_VECTOR_REST_URL!,
  token: process.env.UPSTASH_VECTOR_REST_TOKEN!,
});

export interface RetrievalResult {
  content: string;
  score: number;
  metadata: {
    source: string;
    chunkIndex: number;
    category: string;
  };
}

export async function retrieveContext(
  query: string,
  topK: number = 5
): Promise<{ results: RetrievalResult[]; confident: boolean }> {
  // Query Upstash Vector with the user's raw text
  const queryResult = await index.query({
    data: query,
    topK,
    includeData: true,
    includeMetadata: true,
  });

  // Map results into a clean interface
  const results: RetrievalResult[] = queryResult.map((item) => ({
    content: (item.data as string) ?? '',
    score: item.score,
    metadata: item.metadata as RetrievalResult['metadata'],
  }));

  // Confidence check: is the top result relevant enough?
  const confident = results.length > 0 && results[0].score >= 0.7;

  return { results, confident };
}

What does this code do?

The Index instance connects to your Upstash Vector database using the environment variables you set up in Step 1.

The retrieveContext function passes the user's raw question text via the data field. Because your index uses the built-in BAAI/bge-m3 embedding model, Upstash automatically converts both the query and stored documents into vectors for similarity comparison.

The confident boolean checks whether the top result has a similarity score of 0.7 or higher. If it does not, the chatbot knows it cannot answer reliably and should suggest contacting a human.

  • Save lib/rag/retrieve.ts.
  • You will not see a change in the browser yet. The next chunk uses this retrieval function to build the full chat context.

Seeing a TypeScript error on the query call?

Make sure your @upstash/vector package is installed. Return to your terminal and run npm install @upstash/vector if needed.

Also verify that UPSTASH_VECTOR_REST_URL and UPSTASH_VECTOR_REST_TOKEN are present in your .env.local file.

Help me debug a TypeScript error in my Upstash Vector query call

✔️ Awesome, I've got everything!

Great. Make sure you have saved lib/rag/retrieve.ts before moving on.

ⓧ I'd like to double check the full code

import { Index } from '@upstash/vector';

const index = new Index({
  url: process.env.UPSTASH_VECTOR_REST_URL!,
  token: process.env.UPSTASH_VECTOR_REST_TOKEN!,
});

export interface RetrievalResult {
  content: string;
  score: number;
  metadata: {
    source: string;
    chunkIndex: number;
    category: string;
  };
}

export async function retrieveContext(
  query: string,
  topK: number = 5
): Promise<{ results: RetrievalResult[]; confident: boolean }> {
  const queryResult = await index.query({
    data: query,
    topK,
    includeData: true,
    includeMetadata: true,
  });

  const results: RetrievalResult[] = queryResult.map((item) => ({
    content: (item.data as string) ?? '',
    score: item.score,
    metadata: item.metadata as RetrievalResult['metadata'],
  }));

  const confident = results.length > 0 && results[0].score >= 0.7;

  return { results, confident };
}
Create the chat service and API route

The retrieval service finds relevant documents, but you still need a layer that turns those documents into a prompt for the LLM. This separation keeps the chat logic reusable across different delivery channels (browser, webhooks, etc.).

  • Create the folder lib/services/ if it does not already exist.
  • Create a new file lib/services/chat-service.ts and add the following code:
import { retrieveContext, RetrievalResult } from '@/lib/rag/retrieve';

export interface ChatContext {
  systemPrompt: string;
  sources: RetrievalResult[];
  confident: boolean;
}

export async function buildChatContext(
  userMessage: string,
  config: { assistantName: string; tone: string }
): Promise<ChatContext> {
  // Retrieve relevant document chunks for the user's question
  const { results, confident } = await retrieveContext(userMessage);

  // Format each chunk with a source label for citation
  const contextText = results
    .map((r, i) => `[Source ${i + 1}: ${r.metadata.source}]\n${r.content}`)
    .join('\n\n');

  // Build the system prompt with personality and retrieved context
  const systemPrompt = `Tu es ${config.assistantName}, un assistant client professionnel.\n\nTon et style: ${config.tone}\n\nInstructions:\n- Reponds UNIQUEMENT en te basant sur le contexte fourni ci-dessous.\n- Si le contexte ne contient pas l'information, dis clairement que tu ne disposes pas de cette information et suggere de contacter un humain.\n- Cite tes sources en mentionnant [Source X] quand tu utilises une information.\n- Sois concis, professionnel et utile.\n\nContexte documentaire:\n${contextText || 'Aucun document pertinent trouve.'}`;

  return { systemPrompt, sources: results, confident };
}

What does this code do?

The buildChatContext function is the bridge between retrieval and generation. It calls retrieveContext with the user's message, then formats the returned chunks into labeled blocks like [Source 1: faq-general].

The system prompt instructs the LLM to only use the provided context and to cite sources by number. This prevents hallucination and makes answers traceable. The config parameter lets you swap the assistant's name and tone without changing code.

  • Save lib/services/chat-service.ts.

Import path error on @/lib/rag/retrieve?

The @/ alias is set up by create-next-app in your tsconfig.json. Make sure you have a paths entry mapping @/* to ./* in your tsconfig.json.

Help me fix a path alias import error in my Next.js project

Now create the API route that wires everything together. The Vercel AI SDK provides streamText to stream the LLM response token by token back to the browser.

  • Create the folder app/api/chat/ inside your project.
  • Create a new file app/api/chat/route.ts and add the following code:
import { streamText } from 'ai';
import { google } from '@ai-sdk/google';
import { buildChatContext } from '@/lib/services/chat-service';

export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  // Extract the latest user message for retrieval
  const lastUserMessage = messages[messages.length - 1]?.content ?? '';

  // Build context: retrieve documents + construct system prompt
  const { systemPrompt, sources, confident } = await buildChatContext(
    lastUserMessage,
    {
      assistantName: 'Assistant IA',
      tone: 'Professionnel, chaleureux et direct. Tu tutoies le client.',
    }
  );

  // Stream the LLM response using Gemini 2.5 Flash
  const result = streamText({
    model: google('gemini-2.5-flash'),
    system: systemPrompt,
    messages,
  });

  // Return streaming response with source metadata in headers
  return result.toDataStreamResponse({
    getResponseHeaders: () => ({
      'X-Sources': JSON.stringify(
        sources.map((s) => ({
          source: s.metadata.source,
          score: Math.round(s.score * 100) / 100,
        }))
      ),
      'X-Confident': String(confident),
    }),
  });
}

What does this code do?

The route extracts the latest user message from the conversation history and passes it to buildChatContext. This retrieves relevant documents and returns a system prompt containing the context.

The streamText function calls gemini-2.5-flash with the system prompt and full message history, streaming tokens as they are generated.

The toDataStreamResponse method converts the stream into an HTTP response. The getResponseHeaders callback attaches X-Sources (which documents were used) and X-Confident (whether retrieval was reliable) as custom headers. The UI will use these for source citation display.

  • Save app/api/chat/route.ts.
  • Your dev server should still be running. You will test this route in the next substep.

Seeing a module not found error?

Verify that the ai and @ai-sdk/google packages are installed. Check your package.json for both entries. If either is missing, install them by running:

npm install ai @ai-sdk/google

Help me fix a module not found error in my chat API route

✔️ Awesome, I've got everything!

Great. Make sure you have saved both lib/services/chat-service.ts and app/api/chat/route.ts.

ⓧ I'd like to double check the full code

import { retrieveContext, RetrievalResult } from '@/lib/rag/retrieve';

export interface ChatContext {
  systemPrompt: string;
  sources: RetrievalResult[];
  confident: boolean;
}

export async function buildChatContext(
  userMessage: string,
  config: { assistantName: string; tone: string }
): Promise<ChatContext> {
  const { results, confident } = await retrieveContext(userMessage);

  const contextText = results
    .map((r, i) => `[Source ${i + 1}: ${r.metadata.source}]\n${r.content}`)
    .join('\n\n');

  const systemPrompt = `Tu es ${config.assistantName}, un assistant client professionnel.\n\nTon et style: ${config.tone}\n\nInstructions:\n- Reponds UNIQUEMENT en te basant sur le contexte fourni ci-dessous.\n- Si le contexte ne contient pas l'information, dis clairement que tu ne disposes pas de cette information et suggere de contacter un humain.\n- Cite tes sources en mentionnant [Source X] quand tu utilises une information.\n- Sois concis, professionnel et utile.\n\nContexte documentaire:\n${contextText || 'Aucun document pertinent trouve.'}`;

  return { systemPrompt, sources: results, confident };
}
import { streamText } from 'ai';
import { google } from '@ai-sdk/google';
import { buildChatContext } from '@/lib/services/chat-service';

export const maxDuration = 30;

export async function POST(req: Request) {
  const { messages } = await req.json();

  const lastUserMessage = messages[messages.length - 1]?.content ?? '';

  const { systemPrompt, sources, confident } = await buildChatContext(
    lastUserMessage,
    {
      assistantName: 'Assistant IA',
      tone: 'Professionnel, chaleureux et direct. Tu tutoies le client.',
    }
  );

  const result = streamText({
    model: google('gemini-2.5-flash'),
    system: systemPrompt,
    messages,
  });

  return result.toDataStreamResponse({
    getResponseHeaders: () => ({
      'X-Sources': JSON.stringify(
        sources.map((s) => ({
          source: s.metadata.source,
          score: Math.round(s.score * 100) / 100,
        }))
      ),
      'X-Confident': String(confident),
    }),
  });
}
Test source citations in the API response

The chat route attaches source metadata as custom HTTP headers. This lets the frontend know which documents informed the answer without embedding that data in the streamed text itself.

How do the source citation headers work?

The X-Sources header contains a JSON array of objects, each with the document source name and its similarity score. The frontend will parse this to show the user which FAQ or product page was used.

The X-Confident header is either true or false. When false, the UI can warn the customer that the answer may be incomplete. This is the foundation for the human handover feature in the Secret Mission.

  • Test the full RAG pipeline by running this curl command in a new terminal tab:
curl -X POST http://localhost:3000/api/chat \
  -H "Content-Type: application/json" \
  -d '{"messages":[{"role":"user","content":"Quels sont vos horaires ?"}]}' \
  -i
  • Look at the response headers near the top of the output. You should see X-Sources containing a JSON array with faq-general and a score close to 1.0.
  • You should also see X-Confident: true because the question matches your seeded FAQ data well.
  • Below the headers, streaming data will appear. This is the AI SDK's data stream format containing the LLM's response tokens.

Getting a 500 error or empty response?

Check that your dev server is still running at localhost:3000. If it crashed, restart it with npm run dev.

Verify your GOOGLE_GENERATIVE_AI_API_KEY is correct in .env.local. An invalid key will cause the Gemini call to fail. You may need to restart the dev server after editing .env.local.

If you see an Upstash error, confirm your Vector index was seeded in Step 2 by checking the Upstash console for stored vectors.

Help me debug a 500 error from my chat API route

Your RAG engine is fully operational. The retrieval, prompt construction, and streaming generation are all working end to end. Next up, you will build a professional chat interface that streams these responses in real time and displays source citations to the user.

Build the Professional Chat Interface

Your RAG engine is fully functional. It can ingest documents, store them as vectors, retrieve relevant chunks, and generate streaming responses through an API route.

Now you need a way for users to actually interact with it. In this step, you will build a professional chat interface that connects to your /api/chat route and displays streaming responses in real time.

In this step, get ready to:
  • Build a reusable ChatMessage component with user/assistant styling and source citations.
  • Create the main chat interface with real-time streaming and suggested questions.
  • Wire up the home page and test your full RAG chatbot end to end.
Add the shadcn/ui components

The chat interface uses four shadcn/ui components for a polished SaaS look: Card, Button, Input, and ScrollArea.

  • Install the required shadcn/ui components by running this command:
npx shadcn@latest add card button input scroll-area

What does this command do?

The shadcn CLI generates the component source code directly into your components/ui/ folder. Unlike traditional component libraries, you own and can customize these files.

  • Confirm the components were added by checking your components/ui/ folder. You should see card.tsx, button.tsx, input.tsx, and scroll-area.tsx listed.

Command not found or permission error?

Make sure you are running the command from your project root folder (the one containing package.json). If npx prompts you to install the shadcn package, accept it.

Help me troubleshoot this shadcn/ui install error

Create the ChatMessage component

Each message in the chat needs different styling depending on who sent it. User messages align right with a primary colour background. Assistant messages align left with a muted background and include a toggle to reveal source documents.

  • Create a new folder components/chat/ inside your project.
  • Create a new file components/chat/chat-message.tsx and paste the following code:
'use client';

import { Button } from '@/components/ui/button';

interface ChatMessageProps {
  role: 'user' | 'assistant' | 'system';
  content: string;
  showSources: boolean;
  onToggleSources: () => void;
}

export function ChatMessage({
  role,
  content,
  showSources,
  onToggleSources,
}: ChatMessageProps) {
  return (
    <div
      className={`flex ${role === 'user' ? 'justify-end' : 'justify-start'}`}
    >
      <div
        className={`max-w-[80%] rounded-lg p-3 ${
          role === 'user'
            ? 'bg-primary text-primary-foreground'
            : 'bg-muted'
        }`}
      >
        <p className="text-sm whitespace-pre-wrap">{content}</p>

        {role === 'assistant' && (
          <div className="mt-2">
            <Button
              variant="ghost"
              size="sm"
              className="text-xs h-6 px-2"
              onClick={onToggleSources}
            >
              {showSources ? 'Masquer sources' : 'Voir sources'}
            </Button>

            {showSources && (
              <div className="mt-2 p-2 bg-background rounded text-xs text-muted-foreground">
                <p>Sources utilisees pour cette reponse.</p>
              </div>
            )}
          </div>
        )}
      </div>
    </div>
  );
}

What does this component do?

  • The role prop determines alignment and colours. User messages get bg-primary (dark). Assistant messages get bg-muted (light grey).
  • The showSources prop controls whether the source panel is visible. The parent component manages this state.
  • Only assistant messages show the "Voir sources" button. This lets users verify which documents informed the response.
  • Save components/chat/chat-message.tsx.

You will not see a change yet because nothing imports this component. The next chunk connects it to the main chat interface.

Build the chat interface with streaming

The Vercel AI SDK provides a useChat hook that handles the entire streaming lifecycle: sending messages, receiving tokens, and managing conversation state. Your interface just needs to render the data the hook provides.

  • Create a new file components/chat/chat-interface.tsx and paste the following code:
'use client';

import { useChat } from 'ai/react';
import { useState } from 'react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { Card } from '@/components/ui/card';
import { ScrollArea } from '@/components/ui/scroll-area';
import { ChatMessage } from '@/components/chat/chat-message';

export function ChatInterface() {
  const [showSources, setShowSources] = useState<Record<string, boolean>>({});

  const { messages, input, handleInputChange, handleSubmit, isLoading } =
    useChat({
      api: '/api/chat',
    });

  const suggestedQuestions = [
    'Quels sont vos horaires ?',
    'Comment fonctionne la livraison ?',
    'Quelles sont vos formules ?',
  ];

  return (
    <Card className="flex h-[600px] w-full max-w-2xl flex-col mx-auto">
      <div className="border-b p-4">
        <h2 className="text-lg font-semibold">Assistant IA</h2>
        <p className="text-sm text-muted-foreground">
          Support client automatise
        </p>
      </div>

      <ScrollArea className="flex-1 p-4">
        {messages.length === 0 && (
          <div className="flex flex-col items-center justify-center h-full gap-4">
            <p className="text-muted-foreground text-center">
              Bonjour ! Comment puis-je vous aider ?
            </p>
            <div className="flex flex-wrap gap-2 justify-center">
              {suggestedQuestions.map((q) => (
                <Button
                  key={q}
                  variant="outline"
                  size="sm"
                  onClick={() => {
                    handleInputChange({
                      target: { value: q },
                    } as React.ChangeEvent<HTMLInputElement>);
                  }}
                >
                  {q}
                </Button>
              ))}
            </div>
          </div>
        )}

        <div className="space-y-4">
          {messages.map((message) => (
            <ChatMessage
              key={message.id}
              role={message.role}
              content={message.content}
              showSources={showSources[message.id] ?? false}
              onToggleSources={() =>
                setShowSources((prev) => ({
                  ...prev,
                  [message.id]: !prev[message.id],
                }))
              }
            />
          ))}
        </div>
      </ScrollArea>

      <form onSubmit={handleSubmit} className="border-t p-4 flex gap-2">
        <Input
          value={input}
          onChange={handleInputChange}
          placeholder="Posez votre question..."
          disabled={isLoading}
        />
        <Button type="submit" disabled={isLoading}>
          Envoyer
        </Button>
      </form>
    </Card>
  );
}

What does this component do?

  • useChat({ api: '/api/chat' }) connects to your streaming API route. It returns the messages array, the current input value, and handlers for form submission.
  • The showSources state tracks which messages have their sources panel expanded, keyed by message ID.
  • When messages is empty, the welcome state renders with three suggested questions that populate the input field on click.
  • The form at the bottom disables during loading to prevent duplicate submissions.
  • Save components/chat/chat-interface.tsx.

The component is ready but nothing renders it yet. The next chunk wires it into your home page.

Wire up the home page and test your chatbot

The default app/page.tsx created by Next.js contains placeholder content. You will replace it with a single centered ChatInterface component.

  • Open app/page.tsx and replace its entire contents with:
import { ChatInterface } from '@/components/chat/chat-interface';

export default function Home() {
  return (
    <main className="flex min-h-screen items-center justify-center p-4 bg-background">
      <ChatInterface />
    </main>
  );
}

What does this page do?

This is a server component that renders the ChatInterface client component. The main element uses Tailwind's flexbox utilities to centre the chat card both horizontally and vertically on the screen.

  • Save app/page.tsx.
  • Return to your browser at http://localhost:3000. You should see a card with the header "Assistant IA", three suggested question buttons, and an input field at the bottom.
  • Click one of the suggested questions (for example, "Quels sont vos horaires ?"). The question populates the input field.
  • Click Envoyer to send the message.
  • Watch the assistant's response stream in token by token. Once complete, click Voir sources below the response to confirm the source citation panel expands.

Not seeing a response or getting an error?

  • Check that your dev server is still running (npm run dev in your terminal).
  • Open the browser console (F12) and look for network errors on the /api/chat request. A 500 error usually means your .env.local keys are missing or incorrect.
  • Make sure you ran the seed script in the previous step. Without ingested documents, the assistant has no context to answer from.

Help me debug my chat streaming connection

✔️ Awesome, I've got everything!

Great. Double check you have saved all three files before moving on.

ⓧ I'd like to double check the full code

Here is the full code for each file you created in this step.

components/chat/chat-message.tsx

'use client';

import { Button } from '@/components/ui/button';

interface ChatMessageProps {
  role: 'user' | 'assistant' | 'system';
  content: string;
  showSources: boolean;
  onToggleSources: () => void;
}

export function ChatMessage({
  role,
  content,
  showSources,
  onToggleSources,
}: ChatMessageProps) {
  return (
    <div
      className={`flex ${role === 'user' ? 'justify-end' : 'justify-start'}`}
    >
      <div
        className={`max-w-[80%] rounded-lg p-3 ${
          role === 'user'
            ? 'bg-primary text-primary-foreground'
            : 'bg-muted'
        }`}
      >
        <p className="text-sm whitespace-pre-wrap">{content}</p>

        {role === 'assistant' && (
          <div className="mt-2">
            <Button
              variant="ghost"
              size="sm"
              className="text-xs h-6 px-2"
              onClick={onToggleSources}
            >
              {showSources ? 'Masquer sources' : 'Voir sources'}
            </Button>

            {showSources && (
              <div className="mt-2 p-2 bg-background rounded text-xs text-muted-foreground">
                <p>Sources utilisees pour cette reponse.</p>
              </div>
            )}
          </div>
        )}
      </div>
    </div>
  );
}

components/chat/chat-interface.tsx

'use client';

import { useChat } from 'ai/react';
import { useState } from 'react';
import { Button } from '@/components/ui/button';
import { Input } from '@/components/ui/input';
import { Card } from '@/components/ui/card';
import { ScrollArea } from '@/components/ui/scroll-area';
import { ChatMessage } from '@/components/chat/chat-message';

export function ChatInterface() {
  const [showSources, setShowSources] = useState<Record<string, boolean>>({});

  const { messages, input, handleInputChange, handleSubmit, isLoading } =
    useChat({
      api: '/api/chat',
    });

  const suggestedQuestions = [
    'Quels sont vos horaires ?',
    'Comment fonctionne la livraison ?',
    'Quelles sont vos formules ?',
  ];

  return (
    <Card className="flex h-[600px] w-full max-w-2xl flex-col mx-auto">
      <div className="border-b p-4">
        <h2 className="text-lg font-semibold">Assistant IA</h2>
        <p className="text-sm text-muted-foreground">
          Support client automatise
        </p>
      </div>

      <ScrollArea className="flex-1 p-4">
        {messages.length === 0 && (
          <div className="flex flex-col items-center justify-center h-full gap-4">
            <p className="text-muted-foreground text-center">
              Bonjour ! Comment puis-je vous aider ?
            </p>
            <div className="flex flex-wrap gap-2 justify-center">
              {suggestedQuestions.map((q) => (
                <Button
                  key={q}
                  variant="outline"
                  size="sm"
                  onClick={() => {
                    handleInputChange({
                      target: { value: q },
                    } as React.ChangeEvent<HTMLInputElement>);
                  }}
                >
                  {q}
                </Button>
              ))}
            </div>
          </div>
        )}

        <div className="space-y-4">
          {messages.map((message) => (
            <ChatMessage
              key={message.id}
              role={message.role}
              content={message.content}
              showSources={showSources[message.id] ?? false}
              onToggleSources={() =>
                setShowSources((prev) => ({
                  ...prev,
                  [message.id]: !prev[message.id],
                }))
              }
            />
          ))}
        </div>
      </ScrollArea>

      <form onSubmit={handleSubmit} className="border-t p-4 flex gap-2">
        <Input
          value={input}
          onChange={handleInputChange}
          placeholder="Posez votre question..."
          disabled={isLoading}
        />
        <Button type="submit" disabled={isLoading}>
          Envoyer
        </Button>
      </form>
    </Card>
  );
}

app/page.tsx

import { ChatInterface } from '@/components/chat/chat-interface';

export default function Home() {
  return (
    <main className="flex min-h-screen items-center justify-center p-4 bg-background">
      <ChatInterface />
    </main>
  );
}

Your RAG chatbot is fully functional. Users can ask questions, see answers stream in real time, and verify which documents the assistant used. Next up, you will push this further with a Secret Mission that adds intelligent human handover detection.

Secret mission

Add Human Handover Detection

A great AI assistant knows its limits. In this bonus challenge, you'll add confidence scoring to the RAG pipeline so the chatbot detects when it cannot answer reliably. When confidence is low, the UI shows an orange escalation banner and a "Transfer to Human" button that logs the conversation for review.

Clean Up Your Resources

Clean Up Your Resources

This project uses free-tier services with no ongoing costs. Your Upstash Vector index persists indefinitely on the free plan, and your Google AI Studio API key has no idle charges.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Upstash Vector index (free plan, stores your document embeddings).
  • Google AI Studio API key (free tier, stored in .env.local).
  • Local Next.js project folder on your Desktop.

Keep everything running

No action needed. Choose this if you plan to keep building on this project or revisiting the chatbot.

  • Your Upstash Vector index stays on the free plan with no idle costs (10K queries/day limit).
  • Your Google AI Studio API key remains active on the free tier with rate limits.
  • Your local project folder stays on your Desktop.

Pause - I'll come back to this later

Shut down running processes to free up memory. Your files and cloud resources stay intact for when you return.

  • Stop the dev server by pressing Ctrl+C in the terminal where npm run dev is running.
  • Your Upstash Vector index and Google AI Studio API key persist at no cost on free tier.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each step below to fully clean up.

  • Stop the dev server by pressing Ctrl+C in the terminal.
  • Delete your Upstash Vector index: go to the Upstash Console at console.upstash.com, click the Vector tab, select your index, and look for the delete option in the index details page.
  • Delete your Google AI Studio API key: go to aistudio.google.com/apikey, find the key you created for this project, and delete it from the API Keys page.
  • Delete your local project folder by running these commands:
cd ~/Desktop
rm -rf assistant-client-intelligent

On Windows?

Use this command instead to remove the project folder:

cd ~\Desktop
rmdir /s /q assistant-client-intelligent

Nice Work!

Nice Work!

You just built a production-ready RAG chatbot from scratch. Your assistant ingests business documents, retrieves relevant context via semantic search, and streams professional responses with source citations.

You've learned how to:

  • Build a complete RAG pipeline with document chunking, vector embeddings via Upstash Vector, and semantic retrieval.
  • Create a streaming chat API with Google Gemini via Vercel AI SDK, fully separated from the UI layer for multi-channel delivery.
  • Deliver a professional SaaS-quality chat interface with real-time streaming, source citations, and suggested questions using shadcn/ui.
  • Secret Mission: Implemented intelligent human handover detection based on retrieval confidence scores.

Ready to quiz yourself?