Build a Document Q&A API with RAG

Build a NestJS REST API that answers questions about uploaded docs using Gemini.

Introduction

30 Second Summary

Think about how many times you've searched through a long document trying to find one specific detail. Now imagine having an API that reads your documents for you and answers your questions in plain English.

In this project, you will build a Document Q&A API using NestJS and the RAG pattern. You'll upload text documents, store them as vector embeddings in ChromaDB, and ask natural-language questions that get answered by Google Gemini.

What You'll Build

You'll have a locally-running API where you POST a document, then POST a question, and watch as the server retrieves the most relevant passages and generates a grounded answer in seconds.

By the end of this project, you'll have:

  • Upload documents and see them split into chunks, embedded as vectors, and stored in a running ChromaDB instance.
  • Ask natural-language questions and get back AI-generated answers with the source passages included, all from a single API call.
  • A modular NestJS codebase structured with feature-based modules, shared services, DTOs, and validation the way senior engineers organize production backends.
  • Secret Mission: Add metadata filtering so you can scope questions to specific documents by name.

Are there any prerequisites?

This project assumes intermediate TypeScript knowledge, basic REST API concepts, and familiarity with npm. You do not need any prior experience with AI, embeddings, or vector databases.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

You're creating a backend service that combines traditional REST API design with AI-powered document understanding.

Set Up the Project and Infrastructure

To build your Document Q&A API, you need two pieces of infrastructure up and running before you write any feature code.

First, a NestJS project scaffold that gives you a working server. Second, a ChromaDB vector database running in Docker that will store your document embeddings. Getting both running now means every subsequent step produces visible results immediately.

In this step, get ready to:
  • Scaffold the NestJS project and confirm it serves a response.
  • Start ChromaDB in a Docker container.
  • Install project dependencies and configure the Gemini API key.
Scaffold the project and start the dev server
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Scaffold a new NestJS project by running this command:
nest new doc-qa-api --strict --package-manager npm

What does this command do?

  • nest new creates a complete project directory with modules, controllers, services, and TypeScript configuration.
  • --strict enables strict TypeScript checks like strictNullChecks and noImplicitAny. This catches bugs at compile time instead of at runtime.
  • --package-manager npm tells NestJS to use npm and skip the interactive package manager prompt.

You will see NestJS generate files and install packages. This takes about 30 seconds.

  • Navigate into the project directory by running:
cd doc-qa-api

About the next command

The next command starts a development server that takes over your terminal. You will not be able to type further commands in this terminal while the server runs. You will open a new terminal tab for the remaining commands in this step.

  • Start the development server by running:
npm run start:dev

Before you check your browser, what do you think you will see at http://localhost:3000?

  • Open your browser and navigate to http://localhost:3000.

You should see the text "Hello World!" in your browser. This confirms NestJS scaffolded your project correctly and the server is running.

Not seeing Hello World?

  • Check your terminal for error messages. If you see a port conflict, another process is using port 3000. Stop it or change the port in src/main.ts.
  • Make sure you ran the command from inside the doc-qa-api folder, not from ~/Desktop.
  • If the terminal shows the server started but the browser says "This site can't be reached", try http://127.0.0.1:3000 instead.

Help me troubleshoot my NestJS dev server not responding on localhost:3000.

Start ChromaDB in Docker

ChromaDB is a vector database that stores embeddings and performs similarity search. You will run it as a Docker container so it stays isolated from your system.

  • Open a new terminal tab by pressing Cmd+T (macOS) or Ctrl+Shift+T (Windows/Linux).
  • Navigate to your project folder by running:
cd ~/Desktop/doc-qa-api
  • Pull and run the ChromaDB container by running this command:
docker run -d -v ./chroma-data:/data -p 8000:8000 --name chromadb chromadb/chroma

What does this command do?

  • -d runs the container in detached mode so it does not take over your terminal.
  • -v ./chroma-data:/data mounts a local folder for persistent storage. Your vector data survives container restarts.
  • -p 8000:8000 maps port 8000 on your machine to port 8000 in the container.
  • --name chromadb gives the container a friendly name so you can stop and start it easily.
  • Confirm the container is running by running:
docker ps

You should see a row with chromadb/chroma in the IMAGE column and 0.0.0.0:8000->8000/tcp in the PORTS column.

  • Verify ChromaDB is responding by opening http://localhost:8000/api/v2/heartbeat in your browser.

You should see a JSON response containing a nanosecond heartbeat field with a large integer. This means ChromaDB is alive and accepting requests.

Docker command failed?

  • If you see "Cannot connect to the Docker daemon", make sure Docker Desktop is running. Open it from your applications and wait for it to finish starting up.
  • If you see "name chromadb is already in use", a previous container exists. Remove it by running docker rm -f chromadb and then try the run command again.
  • If localhost:8000 shows nothing in the browser, wait a few seconds for the container to finish starting and refresh.

Help me get my ChromaDB Docker container running.

Install dependencies and create the environment file

Your NestJS project needs several packages to connect to Gemini for AI capabilities, talk to ChromaDB, and validate incoming requests.

  • Install all project dependencies by running this command in your second terminal:
npm install @google/genai chromadb @nestjs/config class-validator class-transformer

What are these packages?

  • @google/genai is Google's unified SDK for calling the Gemini API (embeddings and text generation).
  • chromadb is the TypeScript client for your ChromaDB vector database.
  • @nestjs/config loads environment variables from your .env file into NestJS services.
  • class-validator and class-transformer enable request validation using decorators on your DTOs.

You will see npm download and install the packages. Your dev server in the first terminal will automatically restart when it detects the change.

  • Create a .env file in the root of your doc-qa-api folder by running:
echo "GEMINI_API_KEY=your-api-key-here" > .env
  • Open .env in your editor and replace your-api-key-here with the Gemini API key you created earlier.
  • Save the file.

Why a .env file?

Storing your API key in a .env file keeps secrets out of your source code. The @nestjs/config package reads this file automatically so your services can access the key without hardcoding it.

✔️ Awesome, I've got everything!

Great. Make sure you saved the .env file with your actual API key in place of the placeholder.

ⓧ I'd like to double check the full code

Your .env file should contain exactly one line (with your real key substituted):

GEMINI_API_KEY=your-api-key-here

Now let's confirm everything is working together. Before you check, do you expect your dev server to still be responding after the package install?

  • Refresh http://localhost:3000 in your browser.

You should still see "Hello World!" in the browser. The dev server detected your new packages and restarted cleanly.

  • In your second terminal, confirm Docker is still running by running:
docker ps

You should see the chromadb container listed with status "Up". Both your NestJS server and ChromaDB are ready to go.

Your project scaffold is running and your vector database is live. Next up, you will build the document ingestion endpoint that accepts text, splits it into chunks, and stores it in ChromaDB.

Build the Document Ingestion Endpoint

In Step 1, you got your NestJS server running and ChromaDB ready in Docker. Now it's time to build something real.

Before your API can answer questions about documents, it needs a way to accept and store them. In this step, you'll build the "write path" of a RAG pipeline: an endpoint that accepts text, splits it into manageable chunks, and stores those chunks in ChromaDB.

In this step, get ready to:
  • Generate a documents module using the NestJS CLI.
  • Create a validated DTO and wire up global validation.
  • Implement chunking and ChromaDB storage logic.
Create the documents module with NestJS CLI

NestJS organizes code into feature modules. Each module groups a controller, service, and DTOs for one domain concern.

  • Generate the documents resource by running this command:
nest generate resource documents --no-spec

What will this command ask?

The CLI will prompt you with two questions:

  • "What transport layer do you use?" Select REST API.
  • "Would you like to generate CRUD entry points?" Select Yes.

You should see output listing the created files and an update to app.module.ts.

What did this generate?

The CLI created a full CRUD scaffold inside src/documents/: a module, controller, service, DTOs, and an entities folder.

It also auto-imported DocumentsModule into your app.module.ts. You'll trim the generated code down to just what your ingestion endpoint needs.

  • Replace the contents of src/documents/dto/create-document.dto.ts with this validated DTO:
import { IsNotEmpty, IsString } from 'class-validator';

export class CreateDocumentDto {
  @IsString()
  @IsNotEmpty()
  title: string;

  @IsString()
  @IsNotEmpty()
  content: string;
}

What does this code do?

The @IsString() and @IsNotEmpty() decorators come from class-validator. When a request arrives, NestJS checks these rules automatically and rejects invalid payloads with a 400 error before your service code runs.

  • Now update src/main.ts to enable validation globally. Replace the entire file with:
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';
import { ValidationPipe } from '@nestjs/common';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  // Reject unknown fields and auto-transform payloads to DTO instances
  app.useGlobalPipes(new ValidationPipe({ whitelist: true, transform: true }));
  await app.listen(process.env.PORT ?? 3000);
}
bootstrap();

What does the ValidationPipe do?

The whitelist option strips any fields not declared in the DTO. The transform option converts the raw JSON body into a typed DTO class instance so decorators like @IsString() can run.

  • Save both files.
  • Test that validation works by sending an empty POST request. Run this command in a new terminal:
curl -X POST http://localhost:3000/documents -H "Content-Type: application/json" -d '{}'

You should see a 400 response with validation error messages listing title and content as required fields.

Getting a 404 instead of 400?

Make sure your dev server is still running (npm run start:dev in the original terminal). The nest generate resource command auto-imports the module, so the route should exist.

If you see a compilation error in the dev server terminal, check that src/documents/dto/create-document.dto.ts has the exact code shown above.

Implement the chunking and storage logic

Your endpoint now validates incoming data. Next, you'll wire it up to actually store documents in ChromaDB as chunked text.

  • Replace the contents of src/app.module.ts to add ConfigModule and clean up the default scaffold:
import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import { DocumentsModule } from './documents/documents.module';

@Module({
  imports: [
    ConfigModule.forRoot({ isGlobal: true }),
    DocumentsModule,
  ],
})
export class AppModule {}

What does ConfigModule.forRoot do?

This loads your .env file and makes its values available throughout the app via ConfigService. The isGlobal: true option means you don't need to import ConfigModule in every feature module.

  • Replace the contents of src/documents/documents.controller.ts with a simplified controller that only has the POST endpoint:
import { Controller, Post, Body } from '@nestjs/common';
import { DocumentsService } from './documents.service';
import { CreateDocumentDto } from './dto/create-document.dto';

@Controller('documents')
export class DocumentsController {
  constructor(private readonly documentsService: DocumentsService) {}

  @Post()
  async create(@Body() createDocumentDto: CreateDocumentDto) {
    return this.documentsService.create(createDocumentDto);
  }
}

What does this controller do?

The @Controller('documents') decorator registers this at the /documents path. The @Post() method receives the validated DTO and delegates all work to the service. NestJS returns 201 for POST by default.

Your dev server will show compilation errors until you update the service next. That is expected.

  • Replace the contents of src/documents/documents.service.ts with this code that sets up the ChromaDB connection and text chunking:
import { Injectable } from '@nestjs/common';
import { ChromaClient } from 'chromadb';
import { CreateDocumentDto } from './dto/create-document.dto';

@Injectable()
export class DocumentsService {
  private readonly chroma: ChromaClient;

  constructor() {
    // Connect to the local ChromaDB container
    this.chroma = new ChromaClient({ host: 'localhost', port: 8000 });
  }

  // Split text into overlapping chunks for better retrieval
  private chunkText(text: string, chunkSize = 500, overlap = 50): string[] {
    const chunks: string[] = [];
    let start = 0;
    while (start < text.length) {
      const end = Math.min(start + chunkSize, text.length);
      chunks.push(text.slice(start, end));
      start += chunkSize - overlap;
    }
    return chunks;
  }
  • Now add the create method directly below the chunkText method, before the closing } of the class:
  async create(dto: CreateDocumentDto) {
    const chunks = this.chunkText(dto.content);
    const collection = await this.chroma.getOrCreateCollection({ name: 'documents' });

    // Generate unique IDs from the document title and chunk index
    const ids = chunks.map((_, i) => `${dto.title.replace(/\s+/g, '-').toLowerCase()}-chunk-${i}`);
    const metadatas = chunks.map((_, i) => ({ title: dto.title, chunkIndex: i }));

    // Store chunks with their metadata in ChromaDB
    await collection.add({ ids, documents: chunks, metadatas });

    return { message: 'Document ingested successfully', title: dto.title, chunksStored: chunks.length };
  }
}

What does this service do?

The chunkText method splits document content into ~500-character pieces with a 50-character overlap. The overlap ensures sentences at chunk boundaries aren't lost during retrieval.

The create method uses getOrCreateCollection to ensure the documents collection exists in ChromaDB, then stores each chunk with a unique ID and metadata (the original title and chunk position).

  • Save documents.service.ts.
  • Check your dev server terminal. It should recompile successfully with no errors.

Seeing a compilation error?

Make sure you pasted BOTH code blocks into the same file. The first block defines the class and chunkText method (without a closing }), and the second block adds create and closes the class.

Also confirm that src/documents/documents.module.ts imports DocumentsService and DocumentsController. The generated module should already have these.

✔️ Awesome, I've got everything!

Great. Make sure you've saved all files in src/documents/ and that your dev server shows no errors.

ⓧ I'd like to double check the full code

Here is the full documents.service.ts for comparison. Scroll up and confirm your file has both the chunkText method and the create method inside the class, in that order.

Also confirm src/documents/documents.controller.ts only has the single @Post() endpoint importing CreateDocumentDto, and src/app.module.ts imports ConfigModule and DocumentsModule only.

Test the full ingestion pipeline

Everything is wired up. Before you run the test, what do you think will happen when you POST a document with a long content field?

  • Send a sample document to your API by running:
curl -X POST http://localhost:3000/documents \
  -H "Content-Type: application/json" \
  -d '{"title": "NestJS Guide", "content": "NestJS is a progressive Node.js framework for building efficient and scalable server-side applications. It uses TypeScript by default and combines elements of OOP, FP, and FRP. Under the hood, NestJS uses Express as its HTTP server framework, though it can be configured to use Fastify instead. The framework provides an out-of-the-box application architecture that allows developers and teams to create highly testable, scalable, loosely coupled, and easily maintainable applications. The architecture is heavily inspired by Angular."}'

You should see a 201 response with JSON containing the title and the number of chunks stored:

{
  "message": "Document ingested successfully",
  "title": "NestJS Guide",
  "chunksStored": 2
}

Why 2 chunks?

The sample text is about 530 characters. With a 500-character chunk size and 50-character overlap, it splits into 2 chunks. Shorter documents (under 500 characters) produce a single chunk.

  • Now confirm ChromaDB received the data by visiting this URL in your browser:
http://localhost:8000/api/v2/collections

You should see a JSON response that includes your documents collection.

Getting a connection error from curl?

Confirm your Docker container is still running with docker ps. If the chromadb container isn't listed, restart it with docker start chromadb.

If curl returns an empty object or error for the NestJS endpoint, check your dev server terminal for any runtime exceptions.

Your ingestion pipeline is working. Documents go in, get chunked, and land in ChromaDB. Next up, you'll add Gemini embeddings so that when you query later, the vector search actually returns meaningful results.

Add Gemini Embeddings to the Pipeline

Your document ingestion pipeline stores text in ChromaDB, but it relies on ChromaDB's default embedding model. When you build the query endpoint in the next step, you will embed questions using Gemini.

If your documents and questions use different embedding models, they live in different vector spaces and retrieval becomes meaningless. In this step, you will create a dedicated AI service that generates Gemini embeddings, then integrate it into your ingestion pipeline so both sides share the same model.

In this step, get ready to:
  • Create a shared AI module with an embedding method powered by Gemini.
  • Integrate the AI service into your document ingestion so every chunk gets a Gemini embedding.
  • Confirm embeddings are stored by checking log output for the expected dimension.
Create a dedicated Gemini service module

NestJS encourages isolating each concern into its own module. The AI logic belongs in a shared module that any feature module can import.

  • Generate the AI module and service by running these two commands:
nest generate module ai
nest generate service ai --no-spec

What did the CLI just do?

The nest generate module ai command created src/ai/ai.module.ts and automatically registered it in AppModule's imports array.

The nest generate service ai --no-spec command created src/ai/ai.service.ts and added it to AiModule's providers. The --no-spec flag skips generating a test file.

  • Confirm the files were created by running:
ls src/ai/

You should see ai.module.ts and ai.service.ts listed.

The generated module needs one change: it must export AiService so other modules can inject it.

  • Open src/ai/ai.module.ts and find this code:
@Module({
  providers: [AiService],
})
  • Add an exports array so the decorator now looks like this:
@Module({
  providers: [AiService],
  exports: [AiService],
})
  • Save src/ai/ai.module.ts.

Now replace the generated service scaffold with the full Gemini embedding implementation.

  • Open src/ai/ai.service.ts and replace its entire contents with the following:
import { Injectable } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { GoogleGenAI } from '@google/genai';

@Injectable()
export class AiService {
  private readonly ai: GoogleGenAI;

  constructor(private readonly configService: ConfigService) {
    // Initialize the Gemini SDK with our API key from .env
    this.ai = new GoogleGenAI({
      apiKey: this.configService.get<string>('GEMINI_API_KEY'),
    });
  }

  async embedTexts(texts: string[]): Promise<number[][]> {
    // Generate one embedding per text to stay within free tier rate limits
    const embeddings: number[][] = [];
    for (const text of texts) {
      const response = await this.ai.models.embedContent({
        model: 'gemini-embedding-001',
        contents: text,
      });
      embeddings.push(response.embedding.values);
    }
    return embeddings;
  }
}

What does this code do?

  • The constructor uses dependency injection to pull ConfigService, which reads GEMINI_API_KEY from your .env file.
  • The embedTexts method takes an array of strings and returns an array of number arrays (vectors).
  • It loops through each text individually because the Gemini free tier has per-minute rate limits.
  • The model gemini-embedding-001 produces 768-dimensional vectors.
  • Save src/ai/ai.service.ts.
  • Check the terminal where npm run start:dev is running. The dev server should recompile with no errors.

Seeing a compilation error?

  • If you see a message about @google/genai not being found, confirm you installed the package in Step 1. Run npm install @google/genai if needed.
  • If the error mentions ConfigService, make sure your app.module.ts still has ConfigModule.forRoot({ isGlobal: true }) in its imports.

Still stuck? Help me fix this NestJS compilation error in my AiService

Now verify that app.module.ts was updated correctly by the CLI.

  • Open src/app.module.ts and confirm it includes AiModule in the imports array. It should look like this:
import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import { DocumentsModule } from './documents/documents.module';
import { AiModule } from './ai/ai.module';

@Module({
  imports: [
    ConfigModule.forRoot({ isGlobal: true }),
    DocumentsModule,
    AiModule,
  ],
})
export class AppModule {}
  • If the CLI did not add AiModule automatically, paste the code above and save the file.

✔️ Awesome, I've got everything!

Make sure you have saved all files in src/ai/ and that the dev server shows no errors.

ⓧ I'd like to double check the full code

Here is the full src/ai/ai.module.ts:

import { Module } from '@nestjs/common';
import { AiService } from './ai.service';

@Module({
  providers: [AiService],
  exports: [AiService],
})
export class AiModule {}

Here is the full src/ai/ai.service.ts:

import { Injectable } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { GoogleGenAI } from '@google/genai';

@Injectable()
export class AiService {
  private readonly ai: GoogleGenAI;

  constructor(private readonly configService: ConfigService) {
    this.ai = new GoogleGenAI({
      apiKey: this.configService.get<string>('GEMINI_API_KEY'),
    });
  }

  async embedTexts(texts: string[]): Promise<number[][]> {
    const embeddings: number[][] = [];
    for (const text of texts) {
      const response = await this.ai.models.embedContent({
        model: 'gemini-embedding-001',
        contents: text,
      });
      embeddings.push(response.embedding.values);
    }
    return embeddings;
  }
}

Here is the full src/app.module.ts:

import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import { DocumentsModule } from './documents/documents.module';
import { AiModule } from './ai/ai.module';

@Module({
  imports: [
    ConfigModule.forRoot({ isGlobal: true }),
    DocumentsModule,
    AiModule,
  ],
})
export class AppModule {}
Integrate embeddings into document ingestion

The AiService is ready. Now you need to wire it into the DocumentsModule so the ingestion pipeline generates embeddings before storing chunks.

  • Open src/documents/documents.module.ts and replace its entire contents with this updated version that imports AiModule:
import { Module } from '@nestjs/common';
import { DocumentsService } from './documents.service';
import { DocumentsController } from './documents.controller';
import { AiModule } from '../ai/ai.module';

@Module({
  imports: [AiModule],
  controllers: [DocumentsController],
  providers: [DocumentsService],
})
export class DocumentsModule {}

Why import AiModule here?

NestJS requires each module to explicitly import the modules it depends on. This makes dependencies visible and testable, even if the module is already registered at the root level.

  • Save src/documents/documents.module.ts.
  • Confirm the dev server recompiles with no errors in your terminal.

Now update the service itself to inject AiService and add logging.

  • Open src/documents/documents.service.ts and find the imports at the top:
import { Injectable } from '@nestjs/common';
import { ChromaClient } from 'chromadb';
import { CreateDocumentDto } from './dto/create-document.dto';
  • Replace those import lines with these updated imports:
import { Injectable, Logger } from '@nestjs/common';
import { ChromaClient } from 'chromadb';
import { AiService } from '../ai/ai.service';
import { CreateDocumentDto } from './dto/create-document.dto';
  • Next, find the class declaration and constructor:
@Injectable()
export class DocumentsService {
  private readonly chroma: ChromaClient;

  constructor() {
    this.chroma = new ChromaClient({ host: 'localhost', port: 8000 });
  }
  • Replace that block with this version that adds a logger and injects AiService:
@Injectable()
export class DocumentsService {
  private readonly logger = new Logger(DocumentsService.name);
  private readonly chroma: ChromaClient;

  constructor(private readonly aiService: AiService) {
    this.chroma = new ChromaClient({ host: 'localhost', port: 8000 });
  }
  • Save the file and confirm the dev server recompiles with no errors before continuing.

The final change is the create method. You need to generate embeddings for all chunks and pass them to ChromaDB.

  • In the same file, find the create method and replace it with this updated version:
  async create(dto: CreateDocumentDto) {
    const chunks = this.chunkText(dto.content);
    const collection = await this.chroma.getOrCreateCollection({ name: 'documents' });

    // Generate Gemini embeddings for each chunk
    this.logger.log(`Generating embeddings for ${chunks.length} chunks...`);
    const embeddings = await this.aiService.embedTexts(chunks);
    this.logger.log(`Embeddings generated. Dimension: ${embeddings[0]?.length}`);

    const ids = chunks.map((_, i) => `${dto.title.replace(/\s+/g, '-').toLowerCase()}-chunk-${i}`);
    const metadatas = chunks.map((_, i) => ({ title: dto.title, chunkIndex: i }));

    // Pass embeddings alongside documents so ChromaDB uses our vectors
    await collection.add({ ids, embeddings, documents: chunks, metadatas });

    return { message: 'Document ingested successfully', title: dto.title, chunksStored: chunks.length };
  }
}

What changed in the create method?

  • Two logging lines bracket the embedding call so you can see progress in the terminal.
  • The aiService.embedTexts(chunks) call sends each chunk to the Gemini API and returns 768-dimensional vectors.
  • The embeddings array is now passed to collection.add, telling ChromaDB to store your custom vectors instead of generating its own.
  • Save src/documents/documents.service.ts.

Before you test, predict what you expect to see in the terminal when you POST a document. Will the embedding dimension be 768, or something else?

  • Test the embedding pipeline by running this curl command in a new terminal:
curl -X POST http://localhost:3000/documents \
  -H "Content-Type: application/json" \
  -d '{"title": "NestJS Guide", "content": "NestJS is a framework for building efficient server-side applications. It uses TypeScript and combines elements of OOP, FP, and FRP. Under the hood it uses Express by default but can also be configured to use Fastify. NestJS provides an out-of-the-box application architecture that allows developers to create highly testable, scalable, and maintainable applications."}'

In the terminal running your dev server, you should see two log lines:

[DocumentsService] Generating embeddings for 1 chunks...
[DocumentsService] Embeddings generated. Dimension: 768

The dimension 768 confirms gemini-embedding-001 produced the vectors. Your curl response should still return the familiar success JSON with chunksStored: 1.

Seeing a 500 error or no embedding logs?

  • Check that your .env file has a valid GEMINI_API_KEY value (not the placeholder your-api-key-here). Get a free key from https://aistudio.google.com/apikey.
  • If you see a rate-limit error, wait 60 seconds and try again. The free tier has per-minute limits.
  • If the error mentions AiService not being available, confirm documents.module.ts has imports: [AiModule] in its @Module decorator.

Still stuck? Help me debug why my Gemini embeddings are not working in NestJS

✔️ Awesome, I've got everything!

Confirm you saved both documents.module.ts and documents.service.ts, and that your logs show Dimension: 768.

ⓧ I'd like to double check the full code

Here is the full src/documents/documents.module.ts:

import { Module } from '@nestjs/common';
import { DocumentsService } from './documents.service';
import { DocumentsController } from './documents.controller';
import { AiModule } from '../ai/ai.module';

@Module({
  imports: [AiModule],
  controllers: [DocumentsController],
  providers: [DocumentsService],
})
export class DocumentsModule {}

For src/documents/documents.service.ts, confirm your file has these sections in order:

  • Imports: Injectable, Logger from @nestjs/common, ChromaClient from chromadb, AiService from ../ai/ai.service, and CreateDocumentDto.
  • Class properties: logger and chroma.
  • Constructor injecting AiService and initializing ChromaClient.
  • The chunkText helper method (unchanged from before).
  • The create method with this.aiService.embedTexts(chunks) and embeddings passed to collection.add.

Your documents are now stored with Gemini-generated vectors. Next up, you will build the question-answering endpoint that embeds a user's question with the same model, retrieves the most relevant chunks, and sends them to Gemini for a grounded answer.

Build the Question-Answering Endpoint

Your documents are chunked, embedded, and stored in ChromaDB. Right now they are just data sitting unused.

This step brings the RAG pattern to life. You will build an endpoint that takes a question, embeds it, retrieves the most relevant document chunks from ChromaDB, and sends them as context to Gemini to generate a grounded answer.

In this step, get ready to:
  • Create a DTO and endpoint for asking questions.
  • Add a generation method that builds a grounded prompt from retrieved context.
  • Wire the full RAG pipeline: embed, retrieve, generate, respond.
Create the ask endpoint and retrieval logic

Before you can query documents, you need a DTO to validate incoming questions and a controller endpoint to receive them.

  • Create a new file at src/documents/dto/ask-question.dto.ts with the following code:
import { IsNotEmpty, IsString } from 'class-validator';

export class AskQuestionDto {
  @IsString()
  @IsNotEmpty()
  question: string;
}

What does this code do?

This DTO validates that every request to the ask endpoint includes a non-empty question string. The ValidationPipe you registered globally in main.ts enforces these decorators automatically.

  • Save ask-question.dto.ts.
  • Check your terminal. The dev server should recompile with no errors.

Seeing a compilation error?

Make sure the file is saved inside src/documents/dto/. If you placed it in the wrong folder, the import paths in the next steps will fail.

help me fix a compilation error after creating ask-question.dto.ts

Implement the augmented generation step

With the DTO ready, you need two things: a method that generates answers from context passages, and a service method that orchestrates the full retrieval-to-generation flow.

  • Open src/ai/ai.service.ts and add the following generateAnswer method below the existing embedTexts method:
  async generateAnswer(question: string, context: string[]): Promise<string> {
    const contextBlock = context.join('\n\n---\n\n');
    const prompt = `You are a helpful assistant. Answer the question based ONLY on the provided context. If the context does not contain enough information to answer, say "I don't have enough information to answer that."

Context:
${contextBlock}

Question: ${question}

Answer:`;

    const response = await this.ai.models.generateContent({
      model: 'gemini-2.5-flash',
      contents: prompt,
    });
    return response.text ?? 'No response generated.';
  }

What does this code do?

This method builds a grounded prompt. It joins the retrieved context passages with separators, instructs the model to answer ONLY from the provided context, and calls gemini-2.5-flash for generation.

The "answer only from context" instruction is key to RAG. It prevents the model from hallucinating answers outside the documents you uploaded.

  • Save ai.service.ts.
  • Check your terminal. The dev server recompiles with no errors.

Seeing a template literal error?

Make sure the backtick-delimited prompt string uses ${contextBlock} and ${question} for interpolation. Regular quotes won't work for multiline template literals.

help me fix a template literal error in my generateAnswer method

Now wire up the service and controller. The ask method in DocumentsService will embed the question, query ChromaDB for the top 3 matching passages, then feed them into generateAnswer.

  • Open src/documents/documents.service.ts and add the AskQuestionDto import at the top, next to the existing CreateDocumentDto import:
import { AskQuestionDto } from './dto/ask-question.dto';
  • Add the following ask method below the existing create method in the same file:
  async ask(dto: AskQuestionDto) {
    const collection = await this.chroma.getOrCreateCollection({
      name: 'documents',
    });

    // Embed the question using the same model as document ingestion
    const [questionEmbedding] = await this.aiService.embedTexts([dto.question]);

    // Query ChromaDB for the 3 most relevant chunks
    const results = await collection.query({
      queryEmbeddings: [questionEmbedding],
      nResults: 3,
    });

    const passages = results.documents?.[0] ?? [];
    const metadatas = results.metadatas?.[0] ?? [];

    if (passages.length === 0) {
      return { answer: 'No relevant documents found.', sources: [] };
    }

    // Generate a grounded answer from the retrieved passages
    const answer = await this.aiService.generateAnswer(dto.question, passages);

    const sources = passages.map((text, i) => ({
      text,
      metadata: metadatas[i],
    }));

    return { answer, sources };
  }

What does this code do?

This is the full RAG read path in one method:

  • It embeds the user's question into the same 768-dimensional vector space as the stored documents.
  • It queries ChromaDB with queryEmbeddings to find the 3 nearest chunks by cosine similarity.
  • It passes those chunks as context to generateAnswer, which produces a grounded response.
  • It returns both the answer and the source passages so the caller can verify where the answer came from.
  • Save documents.service.ts.
  • Check the terminal. The dev server should recompile successfully.

Finally, update the controller to expose the new endpoint.

  • Replace the contents of src/documents/documents.controller.ts with:
import { Controller, Post, Body } from '@nestjs/common';
import { DocumentsService } from './documents.service';
import { CreateDocumentDto } from './dto/create-document.dto';
import { AskQuestionDto } from './dto/ask-question.dto';

@Controller('documents')
export class DocumentsController {
  constructor(private readonly documentsService: DocumentsService) {}

  @Post()
  async create(@Body() createDocumentDto: CreateDocumentDto) {
    return this.documentsService.create(createDocumentDto);
  }

  @Post('ask')
  async ask(@Body() askQuestionDto: AskQuestionDto) {
    return this.documentsService.ask(askQuestionDto);
  }
}

What does this code do?

The new @Post('ask') decorator maps POST requests to /documents/ask. NestJS validates the incoming body against AskQuestionDto before it reaches your service.

  • Save documents.controller.ts.
  • Confirm the dev server recompiles without errors.

Seeing an import error for AskQuestionDto?

Double check that src/documents/dto/ask-question.dto.ts exists and exports the class. The import path is relative: ./dto/ask-question.dto.

help me fix an import error for AskQuestionDto in my documents controller

Time to test the full pipeline. Before you run the next command, think about what you expect to happen. You already uploaded a document in the previous step. Do you think the API will find relevant passages and generate an answer?

  • If you have not uploaded a document yet, do so now by running:
curl -X POST http://localhost:3000/documents \
  -H "Content-Type: application/json" \
  -d '{"title": "Solar System", "content": "The solar system consists of the Sun and the objects that orbit it. The four inner planets are Mercury, Venus, Earth, and Mars. They are called terrestrial planets because they have solid rocky surfaces. The four outer planets are Jupiter, Saturn, Uranus, and Neptune. They are called gas giants because they are made mostly of gas. Pluto was reclassified as a dwarf planet in 2006."}'
  • Now ask a question about the document you uploaded:
curl -X POST http://localhost:3000/documents/ask \
  -H "Content-Type: application/json" \
  -d '{"question": "What are the inner planets?"}'

You should receive a JSON response with an answer field containing Gemini's response (something like "The four inner planets are Mercury, Venus, Earth, and Mars") and a sources array showing the passages it used.

Getting an empty sources array or no relevant documents?

Make sure you uploaded a document first. The ask endpoint queries existing documents in ChromaDB. If the collection is empty, it returns "No relevant documents found."

Also confirm your ChromaDB container is still running with docker ps. If it stopped, restart it with docker start chromadb.

help me debug why my ask endpoint returns no relevant documents

✔️ Awesome, I've got everything!

Great. Double check you have saved all three files.

ⓧ I'd like to double check the full code

Here are the files you modified in this step. Scroll up and confirm each piece is present in your code.

  • src/documents/dto/ask-question.dto.ts exports AskQuestionDto with a validated question field.
  • src/ai/ai.service.ts has embedTexts and the new generateAnswer method.
  • src/documents/documents.service.ts imports AskQuestionDto and has both create and ask methods.
  • src/documents/documents.controller.ts imports AskQuestionDto and has both @Post() and @Post('ask') endpoints.

Your Document Q&A API is fully functional. You can upload any text document and ask natural-language questions about it. Next up, you will tackle the Secret Mission: adding metadata filtering so you can scope questions to specific documents by title.

Secret mission

Add Document Filtering

Right now, your API searches across all uploaded documents at once. In production, users need to scope their questions to specific documents. Can you add metadata filtering so the /documents/ask endpoint only retrieves chunks from a named document?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs locally with Docker and the Gemini free tier, so there are no ongoing costs.

Resources you used:

  • Docker container named chromadb running on port 8000.
  • Local chroma-data folder storing vector data (inside your project directory).
  • The doc-qa-api project folder on your Desktop.
  • NestJS CLI installed globally via npm.

Keep everything running

No action needed. Choose this if you're still actively building or experimenting with the API. Everything runs locally, so there is no cost for leaving it running.

  • Your Docker container continues running on port 8000.
  • Your NestJS dev server stays available at http://localhost:3000.
  • Your Gemini API key remains in your .env file for continued use.

Pause - I'll come back to this later

Stop running processes to free up system memory. Your files and data persist so you can resume later.

  • Stop the NestJS dev server by pressing Ctrl+C in the terminal where it is running.
  • Stop the ChromaDB container by running:
docker stop chromadb
  • When you return, restart ChromaDB by running:
docker start chromadb
  • Then start your dev server again with npm run start:dev from inside the doc-qa-api folder. Your uploaded documents are still stored in ChromaDB.

Delete - I don't want to use this again

Remove all project resources and start fresh. This permanently deletes your uploaded documents and vector data.

  • Stop and remove the ChromaDB container by running:
docker rm -f chromadb
  • Delete the project folder and all its contents (including chroma-data). On macOS/Linux, run:
rm -rf ~/Desktop/doc-qa-api
  • On Windows, run:
rmdir /s /q %USERPROFILE%\Desktop\doc-qa-api
  • Uninstall the NestJS CLI globally by running:
npm uninstall -g @nestjs/cli

That's a Wrap!

That's a Wrap!

Nice work! You just built a production-style Document Q&A API that accepts text uploads and answers natural-language questions using RAG.

You've learned how to:

  • Build a NestJS REST API from scratch with senior-level modular architecture: feature modules, shared services, DTOs, and validation pipes.
  • Run ChromaDB as a vector database in Docker and integrate it with a TypeScript backend.
  • Implement a complete RAG pipeline end-to-end: document chunking, embedding with Gemini, vector storage, semantic retrieval, and augmented generation.
  • Secret Mission: Added metadata filtering for scoped document queries, demonstrating hybrid search capabilities.

Ready to quiz yourself?