Build a RAG App with Bedrock

Build a RAG pipeline on AWS that answers questions grounded in your documents.

Introduction

30 Second Summary

AI chatbots often make things up when they don't know the answer. Retrieval-Augmented Generation (RAG) solves this by giving the AI a set of trusted documents to search before responding.

In this project, you will build a fully managed RAG application on AWS using Amazon Bedrock Knowledge Bases. You'll upload documents, create a knowledge base that automatically processes them into searchable vectors, and query it to get accurate, source-cited answers.

What You'll Build

A RAG pipeline that ingests documents into a vector store and retrieves relevant context to ground a foundation model's responses in your own data.

By the end of this project, you'll have:

An S3 bucket containing your source documents, connected to a Bedrock Knowledge Base as a data source.
A working RAG pipeline that chunks, embeds, and indexes your documents into S3 Vectors for semantic search.
Tested queries showing grounded, cited responses from a foundation model instead of hallucinated answers.
Secret Mission: Connect a Bedrock Agent to your Knowledge Base to create a conversational AI assistant.

Are there any prerequisites?

You'll need an AWS account with billing enabled. No prior experience with Amazon Bedrock or RAG is required.

Set Up Your AWS Environment

Before you can build a RAG pipeline, you need the underlying AWS infrastructure in place. That means access to the AI models that will power your knowledge base, and a storage location for the documents you want to search.

In this step, you will enable the foundation models in Amazon Bedrock and create an S3 bucket to hold your source documents.

In this step, get ready to:

Sign in to AWS and enable Bedrock model access.
Create an S3 bucket and upload sample documents.
Verify your models are ready to use.

Sign in to AWS and enable Bedrock model access

Your RAG pipeline needs two types of AI models. An embedding model converts your documents into numerical vectors for searching. A generation model produces human-readable answers from the retrieved context.

Open your browser and go to the AWS Management Console at https://console.aws.amazon.com.
Sign in with your AWS account credentials.
In the top-right corner of the console, click the region name dropdown.
Select US East (N. Virginia) us-east-1.

Why us-east-1?

Amazon Bedrock supports the widest range of models in us-east-1. Your S3 bucket must be in the same region as your Knowledge Base, so setting your region now keeps everything aligned.

In the console search bar at the top, type Bedrock and select Amazon Bedrock from the dropdown.

Amazon Bedrock automatically enables access to all serverless foundation models by default. Amazon-owned models like Titan Text Embeddings V2 and Amazon Nova Lite are ready to use immediately.

If you plan to use Claude 3 Haiku as your generation model, Anthropic models require a one-time usage form submission before first use.

In the left navigation pane, choose Playgrounds, then select Chat / Text.
Click Select model.
Choose Anthropic as the provider, then select Claude 3 Haiku.
If prompted, you will see a one-time usage form. Fill in a brief description of your use case (e.g., "Building a RAG knowledge base for document Q&A") and submit the form.

What if I want to use Amazon Nova Lite instead?

Amazon Nova Lite is an Amazon-owned model and does not require any usage form. You can select it directly from the playground without extra steps. It is also more cost-effective for this project.

Both Claude 3 Haiku and Amazon Nova Lite work well for this project. Choose whichever you prefer.

Don't see models in the playground?

Make sure you are in the us-east-1 region. The region selector is in the top-right corner of the console. Models are region-specific, so switching to a different region shows different availability.

If you still cannot access models, confirm your IAM user or role has the required Bedrock permissions attached.

Help me troubleshoot my Bedrock model access.

Create an S3 bucket for your documents

Your knowledge base needs a source of truth. You will store your documents in an S3 bucket, which Bedrock will read from when building the vector index.

In the console search bar, type S3 and select S3 from the dropdown.
In the left navigation pane, choose General purpose buckets.
Click Create bucket.
For Bucket name, enter a unique name like nextwork-rag-docs- followed by your initials and a number (e.g., nextwork-rag-docs-jd-2026). Bucket names must be globally unique.
Confirm the AWS Region is set to US East (N. Virginia) us-east-1.
Leave all other settings at their defaults (Block Public Access stays enabled).
Scroll down and click Create bucket.

Why does the bucket name need to be unique?

S3 bucket names are globally unique across all AWS accounts. No two buckets in the world can share the same name. Adding your initials and a number ensures yours does not collide with someone else's.

Now upload your sample documents. These are the files your RAG system will search through to answer questions.

Click your newly created bucket name in the bucket list.
On the Objects tab, click Upload.
Click Add files and select 3-5 documents from your computer. These can be PDFs, text files, or markdown files.
Click Upload to start the upload.
Wait for the green success banner, then click Close.

What documents should I use?

Pick documents you can ask questions about. Company FAQs, technical documentation, meeting notes, or study guides all work well. The more specific the content, the easier it is to test your RAG system later.

Bedrock Knowledge Bases supports PDFs, Word documents, Excel files, HTML, Markdown, TXT, and CSV files.

You should see your 3-5 files listed on the Objects tab of your bucket.

Upload failed?

Check that each file is under 50 MB. If you see a permissions error, ensure your IAM user has S3 write permissions attached to their role or policy.

Help me fix my S3 upload issue.

Verify model access is granted

Before moving on, confirm that both models you need are accessible. The embedding model (Titan Text Embeddings V2) handles document indexing. The generation model (Claude 3 Haiku or Amazon Nova Lite) handles answering queries.

Navigate back to the Amazon Bedrock console by searching Bedrock in the top search bar.
In the left navigation pane, choose Model access under Bedrock configurations.
Look for Titan Text Embeddings V2 in the model list. Its status should show Access granted.
Look for your chosen generation model (Claude 3 Haiku or Amazon Nova Lite). Its status should also show Access granted.

What does Access granted mean?

This status confirms that your AWS account can call the model's API. Amazon Bedrock enables all serverless models by default in commercial regions.

For Anthropic models, the status changes to Access granted after you successfully submit the one-time usage form. If you chose Amazon Nova Lite, it is granted automatically with no extra steps.

Seeing Available to request for Anthropic?

This means you have not yet submitted the one-time usage form for Anthropic models. Go to the Bedrock playground, select an Anthropic model, and complete the form when prompted. The status updates within a few minutes.

Help me get my Anthropic model access approved.

Your AWS environment is ready. You have model access confirmed and your source documents waiting in S3. Next up, you will create the Knowledge Base that ties everything together into a working RAG pipeline.

Create Your Knowledge Base

Your S3 bucket is loaded with documents and your models are ready to go. Now it's time to build the core of your RAG application: the knowledge base itself.

A Knowledge Base in Amazon Bedrock connects your documents to the AI pipeline. It handles splitting your documents into chunks, converting them into vector embeddings, storing those embeddings in a vector store, and retrieving the most relevant ones when you ask a question.

In this step, get ready to:

Create a Bedrock Knowledge Base with S3 Vectors as the vector store.
Connect your S3 bucket as the data source.
Sync your documents to chunk, embed, and index them.

Create the Knowledge Base

The Knowledge Base creation wizard walks you through naming your resource, setting up permissions, and choosing your data source.

In the Bedrock console, click Knowledge bases in the left navigation pane.
Click Create.
Select Knowledge Base with vector store.
Under Knowledge Base details, give your knowledge base a descriptive name (for example, my-rag-knowledge-base).
Under IAM permissions, select Create and use a new service role.
For the data source, choose Amazon S3, then click Next.

What is the service role?

The service role is an IAM role that gives Bedrock permission to access your S3 bucket, call the embedding model, and write to the vector store.

By letting Bedrock create this role for you, the permissions are automatically scoped to only what your knowledge base needs.

Configure the data source and vector store

Now you'll point the knowledge base at your documents and choose how they'll be stored as vectors.

Enter a name for the data source (for example, my-documents-source).
In the S3 URI field, enter the URI of your S3 bucket in the format s3://your-bucket-name.
Leave the default chunking and parsing settings as they are.
Click Next.
In the Embeddings model section, select Titan Text Embeddings v2.
In the Vector database section, select Quick create a new vector store.
Choose Amazon S3 Vectors as the vector store type.
Click Next.
Review your configuration, then click Create knowledge base.

The knowledge base may take a few minutes to create. Wait until the status changes from In progress before continuing.

Why S3 Vectors over OpenSearch Serverless?

Amazon S3 Vectors is a purpose-built vector store that went GA in December 2025.

It reduces vector storage and query costs by up to 90% compared to OpenSearch Serverless, which requires a minimum of 4 OCUs even at zero traffic. For a learning project with a small document set, S3 Vectors is the ideal choice: pay-as-you-go pricing with no minimum cost.

Knowledge base creation failing?

Make sure your S3 bucket is in the same region as your Bedrock knowledge base. If you created the bucket in a different region, you'll need to recreate it in the correct one.

Also confirm that model access for Titan Text Embeddings V2 shows Access granted in the Bedrock model access page.

help me troubleshoot why my Bedrock Knowledge Base creation is failing

Sync your documents

Creating the knowledge base set up the infrastructure. Now you need to sync your documents so Bedrock can chunk them, generate vector embeddings, and store them in S3 Vectors.

On your knowledge base details page, scroll down to the Data source section.
Select your data source by clicking its checkbox.
Click Sync.
Monitor the sync status until it shows Ready.

What happens during sync?

Bedrock reads each document from your S3 bucket and splits it into smaller passages called chunks. The default strategy uses fixed-size chunking, which divides text into segments of roughly equal token count.

Each chunk is then sent to the Titan Text Embeddings V2 model, which converts it into a numerical vector. These vectors are stored in your S3 Vectors vector store, ready for semantic search when you query later.

The sync typically takes 1-3 minutes for a small set of documents. Once the status shows Ready, your documents have been fully processed and indexed.

Sync status stuck or showing errors?

Verify that your S3 bucket contains supported file types (PDFs, Word, Excel, HTML, Markdown, TXT, or CSV files). Unsupported formats will be skipped during ingestion.

If the sync fails entirely, check that the service role has permission to read from your S3 bucket. Using the auto-created service role should handle this automatically.

help me fix a sync error in my Bedrock Knowledge Base

Your knowledge base is live and loaded with your documents. Next up, you'll query it and see how RAG grounds an LLM's responses in your actual content.

Query Your Knowledge Base

Your Knowledge Base has ingested, chunked, and embedded your documents. Now it is time to see RAG in action by querying your knowledge base and observing how it grounds responses in your actual source material.

In this step, you will ask your knowledge base questions, compare its responses to a plain foundation model (without retrieval), and test edge cases to understand how semantic search finds relevant information even when your wording differs from the source text.

In this step, get ready to:

Test your Knowledge Base using the built-in console test window.
Compare RAG responses to non-RAG responses in the Bedrock playground.
Experiment with edge cases and rephrased queries to observe semantic search.

Test in the Bedrock console

The Bedrock console includes a built-in test window that lets you query your Knowledge Base and see generated responses with source citations. This is the fastest way to verify your RAG pipeline works end to end.

In the Amazon Bedrock console, choose Knowledge bases in the left navigation pane.
Select your knowledge base from the list. A test window expands from the right side of the page.
In the test window, turn on Generate responses for your query. This tells Bedrock to use a foundation model to generate a natural language answer from the retrieved chunks.
Click Select model. Choose your generation model (Claude 3 Haiku or Amazon Nova Lite), then click Apply.
Enter a question in the text box that can be answered from your documents. For example, if you uploaded technical documentation, ask a specific factual question about it.
Click Run.

You should see a generated response with numbered footnotes. Each footnote links back to a specific chunk from your source documents.

Click a footnote to see the exact excerpt from your source document that the response is based on.
Click Show source details to see the raw text from each retrieved chunk, its relevance score, and a link to the original S3 object.

What are footnotes and source citations?

When Generate responses is turned on, Bedrock uses the RetrieveAndGenerate API under the hood. This API retrieves relevant chunks from your vector store, passes them as context to the foundation model, and asks the model to generate an answer. The footnotes prove which chunks informed each part of the response.

This is the core value of RAG: every answer is traceable back to your source data, making it auditable and trustworthy.

Not seeing any response?

Make sure Generate responses for your query is turned on. Without it, the test window returns raw chunks instead of a generated answer.

If you see an access error, confirm your generation model still shows Access granted status in the Bedrock model access page.

Help me troubleshoot why my Knowledge Base test is not returning results.

Compare RAG vs. non-RAG responses

To understand the value RAG provides, you will now ask the same question directly to the foundation model without any document context. This comparison shows how RAG prevents hallucination.

In the Bedrock console left navigation pane, choose Text under Playgrounds.
Click Select model and choose the same generation model you used for your Knowledge Base query. Click Apply.
Enter the same question you asked your Knowledge Base in the previous substep.
Click Run.

Compare the two responses. The playground response comes purely from the model's training data, with no access to your documents.

What should I notice?

The RAG response should be specific, accurate, and cite your actual documents. The playground response may be vague, generic, or even incorrect about details specific to your documents.

If your documents contain information the model could not have seen during training (internal policies, recent data, proprietary knowledge), the difference is even more dramatic. The playground model may hallucinate an answer while the RAG response acknowledges its sources.

Both responses look the same?

If your documents contain widely known public information, the foundation model might answer correctly from its training data alone. Try asking about a specific detail unique to your documents (a particular date, policy number, or internal term) to see the difference more clearly.

Help me find a good question to demonstrate RAG vs non-RAG differences.

Experiment with different queries

Now that you have confirmed your RAG pipeline works, test its boundaries. Edge cases reveal how semantic search behaves and where retrieval has limits.

Ask a question your documents cannot answer. For example, ask about a topic completely unrelated to your uploaded content.
Observe how the model responds. A well-configured RAG system acknowledges when it cannot find relevant information rather than hallucinating an answer.
Ask a question that spans multiple documents. For example, ask for a comparison or summary that requires information from more than one source file.
Check the footnotes. You should see citations from multiple source documents in a single response.
Rephrase a question you already asked using different wording. Use synonyms or ask the same thing from a different angle.
Observe that semantic search still retrieves the same relevant chunks, even though your phrasing is different. This demonstrates that vector embeddings capture meaning, not just keywords.

Why does rephrasing still work?

Traditional keyword search requires exact word matches. Semantic search converts both your query and your document chunks into vector embeddings that represent meaning. Two phrases with different words but similar meaning end up close together in vector space, so the retrieval engine finds them regardless of exact wording.

This is why Amazon Titan Text Embeddings V2 is critical to the pipeline. It translates human language into a mathematical representation that captures intent.

Try turning off Generate responses

In your Knowledge Base test window, clear the Generate responses for your query toggle. This switches to the Retrieve API, which returns the raw source chunks in order of relevance without generating a response. It is useful for seeing exactly which chunks the vector search selected.

You have now seen RAG in action from multiple angles. Next up, you will dive deeper into the pipeline mechanics to understand chunking, the two retrieval APIs, and the cost structure of your RAG application.

Understand the RAG Pipeline

You have successfully queried your knowledge base and seen how RAG grounds responses in your documents. Now it is time to look under the hood and understand how the pipeline actually works.

In this step, you will examine how your documents were chunked, explore the two retrieval APIs that power your knowledge base, and review the costs involved in running a RAG application on AWS.

In this step, get ready to:

Review the chunking configuration that controls how your documents are split.
Explore the Retrieve and RetrieveAndGenerate APIs in the console.
Check your AWS billing to understand the cost structure of a RAG application.

Examine chunking and embeddings

When you synced your data source, Bedrock automatically split your documents into smaller pieces called chunks. The size and overlap of these chunks directly affects how well your knowledge base retrieves relevant information.

In the Bedrock console, select Knowledge bases in the left navigation pane.
Select the name of your knowledge base to open its details page.
In the Data source section, select the radio button next to your data source.
Choose Edit to view the data source configuration, including the chunking strategy.

You should see that your data source uses the default chunking strategy.

What is default chunking?

When you do not specify a chunking strategy, Amazon Bedrock splits your content into chunks of approximately 300 tokens while preserving sentence boundaries.

This means no sentence gets cut in half mid-thought. Each chunk is a self-contained piece of text that the embedding model can convert into a meaningful vector.

Chunk size matters because smaller chunks give more precise retrieval, while larger chunks preserve more context. The 300-token default balances both for general-purpose use cases.

Why can't I change the chunking strategy?

You cannot change the chunking configuration after the data source is created. If you need different chunking (fixed-size, hierarchical, or semantic), you must re-create the data source with the new configuration.

Explore the RetrieveAndGenerate vs. Retrieve APIs

Bedrock Knowledge Bases provides two APIs for querying your data. Understanding the difference helps you choose the right one for different application scenarios.

Return to your knowledge base details page in the Bedrock console.
Select your knowledge base to open the test window on the right side of the screen.
Turn off Generate responses for your query (clear the toggle).
Enter a question about your documents and select Run.

You should see raw source chunks returned directly in order of relevance. This is the Retrieve API in action.

What is the Retrieve API?

The Retrieve API queries your knowledge base and returns only the relevant source chunks from your data. It does not generate a natural language answer.

Use Retrieve when you want full control over what happens after retrieval. For example, you might feed the chunks into your own custom prompt template or apply additional processing before showing results to the user.

Now turn on Generate responses for your query (enable the toggle).
Enter the same question and select Run.

This time you should see a full natural language answer with footnote citations pointing back to source chunks. This is the RetrieveAndGenerate API in action.

What is the RetrieveAndGenerate API?

The RetrieveAndGenerate API combines retrieval with the InvokeModel operation. It retrieves the most relevant source chunks, feeds them into a foundation model as context, and returns a generated natural language response with citations.

This is the full RAG process in a single API call. Use it when you want a ready-to-display answer for your end users.

When to use each API

Use Retrieve when you need raw chunks for custom processing, want to build your own prompt, or need to apply business logic before generation.

Use RetrieveAndGenerate when you want a turnkey answer with citations in a single call. It also handles session context automatically for multi-turn conversations.

Review costs

Understanding cost structure is essential for planning production RAG applications. Your project incurred three types of charges: embedding, inference, and storage.

Select your account name in the top-right navigation bar of the AWS console.
Choose Billing and Cost Management.
Review the charges listed under Amazon Bedrock and Amazon S3 in your billing summary.

What are the cost components of a RAG application?

There are three main cost drivers for your knowledge base:

Embedding costs: Amazon Titan Text Embeddings V2 charges $0.00002 per 1,000 input tokens.
Inference costs: Your generation model charges per token for each query. Claude 3 Haiku costs $0.00025 per 1K input tokens and $0.00125 per 1K output tokens. Amazon Nova Lite costs $0.00006 per 1M input tokens and $0.00024 per 1M output tokens.
Storage costs: S3 storage for your documents and S3 Vectors for your embeddings, both with minimal pay-as-you-go charges.

For this project with a small document set, total costs should be between $1 and $3 including all your experimentation. In production, embedding is a one-time cost per sync, while inference costs scale with query volume.

S3 Vectors keeps costs low

S3 Vectors reduces vector storage and query costs by up to 90% compared to alternatives like OpenSearch Serverless. It has no minimum charge when idle, so you only pay when you sync or query.

You now understand the full RAG pipeline from document chunking through vector retrieval to response generation. Next up, you will take on a secret mission to connect a Bedrock Agent to your knowledge base for a conversational AI experience.

Secret mission

Connect a Bedrock Agent

You have a working RAG pipeline that answers questions grounded in your documents. But what if you could turn it into a full conversational AI assistant that maintains context across multiple questions? In this secret mission, you will create a Bedrock Agent that uses your Knowledge Base as a tool for multi-turn conversations.

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project uses pay-as-you-go AWS services, so costs are minimal when you are not actively querying.

Resources you used:

S3 bucket with your source documents.
Bedrock Knowledge Base with S3 Vectors vector store.
Bedrock model access (Titan Text Embeddings V2 and generation model).
(Secret Mission) Bedrock Agent with alias.

Keep everything running

No action needed. Choose this if you plan to continue experimenting with your Knowledge Base.

S3 storage costs are negligible for a few documents (fractions of a cent per month).
S3 Vectors and Knowledge Bases do not charge when idle. You only pay for Bedrock inference when you make queries.
Model access does not incur charges. You are only billed per token when you run queries.

Pause - I'll come back to this later

No active processes to shut down. Your resources are serverless and only cost money when you query them.

Your Knowledge Base, S3 Vectors store, and S3 bucket all remain intact at near-zero idle cost.
When you return, you can query your Knowledge Base immediately without re-syncing (unless you add new documents).

Delete - I don't want to use this again

Remove all project resources to stop all charges immediately. Follow these steps in order.

Delete the Bedrock Agent (Secret Mission only): In the Bedrock console, select Agents from the left navigation pane. Select the radio button next to your agent. Choose Delete. Type delete in the confirmation dialog and select Delete.
Dissociate the Knowledge Base from the Agent (if you completed the Secret Mission): In the Bedrock console, select Agents from the left navigation. Choose your agent. Select Edit in Agent builder. In the Knowledge bases section, select the radio button next to your Knowledge Base and choose Delete. Confirm the deletion.
Delete the Knowledge Base: In the Bedrock console, select Knowledge bases from the left navigation pane. Select the radio button next to your Knowledge Base. Choose Delete. Type delete in the input box and select Delete to confirm. This removes the data from the S3 Vectors store (the default data deletion policy is set to Delete).
Delete the S3 bucket: Open the S3 console at https://console.aws.amazon.com/s3/. In the left navigation, choose General purpose buckets. Select the bucket you created. Choose Empty. Type permanently delete to confirm. Once empty, return to the buckets list, select the bucket again, choose Delete, enter the bucket name to confirm, and choose Delete bucket.

What about the IAM service role?

Bedrock auto-created a service role for your Knowledge Base. This role incurs no charges. You can leave it in place or delete it from the IAM console under Roles if you prefer a clean account.

Cost warning

If you leave your resources in place and continue making queries, you will be charged per token for embedding and inference. For occasional testing, expect costs under $0.05 per session. If you are done with this project, choose Delete to stop all charges immediately.

That's a Wrap!

Nice work! You just built a production-grade RAG application on AWS using Amazon Bedrock Knowledge Bases.

You've learned how to:

Built a Bedrock Knowledge Base that automatically chunks, embeds, and indexes your documents into S3 Vectors.
Queried your knowledge base with a foundation model and received grounded, source-cited responses using the RetrieveAndGenerate and Retrieve APIs.
Compared RAG vs. non-RAG responses to understand how retrieval grounds answers in real documents and reduces hallucination.
Connected a Bedrock Agent to your Knowledge Base to create a conversational AI assistant with multi-turn memory.

Ready to quiz yourself?