RAG with Amazon Bedrock Knowledge Bases

Build a managed RAG system that answers questions from your documents.

Introduction

30 Second Summary

When you ask an AI a question, it draws from its training data. But what if you need answers grounded in your documents? That's where Retrieval-Augmented Generation comes in.

In this project, you will build a fully managed RAG system on AWS that answers questions from your uploaded documents with source citations. Amazon Bedrock Knowledge Bases handles all the heavy lifting so you can focus on understanding the architecture.

What You'll Build

A fully managed RAG system using Amazon Bedrock that automatically chunks, embeds, and indexes your documents for AI-powered question answering.

By the end of this project, you'll have:

  • An S3 bucket containing your source documents connected to a Bedrock Knowledge Base with automatic vector storage.
  • A working RAG pipeline that retrieves relevant document chunks and generates grounded answers with citations.
  • Experience querying your knowledge base through both the AWS console test interface and programmatically via the AWS CLI.
  • Secret Mission: Query your knowledge base programmatically with different generation models and explore how switching models affects answer quality.

Are there any prerequisites?

None. This guide includes full AWS account creation and setup from scratch. No prior cloud experience or coding knowledge is required.

Before We Start

Before We Start...

Create Your AWS Account and Enable Bedrock Models

Before you can build a RAG system, you need access to the AI models that power it. Amazon Bedrock provides fully managed foundation models from multiple providers, but you need an AWS account and model access to use them.

In this step, you will create (or sign into) your AWS account, set the correct region, and enable the two models your knowledge base needs: one for converting documents into vector embeddings, and one for generating answers.

In this step, get ready to:
  • Create or sign into your AWS account and set the region to US East (N. Virginia).
  • Navigate to Amazon Bedrock and enable model access for Titan Text Embeddings V2 and Amazon Nova Lite.
  • Verify that model access is granted.
Create or sign into your AWS account

You need an active AWS account with a payment method on file. If you already have one, skip to the sign-in step below.

  • Go to https://aws.amazon.com and click Create an AWS Account.
  • Follow the prompts to enter your email, set a password, and add a payment method.
  • Once your account is ready, sign in to the AWS Management Console.

Why do I need a payment method?

AWS requires a payment method even for free-tier resources. Amazon Bedrock charges per model invocation (pay-as-you-go), so you will see small charges of a few cents to a few dollars depending on how much you test.

Set your region to US East (N. Virginia)

AWS resources are region-specific. All Bedrock models and features you need for this project are available in the us-east-1 region.

  • Look at the top-right corner of the AWS Management Console, next to your account name.
  • Click the Region name displayed there (it might say something like US East (Ohio) or another default).
  • From the dropdown, select US East (N. Virginia).

The console should now display US East (N. Virginia) in the top-right region selector.

Why US East (N. Virginia)?

This region (us-east-1) has the broadest availability for Amazon Bedrock models, including all Amazon Nova models. The region you request model access from is the only region where you can use those models.

Navigate to Amazon Bedrock and enable model access

Your knowledge base needs two models: an embedding model to convert documents into vectors, and a generation model to produce answers. You will enable both now.

  • In the AWS Management Console search bar at the top, type Bedrock and select Amazon Bedrock from the dropdown.
  • In the left navigation pane, under Bedrock configurations, choose Model access.
  • Choose Enable specific models.
  • From the Base models list, select Titan Text Embeddings V2 and Nova Lite.
  • Click Next.
  • On the Review and submit page, click Submit.

What are these two models for?

Titan Text Embeddings V2 (amazon.titan-embed-text-v2:0) converts your document text into numerical vectors that capture semantic meaning. It supports up to 8,192 tokens of input and outputs 1,024-dimension vectors.

Amazon Nova Lite (amazon.nova-lite-v1:0) is the generation model that reads retrieved document chunks and produces human-readable answers. It is optimized for RAG applications and costs just $0.06 per million input tokens.

Already seeing Access granted?

AWS recently simplified model access for Amazon-owned models. If you see Access granted next to both models already, you are all set and can skip ahead to verifying access below.

Verify model access is granted

Amazon-owned models (like Titan and Nova) approve instantly. You should see confirmation right away.

  • Return to the Model access page (left navigation pane, under Bedrock configurations).
  • Refresh the Base models table.
  • Confirm that both Titan Text Embeddings V2 and Nova Lite show an Access granted status.

✔️ I see Access granted for both models

You are all set. Both models are ready to use in us-east-1.

ⓧ I don't see the Model access page

AWS recently simplified model access. If you do not see a Model access page in the left navigation, Amazon-owned models are already enabled automatically for your account.

  • Verify by clicking Model catalog in the left navigation pane.
  • Search for Titan Text Embeddings V2 and Nova Lite.
  • If both appear in the catalog, your models are ready to use.

You can proceed to the next step.

ⓧ Access is still pending

Amazon-owned models typically approve instantly. If you still see a pending status after a minute, try refreshing the page.

  • Make sure you are in the US East (N. Virginia) region (check the top-right corner).
  • Try clicking Model access again in the left navigation to refresh the page.
  • If access remains pending for more than 5 minutes, sign out and sign back in.

Having trouble with Bedrock?

Make sure your region is set to US East (N. Virginia) in the top-right corner. Some Bedrock features and models are not available in other regions.

If you see a permissions error, your AWS account may need the AmazonBedrockFullAccess policy attached to your user or role. For a new personal account, you are already the root user and have full access.

help me troubleshoot Bedrock model access issues in the AWS console

Your AWS account is ready and both AI models are enabled. Next up, you will upload the documents that your knowledge base will use to answer questions.

Upload Documents to S3

Your Bedrock models are ready to go. Now you need source material for your knowledge base to learn from.

Amazon Bedrock Knowledge Bases reads documents from Amazon S3, AWS's cloud storage service. You'll create a storage bucket, organize it with a folder, and upload the files you want your AI to answer questions about.

In this step, get ready to:
  • Create an S3 bucket with a dedicated folder for your source documents.
  • Upload one or more documents (PDF, TXT, or HTML) to your bucket.
  • Verify your documents are visible in the S3 console.
Create an S3 bucket for your documents

An S3 bucket is a container that holds your files in the cloud. Bedrock Knowledge Bases connects to this bucket to read your source documents.

  • In the AWS console search bar at the top of the page, type S3.
  • Click S3 from the search results to open the S3 console.
  • In the left navigation pane, click General purpose buckets.
  • Click Create bucket.

The Create bucket page opens with several configuration options.

  • In the Bucket name field, enter a unique name following this pattern: nextwork-kb-docs- followed by something unique to you (your name, initials, or a few random numbers). For example, nextwork-kb-docs-jane42.
  • Leave all other settings at their defaults.
  • Scroll to the bottom of the page and click Create bucket.

Why does the bucket name need to be globally unique?

S3 bucket names share a global namespace across all AWS accounts. No two buckets anywhere in the world can have the same name. Adding your name or initials ensures yours is unique.

Bucket names must be 3 to 63 characters long, use only lowercase letters, numbers, and hyphens, and start and end with a letter or number.

You should see a green success banner and your new bucket listed on the General purpose buckets page.

Now create a folder inside the bucket to keep your documents organized.

  • Click your bucket name (for example, nextwork-kb-docs-jane42) to open it.
  • Click Create folder.
  • In the Folder name field, enter documents.
  • Click Create folder.

You should see the documents/ folder listed inside your bucket.

Seeing a "Bucket already exists" error?

This means someone else in the world already has a bucket with that name. Try adding more characters to make your name unique (for example, add today's date or extra digits).

Also confirm your region is still set to US East (N. Virginia) in the top-right corner of the console.

Help me fix this bucket name error.

Upload your source documents

Your bucket needs at least one document for the knowledge base to work with. The document should contain information you want the AI to answer questions about.

What documents should I upload?

Pick any PDF, TXT, or HTML file you have on hand. A company FAQ page, course notes, a product manual, or even a Wikipedia article saved as a text file all work great.

If you don't have a document ready, create a plain text file on your computer with a few paragraphs about any topic you know well. Save it as my-notes.txt. The more content in your file, the better your knowledge base can answer questions.

  • Click the documents/ folder to open it.
  • Click Upload.
  • Click Add files.
  • Select one or more files from your computer (PDF, TXT, or HTML format, each under 50 MB).
  • Click Upload.

You should see a green success banner confirming your files were uploaded. Your documents now appear listed inside the documents/ folder with their file names and sizes displayed.

Don't see your files after uploading?

Make sure you clicked the documents/ folder before uploading. If your files ended up in the bucket root instead, select them, choose Actions then Move, and move them into the documents/ folder.

Help me find my uploaded documents in S3.

Your source documents are ready and waiting in S3. Next up, you'll create a Bedrock Knowledge Base that reads these files, breaks them into chunks, and turns them into searchable vectors.

Create Your Bedrock Knowledge Base

Your documents are uploaded and waiting in S3. Now you need something that can actually read them, understand them, and answer questions about them.

In this step, you will create an Amazon Bedrock Knowledge Base that connects to your S3 bucket, chooses an embedding model, and sets up a vector store to hold the searchable representations of your documents.

In this step, get ready to:
  • Create a new knowledge base in the Amazon Bedrock console.
  • Connect your S3 documents folder as the data source.
  • Configure the embedding model and vector store.
Navigate to Amazon Bedrock Knowledge Bases

The Knowledge Bases feature lives under the Builder tools section of the Bedrock console. This is where you create and manage all your RAG-powered knowledge bases.

  • In the Amazon Bedrock console, find the left navigation pane.
  • Click Knowledge bases in the left navigation pane.
  • Click Create.
  • Select Knowledge Base with vector store.

You should now see the knowledge base creation wizard.

  • Under Knowledge base details, change the default name to nextwork-knowledge-base.
  • Under IAM permissions, select Create and use a new service role.
  • For Data source, choose Amazon S3.
  • Click Next to configure your data source.

What is a service role?

A service role is an IAM role that gives Amazon Bedrock permission to access your S3 bucket and the embedding model on your behalf. By choosing Create and use a new service role, Bedrock automatically creates a role with the right permissions scoped to your knowledge base.

Configure the S3 data source

Now you need to point the knowledge base at the specific folder in your S3 bucket where your documents live. The inclusion prefix tells Bedrock to only ingest files from that folder path.

  • In the S3 URI field, enter the path to your documents folder: s3:// followed by your bucket name and the documents/ folder.

The full S3 URI should look like this:

s3://[[BUCKET_NAME="nextwork-kb-docs-your-unique-id"]]/documents/
  • Leave the chunking and parsing settings at their defaults.
  • Click Next to proceed to the embeddings and vector store configuration.

What is document chunking?

When Bedrock ingests your documents, it splits them into smaller pieces called chunks. Each chunk gets converted into a vector embedding separately.

The default chunking strategy works well for standard documents. You cannot change the chunking configuration after the knowledge base is created, but the default is a good starting point.

Select the embedding model and vector store

This page is where you choose how your documents get converted into searchable vectors and where those vectors are stored.

  • In the Embeddings model section, select Titan Text Embeddings v2.
  • In the Vector database section, select Quick create a new vector store.
  • Choose Amazon S3 Vectors as the vector store type.
  • Click Next to proceed to the review page.

What are vector embeddings?

Vector embeddings are numerical representations of text that capture meaning. The Titan Text Embeddings V2 model (amazon.titan-embed-text-v2:0) converts each chunk of your documents into a 1,024-dimension vector.

When you ask a question, your question also gets converted into a vector. Bedrock then finds the document chunks whose vectors are closest to your question vector. This is called semantic search.

Why S3 Vectors?

Amazon S3 Vectors is the simplest vector store option. When you choose quick create, Bedrock automatically creates an S3 vector bucket and a vector index for you. There is no additional infrastructure to configure.

Review and create the knowledge base
  • On the review page, confirm your settings:
  • Click Create knowledge base.

The creation process takes a few minutes. Wait until the status changes to Available before moving on.

  • Once the status shows Available, note the Knowledge Base ID displayed on the knowledge base details page. You will need this later.

Status stuck on In progress?

Knowledge base creation can take up to a few minutes depending on your configuration. If the status does not change after 5 minutes, try refreshing the page.

Make sure you are in the US East (N. Virginia) region. Both Amazon Bedrock Knowledge Bases and S3 Vectors must be available in your selected region.

Help me troubleshoot my knowledge base creation.

Your knowledge base is now created and ready. It knows where your documents are and how to convert them into vectors. Next, you will sync the data source to actually ingest your documents and then test it with real questions.

Sync and Test Your Knowledge Base

Your knowledge base has all the right pieces in place: documents in S3, an embedding model selected, and a vector store ready to receive data. But right now, it's empty.

Before your knowledge base can answer any questions, it needs to process your documents. This is where syncing comes in. Syncing triggers the full RAG ingestion pipeline: Bedrock reads your source files, breaks them into chunks, generates vector embeddings for each chunk, and stores those embeddings in the vector store.

In this step, get ready to:
  • Sync your data source to ingest documents into the vector store.
  • Test your knowledge base using the built-in console interface.
  • Compare retrieval-only mode with retrieval + generation mode.
Sync your data source

Syncing tells Amazon Bedrock to read through your S3 documents, split them into manageable chunks, convert each chunk into a vector embedding using Titan Text Embeddings V2, and store those embeddings in your S3 Vectors vector store.

  • In the Amazon Bedrock console, click Knowledge bases in the left navigation pane.
  • Click the name of your knowledge base (nextwork-knowledge-base).
  • In the Data source section, select the radio button next to your S3 data source.
  • Click Sync.

The sync process takes a moment depending on the size of your documents. When it finishes, a green success banner appears and the status changes to Ready.

What happens during a sync?

Bedrock reads each file from your S3 documents/ folder and splits the content into smaller pieces called chunks. Each chunk gets converted into a numerical vector (an embedding) by the amazon.titan-embed-text-v2:0 model.

Those embeddings are stored in your S3 Vectors index. When you later ask a question, Bedrock converts your question into a vector, then finds the stored chunks whose vectors are most similar to it.

  • Wait for the green success banner to appear.
  • Click your data source name to view the Sync history section and confirm the sync status shows as complete.

Sync not completing?

Check that your S3 bucket contains at least one supported file (PDF, TXT, HTML, Word, CSV, or Markdown) in the documents/ folder. Files larger than 50 MB are not supported for ingestion.

If the sync shows warnings, click View warnings next to the sync event to see which files failed and why.

Why is my knowledge base sync failing?

Test your knowledge base in the console

Now that your documents are indexed, you can ask your knowledge base questions directly from the Bedrock console. The built-in test interface lets you see exactly how RAG works without writing any code.

  • Return to the knowledge base details page by clicking Knowledge bases in the left navigation, then clicking nextwork-knowledge-base.
  • On the right side of the page, find the Test knowledge base section. A test window should already be visible.
  • Make sure the Generate responses toggle is turned ON. This enables the full RAG pipeline: retrieve relevant chunks, then generate a natural language answer.
  • Select a model for response generation if prompted (choose Amazon Nova Lite).
  • Type a question about your uploaded documents in the text box and click Run.

You should see a generated answer that references information from your documents. Below the answer, you will see citations showing which parts of your documents the answer came from.

Ask questions your documents can answer

Choose questions that you know are covered in your uploaded documents. For example, if you uploaded a company FAQ, ask a specific question from that FAQ. This helps you verify the system is retrieving the right content.

Toggle between retrieval-only and retrieval + generation modes

The test interface supports two modes that show you the two core stages of RAG independently.

  • In the test window, turn OFF the Generate responses toggle.
  • Type the same question you asked before and click Run.

This time, you see only the raw source chunks retrieved from your vector store, ordered by relevance. There is no AI-generated summary.

  • Click Show source details to expand the details for each returned chunk.
  • Inspect the raw text of each chunk, the source file reference (linking back to your S3 object), and any metadata.
  • Turn Generate responses back ON and ask another question to see the full RAG answer with citations again.

Why toggle between modes?

Retrieval-only mode shows you exactly which chunks the semantic search found. This is useful for debugging. If the retrieved chunks are irrelevant, the generated answer will also be poor.

Retrieval + generation mode adds a language model (Amazon Nova Lite) on top. The model reads the retrieved chunks and synthesizes a coherent answer, citing the specific sources it used.

Getting empty or irrelevant results?

If the knowledge base returns no results, make sure your sync completed successfully (green banner appeared). After syncing, it can take a few minutes for the vector embeddings to become available for querying.

If results are irrelevant, try rephrasing your question using different terms that match the language in your documents more closely.

Why is my knowledge base returning irrelevant results?

Your knowledge base is live and answering questions grounded in your documents. Next up, you will query it programmatically using the AWS CLI to see how you would integrate this into a real application.

Query via the AWS CLI

You've tested your knowledge base through the console and confirmed it can retrieve and generate answers from your documents.

Now you'll take this one step further by querying your knowledge base programmatically using the AWS CLI. This is how real applications and automation scripts interact with Bedrock Knowledge Bases in production.

In this step, get ready to:
  • Install and configure the AWS CLI with your credentials.
  • Query your knowledge base programmatically using the retrieve-and-generate command.
  • Examine the CLI response structure including the generated answer and citations.
Install the AWS CLI

The AWS CLI lets you interact with AWS services directly from your terminal. You need it to send programmatic queries to your knowledge base.

  • Check if you already have the AWS CLI installed by opening a terminal and running:
aws --version

✔️ I see a version number

You already have the AWS CLI installed. Move on to the next substep to configure your credentials.

ⓧ Command not found

You need to install the AWS CLI. Follow the instructions for your operating system below.

macOS

  • Download and install the AWS CLI by running these commands in your terminal:
curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
sudo installer -pkg AWSCLIV2.pkg -target /
  • Your Mac will prompt you for your password. Type it and press Enter.
  • Confirm the installation succeeded by running:
aws --version

You should see a version number starting with aws-cli/2.

Windows

  • Download the AWS CLI installer from this URL: https://awscli.amazonaws.com/AWSCLIV2.msi
  • Run the downloaded AWSCLIV2.msi file and follow the on-screen instructions.
  • After installation completes, close and reopen your command prompt.
  • Confirm the installation succeeded by running:
aws --version

You should see a version number starting with aws-cli/2.

Still not seeing a version number?

You may need to restart your terminal completely after installation. If the issue persists, visit the official AWS CLI install guide for troubleshooting steps.

help me troubleshoot my AWS CLI installation

Configure your AWS credentials

The CLI needs credentials to authenticate with your AWS account. You'll create an access key in the IAM console and then feed it to the CLI.

  • In the AWS Management Console, click your account name in the top-right corner.
  • Select Security credentials.
  • Scroll down to the Access keys section.
  • Click Create access key.
  • Select Command Line Interface (CLI) as the use case.
  • Check the confirmation box at the bottom, then click Next.
  • Click Create access key.
  • Copy your Access key and Secret access key values. Store them somewhere safe because you will not be able to view the secret key again.

Why do I need access keys?

Access keys are how the AWS CLI proves your identity when making API calls. They consist of two parts: an access key ID (like a username) and a secret access key (like a password).

The CLI stores these locally so it can sign every request you make to AWS.

  • Back in your terminal, configure the AWS CLI by running:
aws configure
  • When prompted, enter the following values:
  • AWS Access Key ID: paste your access key ID.
  • AWS Secret Access Key: paste your secret access key.
  • Default region name: enter us-east-1.
  • Default output format: enter json.

What does this command do?

The aws configure command saves your credentials and preferences to a file in your home directory (~/.aws/credentials). Every future CLI command uses these settings automatically.

Setting the region to us-east-1 ensures your CLI commands target the same region where you created your knowledge base.

Seeing an invalid credentials error later?

Double-check that you pasted the access key and secret key without any extra spaces. Run aws configure again to re-enter them if needed.

help me fix my AWS CLI credentials configuration

Query your knowledge base from the CLI

With the CLI configured, you can now send a query to your knowledge base and get back an AI-generated answer with citations. You'll need your knowledge base ID for this.

  • Return to the Amazon Bedrock console.
  • In the left navigation, click Knowledge Bases under Builder tools.
  • Click on your nextwork-knowledge-base.
  • Copy the Knowledge base ID from the top of the details page. It looks something like ABCDE12345.
  • In your terminal, query your knowledge base by running this command:
aws bedrock-agent-runtime retrieve-and-generate \
  --input '{"text": "What is this document about?"}' \
  --retrieve-and-generate-configuration '{"type": "KNOWLEDGE_BASE", "knowledgeBaseConfiguration": {"knowledgeBaseId": "[[KB_ID="your knowledge base ID"]]", "modelArn": "arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-lite-v1:0"}}'

What does this command do?

  • aws bedrock-agent-runtime retrieve-and-generate calls the RetrieveAndGenerate API, which retrieves relevant chunks from your knowledge base and passes them to a generation model.
  • --input contains your question as a JSON object with a text field.
  • --retrieve-and-generate-configuration tells the API which knowledge base to search and which model to use for generation.
  • The modelArn points to Amazon Nova Lite in us-east-1, the same model you tested with in the console.

After a few seconds, you should see a JSON response in your terminal. The response includes the generated answer and an array of citations.

  • Look at the response structure. You will see two main parts:
  • output contains the text field with the AI-generated answer.
  • citations is an array that lists which document chunks were used to generate the answer, including the S3 URI of each source file.

Why are citations important?

Citations are what make RAG trustworthy. Each answer is grounded in specific document chunks, so you can trace every claim back to a source. This is critical for enterprise use cases where accuracy matters.

Seeing an access denied error?

Make sure you enabled model access for Amazon Nova Lite in the Bedrock console (Step 1). Also verify your CLI credentials have permissions to call Bedrock APIs.

help me fix access denied errors when querying my Bedrock knowledge base from the AWS CLI

You just queried your knowledge base programmatically. In the Secret Mission, you'll take this further by experimenting with different generation models and exploring retrieval-only queries.

Secret mission

Query Programmatically and Compare Models

You've built a working RAG system and tested it through the console. Now take it further by querying your knowledge base programmatically with different generation models, inspecting raw retrieval results, and expanding your document library.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project uses pay-as-you-go AWS services, so costs only accumulate when you actively query the knowledge base or store data in S3.

Cost warning

Your S3 buckets incur minimal storage charges (fractions of a cent for small documents). The knowledge base itself has no idle cost, but every query triggers embedding and generation model charges. If you are not planning to query again, delete the resources to ensure $0 ongoing charges.

Resources you used:

  • Amazon Bedrock Knowledge Base (nextwork-knowledge-base).
  • S3 source documents bucket (nextwork-kb-docs-[unique-id]).
  • S3 Vectors bucket (auto-created by the knowledge base for vector embeddings).
  • Bedrock model access for Titan Text Embeddings V2, Amazon Nova Lite, and Amazon Nova Pro.

Keep everything running

No action needed. Choose this if you are still actively building or plan to query your knowledge base again soon.

  • S3 storage costs approximately $0.023 per GB per month. For a few documents, this is fractions of a cent.
  • The knowledge base has no idle charge. You only pay when you run queries.
  • Model access permissions cost nothing to keep enabled.
  • AWS CLI remains installed on your machine. It does not consume cloud resources.

Pause - I'll come back to this later

This architecture does not require a traditional pause. Resources exist but do not charge unless actively used.

  • Stop querying the knowledge base. No further embedding or generation charges accumulate.
  • Your S3 buckets remain with negligible storage costs (less than $0.01/month for small documents).
  • When you return, your knowledge base is ready to query immediately with no warm-up time.

Delete - I don't want to use this again

Remove all project resources to ensure $0 ongoing charges. Follow each step below.

  • Delete the Bedrock Knowledge Base:
  • Delete the S3 Vectors bucket:
  • Delete your source documents bucket:
  • Model access does not need to be revoked. Enabled models incur no cost when not in use.

That's a Wrap!

That's a Wrap!

Nice work! You just built a fully managed RAG system on AWS using Amazon Bedrock Knowledge Bases. This is the same architecture enterprises use to power internal knowledge assistants, customer support bots, and compliance document systems.

You've learned how to:

  • Built a managed RAG pipeline that automatically chunks, embeds, and indexes documents using Amazon Bedrock Knowledge Bases and S3 Vectors.
  • Used Amazon Titan Text Embeddings V2 to convert documents into semantic vectors and tested retrieval with source citations through the Bedrock console.
  • Queried your knowledge base programmatically using the AWS CLI retrieve-and-generate command and examined the response structure with citations.
  • Compared generation models (Amazon Nova Lite vs Amazon Nova Pro), inspected raw retrieved chunks with relevance scores, and tested multi-document retrieval.

Ready to quiz yourself?