Build a Document Q&A Assistant

Introduction

30 Second Summary

You have documents full of answers that people keep asking you about. Instead of searching through pages every time, what if you could ask an AI that has actually read the whole thing?

In this project, you will build a command-line Document Q&A Assistant using Python and Amazon Bedrock. You feed it any text document, ask questions in plain English, and get accurate answers grounded in that document's content.

What You'll Build

You paste a question into your terminal, and within seconds your assistant pulls the relevant paragraphs from your document and delivers a clear, accurate answer.

By the end of this project, you'll have:

• A working Q&A assistant that answers natural language questions using only the information in your document.
• First-hand proof of the RAG pattern in action. You will watch a foundation model hallucinate without your data, then answer correctly once you feed it context.
• The ability to swap in any text document and start asking questions about it immediately.
• Secret Mission: Add conversation memory so your assistant can handle follow-up questions like "tell me more about that".

Are there any prerequisites?

You need an AWS account with SSO access and Bedrock permissions already set up.

You also need Python 3 installed on your machine and basic familiarity with Python syntax.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters to you. Choosing a document you actually care about will make every step feel relevant.

Set Up Your Environment

Your Q&A assistant needs three things before it can talk to an AI model: a way to authenticate with AWS from the terminal, a Python environment with the right library installed, and permission to use the Claude model on Bedrock.

In this step, you will wire up the AWS CLI for SSO login, create a Python virtual environment with boto3, and enable model access in the Amazon Bedrock console.

In this step, get ready to:

• Configure the AWS CLI for SSO and authenticate.
• Create a Python virtual environment with boto3 installed.
• Enable Amazon Bedrock model access for Claude Sonnet 5.

Configure AWS CLI for SSO

The AWS CLI needs to know how to reach your AWS account. SSO (Single Sign-On) lets you authenticate through your organization's login page instead of managing long-lived access keys.

• Open your terminal. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
• Type Terminal (macOS) or Command Prompt (Windows) and press Enter to open it.
• Confirm the AWS CLI is installed by running:

aws --version

You should see a version number starting with aws-cli/2.

Command not found?

If your terminal does not recognize aws, you need to install the AWS CLI first. Follow the official installation guide at https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html, then return here.

help me install the AWS CLI on my operating system

Now you will run the SSO configuration wizard. This is an interactive command that prompts you for several values. Have your organization's SSO start URL and region ready before you begin.

• Start the SSO configuration wizard by running:

aws configure sso

What will this command ask me?

The wizard prompts you for the following values, one at a time:

• SSO session name - give it any name you like (e.g. my-sso).
• SSO start URL - your organization's SSO portal URL (looks like https://d-xxxxxxxxxx.awsapps.com/start).
• SSO region - the region your Identity Center is in (e.g. us-east-1).
• SSO registration scopes - enter sso:account:access.

After these prompts, the CLI opens your browser for login. Once authenticated, it asks you to select an AWS account and role, then set your CLI default region (enter us-east-1), output format (enter json), and a profile name. Remember the profile name you choose. You will use it throughout this project.

• After the wizard completes, authenticate your session by running:

aws sso login --profile [[PROFILE_NAME="your-profile-name"]]

Your browser opens again for you to confirm the login. Once approved, the CLI caches your credentials.

• Verify your identity by running:

aws sts get-caller-identity --profile [[PROFILE_NAME="your-profile-name"]]

You should see JSON output showing your UserId, Account, and Arn. This confirms your CLI can reach AWS.

Seeing an expired token error?

If you see a message about an expired token, run aws sso login again with your profile flag. SSO sessions expire after a period of inactivity.

help me fix my AWS SSO login issue

Create your Python environment

A virtual environment keeps this project's packages separate from your system Python. You will install boto3 (the AWS SDK for Python) inside it so your script can call Bedrock.

• Move to your Desktop by running this command:

cd ~/Desktop

• Create your project folder and move into it by running these commands:

mkdir bedrock-qa
cd bedrock-qa

• Confirm you are in the new folder by running:

ls

The folder is empty, which is expected. You will add files in later steps.

• Create a Python virtual environment by running:

python3 -m venv bedrock-env

• Activate the virtual environment by running:

macOS

source bedrock-env/bin/activate

Windows

bedrock-env\Scripts\activate

You should see (bedrock-env) appear at the start of your terminal prompt. This confirms the virtual environment is active.

• Install boto3 by running:

pip install boto3

You should see output ending with a line like Successfully installed boto3-... confirming the package is installed.

Seeing permission denied or command not found?

If python3 is not recognized, make sure Python 3 is installed. You can download it from https://www.python.org/downloads/.

If pip install boto3 fails with a permissions error, confirm (bedrock-env) appears in your prompt. If it does not, re-run the activation command above.

help me fix my Python virtual environment setup

Enable Bedrock model access

Even with valid AWS credentials, you cannot call a model until you explicitly enable access to it in the Bedrock console. You will enable Anthropic's Claude Sonnet 5.

• Open the AWS Console in your browser and make sure you are in the US East (N. Virginia) us-east-1 region. You can check the region in the top-right corner of the console.
• In the search bar at the top, type Bedrock and select Amazon Bedrock.
• In the left navigation pane, under Bedrock configurations, click Model access.
• Click Modify model access.
• Find Anthropic in the list and select Claude Sonnet 5.
• Click Next.
• If prompted to submit use case details for Anthropic, click Submit use case details, fill in the form describing your learning project, and click Submit form.
• Review your access changes, read the terms, then click Submit.
• Wait for the status to change to Access granted. This is usually instant but can take a few minutes.

What is a foundation model?

A foundation model is a large AI model pre-trained on massive datasets. Claude Sonnet 5 (model ID: anthropic.claude-sonnet-5) is Anthropic's current flagship model available through Bedrock.

It has a 1M token context window and is priced at $2 per 1M input tokens during its promotional period through August 31, 2026.

Now verify that the CLI can see the model. Before you run this command, what do you expect to see if model access was granted successfully?

• Back in your terminal, verify model access by running:

aws bedrock list-foundation-models --query "modelSummaries[?modelId=='anthropic.claude-sonnet-5'].modelId" --output text --profile [[PROFILE_NAME="your-profile-name"]]

You should see anthropic.claude-sonnet-5 printed in the terminal. This confirms your CLI can authenticate to AWS and that Bedrock model access is active.

Seeing empty output or an access error?

If the output is blank, return to the Bedrock console and confirm the status shows Access granted next to Claude Sonnet 5. Access can take a few minutes to propagate.

If you see an authorization error, confirm you ran the command with the correct --profile flag matching the profile name you set during aws configure sso.

help me troubleshoot Bedrock model access

Your environment is wired up. You have authenticated CLI access to AWS, a clean Python environment with boto3, and permission to call Claude Sonnet 5. Next up, you will make your first AI call and see what a foundation model does when it has no context about your data.

Make Your First AI Call

Your environment is wired up and ready to talk to Amazon Bedrock.

Before you can build a Q&A system, you need to understand what a foundation model actually does when you send it a question. This step gives you a working "hello world" for AI and introduces the core concepts (model, prompt, tokens, response) through hands-on code.

In this step, get ready to:

• Create a Python script that sends a question to Claude Sonnet 5 on Bedrock and prints the response.
• Experiment with prompts and observe hallucination first-hand.

Create the Bedrock hello script

Your first AI script will do the simplest possible thing: send one question to Claude and print the answer. This proves your credentials, region, and model access all work end-to-end.

• In VS Code, create a new file called bedrock_hello.py in your project folder.
• Add the following setup code that imports boto3 and creates a connection to the Bedrock Runtime service:

import boto3
import json

# Create a session using your SSO profile
session = boto3.Session(profile_name="[[PROFILE_NAME=\"your-profile-name\"]]")

# Create the Bedrock Runtime client
client = session.client("bedrock-runtime", region_name="us-east-1")

# The model ID for Claude Sonnet 5 on Bedrock
model_id = "anthropic.claude-sonnet-5"

What does this code do?

• boto3.Session(profile_name=...) creates a session that uses the SSO profile you configured in Step 1. This is how boto3 picks up your credentials without hard-coding secrets.
• session.client("bedrock-runtime", region_name="us-east-1") creates a client specifically for the Bedrock inference API in the US East 1 region.
• model_id stores the exact identifier Bedrock uses to route your request to Claude Sonnet 5.

• Now add the code that sends a question to the model and prints the response. Paste this below the model_id line:

# Send a simple question to the model
response = client.converse(
    modelId=model_id,
    messages=[
        {
            "role": "user",
            "content": [{"text": "What is Amazon Bedrock and why would a developer use it?"}],
        }
    ],
    inferenceConfig={"maxTokens": 512, "temperature": 0.5},
)

# Extract and print the response text
output_text = response["output"]["message"]["content"][0]["text"]
print(output_text)

What does this code do?

• client.converse() is the Converse API. AWS recommends it as the unified way to talk to any model on Bedrock with the same code structure.
• messages holds the conversation. Each message has a role ("user" or "assistant") and content (the text you send).
• maxTokens caps the response length. Tokens are chunks of text (roughly 3/4 of a word each). 512 tokens gives a concise answer.
• temperature controls randomness. 0.5 balances creativity and consistency.

Before you run this, what do you think Claude will respond with when asked about Amazon Bedrock?

• Save bedrock_hello.py.
• Run the script by entering this command in your terminal:

python3 bedrock_hello.py

You should see Claude respond with a plain-text answer explaining what Amazon Bedrock is. This confirms your SSO credentials, region, and model access all work correctly.

Seeing an error instead of a response?

• If you see ExpiredTokenException or a credentials error, your SSO session may have timed out. Run aws sso login --profile your-profile-name to re-authenticate, then try again.
• If you see AccessDeniedException, make sure you enabled model access for Claude Sonnet 5 in the Bedrock console. Access requests are usually instant but can take a few minutes.
• If the profile name is wrong, check the exact name you used during aws configure sso in Step 1.

Help me troubleshoot my Bedrock API call. I'm getting an error when running bedrock_hello.py with boto3 and SSO credentials.

✔️ Awesome, I've got everything!

Great. Double check you have saved bedrock_hello.py and move on to the next substep.

ⓧ I'd like to double check the full code

import boto3
import json

# Create a session using your SSO profile
session = boto3.Session(profile_name="[[PROFILE_NAME=\"your-profile-name\"]]")

# Create the Bedrock Runtime client
client = session.client("bedrock-runtime", region_name="us-east-1")

# The model ID for Claude Sonnet 5 on Bedrock
model_id = "anthropic.claude-sonnet-5"

# Send a simple question to the model
response = client.converse(
    modelId=model_id,
    messages=[
        {
            "role": "user",
            "content": [{"text": "What is Amazon Bedrock and why would a developer use it?"}],
        }
    ],
    inferenceConfig={"maxTokens": 512, "temperature": 0.5},
)

# Extract and print the response text
output_text = response["output"]["message"]["content"][0]["text"]
print(output_text)

Experiment with your prompt and observe hallucination

You just proved the model can answer general knowledge questions accurately. Now try asking it something about YOUR specific document or company topic.

• In bedrock_hello.py, find this line inside the messages list:

"content": [{"text": "What is Amazon Bedrock and why would a developer use it?"}],

• Replace the question text with a specific question about the document topic you chose in Step 0. For example, if your document is a company FAQ, ask something like "What is our company's refund policy?" The line should now look like this:

"content": [{"text": "[[YOUR_QUESTION=\"a specific question about your document topic\"]]" }],

Before you run this, do you think Claude will know the correct answer to a question about YOUR specific document? Why or why not?

• Save bedrock_hello.py and run it again:

python3 bedrock_hello.py

Claude responds confidently with an answer that sounds plausible. But if you check against your actual document, you will see the information is wrong or completely made up.

What is hallucination?

This is hallucination. The model generates plausible-sounding but incorrect information because it has never seen your document. It fills the gap with patterns from its training data instead of admitting it does not know.

This is the exact problem that retrieval-augmented generation (RAG) solves. In the next steps, you will feed your document to the model so it answers from real data instead of guessing.

• Copy the hallucinated answer and save it somewhere (a text file or a sticky note). You will compare it to a grounded answer in Step 4 to see the difference RAG makes.

Save the hallucinated answer

Paste the hallucinated response into a note or comment at the top of your file. Seeing the before-and-after in Step 4 is the most convincing proof that RAG works.

You now have a working AI call and first-hand evidence of the hallucination problem. Next up, you will load your document and build a retriever that finds the relevant pieces to feed to the model.

Load a Document and Build the Retriever

In the last step, you asked Claude about your document's topic and it answered confidently with made-up information. That hallucination happened because the model has never seen your actual data.

To fix this, you need a way to find the relevant parts of your document and feed them to the model as context. This step builds the "retrieval" half of the RAG pattern: load a document, split it into chunks, and search those chunks for the ones most relevant to any question.

In this step, get ready to:

• Create a knowledge base document with Q&A pairs about a topic you care about.
• Build a retriever that splits text into chunks and finds the most relevant ones for any question.
• Test the retriever and confirm it returns accurate chunks from your document.

Create your knowledge base document

Your assistant needs a document to answer questions about. This is the knowledge base that will ground the model's responses in real information instead of hallucinations.

• Create a new file called company_faq.txt in the same folder as bedrock_hello.py.
• Write 15-20 Q&A pairs or paragraphs about a topic you care about (your company's FAQ, a product runbook, a policy document, or any subject you want to query).
• Separate each entry with a blank line between them so the chunker can split on paragraph boundaries.

Here is an example of the format your file should follow:

Q: What are our office hours?
A: Our office hours are Monday through Friday, 9am to 5pm EST. Support tickets submitted outside of these hours will be addressed the next business day.

Q: How do I reset my password?
A: Visit the login page and click "Forgot Password". Enter your email address and you will receive a reset link within 5 minutes. The link expires after 24 hours.

Q: What is the refund policy?
A: We offer full refunds within 30 days of purchase. After 30 days, we provide store credit for the remaining subscription period. Contact support@example.com to initiate a refund.

What makes a good knowledge base?

Pick a topic you actually want to ask questions about. The more specific your entries, the better the retriever can match questions to answers.

Aim for entries that are 2-4 sentences each. Very short entries (a single word) give the keyword matcher too little to work with.

• Write your own 15-20 entries following the format above, then save company_faq.txt.
• Confirm the file exists by running:

ls company_faq.txt

You should see company_faq.txt printed in the output.

File not found?

Make sure you saved the file in the same directory where bedrock_hello.py lives. Run ls to see all files in your current folder and confirm you are in the right place.

help me find where my company_faq.txt file was saved

Build the document loader and chunker

Now you will build the retriever script that processes your document. It starts by reading the file and splitting the text into manageable pieces called chunks.

• Create a new file called retriever.py and add the import and first two functions by pasting this code:

import sys


def load_document(file_path):
    """Load a text document and return its content."""
    with open(file_path, "r") as f:
        return f.read()


def chunk_document(text, chunk_size=500):
    """Split document into chunks by paragraphs, then by size if needed."""
    paragraphs = [p.strip() for p in text.split("\n\n") if p.strip()]

    chunks = []
    for para in paragraphs:
        if len(para) <= chunk_size:
            chunks.append(para)
        else:
            # Split long paragraphs into smaller pieces
            words = para.split()
            current_chunk = ""
            for word in words:
                if len(current_chunk) + len(word) + 1 > chunk_size:
                    chunks.append(current_chunk.strip())
                    current_chunk = word
                else:
                    current_chunk += " " + word
            if current_chunk.strip():
                chunks.append(current_chunk.strip())

    return chunks

What does this code do?

load_document() reads your entire text file into a single string.

chunk_document() splits that string on double newlines (paragraph breaks). If any paragraph is longer than 500 characters, it gets split further at word boundaries. This keeps each chunk small enough to fit in a model's prompt alongside other context.

• Save retriever.py and test that the loader and chunker work by running:

python3 -c "from retriever import load_document, chunk_document; chunks = chunk_document(load_document('company_faq.txt')); print(f'{len(chunks)} chunks created')"

You should see a number printed, like 15 chunks created or 20 chunks created, depending on how many entries you wrote.

Seeing 0 chunks or a FileNotFoundError?

If you get 0 chunks, check that your company_faq.txt entries are separated by blank lines (double newlines). The chunker splits on \n\n, so entries on consecutive lines without a gap will be treated as one giant paragraph.

If you see a FileNotFoundError, confirm both files are in the same directory by running ls.

help me debug why my retriever cannot find or chunk my document

Add the search function and test the retriever

The chunker breaks your document into pieces. Now you need a search function that scores those pieces against a question and returns the most relevant ones.

• In retriever.py, add the search function below the chunk_document function:

def find_relevant_chunks(question, chunks, top_n=3):
    """Find the most relevant chunks using simple keyword matching."""
    question_words = set(question.lower().split())
    # Remove common stop words
    stop_words = {"what", "is", "the", "a", "an", "how", "does", "do", "can", "i", "my", "to", "and", "or", "of", "in", "for", "on", "with"}
    question_words = question_words - stop_words

    scored_chunks = []
    for chunk in chunks:
        chunk_words = set(chunk.lower().split())
        # Score = number of question words found in the chunk
        score = len(question_words & chunk_words)
        scored_chunks.append((score, chunk))

    # Sort by score descending and return top N
    scored_chunks.sort(key=lambda x: x[0], reverse=True)
    return [chunk for score, chunk in scored_chunks[:top_n] if score > 0]

What does this code do?

find_relevant_chunks() takes a question and a list of chunks, then scores each chunk by counting how many question keywords appear in it. It strips out common stop words (like "what", "is", "the") so only meaningful terms drive the match.

This is a basic keyword retriever. Production systems use vector embeddings for semantic search, but keyword matching teaches the same core concept: find the relevant pieces before calling the model.

• Now add the main block at the bottom of retriever.py so you can run it from the command line:

if __name__ == "__main__":
    if len(sys.argv) < 2:
        print("Usage: python3 retriever.py \"your question here\"")
        sys.exit(1)

    question = sys.argv[1]
    document = load_document("company_faq.txt")
    chunks = chunk_document(document)

    print(f"Document loaded: {len(chunks)} chunks created\n")

    relevant = find_relevant_chunks(question, chunks)

    if relevant:
        print(f"Found {len(relevant)} relevant chunk(s):\n")
        for i, chunk in enumerate(relevant, 1):
            print(f"--- Chunk {i} ---")
            print(chunk)
            print()
    else:
        print("No relevant chunks found for your question.")

What does this code do?

The if __name__ == "__main__" block makes your script runnable from the terminal. It takes a question as a command-line argument, loads and chunks your document, runs the search, and prints the matching chunks.

This is the full retrieval pipeline: load, chunk, search, display. No AI call yet. Just finding the right pieces of your document.

• Save retriever.py.

Before you run this, what do you think the output will look like? Do you expect it to find chunks related to your question, or return nothing?

• Test the retriever with a question that matches something in your document by running:

python3 retriever.py "What is the refund policy?"

You should see 2-3 relevant text chunks from your document printed to the terminal, with no AI call involved. The output looks like this:

Document loaded: 18 chunks created

Found 2 relevant chunk(s):

--- Chunk 1 ---
Q: What is the refund policy?
A: We offer full refunds within 30 days of purchase...

--- Chunk 2 ---
...

Try different questions

Run the script with a few different questions. Try one that clearly matches your document content and one that does not match at all. For out-of-scope questions, you should see "No relevant chunks found for your question."

If the retriever returns irrelevant chunks, try adding more specific keywords to your document entries or adjusting your question wording.

Seeing no relevant chunks for a question that should match?

The keyword matcher only finds exact word overlaps. If your question uses "password" but your document says "credentials", there is no match. Try rephrasing your question to use words that actually appear in your document.

Also check that your question is wrapped in quotes when you pass it on the command line. Without quotes, each word becomes a separate argument and only the first word is used.

help me debug why my retriever is not finding relevant chunks

✔️ Awesome, I've got everything!

Great. Double check you have saved retriever.py and that it runs without errors.

ⓧ I'd like to double check the full code

Your retriever.py should contain these sections in order. Scroll up and confirm each is present:

• The import sys line at the top.
• The load_document() function that opens and reads a file.
• The chunk_document() function that splits text by paragraphs and by size.
• The find_relevant_chunks() function that scores chunks by keyword overlap.
• The if __name__ == "__main__" block that wires everything together for command-line usage.

You now have a working retrieval system that can find relevant information in any document. Next up, you will combine this retriever with Claude to build the full RAG pipeline. The model will finally answer from your data instead of making things up.

Combine Retrieval and Generation

In Step 2, you saw Claude answer questions confidently even when it had no access to your data. In Step 3, you built a retriever that finds the relevant pieces of your document.

Now you combine both halves into one script. The retriever finds the right context, then Claude generates an answer from that context instead of making things up. This is the RAG pattern (retrieval-augmented generation) used in production AI systems.

In this step, get ready to:

• Create a RAG assistant script that retrieves context and generates answers.
• Add an interactive question loop so you can keep asking without restarting.
• Test the hallucination guard with in-document and out-of-scope questions.

Create the RAG assistant setup

Your new script needs the same Bedrock connection from bedrock_hello.py, plus the retriever functions from retriever.py. Instead of duplicating code, you import the retriever functions directly.

• Create a new file called rag_assistant.py in the same folder as your other scripts.
• Add the setup code that connects to Bedrock and loads your document by pasting the following:

import boto3
from retriever import load_document, chunk_document, find_relevant_chunks

# Create a session using your SSO profile
session = boto3.Session(profile_name="your-profile-name")

# Create the Bedrock Runtime client
client = session.client("bedrock-runtime", region_name="us-east-1")

# The model ID for Claude Sonnet 5 on Bedrock
model_id = "anthropic.claude-sonnet-5"

# Load and chunk the document once at startup
document = load_document("company_faq.txt")
chunks = chunk_document(document)
print(f"Document loaded: {len(chunks)} chunks ready for retrieval.\n")

What does this code do?

• from retriever import ... pulls in the three functions you built in the previous step, so you reuse them without copying code.
• The Bedrock session and client setup is the same pattern from bedrock_hello.py.
• Loading and chunking the document at startup means it only happens once, not on every question.

• Save rag_assistant.py and test the setup by running:

python3 rag_assistant.py

You should see something like Document loaded: 18 chunks ready for retrieval. (the number depends on your document size). The script then exits because there is nothing else to run yet.

Seeing an ImportError?

• Make sure retriever.py is in the same folder as rag_assistant.py. Python looks for imports in the current directory.
• Confirm your virtual environment is still active. You should see (bedrock-env) in your terminal prompt.

Still stuck? Help me fix an ImportError when running rag_assistant.py

Build the question-answering function

This function is the core of the RAG pattern. It retrieves relevant chunks, builds a prompt with context and instructions, then sends everything to Claude.

• In rag_assistant.py, below the print line, add the first part of the ask_question function that handles retrieval and prompt construction:

def ask_question(question):
    """Retrieve relevant context and generate an answer."""
    # Step 1: Retrieve relevant chunks
    relevant_chunks = find_relevant_chunks(question, chunks)

    if not relevant_chunks:
        return "I could not find relevant information in the document to answer your question."

    # Step 2: Build the context string
    context = "\n\n".join(relevant_chunks)

    # Step 3: Construct the prompt with context and instructions
    system_prompt = (
        "You are a helpful assistant that answers questions based only on the provided context. "
        "If the answer is not in the context, say 'I don't have enough information in the document to answer that.' "
        "Do not make up information. Keep answers concise and helpful."
    )

    user_message = f"Context from the document:\n\n{context}\n\nQuestion: {question}"

What does this code do?

• find_relevant_chunks() searches your document for paragraphs that match the question. If nothing matches, it returns early with a safe message.
• The system_prompt instructs Claude to answer only from the provided context and say it does not know when the answer is not there. This is your hallucination guard.
• The user_message packages the retrieved chunks and the user's question into a single prompt for the model.

• Still inside the ask_question function, add the model call below the user_message line:

    # Step 4: Call the model
    response = client.converse(
        modelId=model_id,
        messages=[
            {
                "role": "user",
                "content": [{"text": user_message}],
            }
        ],
        system=[{"text": system_prompt}],
        inferenceConfig={"maxTokens": 1024, "temperature": 0.3},
    )

    return response["output"]["message"]["content"][0]["text"]

What does this code do?

• client.converse() sends the prompt to Claude via the Converse API. The system parameter sets the model's behaviour rules separately from the user message.
• temperature is set to 0.3 (lower than the 0.5 in your hello script) because factual Q&A benefits from less creative randomness.
• The return line extracts the text content from the nested response structure.

• Save rag_assistant.py and run it to confirm no syntax errors:

python3 rag_assistant.py

You should see the same Document loaded: X chunks ready for retrieval. message with no traceback. The function exists now but nothing calls it yet, so the script still exits cleanly.

Seeing a SyntaxError or IndentationError?

• Make sure the second code block (the model call) is indented inside the ask_question function. Every line should start with four spaces of indentation.
• Check that user_message appears on the line directly above the model call block. There should be no gap in the function body.

Help me fix a SyntaxError in my ask_question function

Add an interactive question loop

Right now you would need to restart the script for every question. A simple input loop transforms this into a usable assistant where you type questions and get answers continuously.

• At the bottom of rag_assistant.py, below the ask_question function, add the main() function and entry point:

def main():
    """Run the interactive Q&A loop."""
    print("Document Q&A Assistant")
    print("Ask questions about your document. Type 'quit' to exit.\n")

    while True:
        question = input("You: ").strip()

        if not question:
            continue
        if question.lower() == "quit":
            print("Goodbye!")
            break

        answer = ask_question(question)
        print(f"\nAssistant: {answer}\n")


if __name__ == "__main__":
    main()

What does this code do?

• while True keeps the loop running until the user types quit.
• input("You: ") pauses and waits for the user to type a question. Empty input is skipped.
• if __name__ == "__main__" ensures main() only runs when you execute this file directly, not when other scripts import from it.

• Save rag_assistant.py.

Before you run this, what do you think will happen when you ask a question that is clearly answered in your document?

• Run your RAG assistant:

python3 rag_assistant.py

You should see the document loaded message, then a You: prompt waiting for input. Type a question that your document clearly answers and press Enter.

The assistant responds with an accurate answer drawn from your document content. Type quit to exit when you are done exploring.

Getting a credentials error?

• Your SSO session may have expired. Run aws sso login --profile your-profile-name to re-authenticate, then try again.
• Make sure the profile_name in rag_assistant.py matches the exact profile name you set up with aws configure sso.

Help me fix a credentials or access error when calling Bedrock from rag_assistant.py

Test the hallucination guard

The real test of your RAG system is whether it stays honest. In Step 2, Claude hallucinated freely about topics it had no data for. Now the system prompt instructs it to refuse when the answer is not in your document.

• Run the assistant again:

python3 rag_assistant.py

• Ask a question that IS clearly answered in your document. Confirm the assistant gives an accurate, grounded answer.
• Ask a question that is NOT in your document (for example, something about a completely different topic). Confirm the assistant responds with something like "I don't have enough information in the document to answer that."
• Compare this to the hallucinated answer you saved from Step 2. The difference is the RAG pattern at work.
• Type quit to exit.

Why does this stop hallucination?

Without context, Claude fills gaps with plausible-sounding information it learned during training. Your system prompt explicitly tells it to say "I don't know" when the answer is not in the provided context.

The retrieved chunks narrow the model's attention to only the relevant paragraphs from your document. Combined with the instruction to stay within that context, this creates a reliable hallucination guard.

✔️ Awesome, I've got everything!

Great. Double check you have saved rag_assistant.py and confirmed both in-document and out-of-scope questions work as expected.

ⓧ I'd like to double check the full code

Here is the complete rag_assistant.py file. Compare it against yours to find any differences:

Your file has three sections built across this step. Scroll through and confirm each one is present, in order:

• Setup block (imports, session, client, model_id, document loading, print statement).
• The ask_question() function (retrieval, context building, system prompt, user message, model call, return).
• The main() function and if __name__ == "__main__" entry point.

If something does not match, focus on indentation and the exact parameter names in the client.converse() call.

Your Document Q&A Assistant is working. You have a complete RAG pipeline: retrieval finds the context, generation produces the answer, and the system prompt keeps the model honest. Next up, the Secret Mission adds conversation memory so you can ask follow-up questions like "tell me more about that."

Secret mission

Add Conversation Memory

Right now, every question you ask is independent. The model forgets everything between exchanges. Try asking a follow-up like "tell me more about that" and watch it fail. Real assistants remember context. Your mission: add conversation memory so your assistant supports natural follow-up questions.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally with pay-per-use Bedrock calls, so there are no ongoing costs unless you run the script.

Resources you used:

• Python virtual environment (bedrock-env) with boto3 installed.
• Project files: bedrock_hello.py, retriever.py, rag_assistant.py, and company_faq.txt.
• Amazon Bedrock model access for Claude Sonnet 5 (no ongoing cost, pay-per-use only).
• AWS CLI SSO profile configuration (local config file, no ongoing cost).

Keep everything running

No action needed. Choose this if you plan to keep experimenting with your Q&A assistant or want to swap in a different document.

• Your local Python files and virtual environment stay on your machine at zero cost.
• Bedrock charges only when you actually run the script and make API calls.
• Your SSO session will expire on its own after a period of inactivity.

Pause - I'll come back to this later

There are no running services to pause. Your files sit on disk and Bedrock is pay-per-use, so nothing accumulates cost while idle.

• Deactivate your virtual environment to return your terminal to its normal state by running:

deactivate

• Your project files remain on disk and you can reactivate the environment later with source bedrock-env/bin/activate (macOS/Linux) or bedrock-env\Scripts\activate (Windows).

Delete - I don't want to use this again

Remove all project files and the virtual environment from your machine. Your SSO session expires automatically.

• Deactivate your virtual environment first by running:

deactivate

• Remove the project files and virtual environment.

macOS

rm bedrock_hello.py retriever.py rag_assistant.py company_faq.txt
rm -rf bedrock-env

Windows

del bedrock_hello.py, retriever.py, rag_assistant.py, company_faq.txt
rmdir /s /q bedrock-env

• Bedrock model access is free to keep enabled. If you want to disable it, navigate to the Amazon Bedrock console, select Model access in the sidebar, and disable access to Anthropic Claude Sonnet 5.

That's a Wrap!

That's a Wrap!

Nice work. You just built a fully functional Document Q&A Assistant powered by Amazon Bedrock and Claude Sonnet 5. You can now point it at any text document and get accurate, grounded answers in seconds.

You've learned how to:

• Built a working Document Q&A Assistant that answers natural language questions using only information from your document, proving the power of the RAG pattern in action.
• Implemented retrieval-augmented generation from scratch: chunking a document into searchable pieces, retrieving relevant context with keyword matching, injecting that context into a prompt, and generating grounded answers.
• Observed hallucination first-hand by asking the model questions it could not answer from data, then eliminated it by providing real document context.
• Secret Mission: Added conversation memory so your assistant remembers prior exchanges and supports multi-turn follow-up questions.

Ready to quiz yourself?