Build a RAG Document Q&A App

Build a RAG pipeline that answers questions about your documents.

Introduction

30 Second Summary

Think about how frustrating it is to search through long documents for a specific answer. You read pages of text just to find one paragraph that actually answers your question.

In this project, you will build a web app that reads your documents, understands them, and answers your questions using only the information they contain. It retrieves the most relevant passages and generates answers with citations so you always know where the information came from.

What You'll Build

A RAG-powered document Q&A application that chunks documents into passages, stores them as vector embeddings in a local database, and uses an LLM to generate cited answers from the most relevant passages.

By the end of this project, you'll have:

  • A document ingestion pipeline that chunks text and embeds it into a ChromaDB vector store.
  • A RAG query engine that retrieves relevant context and generates grounded answers via Amazon Bedrock (Nova Lite).
  • A Streamlit chat interface with expandable source citations, pushed to GitHub as a showcase-ready portfolio piece.
  • Secret Mission: Add a relevance scoring system that shows confidence levels for each retrieved passage.

Are there any prerequisites?

You need basic programming knowledge, an AWS account with credits, and a GitHub account. This project uses Python 3.10+ and runs on Linux.

Set Up Python, Git, and AWS CLI

Before you can build anything, your machine needs the right tools installed and connected to the cloud.

In this step, you will install Python 3, Git, and the AWS CLI. You will also enable the specific AI models your app needs inside the Amazon Bedrock console.

In this step, get ready to:
  • Install Python 3 and Git, then verify both are working.
  • Install the AWS CLI and configure your credentials.
  • Enable model access for Amazon Nova Lite and Titan Text Embeddings V2 in the Bedrock console.
Install Python 3 and Git

Python is the language your RAG app is written in. Git tracks your code changes and lets you push to GitHub later.

  • Check if Python 3 is already installed by running this command in your terminal:
python3 --version

✔️ I see a version number

You should see something like Python 3.10.x or higher. Python 3.10+ is required for this project.

ⓧ Command not found

  • Install Python 3 by running these commands:
sudo apt update
sudo apt install python3 python3-pip python3-venv -y
  • Verify the installation by running python3 --version again. You should see a version number of 3.10 or higher.
  • Check if Git is already installed by running:
git --version

✔️ I see a version number

You should see something like git version 2.x.x. Any recent version works for this project.

ⓧ Command not found

  • Install Git by running:
sudo apt install git -y
  • Verify the installation by running git --version again. You should see a version number.

Still not seeing a version number?

Try closing your terminal and opening a new one. Some installs require a fresh terminal session to update your PATH.

Help me troubleshoot my Python or Git installation on Linux.

Install and configure the AWS CLI

The AWS CLI lets your local machine talk to AWS services like Bedrock. You need it configured with credentials so your Python code can call the AI models.

  • Check if the AWS CLI is already installed by running:
aws --version

✔️ I see a version number

You should see something like aws-cli/2.x.x. You are ready to configure it.

ⓧ Command not found

  • Download and install the AWS CLI v2 by running these commands:
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

What do these commands do?

The first command downloads the official AWS CLI installer. The second unzips it. The third installs it system-wide so you can run aws from anywhere.

  • Verify the installation by running aws --version. You should now see a version number.
  • Configure your AWS credentials by running:
aws configure
  • The command prompts you for four values. Enter your AWS Access Key ID when prompted.
  • Enter your AWS Secret Access Key when prompted.
  • Enter us-east-1 for the Default region name.
  • Enter json for the Default output format.

Where do I find my AWS access keys?

Log into the AWS Console, click your username in the top right, and select Security credentials. Under Access keys, click Create access key.

Copy both the Access Key ID and Secret Access Key immediately. You will not be able to see the secret key again after closing the dialog.

  • Verify your credentials are working by running:
aws sts get-caller-identity

You should see a JSON response with your AWS account ID, user ARN, and user ID. This confirms your credentials are stored and valid.

Seeing an access denied error?

Double-check you copied the full access key and secret key without extra spaces. Run aws configure again to re-enter them.

Help me fix my AWS CLI credentials configuration.

Enable model access in Amazon Bedrock

Your app uses two AI models from AWS. You need to make sure they are enabled in the Bedrock console before your code can call them.

  • Open the AWS Management Console in your browser.
  • Search for Bedrock in the top search bar and select Amazon Bedrock.
  • Make sure your region is set to US East (N. Virginia) in the top right corner of the console.
  • In the left sidebar, click Model access.
  • Click Manage model access.
  • Find and enable Amazon Nova Lite and Titan Text Embeddings V2 in the model list.
  • Click Request model access to save your selection.

Why these two models?

Amazon Nova Lite is the LLM that generates answers from your documents. It costs $0.06 per million input tokens, which makes it extremely affordable for a learning project.

Titan Text Embeddings V2 converts text into numerical vectors that ChromaDB can search. It costs $0.02 per million tokens. Together, running this entire project costs less than a few cents.

Amazon models enable quickly

Amazon's own models (Nova, Titan) are typically auto-enabled with correct AWS Marketplace permissions. Access is granted within seconds, not hours. If you see the status as Access granted already, you are all set.

  • Confirm both models show Access granted in the status column before moving on.

Model access still pending?

Amazon first-party models usually enable instantly. If the status has not changed after a minute, try refreshing the page. Make sure you are in the us-east-1 region.

Help me enable model access for Amazon Nova Lite and Titan Embeddings in the Bedrock console.

Your development environment is ready. You have Python for writing the app, Git for version control, and a configured AWS CLI that can reach Bedrock. Next up, you will create the project structure and install the Python packages that power your RAG pipeline.

Create Project, Git Repo, and Install Dependencies

Your development tools are installed and configured. Now it's time to create the actual project that will become your RAG application.

Every Python project needs a clean foundation: a dedicated folder, version control, isolated dependencies, and a clear file structure. You'll set all of that up in this step so you can focus on building the RAG logic next.

In this step, get ready to:
  • Create a project folder with Git version control and a Python virtual environment.
  • Install the Python packages your RAG app needs (boto3, ChromaDB, Streamlit).
  • Set up the file structure for your project.
Create the project folder, Git repo, and virtual environment

A virtual environment gives your project its own isolated set of Python packages. This prevents conflicts with other projects on your machine.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder, move into it, and initialize a Git repository by running these commands:
mkdir rag-document-qa
cd rag-document-qa
git init

What do these commands do?

  • mkdir rag-document-qa creates a new folder for your project.
  • cd rag-document-qa moves you inside that folder.
  • git init turns the folder into a Git repository so you can track changes and push to GitHub later.
  • Confirm the folder was created and Git initialized by running:
ls -la

You should see a .git directory in the output. This means Git is tracking the folder.

Don't see the .git folder?

Make sure you ran git init while inside the rag-document-qa folder, not from your Desktop. Run pwd to check your current directory.

help me troubleshoot why git init isn't working in my project folder

  • Create and activate a Python virtual environment by running these commands:
python3 -m venv venv
source venv/bin/activate

What do these commands do?

  • python3 -m venv venv creates a virtual environment in a folder called venv/.
  • source venv/bin/activate activates the virtual environment so any packages you install go here instead of your system Python.

After activation, you should see (venv) at the beginning of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

Make sure you used source before the path. Running venv/bin/activate alone without source will not activate the environment.

Also confirm the venv/ folder exists by running ls. If it does not exist, re-run python3 -m venv venv.

help me activate my Python virtual environment on Linux

Add the .gitignore and install dependencies

A .gitignore file tells Git which files and folders to ignore. You don't want to push your virtual environment, local database files, or secrets to GitHub.

  • Open your project folder in VS Code by running:
code .

Command not found?

If code is not recognized, open VS Code manually. Press the **Super** key (Windows key), type VS Code, and press **Enter**. Then click **File**, select **Open Folder**, and navigate to ~/Desktop/rag-document-qa.

  • In the VS Code Explorer sidebar, click the new file icon (or press **Ctrl+N**).
  • Name the file .gitignore and paste in the following content:
# Python
__pycache__/
*.py[cod]
*$py.class
*.so

# Virtual environment
venv/
.venv/

# ChromaDB local storage
chroma_db/

# Environment variables
.env

# IDE
.vscode/
*.swp
*.swo

# OS
.DS_Store
Thumbs.db

What does this file do?

  • venv/ and .venv/ exclude your virtual environment (hundreds of library files that shouldn't live in a repo).
  • chroma_db/ excludes the local vector database that gets generated when you ingest documents.
  • .env keeps any future secrets or environment variables out of version control.
  • __pycache__/ and compiled Python files are auto-generated and never belong in a repository.
  • Save the file with **Ctrl+S**.
  • Confirm .gitignore appears in the VS Code Explorer sidebar on the left.
  • Install the three packages your RAG app depends on by running this command in your terminal:
pip install boto3 chromadb streamlit

What are these packages?

  • boto3 is the AWS SDK for Python. You'll use it to call Amazon Bedrock for embeddings and text generation.
  • chromadb is a local vector database. It stores your document embeddings and handles similarity search.
  • streamlit is a Python framework for building web apps. It gives you a chat interface with zero frontend code.

The install may take a minute or two. When it finishes, you should see Successfully installed followed by a list of packages in your terminal output.

Seeing a permission error or install failure?

Make sure your virtual environment is active. You should see (venv) in your prompt. If not, run source venv/bin/activate first.

If you see an error about SQLite version (common with ChromaDB), you may need to install pysqlite3-binary. ChromaDB requires SQLite version 3.35 or higher.

help me fix my pip install error

  • Create a requirements.txt file to record your project's dependencies. In VS Code, create a new file called requirements.txt and paste this content:
boto3
chromadb
streamlit

Why not use pip freeze?

Running pip freeze dumps every installed package including sub-dependencies, which creates a long and noisy file. Listing only your direct dependencies keeps things clean and readable.

  • Save the file with **Ctrl+S**.
Create the project file structure

Your RAG application will be split across three Python files and a folder for documents. Creating empty files now gives you a clear map of what you'll build in upcoming steps.

  • Create the empty project files and the documents folder by running these commands in your terminal:
touch app.py rag.py ingest.py README.md
mkdir documents

What is each file for?

  • app.py will hold your Streamlit chat interface.
  • rag.py will contain the retrieval and generation logic that powers your Q&A.
  • ingest.py will handle loading documents, chunking them, and storing embeddings.
  • README.md will describe your project for anyone viewing it on GitHub.
  • documents/ is where you'll place the text files your RAG system will answer questions about.
  • Confirm all files and folders exist by running:
ls

You should see app.py, documents, ingest.py, rag.py, README.md, requirements.txt, and venv in the output.

Missing a file?

Make sure you ran the touch and mkdir commands from inside the rag-document-qa folder. Run pwd to check that your current directory ends with rag-document-qa.

If .gitignore is missing from the ls output, that's expected. Files starting with a dot are hidden by default. Run ls -a to see them.

help me check my project file structure

✔️ Awesome, I've got everything!

Great. Double check you've saved both .gitignore and requirements.txt in VS Code before moving on.

ⓧ I'd like to double check the full code

Here's what your .gitignore should look like:

# Python
__pycache__/
*.py[cod]
*$py.class
*.so

# Virtual environment
venv/
.venv/

# ChromaDB local storage
chroma_db/

# Environment variables
.env

# IDE
.vscode/
*.swp
*.swo

# OS
.DS_Store
Thumbs.db

And here's your requirements.txt:

boto3
chromadb
streamlit

Your project skeleton is ready. Next up, you'll fill those empty files with real code, starting with the document chunking pipeline that breaks your text into bite-sized passages for the vector database.

Build the Document Chunking Pipeline

RAG systems cannot search through entire documents at once. They need documents broken into smaller passages so each passage can be compared to a user's question individually.

In this step, you'll create sample documents and write a chunking function that splits them into overlapping passages. These passages become the building blocks that your vector database will search through later.

In this step, get ready to:
  • Create three sample markdown documents as your knowledge base.
  • Write a chunking function that splits documents into overlapping passages.
  • Test the pipeline to verify chunks are created correctly.
Create sample documents for your knowledge base

Your RAG app needs documents to search through. You'll create three markdown files covering different Python topics. These give the system enough content to demonstrate meaningful retrieval later.

  • In VS Code, right-click the documents folder in the Explorer sidebar.
  • Click New File and name it python_basics.md.
  • Paste the following content into python_basics.md:
# Python Basics

## Variables and Data Types

Python uses dynamic typing, which means you don't need to declare variable types explicitly. Variables are created when you first assign a value to them. Common data types include integers (int), floating-point numbers (float), strings (str), and booleans (bool).

## Lists and Dictionaries

Lists are ordered, mutable collections that can hold items of different types. You create them with square brackets: my_list = [1, 2, 3]. Dictionaries store key-value pairs and are created with curly braces: my_dict = {"name": "Alice", "age": 30}. Both are fundamental data structures used in nearly every Python program.

## Functions

Functions are defined using the def keyword followed by the function name and parameters in parentheses. They allow you to organize code into reusable blocks. Functions can return values using the return statement, and they support default parameter values and keyword arguments.

## Error Handling

Python uses try/except blocks for error handling. You wrap code that might raise an exception in a try block, and handle specific exceptions in except blocks. The finally block runs regardless of whether an exception occurred, making it useful for cleanup operations.
  • Create a second file in the documents folder called python_advanced.md and paste this content:
# Advanced Python Concepts

## Decorators

Decorators are functions that modify the behavior of other functions. They use the @symbol syntax placed above a function definition. Common use cases include logging, timing, authentication, and caching. A decorator takes a function as input, wraps it with additional functionality, and returns the modified function.

## Generators

Generators are special functions that use the yield keyword instead of return. They produce values lazily, one at a time, which makes them memory-efficient for processing large datasets. Generator expressions provide a concise syntax similar to list comprehensions but with parentheses instead of brackets.

## Context Managers

Context managers handle resource management using the with statement. They ensure resources like files, network connections, and database connections are properly opened and closed. You can create custom context managers using the contextlib module or by implementing __enter__ and __exit__ methods.

## Async Programming

Async programming in Python uses async/await syntax to handle concurrent operations efficiently. The asyncio module provides the event loop and tools for writing asynchronous code. This is particularly useful for I/O-bound tasks like web requests, file operations, and database queries where waiting for responses would otherwise block execution.
  • Create a third file in the documents folder called python_web.md and paste this content:
# Python for Web Development

## Flask Framework

Flask is a lightweight web framework that provides the essentials for building web applications. It uses decorators to map URLs to Python functions called views. Flask supports templates via Jinja2, form handling, and can be extended with numerous plugins for databases, authentication, and APIs.

## FastAPI Framework

FastAPI is a modern, high-performance web framework for building APIs. It uses Python type hints for automatic request validation, serialization, and documentation generation. FastAPI supports async/await natively and automatically generates OpenAPI documentation. It is becoming the standard choice for building AI-powered APIs and microservices.

## REST API Design

REST APIs use HTTP methods (GET, POST, PUT, DELETE) to perform operations on resources. Good API design includes consistent URL patterns, proper status codes, pagination for large datasets, and clear error messages. Authentication is typically handled via API keys, OAuth tokens, or JWT tokens in the Authorization header.

## Database Integration

Python web applications commonly use SQLAlchemy as an ORM (Object Relational Mapper) to interact with databases. SQLAlchemy maps Python classes to database tables and provides a high-level API for queries. For simpler use cases, sqlite3 is built into Python's standard library and requires no additional installation.
  • Save all three files.
  • Confirm the files exist by running:
ls documents/

You should see python_advanced.md, python_basics.md, and python_web.md listed in the output.

Don't see all three files?

Make sure you created them inside the documents folder, not in the project root. File names must be lowercase with underscores.

help me find where my document files ended up

Write the document loading and chunking logic

Your RAG pipeline needs to read documents from disk, then split each one into small overlapping passages. Overlapping is important because it prevents ideas that span a chunk boundary from being lost.

Why overlapping chunks?

If you split a document into non-overlapping pieces, a sentence that starts at the end of one chunk and finishes at the beginning of the next gets cut in half. Neither chunk captures the full idea.

Overlapping by 50 characters means each chunk shares a small window with its neighbor. This way, boundary sentences appear in full in at least one chunk.

  • Open ingest.py in VS Code.
  • Add the imports and configuration constants by pasting this code:
import os
import json
import boto3
import chromadb


# Configuration constants
DOCS_DIR = "documents"
CHUNK_SIZE = 500
CHUNK_OVERLAP = 50
COLLECTION_NAME = "document_qa"
CHROMA_PATH = "./chroma_db"

What do these constants control?

  • CHUNK_SIZE = 500 sets the maximum character length for each passage.
  • CHUNK_OVERLAP = 50 means each chunk shares 50 characters with the previous one.
  • CHROMA_PATH is where ChromaDB will persist your vector data to disk (used in the next step).
  • The boto3 and chromadb imports are not used in this step. You'll use them when you add embeddings later.
  • Below the constants, add the load_documents function by pasting this code:
def load_documents(docs_dir):
    """Load all markdown files from the documents directory."""
    documents = []
    for filename in sorted(os.listdir(docs_dir)):
        if filename.endswith(".md"):
            filepath = os.path.join(docs_dir, filename)
            with open(filepath, "r") as f:
                content = f.read()
            documents.append({"filename": filename, "content": content})
    return documents

What does this function do?

  • It scans the documents/ folder for any file ending in .md.
  • Each file gets read into memory and stored as a dictionary with filename and content keys.
  • Using sorted() ensures documents are always processed in alphabetical order, making results predictable.
  • Below the load_documents function, add the chunk_document function by pasting this code:
def chunk_document(document, chunk_size=CHUNK_SIZE, overlap=CHUNK_OVERLAP):
    """Split a document into overlapping chunks."""
    content = document["content"]
    chunks = []
    start = 0
    chunk_index = 0

    while start < len(content):
        end = start + chunk_size
        chunk_text = content[start:end]

        # Only keep chunks that contain actual text
        if chunk_text.strip():
            chunks.append({
                "text": chunk_text.strip(),
                "source": document["filename"],
                "chunk_index": chunk_index,
            })
            chunk_index += 1

        # Move forward by chunk_size minus overlap
        start += chunk_size - overlap

    return chunks

What does this function do?

  • It takes a document dictionary and slides a 500-character window across the content, advancing 450 characters each time (500 minus the 50-character overlap).
  • Each chunk is stored with its text, the source filename, and its position index. These metadata fields help you trace answers back to their origin later.
  • The if chunk_text.strip() check skips any trailing whitespace-only chunks that might appear at the end of a file.
  • Save ingest.py.

Seeing a red underline on an import?

If VS Code shows a warning on import chromadb or import boto3, make sure your virtual environment is selected as the Python interpreter. Press Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows), type Python: Select Interpreter, and choose the one that includes venv in its path.

help me fix import errors in my ingest.py file

✔️ Awesome, I've got everything!

Great. Double check you've saved ingest.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete ingest.py file as it should look right now:

import os
import json
import boto3
import chromadb


DOCS_DIR = "documents"
CHUNK_SIZE = 500
CHUNK_OVERLAP = 50
COLLECTION_NAME = "document_qa"
CHROMA_PATH = "./chroma_db"


def load_documents(docs_dir):
    """Load all markdown files from the documents directory."""
    documents = []
    for filename in sorted(os.listdir(docs_dir)):
        if filename.endswith(".md"):
            filepath = os.path.join(docs_dir, filename)
            with open(filepath, "r") as f:
                content = f.read()
            documents.append({"filename": filename, "content": content})
    return documents


def chunk_document(document, chunk_size=CHUNK_SIZE, overlap=CHUNK_OVERLAP):
    """Split a document into overlapping chunks."""
    content = document["content"]
    chunks = []
    start = 0
    chunk_index = 0

    while start < len(content):
        end = start + chunk_size
        chunk_text = content[start:end]

        if chunk_text.strip():
            chunks.append({
                "text": chunk_text.strip(),
                "source": document["filename"],
                "chunk_index": chunk_index,
            })
            chunk_index += 1

        start += chunk_size - overlap

    return chunks
Test the chunking pipeline

Before building the rest of the pipeline, verify that your loading and chunking functions work correctly. A quick test now saves debugging time later.

  • In your terminal, test the chunking pipeline by running this command:
python3 -c "from ingest import load_documents, chunk_document; docs = load_documents('documents'); print(f'Loaded {len(docs)} documents'); chunks = chunk_document(docs[0]); print(f'First doc produced {len(chunks)} chunks'); print(f'Sample chunk: {chunks[0][\"text\"][:80]}...')"

You should see output confirming 3 documents were loaded and the first document produced several chunks. The sample chunk text should show the beginning of your python_advanced.md content (since it comes first alphabetically).

What should I see?

Your output will look something like this:

Loaded 3 documents
First doc produced 4 chunks
Sample chunk: # Advanced Python Concepts

## Decorators

Decorators are functions that mo...

The exact chunk count depends on file length, but you should see 3-5 chunks per document with ~500 characters of content each.

Seeing ModuleNotFoundError?

If you see ModuleNotFoundError: No module named 'chromadb', your virtual environment may not be activated. Run source venv/bin/activate and try again.

If you see FileNotFoundError, make sure you're running the command from your rag-document-qa project folder (not from inside the documents folder).

help me debug my chunking pipeline test

Your documents are loaded and chunked into searchable passages. Next up, you'll turn these text chunks into vector embeddings and store them in ChromaDB so your app can find relevant passages by meaning, not just keywords.

Create Embeddings and Store in ChromaDB

Your chunking pipeline breaks documents into bite-sized passages. Now you need a way to search those passages by meaning, not just keywords.

Vector embeddings solve this. Each chunk gets converted into a numerical representation that captures its semantic meaning. You'll use Amazon Titan Text Embeddings V2 to generate these vectors and ChromaDB to store them on disk for fast similarity search.

In this step, get ready to:
  • Write a function that embeds text using Amazon Titan Text Embeddings V2.
  • Build the ingestion orchestrator that loads, chunks, embeds, and stores documents in ChromaDB.
  • Run the pipeline and verify vectors are stored successfully.
Write the embedding function

Amazon Titan Text Embeddings V2 converts text into a high-dimensional vector. Your function sends each chunk to the model and collects the resulting vectors into a list.

  • In ingest.py, find the end of the chunk_document() function.
  • Add the following get_embeddings() function below it:
def get_embeddings(texts, bedrock_client):
    """Get embeddings from Amazon Titan Text Embeddings V2."""
    embeddings = []
    for text in texts:
        # Send each chunk to Titan Embeddings and get its vector
        response = bedrock_client.invoke_model(
            modelId="amazon.titan-embed-text-v2:0",
            body=json.dumps({"inputText": text}),
        )
        result = json.loads(response["body"].read())
        embeddings.append(result["embedding"])
    return embeddings

What does this code do?

This function loops through each text chunk and sends it to Amazon Titan Text Embeddings V2 via the Bedrock invoke_model() API.

The model returns a numerical vector (the embedding) that captures the chunk's meaning. Two chunks about similar topics will have vectors that are close together in the vector space.

  • Save the file. You won't see a change yet since the function needs to be called by an orchestrator.
Build the ingestion orchestrator

The orchestrator ties together every piece of the pipeline: loading documents, chunking them, generating embeddings, and storing everything in ChromaDB. It also includes progress messages so you can see what's happening at each stage.

  • In ingest.py, add the following ingest_documents() function and main block below get_embeddings():
def ingest_documents():
    """Main ingestion pipeline: load, chunk, embed, and store."""
    print("Loading documents...")
    documents = load_documents(DOCS_DIR)
    print(f"Loaded {len(documents)} documents.")

    print("Chunking documents...")
    all_chunks = []
    for doc in documents:
        chunks = chunk_document(doc)
        all_chunks.extend(chunks)
    print(f"Created {len(all_chunks)} chunks.")

    print("Generating embeddings...")
    # Create a Bedrock client to call embedding model
    bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
    texts = [chunk["text"] for chunk in all_chunks]
    embeddings = get_embeddings(texts, bedrock)
    print(f"Generated {len(embeddings)} embeddings.")

    print("Storing in ChromaDB...")
    # Use PersistentClient so vectors survive between sessions
    client = chromadb.PersistentClient(path=CHROMA_PATH)
    collection = client.get_or_create_collection(name=COLLECTION_NAME)

    # Store chunks with their embeddings and metadata
    collection.upsert(
        ids=[f"chunk_{i}" for i in range(len(all_chunks))],
        embeddings=embeddings,
        documents=texts,
        metadatas=[
            {"source": chunk["source"], "chunk_index": chunk["chunk_index"]}
            for chunk in all_chunks
        ],
    )
    print(f"Stored {len(all_chunks)} chunks in ChromaDB.")
    print("Ingestion complete!")


if __name__ == "__main__":
    ingest_documents()

What does this code do?

  • load_documents() reads all markdown files from the documents/ folder.
  • chunk_document() splits each document into overlapping passages.
  • get_embeddings() converts each chunk into a vector via Amazon Bedrock.
  • collection.upsert() stores the vectors in ChromaDB alongside their original text and metadata (source file and chunk index). Using upsert means you can re-run the script without creating duplicates.
  • PersistentClient saves the vector store to disk at ./chroma_db, so your vectors persist between sessions.

Why ChromaDB instead of Bedrock Knowledge Bases?

ChromaDB runs locally with zero cost and teaches you what a vector database actually does under the hood. Bedrock Knowledge Bases abstracts away the retrieval logic, which defeats the learning purpose here.

  • Save ingest.py.
Run the ingestion pipeline

Time to run the full pipeline and watch your documents transform into searchable vectors.

  • Make sure your virtual environment is active (you should see (venv) in your terminal prompt).
  • Run the ingestion script by executing this command:
python3 ingest.py

You should see output like this:

Loading documents...
Loaded 3 documents.
Chunking documents...
Created 18 chunks.
Generating embeddings...
Generated 18 embeddings.
Storing in ChromaDB...
Stored 18 chunks in ChromaDB.
Ingestion complete!

What should I see?

The exact chunk count depends on your document lengths, but you should see all stages complete without errors. The numbers for loaded, chunked, embedded, and stored should all match.

  • Confirm the vector store was created by running:
ls chroma_db

You should see files inside the chroma_db/ directory. This is your persisted vector store.

Seeing an error when running the script?

  • If you see NoCredentialsError, your AWS credentials are not configured. Run aws configure and enter your Access Key ID, Secret Access Key, and set the region to us-east-1.
  • If you see AccessDeniedException, your AWS account may not have Bedrock model access enabled. Go to the Amazon Bedrock console, select Model access, and verify that Amazon Titan Text Embeddings V2 is enabled.
  • If you see ModuleNotFoundError, make sure your virtual environment is active and that you installed the packages with pip install boto3 chromadb.

Help me debug my ingest.py script

✔️ Awesome, I've got everything!

Great. Double check you've saved ingest.py and that chroma_db/ exists in your project folder.

ⓧ I'd like to double check the full code

Here is the complete ingest.py file at this point:

import os
import json
import boto3
import chromadb


DOCS_DIR = "documents"
CHUNK_SIZE = 500
CHUNK_OVERLAP = 50
COLLECTION_NAME = "document_qa"
CHROMA_PATH = "./chroma_db"


def load_documents(docs_dir):
    """Load all markdown files from the documents directory."""
    documents = []
    for filename in sorted(os.listdir(docs_dir)):
        if filename.endswith(".md"):
            filepath = os.path.join(docs_dir, filename)
            with open(filepath, "r") as f:
                content = f.read()
            documents.append({"filename": filename, "content": content})
    return documents


def chunk_document(document, chunk_size=CHUNK_SIZE, overlap=CHUNK_OVERLAP):
    """Split a document into overlapping chunks."""
    content = document["content"]
    chunks = []
    start = 0
    chunk_index = 0

    while start < len(content):
        end = start + chunk_size
        chunk_text = content[start:end]

        if chunk_text.strip():
            chunks.append({
                "text": chunk_text.strip(),
                "source": document["filename"],
                "chunk_index": chunk_index,
            })
            chunk_index += 1

        start += chunk_size - overlap

    return chunks


def get_embeddings(texts, bedrock_client):
    """Get embeddings from Amazon Titan Text Embeddings V2."""
    embeddings = []
    for text in texts:
        response = bedrock_client.invoke_model(
            modelId="amazon.titan-embed-text-v2:0",
            body=json.dumps({"inputText": text}),
        )
        result = json.loads(response["body"].read())
        embeddings.append(result["embedding"])
    return embeddings


def ingest_documents():
    """Main ingestion pipeline: load, chunk, embed, and store."""
    print("Loading documents...")
    documents = load_documents(DOCS_DIR)
    print(f"Loaded {len(documents)} documents.")

    print("Chunking documents...")
    all_chunks = []
    for doc in documents:
        chunks = chunk_document(doc)
        all_chunks.extend(chunks)
    print(f"Created {len(all_chunks)} chunks.")

    print("Generating embeddings...")
    bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
    texts = [chunk["text"] for chunk in all_chunks]
    embeddings = get_embeddings(texts, bedrock)
    print(f"Generated {len(embeddings)} embeddings.")

    print("Storing in ChromaDB...")
    client = chromadb.PersistentClient(path=CHROMA_PATH)
    collection = client.get_or_create_collection(name=COLLECTION_NAME)

    collection.upsert(
        ids=[f"chunk_{i}" for i in range(len(all_chunks))],
        embeddings=embeddings,
        documents=texts,
        metadatas=[
            {"source": chunk["source"], "chunk_index": chunk["chunk_index"]}
            for chunk in all_chunks
        ],
    )
    print(f"Stored {len(all_chunks)} chunks in ChromaDB.")
    print("Ingestion complete!")


if __name__ == "__main__":
    ingest_documents()

Your documents are now embedded and stored as searchable vectors. Next up, you'll build the RAG query engine that retrieves those vectors and uses them to generate grounded answers.

Build the RAG Query Engine

Your documents are chunked, embedded, and stored in ChromaDB. Now you need the logic that actually answers questions using those stored vectors.

A RAG query engine has two jobs: retrieve the most relevant passages from your vector store, then send those passages to an LLM so it can generate a grounded answer. In this step, you will build both pieces inside rag.py.

In this step, get ready to:
  • Write a retrieval function that embeds a question and finds the top 3 relevant chunks from ChromaDB.
  • Write a generation function that sends retrieved context to Amazon Nova Lite via the Bedrock Converse API.
  • Design a prompt template that instructs the model to answer only from context and cite its sources.
Write the retrieval function

Retrieval is the "R" in RAG. When a user asks a question, you need to convert that question into an embedding vector, then search ChromaDB for the chunks whose vectors are closest to it.

  • Open rag.py in VS Code.
  • Add the imports, configuration constants, and helper functions by pasting this code:
import json
import boto3
import chromadb


COLLECTION_NAME = "document_qa"
CHROMA_PATH = "./chroma_db"
MODEL_ID = "us.amazon.nova-lite-v1:0"


def get_bedrock_client():
    """Create a Bedrock runtime client."""
    return boto3.client("bedrock-runtime", region_name="us-east-1")


def get_collection():
    """Get the ChromaDB collection."""
    client = chromadb.PersistentClient(path=CHROMA_PATH)
    return client.get_or_create_collection(name=COLLECTION_NAME)


def embed_query(query, bedrock_client):
    """Embed a query using Titan Text Embeddings V2."""
    response = bedrock_client.invoke_model(
        modelId="amazon.titan-embed-text-v2:0",
        body=json.dumps({"inputText": query}),
    )
    result = json.loads(response["body"].read())
    return result["embedding"]

What does this code do?

This sets up the foundation for your RAG engine.

  • COLLECTION_NAME and CHROMA_PATH point to the same vector store you populated during ingestion.
  • MODEL_ID uses the inference profile ID for Amazon Nova Lite, which lets AWS route requests optimally.
  • embed_query() converts a user's question into a vector using the same Titan Text Embeddings V2 model used during ingestion. This ensures the question vector lives in the same mathematical space as your document vectors.
  • Save rag.py.

You won't see a visible change yet. The next chunk uses these helpers to perform the actual similarity search.

Now add the retrieval function that searches ChromaDB for relevant chunks. Paste this code below your existing functions in rag.py:

def retrieve(query, bedrock_client, collection, n_results=3):
    """Retrieve the most relevant chunks for a query."""
    # Embed the question into a vector
    query_embedding = embed_query(query, bedrock_client)

    # Search ChromaDB for the closest matching chunks
    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=n_results,
        include=["documents", "metadatas", "distances"],
    )

    # Package each result into a clean dictionary
    retrieved = []
    for i in range(len(results["documents"][0])):
        retrieved.append({
            "text": results["documents"][0][i],
            "source": results["metadatas"][0][i]["source"],
            "chunk_index": results["metadatas"][0][i]["chunk_index"],
            "distance": results["distances"][0][i],
        })

    return retrieved

What does this code do?

  • embed_query() converts the user's question into a vector so it can be compared against stored document vectors.
  • collection.query() performs a similarity search and returns the 3 closest chunks along with their metadata and distance scores.
  • The loop packages each result into a dictionary containing the text, source filename, chunk position, and distance. Lower distance means higher relevance.
  • Save rag.py.
  • Verify the retrieval function works by running this test in your terminal:
python3 -c "from rag import get_bedrock_client, get_collection, retrieve; b=get_bedrock_client(); c=get_collection(); print(retrieve('What are decorators?', b, c))"

You should see a list of 3 dictionaries printed to the terminal, each containing text about Python decorators, a source filename, chunk index, and distance value.

Seeing an error?

  • If you see NoCredentialsError, make sure your virtual environment is active and your AWS credentials are configured with aws configure.
  • If ChromaDB returns an empty result, confirm that you ran python3 ingest.py successfully in the previous step and that the chroma_db/ directory exists.
  • If you see ModuleNotFoundError, make sure your virtual environment is activated with source venv/bin/activate.

Still stuck? Help me debug my retrieve function in rag.py

Write the generation function

Retrieval gives you the raw passages. Now you need a function that sends those passages to Amazon Nova Lite and asks it to synthesize a clear answer.

  • In rag.py, paste the following function below your retrieve() function:
def generate_answer(query, retrieved_chunks, bedrock_client):
    """Generate an answer using Amazon Nova Lite with retrieved context."""
    # Format each retrieved chunk with a source label
    context = ""
    for i, chunk in enumerate(retrieved_chunks, 1):
        context += f"[Source {i}: {chunk['source']}]\n{chunk['text']}\n\n"

    # System prompt instructs the model to only use provided context
    system_prompt = (
        "You are a helpful assistant that answers questions based only on the "
        "provided context. If the context does not contain enough information "
        "to answer the question, say so. Always cite which source(s) you used "
        "by referencing [Source N] in your answer."
    )

    # Combine context and question into the user message
    user_message = f"Context:\n{context}\nQuestion: {query}"

    # Call the Bedrock Converse API
    response = bedrock_client.converse(
        modelId=MODEL_ID,
        messages=[{"role": "user", "content": [{"text": user_message}]}],
        system=[{"text": system_prompt}],
        inferenceConfig={"maxTokens": 1024, "temperature": 0.1},
    )

    # Extract the answer text from the response
    answer = response["output"]["message"]["content"][0]["text"]
    return answer

What does this code do?

  • The context loop labels each chunk as [Source 1], [Source 2], etc. so the LLM can reference them by number in its answer.
  • The system prompt constrains the model to answer ONLY from the provided context. This prevents hallucination.
  • bedrock_client.converse() uses the Bedrock Converse API, which is model-agnostic. You could swap Nova Lite for Claude or another model by changing the MODEL_ID constant.
  • temperature=0.1 keeps answers deterministic and factual. Higher values make the model more creative but less reliable for document Q&A.
  • Save rag.py.

You won't see a change yet. The next section ties retrieval and generation together into a single callable function.

Why the Converse API instead of invoke_model?

You used invoke_model() for Titan Embeddings because embeddings have a simple input/output format. For chat-style interactions, the Converse API provides a standardized message structure that works the same across all Bedrock models.

This means you can switch from Nova Lite to Claude or another model without rewriting your message formatting code.

Design the prompt template and complete the pipeline

The prompt template you just wrote is the core of your RAG system's quality. It tells the model three critical things: only use the provided context, admit when it does not know, and always cite sources.

Why does the prompt say "only the provided context"?

Without this constraint, the LLM will happily answer questions using its general training data. That defeats the purpose of RAG. You want answers grounded in YOUR documents, not the internet.

The citation requirement ("reference [Source N]") creates accountability. Users can click through to verify the answer against the original text.

The final piece is an orchestrator function that ties retrieval and generation into one clean call. Paste this at the bottom of rag.py:

def query_rag(question):
    """Full RAG pipeline: embed question, retrieve context, generate answer."""
    bedrock_client = get_bedrock_client()
    collection = get_collection()

    # Retrieve relevant chunks then generate an answer from them
    retrieved_chunks = retrieve(question, bedrock_client, collection)
    answer = generate_answer(question, retrieved_chunks, bedrock_client)

    return {"answer": answer, "sources": retrieved_chunks}

What does this code do?

query_rag() is the single entry point for your entire RAG pipeline. It creates the AWS client, connects to ChromaDB, retrieves relevant chunks, generates an answer, and returns both the answer text and the source chunks. Your Streamlit app will call this one function for every user question.

  • Save rag.py.
  • Test the full RAG pipeline by running this in your terminal:
python3 -c "from rag import query_rag; result = query_rag('What are decorators in Python?'); print(result['answer']); print('---Sources---'); print([s['source'] for s in result['sources']])"

You should see a multi-sentence answer about Python decorators that references [Source 1], [Source 2], etc. Below it, you should see a list of source filenames like python_advanced.md.

Getting an AccessDeniedException?

  • This means your AWS account does not have access to the Nova Lite model. Go to the AWS Bedrock console, click **Model access** in the left sidebar, and confirm that Amazon Nova Lite shows as enabled.
  • If you see a timeout error, your region may not support Nova Lite. Confirm your aws configure default region is set to us-east-1.

Need more help? Help me fix my Bedrock model access

✔️ Awesome, I've got everything!

Great. Double check you have saved rag.py and move on to the next step.

ⓧ I'd like to double check the full code

Here is the complete rag.py file. Make sure yours matches exactly:

import json
import boto3
import chromadb


COLLECTION_NAME = "document_qa"
CHROMA_PATH = "./chroma_db"
MODEL_ID = "us.amazon.nova-lite-v1:0"


def get_bedrock_client():
    """Create a Bedrock runtime client."""
    return boto3.client("bedrock-runtime", region_name="us-east-1")


def get_collection():
    """Get the ChromaDB collection."""
    client = chromadb.PersistentClient(path=CHROMA_PATH)
    return client.get_or_create_collection(name=COLLECTION_NAME)


def embed_query(query, bedrock_client):
    """Embed a query using Titan Text Embeddings V2."""
    response = bedrock_client.invoke_model(
        modelId="amazon.titan-embed-text-v2:0",
        body=json.dumps({"inputText": query}),
    )
    result = json.loads(response["body"].read())
    return result["embedding"]


def retrieve(query, bedrock_client, collection, n_results=3):
    """Retrieve the most relevant chunks for a query."""
    query_embedding = embed_query(query, bedrock_client)

    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=n_results,
        include=["documents", "metadatas", "distances"],
    )

    retrieved = []
    for i in range(len(results["documents"][0])):
        retrieved.append({
            "text": results["documents"][0][i],
            "source": results["metadatas"][0][i]["source"],
            "chunk_index": results["metadatas"][0][i]["chunk_index"],
            "distance": results["distances"][0][i],
        })

    return retrieved


def generate_answer(query, retrieved_chunks, bedrock_client):
    """Generate an answer using Amazon Nova Lite with retrieved context."""
    context = ""
    for i, chunk in enumerate(retrieved_chunks, 1):
        context += f"[Source {i}: {chunk['source']}]\n{chunk['text']}\n\n"

    system_prompt = (
        "You are a helpful assistant that answers questions based only on the "
        "provided context. If the context does not contain enough information "
        "to answer the question, say so. Always cite which source(s) you used "
        "by referencing [Source N] in your answer."
    )

    user_message = f"Context:\n{context}\nQuestion: {query}"

    response = bedrock_client.converse(
        modelId=MODEL_ID,
        messages=[{"role": "user", "content": [{"text": user_message}]}],
        system=[{"text": system_prompt}],
        inferenceConfig={"maxTokens": 1024, "temperature": 0.1},
    )

    answer = response["output"]["message"]["content"][0]["text"]
    return answer


def query_rag(question):
    """Full RAG pipeline: embed question, retrieve context, generate answer."""
    bedrock_client = get_bedrock_client()
    collection = get_collection()

    retrieved_chunks = retrieve(question, bedrock_client, collection)
    answer = generate_answer(question, retrieved_chunks, bedrock_client)

    return {"answer": answer, "sources": retrieved_chunks}

Your RAG engine is fully functional. Next up, you will wrap it in a Streamlit chat interface so anyone can ask questions through a web browser.

Build the Streamlit Chat Interface

You have a complete RAG pipeline that can answer questions from your documents. The problem is that it only works from a Python script in the terminal.

In this step, you will build a web-based chat interface using Streamlit. This turns your RAG engine into a demo-ready application that anyone can interact with in their browser.

In this step, get ready to:
  • Create the Streamlit app with a page title, sidebar, and document list.
  • Build a chat interface that persists conversation history.
  • Wire up the RAG engine and display answers with expandable source citations.
Set up the page layout and sidebar

Every Streamlit app starts with a page configuration and layout. The sidebar will show which documents are loaded into the system, giving users context about what they can ask.

  • Open app.py in VS Code and add the following code:
import streamlit as st
from rag import query_rag
import os


st.set_page_config(page_title="Document Q&A", page_icon="📚", layout="wide")
st.title("📚 RAG Document Q&A")
st.caption("Ask questions about your documents. Powered by AWS Bedrock.")

# Sidebar: show loaded documents
with st.sidebar:
    st.header("📁 Loaded Documents")
    docs_dir = "documents"
    if os.path.exists(docs_dir):
        doc_files = [f for f in sorted(os.listdir(docs_dir)) if f.endswith(".md")]
        for doc_file in doc_files:
            st.markdown(f"- {doc_file}")
    else:
        st.warning("No documents directory found.")
    st.divider()
    st.markdown("**How it works:**")
    st.markdown(
        "1. Documents are chunked and embedded\n"
        "2. Your question is embedded and matched to relevant chunks\n"
        "3. An LLM generates an answer using only the matched context"
    )

What does this code do?

The first three lines import Streamlit, your RAG engine from rag.py, and the os module for file operations.

  • st.set_page_config() sets the browser tab title, favicon, and uses a wide layout.
  • The with st.sidebar: block creates a collapsible sidebar that lists all .md files from your documents/ folder.
  • The "How it works" section gives users a quick overview of the RAG process.
  • Save app.py.
  • Launch the app by running this command in your terminal:
streamlit run app.py
  • Your browser opens automatically to http://localhost:8501. You should see the title "RAG Document Q&A" and a sidebar listing your three markdown files.

Browser didn't open?

  • Check your terminal output for a URL like http://localhost:8501 and open it manually in your browser.
  • Make sure your virtual environment is activated. You should see (venv) at the start of your terminal prompt.
  • If you see ModuleNotFoundError: No module named 'streamlit', run pip install streamlit again inside your activated virtual environment.

Why isn't my Streamlit app launching?

Add chat history and message display

Streamlit reruns your entire script on every interaction. To keep the conversation visible between messages, you need to store chat history in session state and replay it each time.

  • Stop the running Streamlit server by pressing Ctrl+C in your terminal.
  • In app.py, add the following code below the sidebar block:
# Initialize chat history
if "messages" not in st.session_state:
    st.session_state.messages = []

# Display chat history
for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])
        if message["role"] == "assistant" and "sources" in message:
            with st.expander("📄 View Sources"):
                for source in message["sources"]:
                    st.markdown(f"**{source['source']}** (chunk {source['chunk_index']})")
                    st.text(source["text"][:200] + "...")
                    st.divider()

What does this code do?

  • st.session_state.messages is a list that survives Streamlit reruns. It stores every user and assistant message.
  • The for loop replays the full conversation history on each rerun so previous messages stay visible.
  • For assistant messages, it also renders an expandable section showing the source documents that were used to generate the answer.
  • Save app.py.

You won't see a visible change yet. The chat history is empty, so this loop has nothing to display. The next chunk adds the input box that populates it.

Wire up the chat input to your RAG engine

This is where everything comes together. When a user types a question, the app will embed it, retrieve relevant chunks from ChromaDB, send them to Amazon Nova Lite, and display the generated answer with citations.

  • In app.py, add the following code below the chat history display loop:
# Chat input
if prompt := st.chat_input("Ask a question about your documents"):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        with st.spinner("Searching documents and generating answer..."):
            result = query_rag(prompt)
            answer = result["answer"]
            sources = result["sources"]

        st.markdown(answer)
        with st.expander("📄 View Sources"):
            for source in sources:
                st.markdown(f"**{source['source']}** (chunk {source['chunk_index']})")
                st.text(source["text"][:200] + "...")
                st.divider()

    st.session_state.messages.append({
        "role": "assistant",
        "content": answer,
        "sources": sources,
    })

What does this code do?

  • st.chat_input() renders a text input at the bottom of the page. The walrus operator (:=) assigns the user's message to prompt and checks if it's not empty.
  • query_rag(prompt) calls the full RAG pipeline you built in rag.py. It returns a dictionary with an answer string and a sources list.
  • st.spinner() shows a loading animation while the RAG pipeline processes the question.
  • Each source is displayed inside an expandable section showing the filename, chunk index, and a 200-character preview of the text.
  • Save app.py.
  • Run the app again:
streamlit run app.py
  • In your browser, type a question into the chat input at the bottom. Try: What are decorators in Python?.
  • You should see a spinner while the app processes, then an answer with source citations. Click the View Sources expander to see which document chunks were used.

Seeing an error when you ask a question?

  • If you see NoCredentialError, your AWS credentials are not configured. Run aws configure in a separate terminal and re-enter your credentials.
  • If you see InvalidCollection or an empty result, make sure you ran python3 ingest.py in a previous step. The chroma_db/ folder must exist with data.
  • If you see AccessDeniedException, confirm that Amazon Nova Lite and Titan Text Embeddings V2 are enabled in your Bedrock console.

Help me debug my Streamlit RAG app error.

✔️ Awesome, I've got everything!

Great. Double check you have saved app.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete app.py file. Compare it with yours and fix any differences.

import streamlit as st
from rag import query_rag
import os


st.set_page_config(page_title="Document Q&A", page_icon="📚", layout="wide")
st.title("📚 RAG Document Q&A")
st.caption("Ask questions about your documents. Powered by AWS Bedrock.")

# Sidebar: show loaded documents
with st.sidebar:
    st.header("📁 Loaded Documents")
    docs_dir = "documents"
    if os.path.exists(docs_dir):
        doc_files = [f for f in sorted(os.listdir(docs_dir)) if f.endswith(".md")]
        for doc_file in doc_files:
            st.markdown(f"- {doc_file}")
    else:
        st.warning("No documents directory found.")
    st.divider()
    st.markdown("**How it works:**")
    st.markdown(
        "1. Documents are chunked and embedded\n"
        "2. Your question is embedded and matched to relevant chunks\n"
        "3. An LLM generates an answer using only the matched context"
    )

# Initialize chat history
if "messages" not in st.session_state:
    st.session_state.messages = []

# Display chat history
for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])
        if message["role"] == "assistant" and "sources" in message:
            with st.expander("📄 View Sources"):
                for source in message["sources"]:
                    st.markdown(f"**{source['source']}** (chunk {source['chunk_index']})")
                    st.text(source["text"][:200] + "...")
                    st.divider()

# Chat input
if prompt := st.chat_input("Ask a question about your documents"):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        with st.spinner("Searching documents and generating answer..."):
            result = query_rag(prompt)
            answer = result["answer"]
            sources = result["sources"]

        st.markdown(answer)
        with st.expander("📄 View Sources"):
            for source in sources:
                st.markdown(f"**{source['source']}** (chunk {source['chunk_index']})")
                st.text(source["text"][:200] + "...")
                st.divider()

    st.session_state.messages.append({
        "role": "assistant",
        "content": answer,
        "sources": sources,
    })

Your RAG app is live and working. Next up, you will push everything to GitHub so you have a clean, professional portfolio piece to share.

Push to GitHub

Your RAG app is fully functional. You can ask questions, get cited answers, and demo the whole thing from your browser.

Now you need to make it visible to the world. A GitHub repository with a clean README turns this from a local experiment into a portfolio piece that hiring managers can browse in 30 seconds.

In this step, get ready to:
  • Write a professional README with an architecture diagram and setup instructions.
  • Commit all project files and push to a new GitHub repository.
  • Verify the repository looks clean and complete.
Write the README

A README is the first thing anyone sees when they land on your repository. It needs to answer three questions quickly: what does this project do, how is it built, and how do I run it myself.

  • Open README.md in VS Code (it should be empty right now).
  • Replace the empty file with the following content:
# RAG Document Q&A

A Retrieval-Augmented Generation (RAG) application that answers questions about your documents using AWS Bedrock.

## Architecture

```
Documents --> Chunking --> Embedding (Titan V2) --> ChromaDB (Vector Store)
                                                          |
User Question --> Embedding (Titan V2) --> Similarity Search --> Top Chunks
                                                                     |
                                                          LLM (Nova Lite) --> Answer with Citations
```

## Features

- Document ingestion with overlapping chunk strategy
- Vector similarity search via ChromaDB
- Grounded answers with source citations using Amazon Nova Lite
- Interactive chat interface built with Streamlit

## Setup

1. Install Python 3.10+ and AWS CLI
2. Configure AWS credentials: `aws configure`
3. Create and activate a virtual environment:
   ```bash
   python3 -m venv venv
   source venv/bin/activate
   ```
4. Install dependencies:
   ```bash
   pip install -r requirements.txt
   ```
5. Ingest documents:
   ```bash
   python3 ingest.py
   ```
6. Run the app:
   ```bash
   streamlit run app.py
   ```

## Tech Stack

- **LLM**: Amazon Nova Lite (via AWS Bedrock)
- **Embeddings**: Amazon Titan Text Embeddings V2
- **Vector DB**: ChromaDB (local persistent storage)
- **Frontend**: Streamlit
- **Cloud**: AWS Bedrock (pay-per-token, no infrastructure to manage)

What does this README include?

The Architecture section uses a text-based diagram showing the two data flows: document ingestion (chunking, embedding, storage) and query handling (embed question, search, generate answer).

The Setup section gives anyone the exact commands to reproduce your project from scratch. The Tech Stack section names the specific models and tools so readers instantly understand the technical depth.

  • Save README.md.
Commit and push to GitHub

With all your files ready, you need to create a commit (a snapshot of your project) and push it to a remote repository on GitHub.

  • Stage all your project files for commit by running this command:
git add .

What does git add . do?

The dot (.) tells Git to stage every file in the current directory. Your .gitignore automatically excludes venv/, chroma_db/, and __pycache__/ from being tracked.

  • Create your first commit by running:
git commit -m "Initial commit: RAG Document Q&A app"
  • You should see a list of files being committed, including app.py, ingest.py, rag.py, README.md, requirements.txt, .gitignore, and the three files inside documents/.

Seeing 'nothing to commit'?

Make sure you saved all your files in VS Code before running git add .. Check that your terminal is inside the rag-document-qa folder by running pwd.

Help me debug my git commit issue

Now you need to create a new repository on GitHub to push your code to.

  • Open your browser and go to github.com/new.
  • Enter rag-document-qa in the Repository name field.
  • Leave it set to Public so anyone can view your project.
  • Do NOT check any of the initialization options (no README, no .gitignore, no license). You already have these files locally.
  • Click Create repository.

GitHub shows you a page with setup commands. You only need the commands for pushing an existing repository.

  • Connect your local repository to GitHub by running this command:
git remote add origin https://github.com/[[GITHUB_USERNAME="your-github-username"]]/rag-document-qa.git
  • Push your commit to GitHub by running:
git push -u origin main

Getting a 'src refspec main' error?

Your default branch might be called master instead of main. Run git branch to check your branch name. If it says master, use git push -u origin master instead.

Help me fix my git push error

  • You should see output showing your files being uploaded to GitHub, ending with a line like Branch 'main' set up to track remote branch 'main'.
Verify the repository on GitHub

A quick check confirms your .gitignore is working correctly and your repository presents well to visitors.

  • Open your repository in the browser at https://github.com/ followed by your username and /rag-document-qa.
  • Confirm you see these files listed: app.py, ingest.py, rag.py, requirements.txt, .gitignore, README.md, and the documents/ folder.
  • Confirm that chroma_db/ and venv/ are NOT in the file list. Your .gitignore excluded them.
  • Scroll down to see your README rendered with the architecture diagram and setup instructions.

Why does this matter?

Anyone reviewing your work (including recruiters or hiring managers) will land on this page first. A rendered README with a clear architecture diagram signals that you understand the system you built, not just how to run it.

Seeing chroma_db/ in your repo?

If you committed before creating your .gitignore, Git already tracked those files. Run git rm -r --cached chroma_db/ to untrack them, then commit and push again.

Help me remove chroma_db from my GitHub repo

✔️ Awesome, I've got everything!

Great. Double check you have saved all your files and your GitHub repository shows the rendered README.

ⓧ I'd like to double check the full code

Here is the complete README.md for comparison:

# RAG Document Q&A

A Retrieval-Augmented Generation (RAG) application that answers questions about your documents using AWS Bedrock.

## Architecture

```
Documents --> Chunking --> Embedding (Titan V2) --> ChromaDB (Vector Store)
                                                          |
User Question --> Embedding (Titan V2) --> Similarity Search --> Top Chunks
                                                                     |
                                                          LLM (Nova Lite) --> Answer with Citations
```

## Features

- Document ingestion with overlapping chunk strategy
- Vector similarity search via ChromaDB
- Grounded answers with source citations using Amazon Nova Lite
- Interactive chat interface built with Streamlit

## Setup

1. Install Python 3.10+ and AWS CLI
2. Configure AWS credentials: `aws configure`
3. Create and activate a virtual environment:
   ```bash
   python3 -m venv venv
   source venv/bin/activate
   ```
4. Install dependencies:
   ```bash
   pip install -r requirements.txt
   ```
5. Ingest documents:
   ```bash
   python3 ingest.py
   ```
6. Run the app:
   ```bash
   streamlit run app.py
   ```

## Tech Stack

- **LLM**: Amazon Nova Lite (via AWS Bedrock)
- **Embeddings**: Amazon Titan Text Embeddings V2
- **Vector DB**: ChromaDB (local persistent storage)
- **Frontend**: Streamlit
- **Cloud**: AWS Bedrock (pay-per-token, no infrastructure to manage)

Secret mission

Add Relevance Scoring

Your RAG app retrieves context and generates answers, but how confident should you be in those answers? In this secret mission, you'll add a relevance scoring system that shows confidence levels for each retrieved passage and filters out low-quality context before it reaches the LLM.

Clean Up Your Resources

Clean Up Your Resources

This project runs locally on your machine, and AWS Bedrock charges are pay-per-token with no running infrastructure. There are no ongoing costs unless you actively send queries to Bedrock.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Python virtual environment (venv/ folder).
  • ChromaDB local vector store (chroma_db/ folder).
  • Project files (app.py, rag.py, ingest.py, documents/).
  • GitHub repository.
  • AWS Bedrock model access (Amazon Nova Lite and Titan Text Embeddings V2).
  • Installed packages (boto3, chromadb, streamlit).

Keep everything running

No action needed. Choose this if you're still actively building or want to keep querying your documents.

  • Your virtual environment, ChromaDB data, and project files all stay on your machine.
  • AWS Bedrock only charges when you send queries, so leaving model access enabled costs nothing.
  • To use the app again later, activate your virtual environment and run streamlit run app.py.

Pause - I'll come back to this later

Shut down running processes to free up memory. Your files and data stay intact for when you return.

  • Stop the Streamlit server by pressing Ctrl+C in the terminal where it is running.
  • Deactivate your virtual environment by running deactivate in the terminal.
  • Your chroma_db/ folder persists your embeddings to disk, so everything will be ready when you come back.
  • To resume, navigate to your project folder, run source venv/bin/activate, and then streamlit run app.py.

Delete - I don't want to use this again

Remove all project resources from your machine and start fresh.

  • Stop the Streamlit server by pressing Ctrl+C in the terminal.
  • Deactivate the virtual environment by running deactivate.
  • Delete the entire project folder from your Desktop. This removes all project files, the virtual environment, and the ChromaDB data.
  • Your GitHub repository will remain online unless you manually delete it from your GitHub account under Settings > Delete this repository.
  • AWS Bedrock model access does not need to be disabled. Amazon models are enabled by default and there is no cost unless you send requests.

Nice Work!

Nice Work!

You just built a production-style RAG application from scratch and published it to GitHub. This is the exact type of project that AI engineering job postings ask for.

Here's what you learned:

  • Built a document ingestion pipeline that chunks text into overlapping passages and embeds them using Amazon Titan Text Embeddings V2 into a ChromaDB vector store.
  • Created a RAG query engine that retrieves relevant context via semantic search and generates grounded, cited answers using Amazon Nova Lite through the Bedrock Converse API.
  • Wrapped the full pipeline in a demo-ready Streamlit chat interface with expandable source citations and pushed it to a clean GitHub repository.
  • Added relevance scoring with color-coded confidence badges and a threshold filter to exclude low-quality passages from the LLM context.

Ready to quiz yourself?