Build a Government AI Assistant

Build a RAG pipeline that answers citizen questions with cited policy docs.

Introduction

30 Second Summary

Every city government has hundreds of pages of policy documents that citizens need answers from. Building permits, park rules, waste collection schedules. Today, finding answers means digging through PDFs or waiting on hold. AI could answer these questions instantly, but only if it sticks to what the official documents actually say.

In this project, you will build a RAG (Retrieval-Augmented Generation) AI assistant that ingests government policy documents and answers citizen questions with accurate, cited responses grounded in official sources.

What You'll Build

You type a question like "What do I need to get a building permit?" and the assistant searches through official city policy documents, pulls the relevant paragraphs, and gives you a plain-language answer citing exactly which document it came from.

By the end of this project, you'll have:

A working RAG pipeline where you ask a policy question and get back a plain-language answer grounded in official documents, with every response citing which source the information came from.
Firsthand experience with the hallucination problem by asking Claude a policy question without context, then building the retrieval system that fixes it.
A vector database of chunked policy documents that you can query semantically, seeing the raw retrieved chunks alongside the final answer to understand how the pipeline works.
Secret Mission: A polished Streamlit web UI so you can demo the assistant live in an interview with an input field, loading spinner, and expandable source panel.

Are there any prerequisites?

You should be comfortable running Python scripts from the terminal. You will also need an Anthropic API key, which the project walks you through setting up in Step 1.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

You're going to build a RAG-based AI assistant for a fictional city that answers citizen questions using official policy documents. Every answer comes with citations back to the source document, so nothing is made up.

This is the exact architecture that government consulting firms deploy for their clients. You'll experience the hallucination problem firsthand, then build the solution that fixes it.

Set Up Your Environment

Your AI assistant needs three things to work: Python to run the code, the Anthropic SDK to talk to Claude, and ChromaDB to store and search your policy documents.

Getting these installed and verified now means zero friction when you start building in the next step.

In this step, get ready to:

Verify Python 3.10+ is installed.
Create a project folder with a virtual environment.
Install the Anthropic and ChromaDB packages and test your API key.

Install Python and verify

Python is the language your AI assistant runs on. ChromaDB and Streamlit (used later in the Secret Mission) both require Python 3.10 or higher.

Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
Type Cursor and press Enter to open it.
Open the integrated terminal by pressing Ctrl+` (backtick key, top-left of your keyboard).
Check your Python version by running this command:

python3 --version

✔️ I see version 3.10 or higher

Your Python version is compatible. Move on to the next substep.

ⓧ I see a version lower than 3.10

You have Python installed, but it is too old for this project.

Visit python.org/downloads and download Python 3.10 or newer.
Run the installer. On macOS, the new version installs alongside the old one. On Windows, check the box that says Add Python to PATH during installation.
Close and reopen your Cursor terminal, then run python3 --version again to confirm.

macOS ships an older Python

macOS often defaults to an older system Python. After installing a newer version, you may need to use python3.10 or python3.11 explicitly if python3 still resolves to the old version.

ⓧ Command not found

Python is not installed yet.

Visit python.org/downloads and download Python 3.10 or newer.
Run the installer. On Windows, check the box that says Add Python to PATH during installation.
Close and reopen your Cursor terminal, then run python3 --version again to confirm.

Create your project and install dependencies

A virtual environment keeps this project's packages isolated from the rest of your system. You will install two packages: the Anthropic SDK (to call Claude) and ChromaDB (to store documents as searchable vectors).

Move to your Desktop by running this command:

cd ~/Desktop

Create your project folder and move into it by running these commands:

mkdir government-ai-assistant
cd government-ai-assistant

Your terminal prompt now shows government-ai-assistant as the current folder.

Create a virtual environment by running:

python3 -m venv venv

Confirm the virtual environment was created by running:

ls

You should see a venv folder listed in the output.

Activate the virtual environment:

macOS

source venv/bin/activate

Windows

venv\Scripts\Activate.ps1

Seeing an execution policy error?

PowerShell may block scripts by default. Run this command first, then try activating again:

Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

Your terminal prompt now starts with (venv). This confirms the virtual environment is active.

Install the project's dependencies by running:

pip install anthropic==0.117.0 chromadb==1.5.4

You will see download and installation progress in your terminal. This may take a minute since ChromaDB pulls in several sub-dependencies.

Verify both packages imported successfully by running:

python3 -c "import anthropic; import chromadb; print('Ready')"

You should see the word Ready printed with no errors.

Seeing a ModuleNotFoundError?

Make sure your virtual environment is active (you see (venv) in your prompt). If not, re-run the activation command above.

If you accidentally ran pip install before activating, deactivate, delete the venv folder, and start this substep again.

Help me fix my ModuleNotFoundError after installing packages.

Configure your API key

The Anthropic SDK needs an API key to authenticate your requests to Claude. You will create a key in the Anthropic Console, then set it as an environment variable in your terminal.

Open console.anthropic.com in your browser.
Create an account if you do not have one. You will need to add a small credit balance (around $5 covers this entire project with plenty to spare).
Navigate to API Keys in the sidebar.
Click Create Key. Give it a name like government-assistant and copy the key value.

Copy your key now

Anthropic only shows the full key once. Copy it immediately and keep it somewhere safe. You will paste it in the next command.

Set the API key as an environment variable in your terminal:

macOS

export ANTHROPIC_API_KEY=your-api-key-here

Windows

$env:ANTHROPIC_API_KEY="your-api-key-here"

Replace your-api-key-here with the key you just copied. Do not add spaces around the = sign.

Why an environment variable?

Storing your key in an environment variable keeps it out of your code files. The Anthropic SDK automatically reads ANTHROPIC_API_KEY from the environment, so you never have to paste it into a Python script.

Before you move on, do you think Claude will respond to a simple "Hello" message successfully?

Test that everything works end-to-end by running:

python3 -c "from anthropic import Anthropic; client = Anthropic(); msg = client.messages.create(model='claude-haiku-4-5', max_tokens=100, messages=[{'role': 'user', 'content': 'Hello'}]); print(msg.content[0].text)"

You should see a friendly greeting from Claude printed in your terminal. This confirms your API key, network connection, and package installation all work.

Seeing an authentication error?

Double-check that you copied the full API key with no extra spaces. Run the export (macOS) or $env: (Windows) command again with the correct key.

If you see a billing or quota error, make sure you added credit to your Anthropic account in the console.

Help me fix my Anthropic API authentication error.

Your environment is locked and loaded. Next up, you will ask Claude a government policy question and see firsthand why a raw LLM is not good enough for citizen services.

See the Problem — Ask Claude Without Context

Your environment is ready and Claude is responding to API calls. Now you need to understand why a basic API call is not enough for government use cases.

When Claude answers government policy questions without access to official documents, it generates plausible but unverifiable responses. No government agency can deploy that. In this step, you will feel the problem firsthand by asking Claude a Riverside City policy question and comparing its answer to the actual source documents.

In this step, get ready to:

Create sample government policy documents for Riverside City.
Ask Claude a policy question without giving it any document context.
Identify the gap between Claude's answer and what the real policy says.

Create sample government policy documents

Your AI assistant needs real policy documents to draw from. You will create three fictional (but realistic) Riverside City policy files covering building permits, park regulations, and waste collection.

Create a documents/ folder inside your project by running this command in your Cursor terminal:

mkdir documents

In Cursor's Explorer sidebar, expand your project folder and right-click the documents folder.
Click New File and name it building_permits.txt.
Paste the following content into building_permits.txt:

RIVERSIDE CITY - BUILDING PERMIT GUIDELINES
Last Updated: January 2026

SECTION 1: WHEN YOU NEED A PERMIT
A building permit is required for any construction, renovation, or demolition work within Riverside City limits. This includes new construction, structural modifications, electrical work exceeding 30 amps, plumbing modifications, and any work that changes the building footprint.

Minor repairs such as painting, replacing fixtures with same-type fixtures, and landscaping that does not alter drainage do not require a permit.

SECTION 2: APPLICATION REQUIREMENTS
To apply for a building permit, you must submit the following:
1. Completed Application Form RC-101 (available at City Hall or riversidecity.gov/permits)
2. Two copies of construction plans drawn to scale
3. Proof of property ownership or written authorization from owner
4. Contractor license number (if using a licensed contractor)
5. Site survey completed within the last 12 months
6. Filing fee of $150 for residential projects or $400 for commercial projects

SECTION 3: REVIEW PROCESS
Applications are reviewed within 10 business days for residential projects and 20 business days for commercial projects. The Building Division may request additional documentation or plan revisions during review.

SECTION 4: INSPECTIONS
All permitted work requires a minimum of two inspections: a foundation/rough-in inspection and a final inspection. Additional inspections may be required for complex projects. Schedule inspections at least 48 hours in advance by calling 555-0142 or online at riversidecity.gov/inspections.

SECTION 5: PERMIT VALIDITY
Permits are valid for 12 months from date of issue. Work must commence within 90 days of permit issuance. Extensions may be granted for up to 6 additional months with written request and justification.

Right-click the documents folder again, click New File, and name it park_regulations.txt.
Paste the following content into park_regulations.txt:

RIVERSIDE CITY - PARKS AND RECREATION REGULATIONS
Effective: March 2025

SECTION 1: PARK HOURS
All Riverside City parks are open from 6:00 AM to 10:00 PM daily unless otherwise posted. The Riverside Trail System is open from dawn to dusk. Overnight camping is prohibited in all city parks except designated campgrounds at Millbrook Regional Park.

SECTION 2: FACILITY RESERVATIONS
Pavilions, athletic fields, and community rooms may be reserved for private events. Reservations must be made at least 14 days in advance through the Parks Department. Fees range from $50 for a pavilion (4-hour block) to $200 for athletic fields (full day). Non-profit organizations receive a 50% discount with valid documentation.

SECTION 3: PERMITTED ACTIVITIES
Parks are available for passive and active recreation including walking, jogging, cycling on designated paths, picnicking, fishing in designated areas with valid state license, and organized sports on athletic fields (reservation required for groups over 15 people).

SECTION 4: PROHIBITED ACTIVITIES
The following are prohibited in all city parks: alcohol consumption (except in reserved facilities with permit), amplified music without permit, motorized vehicles off designated roads, fireworks, commercial activity without vendor permit, and feeding of wildlife. Violations may result in fines from $75 to $500.

SECTION 5: DOG POLICY
Dogs are welcome in all Riverside City parks but must be leashed (6-foot maximum) at all times except in designated off-leash areas. Currently designated off-leash areas are: Bark Park at Millbrook (fenced, 2 acres) and the North Field at Riverside Central Park (unfenced, dawn to 9:00 AM only). Owners must clean up after their pets. Aggressive dogs may be banned from city parks.

SECTION 6: EVENT PERMITS
Events with expected attendance over 50 people require a Special Event Permit. Applications must be submitted 30 days in advance. The permit fee is $100 plus $2 per expected attendee. Events over 200 people require proof of liability insurance.

Confirm your first two files exist by running:

ls documents/

You should see building_permits.txt and park_regulations.txt listed.

Create one more file in the documents folder. Right-click it, click New File, and name it waste_collection.txt.
Paste the following content into waste_collection.txt:

RIVERSIDE CITY - WASTE COLLECTION AND RECYCLING GUIDE
Updated: November 2025

SECTION 1: COLLECTION SCHEDULE
Residential waste and recycling collection occurs weekly. Your collection day is determined by your neighborhood zone:
- Zone A (Downtown, Riverside Heights): Monday
- Zone B (Oakwood, Millbrook): Tuesday
- Zone C (Westside, Harbor District): Wednesday
- Zone D (North End, University Area): Thursday

Place bins curbside by 6:00 AM on your collection day. Bins left out after 8:00 PM the same day may result in a warning notice.

SECTION 2: RECYCLING GUIDELINES
Riverside City uses single-stream recycling. Accepted materials include: paper and cardboard (flattened), glass bottles and jars (rinsed), metal cans and aluminum, and plastics marked #1 through #5. NOT accepted: plastic bags, styrofoam, food-contaminated items, electronics, and batteries.

SECTION 3: YARD WASTE
Yard waste collection is available March through November on the same day as regular collection. Use brown paper bags or clearly labeled containers (maximum 50 lbs each). Branches must be cut to 4-foot lengths and bundled. Christmas trees are collected the first two weeks of January.

SECTION 4: BULK ITEM PICKUP
Residents may schedule up to 4 bulk item pickups per year at no additional charge. Items include furniture, appliances (doors removed), mattresses, and large electronics. Schedule at least 5 business days in advance by calling 555-0198 or online at riversidecity.gov/bulk-pickup.

SECTION 5: HAZARDOUS WASTE
Hazardous materials (paint, chemicals, motor oil, batteries, electronics) must NOT be placed in regular collection bins. The Riverside City Hazardous Waste Facility at 450 Industrial Parkway accepts these materials every Saturday from 8:00 AM to 2:00 PM. Residential drop-off is free with proof of Riverside City address.

SECTION 6: MISSED COLLECTIONS
If your collection is missed, report it within 48 hours at riversidecity.gov/missed-pickup or call 555-0198. A makeup collection will be scheduled within 2 business days. Collections may be delayed by one day following city-observed holidays.

Save all three files.
Confirm all documents are in place by running:

ls documents/

You should see all three files listed: building_permits.txt, park_regulations.txt, and waste_collection.txt.

Folder is empty or missing a file?

Make sure you created each file inside the documents/ folder, not in the project root. Check that each filename ends in .txt and matches exactly.

Help me figure out why my documents folder is missing files.

Why fictional policy documents?

These documents simulate what a real government consulting project would use. In production, you would ingest actual city policy PDFs or web pages. Fictional documents let you control exactly what the "ground truth" is, so you can clearly see when the AI gets things right or makes things up.

Ask Claude a policy question cold

Now you will ask Claude a Riverside City policy question without giving it any access to your documents. This is the baseline you are trying to beat.

Create a new file in your project root called ask_claude.py by right-clicking in the Explorer sidebar and clicking New File.
Paste the following code into ask_claude.py:

import os
from anthropic import Anthropic

client = Anthropic()

question = "What do I need to apply for a building permit in Riverside City?"

message = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": question}
    ]
)

print("Question:", question)
print("\nClaude's answer (without access to city documents):")
print(message.content[0].text)

What does this code do?

Anthropic() creates a client that automatically picks up your ANTHROPIC_API_KEY environment variable.
messages.create() sends a question directly to claude-haiku-4-5 with no additional context or documents.
The question asks about Riverside City specifically, but Claude has never seen your policy documents.

Save ask_claude.py.

Before you run this, think about what Claude will say. It has never seen your Riverside City policy files. Do you think it will give accurate, specific answers about Riverside City's building permit requirements?

Run the script to see Claude's ungrounded answer:

python3 ask_claude.py

Claude will return an answer that sounds helpful and authoritative. It might mention typical building permit requirements like application forms, fees, and inspections.

Seeing an authentication error?

Make sure your ANTHROPIC_API_KEY environment variable is still set in this terminal session. If you opened a new terminal, you need to run the export command again and reactivate your virtual environment.

Help me fix my Anthropic API key authentication error.

Identify the gap

Now compare Claude's answer to what the actual Riverside City building permit policy says. Open documents/building_permits.txt in Cursor and read Section 2: Application Requirements.

Click on building_permits.txt in the Explorer sidebar to open it.
Look at the specific requirements listed under Section 2.

Notice the differences between Claude's answer and the actual policy. The real document specifies Form RC-101, a $150 residential fee, a 10-business-day review period, and inspections scheduled at 555-0142. Claude's answer likely mentioned none of these specifics. It might have even invented plausible-sounding details that contradict the real policy.

Why is this a problem for government?

A citizen who follows Claude's generic answer might miss a required form, pay the wrong fee, or show up at the wrong time. Government services need answers grounded in official documents with citations so citizens can verify the information themselves.

This is exactly the problem that RAG (Retrieval-Augmented Generation) solves. Instead of relying on the model's training data, you retrieve the relevant official documents and inject them into the prompt.

✔️ Awesome, I've got everything!

Great. Make sure you have saved ask_claude.py and all three files inside your documents/ folder.

ⓧ I'd like to double check the full code

Here is the complete ask_claude.py file. Make sure yours matches exactly:

import os
from anthropic import Anthropic

client = Anthropic()

question = "What do I need to apply for a building permit in Riverside City?"

message = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": question}
    ]
)

print("Question:", question)
print("\nClaude's answer (without access to city documents):")
print(message.content[0].text)

For the three document files, scroll up and confirm each one is present in your documents/ folder: building_permits.txt, park_regulations.txt, and waste_collection.txt.

You have seen the hallucination problem firsthand. Claude gives confident, generic answers that could mislead a citizen. In the next step, you will build a searchable knowledge base from your policy documents so the assistant can retrieve the real answers.

Build the Knowledge Base

You saw the problem firsthand. Claude gave a plausible-sounding answer about building permits, but nothing tied it to actual Riverside City policy.

The fix is a searchable knowledge base. ChromaDB stores your policy documents as vector embeddings so you can find the most relevant chunks for any question. It runs entirely on your machine with no external API and no cost.

In this step, get ready to:

Write a script that chunks policy documents and stores them in ChromaDB.
Run a test query to confirm semantic search retrieves relevant results.

Write the document chunking and ingestion script

Before the assistant can search your documents, you need to break them into smaller pieces and store those pieces in a way that supports meaning-based lookup. This script handles both jobs.

Create a new file called ingest.py in your project folder (the same folder that contains ask_claude.py).
Add the imports and the chunking function by pasting this code:

import os
import chromadb


def chunk_text(text, chunk_size=500, overlap=50):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        if chunk.strip():
            chunks.append(chunk.strip())
        start = end - overlap
    return chunks

What does this code do?

The chunk_text function splits a long document into 500-character pieces with a 50-character overlap between each piece.

The overlap ensures that sentences sitting right at a chunk boundary still appear in at least one complete chunk. Without overlap, a key sentence could get split in half and become unsearchable.

Below the chunk_text function, add the ingestion function that reads your documents and stores them in ChromaDB:

def ingest_documents(docs_folder="documents", db_path="./chroma_db"):
    """Load all .txt files, chunk them, and store in ChromaDB."""
    client = chromadb.PersistentClient(path=db_path)

    collection = client.get_or_create_collection(name="city_policies")

    doc_id = 0
    for filename in sorted(os.listdir(docs_folder)):
        if filename.endswith(".txt"):
            filepath = os.path.join(docs_folder, filename)
            with open(filepath, "r") as f:
                text = f.read()

            chunks = chunk_text(text)

            for i, chunk in enumerate(chunks):
                collection.add(
                    documents=[chunk],
                    metadatas=[{"source": filename, "chunk_index": i}],
                    ids=[f"{filename}_chunk_{doc_id}"]
                )
                doc_id += 1

            print(f"Ingested {filename}: {len(chunks)} chunks")

    print(f"\nTotal documents in collection: {collection.count()}")
    return collection

What does this function do?

PersistentClient(path=db_path) creates a ChromaDB database on disk at ./chroma_db/ so your data survives between script runs.
get_or_create_collection(name="city_policies") creates a named collection to hold your document chunks and their embeddings.
collection.add() stores each chunk along with metadata recording which file it came from. ChromaDB automatically generates a vector embedding for each chunk using its built-in model.

Why does ChromaDB not need an embedding API?

ChromaDB bundles a local embedding model (all-MiniLM-L6-v2) that runs on your machine. It converts your text into vectors automatically when you call collection.add(). No API key, no cost, no network call.

Save ingest.py.

Run ingestion and verify the knowledge base

The script can now ingest documents, but you also need a way to confirm the semantic search actually works. A test query at the bottom of the file will search for a known topic and print what it finds.

At the bottom of ingest.py, below the ingest_documents function, add the test query and the main entry point:

def test_query(collection):
    """Run a test query to verify semantic search works."""
    results = collection.query(
        query_texts=["building permit application requirements"],
        n_results=3
    )

    print("\n--- Test Query: 'building permit application requirements' ---")
    for i, doc in enumerate(results["documents"][0]):
        source = results["metadatas"][0][i]["source"]
        print(f"\nResult {i+1} (from {source}):")
        print(doc[:150] + "...")


if __name__ == "__main__":
    collection = ingest_documents()
    test_query(collection)

What does this code do?

test_query searches the collection for chunks semantically similar to "building permit application requirements" and prints the top 3 matches with their source filenames.
The if __name__ == "__main__" block runs both functions in order: first ingest all documents, then verify with a test query.

Save ingest.py.

Before you run this, what do you expect the test query results to show? Which document should rank highest for a question about building permit requirements?

Run the ingestion script in your terminal:

python3 ingest.py

You should see output confirming each file was ingested with its chunk count, followed by test query results pulling from building_permits.txt.

What should I see?

Your terminal prints something like:

Ingested building_permits.txt: 6 chunks
Ingested park_regulations.txt: 8 chunks
Ingested waste_collection.txt: 7 chunks

Total documents in collection: 21

--- Test Query: 'building permit application requirements' ---

Result 1 (from building_permits.txt):
...

Result 2 (from building_permits.txt):
...

Result 3 (from building_permits.txt):
...

The exact chunk counts may vary slightly, but all three test results should come from building_permits.txt. That proves ChromaDB understands the meaning of your query and retrieves the right documents.

Seeing an error?

If you see ModuleNotFoundError: No module named 'chromadb', make sure your virtual environment is activated. Run source venv/bin/activate (macOS/Linux) or venv\Scripts\activate (Windows) and try again.
If you see FileNotFoundError mentioning the documents folder, make sure your documents/ folder is in the same directory as ingest.py and contains your three .txt files.
If the first run downloads a model (you see progress bars), that is normal. ChromaDB downloads its embedding model on first use.

Still stuck? Help me debug my ingest.py script

✔️ Awesome, I've got everything!

Great. Double check you have saved ingest.py and that your terminal shows the ingestion output with chunk counts.

ⓧ I'd like to double check the full code

Scroll up and confirm your ingest.py contains all three sections in this order:

Imports (os and chromadb) plus the chunk_text function.
The ingest_documents function that reads files and stores chunks in ChromaDB.
The test_query function and the if __name__ == "__main__" block that calls both ingest_documents() and test_query(collection).

Your knowledge base is loaded and searchable. Next up, you will wire it to Claude so the assistant retrieves relevant policy chunks and generates cited answers.

Wire Up the RAG Pipeline

In the previous step, you built a searchable knowledge base of city policy documents. Your ChromaDB collection can find relevant chunks for any question.

Now you connect retrieval to generation. The assistant will search the knowledge base for relevant policy chunks, inject them into Claude's prompt as context, and instruct Claude to answer only from those documents with citations. The hallucination problem from Step 2 gets solved here.

In this step, get ready to:

Build a retrieval function that queries ChromaDB for relevant policy chunks.
Construct a RAG pipeline that grounds Claude's answers in official documents with citations.
Test the assistant with different question types and compare against the ungrounded answer from Step 2.

Write the retrieval function

The first piece of the pipeline takes a citizen's question and finds the most relevant document chunks from your knowledge base. This is the "retrieval" in Retrieval-Augmented Generation.

Create a new file called assistant.py in your project folder.
Add the imports and retrieval function by pasting this code:

import chromadb
from anthropic import Anthropic


def get_relevant_context(question, db_path="./chroma_db", n_results=3):
    """Retrieve the most relevant document chunks for a question."""
    client = chromadb.PersistentClient(path=db_path)
    collection = client.get_collection(name="city_policies")

    results = collection.query(
        query_texts=[question],
        n_results=n_results
    )

    contexts = []
    for i, doc in enumerate(results["documents"][0]):
        source = results["metadatas"][0][i]["source"]
        contexts.append({"text": doc, "source": source})

    return contexts

What does this code do?

It connects to the same persistent ChromaDB database you built in the previous step.
It uses get_collection (not get_or_create_collection) because the collection must already exist from running ingest.py.
It queries for the top 3 most semantically similar chunks and returns them with their source filenames.

Save assistant.py.
Verify the retrieval function works by running this quick test:

python3 -c "from assistant import get_relevant_context; r = get_relevant_context('building permit'); print(len(r), 'chunks found'); print('Source:', r[0]['source'])"

You should see 3 chunks found and Source: building_permits.txt confirming semantic search is pulling the right documents.

Seeing an import error?

Make sure your virtual environment is still activated. You should see (venv) at the start of your terminal prompt.

If you see a collection not found error, re-run python3 ingest.py to rebuild the knowledge base.

help me debug my assistant.py import error

Build the generation pipeline

The retrieval function finds relevant chunks. Now you need a function that takes those chunks, constructs a careful prompt telling Claude to answer only from those documents, and returns the grounded response.

In assistant.py, add the ask_with_rag() function below get_relevant_context(). Start with the prompt construction:

def ask_with_rag(question):
    """Send a question through the full RAG pipeline."""
    contexts = get_relevant_context(question)

    context_text = ""
    for i, ctx in enumerate(contexts, 1):
        context_text += f"\n[Source {i}: {ctx['source']}]\n{ctx['text']}\n"

    system_prompt = (
        "You are a helpful city government assistant for Riverside City. "
        "Answer citizen questions using ONLY the provided policy documents. "
        "Always cite which source document your answer comes from using the "
        "source labels provided (e.g., 'According to Source 1: building_permits.txt'). "
        "If the documents don't contain enough information to answer, say so clearly. "
        "Do not make up information that isn't in the provided documents."
    )

    user_message = (
        "Based on the following official city policy documents, "
        "please answer the citizen's question.\n\n"
        f"POLICY DOCUMENTS:\n{context_text}\n\n"
        f"CITIZEN'S QUESTION: {question}\n\n"
        "Provide a clear, helpful answer with citations to the source documents."
    )

What does this prompt do?

The system_prompt sets strict guardrails: answer ONLY from the provided documents, always cite sources, and admit when information is missing. This is what prevents hallucination.

The user_message injects the retrieved chunks with labeled source tags so Claude can reference them by name in its answer.

Still inside ask_with_rag(), add the API call and return statement directly below the user_message block:

    client = Anthropic()

    message = client.messages.create(
        model="claude-haiku-4-5",
        max_tokens=1024,
        system=system_prompt,
        messages=[
            {"role": "user", "content": user_message}
        ]
    )

    return message.content[0].text, contexts

What does this code do?

It sends the question plus retrieved context to claude-haiku-4-5, a fast and cost-effective model at $1 per million input tokens.
The system parameter passes the grounding instructions separately from the user content, giving them higher priority.
It returns both the answer text AND the source contexts, so you can show the learner which documents were used.

Save assistant.py.
Test the full pipeline by running this one-shot command:

python3 -c "from assistant import ask_with_rag; a, s = ask_with_rag('What do I need for a building permit?'); print(a[:200])"

You should see the beginning of a cited answer referencing building_permits.txt. The full pipeline is working.

Seeing an authentication error?

Your ANTHROPIC_API_KEY environment variable may have expired from your terminal session. Re-export it:

export ANTHROPIC_API_KEY=your-api-key-here

help me fix my Anthropic API authentication error

Add an interactive loop and test

A one-shot test proves the pipeline works. Now wrap it in an interactive loop so you can ask multiple questions in one session and really explore how the grounding behaves.

At the bottom of assistant.py, add the interactive loop:

if __name__ == "__main__":
    print("Riverside City AI Assistant")
    print("=" * 40)
    print("Ask questions about city policies. Type 'quit' to exit.\n")

    while True:
        question = input("Your question: ")
        if question.lower() in ["quit", "exit", "q"]:
            print("Goodbye!")
            break

        print("\nSearching city documents...\n")
        answer, sources = ask_with_rag(question)
        print(f"Answer:\n{answer}\n")
        print("Sources used:")
        for i, src in enumerate(sources, 1):
            print(f"  [{i}] {src['source']}")
        print("\n" + "-" * 40 + "\n")

What does this code do?

The loop continuously prompts for questions until the user types quit, exit, or q.
After each answer, it prints which source documents were used so you can verify the grounding yourself.

Save assistant.py.

Before you run the assistant, think back to Step 2. What was wrong with Claude's answer about building permits when it had no access to city documents?

Run the assistant:

python3 assistant.py

When prompted, type the same question from Step 2:

What do I need to apply for a building permit in Riverside City?

This time, the answer cites specific documents and references actual policy details from your files. You should see references like "According to Source 1: building_permits.txt" and specific details like Form RC-101, the $150 filing fee, and the site survey requirement.

Compare this to Step 2

In Step 2, Claude gave a generic, unverifiable answer about building permits. Now every claim traces back to a specific source document. This is the core value of RAG: the AI can only say what the documents actually contain.

Now ask a question that is only partially covered by the documents:

Can I have a barbecue in the park and bring my dog?

The assistant should give a partial answer. It can cite park regulations about dogs (leash rules, off-leash areas) and permitted activities, but it should be honest about any gaps.

Finally, ask something completely outside the documents:

What is the mayor's phone number?

The assistant should clearly state that it does not have information about that in the provided documents. No guessing, no fabrication.

Type quit to exit the assistant.

Getting hallucinated answers?

If Claude still makes up information not in the documents, double-check that your system_prompt string matches exactly. The instruction "Do not make up information" must be present.

Also check that context_text is not empty. If retrieval returns nothing, the model has no grounding to work with.

help me fix hallucination in my RAG assistant

✔️ Awesome, I've got everything!

Great. Double check you have saved assistant.py with all the functions and the interactive loop.

ⓧ I'd like to double check the full code

Here is the complete assistant.py file. Confirm your version matches:

Your file should contain these sections in order:

Imports (chromadb and Anthropic).
The get_relevant_context() function that queries ChromaDB.
The ask_with_rag() function with system prompt, user message, API call, and return.
The if __name__ == "__main__": interactive loop.

Scroll up and compare each chunk you pasted against the sections above. Make sure nothing is missing or duplicated.

You just solved the hallucination problem from Step 2. Your assistant now gives cited, grounded answers that a government agency could actually trust. Next up: take this to the next level with a web interface in the Secret Mission.

Secret mission

Add a Web Demo

Your command-line assistant proves the RAG concept works. But in a consulting interview, you need to demo something polished. In this secret mission, you will turn your assistant into a web app using Streamlit so you can show it off in a browser with a clean interface, loading feedback, and expandable source documents.

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project has no ongoing costs. You only pay for Anthropic API calls when you actively run the assistant.

Resources you used:

Anthropic API account with credit balance (approximately $0.25-0.50 spent during this project).
Local project folder on your Desktop containing ask_claude.py, ingest.py, assistant.py, app.py, the documents/ folder, and the chroma_db/ vector database.
Python virtual environment (venv/ folder inside your project).

Keep everything running

No action needed. Choose this if you plan to keep experimenting with the assistant or want to demo it later.

Your API key stays active but only costs money when you make API calls.
The local chroma_db/ folder and Python files stay on your machine at no cost.
To run the assistant again later, activate your virtual environment with source venv/bin/activate (macOS/Linux) or venv\Scripts\activate (Windows), set your ANTHROPIC_API_KEY environment variable, and run streamlit run app.py.

Pause - I'll come back to this later

Shut down running processes to free up your terminal. Your files and database remain on disk at no cost.

If the Streamlit app is still running in your terminal, press Ctrl+C to stop it.
Deactivate your virtual environment by running deactivate in the terminal.
No costs accrue while the assistant is idle. You only pay when you actively make API calls.

Delete - I don't want to use this again

Remove all project files and disable your API key to stop all possible charges.

If the Streamlit app is still running, press Ctrl+C to stop it.
Deactivate the virtual environment by running deactivate in the terminal.
Navigate to your Desktop and delete the entire project folder (the one containing ask_claude.py, assistant.py, and chroma_db/). On macOS/Linux:

cd ~/Desktop
rm -rf riverside-city-assistant

On Windows, open File Explorer, navigate to your Desktop, right-click the project folder, and select Delete.

Open console.anthropic.com in your browser.
Navigate to Settings, then API Keys.
Find the key you created for this project and disable or delete it. This prevents any future charges from that key.

Disabling vs deleting your key

Disabling your key makes it inactive immediately but lets you re-enable it later if you change your mind. Deleting it is permanent. Either option stops all future charges from that key.

Nice Work!

You just built a production-style government AI assistant from scratch. It retrieves relevant policy documents, generates grounded answers with citations, and refuses to hallucinate beyond its source material.

You've learned how to:

Build a RAG pipeline from scratch: document ingestion, chunking, vector storage with ChromaDB, semantic retrieval, and grounded generation with Claude.
Demonstrate the hallucination problem firsthand and build the solution that fixes it using context grounding with citations.
Create a working government AI assistant prototype using the exact architecture real consulting firms deploy to clients like Virginia Beach and California DOF.
Secret Mission: Built a web-ready Streamlit demo you can show in an interview.

Ready to quiz yourself?