RAG Agent with Claude Tool-Calling

Build a CLI agent that searches documents and answers questions with citations.

Introduction

30 Second Summary

AI assistants are great at general knowledge. Ask one about your company's specific safety procedures or inspection schedules, though, and you will get a confident guess instead of the real answer.

In this project, you will build a Python command-line agent that answers questions using your own documents instead of guesses. You'll load nuclear regulation documents into a vector database, wire up Claude's tool-calling protocol, and watch your agent search, retrieve, and cite real sources in its answers.

What You'll Build

Ask your agent about reactor inspection schedules and watch it search your knowledge base, pull the matching procedures, and answer with the exact figures and source document names.

By the end of this project, you'll have:

  • A ChromaDB vector knowledge base you can search by meaning. Query "inspection schedules" and get back the exact maintenance procedures, even if those exact words never appear in the documents.
  • A working Claude agent with tool-calling that decides when to search your documents, reads the results, and responds with grounded answers citing the source files.
  • A clear before-and-after comparison showing Claude's generic guess without your documents versus its precise, cited answer with RAG enabled.
  • Secret Mission: Add a document browsing tool and metadata filtering so your agent can list available sources and search within a specific document.

Are there any prerequisites?

You should be comfortable with Python basics like functions, dictionaries, loops, and file I/O. No prior AI or LLM experience is needed.

Before We Start

Before diving into code, let's lock in what you're about to build and why it matters.

Set Up Your Python Environment and API Key

Your agent needs two Python libraries before it can do anything useful. The Anthropic SDK connects your scripts to Claude's API. ChromaDB stores and searches your documents as vectors.

In this step, you will create a clean project folder, install both libraries, and configure the API key that authenticates your requests to Claude.

In this step, get ready to:
  • Create a project folder and Python virtual environment.
  • Install the anthropic and chromadb libraries.
  • Set up your Anthropic API key.
Create your project folder and install dependencies
  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or Command Prompt (Windows) and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop

A virtual environment isolates your project's packages from your system Python so nothing conflicts. Let's create a project folder and set one up now.

  • Create your project folder, move into it, and set up a virtual environment by running these commands:

macOS

mkdir rag-agent
cd rag-agent
python3 -m venv venv
source venv/bin/activate

Linux users can run the same commands.

Windows

mkdir rag-agent
cd rag-agent
python -m venv venv
venv\Scripts\activate

Your terminal prompt now shows (venv) at the beginning. This confirms the virtual environment is active.

What is a virtual environment?

A virtual environment is a self-contained Python installation tied to a single project. Packages you install inside it stay separate from your system Python and from other projects.

This prevents version conflicts. If one project needs chromadb version 1 and another needs version 2, virtual environments let both coexist on the same machine.

Don't see (venv) in your prompt?

Make sure you are inside the rag-agent folder before running the activation command. Also confirm you are using the correct command for your operating system.

help me activate my Python virtual environment in the rag-agent folder

  • Install the Anthropic SDK and ChromaDB by running:
pip install anthropic chromadb

You should see pip download and install both packages along with their dependencies. This may take a minute or two.

Install failing or hanging?

If you see network errors, check your internet connection and try again. If the install fails with a build error, make sure your pip is up to date by running:

pip install --upgrade pip

Then run the install command again.

help me fix a pip install error for anthropic and chromadb

Create your Anthropic account and set your API key

To call Claude's API, you need an API key from Anthropic. This key authenticates every request your code sends.

  • Open your browser and go to platform.claude.com.
  • Sign in with an existing account, or create a new one if you do not have one yet.
  • Click Settings in the left sidebar.
  • Click API keys.
  • Click Create key and give the key a name (e.g. rag-agent).
  • Copy the key immediately. It starts with sk-ant- and is only shown once.

Why is the key only shown once?

Anthropic never stores the full key after creation. If you lose it, you cannot retrieve it from the console. You would need to create a new one.

Store it somewhere safe right now, such as a password manager. New accounts receive a small amount of free credits, so this project costs very little to run.

Now set the key as an environment variable so the Anthropic SDK can find it automatically.

macOS

  • In your terminal, set the environment variable by running:
export ANTHROPIC_API_KEY="your-api-key-here"

Replace your-api-key-here with the key you just copied. This variable stays active for your current terminal session.

Linux users can run the same command.

Windows

  • In your terminal, set the environment variable permanently by running:
setx ANTHROPIC_API_KEY "your-api-key-here"

Replace your-api-key-here with the key you just copied.

  • Close your current terminal window.
  • Open a new terminal (the setx command only takes effect in new sessions).
  • Navigate back to your project and reactivate the virtual environment by running:
cd ~\Desktop\rag-agent
venv\Scripts\activate

Not sure if the key is set?

The verification step coming next confirms whether the key is configured correctly. If you see import errors there, double-check that you pasted the full key and that the quotes are matched.

help me set my ANTHROPIC_API_KEY environment variable

Verify your setup

Before you run the next command, what do you expect to see if both libraries installed correctly and the key is set?

  • Check that both libraries are importable by running:
python -c "from anthropic import Anthropic; import chromadb; print('anthropic OK'); print('chromadb', chromadb.__version__)"

You should see anthropic OK on one line, followed by chromadb and a version number like 1.5.9 on the next.

Seeing ModuleNotFoundError?

This means Python cannot find one of the packages. Make sure your virtual environment is active. You should see (venv) at the start of your terminal prompt.

If (venv) is missing, reactivate it using the activation command for your operating system, then run the check again.

help me fix ModuleNotFoundError for anthropic or chromadb

Your environment is set up and both libraries are ready to go. In the next step, you will send your first question to Claude and discover an important gap in its knowledge.

Ask Claude Without Your Documents

Your Python environment is set up, your packages are installed, and your API key is ready. Now it's time to actually talk to Claude.

Before building a retrieval system, you need to see the problem it solves. Claude is a powerful language model, but it has no idea what's inside your organization's internal documents. Let's prove that gap by asking it a very specific question about nuclear maintenance procedures.

In this step, get ready to:
  • Create chat.py to send Claude a question about specific internal nuclear procedures.
  • Run the script and observe how Claude responds without access to your documents.
Create a script to ask Claude about your procedures

You are going to ask Claude about a specific internal procedure: the inspection frequency for Category A components in a reactor maintenance program. This is the kind of question that has a precise, documented answer inside an organization.

  • In your code editor, create a new file called chat.py and add the following code:
# Import the Anthropic SDK
from anthropic import Anthropic

# Create a client (reads ANTHROPIC_API_KEY from your environment)
client = Anthropic()

# A specific question about internal procedures
question = "What is the inspection frequency for Category A components in our reactor maintenance program?"

print(f"Question: {question}\n")

# Send the question to Claude
message = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": question}
    ],
)

# Print Claude's response
for block in message.content:
    if block.type == "text":
        print(f"Claude's answer:\n{block.text}")

What does this code do?

  • Anthropic() creates a client that automatically reads your ANTHROPIC_API_KEY environment variable. No need to pass the key manually.
  • messages.create() sends your question to the claude-sonnet-5 model and returns Claude's response.
  • The for loop at the bottom extracts the text content from the response. Claude returns a list of content blocks, so you iterate over them to find the text.
  • Save chat.py.

✔️ Awesome, I've got everything!

Great. Double check you've saved chat.py and move on to run it.

ⓧ I'd like to double check the full code

Here is the complete chat.py file. Make sure yours matches exactly:

from anthropic import Anthropic

client = Anthropic()

question = "What is the inspection frequency for Category A components in our reactor maintenance program?"

print(f"Question: {question}\n")

message = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": question}
    ],
)

for block in message.content:
    if block.type == "text":
        print(f"Claude's answer:\n{block.text}")
Run the script and observe Claude's gap

Before you run this, think about it: does Claude have any way to know what your specific reactor maintenance procedures say? What kind of answer do you expect?

  • Run your script in the terminal:
python chat.py

Claude responds, but read the answer carefully. It gives a generic, hedged response about typical industry practices. It does not mention an 18-month inspection interval, refueling outages, or any of the specific details that your internal documents contain.

Why is the answer generic?

Claude was trained on public information, so it can speak generally about nuclear maintenance. But it has zero access to your organization's specific procedures, inspection schedules, or priority classifications.

This is exactly the gap that Retrieval-Augmented Generation (RAG) solves. Instead of hoping the model was trained on your data, you give it a tool to search your documents at query time.

Seeing an authentication error?

  • Make sure your ANTHROPIC_API_KEY environment variable is set in the same terminal session where you are running the script. On Windows, if you used setx, you need to open a new terminal window for it to take effect.
  • Verify the variable is set by running echo $ANTHROPIC_API_KEY (macOS/Linux) or echo %ANTHROPIC_API_KEY% (Windows). You should see your key starting with sk-ant-.
  • Check that your virtual environment is still activated. You should see its name in parentheses at the start of your terminal prompt.

Help me debug my Anthropic API authentication error.

You have seen the gap firsthand. Claude gave its best guess, but it had no specific numbers, no document references, and no real authority behind its answer. Next up, you will create the actual documents and build a searchable knowledge base so Claude can find real answers.

Build Your Vector Knowledge Base

In Step 2, you asked Claude about your reactor maintenance program and got a generic, hedged answer. That gap exists because Claude has no way to access your internal documents.

In this step, you'll fix that by creating a vector database of sample nuclear regulation documents. You'll use ChromaDB to store your documents as searchable embeddings, so they can be retrieved at query time.

In this step, get ready to:
  • Create three sample nuclear regulation documents with specific internal data.
  • Build a Python script that loads, chunks, and stores those documents in ChromaDB.
  • Verify that semantic search returns relevant results from your knowledge base.
Create your sample documents

Your knowledge base needs source documents with specific internal data. These documents intentionally include exact numbers and procedures that Claude would not know from its training data.

  • Create the documents folder by running this command:
mkdir documents
  • In your code editor, create a new file called maintenance_procedures.txt inside the documents folder and paste the following content:
REACTOR MAINTENANCE PROGRAM - INTERNAL PROCEDURES

Section 1: Component Inspection Schedule

Category A components (primary coolant system, reactor pressure vessel, steam generators) require inspection every 18 months during scheduled refueling outages. Full volumetric examination of Category A welds must be completed within a 10-year interval, with at least 25% completed by the end of the first third of the interval.

Category B components (pressure-retaining boundaries, piping systems, pump casings) follow a 40-month inspection cycle for surface examination. Volumetric testing is performed on a rotating 10-year basis.

Category C components (support structures, component supports, core support structures) are inspected visually every refueling outage, with detailed examination every 60 months.

Section 2: Corrective Maintenance Priorities

Priority 1 (Safety-Critical): Must be addressed within 24 hours. Includes any degradation of safety-related equipment that could affect reactor trip capability or emergency core cooling.

Priority 2 (Important to Safety): Must be addressed within 7 days. Includes degradation of equipment important to safety but with available redundancy.

Priority 3 (Standard): Scheduled during next available maintenance window. Includes non-safety-related equipment maintenance and cosmetic repairs.

Section 3: Documentation Requirements

All maintenance activities must be documented in the Work Order Management System (WOMS). Each work order must include: task description, applicable procedures, parts list, quality hold points, and post-maintenance testing requirements. Records must be retained for the life of the plant plus 5 years.
  • Create a second file called emergency_planning.txt inside the documents folder and paste this content:
EMERGENCY PLANNING AND RESPONSE PROCEDURES

Section 1: Emergency Planning Zones

The Emergency Planning Zone (EPZ) is divided into two areas:
- Plume Exposure Pathway EPZ: Extends approximately 10 miles (16 km) from the reactor. This zone addresses exposure to radioactive plume and deposited materials. Protective actions include evacuation and sheltering.
- Ingestion Exposure Pathway EPZ: Extends approximately 50 miles (80 km) from the reactor. This zone addresses potential ingestion of contaminated water and food supplies.

Section 2: Emergency Classification Levels

Our facility uses four emergency classification levels:

1. Notification of Unusual Event (NOUE): Events that indicate potential degradation of plant safety. No release of radioactive material requiring offsite response is expected. Notification to state and local agencies within 15 minutes.

2. Alert: Events that involve actual or potential substantial degradation of plant safety. Any releases are expected to be limited to small fractions of EPA Protective Action Guideline (PAG) exposure levels. Notification within 15 minutes, activation of onsite emergency response organization.

3. Site Area Emergency: Events that involve actual or likely major failures of plant functions needed for protection of the public. Releases are not expected to exceed EPA PAG exposure levels beyond the site boundary. Full activation of emergency response organizations.

4. General Emergency: Events that involve actual or imminent substantial core degradation or melting with potential for loss of containment integrity. Releases can be expected to exceed EPA PAG exposure levels offsite. Immediate notification, recommendation of protective actions to offsite authorities.

Section 3: Notification Procedures

The Shift Supervisor is responsible for initial emergency classification. Upon classification, notifications must be made to the NRC Operations Center, state emergency management agency, and local county emergency management within 15 minutes. The Emergency Response Organization (ERO) is activated based on the classification level.
  • Create a third file called safety_systems.txt inside the documents folder. Paste the first two sections:
REACTOR SAFETY SYSTEMS OVERVIEW

Section 1: Defense in Depth

Our reactor design employs a defense-in-depth philosophy with multiple independent barriers and safety systems:

Barrier 1 - Fuel Cladding: Zircaloy cladding encases the uranium fuel pellets, providing the first barrier against fission product release. Cladding integrity is monitored continuously through reactor coolant activity measurements.

Barrier 2 - Reactor Coolant System: The primary pressure boundary contains all radioactive coolant and prevents release to the containment building. System pressure is maintained at approximately 2,235 psi (15.4 MPa) during normal operation.

Barrier 3 - Containment Structure: The reinforced concrete containment building with steel liner provides the final barrier. Design pressure is 60 psig, with leak rate testing performed every 10 years per 10 CFR 50 Appendix J.

Section 2: Emergency Core Cooling System (ECCS)

The ECCS consists of three subsystems designed to provide cooling water to the reactor core in the event of a loss-of-coolant accident (LOCA):

1. High-Pressure Injection (HPI): Automatically activates when reactor coolant system pressure drops below 1,600 psi. Uses centrifugal charging pumps to inject borated water from the Refueling Water Storage Tank (RWST).

2. Low-Pressure Injection (LPI): Activates when RCS pressure drops below 200 psi. Uses residual heat removal pumps to deliver large volumes of borated water directly to the reactor vessel.

3. Accumulators: Passive safety system consisting of pressurized tanks containing borated water at 600 psi. Automatically inject when RCS pressure drops below accumulator pressure, requiring no operator action or electrical power.
  • Now add Section 3 to the same safety_systems.txt file, directly below the Accumulators paragraph:

Section 3: Reactor Protection System (RPS)

The RPS automatically initiates a reactor trip (SCRAM) when monitored parameters exceed predetermined setpoints. Key trip parameters include:
- High neutron flux: 118% of rated thermal power
- High pressurizer pressure: 2,385 psi
- Low pressurizer pressure: 1,860 psi
- High containment pressure: 3 psig
- Low reactor coolant flow: 90% of nominal per loop
- High steam generator level: 90% narrow range

The RPS uses redundant logic channels (2-out-of-4 coincidence) to prevent spurious trips while maintaining reliability. Average reactor trip response time from setpoint to rod insertion is less than 2 seconds.

Why do these documents have specific numbers?

Each document includes precise figures: 18-month inspection intervals, a 10-mile EPZ radius, 2,235 psi operating pressure. Claude does not know these internal specifics from its training data.

When you ask Claude about these details later, the difference between a generic answer and a grounded answer with exact numbers will prove that your RAG pipeline is working.

  • Save all three files.
  • Confirm the files exist by running:
ls documents/

You should see emergency_planning.txt, maintenance_procedures.txt, and safety_systems.txt listed.

Not seeing all three files?

  • Make sure each file is saved inside the documents folder, not in the project root.
  • Check that each filename ends with .txt and matches the exact names above.

Still stuck? Help me figure out why my document files aren't showing up in the documents folder.

Write the document loading and chunking functions

Now you need a Python script that can read those text files and split them into smaller pieces. Splitting documents into chunks is essential because LLMs have limited context windows, and smaller chunks produce more focused search results.

  • Create a new file called knowledge_base.py in your project root (the same folder that contains chat.py) and paste the following code:
import os
import chromadb

def load_documents(directory):
    """Load all .txt files from a directory and return as a list of dicts."""
    documents = []
    for filename in sorted(os.listdir(directory)):
        if filename.endswith(".txt"):
            filepath = os.path.join(directory, filename)
            with open(filepath, "r", encoding="utf-8") as f:
                content = f.read()
            documents.append({
                "filename": filename,
                "content": content,
            })
    return documents

def chunk_text(text, chunk_size=500, overlap=50):
    """Split text into overlapping chunks by character count."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        chunks.append(chunk)
        start = end - overlap
    return chunks

What does this code do?

  • load_documents() reads every .txt file in a directory and returns a list of dicts, each with the filename and its content.
  • chunk_text() splits a long text into 500-character pieces with a 50-character overlap. The overlap ensures that context is not lost at chunk boundaries.
  • Save knowledge_base.py.
  • Verify that the document loading works by running this command:
python -c "from knowledge_base import load_documents; docs = load_documents('documents'); print(f'{len(docs)} documents loaded')"

You should see 3 documents loaded printed in your terminal.

Seeing an import error?

  • Make sure your virtual environment is still activated. You should see (venv) at the start of your terminal prompt.
  • Check that knowledge_base.py is saved in the same folder where you are running the command.

Still stuck? Help me debug why importing from knowledge_base.py is failing.

Build and test the knowledge base

Your loading and chunking functions work. Now you need a function that ties everything together: load the documents, chunk them, and store the chunks in a ChromaDB collection where they can be searched by meaning.

  • Add the following function below chunk_text() in knowledge_base.py:
def build_knowledge_base(documents_dir="documents"):
    """Load documents, chunk them, and store in ChromaDB."""
    client = chromadb.Client()
    collection = client.get_or_create_collection(name="nuclear_docs")

    documents = load_documents(documents_dir)

    all_chunks = []
    all_ids = []
    all_metadatas = []

    for doc in documents:
        chunks = chunk_text(doc["content"])
        for i, chunk in enumerate(chunks):
            chunk_id = f"{doc['filename']}_chunk_{i}"
            all_chunks.append(chunk)
            all_ids.append(chunk_id)
            all_metadatas.append({
                "source": doc["filename"],
                "chunk_index": i,
            })

    collection.add(
        ids=all_ids,
        documents=all_chunks,
        metadatas=all_metadatas,
    )

    print(f"Loaded {len(all_chunks)} chunks from {len(documents)} documents.")
    return collection

What does this function do?

  • chromadb.Client() creates an in-memory database instance. Data lives only while the script runs.
  • get_or_create_collection() creates a named collection called nuclear_docs to hold your document chunks.
  • Each chunk gets a unique ID, the text itself, and metadata recording which source file it came from.
  • collection.add() stores the chunks. ChromaDB automatically converts each chunk into a vector embedding using its default all-MiniLM-L6-v2 model.
  • Save knowledge_base.py.
  • Now add the search function and a test block below build_knowledge_base():
def search_knowledge_base(collection, query, n_results=3):
    """Search the knowledge base for relevant chunks."""
    results = collection.query(
        query_texts=[query],
        n_results=n_results,
    )
    return results

if __name__ == "__main__":
    collection = build_knowledge_base()

    query = "emergency planning zones"
    results = search_knowledge_base(collection, query)

    print(f"\nSearch results for: '{query}'\n")
    for i, doc in enumerate(results["documents"][0]):
        source = results["metadatas"][0][i]["source"]
        distance = results["distances"][0][i]
        print(f"--- Result {i + 1} (source: {source}, distance: {distance:.4f}) ---")
        print(doc[:200] + "...")
        print()

What does this code do?

  • search_knowledge_base() runs a semantic search against your collection. ChromaDB converts the query into an embedding and finds the closest matching chunks by meaning, not exact keywords.
  • The if __name__ == "__main__" block runs only when you execute this file directly. It builds the knowledge base, then searches for "emergency planning zones" and prints the matching chunks with their source filenames and distance scores.
  • Save knowledge_base.py.

First run downloads the embedding model

ChromaDB's default embedding function uses the all-MiniLM-L6-v2 model, which runs locally on your machine. The first time you run the script, ChromaDB downloads the model files (approximately 80 MB). This is a one-time download and may take a moment depending on your connection.

Before you run the script, what do you think the search results for "emergency planning zones" will return? Which of your three documents should contain the most relevant match?

  • Run the knowledge base script by running:
python knowledge_base.py

You should see output starting with Loaded 9 chunks from 3 documents, followed by search results showing matching chunks from emergency_planning.txt with their source filenames and distance scores.

What do the distance scores mean?

Lower distance means closer semantic match. A result from emergency_planning.txt with a low distance score confirms that the vector database found the right document based on meaning, not keyword matching.

Seeing an error about the embedding model?

  • If the download of all-MiniLM-L6-v2 fails, check your internet connection and try running the script again.
  • If you see a FileNotFoundError, make sure your documents folder is in the same directory where you are running the command.

Still stuck? Help me debug why running knowledge_base.py is failing.

✔️ Awesome, I've got everything!

Great. Double check you have saved knowledge_base.py and that your terminal output shows 9 chunks loaded and search results with source metadata.

ⓧ I'd like to double check the full code

Your knowledge_base.py was built in three chunks above. Scroll up and confirm you have all three sections present, in this order:

  • Imports (os and chromadb) plus the load_documents() and chunk_text() functions.
  • The build_knowledge_base() function that creates the ChromaDB collection and stores chunks.
  • The search_knowledge_base() function and the if __name__ == "__main__" test block at the bottom.

Your knowledge base is loaded, chunked, and searchable. Next up, you'll wire this knowledge base to Claude using tool-calling so the agent can search your documents on its own.

Wire Up the Tool-Calling Agent

In the last step, you built a knowledge base that stores your nuclear documents as searchable vector chunks. You proved it works by running a semantic search and seeing the right chunks come back.

The missing piece is clear. Claude still cannot access that knowledge base on its own. In this step, you define a search tool that Claude can call, and wire it into an agentic loop where Claude decides what to search, reads the results, and responds with cited answers.

In this step, get ready to:
  • Define a search_documents tool with a JSON schema that Claude can call.
  • Implement an agentic loop that handles Claude's tool calls and feeds results back.
  • Run the agent and compare its answer to the generic response from Step 2.
Create the agent file and define your search tool

Tool-calling lets Claude request information from your code mid-conversation. You describe a tool using a JSON schema, and Claude decides when to call it based on the user's question.

  • Create a new file called agent.py in your rag-agent project folder.
  • Add the imports, client setup, and tool definition by pasting the following code:
from anthropic import Anthropic
from knowledge_base import build_knowledge_base, search_knowledge_base

client = Anthropic()
collection = build_knowledge_base()

tools = [
    {
        "name": "search_documents",
        "description": "Search the nuclear energy knowledge base for relevant document chunks. Use this when the user asks about nuclear regulations, safety procedures, emergency planning, maintenance procedures, or any domain-specific topic.",
        "input_schema": {
            "type": "object",
            "properties": {
                "query": {
                    "type": "string",
                    "description": "The search query to find relevant documents."
                },
                "n_results": {
                    "type": "integer",
                    "description": "Number of results to return. Defaults to 3.",
                    "default": 3
                }
            },
            "required": ["query"]
        }
    }
]

What does this code do?

The first two lines import the Anthropic client and the two functions from your knowledge_base.py file. The client and collection are created at the module level so they are ready the moment the script loads.

The tools list describes the search_documents tool using a JSON schema. Claude reads this schema to understand what the tool does and what parameters it accepts. The query parameter is required, while n_results is optional and defaults to 3.

The tool definition tells Claude what it can call. Now you need the function that actually runs when Claude calls it.

  • Add the execute_tool function below the tools list by pasting the following code:
def execute_tool(tool_name, tool_input):
    """Execute a tool call and return the result as a string."""
    if tool_name == "search_documents":
        query = tool_input["query"]
        n_results = tool_input.get("n_results", 3)
        results = search_knowledge_base(collection, query, n_results)

        if not results["documents"][0]:
            return "No relevant documents found."

        output_parts = []
        for i, doc in enumerate(results["documents"][0]):
            source = results["metadatas"][0][i]["source"]
            output_parts.append(f"[Source: {source}]\n{doc}")

        return "\n\n---\n\n".join(output_parts)

    return f"Unknown tool: {tool_name}"

What does this function do?

When Claude calls a tool, your code receives the tool name and the inputs Claude chose. execute_tool matches the name to search_documents, then calls search_knowledge_base() with Claude's chosen query.

The results are formatted with a [Source: filename] header on each chunk so Claude can cite where the information came from in its response.

  • Save agent.py.
  • Verify the file loads correctly by running this command in your terminal:
python -c "import agent"

You should see Loaded 9 chunks from 3 documents printed. This confirms that the module-level build_knowledge_base() call ran successfully when agent.py was imported.

Seeing an ImportError?

  • Make sure knowledge_base.py is in the same folder as agent.py. Python looks for modules in the current directory.
  • Check that your virtual environment is still activated. You should see (venv) at the start of your terminal prompt.
  • Make sure you are running the command from inside your rag-agent project folder, not from a parent directory.

Still stuck? Ask for help.

Build the agentic loop

You have a tool definition and an executor. The piece that ties them together is the agentic loop. This is a while loop that sends a message to Claude, checks whether Claude wants to call a tool, executes the tool if so, feeds the result back to Claude, and repeats until Claude produces a final answer.

  • In agent.py, add the first half of the run_agent function below execute_tool. This part sets up the conversation, calls the API, and handles the case where Claude responds directly:
def run_agent(user_message):
    """Run the agent loop until Claude produces a final answer."""
    messages = [{"role": "user", "content": user_message}]
    max_turns = 5
    turns = 0

    while turns < max_turns:
        response = client.messages.create(
            model="claude-sonnet-5",
            max_tokens=1024,
            system="You are a nuclear energy knowledge assistant. Use the search_documents tool to find information before answering questions about nuclear regulations, safety, or procedures. Always cite the source document when providing information. If the search results do not contain relevant information, say so.",
            tools=tools,
            messages=messages,
        )

        if response.stop_reason == "end_turn":
            for block in response.content:
                if block.type == "text":
                    return block.text
            return ""

What does this code do?

The function starts a conversation with Claude by wrapping the user's question in a messages list. Each call to messages.create() passes the tools list so Claude knows what it can call.

After each response, the code checks response.stop_reason. If it is end_turn, Claude has finished thinking and produced a final text answer. The function extracts that text and returns it.

The other possibility is that Claude wants to call a tool before answering. You handle that next.

  • Add the tool-calling handler directly below the return "" line, still inside the while loop. Make sure the indentation matches (two levels in from the function):
        if response.stop_reason == "tool_use":
            tool_use_blocks = [
                b for b in response.content if b.type == "tool_use"
            ]
            messages.append({
                "role": "assistant",
                "content": response.content,
            })

            tool_results = []
            for tool_block in tool_use_blocks:
                print(f"  [Searching: '{tool_block.input.get('query', '')}']")
                result = execute_tool(tool_block.name, tool_block.input)
                tool_results.append({
                    "type": "tool_result",
                    "tool_use_id": tool_block.id,
                    "content": result,
                })

            messages.append({"role": "user", "content": tool_results})
            turns += 1
        else:
            return "[Unexpected stop reason]"

    return "[Max turns reached]"

What does the tool-calling handler do?

When stop_reason is tool_use, Claude is saying "I need to call a tool before I can answer." The code extracts all tool-call blocks from the response, then runs each one through execute_tool().

The results are sent back to Claude as tool_result messages. Claude reads these results, then either responds with a final answer or requests another tool call. The max_turns = 5 guard prevents infinite loops.

Why build the loop manually?

The Anthropic SDK has higher-level helpers, but the manual loop teaches the actual protocol you will encounter in production. You see the raw tool_use and tool_result message flow. Understanding this flow makes it straightforward to add new tools, add logging, or debug unexpected behavior.

  • Save agent.py.
  • Verify the function parses correctly by running:
python -c "from agent import run_agent; print('Agent ready')"

You should see Loaded 9 chunks from 3 documents followed by Agent ready. This confirms that run_agent was defined without syntax errors.

Seeing a SyntaxError or IndentationError?

  • The tool-calling handler block (the code starting with if response.stop_reason == "tool_use":) must be indented at the same level as the if response.stop_reason == "end_turn": block above it. Both sit inside the while loop.
  • The return "[Max turns reached]" line must be at the same indentation level as the while keyword. It is outside the loop but inside the function.

Need a hand? Ask for help debugging your indentation.

The last piece is a CLI loop so you can have an interactive conversation with the agent.

  • Add the following code at the very bottom of agent.py, below the run_agent function:
if __name__ == "__main__":
    print("Nuclear Energy Knowledge Agent")
    print("Type 'quit' to exit.\n")

    while True:
        user_input = input("You: ").strip()
        if user_input.lower() in ("quit", "exit"):
            print("Goodbye!")
            break
        if user_input:
            answer = run_agent(user_input)
            print(f"\nAgent: {answer}\n")

What does the CLI loop do?

This gives you an interactive prompt where you can type questions and see the agent's responses. Each time you type a question, run_agent() handles the full cycle: sending the question to Claude, executing any tool calls, and returning the final answer. Type quit or exit to stop the agent.

  • Save agent.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved agent.py before moving on.

ⓧ I'd like to double check the full code

Your agent.py file should contain these sections in order. Scroll up through your file and confirm each one is present:

  • Two import lines (from anthropic and from knowledge_base).
  • Module-level client and collection setup.
  • The tools list with the search_documents definition.
  • The execute_tool() function.
  • The run_agent() function with the while turns < max_turns loop containing both end_turn and tool_use handlers.
  • The if __name__ == "__main__": CLI loop at the bottom.
Ask Claude with your documents

In Step 2, you asked Claude about Category A component inspection frequencies and got a generic, hedged answer. Now your agent can search your actual documents before responding.

Before you run the agent, think about this: do you expect Claude's answer to be different this time? What specifically should change now that your documents are searchable?

  • Start the agent by running:
python agent.py

You should see Loaded 9 chunks from 3 documents followed by the You: prompt.

  • Type the same question you used in Step 2 and press Enter:
What is the inspection frequency for Category A components in our reactor maintenance program?

Watch the terminal. You should see a [Searching: '...'] line appear first. This means Claude decided to call your search_documents tool before answering. After the search completes, Claude responds with a precise answer that references maintenance_procedures.txt and cites the specific 18-month inspection interval from your document.

Compare this to what you saw in Step 2. Instead of a generic answer, the agent pulls exact figures from your internal documents and tells you where it found them.

What just happened behind the scenes?

  • Your question went to Claude with the search_documents tool definition attached.
  • Claude recognized it needed your documents to answer, so it responded with stop_reason: "tool_use" and a search query.
  • Your agentic loop caught that, ran the search against ChromaDB, and sent the matching chunks back as a tool_result.
  • Claude read the chunks, synthesized a grounded answer, and responded with stop_reason: "end_turn". That is the complete RAG cycle in action.
  • Try a couple more questions to see the agent search different documents. For example:
What are the emergency planning zones and how far do they extend?
  • When you are done exploring, type quit and press Enter to exit the agent.

Agent not returning cited answers?

  • If you do not see the [Searching: '...'] line, Claude may not be calling the tool. Double check that the tools=tools parameter is included in the messages.create() call inside run_agent.
  • If you see an authentication error, make sure your ANTHROPIC_API_KEY environment variable is still set in this terminal session.
  • If you see [Max turns reached] instead of an answer, there may be an issue with how tool results are sent back. Verify that the tool_results list is appended as a user message with "role": "user".

Still stuck? Ask for help.

You now have a working RAG agent that searches your internal documents and responds with grounded, cited answers. This is the same architecture that production AI assistants use to answer questions about internal data. In the Secret Mission, you will extend this agent with a second tool and metadata filtering to make it even more capable.

Secret mission

Add Document Browsing and Filtered Search

Your agent has one tool. Real-world agents juggle multiple capabilities. Add a second tool that lets Claude browse available documents, and add metadata filtering so searches can target a specific source file.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing cloud costs. The only expense was the Claude API usage during the steps, which should have been well under $0.50.

Decide whether to keep your resources in place, clean up your environment variable, or delete everything entirely.

Resources you used:

  • Project folder on your Desktop containing chat.py, knowledge_base.py, agent.py, the documents/ folder, and your Python virtual environment.
  • ANTHROPIC_API_KEY environment variable.

Keep everything running

No action needed. Choose this if you plan to keep experimenting with your RAG agent or adding more documents to your knowledge base.

  • Your project folder stays on your Desktop, ready to use whenever you run python agent.py again.
  • Your ANTHROPIC_API_KEY environment variable stays set so you can call the Claude API from other projects too.
  • ChromaDB runs in-memory, so no database process is running between script executions. There is nothing to pause or stop.

Pause - I'll come back to this later

There is nothing actively running between script executions. Your project folder, virtual environment, and documents are all static files on disk. No processes need to be stopped or paused.

  • If you want to free up terminal sessions, close any terminal windows that have the virtual environment activated.
  • When you come back, reactivate your virtual environment and run python agent.py to pick up where you left off.

Delete - I don't want to use this again

Remove all project files and clean up your environment. This deletes your scripts, documents, and virtual environment permanently.

  • Delete your project folder from the Desktop by running this command:

macOS

cd ~/Desktop
rm -rf [[PROJECT_FOLDER="your project folder name"]]

If you set ANTHROPIC_API_KEY using export in your terminal, it is already gone when you close that terminal session. If you added it to your shell profile (like .zshrc or .bashrc), open that file and remove the line that sets ANTHROPIC_API_KEY.

Windows

cd ~\Desktop
rmdir /s /q [[PROJECT_FOLDER="your project folder name"]]

Since you used setx to set ANTHROPIC_API_KEY, it persists across terminal sessions. Remove it by running:

setx ANTHROPIC_API_KEY ""

Close and reopen your terminal for the change to take effect.

  • Confirm the folder is gone by running ls on your Desktop. Your project folder should no longer appear in the output.

Nice Work!

Nice Work!

You just built a fully functional RAG agent that searches a local knowledge base and answers questions with source citations, all powered by Claude tool-calling.

You've learned how to:

  • Build a RAG pipeline from scratch: loading documents, chunking text with overlap, storing vector embeddings in ChromaDB, and running semantic search queries that return ranked results with source metadata.
  • Implement Claude's tool-calling protocol: defining tools with JSON schemas, writing the agentic loop that checks stop_reason on each response, and handling the tool_use/tool_result message flow.
  • See the before and after of RAG in action: Claude without your documents gave generic, hedged answers, while Claude with tool-calling RAG returned grounded answers citing your exact internal data.
  • Secret Mission: Added multi-tool support with a list_documents tool and metadata filtering on search_documents, showing you can extend an agent's capabilities.

Ready to quiz yourself?