Build a Document Research Assistant

Build a RAG pipeline then upgrade it to an agentic system with Gemini.

Introduction

30 Second Summary

You have a folder full of notes, docs, or research. You need answers that pull from multiple sources and connect the dots. Searching one document at a time does not cut it.

In this project, you will build a document research assistant that starts as a basic RAG pipeline and evolves into an agentic system. You will watch it fail on complex questions, then upgrade it with a ReAct reasoning loop that retrieves iteratively until it finds a complete answer.

What You'll Build

Picture this: you ask your assistant a question that spans three documents, and instead of giving you a half-baked answer, it thinks through what it needs, retrieves from multiple sources, and delivers a fully grounded response with its reasoning trace visible.

By the end of this project, you'll have:

  • A basic RAG pipeline that embeds your documents with Gemini, retrieves relevant chunks via vector similarity search, and answers simple questions with source attribution.
  • A visible failure where you watch basic RAG produce a weak, incomplete answer on a multi-step question that requires information from multiple documents.
  • An agentic RAG system powered by a ReAct agent that reasons step-by-step, decomposes complex queries, and makes multiple retrieval calls to build complete answers.
  • A side-by-side comparison script showing when basic RAG is sufficient and when agentic RAG justifies its extra cost.
  • Secret Mission: Add a web search tool so your agent can combine document knowledge with live information from the internet.

Are there any prerequisites?

You should be comfortable with Python (functions, packages, and async basics). You also need a Google AI Studio account, which gives you free access to the Gemini API with no billing setup required.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters for your understanding of how AI retrieval systems work.

Set Up Your Environment and Get a Gemini API Key

Before you can build anything, you need the right tools in place. Your document research assistant relies on LlamaIndex to manage retrieval and Google's Gemini API to generate answers.

In this step, you will create a project folder, install the packages that power your RAG pipeline, get a free Gemini API key, and create the sample documents your system will search over.

In this step, get ready to:
  • Create a project folder with a virtual environment and install LlamaIndex.
  • Get a Gemini API key from Google AI Studio.
  • Create sample documents for your RAG system to search.
Create your project folder and install LlamaIndex

You need an isolated Python environment so your project's packages do not conflict with anything else on your machine. A virtual environment handles this.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • Open a terminal inside VS Code by pressing Ctrl+` (Windows) or Cmd+` (macOS).
  • Move to your Desktop and create the project folder by running these commands:
cd ~/Desktop
mkdir rag-agent
cd rag-agent
  • Open this folder in VS Code by clicking File in the top menu bar.
  • Click Open Folder.
  • Select the rag-agent folder on your Desktop, then click Open.

You should see rag-agent appear in the VS Code Explorer sidebar on the left.

  • Open a fresh terminal in the new workspace by pressing Ctrl+` (Windows) or Cmd+` (macOS).
  • Create a virtual environment by running:
python -m venv .venv

A .venv folder appears in the Explorer sidebar. This contains your isolated Python environment.

  • Activate the virtual environment by running the command for your operating system:

macOS

source .venv/bin/activate

Windows

.venv\Scripts\activate

Your terminal prompt now shows (.venv) at the beginning. This confirms the virtual environment is active.

  • Install the LlamaIndex packages by running:
pip install llama-index-core llama-index-llms-google-genai llama-index-embeddings-google-genai

What are these packages?

  • llama-index-core is the core framework that handles document loading, indexing, and querying.
  • llama-index-llms-google-genai connects LlamaIndex to Google's Gemini models for generating answers.
  • llama-index-embeddings-google-genai uses Gemini's embedding model to convert your documents into searchable vectors.

Before you run the next command, what do you think will happen if LlamaIndex installed correctly?

  • Confirm the installation succeeded by running:
python -c "from llama_index.core import VectorStoreIndex; print('LlamaIndex ready')"

You should see LlamaIndex ready printed with no errors.

Seeing a ModuleNotFoundError?

  • Make sure your terminal prompt shows (.venv) at the beginning. If it does not, re-run the activation command above.
  • If you activated but still see the error, run pip list and check that llama-index-core appears in the output. If not, re-run the pip install command.
  • On some systems, python might point to a different version. Try python3 instead.

Help me troubleshoot this ModuleNotFoundError

Get your Gemini API key from Google AI Studio

Your RAG pipeline needs a Gemini API key to access the LLM and embedding models. The free tier provides more than enough quota for this project.

  • Open your browser and go to https://aistudio.google.com.
  • In the left sidebar, click API Keys.
  • Click Create API key.
  • Copy the key that appears. You will only see the full key once.

Is this free?

New API keys start on the Free Tier by default. You will not be charged unless you explicitly set up billing and link a Cloud Billing account.

  • Back in VS Code, create a new file by clicking the new file icon in the Explorer sidebar.
  • Name the file .env and press Enter.
  • Paste the following into the file, replacing your-api-key-here with the key you just copied:
GEMINI_API_KEY=your-api-key-here
  • Save the file with Cmd+S (macOS) or Ctrl+S (Windows).
  • Install the python-dotenv package so your Python scripts can load this key automatically. Run:
pip install python-dotenv
  • Confirm your .env file is saved by running:
cat .env

You should see your GEMINI_API_KEY= line with the key value you pasted.

See a blank output or 'No such file'?

  • Make sure you named the file exactly .env (with the leading dot). Files starting with a dot are hidden by default, but they should still appear in the VS Code Explorer.
  • Check that the file is in the root of rag-agent, not inside a subfolder.

Help me find my .env file

Create sample documents for your RAG system

Your RAG pipeline needs documents to search over. You will create three markdown files, each covering a distinct subtopic so that later you can ask questions that require combining information from multiple files.

  • Create a docs folder by running:
mkdir docs
  • Inside the docs folder, create three markdown files. You can use any topic you like, but the files must cover distinct subtopics that relate to each other. Here is a recommended setup for a fictional startup called "NovaTech":
  • docs/architecture.md describing the system architecture (microservices, databases used, how components communicate). Aim for 200-500 words.
  • docs/deployment.md covering the deployment process (CI/CD pipeline, hosting provider, scaling strategy, rollback procedures). Aim for 200-500 words.
  • docs/team-policies.md explaining team policies (code review requirements, on-call rotation, incident response). Aim for 200-500 words.

Use AI to generate your docs

You can paste a prompt like "Write a 300-word technical document about a fictional startup's microservice architecture" into any AI chat tool to generate each file quickly. The content does not need to be real. It just needs to be detailed enough to answer questions from.

  • Create each file by right-clicking the docs folder in the Explorer sidebar and selecting New File.
  • Name the file (e.g. architecture.md), paste your content, and save with Cmd+S (macOS) or Ctrl+S (Windows).
  • Repeat for your second and third files (deployment.md and team-policies.md).
  • Confirm all three files exist by running:
ls docs/

You should see your three markdown files listed (for example: architecture.md, deployment.md, team-policies.md).

Only see one or two files?

  • Check that you created the files inside the docs/ folder, not in the root rag-agent directory. Files in the wrong location will not show up when you run ls docs/.
  • Make sure each file has the .md extension.

Help me find my missing docs files

Why three separate files?

Later in this project, you will ask questions that require pulling information from multiple documents at once. If everything lived in one file, basic RAG would always find the answer in a single retrieval. Separate files force the system to decide which documents are relevant and whether it needs more than one.

Your environment is ready. Next up, you will build the actual RAG pipeline that turns your documents into searchable vectors and answers questions from them.

Build Your Basic RAG Pipeline

Your environment is set up and your documents are ready. Now it's time to build the core pipeline that makes Retrieval-Augmented Generation work: loading your documents, converting them into vector embeddings, and querying them with an LLM.

A basic RAG pipeline is the foundation you need to see working before you can feel its limitations. In this step, you will wire up LlamaIndex with Gemini to answer questions grounded in your own documents.

In this step, get ready to:
  • Configure Google GenAI as your LLM and embedding model.
  • Load documents and build a vector index from them.
  • Run a query and see the answer alongside its source documents.
Create rag_basic.py and configure the LLM

Before you can query your documents, you need to tell LlamaIndex which LLM to use for generating answers and which embedding model to use for converting text into vectors.

  • Create a new file called rag_basic.py in your rag-agent folder.
  • Add the following code to set up your imports, configure the LLM and embedding model, and build a vector index from your documents:
import os
from dotenv import load_dotenv
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.llms.google_genai import GoogleGenAI
from llama_index.embeddings.google_genai import GoogleGenAIEmbedding

load_dotenv()

# Configure LLM and embedding model
Settings.llm = GoogleGenAI(model="gemini-3.6-flash")
Settings.embed_model = GoogleGenAIEmbedding(model_name="models/text-embedding-004")

# Load documents and create index
documents = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(documents, show_progress=True)

What does this code do?

This code sets up the full RAG foundation in four steps:

  • load_dotenv() reads your .env file so the Gemini API key is available as an environment variable.
  • Settings.llm tells LlamaIndex to use gemini-3.6-flash for generating answers.
  • Settings.embed_model tells LlamaIndex to use Google's text-embedding-004 model to convert your document chunks into vector embeddings.
  • SimpleDirectoryReader loads every file from your docs/ folder, and VectorStoreIndex.from_documents splits them into chunks, embeds each chunk, and stores them in a searchable index.
  • Save rag_basic.py.
  • Run the script to verify your documents load and index correctly:
python rag_basic.py

You should see progress bars for parsing nodes and generating embeddings, then the script exits cleanly with no errors.

Seeing an API key error?

  • Make sure your .env file contains GEMINI_API_KEY= followed by your actual key with no spaces around the = sign.
  • Confirm your virtual environment is activated. You should see (.venv) at the start of your terminal prompt.
  • Check that your docs/ folder has at least one file in it. An empty folder causes a loading error.

Why can't LlamaIndex find my API key?

Add a query helper with source transparency

Seeing which document chunks the LLM received is critical for understanding why RAG answers the way it does. You will add a helper function that prints the answer alongside the exact sources it drew from.

  • Below the index line in rag_basic.py, add the query engine and the ask helper function:
# Create query engine
query_engine = index.as_query_engine(similarity_top_k=3)


def ask(question: str) -> None:
    """Ask a question and display the answer with sources."""
    print(f"\n{'='*60}")
    print(f"Question: {question}")
    print(f"{'='*60}")

    response = query_engine.query(question)

    print(f"\nAnswer:\n{response.response}")
    print(f"\n--- Retrieved Sources ---")
    for i, node in enumerate(response.source_nodes, 1):
        print(f"\n[{i}] Score: {node.score:.3f}")
        print(f"    File: {node.metadata.get('file_name', 'unknown')}")
        print(f"    Preview: {node.text[:150]}...")

What does this code do?

  • as_query_engine(similarity_top_k=3) creates a query engine that retrieves the 3 most similar document chunks for every question you ask.
  • The ask function sends a question to the query engine, prints the LLM's answer, then loops through the source nodes to show which file each chunk came from and its similarity score.
  • This transparency will be critical later when you compare basic RAG to the agentic version and need to see exactly what context the LLM had access to.
Run your first query

Time to put the pipeline to work. You will add a main block that asks both a simple question and a complex multi-step question, so you can see how basic RAG handles each.

  • At the bottom of rag_basic.py, add the main block that runs two test questions:
if __name__ == "__main__":
    # Simple single-hop question
    ask("What is the main purpose of the system?")

    print("\n\n" + "="*60)
    print("Now trying a complex multi-step question...")
    print("="*60)

    # Complex multi-step question (will show limitations)
    ask(
        "Based on the architecture and the deployment process, "
        "what would need to change if we doubled the user count?"
    )

What does this code do?

The first call to ask() tests a straightforward question whose answer lives in a single document. The second call tests a complex question that requires information from multiple documents. You will examine both outputs to see how well single-pass retrieval handles each type.

  • Save rag_basic.py.

Before you run this, what do you expect to see? Will both questions get equally good answers, or will one be weaker?

  • Run the script:
python rag_basic.py

You should see the simple question get a solid, grounded answer with relevant source chunks. The complex question also produces an answer, but keep it in mind for the next step.

Getting a timeout or rate limit error?

  • The Gemini free tier has per-minute rate limits. Wait 60 seconds and try again.
  • If you see a network timeout, check your internet connection and confirm you can reach https://aistudio.google.com in your browser.

Why is my RAG query timing out or returning an error?

✔️ Awesome, I've got everything!

Great. Double check you have saved rag_basic.py and confirm your output shows a question, an answer, and at least one retrieved source with a file name and score.

ⓧ I'd like to double check the full code

Your rag_basic.py should contain three sections, in this order:

  • Imports, load_dotenv(), Settings configuration, document loading, and index creation (the first code block you added).
  • The query_engine line and the ask() function definition (the second code block).
  • The if __name__ == "__main__": block calling ask() twice (the third code block).

Scroll up and confirm each section is present in order with no duplicates.

Your basic RAG pipeline is alive. It loads your documents, embeds them, retrieves relevant chunks, and generates grounded answers. Next up, you will push it to its limits with a complex question and see exactly where single-pass retrieval falls short.

Hit the Wall With a Complex Query

Your basic RAG pipeline handles simple questions well. It retrieves the right chunk and generates a grounded answer.

But real-world research questions are rarely simple. They require pulling facts from multiple documents and reasoning across them. A single retrieval pass cannot handle this. Let's experience this limitation firsthand.

In this step, get ready to:
  • Run a complex multi-step query through your basic RAG pipeline.
  • Examine which document chunks were retrieved and which were missed.
  • Observe how single-pass retrieval produces an incomplete answer.
Understand what makes a query "complex" for RAG

Your rag_basic.py already contains a complex query at the bottom of the file. Look at the question it asks:

# Complex multi-step question (will show limitations)
ask(
    "Based on the architecture and the deployment process, "
    "what would need to change if we doubled the user count?"
)

Why is this query hard for basic RAG?

This question requires information from at least two different documents: one covering architecture and one covering deployment. A human would read both documents, pull the relevant facts from each, and combine them into a single answer.

Basic RAG converts your question into a single embedding vector and retrieves the top 3 most similar chunks. Those chunks will cluster around whichever document is closest to the overall question embedding. Chunks from the other document get left behind.

Run the pipeline and observe the failure

Before you run the script, think about what you expect to happen. The query asks about both architecture and deployment. Do you think the retriever will grab chunks from both documents, or just one?

  • Run your basic RAG pipeline by executing this command in your terminal:
python rag_basic.py

The script runs both queries in sequence. The simple question at the top should produce a solid answer. Scroll down to the complex question's output.

  • Look at the --- Retrieved Sources --- section under the complex question.
  • Check the File: field on each retrieved chunk. Note whether the chunks come from one document or multiple documents.

You will see that most or all of the retrieved chunks come from a single document. The retriever grabbed the chunks closest to the overall query embedding, which clustered around one topic. Information from the other document is missing entirely.

What should I see?

The File: values on your 3 retrieved chunks will likely show the same filename repeated, or heavily skewed toward one document. The answer itself will be partial, hedged, or only address one side of the question.

Analyze the limitation

Now look at the generated answer for the complex query. Compare it to what you know is in your documents.

  • Read the answer text. Notice if it only addresses one aspect of the question (e.g., only architecture or only deployment, not both).
  • Check if the answer uses hedging language like "may" or "could" where your documents have definitive information.
  • Compare the answer to what a human who read all three documents would say. The gap between these two answers is what agentic RAG will close.

Why does this happen?

Basic RAG performs exactly one retrieval pass. It embeds the entire question as a single vector and finds the top-k nearest chunks. It cannot decide to search again, reformulate the query, or break it into sub-questions.

This is fine for single-hop questions where the answer lives in one place. It falls apart when the answer requires combining facts from multiple sources. That is the exact gap an agentic system closes.

Getting a good answer anyway?

If your documents are short enough, all the relevant content might fit into the top 3 chunks from one file. Try asking a question that explicitly requires a fact only present in a different document. The key is that at least two documents need to contribute information the other does not have.

help me understand why my basic RAG is giving complete answers to complex queries

You have now felt the limitation that drives the entire field of agentic RAG. A system that cannot decide to search again, break a question apart, or try a different angle will always produce shallow answers on complex questions. In the next step, you will build an agent that does all of these things automatically.

Upgrade to Agentic RAG With a ReAct Agent

You just saw basic RAG fail on a complex question. The answer was incomplete because the system did one retrieval pass and stopped, even though the needed information was spread across multiple documents.

The core problem is that basic RAG cannot decide to retrieve again, reformulate its query, or break a complex question into parts. A ReAct agent solves this by wrapping your RAG pipeline as a tool and adding a reasoning loop: think about what information is needed, act by calling the retrieval tool, observe the results, and repeat until the answer is complete.

In this step, get ready to:
  • Create a new file that wraps your query engine as an agent tool.
  • Run the same failing query and watch the agent reason through it iteratively.
  • Add a second tool that gives the agent a new retrieval strategy.
Wrap your RAG pipeline as an agent tool

Instead of calling the query engine directly, you will expose it to an agent as a named tool with a description. The agent reads the description to decide when and how to use the tool.

  • Create a new file called rag_agent.py in your rag-agent folder by adding the following imports and configuration:
import os
import asyncio
from dotenv import load_dotenv
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.core.agent.workflow import ReActAgent
from llama_index.core.tools import QueryEngineTool, FunctionTool, ToolMetadata
from llama_index.core.workflow import Context
from llama_index.llms.google_genai import GoogleGenAI
from llama_index.embeddings.google_genai import GoogleGenAIEmbedding

load_dotenv()

# Configure LLM and embedding model
Settings.llm = GoogleGenAI(model="gemini-3.6-flash")
Settings.embed_model = GoogleGenAIEmbedding(model_name="models/text-embedding-004")

# Load documents and create index
documents = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(documents, show_progress=True)

What does this code do?

Most of this is familiar from rag_basic.py. The key new imports are:

  • ReActAgent from llama_index.core.agent.workflow is the reasoning engine that thinks before it acts.
  • QueryEngineTool and FunctionTool let you wrap existing code as tools the agent can call.
  • Context manages the agent's session state across async calls.
  • asyncio is needed because the ReAct agent runs asynchronously.
  • Below the index creation, add the QueryEngineTool that wraps your existing query engine:
# Create query engine tool
query_engine = index.as_query_engine(similarity_top_k=3)
search_tool = QueryEngineTool(
    query_engine=query_engine,
    metadata=ToolMetadata(
        name="document_search",
        description=(
            "Searches the project documentation for relevant information. "
            "Use a detailed plain text question as input. "
            "Best for finding specific facts, policies, or technical details."
        ),
    ),
)

What does this code do?

QueryEngineTool wraps your existing query engine so the agent can call it like a function. The metadata is critical. The agent reads the description to decide whether this tool is the right one for a given sub-question.

A vague description leads to poor tool selection. A specific description helps the agent reason about when to use it.

  • Below the search tool, add the ReAct agent and an async function to run it:
# Create ReAct agent with tools
agent = ReActAgent(
    tools=[search_tool],
    llm=Settings.llm,
    verbose=True,
    system_prompt=(
        "You are a research assistant that answers questions by searching "
        "project documentation. Break complex questions into sub-questions "
        "and search for each part separately. Always ground your answers "
        "in retrieved information."
    ),
)


async def ask_agent(question: str) -> None:
    """Ask the agent a question and display the reasoning trace."""
    print(f"\n{'='*60}")
    print(f"Question: {question}")
    print(f"{'='*60}\n")

    ctx = Context(agent)
    response = await agent.run(user_msg=question, ctx=ctx)
    print(f"\nFinal Answer:\n{response}")


if __name__ == "__main__":
    asyncio.run(
        ask_agent(
            "Based on the architecture and the deployment process, "
            "what would need to change if we doubled the user count?"
        )
    )

What does this code do?

  • verbose=True prints the agent's internal Thought/Action/Observation loop so you can watch it reason.
  • The system_prompt instructs the agent to decompose complex questions and search for each part separately.
  • Context(agent) creates a session context, and agent.run() kicks off the reasoning loop asynchronously.
  • The if __name__ block runs the exact same complex query that failed in Step 3.
  • Save rag_agent.py.
Run the agent on your complex query

This is the same multi-step question that produced an incomplete answer in Step 3. Before you run it, think about what you expect to happen differently this time.

  • Before running, predict: will the agent call the search tool once or multiple times? Why?
  • Run your agent by entering this command in the terminal:
python rag_agent.py

You will see the agent's reasoning trace printed in real time. It starts with a Thought (what information do I need?), then an Action (calling document_search with a sub-question), then an Observation (the retrieved results). This loop repeats until the agent decides it has enough information to answer.

What should I see?

The verbose output shows multiple Thought/Action/Observation cycles. The agent breaks your complex question into sub-questions (one about architecture, one about deployment), searches for each separately, and then synthesizes a complete answer from the combined results.

Compare this to Step 3's output. The final answer should be noticeably more complete because the agent retrieved from multiple angles instead of doing a single similarity search.

Seeing an error about async or event loop?

  • Make sure asyncio.run() wraps your ask_agent call in the if __name__ block. The ReAct agent requires async execution.
  • If you see a timeout or rate limit error, wait 60 seconds and try again. The free tier has per-minute quotas.

help me debug an async error when running my ReAct agent

Add a document summarizer tool

Right now the agent has one strategy: similarity search. But sometimes the agent needs a broad overview of an entire document rather than a targeted fact. Adding a second tool gives the agent a choice, and you will see it pick the right tool for each sub-question.

  • In rag_agent.py, find the line that says # Create ReAct agent with tools. Add this code directly above that line:
# Create a summarize-by-filename tool
def summarize_document(filename: str) -> str:
    """Summarize an entire document by its filename. Use when you need a broad overview of a specific document rather than targeted search."""
    for doc in documents:
        if doc.metadata.get("file_name", "") == filename:
            content = doc.text[:3000]
            response = Settings.llm.complete(
                f"Summarize the following document concisely:\n\n{content}"
            )
            return str(response)
    return f"Document '{filename}' not found. Available documents: {[d.metadata.get('file_name', '') for d in documents]}"


summarize_tool = FunctionTool.from_defaults(
    fn=summarize_document,
    name="summarize_document",
    description=(
        "Summarizes an entire document by filename. "
        "Use when you need a broad overview of what a document covers, "
        "rather than searching for a specific fact."
    ),
)

What does this code do?

  • summarize_document is a plain Python function that finds a document by filename, takes the first 3000 characters, and asks the LLM to summarize it.
  • FunctionTool.from_defaults() wraps any Python function as a tool the agent can call. The description tells the agent this tool is for broad overviews, not specific facts.
  • If the file is not found, it returns the list of available filenames so the agent can self-correct.
  • Now update the agent's tool list. Find this line in your agent creation:
    tools=[search_tool],
  • Replace it with this line so the agent has access to both tools:
    tools=[search_tool, summarize_tool],
  • Save rag_agent.py and run it again:
python rag_agent.py

Watch the reasoning trace. The agent now has two tools to choose from. Depending on the question, it may call document_search for specific facts and summarize_document for a broader view of a file.

Agent not using the summarize tool?

  • The agent chooses tools based on what the question requires. For some queries, similarity search alone is enough. Try asking a question like "Give me an overview of [one of your filenames]" to see the summarize tool in action.
  • Check that summarize_tool appears in the tools= list. If it is missing, the agent cannot use it.

help me understand why my agent is not selecting the summarize tool

✔️ Awesome, I've got everything!

Great. Double check you have saved rag_agent.py and that your last run showed the full Thought/Action/Observation loop with a complete final answer.

ⓧ I'd like to double check the full code

Your rag_agent.py should contain these sections in order. Scroll up through your file and confirm each one is present:

  • Imports (os, asyncio, dotenv, LlamaIndex core, agent.workflow, tools, Context, GoogleGenAI, GoogleGenAIEmbedding).
  • load_dotenv() and Settings configuration for LLM and embedding model.
  • Document loading with SimpleDirectoryReader and VectorStoreIndex creation.
  • query_engine and search_tool (QueryEngineTool with name "document_search").
  • summarize_document function and summarize_tool (FunctionTool with name "summarize_document").
  • ReActAgent creation with tools=[search_tool, summarize_tool] and verbose=True.
  • async def ask_agent function using Context(agent) and await agent.run().
  • if __name__ == "__main__" block running the complex multi-step query via asyncio.run().

Your agent is reasoning iteratively and producing more complete answers than basic RAG ever could. Next up, you will put both systems side by side to see exactly when the agentic approach earns its extra cost and latency.

Compare Both Systems Side-by-Side

Your agentic system now reasons through complex queries that stumped basic RAG.

Knowing that agentic RAG is more powerful is one thing. Knowing when to use it is another. In this step, you will build a script that runs the same questions through both systems and prints their answers side-by-side with timing data. This makes the quality and latency trade-off impossible to miss.

In this step, get ready to:
  • Create a comparison script that initializes both systems in one file.
  • Run simple and complex questions through both and observe the quality and speed differences.
Set up compare.py with both retrieval systems

This script reuses the same patterns from your two previous files, but puts both the basic query engine and the agentic system into a single file so you can run the same questions through each.

  • Create a new file called compare.py in your rag-agent folder.
  • Add the imports, configuration, and basic RAG setup by pasting this code:
import os
import asyncio
import time
from dotenv import load_dotenv
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.core.agent.workflow import ReActAgent
from llama_index.core.tools import QueryEngineTool, FunctionTool, ToolMetadata
from llama_index.core.workflow import Context
from llama_index.llms.google_genai import GoogleGenAI
from llama_index.embeddings.google_genai import GoogleGenAIEmbedding

load_dotenv()

# Configure LLM and embedding model
Settings.llm = GoogleGenAI(model="gemini-3.6-flash")
Settings.embed_model = GoogleGenAIEmbedding(model_name="models/text-embedding-004")

# Load documents and create index
documents = SimpleDirectoryReader("./docs").load_data()
index = VectorStoreIndex.from_documents(documents, show_progress=True)

# --- Basic RAG setup ---
query_engine = index.as_query_engine(similarity_top_k=3)

What does this code do?

This is the same foundation you used in rag_basic.py and rag_agent.py. The new import is time, which you will use to measure how long each system takes to answer.

The basic RAG system is just the query_engine with similarity_top_k=3. It does a single retrieval pass and generates an answer from whatever chunks it finds.

  • Below the basic RAG setup, add the agentic RAG tools by pasting this code:
# --- Agentic RAG setup ---
search_tool = QueryEngineTool(
    query_engine=query_engine,
    metadata=ToolMetadata(
        name="document_search",
        description=(
            "Searches the project documentation for relevant information. "
            "Use a detailed plain text question as input."
        ),
    ),
)

def summarize_document(filename: str) -> str:
    """Summarize an entire document by its filename."""
    for doc in documents:
        if doc.metadata.get("file_name", "") == filename:
            content = doc.text[:3000]
            response = Settings.llm.complete(
                f"Summarize the following document concisely:\n\n{content}"
            )
            return str(response)
    return f"Document '{filename}' not found."

summarize_tool = FunctionTool.from_defaults(
    fn=summarize_document,
    name="summarize_document",
    description="Summarizes an entire document by filename for a broad overview.",
)

What's different from rag_agent.py?

The tools are the same ones you built in Step 4, but with slightly shorter descriptions. The agent does not need the extra detail here because you are focused on the comparison output, not the verbose reasoning trace.

  • Save compare.py.
  • Confirm there are no syntax errors by running:
python -c "import compare"

You should see the index-building progress bar output and no error messages. If you see an ImportError, check that your virtual environment is still activated.

  • Now add the agent and test questions below the tools by pasting this code:
agent = ReActAgent(
    tools=[search_tool, summarize_tool],
    llm=Settings.llm,
    verbose=False,
    system_prompt=(
        "You are a research assistant. Break complex questions into sub-questions "
        "and search for each part separately. Ground answers in retrieved information."
    ),
)

# --- Test questions ---
test_questions = [
    "What is the main purpose of the system?",
    (
        "Based on the architecture and the deployment process, "
        "what would need to change if we doubled the user count?"
    ),
]

Why is verbose set to False?

In rag_agent.py you used verbose=True to watch the reasoning trace. Here you want clean side-by-side output without the agent's internal thoughts cluttering the comparison.

The test_questions list includes a simple single-hop question and the same complex multi-step question that failed in Step 3. This lets you see both sides of the trade-off.

Add the comparison loop and run it

The comparison function loops through each question, times both systems, and prints the results formatted for easy reading.

  • Below the test_questions list, add the comparison function and main block:
async def run_comparison():
    for question in test_questions:
        print(f"\n{'='*70}")
        print(f"QUESTION: {question}")
        print(f"{'='*70}")

        # Basic RAG
        start = time.time()
        basic_response = query_engine.query(question)
        basic_time = time.time() - start

        print(f"\n[BASIC RAG] ({basic_time:.1f}s)")
        print(f"{basic_response.response[:500]}")

        # Agentic RAG
        start = time.time()
        ctx = Context(agent)
        agent_response = await agent.run(user_msg=question, ctx=ctx)
        agent_time = time.time() - start

        print(f"\n[AGENTIC RAG] ({agent_time:.1f}s)")
        print(f"{str(agent_response)[:500]}")

        print(f"\n--- Timing: Basic={basic_time:.1f}s | Agentic={agent_time:.1f}s ---")


if __name__ == "__main__":
    asyncio.run(run_comparison())

What does this code do?

  • For each question, it records the start time, runs the query, then calculates elapsed time.
  • It truncates each response to 500 characters so the output stays readable.
  • The timing line at the bottom of each question shows you the latency cost of agentic reasoning.
  • Save compare.py.

Before you run this, think about what you expect to see. For the simple question, which system do you think will be faster? For the complex question, which system do you think will give a more complete answer?

  • Run the comparison by executing:
python compare.py

You will see both answers printed for each question with timing data. For the simple question, basic RAG answers quickly and accurately. For the complex multi-step question, the agentic system produces a noticeably more complete answer because it broke the question into parts and retrieved from multiple documents.

What should I see?

The simple question produces similar quality from both systems, but basic RAG is faster (typically 1-3 seconds vs 5-15 seconds for the agent). The agent's extra reasoning adds latency without improving the answer when the question is straightforward.

The complex question is where the difference is dramatic. Basic RAG gives a partial or hedged answer because it only retrieved chunks similar to the overall query embedding. The agent decomposed the question, searched multiple times, and synthesized information from across your documents.

Seeing an error or unexpected output?

  • If you get a NameError, make sure all four code chunks are in the correct order in compare.py. The agent must be defined after the tools.
  • If the agentic answer looks identical to the basic one, confirm verbose=False is set on the agent. With verbose=True the reasoning trace floods the output.
  • If you hit a rate limit, wait 60 seconds and try again. The free tier resets quickly.

Help me debug my compare.py output

✔️ Awesome, I've got everything!

Great. Double check you have saved compare.py and confirmed the side-by-side output.

ⓧ I'd like to double check the full code

Scroll up and confirm your compare.py contains these sections in order:

  • Imports (including time and asyncio), configuration, document loading, and basic RAG query engine.
  • Agentic RAG setup: search_tool, summarize_document function, and summarize_tool.
  • Agent instantiation with verbose=False and the test_questions list (one simple, one complex).
  • The run_comparison() async function and the if __name__ == "__main__" block.

You now have concrete evidence of when each architecture earns its keep. Next up, you will take on the Secret Mission and give your agent access to live web information.

Secret mission

Add a Web Search Tool

Your agent can answer questions from your documents, but what about questions that need current information from the internet? In this extension, you will add a web search tool so the agent can dynamically choose between searching your private docs and searching the live web.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. The Gemini free tier resets its daily quota automatically and does not charge you unless you explicitly set up billing.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • The rag-agent project folder on your Desktop (Python files, docs, virtual environment).
  • A Gemini API key stored in your .env file (free tier, no charges).

Keep everything running

No action needed. Choose this if you are still actively building or want to experiment further with your RAG agent.

  • Your code and virtual environment stay on your machine, ready to use anytime.
  • Your Gemini API key remains active on the free tier with no charges.
  • You can re-activate the virtual environment anytime by running .venv\Scripts\activate (Windows) or source .venv/bin/activate (macOS/Linux) from the rag-agent folder.

Pause - I'll come back to this later

Shut down running processes to free up terminal resources. Your files and API key stay intact for later.

  • Deactivate your virtual environment by running:
deactivate
  • Close the terminal and VS Code. Your project folder and API key remain on disk, ready to pick up where you left off.

Delete - I don't want to use this again

Remove all project resources and revoke your API key to start completely fresh.

  • Deactivate your virtual environment first by running:
deactivate
  • Delete the project folder from your Desktop by running:
cd ~/Desktop
rm -rf rag-agent

On Windows, use this instead:

cd ~\Desktop
rmdir /s /q rag-agent
  • To revoke your Gemini API key, open the Google Cloud API Credentials page.
  • Find the API key you used for this project and click the Actions icon.
  • Select Delete API key, then confirm by clicking Delete in the modal.

Nice Work!

Nice Work!

Nice work! You just built a document research assistant that progresses from a standard RAG pipeline all the way to a fully agentic system powered by a ReAct reasoning loop.

You've learned how to:

  • Built a basic RAG pipeline that embeds documents with Gemini, retrieves relevant chunks via vector similarity search, and generates grounded answers with source attribution.
  • Upgraded to an agentic RAG system using a ReAct agent that decomposes complex queries into sub-questions, makes multiple retrieval calls, and synthesizes complete answers from combined results.
  • Compared both architectures side-by-side with timing data to understand when basic RAG is sufficient and when agentic RAG justifies its extra latency for dramatically better answer quality.
  • Secret Mission: Added a web search tool giving your agent access to live information from the internet alongside your private documents.

Ready to quiz yourself?