Build a Self-Improving Research Agent

Build a RAG agent that evaluates its own answers and improves with each iteration.

Introduction

30 Second Summary

When you get feedback on something you wrote, you naturally reflect on what went wrong and try again. That loop of attempt, feedback, and improvement is how humans get better at anything. AI agents can follow the exact same pattern.

In this project, you will build a Python agent that answers technical questions, evaluates its own output, reflects on what went wrong, and regenerates improved answers using its reflections as guidance. You will watch its scores climb in real time as it accumulates lessons in memory.

What You'll Build

Run your agent, ask it a question about Python async programming, and watch it retrieve context from a local knowledge base, self-critique, and produce a better answer on every retry.

By the end of this project, you'll have:

A working RAG pipeline where you ask a question and see the agent retrieve relevant chunks from a ChromaDB knowledge base, assemble a structured prompt, and generate an answer.
A self-evaluation loop where the agent scores its own answer, identifies specific flaws, and stores lessons in episodic memory.
The complete Reflexion pattern in action: generate, evaluate, reflect, and regenerate with measurably better scores printed to your terminal on each iteration.
Secret Mission: Add multi-aspect evaluation with weighted scoring across factual accuracy, completeness, and clarity dimensions.

Are there any prerequisites?

This project assumes you have experience writing Python and basic familiarity with calling LLM APIs. You do not need prior experience with RAG or ChromaDB. The project walks you through both from scratch.

Before We Start

A self-improving agent needs a foundation before it can learn anything. You need an environment with the Sarvam AI SDK for LLM calls and ChromaDB for retrieval, all verified before wiring the agent's components together.

In this step, you will create a clean project folder, install the required packages, and confirm your API key connects to the Sarvam AI service.

In this step, get ready to:

Create a project folder and Python virtual environment.
Install the Sarvam AI SDK, ChromaDB, and python-dotenv packages.
Set up your API key and verify the connection.

Create your project folder and virtual environment

Every Python project starts with an isolated environment. A virtual environment keeps your project's packages separate from your system Python, preventing version conflicts.

Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
Type Terminal and press Enter to open it.
Move to your Desktop by running this command:

cd ~/Desktop

Create your project folder and move into it by running these commands:

mkdir self-improving-agent
cd self-improving-agent

Create and activate a virtual environment by running the command for your operating system:

macOS

python3 -m venv venv
source venv/bin/activate

Windows

python -m venv venv
venv\Scripts\Activate.ps1

Your terminal prompt now shows (venv) at the beginning, confirming the virtual environment is active.

What about Linux?

Linux uses the same commands as macOS. Run python3 -m venv venv followed by source venv/bin/activate.

Don't see (venv) in your prompt?

Make sure you ran the activation command (source venv/bin/activate on macOS/Linux or venv\Scripts\Activate.ps1 on Windows) from inside the self-improving-agent folder.

On Windows PowerShell, you may see an execution policy error. Run Set-ExecutionPolicy RemoteSigned -Scope CurrentUser first, then try activation again.

Help me fix my virtual environment activation

Install the project dependencies

Your agent relies on three packages: the Sarvam AI SDK for LLM calls, ChromaDB for vector retrieval, and python-dotenv for loading environment variables from a file.

Install all three packages by running:

pip install sarvamai chromadb python-dotenv

You will see pip downloading and installing each package along with their dependencies. This takes about 30-60 seconds.

Seeing a permissions error or install failure?

Make sure (venv) appears in your prompt. If it does not, you are installing into your system Python instead of the virtual environment. Rerun the activation command from the previous substep.

If you see a network timeout, check your internet connection and try again.

Help me fix my pip install error

Set up your API key and verify the connection

The Sarvam AI API authenticates using an API subscription key. You will store it in a .env file so your code can load it without hardcoding secrets.

Create your .env file by running:

echo "SARVAM_API_KEY=your-api-key-here" > .env

Open .env in your editor and replace your-api-key-here with your actual Sarvam API key.
Save the file.

Where do I get a Sarvam API key?

Sign in to the Sarvam AI dashboard and navigate to your API keys section. Copy the subscription key value. This key authenticates every request your agent makes to the sarvam-30b model.

Confirm the file exists by running:

ls -a

You should see .env and venv listed in the output.

Before you run the final check, what do you think will happen if the packages installed correctly?

Verify your full setup by running:

python -c "from sarvamai import SarvamAI; import chromadb; print('Ready')"

You should see Ready printed to your terminal. This confirms both packages are installed and importable.

Seeing a ModuleNotFoundError?

This means the package did not install into your active environment. Check that (venv) appears in your prompt, then rerun pip install sarvamai chromadb python-dotenv.

If you see an error about python not being found, try python3 instead. Some systems require the versioned command.

Help me fix my Python environment

✔️ Awesome, I've got everything!

Great. Double check you have saved your .env file with your real API key in place.

ⓧ I'd like to double check the full code

Your .env file should look like this (with your actual key replacing the placeholder):

SARVAM_API_KEY=your-api-key-here

Your environment is locked in. Next up, you will build the knowledge base that gives your agent something to be right about.

Set Up the Project

Your self-improving agent has several moving parts: a language model, a vector database, and the orchestration code connecting them. Before you build any of that, you need a clean workspace with the right packages installed and a verified API connection.

In this step, you'll create your project folder, install the Sarvam AI SDK and ChromaDB, and confirm everything works with a test call.

In this step, get ready to:

Create your project folder and virtual environment.
Install the Sarvam AI SDK and ChromaDB.
Verify your API connection with a test call to sarvam-30b.

Create the project folder and virtual environment

A dedicated folder and virtual environment keep your agent's dependencies isolated from the rest of your system.

Move to your Desktop by running this command:

cd ~/Desktop

Create your project folder and move into it by running these commands:

mkdir self-improving-agent
cd self-improving-agent

Create a Python virtual environment by running:

python3 -m venv venv

Activate the virtual environment:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

Your terminal prompt now shows (venv) at the beginning, confirming the virtual environment is active.

Don't see (venv) in your prompt?

Make sure you ran the activate command from inside the self-improving-agent folder.

On macOS, confirm the venv folder exists by running ls. If it is missing, re-run the python3 -m venv venv command.

help me troubleshoot my virtual environment activation

Install the dependencies

Your agent needs three packages: the Sarvam AI SDK for LLM calls, ChromaDB for vector storage, and python-dotenv for loading environment variables.

Install all three packages by running:

pip install sarvamai chromadb python-dotenv

What do these packages do?

sarvamai is the official Python SDK for Sarvam AI's language models.
chromadb is an in-memory vector database that handles embedding and retrieval with zero infrastructure.
python-dotenv loads environment variables from a .env file so you never hard-code secrets.

Before you assume the install succeeded, what do you expect to see if you try importing both main libraries?

Verify both packages are importable by running:

python -c "from sarvamai import SarvamAI; import chromadb; print('Ready')"

You should see Ready printed to your terminal.

Seeing a ModuleNotFoundError?

Make sure your virtual environment is active. You should see (venv) in your prompt. If not, re-run the activate command from the previous substep.

If the environment is active but the import still fails, re-run pip install sarvamai chromadb python-dotenv and check for error messages during install.

help me fix my package import error

Create your .env file and test the API connection

The Sarvam AI SDK authenticates using an API key passed via the api_subscription_key parameter. You'll store this key in a .env file so it stays out of your code.

Create a .env file in your self-improving-agent folder by running:

echo "SARVAM_API_KEY=your-api-key-here" > .env

Open .env in your editor and replace your-api-key-here with your actual Sarvam AI API key.
Save the file.

Where do I find my Sarvam API key?

You can generate an API key from the Sarvam AI dashboard. Keys start with sk_ followed by a unique string. The key is passed to the SDK as the api_subscription_key parameter, not as a Bearer token.

Now for the real test. Do you think the API call will succeed, or will something go wrong?

Test your API connection by running this command:

python << 'EOF'
import os
from dotenv import load_dotenv
from sarvamai import SarvamAI

# Load API key from .env file
load_dotenv()

# Initialize the client
client = SarvamAI(api_subscription_key=os.environ.get("SARVAM_API_KEY"))

# Make a test call to sarvam-30b
response = client.chat.completions(
    model="sarvam-30b",
    messages=[{"role": "user", "content": "Say hello in one sentence."}],
    max_tokens=50,
    reasoning_effort=None,
)
print(response.choices[0].message.content)
EOF

You should see a short greeting printed to your terminal. The exact wording varies, but any coherent response confirms your API key and SDK are working.

What does this test do?

load_dotenv() reads your .env file and loads the API key into the environment.
SarvamAI(api_subscription_key=...) creates an authenticated client.
client.chat.completions() sends a request to sarvam-30b with reasoning_effort=None for the fastest response.

Getting an authentication error?

Open your .env file and check that there are no extra spaces or quotes around your key. The format should be exactly SARVAM_API_KEY=sk_your-actual-key with no surrounding quotation marks.

If you copied the key from the dashboard, try regenerating it. Some keys are single-use on reveal.

help me fix my Sarvam AI authentication error

✔️ Awesome, I've got everything!

Great. Your environment is ready and your API connection is live.

ⓧ I'd like to double check the full code

Your .env file should look like this (with your real key in place of the placeholder):

SARVAM_API_KEY=your-api-key-here

Your environment is locked in and your API connection is live. Next up, you'll build the RAG knowledge base that gives your agent something to be right about.

Build the RAG Knowledge Base

Your environment is set up and your Sarvam AI connection is verified. Now your agent needs something to be smart about.

Without a source of truth, an LLM can only hallucinate. In this step, you will build a local knowledge base and a retrieval pipeline so the agent can ground its answers in real content.

In this step, get ready to:

Create a set of technical documents as the agent's knowledge source.
Build an ingestion pipeline that stores documents in a ChromaDB collection.
Build a retrieval function that returns the most relevant chunks for a query.

Create the knowledge base documents

The agent needs a collection of documents that contain real information it can retrieve from. You will create a set of short technical documents about Python async programming. This topic has enough nuance that the agent can be partially wrong, which makes the self-improvement loop meaningful.

Create a new file called knowledge_base.py in your self-improving-agent folder.
Add the following import and document definitions:

import chromadb


DOCUMENTS = [
    {
        "id": "async-basics",
        "content": "Python's asyncio module provides infrastructure for writing single-threaded concurrent code using coroutines. A coroutine is declared with async def and is meant to be run with asyncio.run() or awaited inside another coroutine. The event loop is the core of asyncio - it runs async tasks and callbacks, performs network IO operations, and runs subprocesses."
    },
    {
        "id": "await-keyword",
        "content": "The await keyword is used to pause the execution of a coroutine until the awaited coroutine completes. You can only use await inside an async function. When a coroutine awaits, it yields control back to the event loop, which can then run other tasks. This is how asyncio achieves concurrency without threads."
    },
    {
        "id": "tasks-gather",
        "content": "asyncio.create_task() schedules a coroutine to run concurrently as a Task. asyncio.gather() runs multiple coroutines concurrently and waits for all of them to complete. The key difference: create_task returns immediately and the task runs in the background, while gather waits for all tasks to finish and returns their results in order."
    },
    {
        "id": "async-vs-threading",
        "content": "asyncio is best for IO-bound tasks like network requests, file operations, and database queries. Threading is better for CPU-bound tasks or when you need true parallelism. asyncio uses cooperative multitasking (coroutines voluntarily yield control), while threading uses preemptive multitasking (the OS switches between threads). asyncio has lower overhead since it runs in a single thread."
    },
    {
        "id": "error-handling",
        "content": "Error handling in asyncio uses standard try/except blocks inside coroutines. For tasks created with create_task(), exceptions are stored and re-raised when you await the task or call task.result(). asyncio.gather() has a return_exceptions parameter - when True, exceptions are returned as results instead of being raised. Unhandled exceptions in tasks trigger asyncio's exception handler."
    },
    {
        "id": "async-context-managers",
        "content": "Async context managers use async with syntax and implement __aenter__ and __aexit__ methods. They are essential for managing async resources like database connections, HTTP sessions, and file handles. The aiohttp library uses async context managers for both the session and individual requests: async with aiohttp.ClientSession() as session."
    }
]

What does this code do?

The DOCUMENTS list is your agent's entire knowledge base. Each document is a short paragraph covering one aspect of Python's asyncio module.

Each entry has an id for tracking which source the agent retrieves, and content with the actual text. ChromaDB will handle embedding these into vectors automatically when you add them to a collection.

Save knowledge_base.py.

Build the ingestion pipeline

Documents sitting in a Python list are not searchable. You need to load them into a vector database that can find the most relevant ones for any given question.

In knowledge_base.py, add the following function below the DOCUMENTS list:

def build_knowledge_base():
    """Create and populate a ChromaDB collection with technical documents."""
    client = chromadb.Client()
    collection = client.get_or_create_collection(name="knowledge_base")

    collection.upsert(
        ids=[doc["id"] for doc in DOCUMENTS],
        documents=[doc["content"] for doc in DOCUMENTS]
    )

    return collection

What does this code do?

This function creates an in-memory ChromaDB client and a collection called knowledge_base. No database server needed.

The upsert call adds all 6 documents at once. ChromaDB automatically converts the text into vector embeddings so it can perform similarity search later. The function returns the collection object so other code can query it.

Save knowledge_base.py.
Verify the ingestion works by running this command:

python -c "from knowledge_base import build_knowledge_base; collection = build_knowledge_base(); print(f'Collection created with {collection.count()} documents')"

You should see Collection created with 6 documents printed to your terminal.

Seeing a ModuleNotFoundError?

Make sure your virtual environment is active. You should see (venv) at the start of your terminal prompt. If not, run source venv/bin/activate to reactivate it.

Also confirm you are in the self-improving-agent folder where knowledge_base.py lives.

Help me debug my ChromaDB import error

Build the retrieval function

Now that documents are stored as vectors, you need a function that takes a question and returns the most relevant chunks. This is the RAG retrieval step. It gives the agent factual grounding instead of relying on its own memory.

In knowledge_base.py, add the retrieval function below build_knowledge_base():

def retrieve_context(collection, query, n_results=3):
    """Retrieve the most relevant chunks for a given query."""
    results = collection.query(
        query_texts=[query],
        n_results=n_results
    )

    chunks = []
    for i, doc in enumerate(results["documents"][0]):
        chunks.append({
            "content": doc,
            "id": results["ids"][0][i],
            "distance": results["distances"][0][i]
        })

    return chunks

What does this code do?

The collection.query() call performs a similarity search. ChromaDB compares the query's embedding against all stored document embeddings and returns the closest matches.

The function packages each result into a dictionary with the document content, its source id, and a distance score. Lower distance means higher relevance. This structured format makes it easy for the context engine to assemble prompts later.

Save knowledge_base.py.

Before you run the full retrieval test, what do you think the agent will retrieve for a question about concurrency? Which document IDs do you expect to see?

Test the complete pipeline by running:

python -c "
from knowledge_base import build_knowledge_base, retrieve_context
collection = build_knowledge_base()
chunks = retrieve_context(collection, 'How does asyncio achieve concurrency?')
for chunk in chunks:
    print(f'[{chunk[\"id\"]}] distance: {chunk[\"distance\"]:.4f}')
    print(f'  {chunk[\"content\"][:80]}...')
    print()
"

You should see 3 chunks printed with their IDs and distance scores. The closest matches (lowest distance) will be documents about asyncio basics, the await keyword, and async vs threading.

Seeing zero results or a KeyError?

If you see a KeyError on distances, make sure you are using collection.query() and not collection.get(). Only query returns distance scores.

If you see zero results, confirm that build_knowledge_base() is being called before retrieve_context(). The collection must be populated first.

Help me debug my ChromaDB retrieval

✔️ Awesome, I've got everything!

Great. Make sure you have saved knowledge_base.py with all three sections: the DOCUMENTS list, build_knowledge_base(), and retrieve_context().

ⓧ I'd like to double check the full code

Scroll up and confirm your knowledge_base.py contains these sections in order:

The import chromadb line at the top.
The DOCUMENTS list with all 6 document dictionaries (async-basics, await-keyword, tasks-gather, async-vs-threading, error-handling, async-context-managers).
The build_knowledge_base() function that creates a ChromaDB client, a collection named knowledge_base, and upserts all documents.
The retrieve_context() function that queries the collection and returns chunks with content, id, and distance.

Your agent now has a source of truth it can search. Next up, you will wire a generator that retrieves this context and assembles it into a structured prompt to produce grounded answers.

Wire the Generator with Context Engineering

In the previous step, you built a knowledge base that retrieves relevant chunks for any question. Retrieval alone is not enough. A naive agent would dump all retrieved text into one unstructured blob and hope the LLM figures it out.

In this step, you will build structured context assembly where each role in the agent gets a carefully composed prompt with distinct sections for system instructions, retrieved chunks, and the user question. This is where context engineering starts.

In this step, get ready to:

Build role-specific system prompts and context assemblers for four agent roles.
Wire the generator to the Sarvam API with structured context.
Run the full pipeline end-to-end and see a formatted answer.

Build the context assembler with role-specific prompts

Each role in the Reflexion loop needs a different context window. The generator needs retrieved chunks and the question. The evaluator needs the question, the context, and the answer to judge. The reflector needs flaws to distill into lessons. The regenerator needs everything: chunks, lessons from memory, and the question.

Create a new file called context_engine.py in your self-improving-agent folder.
Add the four role-specific system prompts by pasting this code:

GENERATOR_SYSTEM_PROMPT = """You are a technical expert answering questions based on provided context.
Use ONLY the information in the context below to answer. If the context does not contain enough information, say so explicitly.
Be precise, accurate, and comprehensive in your answers."""

EVALUATOR_SYSTEM_PROMPT = """You are a strict evaluator assessing the quality of an answer.
You will receive the original question, the reference context, and the generated answer.
Score the answer from 1-10 and list specific flaws.

Respond in this exact JSON format:
{"score": <number>, "flaws": ["<flaw1>", "<flaw2>", ...]}

Scoring criteria:
- 9-10: Accurate, complete, well-structured, uses context effectively
- 7-8: Mostly accurate with minor gaps or unclear phrasing
- 5-6: Partially correct but missing key information or contains inaccuracies
- 3-4: Significant errors or mostly irrelevant
- 1-2: Completely wrong or does not address the question"""

REFLECTOR_SYSTEM_PROMPT = """You are a reflection agent analyzing what went wrong with an answer.
Given the flaws identified by the evaluator, produce a concise lesson learned.

Respond in this exact format:
LESSON: <one sentence describing what went wrong and what to do differently next time>"""

REGENERATOR_SYSTEM_PROMPT = """You are a technical expert answering questions based on provided context.
You have attempted this question before and learned from your mistakes.
Use the context AND the lessons from your previous attempts to produce an improved answer.
Pay special attention to the lessons learned - they tell you exactly what to fix."""

What do these prompts do?

Each prompt gives one role a specific job. The generator answers from context only. The evaluator scores and finds flaws. The reflector turns flaws into reusable lessons. The regenerator uses those lessons to produce a better answer.

You are defining all four now, but only the generator is wired in this step. The other three roles come alive in the next steps.

Below the prompts in context_engine.py, add the generator's context assembly function:

def assemble_generator_context(question, chunks):
    """Assemble the context window for the generator role."""
    context_text = "\n\n---\n\n".join(
        [f"[Source: {chunk['id']}]\n{chunk['content']}" for chunk in chunks]
    )

    messages = [
        {"role": "system", "content": GENERATOR_SYSTEM_PROMPT},
        {"role": "user", "content": f"## Retrieved Context\n\n{context_text}\n\n## Question\n\n{question}"}
    ]

    return messages

What does this function do?

This is the core of context engineering. Instead of dumping raw text into the LLM, assemble_generator_context builds a structured messages array with clearly delimited sections.

The system message sets the role and constraints.
The user message separates retrieved context from the question using markdown headers.
Each chunk is labeled with its source ID so the LLM can attribute information.

Verify the file loads correctly by running:

python -c "from context_engine import assemble_generator_context; print('Context engine loaded')"

You should see Context engine loaded printed to your terminal.

Seeing a ModuleNotFoundError?

Make sure you are running the command from inside the self-improving-agent folder where context_engine.py lives. Also confirm your virtual environment is still active (you should see (venv) in your terminal prompt).

help me fix my ModuleNotFoundError when importing context_engine

Now add the remaining three assembly functions below assemble_generator_context in context_engine.py:

def assemble_evaluator_context(question, chunks, answer):
    """Assemble the context window for the evaluator role."""
    context_text = "\n\n".join([chunk["content"] for chunk in chunks])

    messages = [
        {"role": "system", "content": EVALUATOR_SYSTEM_PROMPT},
        {"role": "user", "content": f"## Question\n{question}\n\n## Reference Context\n{context_text}\n\n## Generated Answer\n{answer}\n\nEvaluate this answer:"}
    ]

    return messages


def assemble_reflector_context(flaws):
    """Assemble the context window for the reflector role."""
    flaws_text = "\n".join([f"- {flaw}" for flaw in flaws])

    messages = [
        {"role": "system", "content": REFLECTOR_SYSTEM_PROMPT},
        {"role": "user", "content": f"## Identified Flaws\n\n{flaws_text}\n\nWrite a concise lesson learned:"}
    ]

    return messages

Why are these different from the generator's context?

The evaluator receives the answer alongside the question and reference context so it can judge quality. The reflector only receives the list of flaws because its job is narrow: distill those into one reusable lesson. Each role gets exactly the information it needs and nothing more.

Add the final assembly function below assemble_reflector_context:

def assemble_regenerator_context(question, chunks, episodic_memory):
    """Assemble the context window for the regenerator role - includes lessons from memory."""
    context_text = "\n\n---\n\n".join(
        [f"[Source: {chunk['id']}]\n{chunk['content']}" for chunk in chunks]
    )

    memory_text = "\n".join(
        [f"- {lesson}" for lesson in episodic_memory]
    )

    messages = [
        {"role": "system", "content": REGENERATOR_SYSTEM_PROMPT},
        {"role": "user", "content": (
            f"## Retrieved Context\n\n{context_text}\n\n"
            f"## Lessons from Previous Attempts\n\n{memory_text}\n\n"
            f"## Question\n\n{question}"
        )}
    ]

    return messages

What makes the regenerator special?

The regenerator receives both the original RAG chunks AND all accumulated lessons from episodic memory. This is the key context engineering insight: the regenerator's context window grows richer with each iteration as the agent learns from its mistakes.

Save context_engine.py and confirm all four functions load by running:

python -c "from context_engine import assemble_generator_context, assemble_evaluator_context, assemble_reflector_context, assemble_regenerator_context; print('All 4 assemblers loaded')"

You should see All 4 assemblers loaded printed.

Seeing a SyntaxError?

Check that each function definition starts with def and that parentheses and quotes are balanced. Triple-quoted strings are easy to break if you accidentally delete a quote character.

help me fix the SyntaxError in my context_engine.py file

✔️ Awesome, I've got everything!

Great. Make sure you have saved context_engine.py with all four system prompts and four assembly functions.

ⓧ I'd like to double check the full code

Scroll up and confirm each chunk is present in context_engine.py, in this order:

GENERATOR_SYSTEM_PROMPT, EVALUATOR_SYSTEM_PROMPT, REFLECTOR_SYSTEM_PROMPT, REGENERATOR_SYSTEM_PROMPT (the four prompt constants).
assemble_generator_context() function.
assemble_evaluator_context() function.
assemble_reflector_context() function.
assemble_regenerator_context() function.

Wire the assembler to the Sarvam API

The context assembler builds the messages array, but you still need a client that sends those messages to the LLM and returns the response. You will create a new file for the agent logic.

Create a new file called agent.py in your self-improving-agent folder and add the imports and client initialization:

import json
import os

from sarvamai import SarvamAI
from knowledge_base import build_knowledge_base, retrieve_context
from context_engine import (
    assemble_generator_context,
    assemble_evaluator_context,
    assemble_reflector_context,
    assemble_regenerator_context,
)


def create_client():
    """Initialize the Sarvam AI client."""
    api_key = os.environ.get("SARVAM_API_KEY")
    if not api_key:
        raise ValueError("SARVAM_API_KEY environment variable not set")
    return SarvamAI(api_subscription_key=api_key)

What does this code do?

The imports bring in all four assembly functions from context_engine.py and the knowledge base helpers.
create_client() reads your API key from the environment and initializes the Sarvam AI client using the api_subscription_key parameter (this is how Sarvam authenticates requests).

Below create_client in agent.py, add the LLM call helper and the generator function:

def call_llm(client, messages, temperature=0.7):
    """Make a chat completion call to sarvam-30b."""
    response = client.chat.completions(
        model="sarvam-30b",
        messages=messages,
        temperature=temperature,
        max_tokens=1000,
        reasoning_effort=None,
    )
    return response.choices[0].message.content


def generate(client, question, chunks):
    """Generate an initial answer using RAG context."""
    messages = assemble_generator_context(question, chunks)
    print(f"\n[Generator] Context: {len(chunks)} chunks, system prompt + user message")
    answer = call_llm(client, messages, temperature=0.7)
    return answer

What does this code do?

call_llm() is a reusable wrapper that calls sarvam-30b with any messages array. Setting reasoning_effort=None disables the model's extended thinking mode for faster responses.
generate() ties it together: it calls assemble_generator_context to build the structured messages, prints a debug log showing how many chunks were assembled, and returns the answer.

Save agent.py.

Why this approach instead of calling the API directly?

Wrapping the API call in call_llm() means each role (generator, evaluator, reflector, regenerator) can reuse the same function with different temperatures and messages. The generator uses temperature=0.7 for diverse answers. Later, the evaluator will use 0.2 for consistent scoring.

Test the full pipeline end-to-end

The generator and context engine are wired. Time to run the complete retrieve-assemble-generate pipeline and see an answer.

Install python-dotenv so the script can load your API key from the .env file by running:

pip install python-dotenv

Add the test block at the bottom of agent.py:

if __name__ == "__main__":
    from dotenv import load_dotenv
    load_dotenv()

    from knowledge_base import build_knowledge_base, retrieve_context
    client = create_client()
    collection = build_knowledge_base()
    question = "How does asyncio achieve concurrency without threads, and when should I use it versus threading?"
    chunks = retrieve_context(collection, question, n_results=3)
    answer = generate(client, question, chunks)
    print(f"\nAnswer: {answer}")

What does this test block do?

load_dotenv() reads your .env file and sets SARVAM_API_KEY as an environment variable.
It builds the knowledge base, retrieves the 3 most relevant chunks for the question, assembles structured context, and calls the LLM.
The debug log from generate() shows you exactly what the context window looks like before the API call fires.

Save agent.py.

Before you run this, what do you expect to see? Think about it: the generator has 3 chunks of asyncio documentation as context. Will it produce a good answer, or will something be missing?

Run the agent by executing:

python agent.py

You should see a debug log line showing the assembled context structure, followed by a technical answer about asyncio concurrency. The output looks something like this:

[Generator] Context: 3 chunks, system prompt + user message

Answer: asyncio achieves concurrency without threads by using cooperative multitasking...

The answer draws directly from your knowledge base chunks. It is grounded in real context, not hallucinated.

Seeing a ValueError about SARVAM_API_KEY?

Make sure your .env file is in the same directory where you are running python agent.py. Open the file and confirm it contains SARVAM_API_KEY= followed by your actual key (not the placeholder text).

help me fix the SARVAM_API_KEY not found error

Getting a connection error or 401 response?

Confirm your API key is valid by checking your Sarvam AI dashboard. Also verify you have internet connectivity. The key must be the full string with no extra spaces or newlines.

help me debug my Sarvam API connection issue

Your generator is producing grounded answers from structured context. But how do you know if the answer is actually good? Next up, you will build an evaluator that scores the answer and identifies specific flaws.

Add the Evaluator

Your generator produces an answer, but right now you have no way of knowing whether that answer is any good. Is it accurate? Does it fully address the question? You're flying blind.

The evaluator is a second LLM call that acts as a judge. It receives the same retrieved context plus the generated answer, scores it 1-10, and lists specific flaws in structured JSON. This feedback signal is what will eventually let the agent learn from its mistakes.

In this step, get ready to:

Build the evaluate function with its own context assembly and temperature settings.
Run the full pipeline and observe the generator's first imperfect score.

Build the evaluate function

The evaluator has a different job than the generator, so it gets a different context window. Your assemble_evaluator_context() function in context_engine.py already handles that assembly. Now you need the function that calls it.

In agent.py, find the generate() function. Add the following evaluate() function directly below it:

def evaluate(client, question, chunks, answer):
    # Send the answer to the evaluator for scoring
    messages = assemble_evaluator_context(question, chunks, answer)
    raw_response = call_llm(client, messages, temperature=0.2)

    # Parse the structured JSON response
    try:
        result = json.loads(raw_response)
        score = result.get("score", 5)
        flaws = result.get("flaws", [])
    except json.JSONDecodeError:
        score = 5
        flaws = ["Could not parse evaluator response"]

    return score, flaws

What does this code do?

assemble_evaluator_context() builds a context window with the original question, the retrieved chunks, and the generated answer. The evaluator's system prompt tells it to score 1-10 and return structured JSON.
temperature=0.2 keeps the evaluator deterministic. A creative judge would give wildly different scores each time, which makes the feedback signal unreliable.
The try/except block handles cases where the LLM returns malformed JSON. It falls back to a score of 5 with a descriptive flaw message instead of crashing.

Why a lower temperature for the evaluator?

Each role in the agent has its own temperature tuned to its job. The generator uses 0.7 for diverse, natural answers. The evaluator uses 0.2 because you want consistent, reliable scoring. If the evaluator were creative, it might rate the same answer 4/10 one time and 8/10 the next.

Wire the evaluator into the pipeline

The evaluate function exists, but the script doesn't call it yet. You need to update the __main__ block to run the full pipeline: retrieve, generate, then evaluate.

In agent.py, find the if __name__ == "__main__": block at the bottom of the file.
Find this code:

if __name__ == "__main__":
    from dotenv import load_dotenv
    load_dotenv()

    from knowledge_base import build_knowledge_base, retrieve_context
    client = create_client()
    collection = build_knowledge_base()
    question = "How does asyncio achieve concurrency without threads, and when should I use it versus threading?"
    chunks = retrieve_context(collection, question, n_results=3)
    answer = generate(client, question, chunks)
    print(f"\nAnswer: {answer}")

Replace the entire block with this updated version:

if __name__ == "__main__":
    from dotenv import load_dotenv
    load_dotenv()

    # Run the full pipeline: retrieve, generate, evaluate
    client = create_client()
    collection = build_knowledge_base()
    question = "How does asyncio achieve concurrency without threads, and when should I use it versus threading?"
    chunks = retrieve_context(collection, question, n_results=3)
    answer = generate(client, question, chunks)
    print(f"\nAnswer: {answer[:200]}...")

    # Evaluate the answer
    score, flaws = evaluate(client, question, chunks, answer)
    print(f"\nScore: {score}/10")
    print(f"Flaws: {flaws}")

What changed here?

Removed the duplicate from knowledge_base import ... line since those imports already exist at the top of the file.
Truncated the answer output to 200 characters with answer[:200] so the score and flaws are easy to spot in the terminal.
Added the evaluate() call to score the answer and print both the numeric score and the list of flaws.

Save agent.py.

Run the pipeline and observe the gap

Before you run this, think about what score you expect. The generator has access to relevant context and a clear system prompt. Do you think it will score a perfect 10/10 on its first attempt?

Run the full pipeline by executing this command:

python agent.py

You will see the generated answer (truncated), followed by a score between 5 and 7 out of 10, and a list of specific flaws the evaluator identified.

The score is not 10. That is exactly what you want to see. The generator's first attempt has gaps, and now you have concrete feedback about what those gaps are.

Why is the first answer imperfect?

Even with good context, the generator often misses nuance, omits key details, or structures the answer in a way that loses clarity. The evaluator catches these issues because it has a strict scoring rubric. This gap between "good enough" and "actually great" is what motivates the entire self-improvement loop you'll build next.

Seeing a JSONDecodeError in the flaws?

If your flaws list shows Could not parse evaluator response, the LLM returned text that was not valid JSON. This is handled gracefully by the fallback in your except block. Run the script again. LLM outputs can vary between calls, and the next attempt usually returns properly formatted JSON.

If it keeps happening, check that your EVALUATOR_SYSTEM_PROMPT in context_engine.py includes the exact JSON format instruction.

help me debug my evaluator's JSON parsing

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before moving on.

ⓧ I'd like to double check the full code

Your agent.py is getting longer, so confirm each section is present in this order:

Imports at the top: json, os, SarvamAI, and the four context_engine functions.
create_client() function that reads the API key and returns a SarvamAI client.
call_llm() function that calls sarvam-30b with the given messages and temperature.
generate() function that assembles generator context and calls the LLM.
evaluate() function that calls the evaluator with temperature 0.2 and parses the JSON response.
if __name__ == "__main__": block that runs retrieve, generate, evaluate, and prints score and flaws.

You now have a feedback signal. The generator's first attempt scored below perfect, and you know exactly what's wrong with it. In the next step, you will close this gap by building the reflect-and-regenerate loop that turns those flaws into reusable lessons.

Complete the Reflexion Loop with Episodic Memory

Your agent can now generate an answer and identify exactly what's wrong with it. But each generation starts from scratch. The evaluator finds flaws, prints them, and then the agent does nothing with that knowledge.

In this step, you'll close the self-improvement loop. A reflector will convert evaluation flaws into reusable lessons stored in episodic memory. On each regeneration attempt, those lessons get injected into the context window alongside the original RAG chunks, giving the agent a second chance informed by what went wrong.

In this step, get ready to:

Build the reflector that converts flaws into reusable lessons.
Build the regenerator that uses episodic memory alongside RAG context.
Wire the complete Reflexion loop and watch scores climb across iterations.

Build the reflector

The reflector is the bridge between evaluation and improvement. It takes the list of flaws from the evaluator and distills them into a single actionable lesson that future iterations can use.

In agent.py, add the reflect() function below your existing evaluate() function:

def reflect(client, flaws):
    """Reflect on flaws and produce a lesson learned."""
    messages = assemble_reflector_context(flaws)
    response = call_llm(client, messages, temperature=0.3)

    if "LESSON:" in response:
        lesson = response.split("LESSON:")[1].strip()
    else:
        lesson = response.strip()

    return lesson

What does this code do?

The reflector calls the LLM with a low temperature (0.3) for consistent, focused output. It uses assemble_reflector_context() which you built in the previous step to format the flaws into a structured prompt.

The response parsing looks for the LESSON: prefix and extracts everything after it. If the model doesn't follow the format exactly, it falls back to using the raw response.

Build the regenerator and test both new roles

The regenerator is where context engineering pays off. Unlike the initial generator, it receives both the RAG chunks AND the accumulated lessons from episodic memory. Each of the four roles (generator, evaluator, reflector, regenerator) now has a distinct context window engineered for its specific job.

Add the regenerate() function directly below reflect():

def regenerate(client, question, chunks, episodic_memory):
    """Regenerate an answer using RAG context + episodic memory."""
    messages = assemble_regenerator_context(question, chunks, episodic_memory)
    print(f"[Regenerator] Context: {len(chunks)} chunks + {len(episodic_memory)} lessons from memory")
    answer = call_llm(client, messages, temperature=0.7)
    return answer

Why is this the key context engineering step?

The assemble_regenerator_context() function (in context_engine.py) builds a prompt with three distinct sections: retrieved RAG chunks, lessons from previous attempts, and the original question. This is fundamentally different from the initial generator's context window.

Each role gets a different context window. The generator sees chunks + question. The evaluator sees question + chunks + answer. The reflector sees only flaws. The regenerator sees chunks + lessons + question. This is structured prompt assembly in action.

Now test that both functions work before wiring the full loop.

In agent.py, find the if __name__ == "__main__": block at the bottom of the file.
Find the last two lines:

    print(f"\nScore: {score}/10")
    print(f"Flaws: {flaws}")

Add these lines directly below them, still inside the if __name__ block:

    # Test the reflector and regenerator
    lesson = reflect(client, flaws)
    print(f"\nLesson learned: {lesson}")
    answer2 = regenerate(client, question, chunks, [lesson])
    print(f"\nRegenerated answer: {answer2[:200]}...")

Save agent.py and run it:

python agent.py

You should see the original answer and score from before, followed by a new "Lesson learned" line and a regenerated answer that addresses the flaws.

Not seeing a lesson or regenerated answer?

Make sure both reflect() and regenerate() are placed after evaluate() but before the if __name__ block. Check that the new test lines are indented at the same level as the existing code inside the __main__ block.

If you see a JSON decode error, that means the evaluator's response wasn't valid JSON. The fallback handles this, so you should still get a score of 5 and a generic flaw message.help me debug my reflect or regenerate function

Wire the complete Reflexion loop

Both new roles work individually. Now you'll wire them into a single orchestration function that runs the full generate-evaluate-reflect-regenerate cycle for multiple iterations, accumulating lessons as it goes.

In agent.py, add the run_reflexion_loop() function below regenerate(). Start with the setup and first iteration:

def run_reflexion_loop(question, max_iterations=3, score_threshold=8):
    """Run the complete Reflexion self-improvement loop."""
    client = create_client()
    collection = build_knowledge_base()
    episodic_memory = []

    print(f"\n{'='*60}")
    print(f"Question: {question}")
    print(f"{'='*60}")

    # Retrieve context once (shared across all iterations)
    chunks = retrieve_context(collection, question, n_results=3)
    print(f"\nRetrieved {len(chunks)} chunks:")
    for chunk in chunks:
        print(f"  - [{chunk['id']}] distance: {chunk['distance']:.4f}")

    # Iteration 1: Initial generation
    print(f"\n--- Iteration 1 ---")
    answer = generate(client, question, chunks)
    print(f"\nAnswer: {answer[:200]}...")

    score, flaws = evaluate(client, question, chunks, answer)
    print(f"\nScore: {score}/10")
    print(f"Flaws: {flaws}")

    if score >= score_threshold:
        print(f"\nScore meets threshold ({score_threshold}). Done!")
        return answer, score, episodic_memory

What does this first section do?

It initializes the client, builds the knowledge base, and creates an empty episodic_memory list that will accumulate lessons across iterations.
Context is retrieved once and shared across all iterations. The agent refines its answer, not its search.
If the first attempt already meets the threshold, it returns early. No reflection needed.

Now add the reflection and regeneration loop directly below (still inside the same function). This goes right after the early return:

    # Reflection and regeneration loop
    for iteration in range(2, max_iterations + 1):
        print(f"\n--- Iteration {iteration} ---")

        # Reflect on flaws
        lesson = reflect(client, flaws)
        episodic_memory.append(lesson)
        print(f"\nLesson learned: {lesson}")

        # Regenerate with memory
        answer = regenerate(client, question, chunks, episodic_memory)
        print(f"\nImproved answer: {answer[:200]}...")

        # Re-evaluate
        score, flaws = evaluate(client, question, chunks, answer)
        print(f"\nScore: {score}/10")
        print(f"Flaws: {flaws}")

        if score >= score_threshold:
            print(f"\nScore meets threshold ({score_threshold}). Done!")
            break

    print(f"\n{'='*60}")
    print(f"Final score: {score}/10 after {min(iteration, max_iterations)} iterations")
    print(f"Lessons accumulated: {len(episodic_memory)}")
    for i, lesson in enumerate(episodic_memory, 1):
        print(f"  {i}. {lesson}")
    print(f"{'='*60}")

    return answer, score, episodic_memory

What does the loop do?

Each iteration reflects on the previous flaws, appends the lesson to episodic_memory, then regenerates with ALL accumulated lessons in the context window.
The loop breaks early if the score meets the threshold, or runs for max_iterations total attempts.
The final summary prints every lesson the agent accumulated, showing you what it learned.

Finally, replace the entire existing if __name__ == "__main__": block with a clean version that calls run_reflexion_loop().

Delete everything inside your current if __name__ == "__main__": block (all the test code you wrote earlier) and replace it with:

if __name__ == "__main__":
    from dotenv import load_dotenv
    load_dotenv()

    question = "How does asyncio achieve concurrency without threads, and when should I use it versus threading?"
    run_reflexion_loop(question, max_iterations=3, score_threshold=9)

Why set the threshold to 9?

A threshold of 9 ensures the agent has to work hard across multiple iterations. If you set it lower, the loop might exit after the first evaluation. Setting it high forces the full reflect-regenerate cycle to run multiple times so you can observe the improvement.

Before you run this, think about what you expect to see. Will the scores go up with each iteration? Will the agent reach 9 out of 10?

Save agent.py and run the full Reflexion loop:

python agent.py

You should see output for 3 iterations with scores trending upward (for example, 6/10, then 7/10, then 8/10). Each iteration prints the lesson it learned and the improved answer. The final summary shows all accumulated lessons.

What should I see?

The exact scores depend on the LLM's responses, but you should observe a clear upward trend. Iteration 1 typically scores 5-7 as the initial answer misses nuance. Iterations 2 and 3 score progressively higher as each lesson teaches the agent what to fix.

The key proof of self-improvement: the lessons in episodic memory are concrete ("Include the comparison between cooperative and preemptive multitasking") and each regenerated answer addresses those lessons directly.

Scores not improving across iterations?

If all iterations return the same score, check that episodic_memory.append(lesson) is inside the loop and that assemble_regenerator_context in context_engine.py includes the memory_text section. Without memory being passed to the regenerator, it produces the same answer each time.

If scores fluctuate randomly, that's normal with LLM-based evaluation. The overall trend should still be upward across 3 iterations.help me debug my Reflexion loop scores

✔️ Awesome, I've got everything!

Great. Double check you've saved agent.py with all the new functions.

ⓧ I'd like to double check the full code

Your agent.py should contain these constructs in order. Scroll through your file and confirm each is present:

Imports (json, os, SarvamAI, knowledge_base functions, context_engine functions).
create_client() function.
call_llm() function.
generate() function.
evaluate() function.
reflect() function.
regenerate() function.
run_reflexion_loop() function (setup, iteration 1, early exit check, loop with reflect/regenerate/evaluate, final summary).
if __name__ == "__main__": block calling run_reflexion_loop() with score_threshold=9.

Your agent now demonstrates the complete Reflexion pattern: generate, evaluate, reflect, and regenerate with episodic memory. Each iteration builds on the last, proving that structured self-critique and memory make LLM output measurably better. Next up, you'll push this further in the Secret Mission with multi-aspect evaluation.

Secret mission

Add Multi-Aspect Evaluation with Weighted Scoring

Your current evaluator gives a single score, but that hides which dimensions are actually improving. In production systems, agent output is scored across multiple quality dimensions with configurable weights. Can you upgrade your evaluator to score factual accuracy, completeness, and clarity separately, then use the weakest dimension to guide reflection?

Clean Up Your Resources

This project runs entirely locally, with API calls made only when you manually run the script. There are no ongoing costs or running processes to worry about.

Decide whether to keep your project files for future experimentation, or delete them entirely.

Resources you used:

The self-improving-agent project folder on your Desktop (contains all Python files, virtual environment, and .env with your API key).

Keep everything running

No action needed. Choose this if you want to keep experimenting with the agent, adding new knowledge base documents, or tweaking the evaluation prompts.

Your project folder stays on the Desktop ready for you to return to.
To run the agent again later, activate your virtual environment first with source venv/bin/activate (macOS/Linux) or venv\Scripts\activate (Windows).
Your Sarvam API key remains in .env inside the project folder. No API charges occur unless you run the script.

Pause - I'll come back to this later

Since the project is local and ChromaDB runs in-memory, there is nothing to pause. The agent only makes API calls when you explicitly run it.

Leave the folder as-is on your Desktop.
No processes are running in the background. No costs accumulate while the project sits idle.

Delete - I don't want to use this again

Remove all project files and the virtual environment from your machine.

Delete the project folder by running the command for your operating system.

On macOS/Linux:

cd ~/Desktop
rm -rf self-improving-agent

On Windows:

cd ~\Desktop
rmdir /s /q self-improving-agent

This removes all Python files, the virtual environment, and the .env file containing your API key.
If you want to revoke your Sarvam API key, visit your Sarvam AI dashboard and delete the key from your account settings.

Nice Work!

You just built a self-improving research agent from scratch. Your agent retrieves context, generates answers, critiques its own output, and regenerates better answers using lessons stored in episodic memory.

You've learned how to:

Build a RAG pipeline with ChromaDB that retrieves relevant context from a local knowledge base and grounds your agent's answers in real documents.
Engineer distinct context windows for four different agent roles (generator, evaluator, reflector, regenerator), each tailored to its specific job.
Implement the Reflexion pattern with episodic memory, producing measurable score improvements across iterations printed to your terminal.
Secret Mission: Added multi-aspect evaluation with weighted scoring across factual accuracy, completeness, and clarity dimensions.

Ready to quiz yourself?