Build an Adaptive RAG Agent with LangGraph

Build a self-correcting RAG agent that grades and rewrites queries using LangGraph.

Introduction

30 Second Summary

Have you ever asked a search engine a vague question and gotten back results that had nothing to do with what you meant? A smarter system would realize the results are bad, rephrase your question, and try again on its own.

In this project, you will build a Python-based research agent that answers questions about NFL rules and strategy. Unlike a basic retrieval pipeline that blindly trusts whatever documents it finds, your agent evaluates its own retrieval quality and rewrites queries when the results are not relevant.

What You'll Build

You ask your agent a vague question like "What happens when the clock runs out but they're still playing?", watch it retrieve documents, grade them for relevance, rewrite the query to something more precise, and generate a grounded answer about NFL overtime rules.

By the end of this project, you'll have:

  • A working adaptive RAG agent that retrieves NFL rules documents and grades them for relevance before deciding whether to answer.
  • A self-correction loop where the agent automatically rewrites vague queries and re-retrieves, with a retry limit that prevents infinite loops.
  • A graph-based orchestration pipeline built with LangGraph's StateGraph, conditional edges, and routing functions.
  • Secret Mission: A direct-answer routing path so the agent can skip retrieval entirely for general knowledge questions unrelated to football.

Are there any prerequisites?

This project assumes you are familiar with Python and have basic experience with LangChain (chains and prompts). You also need a free Google AI Studio API key, which you can create at aistudio.google.com.

Before We Start

Before We Start...

Before we dive in, let's lock in what you're about to build and why it matters for your growth as an AI engineer.

Set Up Your Environment

You need LangGraph and the Gemini integration packages installed before you can build the agent.

Verifying your API key works up front prevents debugging headaches later. In this step, you will install the dependencies, connect to the Gemini API, and confirm everything talks to each other.

In this step, get ready to:
  • Confirm Python 3.10+ is installed and create your project folder.
  • Install LangGraph, LangChain, and FAISS packages.
  • Connect to the Gemini API and verify you get a response.
Check Python and install dependencies

This project requires Python 3.10 or higher because LangGraph uses modern typing features introduced in that version.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • Open the integrated terminal by pressing Ctrl+` (backtick key, both macOS and Windows).
  • Check your Python version by running this command:
python3 --version

✔️ I see version 3.10 or higher

Your Python version is ready. Move on to the next instruction below.

ⓧ I see an older version

Your system has an older Python that does not support the typing features LangGraph needs. Install a newer version alongside it using pyenv.

  • On macOS, install pyenv with Homebrew, then install Python 3.10:
  • On Windows, download and install the latest Python 3.10+ from python.org/downloads. During installation, check the box that says Add Python to PATH.
# macOS only - install pyenv and Python 3.10
brew update
brew install pyenv
pyenv install 3.10
pyenv global 3.10

After installing, close and reopen your terminal, then run python3 --version again to confirm you see 3.10 or higher.

ⓧ Command not found

Python is not installed on your machine yet.

  • On macOS, install pyenv with Homebrew, then install Python:
  • On Windows, download and install Python from python.org/downloads. Check the box that says Add Python to PATH during installation.
# macOS only - install pyenv and Python
brew update
brew install pyenv
pyenv install 3.10
pyenv global 3.10

After installing, close and reopen your terminal, then run python3 --version to confirm it prints a version number.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir nfl-rag-agent
cd nfl-rag-agent
  • Confirm the folder exists by running:
ls

You should see an empty output because the folder is brand new.

  • Install the project packages by running this command:
pip install langgraph langchain-google-genai langchain-community langchain-text-splitters faiss-cpu

What does this install?

  • langgraph is the graph-based agent orchestration framework.
  • langchain-google-genai connects LangChain to the Gemini API.
  • langchain-community provides the FAISS vector store integration.
  • faiss-cpu is the vector similarity search library your retriever uses.

You will see pip downloading and installing several packages. This may take a minute.

  • Verify all packages imported correctly by running:
python3 -c "from langgraph.graph import StateGraph; from langchain_google_genai import ChatGoogleGenerativeAI; print('All imports OK')"

You should see All imports OK printed to the terminal.

Seeing ModuleNotFoundError?

  • Make sure you ran the pip install command from inside the nfl-rag-agent folder.
  • If you use python3 to run scripts, try pip3 install instead of pip install to ensure packages install for the correct Python version.
  • help me fix a ModuleNotFoundError when importing langgraph or langchain_google_genai
Set up your Gemini API key

Now you will create the main Python file and connect it to the Gemini API using the key from your Google AI Studio account.

  • Open the nfl-rag-agent folder in VS Code by clicking File in the top menu bar.
  • Click Open Folder.
  • Select the nfl-rag-agent folder on your Desktop, then click Open.
  • Create a new file by clicking File then New File.
  • Save it immediately as agent.py by pressing Cmd+S (macOS) or Ctrl+S (Windows) and typing agent.py as the filename.
  • Copy the following code into agent.py:
import os
from typing import Literal
from typing_extensions import TypedDict

from langchain_google_genai import ChatGoogleGenerativeAI, GoogleGenerativeAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_core.documents import Document
from langchain_core.prompts import ChatPromptTemplate
from langgraph.graph import StateGraph, START, END

# --- Configuration ---
os.environ["GOOGLE_API_KEY"] = "your-api-key-here"

llm = ChatGoogleGenerativeAI(model="gemini-3.1-flash-lite", temperature=0)
embeddings = GoogleGenerativeAIEmbeddings(model="models/gemini-embedding-001")

What does this code do?

  • The imports bring in LangGraph, LangChain, FAISS, and the Gemini integrations you installed.
  • os.environ["GOOGLE_API_KEY"] sets your API key as an environment variable so the Gemini SDK can find it.
  • llm creates a connection to the gemini-3.1-flash-lite model with temperature=0 for deterministic outputs.
  • embeddings creates an embedding model for turning text into vectors later.
  • Replace your-api-key-here with your actual Google AI Studio API key.
  • Save the file with Cmd+S (macOS) or Ctrl+S (Windows).

Keep your API key safe

For this learning project, pasting the key directly into the file is fine. In production code, you would store it in a .env file and load it with a library like python-dotenv.

Verify the setup works end-to-end

The imports and config are in place. Now add a quick test that actually calls the Gemini API to confirm everything is wired up.

  • Add the following code at the bottom of agent.py, below the embeddings line:
# Quick test
if __name__ == "__main__":
    response = llm.invoke("Say hello!")
    print(response.content)

What does this code do?

The if __name__ == "__main__" block runs only when you execute agent.py directly. It sends a simple prompt to Gemini and prints whatever the model responds with.

  • Save agent.py.

Before you run this, what do you expect to see? A greeting from the model, or an error?

  • Open the terminal in VS Code (Ctrl+`) and run:
python3 agent.py

You should see a friendly greeting from Gemini printed in your terminal, something like "Hello! How can I help you today?".

Seeing an authentication error?

  • Double-check that you replaced your-api-key-here with your actual API key from Google AI Studio.
  • Make sure there are no extra spaces or quotation marks around the key.
  • If you see a rate limit error, wait 10 seconds and try again. The Gemini free tier can be briefly unavailable.
  • help me fix a Gemini API authentication error in my Python script

✔️ Awesome, I've got everything!

Great. Double check you have saved agent.py and that a Gemini response printed in your terminal.

ⓧ I'd like to double check the full code

Here is the complete agent.py you should have so far:

import os
from typing import Literal
from typing_extensions import TypedDict

from langchain_google_genai import ChatGoogleGenerativeAI, GoogleGenerativeAIEmbeddings
from langchain_community.vectorstores import FAISS
from langchain_core.documents import Document
from langchain_core.prompts import ChatPromptTemplate
from langgraph.graph import StateGraph, START, END

# --- Configuration ---
os.environ["GOOGLE_API_KEY"] = "your-api-key-here"

llm = ChatGoogleGenerativeAI(model="gemini-3.1-flash-lite", temperature=0)
embeddings = GoogleGenerativeAIEmbeddings(model="models/gemini-embedding-001")

# Quick test
if __name__ == "__main__":
    response = llm.invoke("Say hello!")
    print(response.content)

Your environment is ready and Gemini is responding. Next up, you will build a simple RAG pipeline that retrieves NFL documents and generates answers using a LangGraph StateGraph.

Build a Simple RAG Pipeline

In the previous step, you confirmed that your Gemini connection works. Now you need something for the agent to actually reason about.

Before adding any intelligence, you need a working baseline. This step creates a naive retrieve-then-generate pipeline using LangGraph's StateGraph so you can see the basic graph pattern in action with your NFL knowledge base.

In this step, get ready to:
  • Create an NFL knowledge base with FAISS vector search.
  • Define a state schema and two graph nodes (retrieve and generate).
  • Wire the nodes into a working StateGraph and run it.
Create the NFL knowledge base

Every RAG pipeline starts with a knowledge base. Your agent will answer questions about NFL rules, so you need documents covering different topics and a way to search them by meaning.

  • In agent.py, find and delete the entire # Quick test section at the bottom of the file (the if __name__ block and the two lines inside it).
  • Add the following NFL knowledge base code below your embeddings line:
# --- NFL Knowledge Base ---
documents = [
    Document(page_content="NFL Scoring: A touchdown is worth 6 points and is scored when a player carries the ball into or catches it in the opposing team's end zone. After a touchdown, the team can attempt an extra point kick (1 point) or a two-point conversion by running or passing the ball into the end zone again. A field goal is worth 3 points and is kicked through the uprights. A safety is worth 2 points and is scored when the defense tackles an offensive player in their own end zone."),
    Document(page_content="NFL Penalties: Pass interference occurs when a defender or receiver significantly hinders the other player's ability to catch the ball. Defensive pass interference results in an automatic first down at the spot of the foul. Holding is when a player grabs an opponent other than the ball carrier. Offensive holding results in a 10-yard penalty. Offsides occurs when a defensive player crosses the line of scrimmage before the snap, resulting in a 5-yard penalty."),
    Document(page_content="NFL Overtime Rules: If a game is tied at the end of regulation, a 10-minute overtime period begins. Each team gets at least one possession unless the team that receives the opening kickoff scores a touchdown on its first drive. If the score is still tied after both teams have had a possession, the next score wins (sudden death). In the playoffs, overtime periods continue until there is a winner. During the regular season, games can end in a tie if neither team scores in overtime."),
    Document(page_content="NFL Two-Minute Warning: The two-minute warning is an automatic timeout that occurs when the game clock reaches 2:00 remaining in each half (end of 2nd and 4th quarters). It serves as a television timeout and a strategic break. Teams cannot call timeout to prevent the two-minute warning. After the two-minute warning, different clock rules apply: the clock stops after incomplete passes, out-of-bounds plays, and first downs until the ball is set."),
    Document(page_content="NFL Positions - Offense: The quarterback (QB) leads the offense and throws or hands off the ball. Running backs (RB) carry the ball on rushing plays. Wide receivers (WR) run routes and catch passes. Tight ends (TE) block and catch passes. The offensive line (center, guards, tackles) protects the QB and opens running lanes. Defense: Defensive linemen rush the passer and stop the run. Linebackers cover short passes and stop the run. Cornerbacks and safeties cover receivers and defend against the pass."),
    Document(page_content="NFL Draft and Salary Cap: The NFL Draft is held annually with 7 rounds. Teams select in reverse order of their previous season record (worst team picks first). The salary cap is a limit on the total amount teams can spend on player salaries. Teams must manage the cap by structuring contracts with signing bonuses, incentives, and guaranteed money. Teams that exceed the cap face penalties. The franchise tag allows teams to retain a key player by offering them a one-year contract at the average of the top 5 salaries at their position."),
]

vectorstore = FAISS.from_documents(documents, embeddings)
retriever = vectorstore.as_retriever(search_kwargs={"k": 2})

What does this code do?

  • Six Document objects hold plain-text NFL knowledge covering scoring, penalties, overtime, the two-minute warning, positions, and the draft.
  • FAISS creates a local vector store by converting each document into an embedding (a list of numbers representing meaning) and indexing them for fast similarity search.
  • The retriever wraps the vector store and returns the 2 most relevant documents for any query (k=2).
  • Add a quick retrieval test at the bottom of agent.py so you can confirm the knowledge base works:
# Quick test - verify retrieval works
if __name__ == "__main__":
    results = retriever.invoke("How does scoring work in football?")
    print(f"Retrieved {len(results)} documents:")
    for doc in results:
        print(f"  - {doc.page_content[:80]}...")
  • Save agent.py and run it by running:
python3 agent.py

You should see 2 retrieved documents, with the first starting with "NFL Scoring" since it is the most relevant match.

Seeing an error about FAISS?

  • Make sure you installed faiss-cpu (not faiss-gpu). Re-run pip install faiss-cpu if needed.
  • If you see a timeout or API error from the embeddings call, wait a few seconds and run again. The Gemini free tier can be intermittent.

help me debug a FAISS import or embedding error in my LangChain agent

Define the agent state and nodes

In LangGraph, a StateGraph passes a shared state dictionary between nodes. Each node is a Python function that reads from and writes to that state.

  • Delete the # Quick test section you just added (the if __name__ block and everything below it).
  • Add the state definition and your first node below the retriever line:
# --- Define State ---
class AgentState(TypedDict):
    question: str
    documents: list[Document]
    generation: str
    retries: int


# --- Define Nodes ---
def retrieve(state: AgentState) -> dict:
    """Retrieve documents from the vector store."""
    print(f"--- RETRIEVING for: {state['question']} ---")
    question = state["question"]
    documents = retriever.invoke(question)
    return {"documents": documents}

What does this code do?

  • AgentState is a TypedDict that defines every field the graph can read and write. All nodes share this state.
  • The retrieve node reads question from the state, queries the vector store, and writes the results back into documents.
  • The print statement gives you visibility into what the agent is doing at each step when it runs.
  • Now add the generate node directly below the retrieve function:
def generate(state: AgentState) -> dict:
    """Generate an answer using retrieved documents."""
    question = state["question"]
    documents = state.get("documents", [])

    docs_content = "\n\n".join(doc.page_content for doc in documents)

    prompt = ChatPromptTemplate.from_messages([
        ("system",
         "You are a helpful NFL rules expert. Answer the question based ONLY on the "
         "following context. If the context doesn't contain relevant information, "
         "say 'I don't have enough information to answer this question.'\n\n"
         "Context:\n{context}"),
        ("human", "{question}")
    ])

    chain = prompt | llm
    response = chain.invoke({"context": docs_content, "question": question})
    print("--- GENERATED ANSWER ---")
    return {"generation": response.content}

What does this code do?

  • The generate node joins the retrieved documents into a single context string.
  • A ChatPromptTemplate instructs the LLM to answer ONLY from the given context. This prevents hallucination when the context is relevant.
  • The prompt | llm pipe chains the template and the model together using LangChain's expression language.
  • Save agent.py.
Build and run the simple graph

You have two nodes ready. Now you wire them into a graph with a clear flow: start, retrieve documents, generate an answer, end.

  • Add the graph construction and test invocation at the bottom of agent.py:
# --- Build the Graph ---
workflow = StateGraph(AgentState)

workflow.add_node("retrieve", retrieve)
workflow.add_node("generate", generate)

workflow.add_edge(START, "retrieve")
workflow.add_edge("retrieve", "generate")
workflow.add_edge("generate", END)

app = workflow.compile()

# --- Run the Agent ---
if __name__ == "__main__":
    result = app.invoke({"question": "What is a safety in football?", "retries": 0})
    print(f"\nAnswer: {result['generation']}\n")

What does this code do?

  • StateGraph(AgentState) creates a new graph that passes AgentState between every node.
  • add_node registers each function as a named node. add_edge connects them in sequence: START to retrieve, retrieve to generate, generate to END.
  • compile() freezes the graph into a runnable application. invoke() sends an initial state through the entire graph and returns the final state.

Before you run this, what do you think the agent will print? Will it retrieve relevant documents about safeties, or something unrelated?

  • Save agent.py and run it:
python3 agent.py

You should see the agent print "--- RETRIEVING for: What is a safety in football? ---", then "--- GENERATED ANSWER ---", followed by a correct explanation that a safety is worth 2 points and is scored when the defense tackles an offensive player in their own end zone.

Getting an empty answer or error?

  • If you see a KeyError, make sure you pass "retries": 0 in the invoke() call. LangGraph requires all state fields to be present at invocation.
  • If you see a rate limit error from Gemini, wait 10 seconds and run again. The free tier is intermittent under load.
  • Check that your GOOGLE_API_KEY is set correctly at the top of the file.

help me debug why my LangGraph agent is not returning an answer

✔️ Awesome, I've got everything!

Great. Double check you have saved agent.py and the agent prints a correct answer about safeties.

ⓧ I'd like to double check the full code

Your agent.py should contain these sections in order. Scroll through your file and confirm each is present:

  • Imports (os, typing, typing_extensions, langchain imports, langgraph imports).
  • Configuration section with GOOGLE_API_KEY, llm, and embeddings.
  • NFL Knowledge Base with 6 Document objects, vectorstore, and retriever.
  • Define State section with AgentState TypedDict (4 fields).
  • Define Nodes section with retrieve and generate functions.
  • Build the Graph section with StateGraph, nodes, edges, and compile().
  • Run the Agent section with if __name__ == "__main__" invoking the question about safeties.

Your baseline RAG pipeline is working. It retrieves documents and generates grounded answers. Next up, you will expose a critical flaw in this pipeline: it trusts its retrieval results even when they are completely irrelevant.

Add Document Grading

Your simple RAG pipeline from the last step retrieves documents and generates an answer every time. It has no way to evaluate whether the documents it found are actually helpful.

This is a real problem. When a user asks something outside the knowledge base, the pipeline still pulls the closest documents and confidently generates an answer from irrelevant content. In this step, you will expose that hallucination problem and fix it with a self-evaluation layer.

In this step, get ready to:
  • Expose the naive pipeline's hallucination problem with an out-of-scope question.
  • Create a document grading function that uses LLM self-evaluation.
  • Wire grading into the graph as a conditional edge.
Expose the limitation

Right now, your agent always trusts whatever documents the retriever returns. What happens if someone asks a question that has nothing to do with NFL rules?

  • In agent.py, scroll to the if __name__ == "__main__": block at the bottom of the file.
  • Add a second test question below the existing one by pasting this code after the last print statement:
result = app.invoke({"question": "What is the best strategy for fantasy basketball drafts?", "retries": 0})
print(f"\nAnswer: {result['generation']}\n")

What does this test?

This question is about basketball, not football. Your NFL knowledge base has zero useful information about fantasy basketball drafts. A smart agent should recognise this and say so.

Before you run this, what do you think will happen? Will the agent admit it does not know, or will it try to answer anyway?

  • Run the agent to find out:
python3 agent.py

You will see the agent confidently answer the basketball question using NFL rules content. It pulls documents about scoring or the draft and weaves them into a response that sounds authoritative but is completely wrong.

This hallucination is the exact problem we are solving

The pipeline retrieves the closest documents it can find, even when those documents are irrelevant. Then the generate step dutifully answers using that irrelevant context. There is no quality gate between retrieval and generation.

The fix: add a grading step that evaluates document relevance before passing them to generation.

Create the document grading function

The grading function acts as a quality gate. It asks the LLM a simple yes-or-no question: are these documents relevant to the user's question?

  • In agent.py, find the # --- Build the Graph --- comment. Add the following function directly above that comment, after the generate function:
def grade_documents(state: AgentState) -> Literal["generate", "rewrite_question"]:
    """Grade retrieved documents for relevance."""
    print("--- GRADING DOCUMENTS ---")
    question = state["question"]
    documents = state["documents"]

    docs_content = "\n\n".join(doc.page_content for doc in documents)

    prompt = ChatPromptTemplate.from_messages([
        ("system",
         "You are a relevance grader. Given a question and retrieved documents, "
         "determine if the documents contain information relevant to answering "
         "the question. Respond with ONLY the word 'relevant' or 'irrelevant'."),
        ("human",
         "Question: {question}\n\nDocuments:\n{documents}\n\n"
         "Are these documents relevant to the question?")
    ])

    chain = prompt | llm
    response = chain.invoke({"question": question, "documents": docs_content})

    grade = response.content.strip().lower()
    if "irrelevant" in grade:
        print("Documents graded: irrelevant")
        return "rewrite_question"
    else:
        print("Documents graded: relevant")
        return "generate"

What does this code do?

This function serves double duty in LangGraph. It grades the documents AND acts as a routing function that returns the name of the next node to execute.

  • It joins all retrieved documents into a single string and sends them to the LLM alongside the original question.
  • The LLM responds with only the word "relevant" or "irrelevant".
  • If irrelevant, it returns "rewrite_question" to route toward a rewrite path (which you will build in the next step).
  • If relevant, it returns "generate" to proceed with answer generation.

Why is a conditional edge function different from a regular node?

A regular node function returns state updates (a dict). A conditional edge function returns a string naming the next node. LangGraph uses the return value to decide which path the graph takes. This is how you build branching logic without if-else spaghetti.

Wire grading into the graph as a conditional edge

Now you need to replace the direct edge between retrieve and generate with a conditional edge that passes through your grading function.

  • In agent.py, scroll to the # --- Build the Graph --- section.
  • Find this line:
workflow.add_edge("retrieve", "generate")
  • Replace it with this conditional edge:
workflow.add_conditional_edges("retrieve", grade_documents)

What changed?

Instead of always routing from retrieve to generate, the graph now calls grade_documents after retrieval. Whatever string that function returns becomes the next node in the execution path.

  • Now replace the entire if __name__ == "__main__": block at the bottom with two clearly labeled tests:
# --- Run the Agent ---
if __name__ == "__main__":
    print("=" * 50)
    print("TEST 1: Question within knowledge base scope")
    print("=" * 50)
    result = app.invoke({"question": "What is a safety in football?", "retries": 0})
    print(f"\nAnswer: {result['generation']}\n")

    print("=" * 50)
    print("TEST 2: Question outside knowledge base scope")
    print("=" * 50)
    result = app.invoke({"question": "What is the best strategy for fantasy basketball drafts?", "retries": 0})
    print(f"\nAnswer: {result['generation']}\n")

What do the two tests prove?

  • Test 1 asks about a safety in football. The documents are relevant, so grading should return "generate" and you get a correct answer.
  • Test 2 asks about fantasy basketball. The NFL documents are irrelevant, so grading should detect this and print Documents graded: irrelevant.
  • Save agent.py.

Before you run this, what do you expect to see for Test 2? Will the agent still hallucinate, or will the grader catch the irrelevant documents?

  • Run the updated agent:
python3 agent.py

For Test 1, you should see Documents graded: relevant followed by a correct answer about safeties. For Test 2, you should see Documents graded: irrelevant printed to the terminal.

Not seeing the expected output?

  • If you get a NameError for grade_documents, make sure you defined the function above the # --- Build the Graph --- section, not below it. Python reads top-to-bottom.
  • If both questions grade as "relevant", check that your grading prompt matches the code above exactly. The LLM needs the precise instruction to respond with only "relevant" or "irrelevant".
  • If you see a rate limit error, wait 10 seconds and try again. The Gemini free tier can be intermittent.

Help me debug my document grading output.

✔️ Awesome, I've got everything!

Great. Double check you have saved agent.py and both tests produce the expected grading output.

ⓧ I'd like to double check the full code

Here is the complete agent.py at this point. Scroll through and confirm each section matches yours.

Your file has these sections in order:

  • Imports and configuration (lines 1-15).
  • NFL Knowledge Base with 6 Document objects (lines 17-28).
  • Vectorstore and retriever setup (lines 30-31).
  • AgentState TypedDict (lines 34-39).
  • Three functions: retrieve, generate, and grade_documents.
  • Graph wiring with add_conditional_edges("retrieve", grade_documents).
  • Two test invocations in the __main__ block.

Your agent can now tell the difference between relevant and irrelevant retrieval results. But detecting irrelevance is only half the solution. Right now, when documents are graded irrelevant, the agent has no recovery path. Next up, you will add a self-correction loop that rewrites the question and tries again.

Add Self-Correction with Query Rewriting

Your grader now catches irrelevant documents, but the agent is stuck. When the basketball question triggers an "irrelevant" grade, the graph tries to route to a rewrite_question node that does not exist yet.

In this step, you will give the agent the ability to recover. You will build a rewrite-and-retry loop that reformulates the question and tries retrieval again, completing the self-correction cycle that makes this agent adaptive.

In this step, get ready to:
  • Create a query rewriting node that reformulates vague questions.
  • Add a retry safety check that prevents infinite loops.
  • Wire the complete self-correction loop into your graph.
Create the rewrite_question node

When the grader decides retrieved documents are irrelevant, the agent needs to try a different question. A rewrite node takes the original question, asks the LLM to reformulate it for better retrieval, and increments a retry counter.

  • In agent.py, find the grade_documents function. Add the following rewrite_question function directly below it:
def rewrite_question(state: AgentState) -> dict:
    """Rewrite the question to improve retrieval."""
    question = state["question"]
    retries = state.get("retries", 0)

    prompt = ChatPromptTemplate.from_messages([
        ("system",
         "You are a question rewriter. Rewrite the following question to be more "
         "specific and better suited for retrieving relevant documents from an "
         "NFL rules and strategy knowledge base. "
         "Return ONLY the rewritten question, nothing else."),
        ("human", "{question}")
    ])

    chain = prompt | llm
    response = chain.invoke({"question": question})
    new_question = response.content.strip()
    print(f"--- REWRITING: '{question}' -> '{new_question}' ---")
    return {"question": new_question, "retries": retries + 1}

What does this code do?

  • The prompt instructs the LLM to rewrite the question so it better matches the NFL knowledge base. A vague question like "What happens when the clock runs out?" might become "What are NFL overtime rules?"
  • retries = state.get("retries", 0) reads the current retry count (defaulting to 0 on the first pass).
  • The function returns the rewritten question and increments the retry counter by 1. This counter is what prevents infinite loops.
  • Save agent.py.
Add a retry safety check

Without a limit, the agent could rewrite and re-retrieve forever. A routing function checks the retry count and decides whether to loop back for another attempt or give up and generate with whatever documents are available.

  • In agent.py, add the following should_retry function directly below rewrite_question:
def should_retry(state: AgentState) -> Literal["retrieve", "generate"]:
    """Check if we should retry retrieval or give up."""
    retries = state.get("retries", 0)
    if retries >= 2:
        print("--- MAX RETRIES REACHED, generating with available docs ---")
        return "generate"
    return "retrieve"

What does this code do?

  • This is a conditional edge function. It returns the name of the next node to route to.
  • If retries >= 2, the agent stops trying and routes to generate with whatever documents it has. This prevents infinite loops in production systems.
  • If retries are still under the limit, it routes back to retrieve to try the rewritten question against the knowledge base.
  • Save agent.py.
  • Verify you have no syntax errors by running this command:
python3 -c "compile(open('agent.py').read(), 'agent.py', 'exec'); print('Syntax OK')"

You should see Syntax OK printed. If you see a SyntaxError, check for missing colons or mismatched parentheses in the two new functions.

Seeing a SyntaxError?

  • Check that the triple-quoted docstrings use straight double quotes, not curly quotes.
  • Make sure the f"--- REWRITING: line uses an f-string prefix and matching quotes.
  • Confirm both functions are at the top indentation level (not nested inside another function).

Help me debug a SyntaxError in my rewrite_question or should_retry function.

Complete the graph with the self-correction loop

Now you will wire everything together. The rewrite_question node needs to be registered in the graph, and the conditional edges need to connect it to the retry logic.

  • In agent.py, find the # --- Build the Graph --- section. Replace everything from that comment down to app = workflow.compile() with the following:
# --- Build the Graph ---
workflow = StateGraph(AgentState)

# Add nodes
workflow.add_node("retrieve", retrieve)
workflow.add_node("generate", generate)
workflow.add_node("rewrite_question", rewrite_question)

# Add edges
workflow.add_edge(START, "retrieve")
workflow.add_conditional_edges("retrieve", grade_documents)
workflow.add_conditional_edges("rewrite_question", should_retry)
workflow.add_edge("generate", END)

# Compile
app = workflow.compile()

What does this code do?

  • workflow.add_node("rewrite_question", rewrite_question) registers the new node so the graph can route to it.
  • workflow.add_conditional_edges("retrieve", grade_documents) routes to either generate (relevant docs) or rewrite_question (irrelevant docs) based on what grade_documents returns.
  • workflow.add_conditional_edges("rewrite_question", should_retry) creates the loop. After rewriting, the agent either goes back to retrieve (retry) or forward to generate (give up).

This forms a cycle in the graph: retrieve, grade, rewrite, retrieve again. The retry limit breaks the cycle after 2 attempts.

  • Now replace the entire # --- Run the Agent --- section at the bottom of the file with this updated version that adds a third test question:
# --- Run the Agent ---
if __name__ == "__main__":
    print("=" * 50)
    print("TEST 1: Question within knowledge base scope")
    print("=" * 50)
    result = app.invoke({"question": "What is a safety in football?", "retries": 0})
    print(f"\nAnswer: {result['generation']}\n")

    print("=" * 50)
    print("TEST 2: Question outside knowledge base scope")
    print("=" * 50)
    result = app.invoke({"question": "What is the best strategy for fantasy basketball drafts?", "retries": 0})
    print(f"\nAnswer: {result['generation']}\n")

    print("=" * 50)
    print("TEST 3: Vague question that benefits from rewriting")
    print("=" * 50)
    result = app.invoke({"question": "What happens when the clock runs out but they're still playing?", "retries": 0})
    print(f"\nAnswer: {result['generation']}\n")

What does the new test check?

Test 3 asks a deliberately vague question. The agent should retrieve documents, grade them as relevant enough (or rewrite if needed), and produce an answer about NFL overtime rules. This demonstrates the full adaptive loop in action.

  • Save agent.py.

Before you run the agent, think about what you expect to happen for each test. Will all three produce answers? Will any trigger the rewrite path?

  • Run the agent by executing this command:
python3 agent.py

Watch the output carefully. For Test 1, you should see the agent retrieve and generate directly. For Test 2, the grader marks documents as irrelevant, triggers a rewrite, retries, and eventually generates a graceful "I don't have enough information" response. For Test 3, the vague question triggers retrieval and you should see a correct answer about overtime procedures.

What should I see?

Your terminal output shows the full execution path for each test. Look for these key indicators:

  • Test 1: --- RETRIEVING --- then Documents graded: relevant then --- GENERATED ANSWER ---.
  • Test 2: Documents graded: irrelevant followed by --- REWRITING --- showing the question transformation. After max retries, the answer says it does not have enough information.
  • Test 3: Retrieval followed by grading. The vague wording may trigger a rewrite, or the agent might grade the overtime docs as relevant and answer directly.

Seeing a ValueError about a missing node?

  • Make sure workflow.add_node("rewrite_question", rewrite_question) appears before the add_conditional_edges calls. LangGraph requires all target nodes to exist before you wire edges to them.
  • Confirm the string "rewrite_question" in add_node matches the return value of grade_documents exactly (case-sensitive).

Why is my LangGraph agent throwing a ValueError about a missing node when I run it?

✔️ Awesome, I've got everything!

Great. Double check you have saved agent.py and all three tests produce output.

ⓧ I'd like to double check the full code

Your agent.py file is now over 30 lines, so confirm each section is present in this order:

  • Imports and configuration (os.environ, llm, embeddings).
  • NFL Knowledge Base (6 Document objects, vectorstore, retriever).
  • AgentState TypedDict with question, documents, generation, retries.
  • Node functions: retrieve, generate, grade_documents, rewrite_question, should_retry.
  • Graph wiring: 3 nodes (retrieve, generate, rewrite_question), conditional edges from retrieve and rewrite_question.
  • Three test invocations: safety, basketball, and overtime questions.

Your agent now evaluates its own retrieval quality and self-corrects when documents are not relevant. In the Secret Mission, you will add an even smarter routing layer that decides whether retrieval is needed at all.

Secret mission

Add Direct-Answer Routing

Your agent always retrieves from the NFL knowledge base, even for questions that have nothing to do with football. What if it could decide whether retrieval is even necessary, and answer general knowledge questions directly without wasting a retrieval step?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally with no cloud resources, so there are no ongoing costs.

Resources you used:

  • Local project directory nfl-rag-agent on your Desktop.
  • Python packages installed via pip (langgraph, langchain-google-genai, langchain-community, langchain-text-splitters, faiss-cpu).
  • Google AI Studio API key (pre-existing, used for Gemini API calls).

Keep everything running

No action needed. Choose this if you are still actively building or want to extend the agent with new tools or knowledge bases.

  • Your agent.py file stays on your Desktop, ready to run anytime.
  • Your installed pip packages remain available for future Python projects.
  • Your Google AI Studio API key stays active for other Gemini projects.

Pause - I'll come back to this later

Since everything runs locally and there are no cloud resources, there is nothing to pause. Your files and packages sit idle on disk at no cost.

  • Your project folder and installed packages use a small amount of disk space but cost nothing.
  • When you are ready to return, open agent.py in VS Code and run it again.

Delete - I don't want to use this again

Remove all project files and uninstall the packages you installed for this project.

  • Delete the project folder by running the following command in your terminal.

macOS

rm -rf ~/Desktop/nfl-rag-agent

Windows

rmdir /s /q %USERPROFILE%\Desktop\nfl-rag-agent
  • Uninstall the pip packages you installed for this project.
pip uninstall langgraph langchain-google-genai langchain-community langchain-text-splitters faiss-cpu -y
  • Your Google AI Studio API key remains active in your Google account. If you want to revoke it, visit the Google AI Studio keys page and delete the key.

Nice Work!

Nice Work!

Nice work! You just built an adaptive RAG agent that evaluates its own retrieval quality, rewrites queries when documents fall short, and makes intelligent routing decisions. That is production-grade AI orchestration.

You've learned how to:

  • Build a LangGraph StateGraph with nodes, edges, and conditional routing to orchestrate multi-step agent behavior.
  • Implement document grading as a self-evaluation mechanism so your agent knows when its retrieval results are not useful.
  • Add query rewriting as a self-correction loop with retry limits that prevent infinite cycles.
  • Secret Mission: Added intelligent routing that decides whether retrieval is needed at all, letting the agent skip the knowledge base for general knowledge questions.

Ready to quiz yourself?