Build a Multi-Tool AI Agent API

Build a FastAPI service with a LangChain agent that uses tools to answer queries.

Introduction

30 Second Summary

AI assistants like ChatGPT don't just generate text. They decide which tools to use (calculator, web search, code runner) without you telling them which one to pick. That decision-making ability is called "tool use," and it's the most in-demand skill for production AI systems today.

In this project, you will build a REST API that wraps an AI agent capable of autonomously picking the right tool and returning structured JSON responses to any natural language query.

What You'll Build

A FastAPI-powered AI Agent API that receives queries and uses Groq's free Llama 3.3 70B model through LangChain to autonomously select and call tools before composing a final answer.

By the end of this project, you'll have:

  • A FastAPI application with typed Pydantic request/response models and auto-generated Swagger documentation.
  • A LangChain agent that autonomously selects and calls tools (calculator, text analyzer, datetime) based on the user's query.
  • Structured JSON responses with proper HTTP error handling and status codes.
  • Secret Mission: Add conversation memory so the agent remembers context across multiple requests in the same session.

Are there any prerequisites?

You should be comfortable with Python basics (functions, classes, and pip). Familiarity with APIs is helpful but not required.

Before We Start

Production AI systems need more than a chatbot interface. They need typed API contracts, tool orchestration, and proper error handling.

Before writing any application code, you need a Python environment with the right packages and a free API key from Groq. This step gets everything ready.

In this step, get ready to:
  • Create a project folder and Python virtual environment.
  • Install LangChain, FastAPI, and the Groq integration.
  • Create a free Groq API key and verify the connection.
Create a project folder and virtual environment

You need an isolated Python environment so the packages you install for this project don't conflict with anything else on your machine. A virtual environment handles this.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or Command Prompt (Windows) and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir ai-agent-api
cd ai-agent-api
  • Confirm the folder was created by running:
ls

Since you just created an empty folder and moved into it, ls should show an empty directory.

  • Create a Python virtual environment by running this command:

macOS

python3 -m venv venv

Windows

python -m venv venv
  • Activate the virtual environment:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the start of your terminal prompt. This confirms the virtual environment is active.

  • Verify your Python version by running:
python --version

You should see Python 3.11.x in the output.

Don't see (venv) in your prompt?

Make sure you ran the activate command for your operating system. On macOS it's source venv/bin/activate and on Windows it's venv\Scripts\activate.

If you see a permissions error on Windows, try running your terminal as Administrator.

Why isn't my virtual environment activating?

Install dependencies

You need three main packages: LangChain for agent orchestration, the Groq integration for the LLM, and FastAPI for the web server.

  • Install all required packages by running this command:
pip install langchain==1.3.8 langchain-groq==1.1.3 "fastapi[standard]"

What does this install?

  • langchain==1.3.8 is the core orchestration framework for building AI agents with tool-calling loops.
  • langchain-groq==1.1.3 connects LangChain to Groq's ultra-fast inference API.
  • fastapi[standard] installs FastAPI along with uvicorn (the web server), Pydantic v2 (for typed models), and other essentials.

The install may take a minute. You should see a success message ending with Successfully installed followed by a list of packages.

Seeing an installation error?

Make sure your virtual environment is active. You should see (venv) in your terminal prompt. If not, re-run the activate command from the previous substep.

If you see a network timeout, check your internet connection and try the pip install command again.

Help me fix my pip install error

Create a free Groq API key

Groq provides free access to Llama 3.3 70B, a powerful open-source LLM. You need an API key to send requests to their inference service.

  • Open your browser and go to console.groq.com.
  • Sign up for a free account using Google or GitHub. No credit card is required.
  • Once logged in, navigate to console.groq.com/keys to open the API Keys page.
  • Click Create API Key.
  • Give your key a name (e.g. ai-agent-api) and click Submit.
  • Copy the key immediately. You won't be able to see it again after closing the dialog.

Why Groq?

Groq's free tier gives you access to Llama 3.3 70B, a model far more capable than what most consumer hardware can run locally. You don't need to download multi-GB model files or have a GPU.

The free tier provides 30 requests per minute and 1,000 requests per day. That's more than enough for development and portfolio demos.

  • Set the GROQ_API_KEY environment variable in your terminal so your code can access it:

macOS

export GROQ_API_KEY=your-api-key-here

Replace your-api-key-here with the key you just copied from the Groq console.

Windows

If using Command Prompt:

set GROQ_API_KEY=your-api-key-here

If using PowerShell:

$env:GROQ_API_KEY="your-api-key-here"

Replace your-api-key-here with the key you just copied from the Groq console.

Why an environment variable?

Storing API keys in environment variables keeps them out of your source code. This is a security best practice. The langchain-groq package automatically reads from GROQ_API_KEY so you don't need to paste the key into your Python files.

  • Verify everything works by creating a test script. Create a file called test_setup.py in your ai-agent-api folder and paste the following code:
from langchain_groq import ChatGroq

# Create a ChatGroq instance using the free Llama 3.3 70B model
llm = ChatGroq(model="llama-3.3-70b-versatile", temperature=0)

# Send a test message to verify the connection
response = llm.invoke("Say hello in one sentence.")
print(response.content)

What does this code do?

  • ChatGroq creates a connection to Groq's API. It automatically reads GROQ_API_KEY from your environment.
  • model="llama-3.3-70b-versatile" tells Groq which LLM to use. This is the same model your agent will use throughout the project.
  • llm.invoke() sends a message to the model and returns the response.
  • Run the test script:
python test_setup.py

You should see a friendly greeting printed in your terminal. This confirms that your packages are installed correctly and your Groq API key is working.

Seeing an authentication error?

Double-check that you set the GROQ_API_KEY environment variable in the same terminal session where you're running the script. If you opened a new terminal window, you need to set it again.

Make sure you copied the full API key from the Groq console without any extra spaces.

Help me fix my Groq API authentication error

Your environment is ready. Next up, you'll build the FastAPI application with typed Pydantic models that define the exact shape of your API's requests and responses.

Build the FastAPI App with Pydantic Models

Your environment is ready and your connection to Groq's LLM is confirmed. Now it's time to build the actual API layer that will expose your agent to the outside world.

In this step, you'll define typed request and response contracts using Pydantic models, then create a FastAPI skeleton with two endpoints. These models guarantee that every request and response follows a strict shape, so your API is predictable and self-documenting.

In this step, get ready to:
  • Define Pydantic request and response models for the agent API.
  • Create the FastAPI app with a health check and query endpoint.
  • Test the skeleton with Swagger UI.
Define Pydantic models for the API contract

Before writing any endpoint logic, you need to define the exact shape of data that flows in and out of your API. Pydantic models act as contracts that validate incoming requests and serialize outgoing responses automatically.

  • In the VS Code Explorer sidebar, hover over your project folder name and click the New File icon.
  • Name the file schemas.py and press Enter.
  • Add the request model by pasting this code into schemas.py:
from pydantic import BaseModel, Field


class AgentQuery(BaseModel):
    query: str = Field(..., min_length=1, max_length=1000, description="The question or task for the agent")
    session_id: str | None = Field(default=None, description="Optional session ID for conversation memory")

What does this code do?

AgentQuery defines the shape of every request your API accepts. The query field is required, must be between 1 and 1000 characters, and holds the user's question.

The session_id field is optional (defaults to None). You'll use this later for conversation memory in the Secret Mission.

  • Below the AgentQuery class, add the response model:
class AgentResponse(BaseModel):
    answer: str = Field(..., description="The agent's response")
    tools_used: list[str] = Field(default_factory=list, description="List of tools the agent called")
    model: str = Field(default="llama-3.3-70b-versatile", description="The LLM model used")
    session_id: str | None = Field(default=None, description="Session ID if conversation memory is active")

What does this code do?

AgentResponse defines the shape of every response your API returns. It guarantees clients always get a structured JSON object with the agent's answer, the list of tools it used, and the model name.

The default_factory=list on tools_used means an empty list is created for each response by default. This avoids sharing a single list object between responses.

  • Save schemas.py.
  • Confirm the file exists by running:
ls schemas.py

You should see schemas.py listed in the output.

Don't see the file?

Make sure you saved the file in your project folder, not in a parent or child directory. The ls command only lists files in your current working directory.

help me find where my schemas.py file was saved

✔️ Awesome, I've got everything!

Great. Double check you've saved your file.

ⓧ I'd like to double check the full code

from pydantic import BaseModel, Field


class AgentQuery(BaseModel):
    query: str = Field(..., min_length=1, max_length=1000, description="The question or task for the agent")
    session_id: str | None = Field(default=None, description="Optional session ID for conversation memory")


class AgentResponse(BaseModel):
    answer: str = Field(..., description="The agent's response")
    tools_used: list[str] = Field(default_factory=list, description="List of tools the agent called")
    model: str = Field(default="llama-3.3-70b-versatile", description="The LLM model used")
    session_id: str | None = Field(default=None, description="Session ID if conversation memory is active")
Create the FastAPI app skeleton

With your data contracts defined, you can now create the actual API application. FastAPI uses the metadata you pass (title, description, version) to auto-generate interactive documentation.

  • In the VS Code Explorer sidebar, click the New File icon again.
  • Name the file main.py and press Enter.
  • Add the FastAPI app instance by pasting this code into main.py:
from fastapi import FastAPI

app = FastAPI(
    title="AI Agent API",
    description="A multi-tool AI agent powered by LangChain and Groq",
    version="1.0.0",
)

What does this code do?

The FastAPI() call creates your application instance. The title, description, and version parameters populate the auto-generated OpenAPI specification. This means your Swagger UI page will display these values as a header.

  • Below the app instance, add the two endpoint stubs:
@app.get("/health")
async def health_check():
    return {"status": "healthy"}


@app.post("/agent/query")
async def query_agent():
    pass

What does this code do?

The GET /health endpoint returns a simple JSON object confirming the server is alive. This is a standard pattern for health checks in production APIs.

The POST /agent/query endpoint is a placeholder stub. It does nothing yet (pass). You'll wire it up to the LangChain agent in a later step.

  • Save main.py.

You won't see a change yet. The next substep starts the server so you can verify everything works.

✔️ Awesome, I've got everything!

Great. Double check you've saved your file.

ⓧ I'd like to double check the full code

from fastapi import FastAPI

app = FastAPI(
    title="AI Agent API",
    description="A multi-tool AI agent powered by LangChain and Groq",
    version="1.0.0",
)


@app.get("/health")
async def health_check():
    return {"status": "healthy"}


@app.post("/agent/query")
async def query_agent():
    pass
Test the skeleton

FastAPI ships with a built-in development server called uvicorn. When you start it with the --reload flag, it automatically restarts whenever you save a file.

  • Start the development server by running this command in your terminal:
uvicorn main:app --reload

What does this command do?

uvicorn is the ASGI server that runs your FastAPI app. main:app tells it to look in the file main.py for the object named app. The --reload flag watches for file changes and restarts the server automatically.

  • Open your browser and go to http://localhost:8000/docs.

You should see the Swagger UI page with the title "AI Agent API" at the top. Both endpoints (GET /health and POST /agent/query) are listed.

  • Click the GET /health row to expand it.
  • Click Try it out.
  • Click Execute.

The response body should show {"status": "healthy"} with a 200 status code.

Server won't start?

Make sure your terminal is in the same directory as main.py. If you see a "module not found" error, run ls to confirm main.py is in your current directory.

If you see a port-in-use error, another process is already running on port 8000. Stop it or use a different port with --port 8001.

help me troubleshoot why uvicorn won't start my FastAPI app

Your API skeleton is live and serving interactive documentation. Next up, you'll build the LangChain tools and agent that bring the POST /agent/query endpoint to life.

Create LangChain Tools and Wire Up the Agent

Your API skeleton is live and serving typed endpoints. Now it needs a brain.

In this step, you will build the AI agent that powers the API. The agent uses LangChain to orchestrate tool calls and Groq's free Llama 3.3 70B model to reason about which tool to use for each query.

In this step, get ready to:
  • Define three custom tools (calculator, text analyzer, datetime utility).
  • Create a LangChain agent that autonomously selects and calls tools.
  • Test the agent standalone to confirm it picks the right tool for each query.
Define custom tools

LangChain tools are regular Python functions decorated with @tool. The decorator turns each function into something the agent can discover and call by name.

  • Create a new file called tools.py in your project folder.
  • Add the imports and the first tool, calculate, by pasting this code:
from langchain_core.tools import tool
from datetime import datetime


@tool
def calculate(expression: str) -> str:
    """Evaluate a mathematical expression and return the result. Use this for any math calculations."""
    # Only allow safe characters in the expression
    allowed_chars = set("0123456789+-*/.() ")
    if not all(c in allowed_chars for c in expression):
        return "Error: Invalid characters in expression. Only numbers and +-*/.() are allowed."
    try:
        result = eval(expression)
        return str(result)
    except Exception as e:
        return f"Error evaluating expression: {e}"

What does this code do?

The @tool decorator from langchain_core.tools registers calculate as a tool the agent can call. The docstring tells the agent when to use it.

The allowed_chars check blocks any dangerous input before it reaches eval(). Only digits and basic math operators are permitted.

  • Add the second tool, analyze_text, below the calculate function:
@tool
def analyze_text(text: str) -> str:
    """Analyze text and return statistics including word count, character count, and sentence count."""
    word_count = len(text.split())
    char_count = len(text)
    # Count sentence-ending punctuation
    sentence_count = text.count(".") + text.count("!") + text.count("?")
    if sentence_count == 0:
        sentence_count = 1
    return (
        f"Text Analysis:\n"
        f"- Words: {word_count}\n"
        f"- Characters: {char_count}\n"
        f"- Sentences: {sentence_count}"
    )

What does this code do?

This tool splits the input text to count words, measures total characters, and counts sentence-ending punctuation marks. If no punctuation is found, it defaults to 1 sentence so the result is always meaningful.

  • Add the third tool, get_current_datetime, below analyze_text:
@tool
def get_current_datetime(format: str = "%Y-%m-%d %H:%M:%S") -> str:
    """Get the current date and time. Optionally specify a format string."""
    try:
        return datetime.now().strftime(format)
    except ValueError as e:
        return f"Error with format string: {e}"

What does this code do?

This tool returns the current date and time using Python's datetime module. The caller can optionally pass a custom format string. If the format is invalid, the tool returns a helpful error instead of crashing.

  • Save tools.py.

Seeing an import error?

Make sure you installed langchain in the previous step. The langchain_core package is included automatically as a dependency.

Help me fix my LangChain import error in tools.py

✔️ Awesome, I've got everything!

Great. Double check you have saved tools.py with all three tools.

ⓧ I'd like to double check the full code

from langchain_core.tools import tool
from datetime import datetime


@tool
def calculate(expression: str) -> str:
    """Evaluate a mathematical expression and return the result. Use this for any math calculations."""
    allowed_chars = set("0123456789+-*/.() ")
    if not all(c in allowed_chars for c in expression):
        return "Error: Invalid characters in expression. Only numbers and +-*/.() are allowed."
    try:
        result = eval(expression)
        return str(result)
    except Exception as e:
        return f"Error evaluating expression: {e}"


@tool
def analyze_text(text: str) -> str:
    """Analyze text and return statistics including word count, character count, and sentence count."""
    word_count = len(text.split())
    char_count = len(text)
    sentence_count = text.count(".") + text.count("!") + text.count("?")
    if sentence_count == 0:
        sentence_count = 1
    return (
        f"Text Analysis:\n"
        f"- Words: {word_count}\n"
        f"- Characters: {char_count}\n"
        f"- Sentences: {sentence_count}"
    )


@tool
def get_current_datetime(format: str = "%Y-%m-%d %H:%M:%S") -> str:
    """Get the current date and time. Optionally specify a format string."""
    try:
        return datetime.now().strftime(format)
    except ValueError as e:
        return f"Error with format string: {e}"
Create the agent with LangChain

The agent is the decision-maker. It receives a query, reads the tool descriptions, and decides which tool (if any) to call before composing its final answer.

  • Create a new file called agent.py in your project folder.
  • Add the imports and the get_agent function by pasting this code:
import os
from langchain.agents import create_agent
from langchain_groq import ChatGroq
from langchain_core.messages import HumanMessage
from tools import calculate, analyze_text, get_current_datetime


def get_agent():
    # Initialize the Groq LLM with Llama 3.3 70B
    llm = ChatGroq(
        model="llama-3.3-70b-versatile",
        temperature=0,
        api_key=os.environ.get("GROQ_API_KEY"),
    )
    # Create a LangChain agent with our three tools
    agent = create_agent(llm, tools=[calculate, analyze_text, get_current_datetime])
    return agent

What does this code do?

The get_agent function creates a fresh agent instance. It initializes ChatGroq with the llama-3.3-70b-versatile model and temperature=0 for deterministic responses.

The create_agent function from langchain.agents builds a ReAct-style agent that can reason about which tools to call. It takes the LLM and a list of tools, then returns a graph you can invoke with messages.

  • Add the run_agent async function below get_agent:
async def run_agent(query: str, message_history: list | None = None) -> dict:
    agent = get_agent()
    messages = []
    # Prepend any existing conversation history
    if message_history:
        messages.extend(message_history)
    # Add the current user query
    messages.append(HumanMessage(content=query))

    # Invoke the agent asynchronously
    response = await agent.ainvoke({"messages": messages})

    # Extract the final answer from the last message
    agent_messages = response["messages"]
    final_message = agent_messages[-1]

    # Collect names of any tools the agent called
    tools_used = []
    for msg in agent_messages:
        if hasattr(msg, "tool_calls") and msg.tool_calls:
            for tool_call in msg.tool_calls:
                tools_used.append(tool_call["name"])

    return {
        "answer": final_message.content,
        "tools_used": tools_used,
        "all_messages": agent_messages,
    }

What does this code do?

The run_agent function is the main entry point for querying the agent. It builds a message list (optionally including history), invokes the agent, and parses the response.

The loop at the end inspects each message for tool_calls to build a list of which tools were used. This information powers the tools_used field in the API response.

  • Save agent.py.

Import error for create_agent?

If you see an error about create_agent not being found, double check that you installed langchain==1.3.8. The create_agent function was introduced in LangChain v1. Older versions do not have it.

Help me resolve my create_agent import error

✔️ Awesome, I've got everything!

Great. Double check you have saved agent.py with both functions.

ⓧ I'd like to double check the full code

import os
from langchain.agents import create_agent
from langchain_groq import ChatGroq
from langchain_core.messages import HumanMessage
from tools import calculate, analyze_text, get_current_datetime


def get_agent():
    llm = ChatGroq(
        model="llama-3.3-70b-versatile",
        temperature=0,
        api_key=os.environ.get("GROQ_API_KEY"),
    )
    agent = create_agent(llm, tools=[calculate, analyze_text, get_current_datetime])
    return agent


async def run_agent(query: str, message_history: list | None = None) -> dict:
    agent = get_agent()
    messages = []
    if message_history:
        messages.extend(message_history)
    messages.append(HumanMessage(content=query))

    response = await agent.ainvoke({"messages": messages})

    agent_messages = response["messages"]
    final_message = agent_messages[-1]

    tools_used = []
    for msg in agent_messages:
        if hasattr(msg, "tool_calls") and msg.tool_calls:
            for tool_call in msg.tool_calls:
                tools_used.append(tool_call["name"])

    return {
        "answer": final_message.content,
        "tools_used": tools_used,
        "all_messages": agent_messages,
    }
Test the agent standalone

Before wiring the agent into the API, you will test it directly to confirm it picks the correct tools.

  • Open a new terminal window. Your uvicorn server is still running in the first terminal, so you need a second one.
  • Make sure you are in your project folder and your virtual environment is active.
  • Create a test script called test_agent.py with this code:
import asyncio
from agent import run_agent


async def main():
    # Test 1: Math query should call the calculate tool
    print("Test 1: Math query")
    result = await run_agent("What is 25 * 17 + 3?")
    print(f"Answer: {result['answer']}")
    print(f"Tools used: {result['tools_used']}")
    print()

    # Test 2: Text analysis should call the analyze_text tool
    print("Test 2: Text analysis")
    result = await run_agent("Analyze the text: Hello world")
    print(f"Answer: {result['answer']}")
    print(f"Tools used: {result['tools_used']}")


asyncio.run(main())
  • Run the test script:
python test_agent.py
  • You should see the agent return a numeric answer for Test 1 with calculate in the tools used list.
  • Test 2 should show text statistics with analyze_text in the tools used list.

What should I see?

Test 1 should output an answer of 428 (25 * 17 + 3) and list calculate as the tool used. Test 2 should output word count, character count, and sentence count statistics, listing analyze_text as the tool used.

Getting a connection or authentication error?

Make sure your GROQ_API_KEY environment variable is set in this terminal session. If you opened a new terminal, you need to set it again.

Also confirm you are inside your virtual environment (you should see the environment name in your terminal prompt).

Help me debug my agent test script

Your agent is alive and making smart tool choices. Next up, you will wire it into the FastAPI endpoints so external clients can query it over HTTP.

Connect Agent to API with Error Handling

In the previous step, you confirmed the agent can call the right tools and return answers. That's great for a standalone script, but useless to the outside world.

In this step, you'll wire the agent into your FastAPI endpoints so any HTTP client can send a query and receive a validated JSON response. You'll also add proper error handling so failed requests return meaningful status codes instead of crashing the server.

In this step, get ready to:
  • Connect the agent to the POST endpoint with structured responses.
  • Add error handling for failed requests and session cleanup.
  • Test the full API end-to-end through Swagger UI.
Integrate the agent into the POST endpoint

Right now, main.py has a stub query_agent function that does nothing. You need to replace the file's contents so the endpoint accepts a typed request body, calls the agent, stores conversation history, and returns a validated response.

  • In VS Code, open main.py and replace its entire contents with the following code:
import uuid
from fastapi import FastAPI, HTTPException
from schemas import AgentQuery, AgentResponse
from agent import run_agent
from langchain_core.messages import HumanMessage, AIMessage

app = FastAPI(
    title="AI Agent API",
    description="A multi-tool AI agent powered by LangChain and Groq",
    version="1.0.0",
)

# In-memory storage for conversation sessions
sessions: dict[str, list] = {}


@app.get("/health")
async def health_check():
    return {"status": "healthy"}


@app.post("/agent/query", response_model=AgentResponse)
async def query_agent(request: AgentQuery):
    # Generate a session ID if the client didn't provide one
    session_id = request.session_id or str(uuid.uuid4())
    message_history = sessions.get(session_id, [])

    try:
        result = await run_agent(query=request.query, message_history=message_history)
    except Exception as e:
        raise HTTPException(status_code=500, detail=f"Agent error: {str(e)}")

    # Save conversation history for this session
    sessions[session_id] = message_history + [
        HumanMessage(content=request.query),
        AIMessage(content=result["answer"]),
    ]

    return AgentResponse(
        answer=result["answer"],
        tools_used=result["tools_used"],
        model="llama-3.3-70b-versatile",
        session_id=session_id,
    )

What does this code do?

The imports bring in your Pydantic models (AgentQuery, AgentResponse), the run_agent function from the previous step, and LangChain's message types for tracking conversation history.

The sessions dictionary stores message history keyed by session ID. This enables multi-turn conversations where the agent remembers what was said earlier.

The response_model=AgentResponse parameter tells FastAPI to validate the return value against your Pydantic schema before sending it to the client. If anything is missing or the wrong type, FastAPI catches it before the response leaves the server.

  • Save main.py.
  • Switch back to your terminal. Because you started uvicorn with --reload, it should automatically pick up the changes. Look for a line like WARNING: WatchFiles detected changes in 'main.py'. Reloading... followed by Application startup complete.

Seeing an import error in the terminal?

If the terminal shows ModuleNotFoundError, make sure your virtual environment is still active (you should see the (venv) prefix in your terminal prompt).

If you see a SyntaxError, double-check you replaced the entire file contents and didn't accidentally leave the old stub code at the bottom.

help me fix a uvicorn reload error after updating main.py

Add error handling and session cleanup

A production API needs to handle the case where someone wants to reset their conversation. You'll add a DELETE endpoint that clears a session's history. This endpoint also demonstrates a different error handling pattern: returning a 404 when the requested resource doesn't exist.

  • In main.py, add the following code below the query_agent function (after the closing parenthesis of the return AgentResponse(...) block):
@app.delete("/agent/sessions/{session_id}")
async def clear_session(session_id: str):
    if session_id in sessions:
        del sessions[session_id]
        return {"status": "session cleared"}
    raise HTTPException(status_code=404, detail="Session not found")

How does error handling work across this API?

Your API now uses three distinct HTTP error patterns. The 422 Unprocessable Entity response fires automatically when a request fails Pydantic validation (missing query field, empty string, over 1000 characters). You wrote zero code for this.

The 500 Internal Server Error fires when the agent itself crashes (network timeout to Groq, rate limit exceeded, malformed model response). The try/except catches any exception and wraps it in a clean error message.

The 404 Not Found fires when a client tries to delete a session that doesn't exist. This is standard REST: if the resource isn't there, return 404.

Groq free tier rate limits

The Groq free tier allows 30 requests per minute. If you send queries too quickly during testing, the agent will fail and your try/except will return a 500 error with the rate limit message. Wait a minute and try again.

  • Save main.py.
  • Confirm uvicorn reloads without errors in the terminal.

Server not reloading?

If you don't see a reload message, try saving the file again. If uvicorn still doesn't react, stop the server with Ctrl+C and restart it manually.

help me fix uvicorn not reloading when I save main.py

✔️ Awesome, I've got everything!

Great. Double-check you've saved main.py and the server reloaded cleanly.

ⓧ I'd like to double check the full code

Here's the complete main.py at this point:

import uuid
from fastapi import FastAPI, HTTPException
from schemas import AgentQuery, AgentResponse
from agent import run_agent
from langchain_core.messages import HumanMessage, AIMessage

app = FastAPI(
    title="AI Agent API",
    description="A multi-tool AI agent powered by LangChain and Groq",
    version="1.0.0",
)

sessions: dict[str, list] = {}


@app.get("/health")
async def health_check():
    return {"status": "healthy"}


@app.post("/agent/query", response_model=AgentResponse)
async def query_agent(request: AgentQuery):
    session_id = request.session_id or str(uuid.uuid4())
    message_history = sessions.get(session_id, [])

    try:
        result = await run_agent(query=request.query, message_history=message_history)
    except Exception as e:
        raise HTTPException(status_code=500, detail=f"Agent error: {str(e)}")

    sessions[session_id] = message_history + [
        HumanMessage(content=request.query),
        AIMessage(content=result["answer"]),
    ]

    return AgentResponse(
        answer=result["answer"],
        tools_used=result["tools_used"],
        model="llama-3.3-70b-versatile",
        session_id=session_id,
    )


@app.delete("/agent/sessions/{session_id}")
async def clear_session(session_id: str):
    if session_id in sessions:
        del sessions[session_id]
        return {"status": "session cleared"}
    raise HTTPException(status_code=404, detail="Session not found")
Test the full API end-to-end

Your agent is now fully wired into the API. Time to prove it works by sending real queries through Swagger UI and checking the structured responses.

  • Open http://localhost:8000/docs in your browser.
  • Click on the POST /agent/query endpoint to expand it.
  • Click Try it out.
  • Replace the request body with this math query:
{
  "query": "What is 25 * 17 + 3?"
}
  • Click Execute.

You should see a 200 response with a JSON body that includes an answer field containing the calculation result, tools_used containing ["calculate"], and model set to llama-3.3-70b-versatile.

  • Now test a text analysis query. Replace the request body with:
{
  "query": "Analyze the text: FastAPI makes building APIs easy and fun"
}
  • Click Execute.

This time, tools_used should contain ["analyze_text"] and the answer should include word count, character count, and sentence count.

  • Test the validation by sending an empty query. Replace the request body with:
{
  "query": ""
}
  • Click Execute.

You should see a 422 Unprocessable Entity response. Pydantic rejected the empty string because your AgentQuery model requires min_length=1.

What should I see?

Successful queries return a 200 response matching the AgentResponse schema exactly: answer, tools_used, model, and session_id. No extra fields, no missing fields.

Invalid requests return 422 with a detailed error explaining which field failed and why. Agent failures (if Groq is down or rate-limited) return 500 with a human-readable detail message.

Getting a 500 error on valid queries?

Check that your GROQ_API_KEY environment variable is still set in the terminal where uvicorn is running. If you restarted your terminal, you'll need to set it again.

If you see a rate limit message in the error detail, wait 60 seconds before trying again. The Groq free tier allows 30 requests per minute.

help me debug a 500 error from my FastAPI agent endpoint

Your AI agent API is fully operational. Every query gets a validated, structured JSON response. Every error gets a proper HTTP status code. Next up, you'll push this further by giving your agent memory so it can handle multi-turn conversations.

Secret mission

Add Conversation Memory

Your agent handles one-off queries perfectly. But what if a user wants to have a back-and-forth conversation? Right now they'd have to repeat all context every time. In this mission, you'll explore and verify the session-based memory system already wired into your API, proving the agent can remember previous messages across multiple requests.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally and uses Groq's free tier, so there are no ongoing costs.

Resources you used:

  • Local project folder with Python virtual environment and installed packages.
  • Groq API key (free tier, no charges).

Keep everything running

No action needed. Choose this if you're still actively building or want to keep the project ready for portfolio demos.

  • Your project folder stays on your Desktop.
  • Your Groq API key remains active on the free tier with no charges.
  • To restart the server later, activate your virtual environment and run uvicorn main:app --reload.

Pause - I'll come back to this later

Shut down the running server and deactivate your virtual environment to free up terminal resources. Your files stay intact.

  • Press Ctrl+C in the terminal where uvicorn is running to stop the server.
  • Deactivate your virtual environment by running:
deactivate
  • To resume later, navigate back to your project folder, reactivate the virtual environment, and start the server again.

Delete - I don't want to use this again

Remove all project files and optionally revoke your Groq API key. This cannot be undone.

  • Press Ctrl+C in the terminal to stop the server if it is still running.
  • Deactivate your virtual environment by running:
deactivate
  • Navigate to your Desktop and delete the project folder.

macOS

  • Remove the project folder by running:
cd ~/Desktop
rm -rf ai-agent-api

Windows

  • Remove the project folder by running:
cd ~/Desktop
rmdir /s /q ai-agent-api
  • To revoke your Groq API key, go to console.groq.com.
  • Click API Keys in the left sidebar.
  • Find your key and click the delete icon to revoke it.

Nice Work!

Nice Work!

You just built a production-grade AI agent API from scratch. Your FastAPI service receives natural language queries, routes them through a LangChain agent powered by Groq's free Llama 3.3 70B model, and returns structured JSON responses with full Swagger documentation.

You've learned how to:

  • Build a FastAPI service with typed Pydantic request and response models that auto-generate interactive Swagger documentation.
  • Create a LangChain v1 agent that autonomously selects and calls custom tools (calculator, text analyzer, datetime) using Groq's free Llama 3.3 70B model.
  • Implement structured JSON responses with proper HTTP error handling and status codes for a production-ready API.
  • Secret Mission: Added session-based conversation memory so the agent remembers context across multiple requests in the same session.

Ready to quiz yourself?