Build an AI Agent API with LangChain

Build a FastAPI server that exposes a LangChain AI agent with custom tools.

Introduction

30 Second Summary

Behind every AI chatbot, coding assistant, and smart search tool is an API that takes a question, sends it to an AI model, and returns a useful answer. Building that API yourself means you control what the AI can do, how it responds, and which tools it has access to.

In this project, you will build a REST API that connects clients to an intelligent AI agent. The agent can reason about problems and call tools (like a calculator or a clock) to give accurate answers.

What You'll Build

A FastAPI application that serves a LangChain agent with custom tools, exposing chat and streaming endpoints for real-time AI responses.

By the end of this project, you'll have:

  • A FastAPI server with health check, chat, and streaming endpoints that clients can call.
  • A LangChain agent powered by Google Gemini with custom tools (a calculator and a current-time tool) that reasons about which tool to use.
  • Auto-generated interactive API documentation (Swagger UI) for testing your endpoints directly in the browser.
  • Secret Mission: A conversation memory system so the agent remembers previous messages in a session.

Are there any prerequisites?

You should be comfortable with basic Python (functions, classes, and async/await). Familiarity with REST APIs is helpful but not required.

Before We Start

Every AI-powered product you interact with has a backend API connecting a language model to the outside world. In this project, you'll build that backend yourself using LangChain and FastAPI.

Before writing any application code, you need a working Python environment with the right packages installed. You also need an API key to connect to Google's Gemini model.

In this step, get ready to:
  • Install Python 3.10+ and set up a project directory with a virtual environment.
  • Install project dependencies (FastAPI, LangChain, and more).
  • Get a free Google Gemini API key.
Install Python and create your project directory

This project requires Python 3.10 or higher. Let's confirm you have the right version installed.

  • Open your terminal. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.
  • Check your Python version by running this command:
python3 --version

✔️ I see version 3.10 or higher

You're all set. Your output looks something like Python 3.10.x or higher. Move on to the next instruction below.

ⓧ I see an older version

You have Python installed, but the version is below 3.10. FastAPI and LangChain both require 3.10 or higher, so you need to upgrade.

  • Visit python.org/downloads and download the latest Python installer for your operating system.
  • Run the installer. On Windows, make sure to check the box that says Add Python to PATH before clicking Install.
  • After installation, close and reopen your terminal, then run python3 --version again.

Windows users: python3 not recognized?

On Windows, the python3 command may not be available. Try python --version or py --version instead. If the output shows 3.10 or higher, you're good.

ⓧ Command not found

Python is not installed on your machine yet.

  • Visit python.org/downloads and download the latest Python installer for your operating system.
  • Run the installer. On Windows, make sure to check the box that says Add Python to PATH before clicking Install.
  • After installation, close and reopen your terminal, then run python3 --version again to confirm it shows 3.10 or higher.

Windows users: python3 not recognized?

On Windows, try python --version or py --version instead. The Python installer for Windows registers the python and py commands rather than python3.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir ai-agent-api
cd ai-agent-api
  • Create a Python virtual environment by running:
python3 -m venv venv

What is a virtual environment?

A virtual environment is an isolated Python workspace. It keeps the packages you install for this project separate from your system-wide Python installation.

This prevents version conflicts between different projects on your machine.

  • Activate the virtual environment by running:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

  • Make sure you ran the activation command for your specific operating system (macOS vs Windows).
  • On Windows, if you see a permissions error, try running PowerShell as Administrator and executing Set-ExecutionPolicy RemoteSigned first, then try the activation command again.
  • Confirm the venv folder exists by running ls (macOS) or dir (Windows) in your ai-agent-api folder.

Still stuck? Help me activate my Python virtual environment.

Install project dependencies
  • Install all the packages your project needs by running:
pip install "fastapi[standard]" langchain langchain-google-genai python-dotenv

What do these packages do?

  • fastapi[standard] is the web framework that serves your API. The [standard] extra bundles Uvicorn (the server) and the FastAPI CLI.
  • langchain is the orchestration framework that builds and runs AI agents with tools.
  • langchain-google-genai connects LangChain to Google's Gemini model.
  • python-dotenv loads environment variables from a .env file so you don't hard-code secrets.
  • Wait for the installation to complete. You should see output ending with a line that says Successfully installed followed by a list of package names.

Installation failed?

  • Make sure your virtual environment is active. You should see (venv) in your terminal prompt. If not, re-run the activation command from the previous substep.
  • Check your internet connection. pip needs to download packages from the internet.
  • If you see a permissions error, never use sudo pip install. Instead, ensure you activated the virtual environment first.

Still stuck? Help me fix my pip install error.

Get your Google Gemini API key

Google's Gemini API has a free tier with no credit card required. You'll generate an API key from Google AI Studio and store it in a local file.

  • Open https://aistudio.google.com/apikey in your browser.
  • Sign in with your Google account if prompted.
  • Click Create API Key.
  • If asked to select a project, choose the default project that Google AI Studio created for you.
  • Copy the generated API key and keep it somewhere safe. You will not be able to view it again after closing the dialog.

Why is this free?

Google Gemini offers an ongoing free tier with rate limits. No credit card is required to start. For a learning project like this, you will stay well within the free tier limits.

Now you need to store your API key in a file that your Python code can read.

  • Open your ai-agent-api folder in a code editor (Cursor or VS Code recommended). Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar, type the editor name, and press Enter.
  • In your editor, open the ai-agent-api folder from your Desktop.
  • Create a new file called .env in the root of your ai-agent-api folder.
  • Add the following content to .env, replacing your-api-key-here with the actual key you copied:
GOOGLE_API_KEY=your-api-key-here

What does this file do?

The .env file stores sensitive configuration like API keys outside of your code. The python-dotenv package reads this file and makes the values available as environment variables in your Python scripts.

This keeps your secret key out of your source code. Never commit .env files to version control.

  • Save .env.

✔️ Awesome, I've got everything!

Great. Double check you've saved your .env file with your real API key in place of your-api-key-here.

ⓧ I'd like to double check the full code

Here is what your .env file should look like (with your actual key substituted):

GOOGLE_API_KEY=your-api-key-here

Having trouble getting an API key?

  • Make sure you are signed into a personal Google account. Organization-managed accounts may have restrictions that block API key creation.
  • If you see a permissions error, try creating a new Google Cloud project from the same page and generating the key within that project.

Still stuck? Help me get a Gemini API key from Google AI Studio.

Your environment is ready. Next up, you'll build the LangChain agent that powers your API's intelligence.

Build the LangChain Agent with Custom Tools

Your environment is ready with all the dependencies installed and your API key configured. Now it's time to build the brain of your application.

A LangChain agent is an AI that can reason about problems and decide which tools to call. Instead of just generating text, it can take actions like running calculations or checking the current time. In this step, you'll create custom tools and wire them to the Gemini model.

In this step, get ready to:
  • Create custom tools that the agent can call.
  • Build an agent that connects Gemini to your tools.
  • Test the agent with a question that requires tool use.
Create custom tools in tools.py

Tools are Python functions that the agent can call when it needs to perform a specific action. LangChain's @tool decorator turns any function into a tool the agent understands. The agent reads each tool's docstring to decide when to use it.

  • Create a new file called tools.py in your project folder.
  • Add the imports and the calculator tool by pasting this code:
from langchain_core.tools import tool
from datetime import datetime


@tool
def calculator(expression: str) -> str:
    """Evaluate a mathematical expression. Use this for any math calculations.
    
    Args:
        expression: A valid Python math expression (e.g., '247 * 38', '(15 + 7) / 2')
    """
    try:
        # Only allow safe characters (digits and math operators)
        allowed_chars = set("0123456789+-*/.() ")
        if not all(c in allowed_chars for c in expression):
            return "Error: Invalid characters in expression. Only numbers and basic math operators allowed."
        result = eval(expression)
        return str(result)
    except Exception as e:
        return f"Error evaluating expression: {e}"

What does this code do?

The @tool decorator registers calculator as a LangChain tool. The agent reads the docstring to understand when to call it.

The allowed_chars set restricts eval() to only process numbers and basic math operators. This prevents arbitrary code execution by rejecting anything that isn't a safe math expression.

You won't see any output yet. The next chunk adds a second tool to this file.

  • Below the calculator function, add the current-time tool:


@tool
def get_current_time() -> str:
    """Get the current date and time. Use this when the user asks about the current time or date."""
    now = datetime.now()
    return now.strftime("%Y-%m-%d %H:%M:%S")

What does this code do?

The get_current_time tool returns the current date and time as a formatted string. The docstring tells the agent exactly when to use it.

Notice this tool takes no arguments. It simply checks the system clock and returns the result.

  • Save tools.py.
  • Confirm the file exists by running:
ls tools.py

You should see tools.py listed in the output.

Don't see tools.py?

Make sure you created the file inside your project folder, not somewhere else on your system. Run pwd to check which directory your terminal is in.

help me find where I created tools.py

✔️ Awesome, I've got everything!

Great. Double check you've saved tools.py before moving on.

ⓧ I'd like to double check the full code

from langchain_core.tools import tool
from datetime import datetime


@tool
def calculator(expression: str) -> str:
    """Evaluate a mathematical expression. Use this for any math calculations.
    
    Args:
        expression: A valid Python math expression (e.g., '247 * 38', '(15 + 7) / 2')
    """
    try:
        allowed_chars = set("0123456789+-*/.() ")
        if not all(c in allowed_chars for c in expression):
            return "Error: Invalid characters in expression. Only numbers and basic math operators allowed."
        result = eval(expression)
        return str(result)
    except Exception as e:
        return f"Error evaluating expression: {e}"


@tool
def get_current_time() -> str:
    """Get the current date and time. Use this when the user asks about the current time or date."""
    now = datetime.now()
    return now.strftime("%Y-%m-%d %H:%M:%S")
Create the agent in agent.py

The agent is the orchestrator that connects the Gemini LLM to your tools. It receives a user message, decides whether a tool is needed, calls the tool, and returns a final answer.

  • Create a new file called agent.py in your project folder.
  • Add the agent setup code by pasting the following:
import os
from dotenv import load_dotenv
from langchain_google_genai import ChatGoogleGenerativeAI
from langchain.agents import create_agent
from tools import calculator, get_current_time

# Load environment variables from .env
load_dotenv()

def build_agent():
    # Initialize the Gemini model
    llm = ChatGoogleGenerativeAI(
        model="gemini-2.0-flash",
        google_api_key=os.getenv("GOOGLE_API_KEY"),
    )

    # Create the agent, wiring the model to our custom tools
    agent = create_agent(
        model=llm,
        tools=[calculator, get_current_time],
        system_prompt="You are a helpful assistant. Use the available tools when needed to answer questions accurately. Always explain your reasoning.",
    )

    return agent

What does this code do?

load_dotenv() reads your .env file so GOOGLE_API_KEY is available as an environment variable.

ChatGoogleGenerativeAI creates a connection to Google's Gemini model. The gemini-2.0-flash model is fast and available on the free tier.

create_agent wires everything together. It takes the model, a list of tools, and a system prompt that guides how the agent behaves.

  • Save agent.py.

You won't see any output yet. The next substep adds a test block so you can run the agent.

Seeing a red underline on the imports?

Make sure your virtual environment is activated. If you're using Cursor or VS Code, select the Python interpreter from your virtual environment by pressing Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows) and choosing Python: Select Interpreter.

help me fix import errors in agent.py

Test the agent locally

The agent is built, but you haven't run it yet. Adding a test block at the bottom of agent.py lets you invoke the agent directly from the terminal.

  • In agent.py, add the following test block at the very bottom of the file (below the build_agent function):


if __name__ == "__main__":
    # Test the agent with a math question
    agent = build_agent()
    result = agent.invoke({"messages": [{"role": "user", "content": "What is 247 * 38?"}]})
    print(result["messages"][-1].content)

What does this code do?

The if __name__ == "__main__": block only runs when you execute the file directly (not when it's imported by another file).

It builds the agent, sends it a math question, and prints the final response. The agent should recognize it needs the calculator tool and use it to compute 247 * 38.

  • Save agent.py.
  • Run the agent by executing this command:
python3 agent.py

You should see a response that includes the answer 9386 (the result of 247 * 38). The agent may also include an explanation of how it arrived at the answer.

Seeing an API key error?

Check that your .env file contains GOOGLE_API_KEY=your-actual-key (with your real key, not the placeholder). Also confirm the .env file is in the same folder as agent.py.

If you see a ModuleNotFoundError, make sure your virtual environment is activated before running the command.

help me debug my agent.py errors

What is tool-calling?

When the agent receives "What is 247 * 38?", it doesn't try to do math in its head. It recognizes that the calculator tool is designed for math and delegates the work.

This is the power of tool-calling. The LLM reasons about which tool fits the question, calls it with the right arguments, and incorporates the result into its response.

✔️ Awesome, I've got everything!

Great. Double check you've saved agent.py before moving on.

ⓧ I'd like to double check the full code

import os
from dotenv import load_dotenv
from langchain_google_genai import ChatGoogleGenerativeAI
from langchain.agents import create_agent
from tools import calculator, get_current_time

load_dotenv()

def build_agent():
    llm = ChatGoogleGenerativeAI(
        model="gemini-2.0-flash",
        google_api_key=os.getenv("GOOGLE_API_KEY"),
    )

    agent = create_agent(
        model=llm,
        tools=[calculator, get_current_time],
        system_prompt="You are a helpful assistant. Use the available tools when needed to answer questions accurately. Always explain your reasoning.",
    )

    return agent


if __name__ == "__main__":
    agent = build_agent()
    result = agent.invoke({"messages": [{"role": "user", "content": "What is 247 * 38?"}]})
    print(result["messages"][-1].content)

Your agent is working and can answer questions using real tools. Next up, you'll wrap this agent in a FastAPI application so any client can talk to it over HTTP.

Create the FastAPI Application

Your agent can reason about questions and use tools. Now you need a way for other programs to talk to it.

In this step, you will wrap your LangChain agent in a FastAPI application that exposes it as a REST API. Clients will send questions via HTTP and receive structured JSON responses.

In this step, get ready to:
  • Define typed request and response models with Pydantic.
  • Build a FastAPI app with a health-check endpoint and a chat endpoint.
  • Run the dev server and test your API through auto-generated interactive docs.
Create the request and response schemas

Before building the API itself, you need to define the shape of the data flowing in and out. Pydantic models act as contracts that validate every request automatically.

  • Create a new file called schemas.py in your project folder and add the following code:
from pydantic import BaseModel


# Define the shape of incoming chat requests
class ChatRequest(BaseModel):
    message: str


# Define the shape of outgoing chat responses
class ChatResponse(BaseModel):
    response: str
    tools_used: list[str]

What does this code do?

ChatRequest accepts a single message string from the client. FastAPI will reject any request that doesn't include this field.

ChatResponse sends back the AI's response text plus a tools_used list showing which tools the agent called. This gives callers transparency into the agent's reasoning process.

  • Save schemas.py.
  • Confirm the file exists by running:
ls schemas.py

You should see schemas.py listed in the output.

File not showing up?

  • Make sure you saved the file in your project folder (the same folder containing agent.py and tools.py).
  • Check the filename is exactly schemas.py with no typos or extra extensions.

Why isn't my schemas.py file appearing?

✔️ Awesome, I've got everything!

Great. Make sure you've saved schemas.py before moving on.

ⓧ I'd like to double check the full code

from pydantic import BaseModel


class ChatRequest(BaseModel):
    message: str


class ChatResponse(BaseModel):
    response: str
    tools_used: list[str]
Build the FastAPI app with a health-check endpoint

Now you will create main.py. This file initializes the FastAPI application, loads your agent, and defines the endpoints clients can call.

  • Create a new file called main.py in your project folder and add the following code:
import asyncio
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from schemas import ChatRequest, ChatResponse
from agent import build_agent

# Initialize the FastAPI application with metadata
app = FastAPI(
    title="AI Agent API",
    description="A LangChain-powered AI agent served via FastAPI",
)

# Build the agent once at startup so every request reuses it
agent = build_agent()


@app.get("/")
async def health_check():
    return {"status": "healthy", "agent": "LangChain + Gemini"}

What does this code do?

  • The FastAPI() constructor creates the app object. The title and description appear in the auto-generated API docs.
  • build_agent() is called once at module level. This means the LLM connection and tool wiring happen at server startup, not on every request.
  • The GET / health-check endpoint returns a simple JSON object confirming the server is running.
  • StreamingResponse and asyncio are imported here because you will use them in the next step for streaming responses.
  • Save main.py. You won't see a change yet. The chat endpoint in the next chunk completes the API.
Add the chat endpoint

The chat endpoint is where the real work happens. It receives a message, passes it to the agent, and extracts which tools the agent decided to use.

  • In main.py, add the following code below the health_check function:
@app.post("/chat", response_model=ChatResponse)
async def chat(request: ChatRequest):
    # Invoke the agent asynchronously with the user's message
    result = await agent.ainvoke(
        {"messages": [{"role": "user", "content": request.message}]}
    )

    # Extract which tools the agent called during reasoning
    tools_used = []
    for message in result["messages"]:
        if hasattr(message, "tool_calls") and message.tool_calls:
            for tool_call in message.tool_calls:
                tools_used.append(tool_call["name"])

    # Get the final AI response text
    ai_response = result["messages"][-1].content

    return ChatResponse(response=ai_response, tools_used=tools_used)

What does this code do?

  • The @app.post("/chat") decorator registers this function as a POST endpoint. The response_model=ChatResponse ensures the output matches your Pydantic schema.
  • agent.ainvoke() calls the agent asynchronously so the server can handle other requests while waiting for the LLM.
  • The loop inspects each message in the agent's response chain to find any tool_calls. This reveals whether the agent used the calculator, the time tool, or neither.
  • result["messages"][-1].content grabs the final message in the chain, which is always the agent's answer to the user.
  • Save main.py. You will test both endpoints in the next substep when you start the server.

Seeing a red underline on the imports?

  • Make sure schemas.py is saved in the same folder as main.py. Your editor resolves imports relative to the project root.
  • Confirm your virtual environment is active. Your editor's Python interpreter should point to the venv where you installed fastapi and langchain.

Why is my editor showing import errors for schemas or agent?

✔️ Awesome, I've got everything!

Great. Double check you've saved main.py with both the health-check and chat endpoints.

ⓧ I'd like to double check the full code

import asyncio
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from schemas import ChatRequest, ChatResponse
from agent import build_agent

app = FastAPI(
    title="AI Agent API",
    description="A LangChain-powered AI agent served via FastAPI",
)

agent = build_agent()


@app.get("/")
async def health_check():
    return {"status": "healthy", "agent": "LangChain + Gemini"}


@app.post("/chat", response_model=ChatResponse)
async def chat(request: ChatRequest):
    result = await agent.ainvoke(
        {"messages": [{"role": "user", "content": request.message}]}
    )

    tools_used = []
    for message in result["messages"]:
        if hasattr(message, "tool_calls") and message.tool_calls:
            for tool_call in message.tool_calls:
                tools_used.append(tool_call["name"])

    ai_response = result["messages"][-1].content

    return ChatResponse(response=ai_response, tools_used=tools_used)
Run the server and test with Swagger UI

FastAPI comes with a built-in interactive documentation page called Swagger UI. You can use it to send requests to your API directly from the browser.

  • Start the development server by running:
fastapi dev main.py

What does this command do?

fastapi dev auto-discovers the FastAPI app in main.py and starts a Uvicorn server with auto-reload enabled. Any time you save a file, the server restarts automatically.

  • Open your browser and navigate to http://127.0.0.1:8000/docs.

You should see an interactive documentation page listing your two endpoints: GET / and POST /chat.

  • Click the GET / endpoint to expand it.
  • Click Try it out, then click Execute.

You should see a response body containing {"status": "healthy", "agent": "LangChain + Gemini"}.

  • Now click the POST /chat endpoint to expand it.
  • Click Try it out.
  • Replace the request body with:
{
  "message": "What is 247 * 38?"
}
  • Click Execute.

After a few seconds, you should see a JSON response with the agent's answer and "tools_used": ["calculator"] showing the agent chose the calculator tool.

Seeing a 500 Internal Server Error?

  • Check your terminal for the full error traceback. A missing or invalid GOOGLE_API_KEY in your .env file is the most common cause.
  • Make sure your .env file is in the same directory where you started the server.
  • If you see an import error, confirm that schemas.py, agent.py, and tools.py are all in the same folder as main.py.

Why is my FastAPI chat endpoint returning a 500 error?

Why FastAPI for serving an AI agent?

FastAPI is async-native, which matters when your endpoint spends most of its time waiting for an LLM response over the network. While one request waits, the server can handle other requests without blocking.

It also generates typed API documentation from your Pydantic models automatically. Clients calling your API get a contract they can rely on without you writing separate docs.

Your API is live and responding to questions. Next up, you will add a streaming endpoint so clients can receive tokens in real-time instead of waiting for the full response.

Add Streaming Responses

Your API now accepts questions and returns complete answers. That works, but real AI products feel snappy because they stream tokens to the user as the model generates them.

In this step, you'll add a streaming endpoint that delivers the agent's response token-by-token using Server-Sent Events (SSE). Instead of waiting for the full response, clients see text appear in real-time.

In this step, get ready to:
  • Add a streaming chat endpoint that sends tokens as Server-Sent Events.
  • Build an async generator that streams tokens from the LangChain agent.
  • Test real-time streaming with curl and watch tokens arrive live.
Add the streaming chat endpoint

Your existing /chat endpoint waits for the agent to finish its entire response before sending anything back. A streaming endpoint sends each token the moment the model produces it.

  • In main.py, add this new endpoint below the existing chat function:
@app.post("/chat/stream")
async def chat_stream(request: ChatRequest):
    async def event_generator():
        async for event in agent.astream_events(
            {"messages": [{"role": "user", "content": request.message}]},
            version="v2",
        ):
            if event["event"] == "on_chat_model_stream":
                chunk = event["data"]["chunk"]
                if chunk.content:
                    yield f"data: {chunk.content}\n\n"
        yield "data: [DONE]\n\n"

    return StreamingResponse(event_generator(), media_type="text/event-stream")

What does this code do?

This endpoint combines two ideas: an async generator that produces token chunks, and a StreamingResponse that sends each chunk to the client immediately.

  • event_generator() is an async generator defined inside the endpoint. It calls agent.astream_events() with version="v2" to get a stream of events from LangChain.
  • Events with event["event"] == "on_chat_model_stream" contain the actual token chunks from the LLM. Other events (tool calls, agent reasoning) are filtered out.
  • Each token is yielded as an SSE-formatted string: data: <content>\n\n. The double newline tells the client one event is complete.
  • After all tokens are sent, a final data: [DONE]\n\n signals the stream is finished.
  • StreamingResponse with media_type="text/event-stream" tells the client this is an SSE connection, not a regular JSON response.

Why streaming instead of waiting?

LLMs can take several seconds to produce a full response. Without streaming, the user stares at a blank screen the entire time. With streaming, the first token appears almost instantly, and the rest flows in naturally. This is the same technique ChatGPT and other AI products use to feel responsive.

  • Save main.py.
  • If your dev server is still running, it auto-reloads. If not, start it again with fastapi dev main.py.
  • Open http://127.0.0.1:8000/docs in your browser. You should see the new POST /chat/stream endpoint listed in the Swagger UI alongside your existing endpoints.

Don't see the new endpoint in Swagger?

  • Make sure you saved main.py. The dev server only reloads when the file is saved to disk.
  • Check your terminal for syntax errors. If you see a traceback, the indentation of the nested event_generator() function is the most common culprit.
  • Hard-refresh the Swagger page (Cmd+Shift+R on macOS, Ctrl+Shift+R on Windows) to clear cached docs.

Why isn't my new streaming endpoint showing up in Swagger UI?

Test streaming with curl

Swagger UI shows the endpoint exists, but it can't display a streaming response well. To see tokens arriving in real-time, you'll use curl with the --no-buffer flag, which disables output buffering so each chunk appears the instant it arrives.

  • Open a new terminal window (keep the FastAPI server running in the other one).
  • Test the streaming endpoint by running this command:
curl --no-buffer -X POST http://127.0.0.1:8000/chat/stream \
  -H "Content-Type: application/json" \
  -d '{"message": "What is 15 * 23?"}'

What does this command do?

  • --no-buffer disables curl's output buffering so you see each SSE event the moment the server sends it.
  • -X POST sends a POST request to the streaming endpoint.
  • -H "Content-Type: application/json" tells the server the request body is JSON.
  • -d sends the JSON body with your message. The agent will use the calculator tool to solve 15 * 23.

You should see text appearing in your terminal piece by piece, not all at once. Each line starts with data: followed by a token fragment. The final line reads data: [DONE].

Seeing all output at once instead of streaming?

  • Make sure you included the --no-buffer flag. Without it, curl buffers the entire response before displaying anything.
  • If you get a connection refused error, confirm your FastAPI dev server is still running in the other terminal window.
  • If you see an error about the GOOGLE_API_KEY, make sure your .env file is in the same directory where the server is running.

Why is my streaming endpoint not returning tokens in real-time?

✔️ Awesome, I've got everything!

Great. Double check you've saved main.py and that your server is running with no errors in the terminal.

ⓧ I'd like to double check the full code

Here is the complete main.py file with the streaming endpoint included:

import asyncio
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from schemas import ChatRequest, ChatResponse
from agent import build_agent

app = FastAPI(
    title="AI Agent API",
    description="A LangChain-powered AI agent served via FastAPI",
)

agent = build_agent()


@app.get("/")
async def health_check():
    return {"status": "healthy", "agent": "LangChain + Gemini"}


@app.post("/chat", response_model=ChatResponse)
async def chat(request: ChatRequest):
    result = await agent.ainvoke(
        {"messages": [{"role": "user", "content": request.message}]}
    )

    tools_used = []
    for message in result["messages"]:
        if hasattr(message, "tool_calls") and message.tool_calls:
            for tool_call in message.tool_calls:
                tools_used.append(tool_call["name"])

    ai_response = result["messages"][-1].content

    return ChatResponse(response=ai_response, tools_used=tools_used)


@app.post("/chat/stream")
async def chat_stream(request: ChatRequest):
    async def event_generator():
        async for event in agent.astream_events(
            {"messages": [{"role": "user", "content": request.message}]},
            version="v2",
        ):
            if event["event"] == "on_chat_model_stream":
                chunk = event["data"]["chunk"]
                if chunk.content:
                    yield f"data: {chunk.content}\n\n"
        yield "data: [DONE]\n\n"

    return StreamingResponse(event_generator(), media_type="text/event-stream")

Your API now has three endpoints: a health check, a standard chat endpoint, and a real-time streaming endpoint. Next up, you'll tackle the Secret Mission and give your agent a memory so it can hold multi-turn conversations.

Secret mission

Add Conversation Memory

Right now, every request to your agent is a blank slate. It has no idea what you asked five seconds ago. Build a session-based memory system so your agent can hold a real conversation across multiple messages.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally and uses the Google Gemini free tier, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Project folder with Python files (main.py, agent.py, tools.py, schemas.py, .env).
  • Python virtual environment with installed packages.
  • Google Gemini API key in Google AI Studio.

Keep everything running

No action needed. Choose this if you're still actively building or want to extend the project further.

  • Your project folder stays on your Desktop with all files intact.
  • Your virtual environment remains ready to activate.
  • Your Google Gemini API key stays active in Google AI Studio. The free tier has no charges, so leaving it active costs nothing.

Pause - I'll come back to this later

Shut down the running FastAPI server to free up your terminal. Your files and API key remain available for when you return.

  • Stop the FastAPI dev server by pressing Ctrl+C in the terminal where it is running.
  • Deactivate your virtual environment by running:
deactivate

When you come back, navigate to your project folder, activate the virtual environment, and run fastapi dev main.py again.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each step below.

  • Stop the FastAPI dev server by pressing Ctrl+C in the terminal.
  • Deactivate and delete the project folder (which includes the virtual environment) by running:
deactivate
cd ~/Desktop
rm -rf ai-agent-api

On Windows, use this instead:

deactivate
cd ~\Desktop
rmdir /s /q ai-agent-api

Nice Work!

Nice Work!

Congratulations! You've built a production-style AI agent API from scratch, combining LangChain's intelligent orchestration with FastAPI's high-performance serving layer.

This is the same architecture behind modern AI products like chatbots, coding assistants, and intelligent search tools.

You've learned how to:

  • Build a LangChain agent with custom tools that reasons about which tool to call for each question.
  • Create a FastAPI application with typed Pydantic request and response models for type-safe API contracts.
  • Implement real-time streaming responses via Server-Sent Events so clients receive tokens as they're generated.
  • Added session-based conversation memory so the agent remembers context across multi-turn interactions.

Ready to quiz yourself?