Build a Personal Notes Q&A Agent

Build a local LangChain agent that searches your notes using ChromaDB.

Introduction

30 Second Summary

Think about how many notes, meeting summaries, and ideas you jot down every week. Finding that one piece of information later usually means scrolling through pages of text, hoping your eyes land on the right paragraph.

In this project, you will build a personal notes Q&A agent that answers your questions by automatically searching through your notes. The agent runs entirely on your machine using LangChain and a local AI model, so there is no API cost and no data leaves your computer.

What You'll Build

A terminal-based Q&A system where a LangChain agent autonomously decides when to search your personal notes stored in ChromaDB.

By the end of this project, you'll have:

  • A LangChain agent running locally via Ollama that answers questions about your personal notes.
  • A ChromaDB vector store that indexes and retrieves your notes using semantic search.
  • A custom retrieval tool decorated with LangChain's @tool that the agent calls autonomously when it needs information from your notes.
  • Secret Mission: Add conversation memory so the agent remembers previous questions within a session.

Are there any prerequisites?

You should have basic Python knowledge and familiarity with RAG concepts like loading data, embeddings, and similarity search. You also need Python, VS Code, and Ollama already installed with the nomic-embed-text model pulled.

Before We Start

Your previous RAG project manually orchestrated the entire retrieval flow: query the vector store, pass results to the LLM, return the answer.

In this project, LangChain's agent does that orchestration for you. You define what tools exist (a notes search tool), and the agent autonomously decides when and how to use them.

In this step, get ready to:
  • Pull the qwen2.5:3b model in Ollama.
  • Create a Python virtual environment and install all packages.
  • Create the project file that will hold all your code.
Pull the qwen2.5:3b model

The qwen2.5:3b model is a compact language model that supports tool-calling. This is what your agent will use to decide when to search your notes.

  • Pull the model by running this command in your terminal:
ollama pull qwen2.5:3b

You should see a progress bar downloading the model layers, followed by a success message once it finishes.

Seeing a connection error?

Make sure Ollama is running in the background. Open a separate terminal window and run ollama serve to start it.

Why can't I connect to Ollama?

Set up your Python project

You need a dedicated project folder and a virtual environment to keep your packages isolated from the rest of your system.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir notes-agent
cd notes-agent
  • Confirm the folder exists by running:
ls

You should see the notes-agent folder listed on your Desktop.

  • Create a virtual environment by running:
python3 -m venv venv
  • Activate the virtual environment:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

What is a virtual environment?

A virtual environment is an isolated Python installation inside your project folder. Packages you install here stay separate from your system Python, so different projects can use different package versions without conflict.

  • Install all required packages by running:
pip install langchain langchain-ollama langchain-chroma langchain-text-splitters

You should see download progress for each package, followed by a "Successfully installed" message listing all packages and their versions.

Seeing a pip error?

Make sure you see (venv) in your terminal prompt. If it is missing, re-run the activate command from the platform tab above.

Help me fix this pip error

Verify the installation

A quick check confirms that both Ollama and your Python packages are ready to work together.

  • Confirm both models are available in Ollama by running:
ollama list

You should see both nomic-embed-text and qwen2.5:3b in the output.

  • Verify the LangChain package is importable by running:
python3 -c "import langchain; print(langchain.__version__)"

You should see a version number printed (such as 1.3.11). This confirms Python can find and load the LangChain library.

Seeing ModuleNotFoundError?

This usually means your virtual environment is not active. Check for (venv) at the start of your prompt. If it is missing, re-run the activate command above, then try the import again.

Help me debug this import error

  • Create your project file by running:
touch notes_agent.py
  • Confirm the file was created by running:
ls

You should see notes_agent.py and your venv folder listed in the output.

  • Open your project in VS Code by running:
code .

What is notes_agent.py?

This single file will hold all your code for the project. You will build up the agent piece by piece inside it across the remaining steps.

Your environment is ready. Next up, you will talk to your local Ollama model through LangChain and see your first agent response.

Talk to Ollama Through LangChain

Your environment is ready with all the tools installed. Now it is time to actually talk to your local model through LangChain.

LangChain's create_agent function is the core abstraction. You give it a model, a list of tools, and a system prompt. It returns an agent you can invoke with messages and get structured responses back.

In this step, get ready to:
  • Create a LangChain agent connected to your local Ollama model.
  • Send messages and observe the agent's responses.
  • Explore the response structure to understand what LangChain returns.
Add imports to your script

LangChain organises its functionality into separate packages. You need imports from several of them to build your agent across the next few steps.

  • Switch back to notes_agent.py in VS Code and paste the following imports at the top of the file:
from langchain.agents import create_agent
from langchain.tools import tool
from langchain_ollama import OllamaEmbeddings
from langchain_chroma import Chroma
from langchain_core.documents import Document
from langchain_text_splitters import RecursiveCharacterTextSplitter

What are these imports for?

  • create_agent and tool are the core agent-building blocks you will use in this step and Step 4.
  • OllamaEmbeddings and Chroma will connect to your vector store in Step 3.
  • Document and RecursiveCharacterTextSplitter will prepare your notes for indexing in Step 3.

You will not see any output yet. These imports prepare the building blocks for the rest of the project.

Create your first agent function

The agent wraps your Ollama model so you can send it messages in a standard format and get structured responses back. The system_prompt parameter shapes the agent's personality and behaviour.

  • Below the imports in notes_agent.py, add the following function:
# --- Step 2: Basic agent setup ---

def create_basic_agent():
    agent = create_agent(
        model="ollama:qwen2.5:3b",
        tools=[],
        system_prompt="You are a helpful assistant. Be concise and accurate.",
    )
    result = agent.invoke(
        {"messages": [{"role": "user", "content": "What is LangChain in one sentence?"}]}
    )
    print(result["messages"][-1].content)

What does this code do?

  • create_agent() builds an agent connected to your local qwen2.5:3b model through Ollama. The "ollama:qwen2.5:3b" string tells LangChain which provider and model to use.
  • tools=[] means the agent has no tools yet. It can only respond from its own knowledge.
  • system_prompt sets the personality of the agent. The model reads this instruction before every response. Changing this string changes how the agent answers.
  • agent.invoke() sends a message in the standard chat format (role + content). The result is a dictionary containing a messages list.
  • result["messages"][-1].content grabs the text content of the last message in the response, which is the agent's reply.
  • Save notes_agent.py.
  • Test the function by running this command in your terminal:
python -c "from notes_agent import create_basic_agent; create_basic_agent()"

You should see a one-sentence description of LangChain printed in your terminal. The exact wording varies each time since the model generates fresh responses.

Not seeing a response?

  • Make sure Ollama is running. Open a separate terminal and run ollama list to confirm qwen2.5:3b is listed.
  • Check that your virtual environment is activated. You should see (venv) or similar at the start of your terminal prompt.
  • If you see an import error, confirm you installed all packages in Step 1.

Help me debug my LangChain agent connection to Ollama

Explore the agent's response structure

When you call agent.invoke(), LangChain returns a dictionary with more than just the text reply. Understanding this structure helps you work with tool calls in later steps.

  • In notes_agent.py, find this line inside create_basic_agent():
    print(result["messages"][-1].content)
  • Temporarily change it to print the full result dictionary:
    print(result)
  • Save notes_agent.py and run the same test command again:
python -c "from notes_agent import create_basic_agent; create_basic_agent()"

You should see a dictionary with a messages key containing a list of message objects.

What is inside the result dictionary?

The result is a state dictionary. The messages key holds the full conversation history: your input message (a HumanMessage) followed by the agent's reply (an AIMessage).

Each message object has a content attribute with the text, plus metadata like tool_calls (empty for now since no tools are configured). In the next steps you will see tool calls appear here when the agent decides to search your notes.

  • Change the print line back to its original form so only the response text is printed:
    print(result["messages"][-1].content)
  • Save notes_agent.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved notes_agent.py with the final version of create_basic_agent() that prints only the content.

ⓧ I'd like to double check the full code

from langchain.agents import create_agent
from langchain.tools import tool
from langchain_ollama import OllamaEmbeddings
from langchain_chroma import Chroma
from langchain_core.documents import Document
from langchain_text_splitters import RecursiveCharacterTextSplitter


# --- Step 2: Basic agent setup ---

def create_basic_agent():
    agent = create_agent(
        model="ollama:qwen2.5:3b",
        tools=[],
        system_prompt="You are a helpful assistant. Be concise and accurate.",
    )
    result = agent.invoke(
        {"messages": [{"role": "user", "content": "What is LangChain in one sentence?"}]}
    )
    print(result["messages"][-1].content)

You now have a working LangChain agent that talks to your local Ollama model. Next up, you will index your personal notes into ChromaDB so the agent has something to search.

Index Notes into ChromaDB

In Step 2, you proved that your LangChain agent can talk to Ollama and return a response.

Now it's time to give the agent something to search. In this step, you'll store personal notes in ChromaDB using LangChain's vector store abstraction. Instead of calling the chromadb library directly, LangChain wraps it so your agent can later access the data with a single tool call.

In this step, get ready to:
  • Define sample notes as LangChain Document objects with metadata.
  • Index those notes into ChromaDB using LangChain's embeddings and vector store wrappers.
  • Verify retrieval by running a similarity search against your stored notes.
Define your sample notes

Before you can search anything, the agent needs data to search through. LangChain represents documents as Document objects, each containing a page_content string and a metadata dictionary.

The metadata lets you filter results later. You'll tag each note with a user field so your agent can retrieve notes for a specific person.

  • In notes_agent.py, find the end of the create_basic_agent() function.
  • Below it, add the following SAMPLE_NOTES list and a comment to mark the new section:
# --- Step 3: Index notes into ChromaDB ---

SAMPLE_NOTES = [
    Document(
        page_content="Meeting with design team on Monday. Discussed new dashboard layout. "
        "Agreed to use a sidebar navigation pattern. Follow-up scheduled for Thursday.",
        metadata={"user": "alice", "source": "work_notes"},
    ),
    Document(
        page_content="Python project ideas: build a CLI tool for managing dotfiles, "
        "create a FastAPI service for bookmark management, "
        "experiment with LangChain for document Q&A.",
        metadata={"user": "alice", "source": "personal_notes"},
    ),
    Document(
        page_content="Grocery list: eggs, milk, bread, avocados, chicken breast, "
        "rice, black beans, salsa, tortillas, lime.",
        metadata={"user": "alice", "source": "personal_notes"},
    ),
    Document(
        page_content="Sprint retrospective notes: deployment pipeline is too slow, "
        "need to investigate caching for CI builds. "
        "Team morale is good. Next sprint focus on performance.",
        metadata={"user": "alice", "source": "work_notes"},
    ),
    Document(
        page_content="Book recommendations from Bob: Designing Data-Intensive Applications, "
        "The Pragmatic Programmer, Clean Code. "
        "Start with the first one since it covers distributed systems.",
        metadata={"user": "alice", "source": "personal_notes"},
    ),
]

What does this code do?

Each Document object has two parts:

  • page_content holds the actual text of the note.
  • metadata is a dictionary with user and source fields. The user field enables per-user filtering so the agent only searches one person's notes at a time.

These five notes simulate a realistic personal knowledge base with work notes, project ideas, grocery lists, and book recommendations.

  • Save notes_agent.py.

Seeing a syntax error?

Make sure each Document( call has a matching closing parenthesis and a trailing comma after it. Multi-line strings joined with "..." "..." must be on separate lines without a + between them.

help me fix a syntax error in my SAMPLE_NOTES list

Write the indexing function

With your notes defined, you need a function that converts them into searchable embeddings and stores them in ChromaDB. LangChain handles the embedding step through OllamaEmbeddings, which calls your local nomic-embed-text model under the hood.

  • Below the SAMPLE_NOTES list, add the index_notes() function:
def index_notes():
    embeddings = OllamaEmbeddings(model="nomic-embed-text")

    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=500,
        chunk_overlap=50,
    )
    all_splits = text_splitter.split_documents(SAMPLE_NOTES)

    vector_store = Chroma(
        collection_name="personal_notes",
        embedding_function=embeddings,
        persist_directory="./chroma_langchain_db",
    )

    document_ids = vector_store.add_documents(documents=all_splits)
    print(f"Indexed {len(document_ids)} document chunks.")

    # Test retrieval
    results = vector_store.similarity_search("What books were recommended?", k=2)
    for doc in results:
        print(f"Source: {doc.metadata}")
        print(f"Content: {doc.page_content[:100]}...")
        print()

    return vector_store

What does this function do?

This function does four things in sequence:

  • Creates an OllamaEmbeddings instance that uses your local nomic-embed-text model to convert text into numerical vectors.
  • Splits notes into smaller chunks with RecursiveCharacterTextSplitter. A chunk_size of 500 characters with 50-character overlap ensures each chunk is small enough for precise retrieval while preserving context at boundaries.
  • Initializes a Chroma vector store with persist_directory so your indexed data survives between script runs.
  • Calls add_documents() to embed each chunk and store it in ChromaDB, then runs a test query with similarity_search() to confirm everything works.

Why split into chunks?

Embedding models work best with shorter text passages. A 500-character chunk captures a single topic cleanly. When the agent searches later, it retrieves only the most relevant chunk instead of an entire long document.

  • Save notes_agent.py.

Indentation error?

Everything inside index_notes() must be indented by exactly 4 spaces. Make sure the embeddings = line is indented one level from def index_notes():.

help me fix an indentation error in my index_notes function

Run the indexer and verify retrieval

Your function is defined but hasn't run yet. You need to call it once to actually embed and store the notes in ChromaDB.

  • At the very bottom of notes_agent.py, add a temporary call to run the function:
index_notes()
  • Save notes_agent.py.
  • Run the script in your terminal by running this command:
python notes_agent.py

You should see output showing how many chunks were indexed, followed by retrieval results for the test query about book recommendations.

What should I see?

The output will show a line like Indexed 5 document chunks. followed by metadata and content snippets from the book recommendations note. The number of chunks may vary slightly depending on how the text splits.

The key thing is that the retrieval results are semantically relevant to the query. You asked about books, and the returned content should be about book recommendations.

  • Once you've confirmed the output, remove the index_notes() line from the bottom of your file.
  • Save notes_agent.py.

Seeing a connection error?

Make sure Ollama is running in the background. If you closed it, restart it before running the script again. The OllamaEmbeddings class connects to Ollama's local server to generate embeddings.

help me debug a connection error when running index_notes

✔️ Awesome, I've got everything!

Great. Double check you've saved your file with the index_notes() call removed from the bottom.

ⓧ I'd like to double check the full code

Here is the complete notes_agent.py file at this point:

from langchain.agents import create_agent
from langchain.tools import tool
from langchain_ollama import OllamaEmbeddings
from langchain_chroma import Chroma
from langchain_core.documents import Document
from langchain_text_splitters import RecursiveCharacterTextSplitter


# --- Step 2: Basic agent setup ---

def create_basic_agent():
    agent = create_agent(
        model="ollama:qwen2.5:3b",
        tools=[],
        system_prompt="You are a helpful assistant. Be concise and accurate.",
    )
    result = agent.invoke(
        {"messages": [{"role": "user", "content": "What is LangChain in one sentence?"}]}
    )
    print(result["messages"][-1].content)


# --- Step 3: Index notes into ChromaDB ---

SAMPLE_NOTES = [
    Document(
        page_content="Meeting with design team on Monday. Discussed new dashboard layout. "
        "Agreed to use a sidebar navigation pattern. Follow-up scheduled for Thursday.",
        metadata={"user": "alice", "source": "work_notes"},
    ),
    Document(
        page_content="Python project ideas: build a CLI tool for managing dotfiles, "
        "create a FastAPI service for bookmark management, "
        "experiment with LangChain for document Q&A.",
        metadata={"user": "alice", "source": "personal_notes"},
    ),
    Document(
        page_content="Grocery list: eggs, milk, bread, avocados, chicken breast, "
        "rice, black beans, salsa, tortillas, lime.",
        metadata={"user": "alice", "source": "personal_notes"},
    ),
    Document(
        page_content="Sprint retrospective notes: deployment pipeline is too slow, "
        "need to investigate caching for CI builds. "
        "Team morale is good. Next sprint focus on performance.",
        metadata={"user": "alice", "source": "work_notes"},
    ),
    Document(
        page_content="Book recommendations from Bob: Designing Data-Intensive Applications, "
        "The Pragmatic Programmer, Clean Code. "
        "Start with the first one since it covers distributed systems.",
        metadata={"user": "alice", "source": "personal_notes"},
    ),
]


def index_notes():
    embeddings = OllamaEmbeddings(model="nomic-embed-text")

    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=500,
        chunk_overlap=50,
    )
    all_splits = text_splitter.split_documents(SAMPLE_NOTES)

    vector_store = Chroma(
        collection_name="personal_notes",
        embedding_function=embeddings,
        persist_directory="./chroma_langchain_db",
    )

    document_ids = vector_store.add_documents(documents=all_splits)
    print(f"Indexed {len(document_ids)} document chunks.")

    # Test retrieval
    results = vector_store.similarity_search("What books were recommended?", k=2)
    for doc in results:
        print(f"Source: {doc.metadata}")
        print(f"Content: {doc.page_content[:100]}...")
        print()

    return vector_store

Your notes are now embedded and stored in ChromaDB, ready for the agent to search. Next up, you'll wrap that search as a LangChain tool so the agent can call it autonomously.

Build the Retrieval Tool

Your notes are indexed and searchable in ChromaDB. Now you need a way for the agent to access that search capability on its own.

In LangChain, a tool is a Python function the agent can decide to call. You define the function, describe what it does in a docstring, and LangChain handles the rest. The agent reads the description and decides when the tool is relevant to the user's question.

In this step, get ready to:
  • Connect to your existing ChromaDB vector store at the module level.
  • Write a retrieval tool using LangChain's @tool decorator with metadata filtering.
  • Test the tool by calling it directly as a Python function.
Connect to your existing vector store at module level

Your tool function needs access to the ChromaDB vector store to search notes. Instead of creating the connection inside the function (which would reconnect on every call), you create it once at the module level so it is available to any function in the file.

  • In notes_agent.py, scroll to the bottom of the file (after the index_notes() function) and add the following code:
# --- Step 4: Build the retrieval tool ---

embeddings = OllamaEmbeddings(model="nomic-embed-text")
vector_store = Chroma(
    collection_name="personal_notes",
    embedding_function=embeddings,
    persist_directory="./chroma_langchain_db",
)

What does this code do?

This creates two module-level objects that load once when the script starts.

  • embeddings is your Ollama embedding model. It converts text queries into vectors for comparison.
  • vector_store connects to the same ChromaDB collection you indexed in the previous step. The persist_directory points to your existing database on disk.
  • Save notes_agent.py.
  • You won't see a change yet. The next chunk uses this vector store connection inside the tool function.
Write the search tool with metadata filtering

LangChain's @tool decorator turns a regular Python function into something the agent can call. The function's docstring becomes the tool's description that the model reads to decide when to use it.

  • In notes_agent.py, add the following function directly below the vector_store assignment you just created:
@tool
def search_notes(query: str) -> str:
    """Search the user's personal notes for relevant information. Use this tool when the user asks about their notes, meetings, tasks, lists, or anything they have previously written down."""
    retrieved_docs = vector_store.similarity_search(
        query,
        k=2,
        filter={"user": "alice"},
    )
    if not retrieved_docs:
        return "No relevant notes found."
    serialized = "\n\n".join(
        f"Source: {doc.metadata.get('source', 'unknown')}\nContent: {doc.page_content}"
        for doc in retrieved_docs
    )
    return serialized

What does this code do?

  • @tool registers this function as a LangChain tool. The agent can now discover it and call it autonomously.
  • The docstring is critical. The model reads it to decide whether this tool is relevant to the user's question. A vague description means the agent won't know when to call it.
  • filter={"user": "alice"} ensures the search only returns notes belonging to a specific user. This is per-user isolation using ChromaDB's metadata filtering.
  • The function returns a formatted string (not raw Document objects) because the agent needs plain text it can read and summarize for the user.

Why filter by metadata instead of a separate collection?

A single collection with metadata filters scales better than one collection per user. You can add more users to the same collection and filter at query time without managing multiple databases.

  • Save notes_agent.py.

Seeing a syntax error on save?

  • Make sure the @tool decorator sits directly above the def search_notes line with no blank lines between them.
  • Check that the triple-quoted docstring is on one line or properly continued. Python expects the closing """ before the function body starts.
  • Verify the filter dictionary uses curly braces, not square brackets.

Ask for help: Why am I getting a syntax error in my @tool function?

Test the tool directly

Before wiring the tool to an agent, you can call it directly as a Python function. This lets you verify it returns the expected notes without involving the LLM.

  • Add this temporary test code at the very bottom of notes_agent.py:
# Temporary test - remove after verifying
result = search_notes.invoke("What books were recommended?")
print(result)
  • Run the script to test your tool by running this command:
python notes_agent.py

You should see output showing the book recommendations note with its source metadata. It looks something like this:

Source: personal_notes
Content: Book recommendations from Bob: Designing Data-Intensive Applications, The Pragmatic Programmer, Clean Code. Start with the first one since it covers distributed systems.

Why test without the agent?

Testing a tool in isolation confirms that your vector store connection, metadata filter, and output formatting all work correctly. If the agent gives a wrong answer later, you know the issue is in the agent's decision-making, not in the tool itself.

  • Now remove the two temporary test lines (the result = search_notes.invoke(...) and print(result) lines) from the bottom of notes_agent.py.
  • Save notes_agent.py.

Not seeing any output or getting an error?

  • Make sure Ollama is running. The embedding model needs to be available for the similarity search. Run ollama list to check.
  • Verify that the ./chroma_langchain_db directory exists from the indexing step. If you deleted it, you need to run index_notes() again first.
  • Check that collection_name in the module-level Chroma(...) matches what you used during indexing: "personal_notes".

Ask for help: My search_notes tool is not returning any results when I test it directly.

✔️ Awesome, I've got everything!

Great. Double check you've saved notes_agent.py and that the temporary test lines have been removed.

ⓧ I'd like to double check the full code

Here is the full notes_agent.py file up to this point:

from langchain.agents import create_agent
from langchain.tools import tool
from langchain_ollama import OllamaEmbeddings
from langchain_chroma import Chroma
from langchain_core.documents import Document
from langchain_text_splitters import RecursiveCharacterTextSplitter


# --- Step 2: Basic agent setup ---

def create_basic_agent():
    agent = create_agent(
        model="ollama:qwen2.5:3b",
        tools=[],
        system_prompt="You are a helpful assistant. Be concise and accurate.",
    )
    result = agent.invoke(
        {"messages": [{"role": "user", "content": "What is LangChain in one sentence?"}]}
    )
    print(result["messages"][-1].content)


# --- Step 3: Index notes into ChromaDB ---

SAMPLE_NOTES = [
    Document(
        page_content="Meeting with design team on Monday. Discussed new dashboard layout. "
        "Agreed to use a sidebar navigation pattern. Follow-up scheduled for Thursday.",
        metadata={"user": "alice", "source": "work_notes"},
    ),
    Document(
        page_content="Python project ideas: build a CLI tool for managing dotfiles, "
        "create a FastAPI service for bookmark management, "
        "experiment with LangChain for document Q&A.",
        metadata={"user": "alice", "source": "personal_notes"},
    ),
    Document(
        page_content="Grocery list: eggs, milk, bread, avocados, chicken breast, "
        "rice, black beans, salsa, tortillas, lime.",
        metadata={"user": "alice", "source": "personal_notes"},
    ),
    Document(
        page_content="Sprint retrospective notes: deployment pipeline is too slow, "
        "need to investigate caching for CI builds. "
        "Team morale is good. Next sprint focus on performance.",
        metadata={"user": "alice", "source": "work_notes"},
    ),
    Document(
        page_content="Book recommendations from Bob: Designing Data-Intensive Applications, "
        "The Pragmatic Programmer, Clean Code. "
        "Start with the first one since it covers distributed systems.",
        metadata={"user": "alice", "source": "personal_notes"},
    ),
]


def index_notes():
    embeddings = OllamaEmbeddings(model="nomic-embed-text")

    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=500,
        chunk_overlap=50,
    )
    all_splits = text_splitter.split_documents(SAMPLE_NOTES)

    vector_store = Chroma(
        collection_name="personal_notes",
        embedding_function=embeddings,
        persist_directory="./chroma_langchain_db",
    )

    document_ids = vector_store.add_documents(documents=all_splits)
    print(f"Indexed {len(document_ids)} document chunks.")

    # Test retrieval
    results = vector_store.similarity_search("What books were recommended?", k=2)
    for doc in results:
        print(f"Source: {doc.metadata}")
        print(f"Content: {doc.page_content[:100]}...")
        print()

    return vector_store


# --- Step 4: Build the retrieval tool ---

embeddings = OllamaEmbeddings(model="nomic-embed-text")
vector_store = Chroma(
    collection_name="personal_notes",
    embedding_function=embeddings,
    persist_directory="./chroma_langchain_db",
)


@tool
def search_notes(query: str) -> str:
    """Search the user's personal notes for relevant information. Use this tool when the user asks about their notes, meetings, tasks, lists, or anything they have previously written down."""
    retrieved_docs = vector_store.similarity_search(
        query,
        k=2,
        filter={"user": "alice"},
    )
    if not retrieved_docs:
        return "No relevant notes found."
    serialized = "\n\n".join(
        f"Source: {doc.metadata.get('source', 'unknown')}\nContent: {doc.page_content}"
        for doc in retrieved_docs
    )
    return serialized

Your retrieval tool is tested and ready. In the next step, you'll wire it into a LangChain agent and watch it decide when to search your notes versus answering directly.

Create the Agent and Run It

Your retrieval tool is ready and tested. You have all the pieces in place: a local LLM via Ollama, notes indexed in ChromaDB, and a search tool that retrieves relevant documents filtered by user.

Now it's time to wire everything together into a LangChain agent that autonomously decides when to search your notes. You'll create the agent, give it your tool, and build an interactive loop so you can have a real conversation.

In this step, get ready to:
  • Create a LangChain agent that uses your retrieval tool via tool-calling.
  • Observe the agent deciding when to search notes vs. answering directly.
  • Run a complete interactive Q&A session in your terminal.
Wire up the agent with your retrieval tool

The create_agent function connects a model to tools and a system prompt. The system prompt tells the agent when it should reach for the tool vs. when it should answer from its own knowledge.

  • In notes_agent.py, scroll to the bottom of the file (below the search_notes function).
  • Add the following code to create your agent and interactive loop:
# --- Step 5: Create the full agent and interactive loop ---

def main():
    agent = create_agent(
        model="ollama:qwen2.5:3b",
        tools=[search_notes],
        system_prompt=(
            "You are a helpful personal assistant. You have access to a tool that "
            "searches the user's personal notes. Use it when the user asks about "
            "their notes, meetings, tasks, lists, or things they have written down. "
            "If the tool returns no relevant information, say you could not find "
            "anything in their notes. For general knowledge questions, answer "
            "directly without using the tool."
        ),
    )

    print("Personal Notes Q&A Agent")
    print("Type your questions below. Type 'quit' to exit.\n")

    while True:
        user_input = input("You: ").strip()
        if user_input.lower() in ("quit", "exit", "q"):
            print("Goodbye!")
            break
        if not user_input:
            continue

        result = agent.invoke(
            {"messages": [{"role": "user", "content": user_input}]}
        )
        response = result["messages"][-1].content
        print(f"\nAgent: {response}\n")


if __name__ == "__main__":
    # Uncomment the next line to index notes (run once)
    # index_notes()
    main()

What does this code do?

This is where everything comes together. Here's what each piece does:

  • create_agent(model="ollama:qwen2.5:3b", tools=[search_notes], ...) creates an agent that talks to your local Ollama model and can call your search_notes tool.
  • The system_prompt instructs the agent to use the tool for note-related questions and answer general knowledge questions directly.
  • The while True loop reads user input, invokes the agent, and prints the response until the user types quit.
  • The if __name__ == "__main__" block ensures main() runs only when you execute the script directly.

Why is index_notes() commented out?

You already indexed your notes into ChromaDB in the previous step. Running index_notes() again would create duplicate documents. The comment is there so you can re-index later if you add new notes.

  • Save notes_agent.py.
  • Run the script by running this command in your terminal:
python notes_agent.py

You should see the agent prompt appear:

Personal Notes Q&A Agent
Type your questions below. Type 'quit' to exit.

You:
  • Type a question about your notes, such as What books were recommended? and press Enter.

The agent should respond with information from your indexed notes about book recommendations from Bob.

Not getting a response?

  • Make sure Ollama is running. Open a separate terminal and run ollama list to verify both qwen2.5:3b and nomic-embed-text appear.
  • If you see an error about the ChromaDB collection, you may need to re-index. Temporarily uncomment index_notes() in the if __name__ block, run the script once, then comment it out again.
  • The first response may take 10-30 seconds as the model loads into memory. Subsequent responses are faster.

Help me debug my LangChain agent not responding to notes questions.

Test the agent's decision-making

A key feature of a LangChain agent is that it decides on its own when to call a tool. It reads your question, considers the tool's description (the docstring you wrote), and chooses whether to use it.

  • With the agent still running, type a general knowledge question like What is Python? and press Enter.

The agent answers directly from its own knowledge without searching your notes. It read the question, checked the tool description, and decided the tool was not relevant.

  • Now ask What was discussed in my design team meeting? and press Enter.

This time the agent uses your tool, retrieves the relevant note about the Monday meeting, and responds with details about the dashboard layout and sidebar navigation.

How does the agent decide?

The agent reads two things to make its decision: the system prompt (which says to use the tool for notes-related questions) and the tool's docstring (which describes what kind of information it searches). Together, these tell the model when the tool is appropriate.

This is the core pattern of tool-calling agents. You define tools with clear descriptions, and the model figures out the right moment to use them.

Explore the interactive loop

Your Q&A system is fully functional. The interactive loop keeps running until you explicitly exit, so you can ask as many questions as you want in a single session.

  • Try asking What's on my grocery list? to see the agent retrieve a different note.
  • Try pressing Enter on an empty line. The loop skips blank input and prompts you again.
  • Type quit and press Enter to exit the agent.

You should see Goodbye! and the script ends cleanly.

What makes this different from your previous RAG project?

In your previous project, you manually orchestrated the flow: query the vector store, format results, pass to the LLM. Here, the agent handles that orchestration. You defined the tool, and the agent autonomously decides when and how to use it.

This is the core LangChain mental model. Agent = Model + Tools + Prompt. You supply the pieces, and the agent handles the logic.

✔️ Awesome, I've got everything!

Great. Double check you've saved notes_agent.py and your agent responds correctly to both types of questions.

ⓧ I'd like to double check the full code

Here is the complete notes_agent.py file at this point:

from langchain.agents import create_agent
from langchain.tools import tool
from langchain_ollama import OllamaEmbeddings
from langchain_chroma import Chroma
from langchain_core.documents import Document
from langchain_text_splitters import RecursiveCharacterTextSplitter


# --- Step 2: Basic agent setup ---

def create_basic_agent():
    agent = create_agent(
        model="ollama:qwen2.5:3b",
        tools=[],
        system_prompt="You are a helpful assistant. Be concise and accurate.",
    )
    result = agent.invoke(
        {"messages": [{"role": "user", "content": "What is LangChain in one sentence?"}]}
    )
    print(result["messages"][-1].content)


# --- Step 3: Index notes into ChromaDB ---

SAMPLE_NOTES = [
    Document(
        page_content="Meeting with design team on Monday. Discussed new dashboard layout. "
        "Agreed to use a sidebar navigation pattern. Follow-up scheduled for Thursday.",
        metadata={"user": "alice", "source": "work_notes"},
    ),
    Document(
        page_content="Python project ideas: build a CLI tool for managing dotfiles, "
        "create a FastAPI service for bookmark management, "
        "experiment with LangChain for document Q&A.",
        metadata={"user": "alice", "source": "personal_notes"},
    ),
    Document(
        page_content="Grocery list: eggs, milk, bread, avocados, chicken breast, "
        "rice, black beans, salsa, tortillas, lime.",
        metadata={"user": "alice", "source": "personal_notes"},
    ),
    Document(
        page_content="Sprint retrospective notes: deployment pipeline is too slow, "
        "need to investigate caching for CI builds. "
        "Team morale is good. Next sprint focus on performance.",
        metadata={"user": "alice", "source": "work_notes"},
    ),
    Document(
        page_content="Book recommendations from Bob: Designing Data-Intensive Applications, "
        "The Pragmatic Programmer, Clean Code. "
        "Start with the first one since it covers distributed systems.",
        metadata={"user": "alice", "source": "personal_notes"},
    ),
]


def index_notes():
    embeddings = OllamaEmbeddings(model="nomic-embed-text")

    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=500,
        chunk_overlap=50,
    )
    all_splits = text_splitter.split_documents(SAMPLE_NOTES)

    vector_store = Chroma(
        collection_name="personal_notes",
        embedding_function=embeddings,
        persist_directory="./chroma_langchain_db",
    )

    document_ids = vector_store.add_documents(documents=all_splits)
    print(f"Indexed {len(document_ids)} document chunks.")

    # Test retrieval
    results = vector_store.similarity_search("What books were recommended?", k=2)
    for doc in results:
        print(f"Source: {doc.metadata}")
        print(f"Content: {doc.page_content[:100]}...")
        print()

    return vector_store


# --- Step 4: Build the retrieval tool ---

embeddings = OllamaEmbeddings(model="nomic-embed-text")
vector_store = Chroma(
    collection_name="personal_notes",
    embedding_function=embeddings,
    persist_directory="./chroma_langchain_db",
)


@tool
def search_notes(query: str) -> str:
    """Search the user's personal notes for relevant information. Use this tool when the user asks about their notes, meetings, tasks, lists, or anything they have previously written down."""
    retrieved_docs = vector_store.similarity_search(
        query,
        k=2,
        filter={"user": "alice"},
    )
    if not retrieved_docs:
        return "No relevant notes found."
    serialized = "\n\n".join(
        f"Source: {doc.metadata.get('source', 'unknown')}\nContent: {doc.page_content}"
        for doc in retrieved_docs
    )
    return serialized


# --- Step 5: Create the full agent and interactive loop ---

def main():
    agent = create_agent(
        model="ollama:qwen2.5:3b",
        tools=[search_notes],
        system_prompt=(
            "You are a helpful personal assistant. You have access to a tool that "
            "searches the user's personal notes. Use it when the user asks about "
            "their notes, meetings, tasks, lists, or things they have written down. "
            "If the tool returns no relevant information, say you could not find "
            "anything in their notes. For general knowledge questions, answer "
            "directly without using the tool."
        ),
    )

    print("Personal Notes Q&A Agent")
    print("Type your questions below. Type 'quit' to exit.\n")

    while True:
        user_input = input("You: ").strip()
        if user_input.lower() in ("quit", "exit", "q"):
            print("Goodbye!")
            break
        if not user_input:
            continue

        result = agent.invoke(
            {"messages": [{"role": "user", "content": user_input}]}
        )
        response = result["messages"][-1].content
        print(f"\nAgent: {response}\n")


if __name__ == "__main__":
    # Uncomment the next line to index notes (run once)
    # index_notes()
    main()

You now have a fully working personal notes Q&A agent running 100% locally. In the Secret Mission, you'll add conversation memory so the agent remembers what you asked earlier in the session.

Secret mission

Add Conversation Memory

Your agent answers each question in isolation. It has no idea what you asked 10 seconds ago. Can you give it a memory so follow-up questions actually work?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources on disk, or clean them up to free space.

Resources you used:

  • ChromaDB database stored in ./chroma_langchain_db.
  • Ollama model qwen2.5:3b stored in ~/.ollama/models.
  • Project directory containing notes_agent.py and your Python virtual environment.

Keep everything running

No action needed. Choose this if you plan to keep experimenting with your notes agent or adding more notes.

  • Your ChromaDB database stays on disk, ready to query anytime you run the script.
  • The qwen2.5:3b model stays available for other Ollama projects.
  • The nomic-embed-text model remains available for embedding tasks.

Pause - I'll come back to this later

Stop any running processes to free up memory. Your files and models stay on disk for next time.

  • Stop the Ollama server if it is running in the background.
  • Deactivate your Python virtual environment by running:
deactivate

Everything stays on disk. When you come back, reactivate your virtual environment and run the script again.

Delete - I don't want to use this again

Remove all project resources and free up disk space completely.

  • Delete the ChromaDB database by running:
rm -rf ./chroma_langchain_db
  • Remove the qwen2.5:3b model from Ollama by running:
ollama rm qwen2.5:3b

Why keep nomic-embed-text?

The outline keeps nomic-embed-text because it was a pre-existing model you may use for other embedding projects. If you want to remove it too, run ollama rm nomic-embed-text.

  • Delete the project directory (adjust the path if you created it somewhere other than your Desktop):
cd ~/Desktop
rm -rf notes-agent

On Windows, use:

cd ~\Desktop
rmdir /s /q notes-agent
  • Deactivate your virtual environment first if it is still active:
deactivate

Nice Work!

Nice Work!

You just built a fully local, zero-cost Q&A agent powered by LangChain and Ollama. This is the exact pattern used in production RAG systems, running entirely on your own machine.

Here's what you accomplished:

  • Built a LangChain agent from scratch that runs 100% locally via Ollama, using create_agent to connect to the qwen2.5:3b model.
  • Indexed personal notes into ChromaDB using LangChain's OllamaEmbeddings and Chroma vector store wrapper with metadata filtering for per-user isolation.
  • Created a custom retrieval tool with the @tool decorator that the agent calls autonomously for note-related questions and skips for general knowledge.
  • Secret Mission: Added conversation memory with InMemorySaver so the agent maintains context across turns within a session.

Ready to quiz yourself?