Build a Local Second Brain with RAG

Build a private, offline AI assistant that answers questions from your PDFs.

Introduction

30 Second Summary

You read articles, save PDFs, and bookmark research papers. But when you need to find that one specific detail weeks later, it feels like searching for a needle in a haystack.

In this project, you will build a local AI-powered "second brain" that reads your PDF documents and answers questions about them through a chat interface. Everything runs on your own machine, so your data stays private and you never pay for API calls.

What You'll Build

A Python application that ingests PDF files into a vector database, then uses a local AI model to answer questions grounded exclusively in your documents.

By the end of this project, you'll have:

  • A document ingestion pipeline that extracts text from PDFs, splits it into chunks, and stores embeddings in a ChromaDB vector database.
  • A RAG query engine that retrieves relevant document chunks and generates answers using a local Llama 3.2 model via Ollama.
  • A Streamlit chat interface for uploading PDFs and asking questions about your documents in the browser.
  • Secret Mission: Add source citations with page numbers to every answer so you can verify exactly where the AI found its information.

Are there any prerequisites?

No prior experience with AI or Retrieval-Augmented Generation is needed. This guide walks you through installing Python, Ollama, and all required libraries from scratch.

Set Up Your Environment

Before you can build your second brain, you need a few tools installed on your machine.

In this step, you will install Python, set up a project folder with a virtual environment, install Ollama (a tool for running AI models locally), and grab the Python libraries that power your RAG pipeline.

In this step, get ready to:
  • Install Python and create a project folder with a virtual environment.
  • Install Ollama and pull the Llama 3.2 model.
  • Install the Python libraries for document processing, embeddings, and the chat UI.
Install Python and create a project folder

Python is the programming language you will use to build every piece of this project. You need version 3.10 or higher installed.

  • Check if Python is already installed by opening your terminal and running this command:

How to open a terminal

Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.

Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.

python --version

✔️ I see version 3.10 or higher

Your Python version is compatible. Move on to creating the project folder below.

ⓧ I see an older version

You have Python installed, but it is too old for this project. You need 3.10 or higher.

  • Visit python.org/downloads and download the latest Python 3 installer for your operating system.
  • Run the installer. On Windows, check the box that says Add Python to PATH before clicking Install Now.
  • Close and reopen your terminal, then run python --version again to confirm the new version appears.

Still seeing the old version?

On Windows, the old version may shadow the new install. Try running py --version instead. If that shows the correct version, use py in place of python for the rest of this project.

Help me fix my Python PATH

ⓧ Command not found

Python is not installed on your machine yet.

  • Visit python.org/downloads and download the latest Python 3 installer for your operating system.
  • Run the installer. On Windows, check the box that says Add Python to PATH before clicking Install Now.
  • Close and reopen your terminal, then run python --version to confirm the installation worked.

Installation not working?

Make sure you downloaded the correct installer for your operating system (Windows or macOS). On macOS, you may need to use python3 --version instead of python --version.

Help me troubleshoot my Python installation

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir second-brain
cd second-brain
  • Create a virtual environment by running this command:
python -m venv venv

What is a virtual environment?

A virtual environment is an isolated space for your project's Python packages. It keeps this project's libraries separate from other Python projects on your machine, preventing version conflicts.

  • Activate the virtual environment by running this command:

Windows

venv\Scripts\activate

macOS

source venv/bin/activate

You should see (venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

On Windows, if you get an error about execution policies, run this command first, then try activating again:

Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned

Help me activate my virtual environment

  • Open your project folder in VS Code. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop, select the second-brain folder, and click Open.
Install Ollama and pull a model

Ollama is a tool that runs large language models directly on your computer. Your second brain uses it to generate answers from your documents without sending data to the cloud.

  • Check if Ollama is already installed by running this command in your terminal:
ollama --version

✔️ I see a version number

Ollama is already installed. Skip ahead to pulling the model below.

ⓧ Command not found

Ollama is not installed yet. Install it now.

Windows

  • Install Ollama by pasting this command into PowerShell:
irm https://ollama.com/install.ps1 | iex

The installer does not require Administrator rights. It installs in your home directory and Ollama will run in the background automatically.

macOS

  • Install Ollama by pasting this command into your terminal:
curl -fsSL https://ollama.com/install.sh | sh
  • After installation finishes, close and reopen your terminal.
  • Run ollama --version again to confirm Ollama is now available.

Still getting command not found?

On Windows, make sure you used PowerShell (not Command Prompt) to run the install command. You may also need to restart your computer for the PATH change to take effect.

Help me troubleshoot my Ollama installation

  • Download the Llama 3.2 model by running this command (this is a 2 GB download):
ollama pull llama3.2

What is Llama 3.2?

Llama 3.2 is a 3-billion parameter language model from Meta. At 2 GB, it is small enough to run comfortably on your machine while still being capable of answering document-based questions.

Your machine has 32 GB of RAM. This model only needs around 3 GB of RAM to run, leaving plenty of room for the rest of the application.

  • Verify Ollama is running and the model is available by running:
ollama list

You should see llama3.2 listed in the output. This confirms the model is downloaded and ready to use.

Don't see llama3.2 in the list?

Make sure the ollama pull llama3.2 command completed without errors. If you see a network error, check your internet connection and try again.

If Ollama reports it is not running, start it manually by searching for Ollama in your Start menu (Windows) or Applications (macOS) and opening it.

Help me get Ollama working

  • Confirm the Ollama API is accessible by visiting this URL in your browser:
http://localhost:11434

You should see a page that says "Ollama is running". This confirms the API is ready for your Python code to connect to.

Install Python libraries

Your project depends on four libraries. Each one handles a different part of the RAG pipeline.

  • Make sure your virtual environment is active (you should see (venv) in your terminal prompt). Then install all dependencies by running this command:
pip install chromadb pymupdf ollama streamlit

What do these libraries do?

  • chromadb is a vector database that stores your document chunks as embeddings and retrieves the most relevant ones when you ask a question.
  • pymupdf extracts text from PDF files page by page.
  • ollama is the Python client that lets your code talk to the Ollama server running on your machine.
  • streamlit is a framework that turns Python scripts into browser-based apps. You will use it to build the chat interface.

The installation may take a minute or two. Wait until you see the "Successfully installed" message.

  • Verify all four libraries imported correctly by running this command:
python -c "import chromadb; import pymupdf; import ollama; import streamlit; print('All imports successful!')"

You should see All imports successful! printed in your terminal.

Seeing an import error?

Make sure your virtual environment is active. Look for (venv) at the start of your terminal prompt. If it is missing, activate the environment again using the command from earlier in this step.

If a specific library failed to install, try installing it individually (for example, pip install chromadb) to see a more detailed error message.

Help me fix my import errors

Your environment is ready. Next up, you will build the document ingestion pipeline that turns your PDFs into searchable knowledge.

Build the Document Ingestion Pipeline

Your environment is set up with Python, Ollama, and all the libraries you need. Now it's time to build the core of your second brain: the document ingestion pipeline.

When you ask your AI a question, it needs to search through your documents for relevant information. Raw PDFs are too large to process all at once. The solution is to break them into small, searchable chunks and store them in a vector database.

In this step, get ready to:
  • Extract text from PDFs using PyMuPDF.
  • Split text into overlapping chunks for better retrieval.
  • Store chunks in a persistent ChromaDB vector database.
Extract text from PDFs

The first piece of the pipeline is reading PDF files and pulling out their text content. You'll also track which page each piece of text came from, so you can reference it later.

  • In VS Code, create a new file by pressing Ctrl+N (Windows) or Cmd+N (macOS).
  • Save the file immediately with Ctrl+S (Windows) or Cmd+S (macOS).
  • Name it ingest.py and save it in your project folder.

Every Python script starts with its imports and configuration. Add the following imports and constants to the top of ingest.py:

import os
import sys
import pymupdf
import chromadb

# Configuration constants
CHROMA_PATH = "./chroma_db"
COLLECTION_NAME = "second_brain"
CHUNK_SIZE = 500
CHUNK_OVERLAP = 100

What does this code do?

These constants control how your ingestion pipeline behaves.

  • CHROMA_PATH sets where your vector database files are stored on disk.
  • CHUNK_SIZE controls how many characters each text chunk contains (500 characters).
  • CHUNK_OVERLAP makes chunks overlap by 100 characters so you don't lose context at the boundaries.

You won't see any output yet. The next function uses these imports and constants.

  • Add the extract_text_from_pdf function below the constants in ingest.py:
def extract_text_from_pdf(pdf_path):
    """Extract text from a PDF file, returning a list of (page_number, text) tuples."""
    doc = pymupdf.open(pdf_path)
    pages = []
    # Loop through each page and grab its text
    for page_num, page in enumerate(doc, start=1):
        text = page.get_text()
        # Only keep pages that have actual content
        if text.strip():
            pages.append((page_num, text))
    doc.close()
    return pages

What does this function do?

This function opens a PDF file and extracts the text from every page. It returns a list of tuples pairing each page number with its text content.

The enumerate(doc, start=1) gives you human-readable page numbers starting from 1 instead of 0. Empty pages are skipped to avoid storing blank chunks later.

  • Save ingest.py.
  • Confirm ingest.py appears in the Explorer sidebar in VS Code.

Don't see ingest.py in the sidebar?

Make sure you saved the file inside your project folder (the same folder open in VS Code), not in a different location. If you accidentally saved it elsewhere, use File > Save As to move it to the correct folder.

Help me find where my file was saved

Chunk the text

A full page of text is still too large for effective retrieval. Splitting text into smaller, overlapping chunks gives the search engine more precise matches when you ask questions.

  • Add the chunk_text function below extract_text_from_pdf in ingest.py:
def chunk_text(text, chunk_size=CHUNK_SIZE, overlap=CHUNK_OVERLAP):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    # Slide a window across the text, stepping back by the overlap amount
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks

Why overlapping chunks?

Imagine a sentence that spans the boundary between two chunks. Without overlap, that sentence gets split in half and becomes unsearchable.

By overlapping each chunk by 100 characters, the important context near boundaries appears in both chunks. This gives the retrieval engine a better chance of finding relevant passages.

  • Save ingest.py.

You won't see output yet. The next section ties these functions together into the full ingestion pipeline.

Store chunks in ChromaDB

Now you need to connect the text extraction and chunking into a pipeline that stores everything in ChromaDB. ChromaDB handles embeddings automatically using its built-in model, so you don't need to run a separate embedding step.

  • Add the ingest_pdf function below chunk_text in ingest.py:
def ingest_pdf(pdf_path, collection):
    """Ingest a single PDF into the ChromaDB collection."""
    filename = os.path.basename(pdf_path)
    pages = extract_text_from_pdf(pdf_path)

    documents = []
    metadatas = []
    ids = []

    # Build lists of chunks with their metadata
    for page_num, text in pages:
        chunks = chunk_text(text)
        for i, chunk in enumerate(chunks):
            documents.append(chunk)
            metadatas.append({"source": filename, "page": page_num})
            ids.append(f"{filename}_p{page_num}_c{i}")

    # Add all chunks to ChromaDB in one batch
    if documents:
        collection.add(documents=documents, metadatas=metadatas, ids=ids)

    return len(documents)

What does this function do?

This function orchestrates the full ingestion of one PDF. It extracts text, chunks it, and stores everything in ChromaDB with metadata.

  • Each chunk gets a unique ID formatted as {filename}_p{page_num}_c{chunk_index} so you can trace it back to its source.
  • The metadatas list attaches the source filename and page number to each chunk for later reference.
  • ChromaDB automatically generates embeddings when you call collection.add() with raw text documents.

You won't see output yet. The next function ties everything together and runs the pipeline.

  • Add the main function and the entry point below ingest_pdf in ingest.py:
def main():
    docs_folder = "./docs"
    # Create the docs folder if it doesn't exist yet
    if not os.path.exists(docs_folder):
        os.makedirs(docs_folder)
        print(f"Created '{docs_folder}' folder. Add your PDF files there and run again.")
        return

    # Find all PDF files in the docs folder
    pdf_files = [f for f in os.listdir(docs_folder) if f.lower().endswith(".pdf")]
    if not pdf_files:
        print(f"No PDF files found in '{docs_folder}'. Add some PDFs and run again.")
        return

    # Connect to ChromaDB and get or create the collection
    client = chromadb.PersistentClient(path=CHROMA_PATH)
    collection = client.get_or_create_collection(name=COLLECTION_NAME)

    # Process each PDF file
    total_chunks = 0
    for pdf_file in pdf_files:
        pdf_path = os.path.join(docs_folder, pdf_file)
        num_chunks = ingest_pdf(pdf_path, collection)
        print(f"Ingested '{pdf_file}': {num_chunks} chunks")
        total_chunks += num_chunks

    print(f"\nDone! Total chunks stored: {total_chunks}")
    print(f"Collection now has {collection.count()} documents total.")


if __name__ == "__main__":
    main()

What does this code do?

The main function handles the full workflow. It checks for a ./docs folder, finds all PDFs inside, and processes each one.

  • chromadb.PersistentClient(path=CHROMA_PATH) creates a database that saves to disk so your data survives between runs.
  • get_or_create_collection either opens an existing collection or creates a fresh one named second_brain.
  • The if __name__ == "__main__" guard means main() only runs when you execute the script directly, not when importing it.
  • Save ingest.py.

Before you can run the script, you need a PDF to ingest.

  • Create a docs folder inside your project folder by running this command in your terminal:
mkdir docs
  • Place any PDF file into the docs folder. This can be a research paper, an ebook chapter, or any document you want to ask questions about.

Need a sample PDF?

If you don't have a PDF handy, search for any free research paper or public-domain document online and download it. Even a short 2-3 page document works perfectly for testing.

  • Run the ingestion script by running this command:
python ingest.py
  • You should see output showing how many chunks were stored from your PDF. The exact numbers depend on the length of your document.

What should I see?

You should see something like this (with different numbers based on your PDF):

Ingested 'your_document.pdf': 42 chunks

Done! Total chunks stored: 42
Collection now has 42 documents total.

The script also creates a ./chroma_db folder in your project directory. This folder contains your persistent vector database.

  • Confirm the chroma_db folder was created by running:
ls
  • You should see chroma_db, docs, and ingest.py in the output.

Seeing an error when running ingest.py?

  • If you see ModuleNotFoundError, make sure your virtual environment is still activated. You should see the environment name in parentheses at the start of your terminal prompt.
  • If you see No PDF files found, make sure you placed a PDF file inside the docs folder (not the project root).
  • If you see a permission error, make sure the PDF file is not open in another application.

Help me debug my ingest.py error

✔️ Awesome, I've got everything!

Great. Double check you've saved ingest.py and move on to the next step.

ⓧ I'd like to double check the full code

Here is the complete ingest.py file for comparison:

import os
import sys
import pymupdf
import chromadb

CHROMA_PATH = "./chroma_db"
COLLECTION_NAME = "second_brain"
CHUNK_SIZE = 500
CHUNK_OVERLAP = 100


def extract_text_from_pdf(pdf_path):
    """Extract text from a PDF file, returning a list of (page_number, text) tuples."""
    doc = pymupdf.open(pdf_path)
    pages = []
    for page_num, page in enumerate(doc, start=1):
        text = page.get_text()
        if text.strip():
            pages.append((page_num, text))
    doc.close()
    return pages


def chunk_text(text, chunk_size=CHUNK_SIZE, overlap=CHUNK_OVERLAP):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks


def ingest_pdf(pdf_path, collection):
    """Ingest a single PDF into the ChromaDB collection."""
    filename = os.path.basename(pdf_path)
    pages = extract_text_from_pdf(pdf_path)

    documents = []
    metadatas = []
    ids = []

    for page_num, text in pages:
        chunks = chunk_text(text)
        for i, chunk in enumerate(chunks):
            documents.append(chunk)
            metadatas.append({"source": filename, "page": page_num})
            ids.append(f"{filename}_p{page_num}_c{i}")

    if documents:
        collection.add(documents=documents, metadatas=metadatas, ids=ids)

    return len(documents)


def main():
    docs_folder = "./docs"
    if not os.path.exists(docs_folder):
        os.makedirs(docs_folder)
        print(f"Created '{docs_folder}' folder. Add your PDF files there and run again.")
        return

    pdf_files = [f for f in os.listdir(docs_folder) if f.lower().endswith(".pdf")]
    if not pdf_files:
        print(f"No PDF files found in '{docs_folder}'. Add some PDFs and run again.")
        return

    client = chromadb.PersistentClient(path=CHROMA_PATH)
    collection = client.get_or_create_collection(name=COLLECTION_NAME)

    total_chunks = 0
    for pdf_file in pdf_files:
        pdf_path = os.path.join(docs_folder, pdf_file)
        num_chunks = ingest_pdf(pdf_path, collection)
        print(f"Ingested '{pdf_file}': {num_chunks} chunks")
        total_chunks += num_chunks

    print(f"\nDone! Total chunks stored: {total_chunks}")
    print(f"Collection now has {collection.count()} documents total.")


if __name__ == "__main__":
    main()

Your ingestion pipeline is working. Next up, you'll build the query engine that searches these chunks and generates answers using your local AI model.

Build the RAG Query Engine

Your ingestion pipeline is working. Documents are chunked and stored in ChromaDB, ready to be searched. Now you need a way to actually ask questions about them and get useful answers back.

In this step, you'll build the retrieval and generation engine. This is the "R" and "G" in RAG. Your code will find the most relevant chunks for a question, then send those chunks to Ollama's local LLM to generate a grounded answer.

In this step, get ready to:
  • Connect to the ChromaDB collection and retrieve relevant document chunks.
  • Generate AI answers constrained to your document context using Ollama.
  • Test the full pipeline from the command line.
Query ChromaDB for relevant chunks

The retrieval step is what makes RAG different from a plain chatbot. Instead of sending your question directly to the AI, you first search your vector database for the most relevant pieces of text. The AI then answers using only that context.

  • Create a new file called rag.py in your second-brain project folder.
  • Add the imports, configuration constants, and retrieval functions by pasting this code:
import sys
import chromadb
from ollama import chat

CHROMA_PATH = "./chroma_db"
COLLECTION_NAME = "second_brain"
MODEL_NAME = "llama3.2"
TOP_K = 3


def get_collection():
    """Connect to the existing ChromaDB collection."""
    client = chromadb.PersistentClient(path=CHROMA_PATH)
    collection = client.get_or_create_collection(name=COLLECTION_NAME)
    return collection


def retrieve_context(question, collection, top_k=TOP_K):
    """Query ChromaDB for the most relevant chunks."""
    results = collection.query(query_texts=[question], n_results=top_k)
    documents = results["documents"][0]
    metadatas = results["metadatas"][0]
    return documents, metadatas

What does this code do?

The configuration constants at the top point to the same ChromaDB database and collection that ingest.py wrote to. TOP_K = 3 means you'll retrieve the 3 most relevant chunks per question.

The get_collection() function opens the persistent database connection. The retrieve_context() function uses ChromaDB's query_texts parameter to search by semantic similarity and returns both the document text and the metadata (source file and page number).

You won't see a change yet. The next chunk uses these functions to generate answers.

Generate answers with Ollama

Now you need a function that takes the retrieved chunks, formats them into a prompt, and sends everything to your local Llama 3.2 model. The system prompt constrains the model to answer only from the provided context.

  • In rag.py, add the generate_answer() function below your existing code:
def generate_answer(question, context_docs, context_metadatas):
    """Generate an answer using Ollama with retrieved context."""
    # Format each chunk with its source info
    context_parts = []
    for doc, meta in zip(context_docs, context_metadatas):
        source = meta.get("source", "unknown")
        page = meta.get("page", "?")
        context_parts.append(f"[Source: {source}, Page {page}]\n{doc}")

    context_text = "\n\n---\n\n".join(context_parts)

    # Instruct the model to answer ONLY from the provided context
    system_prompt = (
        "You are a helpful assistant that answers questions based ONLY on the "
        "provided context. If the context does not contain enough information to "
        "answer the question, say 'I don't have enough information in my documents "
        "to answer that.' Do not make up information."
    )

    user_prompt = f"Context:\n{context_text}\n\nQuestion: {question}"

    # Send the prompt to the local Llama 3.2 model via Ollama
    response = chat(
        model=MODEL_NAME,
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt},
        ],
    )

    return response.message.content

What does this code do?

  • The loop formats each retrieved chunk with its source filename and page number, separated by --- dividers. This gives the model clear boundaries between chunks.
  • The system_prompt tells the model to only use the provided context. If it can't find the answer, it admits it rather than hallucinating.
  • The chat() function from the ollama library sends both the system and user messages to your local llama3.2 model and returns the response.

Why constrain the model to the context?

Without the constraint, the model would happily answer any question using its training data. That defeats the purpose of a "second brain" that only knows what YOUR documents contain. The system prompt is what makes this a RAG system rather than a generic chatbot.

You won't see a change yet. The next chunk ties everything together into a runnable script.

Test the full RAG pipeline

You have retrieval and generation. Now you need an orchestrator function that connects them, plus a command-line entry point so you can test from your terminal.

  • In rag.py, add the ask() function and the __main__ block below your existing code:
def ask(question):
    """Full RAG pipeline: retrieve context and generate answer."""
    collection = get_collection()

    # Check if there are any documents to search
    if collection.count() == 0:
        return "No documents ingested yet. Run ingest.py first."

    # Retrieve relevant chunks, then generate an answer from them
    documents, metadatas = retrieve_context(question, collection)
    answer = generate_answer(question, documents, metadatas)
    return answer


if __name__ == "__main__":
    if len(sys.argv) < 2:
        print("Usage: python rag.py \"your question here\"")
        sys.exit(1)

    question = " ".join(sys.argv[1:])
    print(f"\nQuestion: {question}")
    print(f"\nAnswer: {ask(question)}")

What does this code do?

  • The ask() function is the full RAG pipeline in one call. It connects to ChromaDB, retrieves context, and generates an answer. It also checks for an empty collection so you get a helpful error instead of a crash.
  • The __main__ block lets you run the script from the command line with a question as an argument. It joins all arguments into one string so you can pass multi-word questions.
  • Save rag.py.
  • Test the full pipeline by running this command:
python rag.py "What is this document about?"

You should see your question printed, followed by an answer that references specific content from the PDF you ingested earlier. The answer should relate to your document, not general knowledge.

What should I see?

The first time you run this, Ollama may take 10-30 seconds to respond as it loads the model into memory. Subsequent questions will be faster. Your output will look something like:

Question: What is this document about?

Answer: Based on the provided context, this document discusses...

Not getting an answer?

  • Make sure Ollama is running. Check by visiting http://localhost:11434 in your browser. You should see "Ollama is running".
  • Confirm the llama3.2 model is pulled. Run ollama list and check that llama3.2 appears.
  • Make sure you ran python ingest.py first. The query engine needs documents in the collection to search.

Help me troubleshoot my RAG query engine. I ran python rag.py with a question but did not get an answer.

✔️ Awesome, I've got everything!

Great. Double check you've saved rag.py and move on to the next step.

ⓧ I'd like to double check the full code

Here's the complete rag.py file. Make sure yours matches:

import sys
import chromadb
from ollama import chat

CHROMA_PATH = "./chroma_db"
COLLECTION_NAME = "second_brain"
MODEL_NAME = "llama3.2"
TOP_K = 3


def get_collection():
    """Connect to the existing ChromaDB collection."""
    client = chromadb.PersistentClient(path=CHROMA_PATH)
    collection = client.get_or_create_collection(name=COLLECTION_NAME)
    return collection


def retrieve_context(question, collection, top_k=TOP_K):
    """Query ChromaDB for the most relevant chunks."""
    results = collection.query(query_texts=[question], n_results=top_k)
    documents = results["documents"][0]
    metadatas = results["metadatas"][0]
    return documents, metadatas


def generate_answer(question, context_docs, context_metadatas):
    """Generate an answer using Ollama with retrieved context."""
    context_parts = []
    for doc, meta in zip(context_docs, context_metadatas):
        source = meta.get("source", "unknown")
        page = meta.get("page", "?")
        context_parts.append(f"[Source: {source}, Page {page}]\n{doc}")

    context_text = "\n\n---\n\n".join(context_parts)

    system_prompt = (
        "You are a helpful assistant that answers questions based ONLY on the "
        "provided context. If the context does not contain enough information to "
        "answer the question, say 'I don't have enough information in my documents "
        "to answer that.' Do not make up information."
    )

    user_prompt = f"Context:\n{context_text}\n\nQuestion: {question}"

    response = chat(
        model=MODEL_NAME,
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_prompt},
        ],
    )

    return response.message.content


def ask(question):
    """Full RAG pipeline: retrieve context and generate answer."""
    collection = get_collection()

    if collection.count() == 0:
        return "No documents ingested yet. Run ingest.py first."

    documents, metadatas = retrieve_context(question, collection)
    answer = generate_answer(question, documents, metadatas)
    return answer


if __name__ == "__main__":
    if len(sys.argv) < 2:
        print("Usage: python rag.py \"your question here\"")
        sys.exit(1)

    question = " ".join(sys.argv[1:])
    print(f"\nQuestion: {question}")
    print(f"\nAnswer: {ask(question)}")

Your RAG pipeline is complete. You can ask questions about your documents from the command line and get grounded answers. Next up, you'll wrap this in a browser-based chat interface using Streamlit so you can upload PDFs and ask questions without touching the terminal.

Create the Streamlit Chat Interface

Your RAG query engine can retrieve context and generate answers from the command line. Now it is time to give your second brain a proper interface.

In this step, you will build a browser-based chat UI using Streamlit. The app lets you upload PDFs directly through the browser and ask questions in a familiar chat format.

In this step, get ready to:
  • Set up a Streamlit app with page config, title, and session state.
  • Wire up PDF ingestion and RAG queries in the interface.
  • Run and test the full chat experience in your browser.
Set up the Streamlit app

Streamlit apps are single Python files that re-run from top to bottom every time the user interacts with the page. You need to set the page title, initialize chat history in session state so it persists between re-runs, and lay out the basic structure.

  • Create a new file called app.py in your second-brain folder.
  • Add the following code to set up imports, page config, and session state:
import os
import tempfile
import streamlit as st
from ingest import ingest_pdf, CHROMA_PATH, COLLECTION_NAME
from rag import ask
import chromadb

# Configure the browser tab title and icon
st.set_page_config(page_title="Local Second Brain", page_icon="\ud83e\udde0")
st.title("Local Second Brain")
st.caption("Ask questions about your documents. Everything runs locally.")

# Initialize chat history so messages persist between re-runs
if "messages" not in st.session_state:
    st.session_state.messages = []

What does this code do?

The imports pull in your ingest_pdf function and ask function from the scripts you built in previous steps. This means the UI can call your existing pipeline code directly.

The st.set_page_config line sets the browser tab title and favicon. The st.session_state.messages list stores the entire chat history so messages don't disappear when Streamlit re-runs the script.

  • Save app.py.

Seeing an import error?

Make sure app.py is in the same folder as ingest.py and rag.py. The from ingest import ... line only works if all three files live in the same directory.

Help me fix my import error in app.py

Wire up ingestion and Q&A

The sidebar handles document uploads. When a PDF is uploaded, it gets saved to a temporary file, ingested into ChromaDB, and then cleaned up. The main area displays chat history and handles new questions.

  • Add the sidebar with the file uploader and ingestion logic below your session state initialization in app.py:
# Sidebar for uploading PDFs
with st.sidebar:
    st.header("Upload Documents")
    uploaded_files = st.file_uploader(
        "Drop your PDFs here",
        type=["pdf"],
        accept_multiple_files=True,
    )

    # Ingest each uploaded file into ChromaDB
    if uploaded_files:
        client = chromadb.PersistentClient(path=CHROMA_PATH)
        collection = client.get_or_create_collection(name=COLLECTION_NAME)

        for uploaded_file in uploaded_files:
            with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as tmp:
                tmp.write(uploaded_file.getbuffer())
                tmp_path = tmp.name

            num_chunks = ingest_pdf(tmp_path, collection)
            st.success(f"Ingested '{uploaded_file.name}': {num_chunks} chunks")
            os.unlink(tmp_path)

    # Show how many chunks are in the knowledge base
    client = chromadb.PersistentClient(path=CHROMA_PATH)
    collection = client.get_or_create_collection(name=COLLECTION_NAME)
    st.info(f"Documents in knowledge base: {collection.count()} chunks")

What does this code do?

The st.file_uploader widget creates a drag-and-drop zone for PDFs in the sidebar. When files are uploaded, each one is written to a temporary file (since Streamlit's uploaded files are in-memory objects, not disk files).

Your ingest_pdf function processes the temp file and adds chunks to ChromaDB. After ingestion, os.unlink deletes the temporary file to keep things clean. The st.info line at the bottom always shows the current total chunk count.

  • Now add the chat display and input handling below the sidebar block in app.py:
# Display all previous messages from chat history
for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

# Handle new user input
if prompt := st.chat_input("Ask a question about your documents..."):
    # Add and display the user message
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    # Generate and display the assistant response
    with st.chat_message("assistant"):
        with st.spinner("Thinking..."):
            answer = ask(prompt)
        st.markdown(answer)

    # Save the assistant response to history
    st.session_state.messages.append({"role": "assistant", "content": answer})

What does this code do?

The for loop at the top re-renders all previous messages every time the page re-runs. This is how Streamlit maintains the appearance of a persistent chat.

When the user types a question, st.chat_input captures it. The code appends the message to session state, displays it, then calls your ask() function from rag.py to get an answer. The st.spinner shows a loading indicator while Ollama generates the response.

  • Save app.py.

Not sure where each block goes?

The sidebar code (with st.sidebar:) goes directly after the st.session_state.messages = [] initialization. The chat display and input code goes after the sidebar block closes (at the same indentation level as with st.sidebar:).

Help me structure my app.py correctly

✔️ Awesome, I've got everything!

Great. Double check you have saved app.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete app.py file:

import os
import tempfile
import streamlit as st
from ingest import ingest_pdf, CHROMA_PATH, COLLECTION_NAME
from rag import ask
import chromadb

st.set_page_config(page_title="Local Second Brain", page_icon="\ud83e\udde0")
st.title("Local Second Brain")
st.caption("Ask questions about your documents. Everything runs locally.")

if "messages" not in st.session_state:
    st.session_state.messages = []

with st.sidebar:
    st.header("Upload Documents")
    uploaded_files = st.file_uploader(
        "Drop your PDFs here",
        type=["pdf"],
        accept_multiple_files=True,
    )

    if uploaded_files:
        client = chromadb.PersistentClient(path=CHROMA_PATH)
        collection = client.get_or_create_collection(name=COLLECTION_NAME)

        for uploaded_file in uploaded_files:
            with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as tmp:
                tmp.write(uploaded_file.getbuffer())
                tmp_path = tmp.name

            num_chunks = ingest_pdf(tmp_path, collection)
            st.success(f"Ingested '{uploaded_file.name}': {num_chunks} chunks")
            os.unlink(tmp_path)

    client = chromadb.PersistentClient(path=CHROMA_PATH)
    collection = client.get_or_create_collection(name=COLLECTION_NAME)
    st.info(f"Documents in knowledge base: {collection.count()} chunks")

for message in st.session_state.messages:
    with st.chat_message(message["role"]):
        st.markdown(message["content"])

if prompt := st.chat_input("Ask a question about your documents..."):
    st.session_state.messages.append({"role": "user", "content": prompt})
    with st.chat_message("user"):
        st.markdown(prompt)

    with st.chat_message("assistant"):
        with st.spinner("Thinking..."):
            answer = ask(prompt)
        st.markdown(answer)

    st.session_state.messages.append({"role": "assistant", "content": answer})
Run and test the app

Streamlit has its own development server. You launch it with a special command instead of running the Python file directly.

  • Launch the Streamlit app by running this command:
streamlit run app.py

Why not python app.py?

Streamlit needs its own server to handle re-runs, session state, and the browser connection. Running python app.py directly skips this server and causes session state errors.

Your default browser opens automatically to a page showing the app title and a chat input at the bottom.

  • In the left sidebar, click the **Browse files** button and upload a PDF document.
  • Wait for the green success message confirming ingestion (e.g. "Ingested 'yourfile.pdf': 42 chunks").
  • Type a question about your document in the chat input at the bottom and press **Enter**.
  • Verify you see a "Thinking..." spinner followed by an answer that references content from your uploaded PDF.

App not opening in your browser?

Check the terminal output for a line that says "Local URL". Copy that URL (typically http://localhost:8501) and paste it into your browser manually.

If you see an error about session state, make sure you used streamlit run app.py and not python app.py. Streamlit's session state only works when launched through the Streamlit CLI.

Help me get my Streamlit app running

Your second brain now has a real interface. You can upload documents, ask questions, and get AI-powered answers grounded in your own files. Next up, take on the Secret Mission to add source citations with page numbers to every answer.

Secret mission

Add Source Citations with Page Numbers

Your second brain answers questions, but how do you know where the information came from? In this mission, you'll add source citations with page numbers to every answer so you can verify the AI's claims against the original document.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing cloud costs. Decide whether to keep your resources on disk, pause Ollama to free up memory, or delete everything entirely.

Resources you used:

  • Your project folder on the Desktop (contains ingest.py, rag.py, app.py, the ./docs folder, and the ./chroma_db vector database).
  • Ollama llama3.2 model (~2 GB stored in your home directory under Ollama's default model storage).
  • Ollama application (installed via OllamaSetup.exe).

Keep everything running

No action needed. Choose this if you plan to keep adding documents to your second brain or want to ask more questions later.

  • Your ./chroma_db vector database stays on disk, ready to query anytime you run the app again.
  • The llama3.2 model remains available for instant use without re-downloading.
  • Ollama continues running in the background. It uses minimal resources when idle.

Pause - I'll come back to this later

Stop the Ollama background process to free up memory while keeping all files on disk for later.

  • Right-click the Ollama icon in your system tray (bottom-right corner of your taskbar).
  • Click Quit Ollama.
  • Your project folder, vector database, and downloaded model all stay on disk. Relaunch Ollama when you want to use your second brain again.

Delete - I don't want to use this again

Remove all project files, the vector database, and the downloaded model to reclaim disk space.

  • Remove the llama3.2 model (~2 GB) by running this command in your terminal:
  • Delete your project folder from the Desktop. This removes all Python files, the ./docs folder, the ./chroma_db database, and your virtual environment.
  • To uninstall the Ollama application itself, open Settings on Windows, go to Apps, find Ollama in the list, and click Uninstall.
ollama rm llama3.2

Delete your project folder from the Desktop by running this command:

macOS

cd ~/Desktop
rm -rf [[PROJECT_FOLDER="your project folder name"]]

Windows

cd ~\Desktop
Remove-Item -Recurse -Force [[PROJECT_FOLDER="your project folder name"]]

Nice Work!

Nice Work!

Nice work! You just built a fully local, private AI assistant that answers questions from your own documents.

You've learned how to:

  • Build a document ingestion pipeline that extracts text from PDFs, chunks it into overlapping segments, and stores embeddings in ChromaDB.
  • Create a RAG query engine that retrieves relevant context from a vector database and generates grounded answers using a local LLM through Ollama.
  • Build a Streamlit chat interface that ties together PDF upload, ingestion, and question-answering in a single browser-based app.
  • Secret Mission: Added source citations with page numbers so you can verify every answer against the original document.

Ready to quiz yourself?