DocuQuery: Multi-Document RAG Assistant

Introduction

30 Second Summary

When you research a topic across multiple documents, you constantly flip between sources trying to remember which paper said what. AI can help, but most basic implementations just dump everything into one prompt and hope the model figures it out.

In this project, you will build DocuQuery, a multi-document research assistant that synthesizes answers across multiple PDFs with cross-document citations. You'll engineer the context window and prompt templates so precisely that the app not only works, it shows HOW it works.

What You'll Build

You upload 2-3 PDF documents, ask a question that spans all of them, and watch DocuQuery return a grounded answer that cites exactly which document, page, and passage supports each claim.

By the end of this project, you'll have:

• A live research assistant at a public URL where you upload PDFs, ask cross-document questions, and get answers with specific citations (document name + page number) for every claim.
• A visible context engineering pipeline with a sidebar showing token budget allocation, retrieval scores, re-ranking decisions, and the final assembled context window.
• An engineered prompt template with XML-tagged sections, JSON schema output, and few-shot examples, all inspectable through a "Prompt Lab" panel that employers can review.
• Secret Mission: Add conversation compaction that summarizes Q&A history into persistent research notes, demonstrating the "compress" technique from context engineering.

Are there any prerequisites?

You'll need basic Python knowledge, a Google account (for the free Gemini API), and a GitHub account. No credit card is required for any part of this project.

Before We Start

You need the tools installed and an API key configured before you can build anything. Getting a visible response from the Gemini API confirms your environment works end-to-end.

In this step, you will create a project folder, install the Python libraries your app needs, get a free API key, and verify everything is connected by making your first call to Gemini.

In this step, get ready to:

• Create a project folder, virtual environment, and install dependencies.
• Get a free Gemini API key and verify it works.
• Create the project file structure.

Create your project folder and install dependencies

Every Python project starts with its own folder and virtual environment to keep dependencies isolated from the rest of your machine.

• Open your terminal.
• Move to your Desktop by running this command:

cd ~/Desktop

• Create your project folder and move into it by running these commands:

mkdir docuquery
cd docuquery

• Create a Python virtual environment by running:

python3 -m venv venv

• Activate the virtual environment.

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt.

Don't see (venv) in your prompt?

Make sure you ran the activate command from inside the docuquery folder. If you see a "file not found" error, confirm the venv folder exists by running ls.

Why isn't my virtual environment activating?

• Create the requirements.txt file that lists your project dependencies by running:

cat > requirements.txt << 'EOF'
google-genai==2.14.0
chromadb==1.5.9
streamlit==1.60.0
pymupdf==1.28.0
EOF

What are these libraries?

• google-genai is Google's official SDK for calling the Gemini API.
• chromadb is a vector database that handles embedding and retrieval automatically.
• streamlit is a Python framework for building interactive web apps with minimal code.
• pymupdf extracts text from PDF files quickly and accurately.

• Install all four dependencies by running:

pip install -r requirements.txt

This takes a minute or two. You should see a "Successfully installed" message listing all four packages when it finishes.

Seeing installation errors?

Make sure (venv) appears in your prompt before running pip install. If you skipped activation, packages install globally and may conflict with system Python.

If you see a compiler error on Windows, chromadb may need the Microsoft C++ Build Tools. Follow the error message link to install them, then retry.

Help me troubleshoot my pip install error.

Get a free Gemini API key and verify your setup

The Gemini API free tier requires no credit card and gives you enough capacity for development and demo usage.

• Open Google AI Studio in your browser and sign in with your Google account.
• Click Get API key in the left sidebar.
• Click Create API key. Google AI Studio creates a default project and generates a key for you automatically.
• Copy the key and keep it somewhere safe. You will only see it once.

Why is this free?

Gemini 2.5 Flash input and output tokens are free of charge on the Free Tier. No credit card or billing setup is required. Your project makes only 1-2 API calls per question, so you will stay well within the free limits.

• Set the API key as an environment variable in your terminal so your code can read it.

macOS

export GEMINI_API_KEY="your-api-key-here"

Windows

$env:GEMINI_API_KEY = "your-api-key-here"

Keep your key out of code

Replace your-api-key-here with the key you just copied. Never paste API keys directly into source files. Using an environment variable keeps the secret separate from your code.

• Create a test script to verify everything is connected. Create a file called test_setup.py and add this code:

import os
from google import genai

client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Say hello and confirm you are working. Keep it to one sentence.",
)

print("Gemini says:", response.text)
print("\nSetup complete! Your environment is ready.")

What does this code do?

• from google import genai imports the official google-genai SDK (not the deprecated google-generativeai package).
• genai.Client(api_key=...) creates a client that authenticates your requests with the API key from your environment variable.
• client.models.generate_content(...) sends a prompt to Gemini 2.5 Flash and returns the model's text response.

Before you run this, what do you expect to see if your API key is set correctly?

• Run the test script:

python test_setup.py

You should see Gemini respond with a short greeting followed by "Setup complete! Your environment is ready." in your terminal.

Seeing an authentication error?

If you see a permissions or authentication error, double-check that you exported the GEMINI_API_KEY variable in the same terminal session where you are running the script. Opening a new terminal tab resets environment variables.

Confirm the variable is set by running echo $GEMINI_API_KEY (macOS) or echo $env:GEMINI_API_KEY (Windows PowerShell). You should see your key printed.

Help me fix my Gemini API key error.

✔️ Awesome, I've got everything!

Great. Make sure you have saved test_setup.py and confirmed Gemini responded in your terminal.

ⓧ I'd like to double check the full code

import os
from google import genai

client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

response = client.models.generate_content(
    model="gemini-2.5-flash",
    contents="Say hello and confirm you are working. Keep it to one sentence.",
)

print("Gemini says:", response.text)
print("\nSetup complete! Your environment is ready.")

Create the project file structure

Your project needs several Python files, a folder for PDF documents, a directory for Agent Skills, and a Streamlit secrets file for later deployment.

• Create all the project files and directories by running these commands:

touch app.py rag.py context_engine.py prompts.py evaluate.py
mkdir -p docs .skills .streamlit

• Confirm everything was created by running:

ls

You should see app.py, rag.py, context_engine.py, prompts.py, evaluate.py, requirements.txt, test_setup.py, and the docs/, .skills/, and .streamlit/ directories listed.

• Create the Streamlit secrets file for deployment later by running:

cat > .streamlit/secrets.toml << 'EOF'
GEMINI_API_KEY = "your-api-key-here"
EOF

What is secrets.toml for?

When you deploy to Streamlit Community Cloud later, this file tells the platform which secrets your app needs. You will paste your real key into the cloud dashboard at deploy time. For now, the placeholder is just a reminder of the format.

• Add 2-3 sample PDF documents to the docs/ folder. These can be any research papers, reports, or articles you have on hand. The app will use these to answer cross-document questions.

No PDFs handy?

Search for open-access research papers on a topic you are interested in. Download 2-3 short papers (under 20 pages each) as PDFs and save them into your docs/ folder.

Your environment is ready and Gemini is responding. Next up, you will build a naive multi-document Q&A so you can see it work (and see where it falls short).

Build a Naive Multi-Document Q&A

Your environment is working and Gemini is responding. Now it is time to build the actual app.

In this step, you will build a first version of DocuQuery that lets you upload PDFs and ask questions about them. The approach is simple: extract all the text, dump it into one prompt, and let Gemini figure it out. This gives you a working app fast, but you will quickly see where the naive approach falls short.

In this step, get ready to:

• Build a Streamlit UI with a multi-file PDF uploader and question input.
• Extract text from PDFs and send everything to Gemini in one prompt.
• Test a cross-document question and observe how the naive approach struggles.

Write the Streamlit UI with file uploader

Every Streamlit app starts with page configuration and input widgets. You need a way for users to upload multiple PDFs and type a question.

• Open app.py in VS Code and add the following code:

import os
import tempfile
import streamlit as st
from google import genai
import pymupdf

# Configure the Streamlit page
st.set_page_config(page_title="DocuQuery", page_icon="\ud83d\udcc4", layout="wide")

# Get API key from environment or Streamlit secrets
api_key = os.environ.get("GEMINI_API_KEY", "")
if not api_key:
    try:
        api_key = st.secrets["GEMINI_API_KEY"]
    except (KeyError, FileNotFoundError):
        st.error("Please set the GEMINI_API_KEY environment variable.")
        st.stop()

# Create the Gemini client
client = genai.Client(api_key=api_key)

# App header
st.title("DocuQuery")
st.caption("Multi-document research assistant")

# Multi-file PDF uploader
uploaded_files = st.file_uploader(
    "Upload PDF documents (2-3 recommended)", type=["pdf"], accept_multiple_files=True
)

What does this code do?

• st.set_page_config sets the browser tab title and page layout before any other Streamlit calls.
• The API key lookup checks the environment variable first, then falls back to Streamlit's secrets system. This lets the app work both locally and when deployed later.
• st.file_uploader with accept_multiple_files=True allows uploading several PDFs at once.

• Save app.py.
• Run the app by entering this command in your terminal:

streamlit run app.py

What happens when you run this?

Streamlit starts a local development server and automatically opens your browser to http://localhost:8501. The command takes over your terminal. To stop the server later, press Ctrl+C in the terminal.

You should see a page titled "DocuQuery" with a file upload area. The app is alive.

Don't see the app in your browser?

• If your browser did not open automatically, visit http://localhost:8501 manually.
• If you see an error about GEMINI_API_KEY, confirm the environment variable is still set in the terminal where you ran the command.
• Make sure your virtual environment is activated in the same terminal session.

help me debug why my Streamlit app is not loading at localhost:8501

Add PDF extraction and send everything to Gemini

Now you need the logic that runs when the user uploads files and asks a question. The naive approach extracts all text from every PDF and dumps it into a single prompt.

• In app.py, add the following code directly below the uploaded_files line:

if uploaded_files:
    question = st.text_input("Ask a research question across your documents:")
    if question:
        # Extract text from all uploaded PDFs
        all_text = ""
        for uploaded_file in uploaded_files:
            with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as tmp:
                tmp.write(uploaded_file.read())
                tmp_path = tmp.name
            doc = pymupdf.open(tmp_path)
            for page in doc:
                # Tag each page's text with its source document name
                all_text += f"\n[{uploaded_file.name}]\n{page.get_text()}"
            doc.close()
        # Build a simple prompt with all text concatenated
        prompt = f"Answer this question based on the documents:\n\n{all_text[:8000]}\n\nQuestion: {question}"
        with st.spinner("Generating answer..."):
            response = client.models.generate_content(
                model="gemini-2.5-flash",
                contents=prompt,
            )
        # Display the raw answer
        st.subheader("Answer")
        st.write(response.text)

What does this code do?

• PyMuPDF opens each uploaded PDF and extracts text page by page. Each page's text is tagged with the document filename so Gemini knows which document it came from.
• all_text[:8000] truncates the concatenated text to 8000 characters. This is a naive safety valve to avoid sending an enormous prompt.
• The entire text dump plus the question goes into one client.models.generate_content call with no retrieval, ranking, or structure.

• Save app.py.
• Switch to your browser. Streamlit automatically reloads when you save.
• Upload one of the sample PDFs from your docs/ folder.
• Type a simple question about that single document and press Enter.

You should see a spinner followed by an answer from Gemini. The app works.

Seeing an error after asking a question?

• If you see a FileNotFoundError, make sure the PDF files in your docs/ folder are valid PDF files (not renamed text files).
• If you get an API error, confirm GEMINI_API_KEY is set in the terminal session running Streamlit. You may need to stop the server with Ctrl+C, re-export the key, and run streamlit run app.py again.

help me fix an error when my Streamlit app tries to read a PDF and call Gemini

✔️ Awesome, I've got everything!

Great. Double check you have saved app.py and that Streamlit is still running in your terminal.

ⓧ I'd like to double check the full code

Your app.py should contain two sections in order:

• The imports, page config, API key handling, Gemini client, title, caption, and file_uploader (first chunk above).
• The if uploaded_files: block with question input, PDF extraction loop, prompt construction, Gemini call, and answer display (second chunk above).

Scroll up and confirm both chunks are present, in order, with no extra code between them.

Test with a cross-document question

The app handles single-document questions fine. The real test is asking something that requires synthesizing information across multiple documents.

• Upload at least two of the sample PDFs from your docs/ folder.
• Type a question that requires comparing both documents, such as: "What do both documents say about X?" or "Where do these documents agree or disagree?"

Before you press Enter, what do you think the answer will look like? Do you expect specific page references and document names in the response?

• Press Enter and read the answer carefully.

You will notice the answer is vague. It might say "both documents mention..." without telling you exactly where. It might conflate information from different sources or skip citations entirely. This is the limitation of dumping raw text into a prompt with no retrieval, no ranking, and no structured instructions.

Why does the naive approach produce weak answers?

Three problems are at play. First, you are sending up to 8000 characters of unfiltered text. Most of it is irrelevant to the question, which dilutes the signal the model can use.

Second, the model has no structured instructions about how to cite sources. It sees document name tags in the text but has no rules forcing it to reference them.

Third, there is no retrieval step. The model gets everything or nothing. It cannot focus on the most relevant passages. These are exactly the problems that context engineering and prompt engineering solve in the next steps.

Your first version of DocuQuery is running. It answers questions, but it cannot pinpoint which document or page its claims come from. In the next step, you will build a proper RAG pipeline with context engineering that retrieves only the most relevant chunks, tags them with provenance, and assembles them within a token budget.

Add the RAG Pipeline with Context Engineering

In Step 2, you saw the naive approach in action. Dumping entire documents into one prompt wastes tokens on irrelevant paragraphs and the model loses track of which information came from where.

A proper context engineering pipeline fixes this by selecting only high-signal chunks, tagging them with provenance (document name + page number), and assembling them within a strict token budget. In this step, you will build that pipeline and make every decision visible in a sidebar.

In this step, get ready to:

• Build a RAG module that extracts, chunks, stores, and retrieves document content with provenance metadata.
• Build a context engine that re-ranks chunks, enforces a token budget, and reports what was selected or pruned.
• Update the app to use the pipeline and display a Context Engineering sidebar.

Build the multi-document RAG pipeline

The RAG pipeline lives in a dedicated file called rag.py. It handles four responsibilities: extracting text page-by-page, chunking text with overlap, storing chunks in a vector database, and retrieving the most relevant chunks for a query.

• Create a new file called rag.py in your project folder (the same folder as app.py).
• Add the imports and the extraction function by pasting this code:

import pymupdf
import chromadb


def extract_text_by_page(pdf_path, doc_name):
    """Extract text from a PDF, returning list of chunk-ready page dicts."""
    doc = pymupdf.open(pdf_path)
    pages = []
    for page_num, page in enumerate(doc, start=1):
        text = page.get_text()
        if text.strip():
            pages.append({"doc_name": doc_name, "page": page_num, "text": text})
    doc.close()
    return pages

What does this code do?

This function opens a PDF and loops through every page. For each page that has text, it records the document name, page number, and the extracted text in a dictionary. This metadata is what lets the app cite sources later.

• Save rag.py.
• Confirm the module loads by running:

python -c "from rag import extract_text_by_page; print('Extraction function loaded.')"

You should see Extraction function loaded. printed with no errors.

• Next, add the chunking function below extract_text_by_page in rag.py:

def chunk_pages(pages, chunk_size=500, overlap=50):
    """Split page text into overlapping chunks with provenance metadata."""
    chunks = []
    for page_data in pages:
        text = page_data["text"]
        start = 0
        while start < len(text):
            end = start + chunk_size
            chunk = text[start:end]
            if chunk.strip():
                chunks.append({
                    "text": chunk,
                    "doc_name": page_data["doc_name"],
                    "page": page_data["page"],
                })
            start += chunk_size - overlap
    return chunks

What does this code do?

Chunking breaks each page into smaller overlapping pieces of 500 characters. The 50-character overlap ensures that sentences split across chunk boundaries still appear in at least one complete chunk.

Each chunk inherits the doc_name and page from its parent page. This provenance travels with the chunk all the way to the final answer.

• Save rag.py and confirm both functions load:

python -c "from rag import extract_text_by_page, chunk_pages; print('Extraction + chunking ready.')"

You should see Extraction + chunking ready. with no errors.

Seeing a ModuleNotFoundError?

• Make sure you are running the command from the same folder where rag.py lives.
• Confirm your virtual environment is activated. If not, activate it again before running.

Help me debug my import error for rag.py

• Now add the function that stores chunks in ChromaDB. Paste this below chunk_pages in rag.py:

def create_collection(chunks, collection_name="documents"):
    """Store chunks in ChromaDB with provenance metadata."""
    client = chromadb.Client()

    existing = client.list_collections()
    for col in existing:
        if col.name == collection_name:
            client.delete_collection(collection_name)
            break

    collection = client.create_collection(name=collection_name)

    documents = [chunk["text"] for chunk in chunks]
    metadatas = [{"doc_name": chunk["doc_name"], "page": chunk["page"]} for chunk in chunks]
    ids = [f"chunk_{i}" for i in range(len(chunks))]

    collection.add(documents=documents, metadatas=metadatas, ids=ids)

    return collection

What does this code do?

ChromaDB is a vector database that handles embedding automatically using its built-in model. You do not need to call any embedding API yourself.

This function creates an in-memory collection, deleting any existing one with the same name first. It stores each chunk's text alongside its provenance metadata (doc_name and page) so that retrieval results always know where they came from.

• Save rag.py.
• Add the retrieval function below create_collection:

def retrieve(collection, query, n_results=5):
    """Retrieve relevant chunks with scores."""
    results = collection.query(query_texts=[query], n_results=n_results)

    retrieved = []
    for i in range(len(results["documents"][0])):
        retrieved.append({
            "text": results["documents"][0][i],
            "doc_name": results["metadatas"][0][i]["doc_name"],
            "page": results["metadatas"][0][i]["page"],
            "distance": results["distances"][0][i],
        })

    return retrieved

What does this code do?

ChromaDB's query method finds the chunks most semantically similar to the user's question. The distance score tells you how close each chunk is to the query. Lower distance means higher relevance.

The function packages each result with its text, provenance, and distance score into a clean list that the context engine will use next.

• Save rag.py and verify the complete module loads:

python -c "from rag import extract_text_by_page, chunk_pages, create_collection, retrieve; print('RAG pipeline ready.')"

You should see RAG pipeline ready. printed.

Seeing an import error for chromadb?

• Confirm chromadb is installed in your virtual environment by running pip list | grep chromadb to check.
• If it is missing, re-run pip install chromadb==1.5.9 with your virtual environment activated.

Help me fix my chromadb import error

✔️ Awesome, I've got everything!

Great. Double check you have saved rag.py with all four functions.

ⓧ I'd like to double check the full code

Your rag.py should contain these four functions in order. Scroll up and confirm each one is present:

• extract_text_by_page (opens a PDF, returns page dicts with doc_name, page, and text).
• chunk_pages (splits pages into overlapping 500-character chunks with metadata).
• create_collection (stores chunks in ChromaDB with provenance).
• retrieve (queries ChromaDB and returns ranked results with distances).

Build the context assembly engine

Retrieval alone is not enough. You need a layer that decides which chunks fit within a token budget and drops the rest. This is the core of context engineering: managing exactly what the model sees.

• Create a new file called context_engine.py in your project folder.
• Paste the token estimator and the selection logic:

def estimate_tokens(text):
    """Rough token estimate: ~4 characters per token."""
    return len(text) // 4


def assemble_context(retrieved_chunks, token_budget=3000, system_tokens=500, question_tokens=100):
    """Re-rank, prune to budget, and assemble context with provenance tags."""
    available_budget = token_budget - system_tokens - question_tokens

    sorted_chunks = sorted(retrieved_chunks, key=lambda c: c["distance"])

    selected = []
    tokens_used = 0
    pruned = []

    for chunk in sorted_chunks:
        chunk_tokens = estimate_tokens(chunk["text"])
        if tokens_used + chunk_tokens <= available_budget:
            selected.append(chunk)
            tokens_used += chunk_tokens
        else:
            pruned.append(chunk)

What does this code do?

The estimate_tokens function gives a rough count (4 characters per token) that is fast enough for budgeting without needing an external tokenizer.

The assembly function sorts chunks by distance (lowest first, meaning highest relevance). It then greedily adds chunks until the available budget is exhausted. Any chunk that would overflow gets placed in the pruned list instead.

• Save context_engine.py.
• Now add the rest of the function directly below the pruned.append(chunk) line (still inside assemble_context):

    context_parts = []
    for i, chunk in enumerate(selected, start=1):
        context_parts.append(
            f"[Source {i}] (Document: {chunk['doc_name']}, Page {chunk['page']}):\n{chunk['text']}"
        )

    assembled_context = "\n\n".join(context_parts)

    budget_report = {
        "total_budget": token_budget,
        "system_tokens": system_tokens,
        "question_tokens": question_tokens,
        "context_tokens_used": tokens_used,
        "context_tokens_available": available_budget,
        "chunks_selected": len(selected),
        "chunks_pruned": len(pruned),
    }

    return assembled_context, selected, pruned, budget_report

What does this code do?

This section formats each selected chunk with a [Source N] tag and its provenance (document name + page). The model can then cite specific sources in its answer.

The budget_report dictionary captures how the budget was spent. You will display this in the sidebar so anyone reviewing your app can see your engineering decisions in action.

• Save context_engine.py and verify the module loads:

python -c "from context_engine import assemble_context, estimate_tokens; print('Context engine ready.')"

You should see Context engine ready. printed with no errors.

Seeing an IndentationError?

• The second code block must be indented inside assemble_context. Make sure the context_parts = [] line starts with 4 spaces, just like the for chunk in sorted_chunks: line above it.
• Check that there is no extra blank line between the two code sections that breaks the function body.

Help me fix my indentation error in context_engine.py

✔️ Awesome, I've got everything!

Great. Make sure context_engine.py is saved with both estimate_tokens and assemble_context as one continuous file.

ⓧ I'd like to double check the full code

Your context_engine.py should contain these two functions. Scroll up and confirm they form one continuous file:

• estimate_tokens (returns character count divided by 4).
• assemble_context (sorts by distance, selects chunks within the token budget, prunes the rest, formats with [Source N] tags, and returns the assembled context plus a budget report).

Update the app to use the pipeline and display context decisions

Now you will replace the naive approach in app.py with your new RAG pipeline. The app will gain a two-column layout: the main column for Q&A and a sidebar that exposes every context engineering decision.

• Open app.py and delete all of its current contents.
• Paste the new imports, configuration, and layout setup:

import os
import tempfile
import streamlit as st
from google import genai
from rag import extract_text_by_page, chunk_pages, create_collection, retrieve
from context_engine import assemble_context, estimate_tokens

st.set_page_config(page_title="DocuQuery", page_icon="📄", layout="wide")

api_key = os.environ.get("GEMINI_API_KEY", "")
if not api_key:
    try:
        api_key = st.secrets["GEMINI_API_KEY"]
    except (KeyError, FileNotFoundError):
        st.error("Please set the GEMINI_API_KEY environment variable.")
        st.stop()

client = genai.Client(api_key=api_key)

if "research_notes" not in st.session_state:
    st.session_state["research_notes"] = []

st.title("DocuQuery")
st.caption("Multi-document research assistant with context engineering")

col_main, col_sidebar = st.columns([2, 1])

What changed from the old app.py?

• The imports now pull from your new rag and context_engine modules instead of using pymupdf directly.
• Session state is initialized for research_notes (used in a later step).
• A two-column layout (col_main and col_sidebar) replaces the single-column view so you can show engineering decisions alongside answers.

• Save app.py.
• Below the code you just pasted, add the file upload and processing logic inside the main column:

with col_main:
    uploaded_files = st.file_uploader(
        "Upload PDF documents (2-3 recommended)", type=["pdf"], accept_multiple_files=True
    )

    if uploaded_files:
        file_names = sorted([f.name for f in uploaded_files])
        current_key = "|".join(file_names)

        if st.session_state.get("files_key") != current_key:
            with st.spinner("Processing documents..."):
                all_chunks = []
                for uploaded_file in uploaded_files:
                    with tempfile.NamedTemporaryFile(delete=False, suffix=".pdf") as tmp:
                        tmp.write(uploaded_file.read())
                        tmp_path = tmp.name
                    pages = extract_text_by_page(tmp_path, uploaded_file.name)
                    chunks = chunk_pages(pages)
                    all_chunks.extend(chunks)

                collection = create_collection(all_chunks)
                st.session_state["collection"] = collection
                st.session_state["all_chunks"] = all_chunks
                st.session_state["files_key"] = current_key
                st.session_state["research_notes"] = []

            st.success(f"Processed {len(uploaded_files)} documents into {len(all_chunks)} chunks.")

What does this code do?

The files_key check prevents re-processing when you ask multiple questions against the same documents. Only when the set of uploaded files changes does the app re-extract, re-chunk, and re-store everything in ChromaDB.

• Save app.py and run it to confirm the layout loads:

streamlit run app.py

You should see a two-column page with the file uploader on the left and an empty right column. Upload your PDFs to confirm the processing spinner appears and a success message shows the chunk count.

App crashes with ImportError?

• Check that rag.py and context_engine.py are in the same folder as app.py.
• Confirm you ran streamlit run app.py from the project folder (not a parent directory).

Help me fix my Streamlit import error

• Stop the app with Ctrl+C in the terminal.
• Add the question handling and response logic below the st.success line (still inside the if uploaded_files: block):

        question = st.text_input("Ask a research question across your documents:")

        if question:
            collection = st.session_state["collection"]
            retrieved_chunks = retrieve(collection, question, n_results=5)

            assembled_context, selected, pruned, budget_report = assemble_context(
                retrieved_chunks, token_budget=3000
            )

            prompt = f"Answer this question based on the sources below with citations:\n\n{assembled_context}\n\nQuestion: {question}"

            with st.spinner("Generating answer..."):
                response = client.models.generate_content(
                    model="gemini-2.5-flash",
                    contents=prompt,
                )

            st.subheader("Answer")
            st.write(response.text)

What does this code do?

Instead of dumping all document text into the prompt, the app now retrieves only the 5 most relevant chunks using ChromaDB, assembles them within a 3000-token budget using your context engine, and sends that focused context to Gemini.

The prompt now includes [Source N] tagged chunks, which encourages the model to cite them in its response.

• Save app.py.
• Now add the sidebar that displays all context engineering decisions. Paste this at the bottom of app.py:

with col_sidebar:
    st.markdown("### Context Engineering")

    if uploaded_files and question:
        st.markdown("**Token Budget Allocation**")
        st.metric("Total Budget", f"{budget_report['total_budget']} tokens")
        st.metric("System Prompt", f"{budget_report['system_tokens']} tokens")
        st.metric("Context Used", f"{budget_report['context_tokens_used']} tokens")
        st.metric("Chunks Selected", budget_report["chunks_selected"])
        st.metric("Chunks Pruned", budget_report["chunks_pruned"])

        st.markdown("**Retrieval Scores**")
        for i, chunk in enumerate(selected, start=1):
            score_display = f"{1 - chunk['distance']:.3f}" if chunk["distance"] < 1 else f"{chunk['distance']:.3f}"
            st.text(f"[Source {i}] {chunk['doc_name']} p.{chunk['page']} (score: {score_display})")

        if pruned:
            st.markdown("**Pruned (over budget):**")
            for chunk in pruned:
                st.text(f"  x {chunk['doc_name']} p.{chunk['page']}")

What does this sidebar show?

The sidebar exposes every context engineering decision: how many tokens were allocated, how many were actually used, which chunks scored highest and made it into the prompt, and which were pruned because they exceeded the budget.

This transparency is what separates a portfolio project from a demo. Anyone reviewing your app can see exactly how you manage the model's context window.

• Save app.py.

Before you run the app, think about this: you asked a cross-document question in Step 2 and got a vague answer. Do you think the answer will be more specific now that the prompt contains only high-relevance, provenance-tagged chunks?

• Run the app:

streamlit run app.py

• Upload the same PDFs you used in Step 2.
• Ask the same cross-document question (for example, "What do both documents say about X?").

You should see a specific, accurate answer with source references. On the right side, the Context Engineering sidebar shows your token budget metrics, retrieval scores for each source chunk, and any chunks that were pruned.

Sidebar shows nothing after asking a question?

• Make sure the with col_sidebar: block is at the same indentation level as with col_main: (both should be at the top level, not nested inside each other).
• Confirm the variables budget_report, selected, and pruned are defined inside the if question: block above the sidebar code.

Help me debug my Context Engineering sidebar

✔️ Awesome, I've got everything!

Great. Confirm you have saved all three files: rag.py, context_engine.py, and app.py.

ⓧ I'd like to double check the full code

Your project now has three key files. Scroll up and confirm each one matches the code blocks you pasted:

• rag.py: 4 functions (extract_text_by_page, chunk_pages, create_collection, retrieve).
• context_engine.py: 2 functions (estimate_tokens, assemble_context).
• app.py: imports the new modules, uses a two-column layout, processes PDFs into chunks, retrieves and assembles context for queries, and displays a Context Engineering sidebar.

Your app now demonstrates real context engineering. Instead of dumping raw documents into the model, it selects, ranks, budgets, and tags every piece of context. Next up, you will engineer the prompt itself with structured citations and few-shot examples.

Engineer the Prompt with Structured Citations

Your RAG pipeline retrieves the right chunks and assembles them within a token budget. But if you ask the model a question where your documents disagree, you still get vague answers with no specific citations.

The missing piece is prompt engineering. In this step, you will build a structured prompt template that uses XML-tagged sections to separate instructions from data, a JSON schema to enforce structured citation output, and few-shot examples to handle edge cases like conflicting sources. You will also expose the full prompt in a "Prompt Lab" panel so employers can inspect your engineering decisions.

In this step, get ready to:

• Build a structured prompt module with XML-tagged sections, few-shot examples, and a JSON citation schema.
• Wire the prompt into your app with enforced JSON output and a Prompt Lab expander.
• Display structured citations with document name, page number, and relevance for each claim.

Build the structured prompt template

A well-engineered prompt separates concerns into labeled sections the model can parse reliably. XML tags like <system_instructions> and <sources> tell the model what is a rule versus what is data. Few-shot examples show the model the exact citation format you expect. A JSON schema enforces that the output is always parseable.

• Create a new file called prompts.py in your project folder, and add the system template:

SYSTEM_TEMPLATE = """<system_instructions>
You are a multi-document research assistant. Your job is to synthesize information across multiple source documents and provide accurate, well-cited answers.

RULES:
- ONLY use information from the provided sources
- Cite every claim using [Source N] notation referencing the source number
- When sources disagree, explicitly note the disagreement and cite both sides
- If sources lack information to answer fully, state what is missing
- Be specific and concise. Prefer direct quotes over paraphrasing when precision matters
</system_instructions>

<few_shot_examples>
USER QUESTION: What do the documents say about the effectiveness of approach X?

GOOD ANSWER: Document A reports that approach X achieved 95% accuracy on benchmark tasks [Source 1], while Document B found only 78% accuracy under different conditions [Source 3]. The discrepancy appears to stem from differences in dataset size: Document A used 10,000 samples [Source 1] while Document B used only 500 [Source 3].

USER QUESTION: What is the recommended temperature for process Y?

GOOD ANSWER: The sources do not contain specific information about temperature recommendations for process Y. Source 2 discusses process Y's duration but does not mention temperature parameters.
</few_shot_examples>"""

What does this template do?

The <system_instructions> section defines the model's role and citation rules. The model treats everything inside these tags as hard constraints.

The <few_shot_examples> section shows the model two scenarios: one where sources agree (cite both with specifics) and one where sources lack information (say so explicitly). This teaches the model the exact format you expect without ambiguity.

• Below SYSTEM_TEMPLATE, add the query template and the build_prompt() function that assembles the full prompt:

QUERY_TEMPLATE = """<sources>
{context}
</sources>

<research_notes>
{research_notes}
</research_notes>

<user_question>
{question}
</user_question>

Provide your answer with citations. Use [Source N] notation for every factual claim."""


def build_prompt(context, question, research_notes="No prior research notes."):
    """Assemble the full engineered prompt."""
    query = QUERY_TEMPLATE.format(
        context=context,
        research_notes=research_notes,
        question=question,
    )
    return SYSTEM_TEMPLATE, query

What does this code do?

QUERY_TEMPLATE uses three XML-tagged sections to separate sources, prior research notes, and the user's question. The model can clearly distinguish retrieved evidence from conversation history from the actual query.

build_prompt() fills in the placeholders and returns two strings: the system prompt (instructions + examples) and the query prompt (sources + notes + question). Splitting them lets you pass the system prompt separately via system_instruction in the API config.

• Save prompts.py and verify it loads correctly by running:

python -c "from prompts import build_prompt; print('prompts.py loaded OK')"

You should see prompts.py loaded OK printed in your terminal.

Seeing a SyntaxError?

Triple-quoted strings are sensitive to indentation. Make sure the opening """ and closing """ of each template are on lines by themselves with no extra spaces before them.

help me fix a SyntaxError in my prompts.py template strings

• Now add the JSON citation schema below build_prompt() in prompts.py. Place this between the QUERY_TEMPLATE definition and the build_prompt() function:

CITATION_SCHEMA = {
    "type": "object",
    "properties": {
        "answer": {
            "type": "string",
            "description": "The complete answer with [Source N] citations inline",
        },
        "sources_used": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "source_number": {"type": "integer"},
                    "doc_name": {"type": "string"},
                    "page": {"type": "integer"},
                    "relevance": {"type": "string", "description": "Why this source was relevant"},
                },
                "required": ["source_number", "doc_name", "page", "relevance"],
            },
        },
        "confidence": {
            "type": "string",
            "enum": ["high", "medium", "low"],
            "description": "Confidence based on source coverage",
        },
    },
    "required": ["answer", "sources_used", "confidence"],
}

What does the citation schema enforce?

When you pass this schema to the Gemini API with response_mime_type='application/json', the model is forced to return valid JSON matching this exact structure. Every response will contain an answer string with inline citations, a sources_used array listing each source with its document name and page, and a confidence level.

This eliminates the guesswork of parsing free-text responses. You always know the output shape.

• Save prompts.py and confirm the full module loads by running:

python -c "from prompts import build_prompt, CITATION_SCHEMA; print('All exports OK')"

You should see All exports OK in your terminal.

Seeing a NameError or ImportError?

Make sure CITATION_SCHEMA is placed between QUERY_TEMPLATE and build_prompt() in the file. The build_prompt() function should be the last item in the file.

help me fix an import error in prompts.py

✔️ Awesome, I've got everything!

Great. Make sure you have saved prompts.py with all three exports: SYSTEM_TEMPLATE, QUERY_TEMPLATE, CITATION_SCHEMA, and build_prompt().

ⓧ I'd like to double check the full code

Your prompts.py should contain these sections in order. Scroll up and confirm each one is present:

• SYSTEM_TEMPLATE with <system_instructions> and <few_shot_examples> XML sections.
• QUERY_TEMPLATE with <sources>, <research_notes>, and <user_question> XML sections.
• CITATION_SCHEMA dictionary with answer, sources_used, and confidence fields.
• build_prompt() function that formats the query and returns (SYSTEM_TEMPLATE, query).

Wire the prompt into app.py with structured citations

With your prompt module ready, you need to update app.py to use the engineered prompt, enforce JSON output, and display the results with structured citations and a Prompt Lab panel.

• In app.py, find the imports at the top of the file:

import os
import tempfile
import streamlit as st
from google import genai
from rag import extract_text_by_page, chunk_pages, create_collection, retrieve
from context_engine import assemble_context, estimate_tokens

• Replace those import lines with the following updated imports:

import os
import json
import tempfile
import streamlit as st
from google import genai
from google.genai import types
from rag import extract_text_by_page, chunk_pages, create_collection, retrieve
from context_engine import assemble_context, estimate_tokens
from prompts import build_prompt, CITATION_SCHEMA

What changed in the imports?

You added json to parse the structured JSON response, types from google.genai to configure generation parameters, and build_prompt plus CITATION_SCHEMA from your new prompts module.

• Now find the if question: block inside with col_main:. Locate the lines from collection = st.session_state["collection"] through st.write(response.text) and replace them with:

            collection = st.session_state["collection"]
            retrieved_chunks = retrieve(collection, question, n_results=5)

            assembled_context, selected, pruned, budget_report = assemble_context(
                retrieved_chunks, token_budget=3000
            )

            # Build research notes from session history
            notes_text = "\n".join(
                [f"- {note}" for note in st.session_state["research_notes"]]
            ) if st.session_state["research_notes"] else "No prior research notes."

            # Assemble the engineered prompt
            system_prompt, query_prompt = build_prompt(assembled_context, question, notes_text)

            with st.spinner("Generating answer..."):
                response = client.models.generate_content(
                    model="gemini-2.5-flash",
                    contents=query_prompt,
                    config=types.GenerateContentConfig(
                        system_instruction=system_prompt,
                        temperature=0.2,
                        max_output_tokens=1024,
                        response_mime_type="application/json",
                        response_json_schema=CITATION_SCHEMA,
                    ),
                )

What changed in the API call?

Instead of a raw string prompt, you now pass query_prompt as contents and system_prompt as the system instruction. The GenerateContentConfig enforces JSON output via response_mime_type="application/json" and constrains the shape with response_json_schema=CITATION_SCHEMA.

The notes_text variable injects any prior research notes into the prompt, giving the model memory of earlier questions in the same session.

• Directly after the with st.spinner block you just added, paste the response parsing and citation display code:

            try:
                result = json.loads(response.text)
                answer_text = result["answer"]
                sources_used = result.get("sources_used", [])
                confidence = result.get("confidence", "unknown")
            except (json.JSONDecodeError, KeyError):
                answer_text = response.text
                sources_used = []
                confidence = "unknown"

            st.subheader("Answer")
            st.write(answer_text)
            st.caption(f"Confidence: {confidence} | Sources cited: {len(sources_used)}/{len(selected)} selected")

            with st.expander("Source Citations"):
                if sources_used:
                    for src in sources_used:
                        st.markdown(
                            f"**[Source {src['source_number']}]** {src['doc_name']}, Page {src['page']}"
                        )
                        st.text(f"Relevance: {src['relevance']}")
                        st.divider()
                else:
                    for i, chunk in enumerate(selected, start=1):
                        st.markdown(f"**[Source {i}]** {chunk['doc_name']}, Page {chunk['page']}")
                        st.text(chunk["text"][:200] + "...")
                        st.divider()

What does this code do?

The try/except block parses the JSON response into answer_text, sources_used, and confidence. If parsing fails (rare edge case), it falls back to displaying raw text.

The Source Citations expander lists each cited source with its document name, page number, and why it was relevant. This gives the reader full traceability from claim to source.

• Finally, add the Prompt Lab expander and research notes tracking directly below the Source Citations expander. This code still sits inside the if question: block:

            with st.expander("Prompt Lab - View Engineered Prompt"):
                st.markdown("**System Instructions (XML-tagged):**")
                st.code(system_prompt, language="xml")
                st.markdown("**Query with Context + Research Notes:**")
                st.code(query_prompt[:2000] + "..." if len(query_prompt) > 2000 else query_prompt, language="xml")

            # Track this Q&A as a research note for future questions
            note = f"Q: {question[:80]} -> Key finding: {answer_text[:120]}"
            st.session_state["research_notes"].append(note)

What does the Prompt Lab show?

The Prompt Lab expander renders your full engineered prompt with syntax highlighting. Anyone reviewing your project can expand it and see exactly how you structured the system instructions, sources, research notes, and question. This makes your engineering decisions transparent.

The research_notes append creates a memory of each Q&A exchange. Future questions inject these notes into the prompt, so the model has context about what you have already explored.

• Now update the sidebar. Find this line in the with col_sidebar: block:

    if uploaded_files and question:

• Replace it with:

    if "budget_report" in dir() and question:

• At the very end of the with col_sidebar: block (after the pruned chunks display), add the research notes panel:

    st.markdown("---")
    st.markdown("### Session Research Notes")
    if st.session_state["research_notes"]:
        for note in st.session_state["research_notes"][-5:]:
            st.text(note[:100])
    else:
        st.text("No notes yet. Ask questions to build research memory.")

What does the research notes panel show?

This sidebar section displays the last 5 research notes from the current session. Each note summarizes a prior question and its key finding. As the learner asks follow-up questions, they can see their research history building up.

• Save app.py.

Before you run the app, predict: what format do you expect the answer to appear in now that you have enforced a JSON schema?

• Run your app by executing:

streamlit run app.py

• Upload your two sample PDFs.
• Ask a question where the documents might disagree or cover the same topic differently.

You should see a structured answer with inline [Source N] citations, a confidence level, and a Source Citations expander listing each source with its document name, page, and relevance. Expand the Prompt Lab to see your full engineered prompt with its XML-tagged sections.

Seeing a JSON parse error in the app?

If the answer displays as raw text instead of structured citations, the JSON fallback is working but the model returned invalid JSON. Check that CITATION_SCHEMA in prompts.py matches the schema above exactly. All required arrays must list the correct field names.

If you see an import error mentioning types, confirm the line from google.genai import types is present in your imports.

help me debug my structured citation output in DocuQuery

✔️ Awesome, I've got everything!

Great. Confirm you have saved both prompts.py and app.py and that your app displays structured citations with document names and page numbers.

ⓧ I'd like to double check the full code

Your app.py should contain these sections in order. Scroll up and confirm each is present:

• Updated imports with json, types, build_prompt, and CITATION_SCHEMA.
• The if question: block with notes_text construction, build_prompt() call, and GenerateContentConfig with JSON schema enforcement.
• JSON response parsing with try/except, answer display with confidence caption, and Source Citations expander.
• Prompt Lab expander with st.code() for system and query prompts.
• Research notes append after each answer.
• Sidebar condition updated to "budget_report" in dir() and a Session Research Notes section at the bottom.

Your research assistant now produces reliable, structured citations that trace every claim back to a specific document and page. The Prompt Lab makes your engineering decisions visible to anyone reviewing the project. Next up, you will deploy this app to a live URL, add Agent Skills for AI coding assistants, and build an evaluation harness that proves your context-engineered pipeline outperforms the naive approach.

Deploy with Agent Skills and Evaluate

In the last step, you engineered a structured prompt that produces cited, high-quality answers. Your app works locally, but the most important question for an employer is still unanswered: can you ship?

In this step, you will prove that you think about AI systems holistically. You will build a comparative evaluation harness, create Agent Skills that make the repo AI-assistant-friendly, and deploy the whole thing to a live public URL.

In this step, get ready to:

• Build a comparative evaluation harness that scores naive vs. context-engineered answers.
• Create Agent Skills (SKILL.md files) that teach AI coding assistants how to extend your project.
• Deploy to Streamlit Community Cloud with a live public URL.

Write the evaluation harness

An evaluation harness runs your pipeline against a set of known questions and checks whether the answers meet quality criteria. This lets you measure the value of context engineering with hard numbers instead of vibes.

• Open evaluate.py in VS Code.
• Add the imports, API client, test PDFs config, and test cases by pasting the following code:

import os
import json
from google import genai
from google.genai import types
from rag import extract_text_by_page, chunk_pages, create_collection, retrieve
from context_engine import assemble_context
from prompts import build_prompt, CITATION_SCHEMA

client = genai.Client(api_key=os.environ["GEMINI_API_KEY"])

TEST_PDFS = [
    {"path": "docs/sample1.pdf", "name": "sample1.pdf"},
    {"path": "docs/sample2.pdf", "name": "sample2.pdf"},
]

TEST_CASES = [
    {
        "question": "What are the main topics covered across both documents?",
        "expected_keywords": [],
        "requires_citation": True,
    },
]

What does this code do?

TEST_PDFS points to the sample documents in your docs/ folder. TEST_CASES defines question-answer pairs. Each test case has a question, optional expected keywords, and a flag for whether citations are required.

You start with one test case here. After you confirm things work, you can add more test cases specific to your own PDFs.

• Below the test cases, add both pipeline functions that the harness will compare:

def run_naive(all_text, question):
    """Naive approach: dump all text into prompt."""
    prompt = f"Answer this question based on the documents:\n\n{all_text[:8000]}\n\nQuestion: {question}"
    response = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=prompt,
        config=types.GenerateContentConfig(temperature=0.1, max_output_tokens=512),
    )
    return response.text


def run_context_engineered(collection, question):
    """Context-engineered approach."""
    retrieved = retrieve(collection, question, n_results=5)
    assembled_context, selected, pruned, budget = assemble_context(retrieved)
    system_prompt, query_prompt = build_prompt(assembled_context, question)

    response = client.models.generate_content(
        model="gemini-2.5-flash",
        contents=query_prompt,
        config=types.GenerateContentConfig(
            system_instruction=system_prompt,
            temperature=0.1,
            max_output_tokens=512,
            response_mime_type="application/json",
            response_json_schema=CITATION_SCHEMA,
        ),
    )

    try:
        result = json.loads(response.text)
        return result["answer"]
    except (json.JSONDecodeError, KeyError):
        return response.text

What do these functions do?

run_naive() simulates the Step 2 approach: dump all text and hope for the best. run_context_engineered() uses your full RAG pipeline with retrieval, context assembly, and the structured prompt.

Running both against the same questions lets you measure the improvement context engineering delivers.

• Save evaluate.py and confirm no import errors by running:

python -c "import evaluate"

You should see no output if everything imports correctly.

Seeing an ImportError?

Check that your virtual environment is activated and all packages from requirements.txt are installed. Confirm rag.py, context_engine.py, and prompts.py are in the same directory as evaluate.py.

Why can't Python find my modules?

• Now add the evaluation logic and main entry point below the pipeline functions:

def evaluate_answer(answer, test_case):
    """Check if answer meets criteria."""
    answer_lower = answer.lower()
    has_citation = "[source" in answer_lower
    keywords_ok = True
    if test_case["expected_keywords"]:
        keywords_ok = any(kw.lower() in answer_lower for kw in test_case["expected_keywords"])
    citation_ok = has_citation if test_case["requires_citation"] else True
    return keywords_ok and citation_ok


def run_evaluation():
    """Run comparative evaluation."""
    print("Loading documents...")
    all_chunks = []
    all_text_parts = []
    for pdf_info in TEST_PDFS:
        pages = extract_text_by_page(pdf_info["path"], pdf_info["name"])
        chunks = chunk_pages(pages)
        all_chunks.extend(chunks)
        all_text_parts.extend([p["text"] for p in pages])

    collection = create_collection(all_chunks)
    all_text = "\n".join(all_text_parts)
    print(f"Processed {len(TEST_PDFS)} documents, {len(all_chunks)} chunks.\n")

    naive_passed = 0
    engineered_passed = 0
    total = len(TEST_CASES)

• Directly below (still inside run_evaluation()), add the test loop and the main guard:

    for i, test in enumerate(TEST_CASES, start=1):
        question = test["question"]
        print(f"Test {i}: {question}")

        naive_answer = run_naive(all_text, question)
        naive_ok = evaluate_answer(naive_answer, test)
        print(f"  Naive:      {'PASS' if naive_ok else 'FAIL'}")

        eng_answer = run_context_engineered(collection, question)
        eng_ok = evaluate_answer(eng_answer, test)
        print(f"  Engineered: {'PASS' if eng_ok else 'FAIL'}")
        print()

        if naive_ok:
            naive_passed += 1
        if eng_ok:
            engineered_passed += 1

    print(f"{'='*40}")
    print(f"RESULTS:")
    print(f"  Naive approach:      {naive_passed}/{total} ({naive_passed/total*100:.0f}%)")
    print(f"  Context-engineered:  {engineered_passed}/{total} ({engineered_passed/total*100:.0f}%)")
    print(f"{'='*40}")


if __name__ == "__main__":
    run_evaluation()

What does the evaluation logic do?

evaluate_answer() checks two things: are expected keywords present, and does the answer include [Source citations when required? A test passes only if both criteria are met.

run_evaluation() loads your test PDFs, runs every question through both pipelines, and prints a side-by-side scorecard showing how many tests each approach passed.

• Save evaluate.py.
• Before you run the evaluation, what do you expect to see? Will the context-engineered pipeline produce citations while the naive one does not?
• Run the evaluation harness:

python evaluate.py

You should see output showing each test with PASS/FAIL for both pipelines, followed by a comparative results summary. The context-engineered pipeline should pass citation checks that the naive approach fails.

Seeing a FileNotFoundError for your PDFs?

Make sure docs/sample1.pdf and docs/sample2.pdf exist in your project. These should be the same sample PDFs you uploaded when testing the app in previous steps. Copy any two PDF files into the docs/ folder and rename them to match.

Why is my evaluation failing to find test PDFs?

✔️ Awesome, I've got everything!

Great. Double check you have saved evaluate.py and that the comparative results printed successfully.

ⓧ I'd like to double check the full code

Your evaluate.py should contain these sections in order. Scroll up and confirm each one is present:

• Imports (os, json, genai, types, plus imports from rag, context_engine, prompts).
• client creation, TEST_PDFS list, and TEST_CASES list.
• run_naive() and run_context_engineered() functions.
• evaluate_answer() and run_evaluation() functions.
• if __name__ == "__main__": guard at the bottom calling run_evaluation().

Create Agent Skills

Agent Skills are SKILL.md files that teach AI coding assistants (like Claude Code, Cursor, or Copilot) how to work with your codebase. They follow the open standard of YAML frontmatter plus procedural Markdown steps.

• Create the file .skills/rag-pipeline/SKILL.md by running:

mkdir -p .skills/rag-pipeline

• Open .skills/rag-pipeline/SKILL.md in VS Code and paste the first part of the skill (frontmatter through procedure):

---
name: rag-pipeline
description: Extend or modify the DocuQuery RAG pipeline. Use when adding new document types, changing chunking strategy, modifying retrieval parameters, or adding new metadata fields to the context assembly.
---

# RAG Pipeline Extension

## When to invoke

Use this skill when the task involves modifying document ingestion, chunking, embedding, retrieval, or context assembly in the DocuQuery project.

## Inputs

- The type of change (new document format, different chunk size, new metadata, retrieval tuning)
- Any constraints (token budget limits, performance requirements)

## Procedure

1. Document ingestion lives in `rag.py` in the `extract_text_by_page()` function. To support a new format, add a new extraction function that returns the same `[{"doc_name": str, "page": int, "text": str}]` structure.
2. Chunking is in `rag.py` in `chunk_pages()`. Default is 500 chars with 50 char overlap. When changing chunk size, also update the token budget in `context_engine.py` to account for different chunk token counts.
3. ChromaDB collection is created in `create_collection()`. Metadata fields (`doc_name`, `page`) are used for provenance. To add new metadata, update both the `metadatas` list in `create_collection()` and the retrieval output in `retrieve()`.
4. Context assembly in `context_engine.py` enforces a 3000-token budget. The `assemble_context()` function re-ranks by distance score and prunes over-budget chunks. Modify `token_budget` parameter or the ranking logic here.
5. After changes, run `python evaluate.py` to verify retrieval quality has not degraded.

• Below the Procedure section, add the Outputs and Failure modes sections:

## Outputs

- Modified pipeline files (`rag.py`, `context_engine.py`)
- Updated evaluation results confirming no regression

## Failure modes

- If ChromaDB throws a dimension mismatch error, the collection was created with a different embedding model. Delete and recreate the collection.
- If token budget is exceeded in the UI, reduce `n_results` in `retrieve()` or increase `token_budget` in `assemble_context()`.

• Save .skills/rag-pipeline/SKILL.md.
• Confirm the file exists by running:

ls .skills/rag-pipeline/

You should see SKILL.md in the output.

• Now create the evaluation harness skill. Set up the directory:

mkdir -p .skills/eval-harness

• Open .skills/eval-harness/SKILL.md in VS Code and paste the first part:

---
name: eval-harness
description: Add test cases to or run the DocuQuery evaluation harness. Use when writing new test questions, validating answer quality, or comparing naive vs context-engineered pipelines.
---

# Evaluation Harness

## When to invoke

Use this skill when adding new evaluation test cases, running the comparative evaluation, or modifying evaluation criteria.

## Inputs

- New test questions and expected keywords (for adding test cases)
- Test PDF paths (must exist in `docs/` directory)

## Procedure

1. Open `evaluate.py`. Test cases are defined in the `TEST_CASES` list as dicts with keys: `question` (str), `expected_keywords` (list of strings the answer should contain), and `requires_citation` (bool).
2. To add a new test case, append a dict to `TEST_CASES`. Choose questions that test cross-document synthesis (referencing both sample PDFs). Include at least 2-3 expected keywords that a correct answer must contain.
3. Ensure test PDFs are listed in `TEST_PDFS` with both `path` and `name` keys matching files in `docs/`.
4. Run with `python evaluate.py`. It tests both naive (full-text dump) and context-engineered (RAG + structured prompt) pipelines and prints comparative scores.
5. A test passes when: expected keywords are found in the answer AND citations are present (if `requires_citation` is True). The citation check looks for "[source" in the lowercase answer.

• Below the Procedure section, add:

## Outputs

- Console output showing per-test PASS/FAIL for both pipelines
- Final comparative score (e.g., "Naive: 4/10, Context-Engineered: 8/10")

## Failure modes

- If all tests fail with "FAIL" on the engineered pipeline, check that `GEMINI_API_KEY` environment variable is set.
- If PDF loading fails, verify file paths in `TEST_PDFS` are relative to the project root.
- If citation checks fail but answers look correct, the model may be using a different citation format. Check the prompt template in `prompts.py`.

• Save .skills/eval-harness/SKILL.md.
• Confirm both skills exist:

ls .skills/*/SKILL.md

You should see both .skills/eval-harness/SKILL.md and .skills/rag-pipeline/SKILL.md listed.

What are Agent Skills for?

When another developer (or you in the future) opens this repo with an AI coding assistant, the assistant automatically reads these SKILL.md files. It then knows how to add new document types, modify chunking, or extend the evaluation without guessing at your architecture.

This is the Agent Skills open standard in action. The YAML frontmatter tells the assistant when to invoke the skill, and the numbered steps tell it exactly how.

Push to GitHub and deploy on Streamlit Community Cloud

With your code tested and documented, it is time to push to GitHub and deploy. Streamlit Community Cloud deploys directly from a GitHub repo with no Docker or server config needed.

• Initialize a Git repo and make your first commit by running these commands:

git init
git add .
git commit -m "DocuQuery: multi-document research assistant"

What about the .streamlit/secrets.toml file?

Your .streamlit/secrets.toml contains a placeholder API key value. It is safe to commit because it holds your-api-key-here (not a real key). Your actual key will be stored securely in Streamlit Community Cloud's secrets management, not in the repo.

• Go to github.com and create a new public repository named docuquery (or any name you prefer).
• Do not initialize it with a README (you already have files locally).
• Push your local repo to GitHub by running:

git remote add origin [[YOUR_REPO_URL="https://github.com/YOUR_USERNAME/docuquery.git"]]
git branch -M main
git push -u origin main

• Confirm the push succeeded by refreshing your GitHub repository page. You should see all your project files listed.

Push rejected or authentication failed?

If Git asks for credentials, you may need to set up a personal access token or SSH key. Check that the remote URL matches what GitHub shows on your new repo page.

Why can't I push to my GitHub repo?

• Now deploy on Streamlit Community Cloud. Go to share.streamlit.io and sign in with your GitHub account.
• In the upper-right corner, click Create app.
• When asked if you already have an app, click Yup, I have an app.
• Fill in your repository name, branch (main), and main file path (app.py).
• Click Advanced settings.
• In the Secrets field, paste your secret in TOML format:

GEMINI_API_KEY = "your-actual-api-key-here"

• Replace the placeholder with your real Gemini API key from Google AI Studio.
• Click Save to close advanced settings.
• Click Deploy.

Streamlit will build your app. Most deploys finish within a few minutes. When complete, your app is live at a public URL.

• Before you check the live URL, do you expect the app to load correctly on the first try? What could go wrong?
• Open your app at https://your-app-name.streamlit.app in a new browser window.
• Upload a PDF and ask a question to confirm the deployed app works end-to-end.

App showing an error after deploy?

Check that your requirements.txt is in the repo root (not a subdirectory). Verify the secret name in the Streamlit Cloud dashboard matches GEMINI_API_KEY exactly. You can view deploy logs by clicking your app's menu in the Streamlit Cloud dashboard.

Why is my Streamlit Cloud app failing to start?

Secret mission

Add Conversation Compaction

In a real research session, users ask follow-up questions. Naively appending full conversation history bloats the context window and causes "context rot" where the model loses focus. Can you implement compaction that summarizes prior Q&A into concise research notes, preserving decisions and findings while freeing token budget for new retrieval?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on free services, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

• Streamlit Community Cloud app (free tier, no charges).
• Google Gemini API key (free tier, no charges).
• GitHub repository (free).
• Local project folder with Python virtual environment.

Keep everything running

No action needed. Choose this if you are still actively building or want to keep your live demo URL available for employers to visit.

• Your Streamlit Community Cloud app stays live indefinitely on the free tier.
• Your Gemini API key remains active within the free tier rate limits.
• Your GitHub repository stays accessible as part of your portfolio.
• Your local project folder stays on your machine.

Pause - I'll come back to this later

Free up local resources while keeping your deployed app and remote services intact. Streamlit Community Cloud apps automatically sleep after inactivity and wake on the next visit.

• Deactivate your Python virtual environment by running:
• No action is needed for Streamlit Community Cloud. It sleeps automatically.
• No action is needed for your Gemini API key or GitHub repository.

deactivate

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each step below to fully clean up.

Delete your Streamlit Community Cloud app:

• Go to your workspace at share.streamlit.io.
• Click the overflow icon next to your DocuQuery app.
• Click Delete.
• Enter the required confirmation string and click Delete again.

Delete your Gemini API key:

• Open the Google Cloud API Credentials page at console.cloud.google.com/apis/credentials.
• Find the API key you created for this project and click the Actions icon.
• Select Delete API key.
• In the Delete credential modal, click Delete to confirm.

Delete your GitHub repository:

• Go to your DocuQuery repository on GitHub.
• Click Settings in the repository navigation bar.
• Scroll to the bottom of the page to the Danger Zone section.
• Click Delete this repository and follow the confirmation prompts.

Delete your local project folder:

• Remove the project folder from your Desktop by running:

macOS

rm -rf ~/Desktop/docuquery

Windows

Remove-Item -Recurse -Force ~\Desktop\docuquery

• Unset the GEMINI_API_KEY environment variable from your terminal session by closing the terminal window.

Nice Work!

Nice Work!

You just built and deployed a multi-document research assistant that demonstrates context engineering, prompt engineering, and the Agent Skills open standard. Employers can visit your live URL, inspect how you manage the model's context window, and their AI coding assistants can immediately understand how to extend your codebase.

You've learned how to:

• Build a multi-document RAG pipeline with cross-document retrieval, provenance tagging, and structured source citations that ground every claim to a specific document and page.
• Demonstrate context engineering with a visible pipeline showing token budgeting, retrieval scoring, re-ranking decisions, and context assembly that employers can inspect in the sidebar.
• Engineer structured prompts using XML-tagged sections, JSON schema output enforcement, and few-shot examples for edge cases like conflicting sources.
• Secret Mission: Implemented conversation compaction that summarizes Q&A history into persistent research notes, demonstrating the "compress" technique from context engineering.

Ready to quiz yourself?