Build an AI Student Second Brain V2
Evolve a RAG app into a full learning platform with folders, maps, and flashcards.
Introduction
30 Second Summary
Students drown in course materials every semester. PDFs, lecture recordings, and handwritten notes pile up with no clear way to connect them into real understanding.
In this project, you will transform E-Hanatra V1 into a complete intelligent learning platform. You will add folder organization, contextual AI chat, auto-generated knowledge maps, and spaced repetition revision cards, then layer on Malagasy language support, a mobile PWA experience, and multi-user authentication.
What You'll Build
A Next.js 15 and FastAPI application that transforms uploaded course materials into organized knowledge, interactive concept maps, and AI-powered revision cards with native Malagasy language support and multi-user data isolation.
By the end of this project, you'll have:
- Hierarchical folder system with drag-and-drop organization and contextual RAG chat scoped to documents, folders, or your entire knowledge base.
- Auto-generated Knowledge Maps using React Flow and intelligent SM-2 spaced repetition revision cards with multiple-choice quiz mode.
- PWA mobile experience with offline caching, Malagasy language transcription via Whisper, and JWT-based multi-user authentication with personal data isolation.
- Secret Mission: Build a Study Analytics dashboard with weekly learning heatmaps, concept mastery radar charts, and exportable study reports.
Are there any prerequisites?
This project assumes you have already built and deployed E-Hanatra V1 with a working Next.js 15 frontend, FastAPI backend, ChromaDB vector store, Groq LLM integration, and EasyOCR document processing.
V2.0 builds directly on top of your existing codebase. If you have not completed V1, finish that project first before starting here.
Build the Hierarchical Folder System — Backend
E-Hanatra V1 can upload documents and chat about them, but everything lives in a single flat list. As your course materials grow, finding a specific lecture becomes painful without organization.
In this step, you will build a hierarchical folder system in your FastAPI backend. This means adding a SQLite database for folder and document metadata, writing CRUD endpoints for managing folders, and enriching ChromaDB chunks with folder context so future features (like scoped chat) know where each piece of content belongs.
In this step, get ready to:
- Create a SQLite database with folders and documents tables using aiosqlite.
- Build FastAPI CRUD endpoints for folder management.
- Update the document upload pipeline to store folder metadata in both SQLite and ChromaDB.
Create the SQLite database schema
Your V1 backend has no relational database. Document metadata (titles, paths, folder locations) needs a structured home that ChromaDB alone cannot provide.
- Install the aiosqlite package by running this command in your FastAPI project directory:
pip install aiosqliteWhat is aiosqlite?
aiosqlite is an async wrapper around Python's built-in sqlite3 module. It lets your FastAPI endpoints interact with SQLite without blocking the event loop, which is critical for keeping your API responsive.
- Create a new file called database.py in your FastAPI project root.
- Add the database connection helper and schema initialization by pasting this code:
import aiosqlite
import uuid
from datetime import datetime
DATABASE_PATH = "e-hanatra.db"
async def get_db():
"""Get a database connection with row factory enabled."""
db = await aiosqlite.connect(DATABASE_PATH)
db.row_factory = aiosqlite.Row
return db
async def init_db():
"""Create tables if they don't exist and seed default folder."""
async with aiosqlite.connect(DATABASE_PATH) as db:
# Enable WAL mode for better read performance
await db.execute("PRAGMA journal_mode=WAL")
# Folders table using adjacency list pattern
# parent_id is nullable (NULL = root-level folder)
await db.execute("""
CREATE TABLE IF NOT EXISTS folders (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
parent_id TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
FOREIGN KEY (parent_id) REFERENCES folders(id)
)
""")
# Documents table linking files to folders
await db.execute("""
CREATE TABLE IF NOT EXISTS documents (
id TEXT PRIMARY KEY,
title TEXT NOT NULL,
file_path TEXT NOT NULL,
folder_id TEXT NOT NULL,
content_type TEXT NOT NULL,
created_at TEXT NOT NULL,
FOREIGN KEY (folder_id) REFERENCES folders(id)
)
""")
# Create the default root folder if it doesn't exist
existing = await db.execute(
"SELECT id FROM folders WHERE name = ? AND parent_id IS NULL",
("My Documents",)
)
row = await existing.fetchone()
if not row:
now = datetime.utcnow().isoformat()
await db.execute(
"INSERT INTO folders (id, name, parent_id, created_at, updated_at) VALUES (?, ?, NULL, ?, ?)",
(str(uuid.uuid4()), "My Documents", now, now)
)
await db.commit()What does this code do?
The folders table uses the adjacency list pattern. Each folder has a parent_id that references another folder's id, creating unlimited nesting. Root-level folders have parent_id set to NULL.
The init_db() function also seeds a default "My Documents" root folder so existing V1 documents have somewhere to land during migration.
- Save database.py.
- Confirm the file exists by running:
ls database.pyYou should see database.py listed in the output.
File not showing up?
Make sure you created the file in your FastAPI project root directory, not a subdirectory. Run pwd to confirm your current location.
help me find where to create database.py in my project
Now you need to wire init_db() into your FastAPI startup so the tables are created when the server launches.
- In your main FastAPI app file (where you define your app = FastAPI() instance), add the startup event that initializes the database:
from database import init_db
# Initialize the SQLite database on startup
@app.on_event("startup")
async def startup():
await init_db()What does this do?
The @app.on_event("startup") decorator runs init_db() once when your FastAPI server starts. This creates the tables if they don't exist and seeds the default folder.
- Save your main app file and start the server by running:
uvicorn main:app --reloadYou should see the server start without errors. The file e-hanatra.db now exists in your project directory.
- Confirm the database was created by running:
ls e-hanatra.dbDon't see e-hanatra.db?
Check your terminal for import errors when the server starts. Make sure database.py is importable from your main app file (same directory or properly configured as a module).
help me debug why my database file isn't being created
Next, create the migration script that moves your existing V1 documents into the new folder structure.
- Create a file called migrate_v1.py in your project root and paste this migration script:
"""Migration script to import existing V1 documents into the new folder system."""
import asyncio
import aiosqlite
import uuid
import os
from datetime import datetime
from database import DATABASE_PATH
async def migrate():
"""Move existing V1 documents into the default 'My Documents' folder."""
async with aiosqlite.connect(DATABASE_PATH) as db:
db.row_factory = aiosqlite.Row
# Find the default root folder
cursor = await db.execute(
"SELECT id FROM folders WHERE name = ? AND parent_id IS NULL",
("My Documents",)
)
root_folder = await cursor.fetchone()
if not root_folder:
print("Error: 'My Documents' folder not found. Run the server first to initialize the database.")
return
root_folder_id = root_folder["id"]
# Scan your V1 upload directory for existing files
upload_dir = "./uploads" # Adjust this path to your V1 upload directory
if not os.path.exists(upload_dir):
print(f"Upload directory '{upload_dir}' not found. No documents to migrate.")
return
migrated_count = 0
now = datetime.utcnow().isoformat()
for filename in os.listdir(upload_dir):
file_path = os.path.join(upload_dir, filename)
if not os.path.isfile(file_path):
continue
# Check if document already exists in the database
existing = await db.execute(
"SELECT id FROM documents WHERE file_path = ?",
(file_path,)
)
if await existing.fetchone():
continue
# Determine content type from file extension
ext = os.path.splitext(filename)[1].lower()
content_type_map = {
".pdf": "application/pdf",
".txt": "text/plain",
".md": "text/markdown",
".docx": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
".png": "image/png",
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
}
content_type = content_type_map.get(ext, "application/octet-stream")
# Insert the document record
doc_id = str(uuid.uuid4())
await db.execute(
"INSERT INTO documents (id, title, file_path, folder_id, content_type, created_at) VALUES (?, ?, ?, ?, ?, ?)",
(doc_id, filename, file_path, root_folder_id, content_type, now)
)
migrated_count += 1
await db.commit()
print(f"Migration complete. {migrated_count} documents imported into 'My Documents'.")
if __name__ == "__main__":
asyncio.run(migrate())What does the migration do?
This script scans your V1 upload directory, creates a metadata record for each file in the documents table, and assigns every document to the default "My Documents" root folder. It skips files that already have a record, so it is safe to run multiple times.
- Save migrate_v1.py and run the migration:
python migrate_v1.pyYou should see output like: Migration complete. X documents imported into 'My Documents'. where X is the number of files in your uploads directory.
Seeing 0 documents migrated?
Check that the upload_dir variable in migrate_v1.py points to the correct path where your V1 stores uploaded files. Adjust it to match your project structure.
help me find my V1 upload directory path
Implement folder CRUD endpoints
With the database schema in place, you need API endpoints that let the frontend create, read, rename, move, and delete folders.
- Create a new file called routers/folders.py (create the routers directory if it doesn't exist).
- Add the folder creation and tree retrieval endpoints by pasting this code:
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
from typing import Optional, List
import uuid
from datetime import datetime
from database import get_db
router = APIRouter(prefix="/folders", tags=["folders"])
class FolderCreate(BaseModel):
"""Request body for creating a new folder."""
name: str
parent_id: Optional[str] = None
class FolderRename(BaseModel):
"""Request body for renaming a folder."""
name: str
class FolderMove(BaseModel):
"""Request body for moving a folder to a new parent."""
parent_id: Optional[str] = None
@router.post("")
async def create_folder(body: FolderCreate):
"""Create a new folder with optional parent_id for nesting."""
db = await get_db()
try:
# Verify parent folder exists if parent_id is provided
if body.parent_id:
cursor = await db.execute(
"SELECT id FROM folders WHERE id = ?", (body.parent_id,)
)
if not await cursor.fetchone():
raise HTTPException(status_code=404, detail="Parent folder not found")
now = datetime.utcnow().isoformat()
folder_id = str(uuid.uuid4())
await db.execute(
"INSERT INTO folders (id, name, parent_id, created_at, updated_at) VALUES (?, ?, ?, ?, ?)",
(folder_id, body.name, body.parent_id, now, now)
)
await db.commit()
return {"id": folder_id, "name": body.name, "parent_id": body.parent_id, "created_at": now}
finally:
await db.close()
async def build_tree(db, parent_id=None) -> List[dict]:
"""Recursively build the folder tree structure."""
if parent_id is None:
cursor = await db.execute(
"SELECT id, name, parent_id, created_at, updated_at FROM folders WHERE parent_id IS NULL ORDER BY name"
)
else:
cursor = await db.execute(
"SELECT id, name, parent_id, created_at, updated_at FROM folders WHERE parent_id = ? ORDER BY name",
(parent_id,)
)
rows = await cursor.fetchall()
tree = []
for row in rows:
# Count documents in this folder
doc_cursor = await db.execute(
"SELECT COUNT(*) FROM documents WHERE folder_id = ?", (row["id"],)
)
doc_count = (await doc_cursor.fetchone())[0]
# Recursively get children
children = await build_tree(db, row["id"])
tree.append({
"id": row["id"],
"name": row["name"],
"parent_id": row["parent_id"],
"created_at": row["created_at"],
"updated_at": row["updated_at"],
"document_count": doc_count,
"children": children
})
return tree
@router.get("/tree")
async def get_folder_tree():
"""Return the full nested folder tree as recursive JSON."""
db = await get_db()
try:
tree = await build_tree(db)
return {"tree": tree}
finally:
await db.close()What does this code do?
POST /folders creates a folder with an optional parent_id. If parent_id is provided, the new folder becomes a subfolder. If omitted, it sits at the root level.
GET /folders/tree uses the recursive build_tree() function to walk the adjacency list and assemble the full nested structure. Each node includes a document_count so the frontend can show a badge.
- Save routers/folders.py.
- Now add the rename, move, delete, and document listing endpoints below the existing code in the same file:
@router.patch("/{folder_id}")
async def rename_folder(folder_id: str, body: FolderRename):
"""Rename a folder."""
db = await get_db()
try:
now = datetime.utcnow().isoformat()
cursor = await db.execute(
"UPDATE folders SET name = ?, updated_at = ? WHERE id = ?",
(body.name, now, folder_id)
)
if cursor.rowcount == 0:
raise HTTPException(status_code=404, detail="Folder not found")
await db.commit()
return {"id": folder_id, "name": body.name, "updated_at": now}
finally:
await db.close()
@router.patch("/{folder_id}/move")
async def move_folder(folder_id: str, body: FolderMove):
"""Move a folder to a new parent (drag-and-drop support)."""
db = await get_db()
try:
# Prevent moving a folder into itself or its own descendants
if body.parent_id:
ancestors = await get_folder_ancestors_list(db, body.parent_id)
if folder_id in ancestors:
raise HTTPException(
status_code=400,
detail="Cannot move a folder into its own subtree"
)
now = datetime.utcnow().isoformat()
cursor = await db.execute(
"UPDATE folders SET parent_id = ?, updated_at = ? WHERE id = ?",
(body.parent_id, now, folder_id)
)
if cursor.rowcount == 0:
raise HTTPException(status_code=404, detail="Folder not found")
await db.commit()
return {"id": folder_id, "parent_id": body.parent_id, "updated_at": now}
finally:
await db.close()
async def get_folder_ancestors_list(db, folder_id: str) -> List[str]:
"""Get list of ancestor folder IDs to prevent circular moves."""
ancestors = []
current_id = folder_id
while current_id:
ancestors.append(current_id)
cursor = await db.execute(
"SELECT parent_id FROM folders WHERE id = ?", (current_id,)
)
row = await cursor.fetchone()
if row and row["parent_id"]:
current_id = row["parent_id"]
else:
break
return ancestors
@router.delete("/{folder_id}")
async def delete_folder(folder_id: str):
"""Delete a folder and cascade delete all subfolders and their documents."""
db = await get_db()
try:
# Recursively collect all subfolder IDs
folder_ids_to_delete = []
await collect_subfolder_ids(db, folder_id, folder_ids_to_delete)
# Delete all documents in these folders
for fid in folder_ids_to_delete:
await db.execute("DELETE FROM documents WHERE folder_id = ?", (fid,))
# Delete all the folders (children first, parent last)
for fid in reversed(folder_ids_to_delete):
await db.execute("DELETE FROM folders WHERE id = ?", (fid,))
await db.commit()
return {"deleted": folder_ids_to_delete}
finally:
await db.close()
async def collect_subfolder_ids(db, folder_id: str, result: List[str]):
"""Recursively collect a folder and all its subfolder IDs."""
result.append(folder_id)
cursor = await db.execute(
"SELECT id FROM folders WHERE parent_id = ?", (folder_id,)
)
children = await cursor.fetchall()
for child in children:
await collect_subfolder_ids(db, child["id"], result)
@router.get("/{folder_id}/documents")
async def get_folder_documents(folder_id: str):
"""List all documents in a specific folder."""
db = await get_db()
try:
# Verify folder exists
cursor = await db.execute(
"SELECT id FROM folders WHERE id = ?", (folder_id,)
)
if not await cursor.fetchone():
raise HTTPException(status_code=404, detail="Folder not found")
# Fetch documents in this folder
cursor = await db.execute(
"SELECT id, title, file_path, folder_id, content_type, created_at FROM documents WHERE folder_id = ? ORDER BY created_at DESC",
(folder_id,)
)
rows = await cursor.fetchall()
documents = [
{
"id": row["id"],
"title": row["title"],
"file_path": row["file_path"],
"folder_id": row["folder_id"],
"content_type": row["content_type"],
"created_at": row["created_at"]
}
for row in rows
]
return {"documents": documents}
finally:
await db.close()What do these endpoints handle?
- PATCH /folders/{id} renames a folder by updating its name field.
- PATCH /folders/{id}/move changes a folder's parent_id to support drag-and-drop reordering. It prevents circular references by checking ancestors.
- DELETE /folders/{id} cascade-deletes all subfolders and their documents recursively.
- GET /folders/{id}/documents lists all documents that belong to a specific folder.
- Save routers/folders.py.
- Register the router in your main app file by adding these lines:
from routers.folders import router as folders_router
# Register the folders router
app.include_router(folders_router)- Save your main app file and restart the server.
- Test the folder creation endpoint by running:
curl -X POST http://localhost:8000/folders -H "Content-Type: application/json" -d '{"name": "Physics 101"}'You should see a JSON response with the new folder's id, name, and created_at fields.
- Verify the tree endpoint returns your folders by running:
curl http://localhost:8000/folders/treeYou should see a JSON tree containing both "My Documents" and your new "Physics 101" folder.
Getting a 404 or connection error?
Make sure the server is running with --reload so it picks up your new router. Check that you have an __init__.py file in the routers directory (it can be empty).
help me debug my folder endpoints
✔️ Awesome, I've got everything!
Great. Double check you have saved routers/folders.py and your main app file.
ⓧ I'd like to double check the full code
Here is the complete routers/folders.py file:
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
from typing import Optional, List
import uuid
from datetime import datetime
from database import get_db
router = APIRouter(prefix="/folders", tags=["folders"])
class FolderCreate(BaseModel):
"""Request body for creating a new folder."""
name: str
parent_id: Optional[str] = None
class FolderRename(BaseModel):
"""Request body for renaming a folder."""
name: str
class FolderMove(BaseModel):
"""Request body for moving a folder to a new parent."""
parent_id: Optional[str] = None
@router.post("")
async def create_folder(body: FolderCreate):
"""Create a new folder with optional parent_id for nesting."""
db = await get_db()
try:
# Verify parent folder exists if parent_id is provided
if body.parent_id:
cursor = await db.execute(
"SELECT id FROM folders WHERE id = ?", (body.parent_id,)
)
if not await cursor.fetchone():
raise HTTPException(status_code=404, detail="Parent folder not found")
now = datetime.utcnow().isoformat()
folder_id = str(uuid.uuid4())
await db.execute(
"INSERT INTO folders (id, name, parent_id, created_at, updated_at) VALUES (?, ?, ?, ?, ?)",
(folder_id, body.name, body.parent_id, now, now)
)
await db.commit()
return {"id": folder_id, "name": body.name, "parent_id": body.parent_id, "created_at": now}
finally:
await db.close()
async def build_tree(db, parent_id=None) -> List[dict]:
"""Recursively build the folder tree structure."""
if parent_id is None:
cursor = await db.execute(
"SELECT id, name, parent_id, created_at, updated_at FROM folders WHERE parent_id IS NULL ORDER BY name"
)
else:
cursor = await db.execute(
"SELECT id, name, parent_id, created_at, updated_at FROM folders WHERE parent_id = ? ORDER BY name",
(parent_id,)
)
rows = await cursor.fetchall()
tree = []
for row in rows:
# Count documents in this folder
doc_cursor = await db.execute(
"SELECT COUNT(*) FROM documents WHERE folder_id = ?", (row["id"],)
)
doc_count = (await doc_cursor.fetchone())[0]
# Recursively get children
children = await build_tree(db, row["id"])
tree.append({
"id": row["id"],
"name": row["name"],
"parent_id": row["parent_id"],
"created_at": row["created_at"],
"updated_at": row["updated_at"],
"document_count": doc_count,
"children": children
})
return tree
@router.get("/tree")
async def get_folder_tree():
"""Return the full nested folder tree as recursive JSON."""
db = await get_db()
try:
tree = await build_tree(db)
return {"tree": tree}
finally:
await db.close()
@router.patch("/{folder_id}")
async def rename_folder(folder_id: str, body: FolderRename):
"""Rename a folder."""
db = await get_db()
try:
now = datetime.utcnow().isoformat()
cursor = await db.execute(
"UPDATE folders SET name = ?, updated_at = ? WHERE id = ?",
(body.name, now, folder_id)
)
if cursor.rowcount == 0:
raise HTTPException(status_code=404, detail="Folder not found")
await db.commit()
return {"id": folder_id, "name": body.name, "updated_at": now}
finally:
await db.close()
@router.patch("/{folder_id}/move")
async def move_folder(folder_id: str, body: FolderMove):
"""Move a folder to a new parent (drag-and-drop support)."""
db = await get_db()
try:
# Prevent moving a folder into itself or its own descendants
if body.parent_id:
ancestors = await get_folder_ancestors_list(db, body.parent_id)
if folder_id in ancestors:
raise HTTPException(
status_code=400,
detail="Cannot move a folder into its own subtree"
)
now = datetime.utcnow().isoformat()
cursor = await db.execute(
"UPDATE folders SET parent_id = ?, updated_at = ? WHERE id = ?",
(body.parent_id, now, folder_id)
)
if cursor.rowcount == 0:
raise HTTPException(status_code=404, detail="Folder not found")
await db.commit()
return {"id": folder_id, "parent_id": body.parent_id, "updated_at": now}
finally:
await db.close()
async def get_folder_ancestors_list(db, folder_id: str) -> List[str]:
"""Get list of ancestor folder IDs to prevent circular moves."""
ancestors = []
current_id = folder_id
while current_id:
ancestors.append(current_id)
cursor = await db.execute(
"SELECT parent_id FROM folders WHERE id = ?", (current_id,)
)
row = await cursor.fetchone()
if row and row["parent_id"]:
current_id = row["parent_id"]
else:
break
return ancestors
@router.delete("/{folder_id}")
async def delete_folder(folder_id: str):
"""Delete a folder and cascade delete all subfolders and their documents."""
db = await get_db()
try:
# Recursively collect all subfolder IDs
folder_ids_to_delete = []
await collect_subfolder_ids(db, folder_id, folder_ids_to_delete)
# Delete all documents in these folders
for fid in folder_ids_to_delete:
await db.execute("DELETE FROM documents WHERE folder_id = ?", (fid,))
# Delete all the folders (children first, parent last)
for fid in reversed(folder_ids_to_delete):
await db.execute("DELETE FROM folders WHERE id = ?", (fid,))
await db.commit()
return {"deleted": folder_ids_to_delete}
finally:
await db.close()
async def collect_subfolder_ids(db, folder_id: str, result: List[str]):
"""Recursively collect a folder and all its subfolder IDs."""
result.append(folder_id)
cursor = await db.execute(
"SELECT id FROM folders WHERE parent_id = ?", (folder_id,)
)
children = await cursor.fetchall()
for child in children:
await collect_subfolder_ids(db, child["id"], result)
@router.get("/{folder_id}/documents")
async def get_folder_documents(folder_id: str):
"""List all documents in a specific folder."""
db = await get_db()
try:
# Verify folder exists
cursor = await db.execute(
"SELECT id FROM folders WHERE id = ?", (folder_id,)
)
if not await cursor.fetchone():
raise HTTPException(status_code=404, detail="Folder not found")
# Fetch documents in this folder
cursor = await db.execute(
"SELECT id, title, file_path, folder_id, content_type, created_at FROM documents WHERE folder_id = ? ORDER BY created_at DESC",
(folder_id,)
)
rows = await cursor.fetchall()
documents = [
{
"id": row["id"],
"title": row["title"],
"file_path": row["file_path"],
"folder_id": row["folder_id"],
"content_type": row["content_type"],
"created_at": row["created_at"]
}
for row in rows
]
return {"documents": documents}
finally:
await db.close()Update the upload endpoint with folder metadata
Right now, your V1 upload endpoint ingests documents into ChromaDB without any folder context. Future features like scoped chat need to know which folder each chunk belongs to, so you need to enrich the metadata.
- Create a new file called utils.py in your project root and add the get_folder_ancestors utility function:
from database import get_db
async def get_folder_ancestors(folder_id: str) -> str:
"""Return the full ancestor path as a comma-separated string of folder IDs.
Example: for a folder nested 3 levels deep, returns
'root_id,parent_id,current_id' (from root to current).
This is stored in ChromaDB metadata for hierarchical queries.
"""
db = await get_db()
try:
ancestors = []
current_id = folder_id
# Walk up the tree from current folder to root
while current_id:
ancestors.append(current_id)
cursor = await db.execute(
"SELECT parent_id FROM folders WHERE id = ?", (current_id,)
)
row = await cursor.fetchone()
if row and row["parent_id"]:
current_id = row["parent_id"]
else:
break
# Reverse so root comes first
ancestors.reverse()
return ",".join(ancestors)
finally:
await db.close()Why store folder_path as a comma-separated string?
ChromaDB metadata values must be strings, integers, floats, or booleans. By storing the full ancestor path as a comma-separated string of IDs, you can later filter chunks by any folder in the hierarchy without needing to query SQLite first.
- Save utils.py.
- Now update your existing document upload endpoint to accept a folder_id parameter and enrich ChromaDB chunks with folder metadata. Find your upload endpoint and update it to include this logic:
import uuid
from datetime import datetime
from database import get_db
from utils import get_folder_ancestors
# Inside your upload endpoint, after processing the file:
# Accept folder_id as a form field or query parameter
async def upload_document(file: UploadFile, folder_id: str):
"""Upload a document, store metadata in SQLite, and ingest into ChromaDB with folder context."""
db = await get_db()
try:
# Verify the target folder exists
cursor = await db.execute(
"SELECT id FROM folders WHERE id = ?", (folder_id,)
)
if not await cursor.fetchone():
raise HTTPException(status_code=404, detail="Folder not found")
# Generate document ID and save the file
doc_id = str(uuid.uuid4())
file_path = f"./uploads/{doc_id}_{file.filename}"
# ... (your existing file save logic) ...
# Store document metadata in SQLite
now = datetime.utcnow().isoformat()
await db.execute(
"INSERT INTO documents (id, title, file_path, folder_id, content_type, created_at) VALUES (?, ?, ?, ?, ?, ?)",
(doc_id, file.filename, file_path, folder_id, file.content_type, now)
)
await db.commit()
# Get the full folder path for ChromaDB metadata
folder_path = await get_folder_ancestors(folder_id)
# When adding chunks to ChromaDB, include folder metadata
# For each chunk in your ingestion pipeline:
chunk_metadata = {
"document_id": doc_id,
"folder_id": folder_id,
"folder_path": folder_path,
"document_title": file.filename
}
# ... pass chunk_metadata to your ChromaDB collection.add() call ...
return {"id": doc_id, "title": file.filename, "folder_id": folder_id}
finally:
await db.close()What metadata fields are added to ChromaDB?
- document_id identifies which document a chunk belongs to.
- folder_id identifies the folder containing that document.
- folder_path stores the comma-separated ancestor IDs for hierarchical queries.
- document_title stores the human-readable filename for display in source citations later.
- Save your updated upload endpoint file.
- Finally, create a script to backfill existing ChromaDB chunks with the new metadata fields. Create backfill_chroma.py and paste:
"""Backfill existing ChromaDB chunks with folder_id, document_id, and folder_path metadata."""
import asyncio
import aiosqlite
import chromadb
from database import DATABASE_PATH
from utils import get_folder_ancestors
async def backfill():
"""Update existing ChromaDB records with folder metadata from SQLite."""
# Connect to ChromaDB
client = chromadb.PersistentClient(path="./chroma_data")
collection = client.get_collection("documents") # Adjust collection name to match your V1
# Get all documents from SQLite
async with aiosqlite.connect(DATABASE_PATH) as db:
db.row_factory = aiosqlite.Row
cursor = await db.execute("SELECT id, title, folder_id FROM documents")
documents = await cursor.fetchall()
updated_count = 0
for doc in documents:
doc_id = doc["id"]
folder_id = doc["folder_id"]
title = doc["title"]
# Get the folder ancestor path
folder_path = await get_folder_ancestors(folder_id)
# Find chunks in ChromaDB that belong to this document
# Adjust the query based on how V1 stored document references
try:
results = collection.get(
where={"source": title}, # Adjust filter to match your V1 metadata key
include=["metadatas"]
)
if results and results["ids"]:
# Update metadata for each matching chunk
collection.update(
ids=results["ids"],
metadatas=[
{
**meta,
"document_id": doc_id,
"folder_id": folder_id,
"folder_path": folder_path,
"document_title": title
}
for meta in results["metadatas"]
]
)
updated_count += len(results["ids"])
except Exception as e:
print(f"Warning: Could not update chunks for '{title}': {e}")
print(f"Backfill complete. Updated {updated_count} chunks with folder metadata.")
if __name__ == "__main__":
asyncio.run(backfill())What does this script do?
This script cross-references your SQLite document records with existing ChromaDB chunks. For each document, it finds matching chunks and adds the document_id, folder_id, folder_path, and document_title fields to their metadata.
You may need to adjust the where filter to match how your V1 stored document references in ChromaDB metadata.
- Save backfill_chroma.py and run the backfill script:
python backfill_chroma.pyYou should see output confirming how many chunks were updated with folder metadata.
Getting errors during backfill?
The most common issue is a mismatch between how V1 stored document references in ChromaDB and the filter used in the script. Check what metadata keys your V1 ingestion pipeline set on each chunk, then update the where filter accordingly.
help me match my V1 ChromaDB metadata to the backfill script
✔️ Awesome, I've got everything!
Great. Make sure you have saved utils.py, your updated upload endpoint, and backfill_chroma.py.
ⓧ I'd like to double check the full code
Here is the complete utils.py file:
from database import get_db
async def get_folder_ancestors(folder_id: str) -> str:
"""Return the full ancestor path as a comma-separated string of folder IDs.
Example: for a folder nested 3 levels deep, returns
'root_id,parent_id,current_id' (from root to current).
This is stored in ChromaDB metadata for hierarchical queries.
"""
db = await get_db()
try:
ancestors = []
current_id = folder_id
# Walk up the tree from current folder to root
while current_id:
ancestors.append(current_id)
cursor = await db.execute(
"SELECT parent_id FROM folders WHERE id = ?", (current_id,)
)
row = await cursor.fetchone()
if row and row["parent_id"]:
current_id = row["parent_id"]
else:
break
# Reverse so root comes first
ancestors.reverse()
return ",".join(ancestors)
finally:
await db.close()Here is the complete database.py file:
import aiosqlite
import uuid
from datetime import datetime
DATABASE_PATH = "e-hanatra.db"
async def get_db():
"""Get a database connection with row factory enabled."""
db = await aiosqlite.connect(DATABASE_PATH)
db.row_factory = aiosqlite.Row
return db
async def init_db():
"""Create tables if they don't exist and seed default folder."""
async with aiosqlite.connect(DATABASE_PATH) as db:
# Enable WAL mode for better read performance
await db.execute("PRAGMA journal_mode=WAL")
# Folders table using adjacency list pattern
# parent_id is nullable (NULL = root-level folder)
await db.execute("""
CREATE TABLE IF NOT EXISTS folders (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
parent_id TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
FOREIGN KEY (parent_id) REFERENCES folders(id)
)
""")
# Documents table linking files to folders
await db.execute("""
CREATE TABLE IF NOT EXISTS documents (
id TEXT PRIMARY KEY,
title TEXT NOT NULL,
file_path TEXT NOT NULL,
folder_id TEXT NOT NULL,
content_type TEXT NOT NULL,
created_at TEXT NOT NULL,
FOREIGN KEY (folder_id) REFERENCES folders(id)
)
""")
# Create the default root folder if it doesn't exist
existing = await db.execute(
"SELECT id FROM folders WHERE name = ? AND parent_id IS NULL",
("My Documents",)
)
row = await existing.fetchone()
if not row:
now = datetime.utcnow().isoformat()
await db.execute(
"INSERT INTO folders (id, name, parent_id, created_at, updated_at) VALUES (?, ?, NULL, ?, ?)",
(str(uuid.uuid4()), "My Documents", now, now)
)
await db.commit()Your backend now has a complete hierarchical folder system with database storage, CRUD endpoints, and enriched ChromaDB metadata. Next up, you will build the frontend folder tree component so students can actually see and interact with their organized folders.
Build the Hierarchical Folder System — Frontend
Your backend folder API is live and ready to serve data. Now it is time to give students a visual way to organize their course materials.
In this step, you will build a recursive folder tree sidebar, add drag-and-drop interactions, and wire up breadcrumb navigation so learners always know where they are in their knowledge hierarchy.
In this step, get ready to:
- Build a recursive FolderTree sidebar component with expand/collapse and document count badges.
- Add right-click context menus, inline rename, and HTML5 drag-and-drop reordering.
- Create breadcrumb navigation, a folder selector in the upload dialog, and a grid/list contents view.
Build the FolderTree sidebar component
A folder tree needs to render itself recursively. Each folder can contain subfolders, and each subfolder can contain more subfolders. React handles this naturally by letting a component render instances of itself.
- Create a new file components/folders/FolderNode.tsx inside your Next.js project. This component renders a single folder and recursively renders its children.
- Add the following code to define the types and recursive node:
'use client'
import { useState } from 'react'
// Shape of a folder returned by GET /folders/tree
export interface FolderTreeNode {
id: string
name: string
parent_id: string | null
document_count: number
children: FolderTreeNode[]
}
interface FolderNodeProps {
folder: FolderTreeNode
depth: number
selectedFolderId: string | null
onSelect: (folder: FolderTreeNode) => void
}
export default function FolderNode({
folder,
depth,
selectedFolderId,
onSelect,
}: FolderNodeProps) {
// Track whether this node's children are visible
const [expanded, setExpanded] = useState(depth === 0)
const hasChildren = folder.children.length > 0
const isSelected = folder.id === selectedFolderId
return (
<div className="folder-node-wrapper">
<div
className={`folder-node ${isSelected ? 'selected' : ''}`}
style={{ paddingLeft: `${depth * 16 + 8}px` }}
onClick={() => onSelect(folder)}
>
{/* Expand/collapse chevron */}
<button
className="chevron-btn"
onClick={(e) => {
e.stopPropagation()
setExpanded(!expanded)
}}
aria-label={expanded ? 'Collapse folder' : 'Expand folder'}
>
{hasChildren ? (expanded ? '▾' : '▸') : '·'}
</button>
{/* Folder icon */}
<span className="folder-icon">{expanded ? '📂' : '📁'}</span>
{/* Folder name */}
<span className="folder-name">{folder.name}</span>
{/* Document count badge */}
{folder.document_count > 0 && (
<span className="doc-count-badge">{folder.document_count}</span>
)}
</div>
{/* Recursively render children when expanded */}
{expanded && hasChildren && (
<div className="folder-children">
{folder.children.map((child) => (
<FolderNode
key={child.id}
folder={child}
depth={depth + 1}
selectedFolderId={selectedFolderId}
onSelect={onSelect}
/>
))}
</div>
)}
</div>
)
}What does this code do?
The FolderTreeNode interface mirrors the recursive JSON your GET /folders/tree endpoint returns. Each node has a children array of the same type.
The component renders itself by mapping over folder.children and returning another FolderNode for each child. The depth prop increases with each level, creating indentation via paddingLeft.
- Save components/folders/FolderNode.tsx.
Now create the parent component that fetches the tree data and renders the root-level nodes.
- Create components/folders/FolderTree.tsx and add this code:
'use client'
import { useState, useEffect } from 'react'
import FolderNode, { FolderTreeNode } from './FolderNode'
interface FolderTreeProps {
onFolderSelect: (folder: FolderTreeNode) => void
selectedFolderId: string | null
}
export default function FolderTree({
onFolderSelect,
selectedFolderId,
}: FolderTreeProps) {
const [tree, setTree] = useState<FolderTreeNode[]>([])
const [loading, setLoading] = useState(true)
// Fetch the full folder tree from the backend
useEffect(() => {
async function fetchTree() {
try {
const res = await fetch('/api/folders/tree')
const data = await res.json()
setTree(data)
} catch (error) {
console.error('Failed to fetch folder tree:', error)
} finally {
setLoading(false)
}
}
fetchTree()
}, [])
if (loading) {
return <div className="folder-tree-loading">Loading folders...</div>
}
return (
<aside className="folder-tree-sidebar">
<div className="folder-tree-header">
<h3>Documents</h3>
</div>
<div className="folder-tree-list">
{tree.map((rootFolder) => (
<FolderNode
key={rootFolder.id}
folder={rootFolder}
depth={0}
selectedFolderId={selectedFolderId}
onSelect={onFolderSelect}
/>
))}
</div>
</aside>
)
}What does this code do?
The FolderTree component fetches the full nested tree from your FastAPI backend on mount. It renders root-level folders (those with no parent_id), including the default My Documents folder.
The onFolderSelect callback propagates the selected folder up to the parent layout so other components (breadcrumbs, content view) can react.
- Save components/folders/FolderTree.tsx.
Next, add the dark theme styles to match V1's visual language.
- Create components/folders/FolderTree.css with the following styles:
/* Sidebar container */
.folder-tree-sidebar {
width: 260px;
background-color: #1a1a1a;
border-right: 1px solid #2a2a2a;
height: 100vh;
overflow-y: auto;
padding: 12px 0;
}
.folder-tree-header {
padding: 8px 16px 12px;
border-bottom: 1px solid #2a2a2a;
}
.folder-tree-header h3 {
color: #e0e0e0;
font-size: 14px;
font-weight: 600;
margin: 0;
}
/* Individual folder node */
.folder-node {
display: flex;
align-items: center;
gap: 6px;
padding: 6px 8px;
cursor: pointer;
border-radius: 4px;
margin: 1px 8px;
transition: background-color 0.15s;
}
.folder-node:hover {
background-color: #2a2a2a;
}
.folder-node.selected {
background-color: #3a3a5c;
border: 1px solid #6366f1;
}
/* Chevron toggle button */
.chevron-btn {
background: none;
border: none;
color: #888;
font-size: 12px;
width: 16px;
cursor: pointer;
padding: 0;
}
.folder-icon {
font-size: 14px;
}
.folder-name {
color: #d0d0d0;
font-size: 13px;
flex: 1;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
/* Document count badge */
.doc-count-badge {
background-color: #3a3a3a;
color: #999;
font-size: 11px;
padding: 1px 6px;
border-radius: 10px;
}
/* Drop target highlight */
.folder-node.drop-target {
background-color: #2a2a4a;
border: 1px dashed #6366f1;
}Why these specific colors?
The dark background (#1a1a1a) and subtle borders (#2a2a2a) match V1's existing dark theme. The indigo accent (#6366f1) highlights the selected folder state, maintaining visual consistency across the app.
- Save components/folders/FolderTree.css.
- Import this CSS file at the top of FolderTree.tsx by adding import './FolderTree.css'.
- Open your app in the browser. You should see the folder tree sidebar on the left with your My Documents root folder, an expand/collapse chevron, and a document count badge.
Sidebar not showing up?
Make sure FolderTree is imported and rendered in your main layout component. Check that the CSS import path is correct relative to the component file.
If the tree loads empty, verify your FastAPI server is running and the GET /folders/tree endpoint returns data. Check the browser console for fetch errors.
help me debug why my FolderTree sidebar is not rendering
Add folder interaction features
A static tree is useful for viewing, but students need to reorganize their folders as courses progress. You will add a right-click context menu, inline renaming, and drag-and-drop reordering.
- Create components/folders/FolderContextMenu.tsx for the right-click menu:
'use client'
import { useEffect, useRef } from 'react'
interface ContextMenuProps {
x: number
y: number
onClose: () => void
onNewSubfolder: () => void
onRename: () => void
onMoveTo: () => void
onDelete: () => void
}
export default function FolderContextMenu({
x,
y,
onClose,
onNewSubfolder,
onRename,
onMoveTo,
onDelete,
}: ContextMenuProps) {
const menuRef = useRef<HTMLDivElement>(null)
// Close the menu when clicking outside
useEffect(() => {
function handleClickOutside(e: MouseEvent) {
if (menuRef.current && !menuRef.current.contains(e.target as Node)) {
onClose()
}
}
document.addEventListener('mousedown', handleClickOutside)
return () => document.removeEventListener('mousedown', handleClickOutside)
}, [onClose])
return (
<div
ref={menuRef}
className="folder-context-menu"
style={{ top: y, left: x }}
>
<button onClick={onNewSubfolder}>New Subfolder</button>
<button onClick={onRename}>Rename</button>
<button onClick={onMoveTo}>Move to...</button>
<div className="context-menu-divider" />
<button className="danger" onClick={onDelete}>Delete</button>
</div>
)
}What does this code do?
The context menu renders at the mouse coordinates (x/y) passed in as props. It listens for clicks outside itself and calls onClose to dismiss.
Each menu item triggers a callback that the parent component handles. The Delete button uses a danger class so you can style it red to signal a destructive action.
- Save components/folders/FolderContextMenu.tsx.
Now update the FolderNode component to support right-click, inline rename, and drag-and-drop.
- Open components/folders/FolderNode.tsx and replace the component with this enhanced version that adds all three interaction features:
'use client'
import { useState, useRef } from 'react'
import FolderContextMenu from './FolderContextMenu'
export interface FolderTreeNode {
id: string
name: string
parent_id: string | null
document_count: number
children: FolderTreeNode[]
}
interface FolderNodeProps {
folder: FolderTreeNode
depth: number
selectedFolderId: string | null
onSelect: (folder: FolderTreeNode) => void
onMove: (folderId: string, newParentId: string) => void
onRename: (folderId: string, newName: string) => void
onDelete: (folderId: string) => void
onNewSubfolder: (parentId: string) => void
}
export default function FolderNode({
folder,
depth,
selectedFolderId,
onSelect,
onMove,
onRename,
onDelete,
onNewSubfolder,
}: FolderNodeProps) {
const [expanded, setExpanded] = useState(depth === 0)
const [isRenaming, setIsRenaming] = useState(false)
const [renameValue, setRenameValue] = useState(folder.name)
const [isDragOver, setIsDragOver] = useState(false)
const [contextMenu, setContextMenu] = useState<{ x: number; y: number } | null>(null)
const inputRef = useRef<HTMLInputElement>(null)
const hasChildren = folder.children.length > 0
const isSelected = folder.id === selectedFolderId
// Handle right-click to show context menu
function handleContextMenu(e: React.MouseEvent) {
e.preventDefault()
setContextMenu({ x: e.clientX, y: e.clientY })
}
// Handle inline rename on double-click
function handleDoubleClick() {
setIsRenaming(true)
setRenameValue(folder.name)
setTimeout(() => inputRef.current?.select(), 0)
}
// Submit rename on Enter or blur
function handleRenameSubmit() {
if (renameValue.trim() && renameValue !== folder.name) {
onRename(folder.id, renameValue.trim())
}
setIsRenaming(false)
}
// HTML5 Drag-and-Drop handlers
function handleDragStart(e: React.DragEvent) {
e.dataTransfer.setData('text/plain', folder.id)
e.dataTransfer.effectAllowed = 'move'
}
function handleDragOver(e: React.DragEvent) {
e.preventDefault()
e.dataTransfer.dropEffect = 'move'
setIsDragOver(true)
}
function handleDragLeave() {
setIsDragOver(false)
}
function handleDrop(e: React.DragEvent) {
e.preventDefault()
setIsDragOver(false)
const draggedFolderId = e.dataTransfer.getData('text/plain')
// Prevent dropping a folder onto itself
if (draggedFolderId !== folder.id) {
onMove(draggedFolderId, folder.id)
}
}
return (
<div className="folder-node-wrapper">
<div
className={`folder-node ${isSelected ? 'selected' : ''} ${isDragOver ? 'drop-target' : ''}`}
style={{ paddingLeft: `${depth * 16 + 8}px` }}
onClick={() => onSelect(folder)}
onContextMenu={handleContextMenu}
onDoubleClick={handleDoubleClick}
draggable
onDragStart={handleDragStart}
onDragOver={handleDragOver}
onDragLeave={handleDragLeave}
onDrop={handleDrop}
>
<button
className="chevron-btn"
onClick={(e) => {
e.stopPropagation()
setExpanded(!expanded)
}}
>
{hasChildren ? (expanded ? '▾' : '▸') : '·'}
</button>
<span className="folder-icon">{expanded ? '📂' : '📁'}</span>
{/* Show input field when renaming, otherwise show name */}
{isRenaming ? (
<input
ref={inputRef}
className="rename-input"
value={renameValue}
onChange={(e) => setRenameValue(e.target.value)}
onBlur={handleRenameSubmit}
onKeyDown={(e) => {
if (e.key === 'Enter') handleRenameSubmit()
if (e.key === 'Escape') setIsRenaming(false)
}}
/>
) : (
<span className="folder-name">{folder.name}</span>
)}
{folder.document_count > 0 && (
<span className="doc-count-badge">{folder.document_count}</span>
)}
</div>
{/* Context menu portal */}
{contextMenu && (
<FolderContextMenu
x={contextMenu.x}
y={contextMenu.y}
onClose={() => setContextMenu(null)}
onNewSubfolder={() => {
onNewSubfolder(folder.id)
setContextMenu(null)
}}
onRename={() => {
handleDoubleClick()
setContextMenu(null)
}}
onMoveTo={() => setContextMenu(null)}
onDelete={() => {
onDelete(folder.id)
setContextMenu(null)
}}
/>
)}
{expanded && hasChildren && (
<div className="folder-children">
{folder.children.map((child) => (
<FolderNode
key={child.id}
folder={child}
depth={depth + 1}
selectedFolderId={selectedFolderId}
onSelect={onSelect}
onMove={onMove}
onRename={onRename}
onDelete={onDelete}
onNewSubfolder={onNewSubfolder}
/>
))}
</div>
)}
</div>
)
}What's happening here?
Three interaction systems are layered onto the folder node:
- Right-click (onContextMenu) captures mouse coordinates and shows the context menu at that position.
- Double-click toggles isRenaming state, swapping the folder name span for an input field that auto-selects the text.
- Drag-and-drop uses onDragStart to store the dragged folder's ID, onDragOver to show the drop indicator, and onDrop to call onMove which triggers PATCH /folders/{id}/move on the backend.
- Save components/folders/FolderNode.tsx.
- Add these styles to your FolderTree.css for the context menu and rename input:
/* Context menu */
.folder-context-menu {
position: fixed;
background: #252525;
border: 1px solid #3a3a3a;
border-radius: 6px;
padding: 4px 0;
min-width: 160px;
box-shadow: 0 4px 12px rgba(0, 0, 0, 0.4);
z-index: 1000;
}
.folder-context-menu button {
display: block;
width: 100%;
padding: 8px 14px;
background: none;
border: none;
color: #d0d0d0;
font-size: 13px;
text-align: left;
cursor: pointer;
}
.folder-context-menu button:hover {
background-color: #3a3a3a;
}
.folder-context-menu button.danger {
color: #ef4444;
}
.context-menu-divider {
height: 1px;
background: #3a3a3a;
margin: 4px 0;
}
/* Inline rename input */
.rename-input {
background: #2a2a2a;
border: 1px solid #6366f1;
border-radius: 3px;
color: #e0e0e0;
font-size: 13px;
padding: 2px 4px;
flex: 1;
outline: none;
}- Save FolderTree.css.
- Refresh your browser. Right-click any folder to see the context menu. Double-click a folder name to enter inline rename mode. Drag one folder onto another to see the dashed drop indicator.
Context menu not appearing on right-click?
Check that you added onContextMenu={handleContextMenu} to the folder node's div. The e.preventDefault() call is essential to suppress the browser's default right-click menu.
If drag-and-drop does not highlight the target folder, verify that the .drop-target class is in your CSS and that handleDragOver calls e.preventDefault() (required to allow drops).
help me debug the folder drag-and-drop interactions
Add breadcrumb navigation and folder contents view
With the tree built, students need two more navigation aids. A breadcrumb bar tells them exactly where they are in the folder hierarchy. A contents view shows the documents and subfolders inside the selected folder.
- Create components/folders/Breadcrumb.tsx to display the current path:
'use client'
interface BreadcrumbItem {
id: string
name: string
}
interface BreadcrumbProps {
path: BreadcrumbItem[]
onNavigate: (folderId: string) => void
}
export default function Breadcrumb({ path, onNavigate }: BreadcrumbProps) {
return (
<nav className="breadcrumb-nav" aria-label="Folder navigation">
{/* Home is always first */}
<button
className="breadcrumb-item"
onClick={() => onNavigate('root')}
>
Home
</button>
{path.map((item, index) => (
<span key={item.id} className="breadcrumb-segment">
<span className="breadcrumb-separator">/</span>
{index === path.length - 1 ? (
// Last item is current folder, not clickable
<span className="breadcrumb-current">{item.name}</span>
) : (
<button
className="breadcrumb-item"
onClick={() => onNavigate(item.id)}
>
{item.name}
</button>
)}
</span>
))}
</nav>
)
}What does this code do?
The breadcrumb renders each ancestor folder as a clickable button. The path array comes from your get_folder_ancestors() backend utility. The last item in the path is the current folder, so it renders as plain text instead of a button.
Clicking any ancestor folder calls onNavigate, which updates the selected folder and refreshes the contents view below.
- Save components/folders/Breadcrumb.tsx.
Now build the folder contents component that shows documents and subfolders in a grid or list layout.
- Create components/folders/FolderContents.tsx with grid/list view toggling and file type icons:
'use client'
import { useState, useEffect } from 'react'
interface Document {
id: string
title: string
content_type: string
created_at: string
}
interface FolderContentsProps {
folderId: string
onDocumentSelect: (doc: Document) => void
onMoveDocument: (docId: string) => void
}
// Map content types to visual icons
function getFileIcon(contentType: string): string {
if (contentType.includes('pdf')) return '📄'
if (contentType.includes('image')) return '🖼️'
if (contentType.includes('audio')) return '🎵'
if (contentType.includes('text')) return '📝'
return '📎'
}
export default function FolderContents({
folderId,
onDocumentSelect,
onMoveDocument,
}: FolderContentsProps) {
const [documents, setDocuments] = useState<Document[]>([])
const [viewMode, setViewMode] = useState<'grid' | 'list'>('grid')
const [docContextMenu, setDocContextMenu] = useState<{
docId: string
x: number
y: number
} | null>(null)
// Fetch documents when the selected folder changes
useEffect(() => {
async function fetchDocuments() {
const res = await fetch(`/api/folders/${folderId}/documents`)
const data = await res.json()
setDocuments(data)
}
if (folderId) fetchDocuments()
}, [folderId])
// Handle right-click on a document
function handleDocContextMenu(e: React.MouseEvent, docId: string) {
e.preventDefault()
setDocContextMenu({ docId, x: e.clientX, y: e.clientY })
}
return (
<div className="folder-contents">
{/* View mode toggle */}
<div className="contents-toolbar">
<span className="doc-count">{documents.length} documents</span>
<div className="view-toggle">
<button
className={viewMode === 'grid' ? 'active' : ''}
onClick={() => setViewMode('grid')}
>
Grid
</button>
<button
className={viewMode === 'list' ? 'active' : ''}
onClick={() => setViewMode('list')}
>
List
</button>
</div>
</div>
{/* Document items */}
<div className={`contents-${viewMode}`}>
{documents.map((doc) => (
<div
key={doc.id}
className={`doc-item doc-item-${viewMode}`}
onClick={() => onDocumentSelect(doc)}
onContextMenu={(e) => handleDocContextMenu(e, doc.id)}
>
<span className="doc-icon">{getFileIcon(doc.content_type)}</span>
<span className="doc-title">{doc.title}</span>
</div>
))}
</div>
{/* Document context menu with Move option */}
{docContextMenu && (
<div
className="folder-context-menu"
style={{ top: docContextMenu.y, left: docContextMenu.x }}
>
<button onClick={() => {
onMoveDocument(docContextMenu.docId)
setDocContextMenu(null)
}}>
Move Document
</button>
</div>
)}
</div>
)
}What does this code do?
The FolderContents component fetches documents from GET /folders/{id}/documents whenever the folderId prop changes. It renders documents in either a grid or list layout based on the toggle.
The getFileIcon helper maps each document's content_type to a visual emoji icon. Right-clicking a document shows a context menu with a Move Document option that triggers the folder selection flow.
- Save components/folders/FolderContents.tsx.
Finally, update your document upload dialog to include a folder selector dropdown.
- Create components/folders/FolderSelector.tsx to provide a dropdown that shows the tree structure in a flat list:
'use client'
import { FolderTreeNode } from './FolderNode'
interface FolderSelectorProps {
tree: FolderTreeNode[]
selectedFolderId: string
onChange: (folderId: string) => void
}
// Flatten the tree into indented options for a select dropdown
function flattenTree(nodes: FolderTreeNode[], depth: number = 0): { id: string; label: string }[] {
const result: { id: string; label: string }[] = []
for (const node of nodes) {
// Use dashes to indicate nesting level
const indent = '—'.repeat(depth)
result.push({ id: node.id, label: `${indent} ${node.name}` })
if (node.children.length > 0) {
result.push(...flattenTree(node.children, depth + 1))
}
}
return result
}
export default function FolderSelector({
tree,
selectedFolderId,
onChange,
}: FolderSelectorProps) {
const options = flattenTree(tree)
return (
<div className="folder-selector">
<label htmlFor="folder-select">Upload to folder:</label>
<select
id="folder-select"
value={selectedFolderId}
onChange={(e) => onChange(e.target.value)}
className="folder-select-input"
>
{options.map((opt) => (
<option key={opt.id} value={opt.id}>
{opt.label}
</option>
))}
</select>
</div>
)
}What does this code do?
The flattenTree utility recursively walks the tree and builds a flat array of options with indentation dashes to represent nesting. This lets a standard HTML <select> element display the hierarchy.
You will import this FolderSelector into your existing upload dialog and pass the folder_id to your upload endpoint so documents land in the correct folder.
- Save components/folders/FolderSelector.tsx.
- Import FolderSelector into your existing document upload dialog component. Pass it the tree data and include the selected folder_id in your upload POST request body.
- Add the following styles to FolderTree.css for the breadcrumb and folder contents:
/* Breadcrumb navigation */
.breadcrumb-nav {
display: flex;
align-items: center;
gap: 2px;
padding: 10px 16px;
background: #1e1e1e;
border-bottom: 1px solid #2a2a2a;
font-size: 13px;
}
.breadcrumb-item {
background: none;
border: none;
color: #6366f1;
cursor: pointer;
font-size: 13px;
padding: 2px 4px;
border-radius: 3px;
}
.breadcrumb-item:hover {
background: #2a2a2a;
}
.breadcrumb-separator {
color: #555;
margin: 0 4px;
}
.breadcrumb-current {
color: #d0d0d0;
}
/* Folder contents */
.folder-contents {
flex: 1;
padding: 16px;
}
.contents-toolbar {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 16px;
}
.doc-count {
color: #888;
font-size: 13px;
}
.view-toggle button {
background: #2a2a2a;
border: 1px solid #3a3a3a;
color: #aaa;
padding: 4px 10px;
font-size: 12px;
cursor: pointer;
}
.view-toggle button.active {
background: #3a3a5c;
color: #e0e0e0;
border-color: #6366f1;
}
/* Grid view */
.contents-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(140px, 1fr));
gap: 12px;
}
.doc-item-grid {
display: flex;
flex-direction: column;
align-items: center;
padding: 16px 8px;
background: #222;
border: 1px solid #2a2a2a;
border-radius: 8px;
cursor: pointer;
transition: border-color 0.15s;
}
.doc-item-grid:hover {
border-color: #6366f1;
}
.doc-item-grid .doc-icon {
font-size: 28px;
margin-bottom: 8px;
}
.doc-item-grid .doc-title {
color: #d0d0d0;
font-size: 12px;
text-align: center;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
max-width: 100%;
}
/* List view */
.contents-list {
display: flex;
flex-direction: column;
gap: 4px;
}
.doc-item-list {
display: flex;
align-items: center;
gap: 10px;
padding: 8px 12px;
border-radius: 4px;
cursor: pointer;
}
.doc-item-list:hover {
background: #2a2a2a;
}
.doc-item-list .doc-title {
color: #d0d0d0;
font-size: 13px;
}
/* Folder selector in upload dialog */
.folder-selector {
margin: 12px 0;
}
.folder-selector label {
display: block;
color: #aaa;
font-size: 12px;
margin-bottom: 4px;
}
.folder-select-input {
width: 100%;
padding: 8px;
background: #2a2a2a;
border: 1px solid #3a3a3a;
border-radius: 4px;
color: #e0e0e0;
font-size: 13px;
}- Save FolderTree.css.
- Refresh your browser. Click folders in the sidebar and confirm the breadcrumb updates to show the full path (Home / My Documents / Subfolder). Toggle between grid and list view in the contents area to see documents with their file type icons.
Breadcrumb not updating when you click folders?
The breadcrumb's path prop depends on calling your backend's folder ancestors endpoint. Make sure your parent component fetches the path array from get_folder_ancestors() whenever the selected folder changes.
If the grid/list toggle does nothing, check that your CSS file includes both .contents-grid and .contents-list classes and that the class name is built dynamically as contents-${viewMode}.
help me debug the breadcrumb and folder contents components
✔️ Awesome, I've got everything!
Great. Double check you have saved all files in components/folders/.
ⓧ I'd like to double check the full code
Here are the complete files you should have:
'use client'
import { useState, useRef } from 'react'
import FolderContextMenu from './FolderContextMenu'
export interface FolderTreeNode {
id: string
name: string
parent_id: string | null
document_count: number
children: FolderTreeNode[]
}
interface FolderNodeProps {
folder: FolderTreeNode
depth: number
selectedFolderId: string | null
onSelect: (folder: FolderTreeNode) => void
onMove: (folderId: string, newParentId: string) => void
onRename: (folderId: string, newName: string) => void
onDelete: (folderId: string) => void
onNewSubfolder: (parentId: string) => void
}
export default function FolderNode({
folder,
depth,
selectedFolderId,
onSelect,
onMove,
onRename,
onDelete,
onNewSubfolder,
}: FolderNodeProps) {
const [expanded, setExpanded] = useState(depth === 0)
const [isRenaming, setIsRenaming] = useState(false)
const [renameValue, setRenameValue] = useState(folder.name)
const [isDragOver, setIsDragOver] = useState(false)
const [contextMenu, setContextMenu] = useState<{ x: number; y: number } | null>(null)
const inputRef = useRef<HTMLInputElement>(null)
const hasChildren = folder.children.length > 0
const isSelected = folder.id === selectedFolderId
function handleContextMenu(e: React.MouseEvent) {
e.preventDefault()
setContextMenu({ x: e.clientX, y: e.clientY })
}
function handleDoubleClick() {
setIsRenaming(true)
setRenameValue(folder.name)
setTimeout(() => inputRef.current?.select(), 0)
}
function handleRenameSubmit() {
if (renameValue.trim() && renameValue !== folder.name) {
onRename(folder.id, renameValue.trim())
}
setIsRenaming(false)
}
function handleDragStart(e: React.DragEvent) {
e.dataTransfer.setData('text/plain', folder.id)
e.dataTransfer.effectAllowed = 'move'
}
function handleDragOver(e: React.DragEvent) {
e.preventDefault()
e.dataTransfer.dropEffect = 'move'
setIsDragOver(true)
}
function handleDragLeave() {
setIsDragOver(false)
}
function handleDrop(e: React.DragEvent) {
e.preventDefault()
setIsDragOver(false)
const draggedFolderId = e.dataTransfer.getData('text/plain')
if (draggedFolderId !== folder.id) {
onMove(draggedFolderId, folder.id)
}
}
return (
<div className="folder-node-wrapper">
<div
className={`folder-node ${isSelected ? 'selected' : ''} ${isDragOver ? 'drop-target' : ''}`}
style={{ paddingLeft: `${depth * 16 + 8}px` }}
onClick={() => onSelect(folder)}
onContextMenu={handleContextMenu}
onDoubleClick={handleDoubleClick}
draggable
onDragStart={handleDragStart}
onDragOver={handleDragOver}
onDragLeave={handleDragLeave}
onDrop={handleDrop}
>
<button
className="chevron-btn"
onClick={(e) => {
e.stopPropagation()
setExpanded(!expanded)
}}
>
{hasChildren ? (expanded ? '▾' : '▸') : '·'}
</button>
<span className="folder-icon">{expanded ? '📂' : '📁'}</span>
{isRenaming ? (
<input
ref={inputRef}
className="rename-input"
value={renameValue}
onChange={(e) => setRenameValue(e.target.value)}
onBlur={handleRenameSubmit}
onKeyDown={(e) => {
if (e.key === 'Enter') handleRenameSubmit()
if (e.key === 'Escape') setIsRenaming(false)
}}
/>
) : (
<span className="folder-name">{folder.name}</span>
)}
{folder.document_count > 0 && (
<span className="doc-count-badge">{folder.document_count}</span>
)}
</div>
{contextMenu && (
<FolderContextMenu
x={contextMenu.x}
y={contextMenu.y}
onClose={() => setContextMenu(null)}
onNewSubfolder={() => {
onNewSubfolder(folder.id)
setContextMenu(null)
}}
onRename={() => {
handleDoubleClick()
setContextMenu(null)
}}
onMoveTo={() => setContextMenu(null)}
onDelete={() => {
onDelete(folder.id)
setContextMenu(null)
}}
/>
)}
{expanded && hasChildren && (
<div className="folder-children">
{folder.children.map((child) => (
<FolderNode
key={child.id}
folder={child}
depth={depth + 1}
selectedFolderId={selectedFolderId}
onSelect={onSelect}
onMove={onMove}
onRename={onRename}
onDelete={onDelete}
onNewSubfolder={onNewSubfolder}
/>
))}
</div>
)}
</div>
)
}'use client'
import { useState, useEffect } from 'react'
import FolderNode, { FolderTreeNode } from './FolderNode'
import './FolderTree.css'
interface FolderTreeProps {
onFolderSelect: (folder: FolderTreeNode) => void
selectedFolderId: string | null
}
export default function FolderTree({
onFolderSelect,
selectedFolderId,
}: FolderTreeProps) {
const [tree, setTree] = useState<FolderTreeNode[]>([])
const [loading, setLoading] = useState(true)
useEffect(() => {
async function fetchTree() {
try {
const res = await fetch('/api/folders/tree')
const data = await res.json()
setTree(data)
} catch (error) {
console.error('Failed to fetch folder tree:', error)
} finally {
setLoading(false)
}
}
fetchTree()
}, [])
// Handler for drag-and-drop move
async function handleMove(folderId: string, newParentId: string) {
await fetch(`/api/folders/${folderId}/move`, {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ parent_id: newParentId }),
})
// Re-fetch tree to reflect new structure
const res = await fetch('/api/folders/tree')
const data = await res.json()
setTree(data)
}
async function handleRename(folderId: string, newName: string) {
await fetch(`/api/folders/${folderId}`, {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ name: newName }),
})
const res = await fetch('/api/folders/tree')
const data = await res.json()
setTree(data)
}
async function handleDelete(folderId: string) {
await fetch(`/api/folders/${folderId}`, { method: 'DELETE' })
const res = await fetch('/api/folders/tree')
const data = await res.json()
setTree(data)
}
async function handleNewSubfolder(parentId: string) {
await fetch('/api/folders', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ name: 'New Folder', parent_id: parentId }),
})
const res = await fetch('/api/folders/tree')
const data = await res.json()
setTree(data)
}
if (loading) {
return <div className="folder-tree-loading">Loading folders...</div>
}
return (
<aside className="folder-tree-sidebar">
<div className="folder-tree-header">
<h3>Documents</h3>
</div>
<div className="folder-tree-list">
{tree.map((rootFolder) => (
<FolderNode
key={rootFolder.id}
folder={rootFolder}
depth={0}
selectedFolderId={selectedFolderId}
onSelect={onFolderSelect}
onMove={handleMove}
onRename={handleRename}
onDelete={handleDelete}
onNewSubfolder={handleNewSubfolder}
/>
))}
</div>
</aside>
)
}'use client'
import { useEffect, useRef } from 'react'
interface ContextMenuProps {
x: number
y: number
onClose: () => void
onNewSubfolder: () => void
onRename: () => void
onMoveTo: () => void
onDelete: () => void
}
export default function FolderContextMenu({
x,
y,
onClose,
onNewSubfolder,
onRename,
onMoveTo,
onDelete,
}: ContextMenuProps) {
const menuRef = useRef<HTMLDivElement>(null)
useEffect(() => {
function handleClickOutside(e: MouseEvent) {
if (menuRef.current && !menuRef.current.contains(e.target as Node)) {
onClose()
}
}
document.addEventListener('mousedown', handleClickOutside)
return () => document.removeEventListener('mousedown', handleClickOutside)
}, [onClose])
return (
<div
ref={menuRef}
className="folder-context-menu"
style={{ top: y, left: x }}
>
<button onClick={onNewSubfolder}>New Subfolder</button>
<button onClick={onRename}>Rename</button>
<button onClick={onMoveTo}>Move to...</button>
<div className="context-menu-divider" />
<button className="danger" onClick={onDelete}>Delete</button>
</div>
)
}'use client'
interface BreadcrumbItem {
id: string
name: string
}
interface BreadcrumbProps {
path: BreadcrumbItem[]
onNavigate: (folderId: string) => void
}
export default function Breadcrumb({ path, onNavigate }: BreadcrumbProps) {
return (
<nav className="breadcrumb-nav" aria-label="Folder navigation">
<button
className="breadcrumb-item"
onClick={() => onNavigate('root')}
>
Home
</button>
{path.map((item, index) => (
<span key={item.id} className="breadcrumb-segment">
<span className="breadcrumb-separator">/</span>
{index === path.length - 1 ? (
<span className="breadcrumb-current">{item.name}</span>
) : (
<button
className="breadcrumb-item"
onClick={() => onNavigate(item.id)}
>
{item.name}
</button>
)}
</span>
))}
</nav>
)
}'use client'
import { useState, useEffect } from 'react'
interface Document {
id: string
title: string
content_type: string
created_at: string
}
interface FolderContentsProps {
folderId: string
onDocumentSelect: (doc: Document) => void
onMoveDocument: (docId: string) => void
}
function getFileIcon(contentType: string): string {
if (contentType.includes('pdf')) return '📄'
if (contentType.includes('image')) return '🖼️'
if (contentType.includes('audio')) return '🎵'
if (contentType.includes('text')) return '📝'
return '📎'
}
export default function FolderContents({
folderId,
onDocumentSelect,
onMoveDocument,
}: FolderContentsProps) {
const [documents, setDocuments] = useState<Document[]>([])
const [viewMode, setViewMode] = useState<'grid' | 'list'>('grid')
const [docContextMenu, setDocContextMenu] = useState<{
docId: string
x: number
y: number
} | null>(null)
useEffect(() => {
async function fetchDocuments() {
const res = await fetch(`/api/folders/${folderId}/documents`)
const data = await res.json()
setDocuments(data)
}
if (folderId) fetchDocuments()
}, [folderId])
function handleDocContextMenu(e: React.MouseEvent, docId: string) {
e.preventDefault()
setDocContextMenu({ docId, x: e.clientX, y: e.clientY })
}
return (
<div className="folder-contents">
<div className="contents-toolbar">
<span className="doc-count">{documents.length} documents</span>
<div className="view-toggle">
<button
className={viewMode === 'grid' ? 'active' : ''}
onClick={() => setViewMode('grid')}
>
Grid
</button>
<button
className={viewMode === 'list' ? 'active' : ''}
onClick={() => setViewMode('list')}
>
List
</button>
</div>
</div>
<div className={`contents-${viewMode}`}>
{documents.map((doc) => (
<div
key={doc.id}
className={`doc-item doc-item-${viewMode}`}
onClick={() => onDocumentSelect(doc)}
onContextMenu={(e) => handleDocContextMenu(e, doc.id)}
>
<span className="doc-icon">{getFileIcon(doc.content_type)}</span>
<span className="doc-title">{doc.title}</span>
</div>
))}
</div>
{docContextMenu && (
<div
className="folder-context-menu"
style={{ top: docContextMenu.y, left: docContextMenu.x }}
>
<button onClick={() => {
onMoveDocument(docContextMenu.docId)
setDocContextMenu(null)
}}>
Move Document
</button>
</div>
)}
</div>
)
}'use client'
import { FolderTreeNode } from './FolderNode'
interface FolderSelectorProps {
tree: FolderTreeNode[]
selectedFolderId: string
onChange: (folderId: string) => void
}
function flattenTree(nodes: FolderTreeNode[], depth: number = 0): { id: string; label: string }[] {
const result: { id: string; label: string }[] = []
for (const node of nodes) {
const indent = '—'.repeat(depth)
result.push({ id: node.id, label: `${indent} ${node.name}` })
if (node.children.length > 0) {
result.push(...flattenTree(node.children, depth + 1))
}
}
return result
}
export default function FolderSelector({
tree,
selectedFolderId,
onChange,
}: FolderSelectorProps) {
const options = flattenTree(tree)
return (
<div className="folder-selector">
<label htmlFor="folder-select">Upload to folder:</label>
<select
id="folder-select"
value={selectedFolderId}
onChange={(e) => onChange(e.target.value)}
className="folder-select-input"
>
{options.map((opt) => (
<option key={opt.id} value={opt.id}>
{opt.label}
</option>
))}
</select>
</div>
)
}Your folder frontend is fully wired up. Students can browse, organize, rename, and drag folders around with visual feedback at every step. Next up, you will build the contextual chat system that lets students ask questions scoped to a single document, an entire folder, or their full knowledge base.
Implement Advanced Contextual Chat — Backend
Your folder system is in place. Documents are organized, and the frontend tree is working. Now you need the intelligence layer that makes folders meaningful.
Right now, E-Hanatra's chat treats every document equally. It searches the entire ChromaDB collection with no awareness of which folder a chunk belongs to. In this step, you will build three scoped RAG query modes so the AI can answer questions about a single document, an entire folder, or the full knowledge base.
In this step, get ready to:
- Enrich ChromaDB metadata with document titles and backfill existing records.
- Create three scoped chat endpoints (document, folder, global).
- Add source attribution and persist conversation history to SQLite.
Enrich ChromaDB metadata and backfill existing records
For scoped queries to work, every chunk in ChromaDB needs to carry enough metadata to identify which document and folder it belongs to. Your ingestion pipeline already stores document_id and folder_id on new chunks. The missing piece is document_title so the chat can cite sources by name.
- In your FastAPI project, open the file that handles document ingestion (where chunks are added to ChromaDB).
- Update the metadata dictionary for each chunk to include document_title by copying the following code into your ingestion function:
# Enrich each chunk's metadata with document title for source attribution
metadata = {
"document_id": doc_id,
"folder_id": folder_id,
"folder_path": folder_path,
"document_title": document_title # human-readable name for citations
}What does this code do?
Each chunk stored in ChromaDB now carries four metadata fields. The document_title field is new. It stores the human-readable filename so the chat can reference sources without an extra database lookup.
Existing chunks uploaded before this change are missing the document_title field. You need a one-time migration script that cross-references SQLite to backfill them.
- Create a new file called backfill_metadata.py in your FastAPI project root by adding the following code:
import asyncio
import aiosqlite
import chromadb
# One-time migration: backfill document_title on existing ChromaDB chunks
DB_PATH = "e-hanatra.db"
CHROMA_PATH = "./chroma_data"
async def backfill():
# Connect to SQLite to get document titles
async with aiosqlite.connect(DB_PATH) as db:
cursor = await db.execute("SELECT id, title FROM documents")
documents = await cursor.fetchall()
# Connect to ChromaDB
client = chromadb.PersistentClient(path=CHROMA_PATH)
collection = client.get_or_create_collection("documents")
for doc_id, doc_title in documents:
# Retrieve all chunk IDs belonging to this document
results = collection.get(
where={"document_id": doc_id},
include=[]
)
if not results["ids"]:
continue
# Update metadata on each chunk with the document title
collection.update(
ids=results["ids"],
metadatas=[{"document_id": doc_id, "document_title": doc_title} for _ in results["ids"]]
)
print(f"Backfilled {len(results['ids'])} chunks for '{doc_title}'")
print("Migration complete.")
if __name__ == "__main__":
asyncio.run(backfill())What does this script do?
The script iterates through every document in SQLite, fetches the matching chunks from ChromaDB using a where filter on document_id, and calls collection.update() to patch each chunk's metadata with document_title.
ChromaDB's update method accepts ids and metadatas arrays of the same length. Each index corresponds to one record.
- Run the migration script by executing this command:
python backfill_metadata.py- You should see output listing how many chunks were backfilled for each document title, ending with Migration complete. in the terminal.
Seeing zero chunks backfilled?
Make sure the document_id values stored in ChromaDB match the id column in your SQLite documents table. If your V1 ingestion used a different ID format, update the where filter to match.
Also confirm your ChromaDB collection name matches what your app uses. Check your existing code for the collection name passed to get_or_create_collection().
Help me debug why my backfill script finds zero chunks for each document.
Create three scoped RAG query endpoints
With metadata in place, you can now filter ChromaDB queries by scope. Each endpoint retrieves relevant chunks from a specific context, builds a prompt with those chunks, and sends it to Groq along with the conversation history.
- Create a new file called chat_router.py in your FastAPI project for the scoped chat endpoints. Add the following imports and helper function:
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
import aiosqlite
import chromadb
import json
from groq import Groq
router = APIRouter()
# Configuration
DB_PATH = "e-hanatra.db"
CHROMA_PATH = "./chroma_data"
groq_client = Groq()
# Request and response models
class ChatRequest(BaseModel):
message: str
conversation_id: str | None = None
class Source(BaseModel):
document_id: str
document_title: str
chunk_preview: str
class ChatResponse(BaseModel):
answer: str
sources: list[Source]
conversation_id: strWhat does this code do?
This sets up the router and defines Pydantic models for the request and response. ChatRequest accepts a message and an optional conversation ID for continuing existing conversations. ChatResponse returns the AI answer, a list of source citations, and the conversation ID.
- Below the models in chat_router.py, add the shared RAG helper function that all three endpoints will use:
async def query_and_respond(
message: str,
conversation_id: str | None,
scope_type: str,
scope_id: str | None,
where_filter: dict | None
) -> ChatResponse:
"""Shared RAG logic: retrieve chunks, build context, call Groq."""
# Connect to ChromaDB and query with optional metadata filter
client = chromadb.PersistentClient(path=CHROMA_PATH)
collection = client.get_or_create_collection("documents")
query_params = {
"query_texts": [message],
"n_results": 5,
"include": ["documents", "metadatas"]
}
if where_filter:
query_params["where"] = where_filter
results = collection.query(**query_params)
# Build source list from retrieved chunks
sources = []
context_parts = []
for i, doc in enumerate(results["documents"][0]):
meta = results["metadatas"][0][i]
sources.append(Source(
document_id=meta.get("document_id", ""),
document_title=meta.get("document_title", "Unknown"),
chunk_preview=doc[:150]
))
context_parts.append(f"[Source: {meta.get('document_title', 'Unknown')}]\n{doc}")
context_text = "\n\n".join(context_parts)
# Load conversation history from SQLite
conv_id = conversation_id or f"{scope_type}_{scope_id or 'all'}_{message[:10]}"
history = []
async with aiosqlite.connect(DB_PATH) as db:
cursor = await db.execute(
"SELECT messages FROM conversations WHERE id = ?",
(conv_id,)
)
row = await cursor.fetchone()
if row:
history = json.loads(row[0])
# Build messages for Groq with context and history
system_prompt = (
"You are E-Hanatra, an intelligent study assistant. "
"Answer questions using ONLY the provided context. "
"Cite sources by document title when relevant.\n\n"
f"Context:\n{context_text}"
)
messages = [{"role": "system", "content": system_prompt}]
# Include last 8 messages for token efficiency
messages.extend(history[-8:])
messages.append({"role": "user", "content": message})
# Call Groq LLM
completion = groq_client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=messages
)
answer = completion.choices[0].message.content
# Save updated conversation history
history.append({"role": "user", "content": message})
history.append({"role": "assistant", "content": answer})
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"""INSERT INTO conversations (id, scope_type, scope_id, messages)
VALUES (?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET messages = ?""",
(conv_id, scope_type, scope_id or "", json.dumps(history), json.dumps(history))
)
await db.commit()
return ChatResponse(answer=answer, sources=sources, conversation_id=conv_id)What does this helper do?
The query_and_respond function is the shared engine behind all three chat endpoints. It queries ChromaDB with an optional where filter, collects source metadata, loads conversation history from SQLite, sends everything to Groq, and persists the updated conversation.
The history is limited to the last 8 messages to stay within Groq's token budget on the free tier.
- Now add the three endpoint definitions below the helper function in chat_router.py:
@router.post("/chat/document/{doc_id}", response_model=ChatResponse)
async def chat_document(doc_id: str, request: ChatRequest):
"""Chat scoped to a single document."""
return await query_and_respond(
message=request.message,
conversation_id=request.conversation_id,
scope_type="document",
scope_id=doc_id,
where_filter={"document_id": doc_id}
)
@router.post("/chat/folder/{folder_id}", response_model=ChatResponse)
async def chat_folder(folder_id: str, request: ChatRequest):
"""Chat scoped to all documents in a folder."""
return await query_and_respond(
message=request.message,
conversation_id=request.conversation_id,
scope_type="folder",
scope_id=folder_id,
where_filter={"folder_id": folder_id}
)
@router.post("/chat/global", response_model=ChatResponse)
async def chat_global(request: ChatRequest):
"""Chat across all documents with no metadata filter."""
return await query_and_respond(
message=request.message,
conversation_id=request.conversation_id,
scope_type="global",
scope_id=None,
where_filter=None
)How do the three scopes differ?
- POST /chat/document/{doc_id} passes where={"document_id": doc_id} so ChromaDB only returns chunks from that specific document.
- POST /chat/folder/{folder_id} passes where={"folder_id": folder_id} to search all documents within a folder.
- POST /chat/global passes None as the filter, searching the entire collection.
- Save chat_router.py.
- Register the router in your main FastAPI app file by adding app.include_router(router) with the chat router import.
Getting a 422 validation error when testing?
Make sure your request body includes the message field as a JSON string. A common mistake is sending the message as a query parameter instead of in the request body.
Check that you imported and registered the router correctly in your main app file.
Help me fix a 422 error on my scoped chat endpoints.
Add the conversations table and verify the full setup
The chat endpoints depend on a conversations table in SQLite for storing message history. You need to create this table before the endpoints can persist data.
- In your database initialization code (the file where you set up SQLite tables), add the conversations table creation by including this SQL:
# Create conversations table for chat history persistence
await db.execute("""
CREATE TABLE IF NOT EXISTS conversations (
id TEXT PRIMARY KEY,
scope_type TEXT NOT NULL,
scope_id TEXT NOT NULL,
messages TEXT NOT NULL DEFAULT '[]',
created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
)
""")
await db.commit()What does this table store?
Each conversation row has a unique id, a scope_type (document, folder, or global), the scope_id identifying what the conversation is scoped to, and messages stored as a JSON array of role/content pairs.
Storing messages as JSON keeps the schema simple while allowing unlimited conversation length. The ON CONFLICT clause in the chat helper handles upserts so existing conversations get appended to.
- Save your database initialization file.
- Start (or restart) your FastAPI server to apply the table creation.
- Test the document-scoped endpoint by sending a POST request to /chat/document/{doc_id} with a JSON body containing a message field. You can use the FastAPI interactive docs at /docs.
You should see a JSON response containing an answer string, a sources array with document titles and chunk previews, and a conversation_id.
Getting a table not found error?
Make sure your database initialization runs the CREATE TABLE IF NOT EXISTS conversations statement before the chat endpoints handle requests. If you are using a lifespan or startup event, confirm it executes before the router is active.
You can verify the table exists by running sqlite3 e-hanatra.db ".tables" in your terminal. You should see conversations listed alongside folders and documents.
Help me debug why my conversations table isn't being created.
✔️ Awesome, I've got everything!
Great. Double check you have saved all files and your server is running without errors.
ⓧ I'd like to double check the full code
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel
import aiosqlite
import chromadb
import json
from groq import Groq
router = APIRouter()
# Configuration
DB_PATH = "e-hanatra.db"
CHROMA_PATH = "./chroma_data"
groq_client = Groq()
# Request and response models
class ChatRequest(BaseModel):
message: str
conversation_id: str | None = None
class Source(BaseModel):
document_id: str
document_title: str
chunk_preview: str
class ChatResponse(BaseModel):
answer: str
sources: list[Source]
conversation_id: str
async def query_and_respond(
message: str,
conversation_id: str | None,
scope_type: str,
scope_id: str | None,
where_filter: dict | None
) -> ChatResponse:
"""Shared RAG logic: retrieve chunks, build context, call Groq."""
# Connect to ChromaDB and query with optional metadata filter
client = chromadb.PersistentClient(path=CHROMA_PATH)
collection = client.get_or_create_collection("documents")
query_params = {
"query_texts": [message],
"n_results": 5,
"include": ["documents", "metadatas"]
}
if where_filter:
query_params["where"] = where_filter
results = collection.query(**query_params)
# Build source list from retrieved chunks
sources = []
context_parts = []
for i, doc in enumerate(results["documents"][0]):
meta = results["metadatas"][0][i]
sources.append(Source(
document_id=meta.get("document_id", ""),
document_title=meta.get("document_title", "Unknown"),
chunk_preview=doc[:150]
))
context_parts.append(f"[Source: {meta.get('document_title', 'Unknown')}]\n{doc}")
context_text = "\n\n".join(context_parts)
# Load conversation history from SQLite
conv_id = conversation_id or f"{scope_type}_{scope_id or 'all'}_{message[:10]}"
history = []
async with aiosqlite.connect(DB_PATH) as db:
cursor = await db.execute(
"SELECT messages FROM conversations WHERE id = ?",
(conv_id,)
)
row = await cursor.fetchone()
if row:
history = json.loads(row[0])
# Build messages for Groq with context and history
system_prompt = (
"You are E-Hanatra, an intelligent study assistant. "
"Answer questions using ONLY the provided context. "
"Cite sources by document title when relevant.\n\n"
f"Context:\n{context_text}"
)
messages = [{"role": "system", "content": system_prompt}]
# Include last 8 messages for token efficiency
messages.extend(history[-8:])
messages.append({"role": "user", "content": message})
# Call Groq LLM
completion = groq_client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=messages
)
answer = completion.choices[0].message.content
# Save updated conversation history
history.append({"role": "user", "content": message})
history.append({"role": "assistant", "content": answer})
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"""INSERT INTO conversations (id, scope_type, scope_id, messages)
VALUES (?, ?, ?, ?)
ON CONFLICT(id) DO UPDATE SET messages = ?""",
(conv_id, scope_type, scope_id or "", json.dumps(history), json.dumps(history))
)
await db.commit()
return ChatResponse(answer=answer, sources=sources, conversation_id=conv_id)
@router.post("/chat/document/{doc_id}", response_model=ChatResponse)
async def chat_document(doc_id: str, request: ChatRequest):
"""Chat scoped to a single document."""
return await query_and_respond(
message=request.message,
conversation_id=request.conversation_id,
scope_type="document",
scope_id=doc_id,
where_filter={"document_id": doc_id}
)
@router.post("/chat/folder/{folder_id}", response_model=ChatResponse)
async def chat_folder(folder_id: str, request: ChatRequest):
"""Chat scoped to all documents in a folder."""
return await query_and_respond(
message=request.message,
conversation_id=request.conversation_id,
scope_type="folder",
scope_id=folder_id,
where_filter={"folder_id": folder_id}
)
@router.post("/chat/global", response_model=ChatResponse)
async def chat_global(request: ChatRequest):
"""Chat across all documents with no metadata filter."""
return await query_and_respond(
message=request.message,
conversation_id=request.conversation_id,
scope_type="global",
scope_id=None,
where_filter=None
)Your contextual chat backend is live. The AI now gives focused answers based on whether the learner is asking about a single document, a folder of related materials, or their entire knowledge base. Next up, you will build the frontend components that let users switch between these scopes and see source citations inline.
Implement Advanced Contextual Chat — Frontend
Your backend now serves three scoped RAG chat endpoints with source attribution. The missing piece is a frontend that lets students switch between those scopes and actually see where answers come from.
In this step, you will redesign the chat panel with a scope selector, add clickable source citations below every AI response, and implement conversation persistence so students can pick up where they left off.
In this step, get ready to:
- Build a scope selector component with colored badges for document, folder, and global modes.
- Add collapsible source citations that link back to the originating documents.
- Implement conversation persistence with a sidebar drawer, search, and an 8-message context window limit.
Build the scope selector component
The scope selector determines which backend endpoint the chat calls. Each scope gets a distinct colored badge so the student always knows what slice of their knowledge base is active.
- Create a new file components/chat/ScopeSelector.tsx inside your components/chat directory.
- Add the following scope selector component:
'use client'
import { useState } from 'react'
// Define the three chat scope types
export type ChatScope = 'document' | 'folder' | 'global'
interface ScopeSelectorProps {
activeScope: ChatScope
onScopeChange: (scope: ChatScope) => void
documentName?: string
folderName?: string
}
// Color mapping for each scope badge
const scopeConfig = {
document: { label: 'This Document', color: '#3b82f6', bgColor: '#1e3a5f' },
folder: { label: 'This Folder', color: '#22c55e', bgColor: '#14532d' },
global: { label: 'All Documents', color: '#a855f7', bgColor: '#3b0764' },
}
export default function ScopeSelector({
activeScope,
onScopeChange,
documentName,
folderName,
}: ScopeSelectorProps) {
return (
<div className="scope-selector">
{/* Render a tab button for each scope */}
{(Object.keys(scopeConfig) as ChatScope[]).map((scope) => (
<button
key={scope}
className={`scope-tab ${activeScope === scope ? 'active' : ''}`}
onClick={() => onScopeChange(scope)}
style={{
borderColor: activeScope === scope ? scopeConfig[scope].color : 'transparent',
}}
>
{/* Colored badge dot */}
<span
className="scope-badge"
style={{ backgroundColor: scopeConfig[scope].color }}
/>
{/* Display scope label with context name */}
<span className="scope-label">
{scope === 'document' && documentName
? `${scopeConfig[scope].label}: ${documentName}`
: scope === 'folder' && folderName
? `${scopeConfig[scope].label}: ${folderName}`
: scopeConfig[scope].label}
</span>
</button>
))}
</div>
)
}What does this code do?
The ScopeSelector component renders three tab buttons, one per scope. Each tab has a colored dot badge (blue, green, purple) and shows the active document or folder name for context.
The scopeConfig object centralizes colors and labels so you can adjust the palette in one place. When a tab is clicked, onScopeChange fires and the parent chat panel updates which API endpoint to call.
- Save components/chat/ScopeSelector.tsx.
Now add styles for the scope selector. The design follows V1's dark theme with subtle borders and accent colors for the selected state.
- In your chat styles file, add the following CSS for the scope selector:
.scope-selector {
display: flex;
gap: 0.25rem;
padding: 0.5rem;
background: #1a1a2e;
border-bottom: 1px solid #2d2d44;
}
.scope-tab {
display: flex;
align-items: center;
gap: 0.5rem;
padding: 0.5rem 0.75rem;
border: none;
border-bottom: 2px solid transparent;
background: transparent;
color: #a0a0b0;
cursor: pointer;
font-size: 0.85rem;
border-radius: 4px 4px 0 0;
transition: all 0.2s ease;
}
.scope-tab:hover {
background: #2d2d44;
}
.scope-tab.active {
color: #ffffff;
background: #2d2d44;
}
.scope-badge {
width: 8px;
height: 8px;
border-radius: 50%;
}
.scope-label {
white-space: nowrap;
overflow: hidden;
text-overflow: ellipsis;
max-width: 160px;
}What does this code do?
The .scope-tab.active class gets a colored bottom border from the inline style in the component. The .scope-badge is a tiny colored dot that gives instant visual feedback about which scope is selected.
The text-overflow: ellipsis on .scope-label prevents long document or folder names from breaking the layout.
- Save your styles file.
- Refresh your browser. You should see three scope tabs at the top of the chat panel with colored dot badges. Clicking each tab should highlight it with the matching accent color.
Tabs not appearing?
Make sure you imported ScopeSelector into your main chat panel component and rendered it inside the chat header area.
Check that the CSS file is imported in your chat component or global styles. If the tabs render but without styling, verify the class names match exactly.
Why aren't my scope selector tabs showing up in the chat panel?
Add source citations to chat responses
When the AI answers a question, students need to know where the information came from. Source citations build trust and let students jump directly to the relevant material in their documents.
- Create components/chat/SourceCitations.tsx with the following code:
'use client'
import { useState } from 'react'
// Source shape returned by the backend
export interface ChatSource {
document_id: string
document_title: string
chunk_preview: string
}
interface SourceCitationsProps {
sources: ChatSource[]
onSourceClick: (documentId: string) => void
}
export default function SourceCitations({
sources,
onSourceClick,
}: SourceCitationsProps) {
// Collapsed by default to keep the chat clean
const [expanded, setExpanded] = useState(false)
if (!sources || sources.length === 0) return null
return (
<div className="source-citations">
{/* Toggle button to expand/collapse sources */}
<button
className="sources-toggle"
onClick={() => setExpanded(!expanded)}
>
<span className="sources-icon">📄</span>
<span>{sources.length} Source{sources.length > 1 ? 's' : ''}</span>
<span className={`chevron ${expanded ? 'expanded' : ''}`}>▾</span>
</button>
{/* Expanded list of clickable source documents */}
{expanded && (
<ul className="sources-list">
{sources.map((source, idx) => (
<li
key={`${source.document_id}-${idx}`}
className="source-item"
onClick={() => onSourceClick(source.document_id)}
>
<span className="source-doc-icon">📑</span>
<div className="source-details">
<span className="source-title">{source.document_title}</span>
<span className="source-preview">{source.chunk_preview}</span>
</div>
</li>
))}
</ul>
)}
</div>
)
}What does this code do?
The SourceCitations component renders a collapsible list below each AI response. It shows the number of sources, and when expanded, displays each document title with a short chunk preview.
The onSourceClick callback passes the document_id up to the parent, which opens the document preview panel scrolled to the relevant section.
- Add styles for the source citations in your chat styles file:
.source-citations {
margin-top: 0.5rem;
border-top: 1px solid #2d2d44;
padding-top: 0.5rem;
}
.sources-toggle {
display: flex;
align-items: center;
gap: 0.5rem;
background: none;
border: none;
color: #8b8ba0;
cursor: pointer;
font-size: 0.8rem;
padding: 0.25rem 0;
}
.sources-toggle:hover {
color: #c0c0d0;
}
.chevron {
transition: transform 0.2s ease;
}
.chevron.expanded {
transform: rotate(180deg);
}
.sources-list {
list-style: none;
padding: 0;
margin: 0.5rem 0 0 0;
display: flex;
flex-direction: column;
gap: 0.25rem;
}
.source-item {
display: flex;
align-items: flex-start;
gap: 0.5rem;
padding: 0.5rem;
border-radius: 6px;
background: #1a1a2e;
cursor: pointer;
transition: background 0.15s ease;
}
.source-item:hover {
background: #2d2d44;
}
.source-details {
display: flex;
flex-direction: column;
gap: 0.15rem;
}
.source-title {
font-size: 0.8rem;
color: #e0e0f0;
font-weight: 500;
}
.source-preview {
font-size: 0.75rem;
color: #6b6b80;
display: -webkit-box;
-webkit-line-clamp: 2;
-webkit-box-orient: vertical;
overflow: hidden;
}What does this code do?
The .source-item rows use a hover effect to signal that they are clickable. The -webkit-line-clamp: 2 on .source-preview truncates long chunk previews to two lines so the source list stays compact.
- Save components/chat/SourceCitations.tsx and your styles file.
- Send a message in the chat. Below the AI response, you should see a "Sources" toggle. Click it to expand and see document titles with chunk previews.
Sources not appearing below responses?
Verify that you are passing the sources array from the API response to the SourceCitations component. The backend returns {answer, sources: [{document_id, document_title, chunk_preview}]} for each chat response.
Check your browser console for fetch errors. If the API response is missing the sources field, make sure you deployed the updated backend from Step 3.
Help me debug why source citations aren't showing below my chat responses.
Implement conversation persistence and history drawer
Students lose their train of thought when chat history disappears between sessions. Conversation persistence loads previous chats scoped to the current context, and the sidebar drawer lets students search and manage their history.
- Create components/chat/ConversationDrawer.tsx with the following code:
'use client'
import { useState } from 'react'
export interface Conversation {
id: string
scope_type: string
scope_id: string
messages: { role: string; content: string }[]
created_at: string
}
interface ConversationDrawerProps {
conversations: Conversation[]
activeConversationId: string | null
onSelect: (conversation: Conversation) => void
onNewConversation: () => void
onDelete: (conversationId: string) => void
isOpen: boolean
onClose: () => void
}
export default function ConversationDrawer({
conversations,
activeConversationId,
onSelect,
onNewConversation,
onDelete,
isOpen,
onClose,
}: ConversationDrawerProps) {
// Search filter for conversation history
const [searchQuery, setSearchQuery] = useState('')
// Filter conversations by search query matching message content
const filtered = conversations.filter((conv) => {
if (!searchQuery) return true
const query = searchQuery.toLowerCase()
return conv.messages.some((msg) =>
msg.content.toLowerCase().includes(query)
)
})
if (!isOpen) return null
return (
<div className="conversation-drawer">
{/* Drawer header with new conversation button */}
<div className="drawer-header">
<h3>Conversations</h3>
<button className="drawer-close" onClick={onClose}>✕</button>
</div>
{/* Search input for filtering history */}
<div className="drawer-search">
<input
type="text"
placeholder="Search conversations..."
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
/>
</div>
{/* New conversation button */}
<button className="new-conversation-btn" onClick={onNewConversation}>
+ New Conversation
</button>
{/* Scrollable conversation list */}
<ul className="conversation-list">
{filtered.map((conv) => (
<li
key={conv.id}
className={`conversation-item ${
activeConversationId === conv.id ? 'active' : ''
}`}
>
<div
className="conversation-content"
onClick={() => onSelect(conv)}
>
{/* Preview: first message content truncated */}
<span className="conversation-preview">
{conv.messages[0]?.content.slice(0, 60) || 'Empty conversation'}...
</span>
<span className="conversation-date">
{new Date(conv.created_at).toLocaleDateString()}
</span>
</div>
{/* Delete button */}
<button
className="conversation-delete"
onClick={(e) => {
e.stopPropagation()
onDelete(conv.id)
}}
>
🗑
</button>
</li>
))}
</ul>
</div>
)
}What does this code do?
The ConversationDrawer renders a slide-over sidebar listing all saved conversations for the current scope. Students can search across message content, start a new conversation, or delete old ones.
Each conversation preview shows the first message truncated to 60 characters plus the creation date. The activeConversationId highlights which conversation is currently loaded in the chat.
- Save components/chat/ConversationDrawer.tsx.
Now wire everything together in the main chat panel. The key integration points are: the scope selector drives the API endpoint, source citations render below each response, and the conversation drawer loads history scoped to the active context.
- Update your main chat panel component (components/chat/ChatPanel.tsx or equivalent) with the following integration logic:
'use client'
import { useState, useEffect, useCallback } from 'react'
import ScopeSelector, { ChatScope } from './ScopeSelector'
import SourceCitations, { ChatSource } from './SourceCitations'
import ConversationDrawer, { Conversation } from './ConversationDrawer'
// Maximum messages sent as context to Groq for token efficiency
const MAX_CONTEXT_MESSAGES = 8
interface ChatMessage {
role: 'user' | 'assistant'
content: string
sources?: ChatSource[]
}
export default function ChatPanel({
currentDocumentId,
currentDocumentName,
currentFolderId,
currentFolderName,
}: {
currentDocumentId?: string
currentDocumentName?: string
currentFolderId?: string
currentFolderName?: string
}) {
const [activeScope, setActiveScope] = useState<ChatScope>('document')
const [messages, setMessages] = useState<ChatMessage[]>([])
const [input, setInput] = useState('')
const [isLoading, setIsLoading] = useState(false)
const [drawerOpen, setDrawerOpen] = useState(false)
const [conversations, setConversations] = useState<Conversation[]>([])
const [activeConversationId, setActiveConversationId] = useState<string | null>(null)
// Build the API URL based on active scope
const getEndpoint = useCallback(() => {
switch (activeScope) {
case 'document':
return `/chat/document/${currentDocumentId}`
case 'folder':
return `/chat/folder/${currentFolderId}`
case 'global':
return '/chat/global'
}
}, [activeScope, currentDocumentId, currentFolderId])
// Load conversations for the current scope on mount or scope change
useEffect(() => {
async function loadConversations() {
const scopeId = activeScope === 'document'
? currentDocumentId
: activeScope === 'folder'
? currentFolderId
: 'global'
const res = await fetch(
`/api/conversations?scope_type=${activeScope}&scope_id=${scopeId}`
)
if (res.ok) {
const data = await res.json()
setConversations(data)
}
}
loadConversations()
}, [activeScope, currentDocumentId, currentFolderId])
// Send message with context window limited to last 8 messages
const sendMessage = async () => {
if (!input.trim() || isLoading) return
const userMessage: ChatMessage = { role: 'user', content: input }
const updatedMessages = [...messages, userMessage]
setMessages(updatedMessages)
setInput('')
setIsLoading(true)
// Only send the last MAX_CONTEXT_MESSAGES for token efficiency
const contextMessages = updatedMessages
.slice(-MAX_CONTEXT_MESSAGES)
.map(({ role, content }) => ({ role, content }))
try {
const res = await fetch(getEndpoint(), {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ messages: contextMessages }),
})
const data = await res.json()
// Append assistant response with source attributions
const assistantMessage: ChatMessage = {
role: 'assistant',
content: data.answer,
sources: data.sources,
}
setMessages((prev) => [...prev, assistantMessage])
} catch (error) {
console.error('Chat error:', error)
} finally {
setIsLoading(false)
}
}
// Handle clicking a source to open document preview
const handleSourceClick = (documentId: string) => {
// Navigate to document preview scrolled to relevant section
window.dispatchEvent(
new CustomEvent('open-document-preview', { detail: { documentId } })
)
}
// Conversation management handlers
const handleNewConversation = () => {
setMessages([])
setActiveConversationId(null)
}
const handleSelectConversation = (conv: Conversation) => {
setMessages(conv.messages as ChatMessage[])
setActiveConversationId(conv.id)
setDrawerOpen(false)
}
const handleDeleteConversation = async (convId: string) => {
await fetch(`/api/conversations/${convId}`, { method: 'DELETE' })
setConversations((prev) => prev.filter((c) => c.id !== convId))
if (activeConversationId === convId) handleNewConversation()
}
return (
<div className="chat-panel">
{/* Scope selector at the top */}
<ScopeSelector
activeScope={activeScope}
onScopeChange={setActiveScope}
documentName={currentDocumentName}
folderName={currentFolderName}
/>
{/* History toggle button */}
<button
className="history-toggle"
onClick={() => setDrawerOpen(!drawerOpen)}
>
History
</button>
{/* Conversation drawer */}
<ConversationDrawer
conversations={conversations}
activeConversationId={activeConversationId}
onSelect={handleSelectConversation}
onNewConversation={handleNewConversation}
onDelete={handleDeleteConversation}
isOpen={drawerOpen}
onClose={() => setDrawerOpen(false)}
/>
{/* Message list with source citations */}
<div className="chat-messages">
{messages.map((msg, idx) => (
<div key={idx} className={`chat-message ${msg.role}`}>
<p>{msg.content}</p>
{msg.role === 'assistant' && msg.sources && (
<SourceCitations
sources={msg.sources}
onSourceClick={handleSourceClick}
/>
)}
</div>
))}
{isLoading && <div className="chat-loading">Thinking...</div>}
</div>
{/* Input area */}
<div className="chat-input-area">
<input
type="text"
value={input}
onChange={(e) => setInput(e.target.value)}
onKeyDown={(e) => e.key === 'Enter' && sendMessage()}
placeholder="Ask a question..."
/>
<button onClick={sendMessage} disabled={isLoading}>
Send
</button>
</div>
</div>
)
}What does this code do?
The ChatPanel ties all three sub-components together. The getEndpoint function builds the correct API path based on activeScope so the backend knows which ChromaDB filter to apply.
The MAX_CONTEXT_MESSAGES constant limits the conversation context sent to Groq to 8 messages. This keeps token usage efficient while still providing enough context for coherent follow-up answers.
The handleSourceClick function dispatches a custom event that the document preview panel listens for. This decouples the chat from the preview, letting each component manage its own state.
- Save components/chat/ChatPanel.tsx.
- Refresh your browser and open the chat panel. Switch between scope tabs and send a message. You should see the AI response appear with a collapsible "Sources" section below it.
- Click the "History" button to open the conversation drawer. You should see any previous conversations for the selected scope listed with previews and dates.
Chat messages not sending?
Check that your getEndpoint function returns a valid URL. If currentDocumentId or currentFolderId is undefined, the API call will fail with a 404. Make sure the parent component passes these props correctly.
If the drawer shows no conversations, verify your backend conversations table in e-hanatra.db has records for the current scope. The scope_type and scope_id must match what the frontend sends.
Help me debug why my chat panel integration isn't working with the scoped endpoints.
Why limit to 8 messages?
Groq's free tier has token limits per request. Sending the full conversation history of 50+ messages would quickly exceed those limits and slow down response times. Eight messages provides enough context for coherent follow-up questions while keeping each request fast and affordable.
The backend already persists the full history to SQLite. The 8-message window is only what gets sent to the LLM for generating the next response.
Your chat panel now intelligently routes queries to the right scope and shows students exactly where answers come from. Next up, you will build interactive Knowledge Maps that visualize how concepts connect across documents.
Build the Knowledge Map — Concept Extraction Backend
Your contextual chat can already retrieve relevant chunks from specific documents or folders. The next challenge is turning that raw knowledge into something visual and structured.
In this step, you will build the backend that extracts key concepts from your documents using Groq, stores them as a connected graph in SQLite, and serves that graph through API endpoints. The frontend will consume this data in the next step to render an interactive Knowledge Map.
In this step, get ready to:
- Build a concept extraction endpoint that sends document text to Groq and parses structured concept data.
- Create SQLite tables for concepts and concept edges with deduplication logic.
- Build aggregation and regeneration endpoints that serve the full concept graph.
Create the knowledge map database tables
Before you can store extracted concepts, you need database tables to hold them. The knowledge map uses two tables: one for individual concepts and one for the edges (relationships) between them.
- Create a new file called knowledge_map.py in your FastAPI backend folder. This file will hold all the Knowledge Map routes and logic.
- Add the database migration function and router setup by pasting the following code:
import json
import aiosqlite
from fastapi import APIRouter, HTTPException
# Database path (same as the rest of the app)
DB_PATH = "e-hanatra.db"
router = APIRouter()
async def create_knowledge_map_tables():
"""Create the concepts and concept_edges tables if they don't exist."""
async with aiosqlite.connect(DB_PATH) as db:
# Store individual concepts extracted from documents
await db.execute("""
CREATE TABLE IF NOT EXISTS concepts (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
definition TEXT,
category TEXT,
document_id TEXT,
folder_id TEXT
)
""")
# Store relationships between concepts
await db.execute("""
CREATE TABLE IF NOT EXISTS concept_edges (
id INTEGER PRIMARY KEY AUTOINCREMENT,
source_concept_id INTEGER NOT NULL,
target_concept_id INTEGER NOT NULL,
relationship_label TEXT,
FOREIGN KEY (source_concept_id) REFERENCES concepts(id),
FOREIGN KEY (target_concept_id) REFERENCES concepts(id)
)
""")
await db.commit()What does this code do?
The concepts table stores each extracted concept with its name, definition, category (theory, definition, formula, example, or method), and which document and folder it belongs to.
The concept_edges table stores the relationships between concepts. Each edge links a source concept to a target concept with a label describing the relationship. This creates the graph structure that React Flow will visualize later.
- Save knowledge_map.py.
Import errors when running the server?
Make sure aiosqlite is already installed from your earlier steps. If you see a module not found error, check your virtual environment is activated.
help me fix my aiosqlite import error in knowledge_map.py
Build the concept extraction endpoint
The extraction endpoint collects text from a document or folder, splits it into sections, and sends each section to Groq with a structured prompt. Groq returns a JSON array of concepts with their names, definitions, categories, and relationships.
- In knowledge_map.py, add the Groq extraction logic and the POST endpoint below your create_knowledge_map_tables function:
import os
from groq import Groq
# Initialize the Groq client
groq_client = Groq(api_key=os.getenv("GROQ_API_KEY"))
EXTRACTION_PROMPT = """Extract key concepts from this text. Return JSON array:
[{"name": "string", "definition": "string", "category": "string", "related_to": ["concept_names"]}].
Categories: theory, definition, formula, example, method.
Only return valid JSON, no other text.
Text:
{text}"""
async def extract_concepts_from_text(text: str) -> list:
"""Send text to Groq and parse the structured concept response."""
response = groq_client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=[
{"role": "system", "content": "You are a concept extraction assistant. Always respond with valid JSON only."},
{"role": "user", "content": EXTRACTION_PROMPT.format(text=text)}
],
temperature=0.3
)
raw = response.choices[0].message.content.strip()
# Strip markdown code fences if present
if raw.startswith("```"):
raw = raw.split("\n", 1)[1]
raw = raw.rsplit("```", 1)[0]
return json.loads(raw)What does the extraction prompt do?
The prompt instructs Groq to return a strictly formatted JSON array. Each concept includes a name, a short definition, a category (one of five types), and a related_to array listing the names of connected concepts.
Setting temperature=0.3 keeps Groq's output deterministic. The code fence stripping handles cases where the LLM wraps its JSON response in markdown formatting.
- Now add the deduplication helper and the main extraction endpoint below the extract_concepts_from_text function:
async def store_concepts_with_dedup(concepts: list, document_id: str, folder_id: str):
"""Store concepts in SQLite with case-insensitive deduplication within scope."""
async with aiosqlite.connect(DB_PATH) as db:
db.row_factory = aiosqlite.Row
for concept in concepts:
name = concept["name"]
definition = concept.get("definition", "")
category = concept.get("category", "")
related_to = concept.get("related_to", [])
# Check for existing concept with same name (case-insensitive) in scope
existing = await db.execute_fetchall(
"SELECT id FROM concepts WHERE LOWER(name) = LOWER(?) AND document_id = ?",
(name, document_id)
)
if existing:
# Concept already exists, skip duplicate insertion
concept_id = existing[0][0]
else:
# Insert new concept
cursor = await db.execute(
"INSERT INTO concepts (name, definition, category, document_id, folder_id) VALUES (?, ?, ?, ?, ?)",
(name, definition, category, document_id, folder_id)
)
concept_id = cursor.lastrowid
# Create edges from this concept to its related concepts
for related_name in related_to:
# Find or create the related concept
related = await db.execute_fetchall(
"SELECT id FROM concepts WHERE LOWER(name) = LOWER(?) AND document_id = ?",
(related_name, document_id)
)
if related:
target_id = related[0][0]
else:
# Create a placeholder concept for the related name
rel_cursor = await db.execute(
"INSERT INTO concepts (name, definition, category, document_id, folder_id) VALUES (?, ?, ?, ?, ?)",
(related_name, "", "", document_id, folder_id)
)
target_id = rel_cursor.lastrowid
# Avoid duplicate edges
edge_exists = await db.execute_fetchall(
"SELECT id FROM concept_edges WHERE source_concept_id = ? AND target_concept_id = ?",
(concept_id, target_id)
)
if not edge_exists:
await db.execute(
"INSERT INTO concept_edges (source_concept_id, target_concept_id, relationship_label) VALUES (?, ?, ?)",
(concept_id, target_id, "related")
)
await db.commit()How does deduplication work?
Before inserting a concept, the function checks if a concept with the same name (case-insensitive) already exists for the same document. If it does, the function skips the duplicate and reuses the existing ID for edge creation.
For related concepts that do not exist yet, the function creates placeholder entries. These placeholders get filled in with proper definitions when the extraction processes the section containing that concept.
- Add the POST extraction endpoint below the store_concepts_with_dedup function:
@router.post("/knowledge-map/extract/{scope_type}/{scope_id}")
async def extract_knowledge_map(scope_type: str, scope_id: str):
"""Extract concepts from a document or folder and store them."""
await create_knowledge_map_tables()
async with aiosqlite.connect(DB_PATH) as db:
db.row_factory = aiosqlite.Row
if scope_type == "document":
# Get text chunks for this document from the documents table
rows = await db.execute_fetchall(
"SELECT id, folder_id FROM documents WHERE id = ?", (scope_id,)
)
if not rows:
raise HTTPException(status_code=404, detail="Document not found")
folder_id = rows[0][1] if rows[0][1] else ""
document_ids = [scope_id]
elif scope_type == "folder":
# Get all documents in this folder
rows = await db.execute_fetchall(
"SELECT id FROM documents WHERE folder_id = ?", (scope_id,)
)
if not rows:
raise HTTPException(status_code=404, detail="No documents in folder")
document_ids = [row[0] for row in rows]
folder_id = scope_id
else:
raise HTTPException(status_code=400, detail="scope_type must be 'document' or 'folder'")
# For each document, collect its text and extract concepts
import chromadb
chroma_client = chromadb.PersistentClient(path="./chroma_data")
collection = chroma_client.get_collection("documents")
for doc_id in document_ids:
# Retrieve all chunks for this document from ChromaDB
results = collection.get(where={"document_id": doc_id}, include=["documents"])
if not results["documents"]:
continue
# Combine chunks into sections for extraction
full_text = "\n\n".join(results["documents"])
# Split into logical sections (roughly 1500 chars each)
sections = [full_text[i:i+1500] for i in range(0, len(full_text), 1500)]
for section in sections:
try:
concepts = await extract_concepts_from_text(section)
await store_concepts_with_dedup(concepts, doc_id, folder_id)
except (json.JSONDecodeError, KeyError):
# Skip sections where Groq returns invalid JSON
continue
return {"status": "success", "message": "Concepts extracted and stored"}What does this endpoint do?
The endpoint accepts a scope_type (either document or folder) and a scope_id. For a single document, it extracts concepts from that document only. For a folder, it extracts concepts from all documents within.
The text is pulled from ChromaDB chunks (which already have document_id metadata), split into manageable sections, and sent to Groq for extraction. The try/except block handles cases where Groq returns malformed JSON for a section.
- Save knowledge_map.py.
Getting 404 errors when calling the endpoint?
Make sure you have documents already uploaded and stored in your documents table. The endpoint looks up the document by its id column, so verify the ID you pass matches an existing record.
help me debug why my knowledge map extraction endpoint returns 404
Build the aggregation and regeneration endpoints
The frontend needs a single endpoint that returns the full concept graph as nodes and edges. You also need a regeneration endpoint that clears stale concepts and re-extracts them when a document is updated.
- In knowledge_map.py, add the GET aggregation endpoint below the extraction endpoint:
@router.get("/knowledge-map/{scope_type}/{scope_id}")
async def get_knowledge_map(scope_type: str, scope_id: str):
"""Return the full concept graph as nodes and edges for visualization."""
await create_knowledge_map_tables()
async with aiosqlite.connect(DB_PATH) as db:
db.row_factory = aiosqlite.Row
if scope_type == "document":
# Get concepts for a single document
concept_rows = await db.execute_fetchall(
"SELECT c.id, c.name, c.definition, c.category, d.title as document_title "
"FROM concepts c LEFT JOIN documents d ON c.document_id = d.id "
"WHERE c.document_id = ?",
(scope_id,)
)
elif scope_type == "folder":
# Get concepts for all documents in a folder
concept_rows = await db.execute_fetchall(
"SELECT c.id, c.name, c.definition, c.category, d.title as document_title "
"FROM concepts c LEFT JOIN documents d ON c.document_id = d.id "
"WHERE c.folder_id = ?",
(scope_id,)
)
else:
raise HTTPException(status_code=400, detail="scope_type must be 'document' or 'folder'")
# Build nodes list
nodes = []
concept_ids = set()
for row in concept_rows:
concept_ids.add(row[0])
nodes.append({
"id": row[0],
"name": row[1],
"definition": row[2] or "",
"category": row[3] or "",
"document_title": row[4] or ""
})
# Get edges where both source and target are in our concept set
if concept_ids:
placeholders = ",".join("?" * len(concept_ids))
edge_rows = await db.execute_fetchall(
f"SELECT source_concept_id, target_concept_id, relationship_label "
f"FROM concept_edges "
f"WHERE source_concept_id IN ({placeholders}) AND target_concept_id IN ({placeholders})",
list(concept_ids) + list(concept_ids)
)
edges = [{"source": row[0], "target": row[1], "label": row[2] or ""} for row in edge_rows]
else:
edges = []
return {"nodes": nodes, "edges": edges}What does the aggregation endpoint return?
The response is a JSON object with two arrays: nodes (each containing id, name, definition, category, and document_title) and edges (each containing source, target, and label). This structure maps directly to what React Flow expects for rendering a graph.
The query only returns edges where both endpoints are within the current scope. This prevents dangling edges that point to concepts in other documents or folders.
- Add the POST regeneration endpoint below the GET endpoint:
@router.post("/knowledge-map/regenerate")
async def regenerate_knowledge_map(scope_type: str, scope_id: str):
"""Clear existing concepts for a scope and re-extract from scratch."""
await create_knowledge_map_tables()
async with aiosqlite.connect(DB_PATH) as db:
if scope_type == "document":
# Delete edges connected to concepts in this document
await db.execute(
"DELETE FROM concept_edges WHERE source_concept_id IN "
"(SELECT id FROM concepts WHERE document_id = ?)",
(scope_id,)
)
await db.execute(
"DELETE FROM concept_edges WHERE target_concept_id IN "
"(SELECT id FROM concepts WHERE document_id = ?)",
(scope_id,)
)
# Delete the concepts themselves
await db.execute("DELETE FROM concepts WHERE document_id = ?", (scope_id,))
elif scope_type == "folder":
# Delete edges connected to concepts in this folder
await db.execute(
"DELETE FROM concept_edges WHERE source_concept_id IN "
"(SELECT id FROM concepts WHERE folder_id = ?)",
(scope_id,)
)
await db.execute(
"DELETE FROM concept_edges WHERE target_concept_id IN "
"(SELECT id FROM concepts WHERE folder_id = ?)",
(scope_id,)
)
# Delete the concepts themselves
await db.execute("DELETE FROM concepts WHERE folder_id = ?", (scope_id,))
else:
raise HTTPException(status_code=400, detail="scope_type must be 'document' or 'folder'")
await db.commit()
# Re-extract concepts
return await extract_knowledge_map(scope_type, scope_id)Why delete edges before concepts?
The concept_edges table has foreign key references to concepts. Deleting edges first avoids foreign key constraint violations. After clearing both tables for the given scope, the endpoint calls the extraction function to rebuild the graph from scratch.
- Save knowledge_map.py.
- Register the new router in your main FastAPI app file by adding this import and inclusion:
from knowledge_map import router as knowledge_map_router
app.include_router(knowledge_map_router, tags=["Knowledge Map"])- Save main.py and restart your FastAPI server.
- Test the extraction by sending a POST request to /knowledge-map/extract/document/{doc_id} using an existing document ID. You should receive a success response.
- Verify the graph by sending a GET request to /knowledge-map/document/{doc_id}. You should see a JSON response with nodes and edges arrays populated with your extracted concepts.
Getting an empty nodes array?
If the GET endpoint returns empty arrays, the extraction may not have completed. Check your terminal for errors from the Groq API call. Common issues include an invalid or missing GROQ_API_KEY environment variable, or rate limiting on the Groq free tier (30 requests per minute).
help me debug why my knowledge map extraction returns empty results
✔️ Awesome, I've got everything!
Great. Double check you have saved both knowledge_map.py and main.py.
ⓧ I'd like to double check the full code
import json
import os
import aiosqlite
import chromadb
from fastapi import APIRouter, HTTPException
from groq import Groq
# Database path (same as the rest of the app)
DB_PATH = "e-hanatra.db"
router = APIRouter()
# Initialize the Groq client
groq_client = Groq(api_key=os.getenv("GROQ_API_KEY"))
EXTRACTION_PROMPT = """Extract key concepts from this text. Return JSON array:
[{"name": "string", "definition": "string", "category": "string", "related_to": ["concept_names"]}].
Categories: theory, definition, formula, example, method.
Only return valid JSON, no other text.
Text:
{text}"""
async def create_knowledge_map_tables():
"""Create the concepts and concept_edges tables if they don't exist."""
async with aiosqlite.connect(DB_PATH) as db:
# Store individual concepts extracted from documents
await db.execute("""
CREATE TABLE IF NOT EXISTS concepts (
id INTEGER PRIMARY KEY AUTOINCREMENT,
name TEXT NOT NULL,
definition TEXT,
category TEXT,
document_id TEXT,
folder_id TEXT
)
""")
# Store relationships between concepts
await db.execute("""
CREATE TABLE IF NOT EXISTS concept_edges (
id INTEGER PRIMARY KEY AUTOINCREMENT,
source_concept_id INTEGER NOT NULL,
target_concept_id INTEGER NOT NULL,
relationship_label TEXT,
FOREIGN KEY (source_concept_id) REFERENCES concepts(id),
FOREIGN KEY (target_concept_id) REFERENCES concepts(id)
)
""")
await db.commit()
async def extract_concepts_from_text(text: str) -> list:
"""Send text to Groq and parse the structured concept response."""
response = groq_client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=[
{"role": "system", "content": "You are a concept extraction assistant. Always respond with valid JSON only."},
{"role": "user", "content": EXTRACTION_PROMPT.format(text=text)}
],
temperature=0.3
)
raw = response.choices[0].message.content.strip()
# Strip markdown code fences if present
if raw.startswith("```"):
raw = raw.split("\n", 1)[1]
raw = raw.rsplit("```", 1)[0]
return json.loads(raw)
async def store_concepts_with_dedup(concepts: list, document_id: str, folder_id: str):
"""Store concepts in SQLite with case-insensitive deduplication within scope."""
async with aiosqlite.connect(DB_PATH) as db:
db.row_factory = aiosqlite.Row
for concept in concepts:
name = concept["name"]
definition = concept.get("definition", "")
category = concept.get("category", "")
related_to = concept.get("related_to", [])
# Check for existing concept with same name (case-insensitive) in scope
existing = await db.execute_fetchall(
"SELECT id FROM concepts WHERE LOWER(name) = LOWER(?) AND document_id = ?",
(name, document_id)
)
if existing:
# Concept already exists, skip duplicate insertion
concept_id = existing[0][0]
else:
# Insert new concept
cursor = await db.execute(
"INSERT INTO concepts (name, definition, category, document_id, folder_id) VALUES (?, ?, ?, ?, ?)",
(name, definition, category, document_id, folder_id)
)
concept_id = cursor.lastrowid
# Create edges from this concept to its related concepts
for related_name in related_to:
# Find or create the related concept
related = await db.execute_fetchall(
"SELECT id FROM concepts WHERE LOWER(name) = LOWER(?) AND document_id = ?",
(related_name, document_id)
)
if related:
target_id = related[0][0]
else:
# Create a placeholder concept for the related name
rel_cursor = await db.execute(
"INSERT INTO concepts (name, definition, category, document_id, folder_id) VALUES (?, ?, ?, ?, ?)",
(related_name, "", "", document_id, folder_id)
)
target_id = rel_cursor.lastrowid
# Avoid duplicate edges
edge_exists = await db.execute_fetchall(
"SELECT id FROM concept_edges WHERE source_concept_id = ? AND target_concept_id = ?",
(concept_id, target_id)
)
if not edge_exists:
await db.execute(
"INSERT INTO concept_edges (source_concept_id, target_concept_id, relationship_label) VALUES (?, ?, ?)",
(concept_id, target_id, "related")
)
await db.commit()
@router.post("/knowledge-map/extract/{scope_type}/{scope_id}")
async def extract_knowledge_map(scope_type: str, scope_id: str):
"""Extract concepts from a document or folder and store them."""
await create_knowledge_map_tables()
async with aiosqlite.connect(DB_PATH) as db:
db.row_factory = aiosqlite.Row
if scope_type == "document":
# Get text chunks for this document from the documents table
rows = await db.execute_fetchall(
"SELECT id, folder_id FROM documents WHERE id = ?", (scope_id,)
)
if not rows:
raise HTTPException(status_code=404, detail="Document not found")
folder_id = rows[0][1] if rows[0][1] else ""
document_ids = [scope_id]
elif scope_type == "folder":
# Get all documents in this folder
rows = await db.execute_fetchall(
"SELECT id FROM documents WHERE folder_id = ?", (scope_id,)
)
if not rows:
raise HTTPException(status_code=404, detail="No documents in folder")
document_ids = [row[0] for row in rows]
folder_id = scope_id
else:
raise HTTPException(status_code=400, detail="scope_type must be 'document' or 'folder'")
# For each document, collect its text and extract concepts
chroma_client = chromadb.PersistentClient(path="./chroma_data")
collection = chroma_client.get_collection("documents")
for doc_id in document_ids:
# Retrieve all chunks for this document from ChromaDB
results = collection.get(where={"document_id": doc_id}, include=["documents"])
if not results["documents"]:
continue
# Combine chunks into sections for extraction
full_text = "\n\n".join(results["documents"])
# Split into logical sections (roughly 1500 chars each)
sections = [full_text[i:i+1500] for i in range(0, len(full_text), 1500)]
for section in sections:
try:
concepts = await extract_concepts_from_text(section)
await store_concepts_with_dedup(concepts, doc_id, folder_id)
except (json.JSONDecodeError, KeyError):
# Skip sections where Groq returns invalid JSON
continue
return {"status": "success", "message": "Concepts extracted and stored"}
@router.get("/knowledge-map/{scope_type}/{scope_id}")
async def get_knowledge_map(scope_type: str, scope_id: str):
"""Return the full concept graph as nodes and edges for visualization."""
await create_knowledge_map_tables()
async with aiosqlite.connect(DB_PATH) as db:
db.row_factory = aiosqlite.Row
if scope_type == "document":
# Get concepts for a single document
concept_rows = await db.execute_fetchall(
"SELECT c.id, c.name, c.definition, c.category, d.title as document_title "
"FROM concepts c LEFT JOIN documents d ON c.document_id = d.id "
"WHERE c.document_id = ?",
(scope_id,)
)
elif scope_type == "folder":
# Get concepts for all documents in a folder
concept_rows = await db.execute_fetchall(
"SELECT c.id, c.name, c.definition, c.category, d.title as document_title "
"FROM concepts c LEFT JOIN documents d ON c.document_id = d.id "
"WHERE c.folder_id = ?",
(scope_id,)
)
else:
raise HTTPException(status_code=400, detail="scope_type must be 'document' or 'folder'")
# Build nodes list
nodes = []
concept_ids = set()
for row in concept_rows:
concept_ids.add(row[0])
nodes.append({
"id": row[0],
"name": row[1],
"definition": row[2] or "",
"category": row[3] or "",
"document_title": row[4] or ""
})
# Get edges where both source and target are in our concept set
if concept_ids:
placeholders = ",".join("?" * len(concept_ids))
edge_rows = await db.execute_fetchall(
f"SELECT source_concept_id, target_concept_id, relationship_label "
f"FROM concept_edges "
f"WHERE source_concept_id IN ({placeholders}) AND target_concept_id IN ({placeholders})",
list(concept_ids) + list(concept_ids)
)
edges = [{"source": row[0], "target": row[1], "label": row[2] or ""} for row in edge_rows]
else:
edges = []
return {"nodes": nodes, "edges": edges}
@router.post("/knowledge-map/regenerate")
async def regenerate_knowledge_map(scope_type: str, scope_id: str):
"""Clear existing concepts for a scope and re-extract from scratch."""
await create_knowledge_map_tables()
async with aiosqlite.connect(DB_PATH) as db:
if scope_type == "document":
# Delete edges connected to concepts in this document
await db.execute(
"DELETE FROM concept_edges WHERE source_concept_id IN "
"(SELECT id FROM concepts WHERE document_id = ?)",
(scope_id,)
)
await db.execute(
"DELETE FROM concept_edges WHERE target_concept_id IN "
"(SELECT id FROM concepts WHERE document_id = ?)",
(scope_id,)
)
# Delete the concepts themselves
await db.execute("DELETE FROM concepts WHERE document_id = ?", (scope_id,))
elif scope_type == "folder":
# Delete edges connected to concepts in this folder
await db.execute(
"DELETE FROM concept_edges WHERE source_concept_id IN "
"(SELECT id FROM concepts WHERE folder_id = ?)",
(scope_id,)
)
await db.execute(
"DELETE FROM concept_edges WHERE target_concept_id IN "
"(SELECT id FROM concepts WHERE folder_id = ?)",
(scope_id,)
)
# Delete the concepts themselves
await db.execute("DELETE FROM concepts WHERE folder_id = ?", (scope_id,))
else:
raise HTTPException(status_code=400, detail="scope_type must be 'document' or 'folder'")
await db.commit()
# Re-extract concepts
return await extract_knowledge_map(scope_type, scope_id)Your concept extraction backend is ready. Next up, you will build the interactive frontend visualization using React Flow to turn this graph data into a clickable, zoomable Knowledge Map.
Build the Knowledge Map — Interactive Visualization
Your concept extraction engine is live. It pulls key ideas from documents, stores them in SQLite, and exposes them as structured graph data through your API.
Now you need a way for students to actually see those connections. In this step, you will build an interactive Knowledge Map using React Flow that renders concept nodes with color-coded categories, auto-positions them in a hierarchical layout, and lets students click into any concept for full details.
In this step, get ready to:
- Install React Flow and dagre, then build a KnowledgeMap page with a custom ConceptNode component and automatic hierarchical layout.
- Add interactive features including a node detail panel, zoom controls, MiniMap, category filters, and search highlighting.
- Wire the map to URL-based scope params, add Generate Map and Regenerate buttons, and show loading skeletons during extraction.
Install React Flow and dagre, then build the KnowledgeMap page
React Flow renders interactive node-based diagrams as native React components. Dagre computes hierarchical positions for each node so you don't have to manually place hundreds of concepts.
- Install both packages by running this command in your Next.js project directory:
npm install @xyflow/react @dagrejs/dagreWhat are these packages?
@xyflow/react is the React Flow library (36.5K GitHub stars, MIT licensed). It gives you the ReactFlow component, built-in plugins like Controls and MiniMap, and hooks for managing node/edge state.
@dagrejs/dagre is the actively maintained dagre library for directed graph layout. It computes x/y positions for nodes arranged in a tree hierarchy.
- Confirm the packages installed by running:
ls node_modules/@xyflow/react node_modules/@dagrejs/dagreYou should see folder contents listed for both packages without errors.
Seeing install errors?
Make sure you are running the command inside your Next.js project folder (the one containing package.json). If you see peer dependency warnings about React versions, ensure your project uses React 18 or higher.
help me fix my @xyflow/react install errors
Now create the custom ConceptNode component. This is a React component that React Flow renders for each concept in your map.
- Create a new file components/ConceptNode.tsx and add the following code:
import { Handle, Position } from '@xyflow/react';
// Color mapping for concept categories
const categoryColors: Record<string, string> = {
theory: '#3b82f6', // blue
definition: '#22c55e', // green
formula: '#f97316', // orange
example: '#a855f7', // purple
method: '#ef4444', // red
};
interface ConceptNodeData {
label: string;
definition: string;
category: string;
document_title: string;
}
// Custom node component displaying concept name with category-colored border
export default function ConceptNode({ data }: { data: ConceptNodeData }) {
const borderColor = categoryColors[data.category] || '#6b7280';
return (
<div
style={{
padding: '10px 16px',
borderRadius: '8px',
border: `2px solid ${borderColor}`,
background: '#1f2937',
color: '#f9fafb',
fontSize: '13px',
fontWeight: 500,
maxWidth: '180px',
textAlign: 'center',
}}
>
<Handle type="target" position={Position.Top} />
<span>{data.label}</span>
<Handle type="source" position={Position.Bottom} />
</div>
);
}What does this code do?
categoryColors maps each concept category to a distinct border color. Theory is blue, definition is green, formula is orange, example is purple, and method is red.
The Handle components from React Flow mark where edges connect. Target handles accept incoming edges at the top. Source handles emit outgoing edges from the bottom.
Next, build the main KnowledgeMap page that fetches graph data and renders it with dagre layout.
- Create the file app/knowledge-map/page.tsx and add this code:
'use client';
import { useCallback, useEffect, useMemo, useState } from 'react';
import { useSearchParams } from 'next/navigation';
import {
ReactFlow,
Controls,
MiniMap,
Background,
useNodesState,
useEdgesState,
} from '@xyflow/react';
import dagre from '@dagrejs/dagre';
import '@xyflow/react/dist/style.css';
import ConceptNode from '@/components/ConceptNode';
// Register the custom node type
const nodeTypes = { concept: ConceptNode };
// Dagre layout configuration
const nodeWidth = 180;
const nodeHeight = 50;
// Compute hierarchical positions using dagre
function getLayoutedElements(nodes: any[], edges: any[], direction = 'TB') {
const dagreGraph = new dagre.graphlib.Graph().setDefaultEdgeLabel(() => ({}));
dagreGraph.setGraph({ rankdir: direction, nodesep: 60, ranksep: 80 });
nodes.forEach((node) => {
dagreGraph.setNode(node.id, { width: nodeWidth, height: nodeHeight });
});
edges.forEach((edge) => {
dagreGraph.setEdge(edge.source, edge.target);
});
dagre.layout(dagreGraph);
const layoutedNodes = nodes.map((node) => {
const position = dagreGraph.node(node.id);
return {
...node,
targetPosition: 'top',
sourcePosition: 'bottom',
position: {
x: position.x - nodeWidth / 2,
y: position.y - nodeHeight / 2,
},
};
});
return { nodes: layoutedNodes, edges };
}
export default function KnowledgeMapPage() {
const searchParams = useSearchParams();
const scopeType = searchParams.get('scope') || 'global';
const scopeId = searchParams.get('id') || '';
const [nodes, setNodes, onNodesChange] = useNodesState([]);
const [edges, setEdges, onEdgesChange] = useEdgesState([]);
const [loading, setLoading] = useState(true);
const [selectedNode, setSelectedNode] = useState<any>(null);
const [categoryFilters, setCategoryFilters] = useState<Record<string, boolean>>({
theory: true,
definition: true,
formula: true,
example: true,
method: true,
});
const [searchQuery, setSearchQuery] = useState('');
// Fetch concept graph data from the API
const fetchGraph = useCallback(async () => {
setLoading(true);
try {
const res = await fetch(
`/api/knowledge-map/${scopeType}/${scopeId}`
);
const data = await res.json();
// Transform API response into React Flow nodes
const rawNodes = data.nodes.map((concept: any) => ({
id: concept.id,
type: 'concept',
data: {
label: concept.name,
definition: concept.definition,
category: concept.category,
document_title: concept.document_title,
},
position: { x: 0, y: 0 },
}));
// Transform API response into React Flow edges
const rawEdges = data.edges.map((edge: any, idx: number) => ({
id: `edge-${idx}`,
source: edge.source,
target: edge.target,
label: edge.label,
animated: true,
}));
// Apply dagre layout
const { nodes: layoutedNodes, edges: layoutedEdges } =
getLayoutedElements(rawNodes, rawEdges);
setNodes(layoutedNodes);
setEdges(layoutedEdges);
} catch (err) {
console.error('Failed to fetch knowledge map:', err);
} finally {
setLoading(false);
}
}, [scopeType, scopeId, setNodes, setEdges]);
useEffect(() => {
if (scopeId) fetchGraph();
}, [fetchGraph, scopeId]);
// Filter nodes by category
const filteredNodes = useMemo(() => {
return nodes.filter((node) => {
const category = (node.data as any).category;
return categoryFilters[category] !== false;
});
}, [nodes, categoryFilters]);
// Highlight nodes matching search query
const displayNodes = useMemo(() => {
if (!searchQuery) return filteredNodes;
return filteredNodes.map((node) => ({
...node,
style: (node.data as any).label
.toLowerCase()
.includes(searchQuery.toLowerCase())
? { opacity: 1 }
: { opacity: 0.3 },
}));
}, [filteredNodes, searchQuery]);
// Handle node click to show side panel
const onNodeClick = useCallback((_: any, node: any) => {
setSelectedNode(node.data);
}, []);
// Regenerate the map from scratch
const handleRegenerate = async () => {
await fetch(`/api/knowledge-map/regenerate`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ scope_type: scopeType, scope_id: scopeId }),
});
fetchGraph();
};
if (loading) {
return (
<div className="knowledge-map-skeleton">
<div className="skeleton-header" />
<div className="skeleton-graph" />
</div>
);
}
return (
<div style={{ display: 'flex', height: '100vh' }}>
{/* Main map area */}
<div style={{ flex: 1 }}>
{/* Toolbar: search, filters, regenerate */}
<div className="map-toolbar">
<input
type="text"
placeholder="Search concepts..."
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
className="map-search"
/>
<div className="category-filters">
{Object.keys(categoryFilters).map((cat) => (
<label key={cat} className="filter-checkbox">
<input
type="checkbox"
checked={categoryFilters[cat]}
onChange={() =>
setCategoryFilters((prev) => ({
...prev,
[cat]: !prev[cat],
}))
}
/>
{cat}
</label>
))}
</div>
<button onClick={handleRegenerate} className="btn-regenerate">
Regenerate
</button>
</div>
{/* React Flow canvas */}
<ReactFlow
nodes={displayNodes}
edges={edges}
onNodesChange={onNodesChange}
onEdgesChange={onEdgesChange}
onNodeClick={onNodeClick}
nodeTypes={nodeTypes}
fitView
>
<Controls />
<MiniMap
nodeColor={(node) => {
const colors: Record<string, string> = {
theory: '#3b82f6',
definition: '#22c55e',
formula: '#f97316',
example: '#a855f7',
method: '#ef4444',
};
return colors[(node.data as any).category] || '#6b7280';
}}
/>
<Background variant="dots" gap={16} size={1} />
</ReactFlow>
</div>
{/* Side panel for selected concept details */}
{selectedNode && (
<div className="concept-panel">
<button onClick={() => setSelectedNode(null)} className="panel-close">
Close
</button>
<h3>{selectedNode.label}</h3>
<p className="panel-category">{selectedNode.category}</p>
<p className="panel-definition">{selectedNode.definition}</p>
<p className="panel-source">
Source: {selectedNode.document_title}
</p>
</div>
)}
</div>
);
}What does this code do?
getLayoutedElements creates a dagre graph, feeds it your nodes and edges with their dimensions, then runs dagre.layout() to compute hierarchical x/y positions. It shifts positions from center-anchor (dagre default) to top-left anchor (React Flow default).
The page reads scope and id from URL search params. This determines whether the map shows concepts for a single document, a folder, or the entire knowledge base.
categoryFilters is a state object with one boolean per category. Unchecking a category removes those nodes from the display. The searchQuery dims non-matching nodes to 30% opacity so matching ones stand out.
- Save both files.
- Start your Next.js dev server if it is not already running.
- Navigate to /knowledge-map?scope=document&id= followed by a valid document ID from your database. You should see a loading skeleton briefly, then a hierarchical graph of color-bordered concept nodes.
Seeing a blank page or import errors?
Ensure @xyflow/react/dist/style.css is imported in your page file. Without it, React Flow renders with zero dimensions. Also check that your tsconfig.json has the @/ path alias pointing to your project root.
help me debug my React Flow Knowledge Map not rendering
Add interactivity with Controls, MiniMap, filters, and search
The core rendering is done. The code above already includes the interactive features. Let's verify each one works correctly.
- Click any concept node on the map. A side panel should slide in from the right showing the concept's full definition, category, and source document title.
- Test the zoom controls in the bottom-left corner. The Controls component provides zoom-in, zoom-out, and fit-to-view buttons.
- Check the MiniMap in the bottom-right corner. It shows a small overview of the entire graph with category-colored dots.
- Uncheck one of the category filters in the toolbar (for example, uncheck theory). All blue-bordered theory nodes should disappear from the map.
- Type a concept name in the search input. Matching nodes stay fully visible while non-matching nodes dim to 30% opacity.
Why use the Background dots pattern?
The dots pattern from React Flow's Background component gives the canvas a subtle grid that helps students gauge distance and position when panning around a large concept graph. It matches E-Hanatra's dark theme without adding visual clutter.
Filters or search not working?
The filtering logic depends on node.data.category matching one of the five category keys (theory, definition, formula, example, method). Verify your Groq extraction prompt returns these exact category strings. If your backend returns different category names, the filters will have no effect.
help me debug my Knowledge Map category filters
Add scope awareness, Generate Map buttons, and loading state
The Knowledge Map page already reads scope from URL params. Now you need to add navigation buttons in the folder and document views so students can launch the map from anywhere in the app.
- Open your existing folder view component (the component that displays folder contents).
- Add a "Generate Map" button that navigates to the Knowledge Map page with the correct scope. Add this button in the folder view's toolbar area:
import { useRouter } from 'next/navigation';
// Inside your folder view component
const router = useRouter();
// Generate Map button handler
function handleGenerateMap(folderId: string) {
router.push(`/knowledge-map?scope=folder&id=${folderId}`);
}What does this code do?
useRouter from Next.js navigation lets you push new routes programmatically. The button constructs a URL with scope=folder and the folder's ID, which the KnowledgeMap page picks up from useSearchParams().
- Add the same pattern in your document view component, but with scope=document:
// Generate Map button for a single document
function handleGenerateMap(documentId: string) {
router.push(`/knowledge-map?scope=document&id=${documentId}`);
}- Render both buttons as a styled button element in their respective views:
<button
onClick={() => handleGenerateMap(currentId)}
className="btn-generate-map"
>
Generate Map
</button>What about the loading skeleton?
The loading skeleton is already built into the KnowledgeMap page. When loading is true, the page renders placeholder div elements with CSS classes skeleton-header and skeleton-graph. Style these with a pulsing animation in your CSS to show students that data is loading.
The Regenerate button is also already in the toolbar. It calls POST /api/knowledge-map/regenerate to clear and re-extract concepts, then re-fetches the graph. Students use this after uploading new documents to refresh a stale map.
- Save all files.
- Navigate to a folder in your app and click the Generate Map button. You should be taken to /knowledge-map?scope=folder&id=... and see the map render with all concepts from that folder's documents.
- Click Regenerate in the map toolbar. The loading skeleton should appear briefly, then the map refreshes with fresh concept data.
Generate Map button not navigating?
Ensure your component imports useRouter from next/navigation (not next/router). In Next.js 15 with the App Router, the correct import path is next/navigation.
help me fix my Generate Map navigation in Next.js 15
✔️ Awesome, I've got everything!
Great. Double check you have saved all your files and that the map renders with colored nodes and working filters.
ⓧ I'd like to double check the full code
Here is the full components/ConceptNode.tsx:
import { Handle, Position } from '@xyflow/react';
// Color mapping for concept categories
const categoryColors: Record<string, string> = {
theory: '#3b82f6', // blue
definition: '#22c55e', // green
formula: '#f97316', // orange
example: '#a855f7', // purple
method: '#ef4444', // red
};
interface ConceptNodeData {
label: string;
definition: string;
category: string;
document_title: string;
}
// Custom node component displaying concept name with category-colored border
export default function ConceptNode({ data }: { data: ConceptNodeData }) {
const borderColor = categoryColors[data.category] || '#6b7280';
return (
<div
style={{
padding: '10px 16px',
borderRadius: '8px',
border: `2px solid ${borderColor}`,
background: '#1f2937',
color: '#f9fafb',
fontSize: '13px',
fontWeight: 500,
maxWidth: '180px',
textAlign: 'center',
}}
>
<Handle type="target" position={Position.Top} />
<span>{data.label}</span>
<Handle type="source" position={Position.Bottom} />
</div>
);
}Here is the full app/knowledge-map/page.tsx:
'use client';
import { useCallback, useEffect, useMemo, useState } from 'react';
import { useSearchParams } from 'next/navigation';
import {
ReactFlow,
Controls,
MiniMap,
Background,
useNodesState,
useEdgesState,
} from '@xyflow/react';
import dagre from '@dagrejs/dagre';
import '@xyflow/react/dist/style.css';
import ConceptNode from '@/components/ConceptNode';
// Register the custom node type
const nodeTypes = { concept: ConceptNode };
// Dagre layout configuration
const nodeWidth = 180;
const nodeHeight = 50;
// Compute hierarchical positions using dagre
function getLayoutedElements(nodes: any[], edges: any[], direction = 'TB') {
const dagreGraph = new dagre.graphlib.Graph().setDefaultEdgeLabel(() => ({}));
dagreGraph.setGraph({ rankdir: direction, nodesep: 60, ranksep: 80 });
nodes.forEach((node) => {
dagreGraph.setNode(node.id, { width: nodeWidth, height: nodeHeight });
});
edges.forEach((edge) => {
dagreGraph.setEdge(edge.source, edge.target);
});
dagre.layout(dagreGraph);
const layoutedNodes = nodes.map((node) => {
const position = dagreGraph.node(node.id);
return {
...node,
targetPosition: 'top',
sourcePosition: 'bottom',
position: {
x: position.x - nodeWidth / 2,
y: position.y - nodeHeight / 2,
},
};
});
return { nodes: layoutedNodes, edges };
}
export default function KnowledgeMapPage() {
const searchParams = useSearchParams();
const scopeType = searchParams.get('scope') || 'global';
const scopeId = searchParams.get('id') || '';
const [nodes, setNodes, onNodesChange] = useNodesState([]);
const [edges, setEdges, onEdgesChange] = useEdgesState([]);
const [loading, setLoading] = useState(true);
const [selectedNode, setSelectedNode] = useState<any>(null);
const [categoryFilters, setCategoryFilters] = useState<Record<string, boolean>>({
theory: true,
definition: true,
formula: true,
example: true,
method: true,
});
const [searchQuery, setSearchQuery] = useState('');
// Fetch concept graph data from the API
const fetchGraph = useCallback(async () => {
setLoading(true);
try {
const res = await fetch(
`/api/knowledge-map/${scopeType}/${scopeId}`
);
const data = await res.json();
// Transform API response into React Flow nodes
const rawNodes = data.nodes.map((concept: any) => ({
id: concept.id,
type: 'concept',
data: {
label: concept.name,
definition: concept.definition,
category: concept.category,
document_title: concept.document_title,
},
position: { x: 0, y: 0 },
}));
// Transform API response into React Flow edges
const rawEdges = data.edges.map((edge: any, idx: number) => ({
id: `edge-${idx}`,
source: edge.source,
target: edge.target,
label: edge.label,
animated: true,
}));
// Apply dagre layout
const { nodes: layoutedNodes, edges: layoutedEdges } =
getLayoutedElements(rawNodes, rawEdges);
setNodes(layoutedNodes);
setEdges(layoutedEdges);
} catch (err) {
console.error('Failed to fetch knowledge map:', err);
} finally {
setLoading(false);
}
}, [scopeType, scopeId, setNodes, setEdges]);
useEffect(() => {
if (scopeId) fetchGraph();
}, [fetchGraph, scopeId]);
// Filter nodes by category
const filteredNodes = useMemo(() => {
return nodes.filter((node) => {
const category = (node.data as any).category;
return categoryFilters[category] !== false;
});
}, [nodes, categoryFilters]);
// Highlight nodes matching search query
const displayNodes = useMemo(() => {
if (!searchQuery) return filteredNodes;
return filteredNodes.map((node) => ({
...node,
style: (node.data as any).label
.toLowerCase()
.includes(searchQuery.toLowerCase())
? { opacity: 1 }
: { opacity: 0.3 },
}));
}, [filteredNodes, searchQuery]);
// Handle node click to show side panel
const onNodeClick = useCallback((_: any, node: any) => {
setSelectedNode(node.data);
}, []);
// Regenerate the map from scratch
const handleRegenerate = async () => {
await fetch(`/api/knowledge-map/regenerate`, {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ scope_type: scopeType, scope_id: scopeId }),
});
fetchGraph();
};
if (loading) {
return (
<div className="knowledge-map-skeleton">
<div className="skeleton-header" />
<div className="skeleton-graph" />
</div>
);
}
return (
<div style={{ display: 'flex', height: '100vh' }}>
{/* Main map area */}
<div style={{ flex: 1 }}>
{/* Toolbar: search, filters, regenerate */}
<div className="map-toolbar">
<input
type="text"
placeholder="Search concepts..."
value={searchQuery}
onChange={(e) => setSearchQuery(e.target.value)}
className="map-search"
/>
<div className="category-filters">
{Object.keys(categoryFilters).map((cat) => (
<label key={cat} className="filter-checkbox">
<input
type="checkbox"
checked={categoryFilters[cat]}
onChange={() =>
setCategoryFilters((prev) => ({
...prev,
[cat]: !prev[cat],
}))
}
/>
{cat}
</label>
))}
</div>
<button onClick={handleRegenerate} className="btn-regenerate">
Regenerate
</button>
</div>
{/* React Flow canvas */}
<ReactFlow
nodes={displayNodes}
edges={edges}
onNodesChange={onNodesChange}
onEdgesChange={onEdgesChange}
onNodeClick={onNodeClick}
nodeTypes={nodeTypes}
fitView
>
<Controls />
<MiniMap
nodeColor={(node) => {
const colors: Record<string, string> = {
theory: '#3b82f6',
definition: '#22c55e',
formula: '#f97316',
example: '#a855f7',
method: '#ef4444',
};
return colors[(node.data as any).category] || '#6b7280';
}}
/>
<Background variant="dots" gap={16} size={1} />
</ReactFlow>
</div>
{/* Side panel for selected concept details */}
{selectedNode && (
<div className="concept-panel">
<button onClick={() => setSelectedNode(null)} className="panel-close">
Close
</button>
<h3>{selectedNode.label}</h3>
<p className="panel-category">{selectedNode.category}</p>
<p className="panel-definition">{selectedNode.definition}</p>
<p className="panel-source">
Source: {selectedNode.document_title}
</p>
</div>
)}
</div>
);
}Your Knowledge Map is fully interactive. Students can now visually explore the connections between concepts extracted from their course materials, filter by category, search for specific ideas, and regenerate maps as they upload new documents. Next up, you will build the revision card engine that turns these same concepts into spaced-repetition flashcards.
Build the Revision Cards — SM-2 Engine Backend
Your knowledge maps give students a visual understanding of how concepts connect. The next question is: how do students remember those concepts long-term?
In this step, you will build a revision card engine powered by the SM-2 spaced repetition algorithm. The system generates flashcards from document content, then intelligently schedules reviews based on how well the student recalls each card.
In this step, get ready to:
- Build the card generation endpoint that uses Groq to create flashcards from documents.
- Implement the SM-2 algorithm and review endpoint for intelligent scheduling.
- Create query endpoints for due cards, stats, and bulk generation.
Create the revision cards table and generation endpoint
Before generating cards, you need a place to store them. The revision_cards table holds each flashcard along with its SM-2 scheduling state.
- Create a new file called sm2.py in your FastAPI project's root directory.
- Add the database migration and table creation for revision_cards by adding the following to your database initialization script:
# Add to your database initialization (e.g., database.py or migrations)
async def create_revision_cards_table(db):
"""Create the revision_cards table for spaced repetition flashcards."""
await db.execute("""
CREATE TABLE IF NOT EXISTS revision_cards (
id TEXT PRIMARY KEY,
document_id TEXT NOT NULL,
folder_id TEXT,
front TEXT NOT NULL,
back TEXT NOT NULL,
difficulty TEXT DEFAULT 'medium',
tags_json TEXT DEFAULT '[]',
interval INTEGER DEFAULT 0,
repetition INTEGER DEFAULT 0,
efactor REAL DEFAULT 2.5,
due_date TEXT DEFAULT (date('now')),
status TEXT DEFAULT 'new',
FOREIGN KEY (document_id) REFERENCES documents(id)
)
""")
await db.commit()What does this table store?
Each row is a single flashcard. The front and back columns hold the question and answer. The scheduling columns (interval, repetition, efactor, due_date) track the SM-2 algorithm's state for each card.
The status field tracks where a card is in the learning lifecycle: new, learning, review, or mastered.
- Now build the card generation endpoint. Add the following to a new file called revision_router.py:
import json
import uuid
from datetime import date
from fastapi import APIRouter, HTTPException, BackgroundTasks
import aiosqlite
from groq_client import groq_client
from database import get_db_path
router = APIRouter(prefix="/revision", tags=["revision"])
# Prompt template for flashcard generation
CARD_GENERATION_PROMPT = """Generate revision flashcards from this content.
Return JSON array: [{"front": "question_or_prompt", "back": "answer_or_explanation", "difficulty": "easy|medium|hard", "tags": ["concept_names"]}].
Create 5-8 cards per logical section.
Vary question types: definitions, applications, comparisons.
Content:
{content}"""
@router.post("/generate/{document_id}")
async def generate_revision_cards(document_id: str, background_tasks: BackgroundTasks):
"""Generate flashcards for a document using Groq LLM."""
db_path = get_db_path()
async with aiosqlite.connect(db_path) as db:
# Get document metadata
cursor = await db.execute(
"SELECT id, folder_id, title FROM documents WHERE id = ?",
(document_id,)
)
doc = await cursor.fetchone()
if not doc:
raise HTTPException(status_code=404, detail="Document not found")
folder_id = doc[1]
# Schedule card generation in the background
background_tasks.add_task(
_generate_cards_for_document, document_id, folder_id
)
return {"message": "Card generation started", "document_id": document_id}
async def _generate_cards_for_document(document_id: str, folder_id: str):
"""Background task: generate and store flashcards."""
db_path = get_db_path()
# Retrieve document text from ChromaDB
from chromadb_client import get_collection
collection = get_collection()
results = collection.get(
where={"document_id": document_id},
include=["documents"]
)
document_text = "\n".join(results["documents"] or [])
if not document_text:
return
# Send to Groq for card generation
prompt = CARD_GENERATION_PROMPT.format(content=document_text)
response = groq_client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=[{"role": "user", "content": prompt}],
temperature=0.7,
response_format={"type": "json_object"}
)
# Parse the generated cards
try:
response_text = response.choices[0].message.content
cards_data = json.loads(response_text)
# Handle both {"cards": [...]} and [...] formats
if isinstance(cards_data, dict):
cards_data = cards_data.get("cards", cards_data.get("flashcards", []))
except (json.JSONDecodeError, IndexError):
return
# Store cards in SQLite
async with aiosqlite.connect(db_path) as db:
for card in cards_data:
card_id = str(uuid.uuid4())
await db.execute("""
INSERT INTO revision_cards
(id, document_id, folder_id, front, back, difficulty, tags_json,
interval, repetition, efactor, due_date, status)
VALUES (?, ?, ?, ?, ?, ?, ?, 0, 0, 2.5, ?, 'new')
""", (
card_id,
document_id,
folder_id,
card.get("front", ""),
card.get("back", ""),
card.get("difficulty", "medium"),
json.dumps(card.get("tags", [])),
date.today().isoformat()
))
await db.commit()What does this code do?
The POST /revision/generate/{document_id} endpoint kicks off card generation as a background task so the API responds immediately.
The background function retrieves document chunks from ChromaDB, sends them to Groq with a structured prompt requesting 5-8 flashcards per section, then stores each card in SQLite with default SM-2 values (interval 0, repetition 0, efactor 2.5, due today).
- Save revision_router.py.
- Register the router in your main FastAPI app file by adding app.include_router(revision_router.router).
- Start your FastAPI server and confirm the new endpoint appears at /docs under the "revision" tag.
Endpoint not showing in /docs?
Make sure you imported the router and called app.include_router(revision_router.router) in your main app file. Also verify aiosqlite is installed in your environment.
Why isn't my revision endpoint appearing in the FastAPI docs?
Implement the SM-2 algorithm and review endpoint
The SM-2 algorithm decides when a student should see a card again based on how well they recalled it. It adjusts both the review interval and an ease factor that personalizes spacing per card.
- Add the SM-2 algorithm function to sm2.py:
from datetime import date, timedelta
def sm2_review(card: dict, quality_grade: int) -> dict:
"""Apply the SM-2 spaced repetition algorithm to a card.
Args:
card: dict with keys 'interval', 'repetition', 'efactor'
quality_grade: integer 0-5 rating how well the card was recalled
Returns:
Updated dict with new interval, repetition, efactor, due_date
"""
if quality_grade < 0 or quality_grade > 5:
raise ValueError("Quality grade must be between 0 and 5")
interval = card["interval"]
repetition = card["repetition"]
efactor = card["efactor"]
if quality_grade >= 3:
# Correct response: increase interval
if repetition == 0:
interval = 1
elif repetition == 1:
interval = 6
else:
interval = round(interval * efactor)
repetition += 1
else:
# Incorrect response: reset to beginning
repetition = 0
interval = 1
# efactor unchanged on failure
# Update ease factor using SM-2 formula
efactor = efactor + (0.1 - (5 - quality_grade) * (0.08 + (5 - quality_grade) * 0.02))
# Floor at 1.3
if efactor < 1.3:
efactor = 1.3
# Calculate next due date
due_date = (date.today() + timedelta(days=interval)).isoformat()
return {
"interval": interval,
"repetition": repetition,
"efactor": round(efactor, 2),
"due_date": due_date
}How does SM-2 work?
The algorithm rates recall quality on a 0-5 scale. Grades 3-5 mean the student remembered (with varying difficulty). Grades 0-2 mean they forgot, which resets the card to the beginning of the schedule.
For correct responses, the first review sets interval to 1 day, the second to 6 days, and subsequent reviews multiply the previous interval by the ease factor. This creates exponential spacing between reviews.
The ease factor formula EF' = EF + (0.1 - (5-q) * (0.08 + (5-q) * 0.02)) adjusts card difficulty. Easy cards (grade 5) get a higher ease factor. Harder cards get a lower one, floored at 1.3.
- Save sm2.py.
- Now add the review endpoint to revision_router.py. Place this below the existing generation code:
from pydantic import BaseModel, Field
from sm2 import sm2_review
class ReviewRequest(BaseModel):
"""Request body for reviewing a card."""
grade: int = Field(ge=0, le=5, description="Quality grade 0-5")
@router.patch("/review/{card_id}")
async def review_card(card_id: str, review: ReviewRequest):
"""Submit a review grade for a card and update its SM-2 schedule."""
db_path = get_db_path()
async with aiosqlite.connect(db_path) as db:
# Fetch current card state
cursor = await db.execute(
"SELECT interval, repetition, efactor, status FROM revision_cards WHERE id = ?",
(card_id,)
)
row = await cursor.fetchone()
if not row:
raise HTTPException(status_code=404, detail="Card not found")
# Apply SM-2 algorithm
card_state = {
"interval": row[0],
"repetition": row[1],
"efactor": row[2]
}
updated = sm2_review(card_state, review.grade)
# Determine new status based on interval
if updated["interval"] > 21:
new_status = "mastered"
elif updated["repetition"] > 0:
new_status = "review"
else:
new_status = "learning"
# Update the card in the database
await db.execute("""
UPDATE revision_cards
SET interval = ?, repetition = ?, efactor = ?,
due_date = ?, status = ?
WHERE id = ?
""", (
updated["interval"],
updated["repetition"],
updated["efactor"],
updated["due_date"],
new_status,
card_id
))
await db.commit()
return {
"card_id": card_id,
"new_interval": updated["interval"],
"new_efactor": updated["efactor"],
"due_date": updated["due_date"],
"status": new_status
}What does the review endpoint do?
When a student rates their recall (0-5), this endpoint fetches the card's current SM-2 state, runs it through sm2_review(), and saves the updated scheduling values back to the database.
The status transitions work like this: a card with interval greater than 21 days is considered mastered. A card with at least one successful repetition is in review. A reset card enters learning.
- Save revision_router.py.
- Test the review endpoint by sending a PATCH request to /revision/review/{card_id} with body {"grade": 4} in your FastAPI /docs interface. You should see updated interval and efactor values in the response.
Getting a 404 when reviewing?
You need to generate cards first before reviewing them. Call POST /revision/generate/{document_id} with a valid document ID, wait a moment for the background task, then try reviewing one of the created cards.
Help me debug the SM-2 review endpoint returning unexpected values.
Build query endpoints and bulk generation
Students need to see which cards are due today and track their progress. You also need a way to generate cards for an entire folder at once.
- Add the following query endpoints to revision_router.py:
@router.get("/due")
async def get_due_cards():
"""Get all cards due for review today or earlier."""
db_path = get_db_path()
today = date.today().isoformat()
async with aiosqlite.connect(db_path) as db:
db.row_factory = aiosqlite.Row
cursor = await db.execute("""
SELECT id, document_id, folder_id, front, back,
difficulty, tags_json, interval, repetition,
efactor, due_date, status
FROM revision_cards
WHERE due_date <= ?
ORDER BY due_date ASC
""", (today,))
rows = await cursor.fetchall()
return [
{
"id": row["id"],
"document_id": row["document_id"],
"folder_id": row["folder_id"],
"front": row["front"],
"back": row["back"],
"difficulty": row["difficulty"],
"tags": json.loads(row["tags_json"]),
"status": row["status"]
}
for row in rows
]
@router.get("/due/folder/{folder_id}")
async def get_due_cards_by_folder(folder_id: str):
"""Get due cards scoped to a specific folder."""
db_path = get_db_path()
today = date.today().isoformat()
async with aiosqlite.connect(db_path) as db:
db.row_factory = aiosqlite.Row
cursor = await db.execute("""
SELECT id, document_id, folder_id, front, back,
difficulty, tags_json, interval, repetition,
efactor, due_date, status
FROM revision_cards
WHERE due_date <= ? AND folder_id = ?
ORDER BY due_date ASC
""", (today, folder_id))
rows = await cursor.fetchall()
return [
{
"id": row["id"],
"document_id": row["document_id"],
"folder_id": row["folder_id"],
"front": row["front"],
"back": row["back"],
"difficulty": row["difficulty"],
"tags": json.loads(row["tags_json"]),
"status": row["status"]
}
for row in rows
]What do these endpoints return?
Both endpoints filter cards where due_date <= today, returning cards that are ready for review. The folder-scoped version adds a folder_id filter so students can review cards from a specific subject.
- Next, add the stats endpoint and bulk folder generation below the query endpoints:
@router.get("/stats")
async def get_revision_stats():
"""Get overall revision statistics."""
db_path = get_db_path()
today = date.today().isoformat()
async with aiosqlite.connect(db_path) as db:
# Total cards
cursor = await db.execute("SELECT COUNT(*) FROM revision_cards")
total = (await cursor.fetchone())[0]
# Due today
cursor = await db.execute(
"SELECT COUNT(*) FROM revision_cards WHERE due_date <= ?", (today,)
)
due_today = (await cursor.fetchone())[0]
# Mastered count
cursor = await db.execute(
"SELECT COUNT(*) FROM revision_cards WHERE status = 'mastered'"
)
mastered = (await cursor.fetchone())[0]
# Average efactor
cursor = await db.execute(
"SELECT AVG(efactor) FROM revision_cards"
)
avg_efactor_row = await cursor.fetchone()
avg_efactor = round(avg_efactor_row[0], 2) if avg_efactor_row[0] else 2.5
# Current streak: count consecutive days with at least 1 review
cursor = await db.execute("""
SELECT DISTINCT due_date FROM revision_cards
WHERE status != 'new'
ORDER BY due_date DESC
""")
review_dates = [row[0] for row in await cursor.fetchall()]
# Calculate streak from consecutive review dates
streak = 0
check_date = date.today()
for _ in range(len(review_dates)):
if check_date.isoformat() in review_dates:
streak += 1
check_date -= timedelta(days=1)
else:
break
return {
"total_cards": total,
"due_today": due_today,
"mastered": mastered,
"average_efactor": avg_efactor,
"current_streak": streak
}
@router.post("/generate/folder/{folder_id}")
async def generate_cards_for_folder(folder_id: str, background_tasks: BackgroundTasks):
"""Bulk-generate revision cards for all documents in a folder."""
db_path = get_db_path()
async with aiosqlite.connect(db_path) as db:
cursor = await db.execute(
"SELECT id FROM documents WHERE folder_id = ?", (folder_id,)
)
documents = await cursor.fetchall()
if not documents:
raise HTTPException(status_code=404, detail="No documents in folder")
# Schedule generation for each document
for doc in documents:
background_tasks.add_task(
_generate_cards_for_document, doc[0], folder_id
)
return {
"message": f"Card generation started for {len(documents)} documents",
"folder_id": folder_id
}What do stats and bulk generation do?
The stats endpoint returns a snapshot of the student's revision progress: total cards, how many are due, how many are mastered (interval > 21 days), the average ease factor, and a streak of consecutive days with reviews.
The folder generation endpoint finds all documents in a folder and schedules card generation for each one as a background task. This lets students generate cards for an entire course folder in one click.
- Save revision_router.py.
- Restart your FastAPI server and visit /docs. You should see all six revision endpoints listed under the "revision" tag: POST /revision/generate/{document_id}, POST /revision/generate/folder/{folder_id}, PATCH /revision/review/{card_id}, GET /revision/due, GET /revision/due/folder/{folder_id}, and GET /revision/stats.
Stats returning zeros?
The stats endpoint queries existing cards. If you have not generated any cards yet, all counts will be zero. Generate cards for a document first, then review a few to see the stats populate.
My revision stats are showing unexpected values after reviewing cards.
✔️ Awesome, I've got everything!
Great. Double check you have saved both sm2.py and revision_router.py, and that the router is registered in your main app file.
ⓧ I'd like to double check the full code
Here is the complete sm2.py:
from datetime import date, timedelta
def sm2_review(card: dict, quality_grade: int) -> dict:
"""Apply the SM-2 spaced repetition algorithm to a card.
Args:
card: dict with keys 'interval', 'repetition', 'efactor'
quality_grade: integer 0-5 rating how well the card was recalled
Returns:
Updated dict with new interval, repetition, efactor, due_date
"""
if quality_grade < 0 or quality_grade > 5:
raise ValueError("Quality grade must be between 0 and 5")
interval = card["interval"]
repetition = card["repetition"]
efactor = card["efactor"]
if quality_grade >= 3:
# Correct response: increase interval
if repetition == 0:
interval = 1
elif repetition == 1:
interval = 6
else:
interval = round(interval * efactor)
repetition += 1
else:
# Incorrect response: reset to beginning
repetition = 0
interval = 1
# efactor unchanged on failure
# Update ease factor using SM-2 formula
efactor = efactor + (0.1 - (5 - quality_grade) * (0.08 + (5 - quality_grade) * 0.02))
# Floor at 1.3
if efactor < 1.3:
efactor = 1.3
# Calculate next due date
due_date = (date.today() + timedelta(days=interval)).isoformat()
return {
"interval": interval,
"repetition": repetition,
"efactor": round(efactor, 2),
"due_date": due_date
}Here is the complete revision_router.py:
import json
import uuid
from datetime import date, timedelta
from fastapi import APIRouter, HTTPException, BackgroundTasks
from pydantic import BaseModel, Field
import aiosqlite
from groq_client import groq_client
from database import get_db_path
from sm2 import sm2_review
router = APIRouter(prefix="/revision", tags=["revision"])
# Prompt template for flashcard generation
CARD_GENERATION_PROMPT = """Generate revision flashcards from this content.
Return JSON array: [{"front": "question_or_prompt", "back": "answer_or_explanation", "difficulty": "easy|medium|hard", "tags": ["concept_names"]}].
Create 5-8 cards per logical section.
Vary question types: definitions, applications, comparisons.
Content:
{content}"""
class ReviewRequest(BaseModel):
"""Request body for reviewing a card."""
grade: int = Field(ge=0, le=5, description="Quality grade 0-5")
@router.post("/generate/{document_id}")
async def generate_revision_cards(document_id: str, background_tasks: BackgroundTasks):
"""Generate flashcards for a document using Groq LLM."""
db_path = get_db_path()
async with aiosqlite.connect(db_path) as db:
# Get document metadata
cursor = await db.execute(
"SELECT id, folder_id, title FROM documents WHERE id = ?",
(document_id,)
)
doc = await cursor.fetchone()
if not doc:
raise HTTPException(status_code=404, detail="Document not found")
folder_id = doc[1]
# Schedule card generation in the background
background_tasks.add_task(
_generate_cards_for_document, document_id, folder_id
)
return {"message": "Card generation started", "document_id": document_id}
async def _generate_cards_for_document(document_id: str, folder_id: str):
"""Background task: generate and store flashcards."""
db_path = get_db_path()
# Retrieve document text from ChromaDB
from chromadb_client import get_collection
collection = get_collection()
results = collection.get(
where={"document_id": document_id},
include=["documents"]
)
document_text = "\n".join(results["documents"] or [])
if not document_text:
return
# Send to Groq for card generation
prompt = CARD_GENERATION_PROMPT.format(content=document_text)
response = groq_client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=[{"role": "user", "content": prompt}],
temperature=0.7,
response_format={"type": "json_object"}
)
# Parse the generated cards
try:
response_text = response.choices[0].message.content
cards_data = json.loads(response_text)
# Handle both {"cards": [...]} and [...] formats
if isinstance(cards_data, dict):
cards_data = cards_data.get("cards", cards_data.get("flashcards", []))
except (json.JSONDecodeError, IndexError):
return
# Store cards in SQLite
async with aiosqlite.connect(db_path) as db:
for card in cards_data:
card_id = str(uuid.uuid4())
await db.execute("""
INSERT INTO revision_cards
(id, document_id, folder_id, front, back, difficulty, tags_json,
interval, repetition, efactor, due_date, status)
VALUES (?, ?, ?, ?, ?, ?, ?, 0, 0, 2.5, ?, 'new')
""", (
card_id,
document_id,
folder_id,
card.get("front", ""),
card.get("back", ""),
card.get("difficulty", "medium"),
json.dumps(card.get("tags", [])),
date.today().isoformat()
))
await db.commit()
@router.patch("/review/{card_id}")
async def review_card(card_id: str, review: ReviewRequest):
"""Submit a review grade for a card and update its SM-2 schedule."""
db_path = get_db_path()
async with aiosqlite.connect(db_path) as db:
# Fetch current card state
cursor = await db.execute(
"SELECT interval, repetition, efactor, status FROM revision_cards WHERE id = ?",
(card_id,)
)
row = await cursor.fetchone()
if not row:
raise HTTPException(status_code=404, detail="Card not found")
# Apply SM-2 algorithm
card_state = {
"interval": row[0],
"repetition": row[1],
"efactor": row[2]
}
updated = sm2_review(card_state, review.grade)
# Determine new status based on interval
if updated["interval"] > 21:
new_status = "mastered"
elif updated["repetition"] > 0:
new_status = "review"
else:
new_status = "learning"
# Update the card in the database
await db.execute("""
UPDATE revision_cards
SET interval = ?, repetition = ?, efactor = ?,
due_date = ?, status = ?
WHERE id = ?
""", (
updated["interval"],
updated["repetition"],
updated["efactor"],
updated["due_date"],
new_status,
card_id
))
await db.commit()
return {
"card_id": card_id,
"new_interval": updated["interval"],
"new_efactor": updated["efactor"],
"due_date": updated["due_date"],
"status": new_status
}
@router.get("/due")
async def get_due_cards():
"""Get all cards due for review today or earlier."""
db_path = get_db_path()
today = date.today().isoformat()
async with aiosqlite.connect(db_path) as db:
db.row_factory = aiosqlite.Row
cursor = await db.execute("""
SELECT id, document_id, folder_id, front, back,
difficulty, tags_json, interval, repetition,
efactor, due_date, status
FROM revision_cards
WHERE due_date <= ?
ORDER BY due_date ASC
""", (today,))
rows = await cursor.fetchall()
return [
{
"id": row["id"],
"document_id": row["document_id"],
"folder_id": row["folder_id"],
"front": row["front"],
"back": row["back"],
"difficulty": row["difficulty"],
"tags": json.loads(row["tags_json"]),
"status": row["status"]
}
for row in rows
]
@router.get("/due/folder/{folder_id}")
async def get_due_cards_by_folder(folder_id: str):
"""Get due cards scoped to a specific folder."""
db_path = get_db_path()
today = date.today().isoformat()
async with aiosqlite.connect(db_path) as db:
db.row_factory = aiosqlite.Row
cursor = await db.execute("""
SELECT id, document_id, folder_id, front, back,
difficulty, tags_json, interval, repetition,
efactor, due_date, status
FROM revision_cards
WHERE due_date <= ? AND folder_id = ?
ORDER BY due_date ASC
""", (today, folder_id))
rows = await cursor.fetchall()
return [
{
"id": row["id"],
"document_id": row["document_id"],
"folder_id": row["folder_id"],
"front": row["front"],
"back": row["back"],
"difficulty": row["difficulty"],
"tags": json.loads(row["tags_json"]),
"status": row["status"]
}
for row in rows
]
@router.get("/stats")
async def get_revision_stats():
"""Get overall revision statistics."""
db_path = get_db_path()
today = date.today().isoformat()
async with aiosqlite.connect(db_path) as db:
# Total cards
cursor = await db.execute("SELECT COUNT(*) FROM revision_cards")
total = (await cursor.fetchone())[0]
# Due today
cursor = await db.execute(
"SELECT COUNT(*) FROM revision_cards WHERE due_date <= ?", (today,)
)
due_today = (await cursor.fetchone())[0]
# Mastered count
cursor = await db.execute(
"SELECT COUNT(*) FROM revision_cards WHERE status = 'mastered'"
)
mastered = (await cursor.fetchone())[0]
# Average efactor
cursor = await db.execute(
"SELECT AVG(efactor) FROM revision_cards"
)
avg_efactor_row = await cursor.fetchone()
avg_efactor = round(avg_efactor_row[0], 2) if avg_efactor_row[0] else 2.5
# Current streak: count consecutive days with at least 1 review
cursor = await db.execute("""
SELECT DISTINCT due_date FROM revision_cards
WHERE status != 'new'
ORDER BY due_date DESC
""")
review_dates = [row[0] for row in await cursor.fetchall()]
# Calculate streak from consecutive review dates
streak = 0
check_date = date.today()
for _ in range(len(review_dates)):
if check_date.isoformat() in review_dates:
streak += 1
check_date -= timedelta(days=1)
else:
break
return {
"total_cards": total,
"due_today": due_today,
"mastered": mastered,
"average_efactor": avg_efactor,
"current_streak": streak
}
@router.post("/generate/folder/{folder_id}")
async def generate_cards_for_folder(folder_id: str, background_tasks: BackgroundTasks):
"""Bulk-generate revision cards for all documents in a folder."""
db_path = get_db_path()
async with aiosqlite.connect(db_path) as db:
cursor = await db.execute(
"SELECT id FROM documents WHERE folder_id = ?", (folder_id,)
)
documents = await cursor.fetchall()
if not documents:
raise HTTPException(status_code=404, detail="No documents in folder")
# Schedule generation for each document
for doc in documents:
background_tasks.add_task(
_generate_cards_for_document, doc[0], folder_id
)
return {
"message": f"Card generation started for {len(documents)} documents",
"folder_id": folder_id
}Your revision engine is complete. Students can now generate flashcards from any document, review them with intelligent SM-2 scheduling, and track their progress across folders. Next up, you will build the frontend UI for flashcard review and quiz mode.
Build the Revision Cards — UI and Quiz Mode
Your SM-2 engine is running on the backend, scheduling review intervals and tracking card difficulty. Now it is time to give students a way to actually use those revision cards.
In this step, you will build three frontend pages that bring the revision system to life. The Flashcard Review page lets students flip through due cards and rate their recall. Quiz Mode generates multiple-choice questions powered by Groq. The Revision Dashboard ties everything together with progress tracking and study streak data.
In this step, get ready to:
- Build a Flashcard Review page with flip animations, rating buttons, and a session timer.
- Create a Quiz Mode backend endpoint and multiple-choice frontend UI.
- Build a Revision Dashboard with due card counts, mastery stats, streak tracking, and a 7-day bar chart.
Build the Flashcard Review page
Students need a focused interface for reviewing due cards one at a time. The review page fetches cards from your existing GET /revision/due endpoint and presents them in a flashcard format with a flip animation.
- Create a new file at app/revision/review/page.tsx and add the following code:
'use client'
import { useState, useEffect, useRef } from 'react'
// Type definition for a revision card from the backend
interface RevisionCard {
id: string
front: string
back: string
difficulty: string
document_id: string
folder_id: string
}
export default function FlashcardReview() {
const [cards, setCards] = useState<RevisionCard[]>([])
const [currentIndex, setCurrentIndex] = useState(0)
const [isFlipped, setIsFlipped] = useState(false)
const [isSliding, setIsSliding] = useState(false)
const [sessionTime, setSessionTime] = useState(0)
const timerRef = useRef<NodeJS.Timeout | null>(null)
// Fetch due cards on mount
useEffect(() => {
async function fetchDueCards() {
const res = await fetch('/api/revision/due')
const data = await res.json()
setCards(data)
}
fetchDueCards()
}, [])
// Session timer that increments every second
useEffect(() => {
timerRef.current = setInterval(() => {
setSessionTime(prev => prev + 1)
}, 1000)
return () => {
if (timerRef.current) clearInterval(timerRef.current)
}
}, [])
// Format seconds into MM:SS display
const formatTime = (seconds: number) => {
const mins = Math.floor(seconds / 60)
const secs = seconds % 60
return `${mins.toString().padStart(2, '0')}:${secs.toString().padStart(2, '0')}`
}
// Submit rating to backend and advance to next card
const handleRating = async (grade: number) => {
const card = cards[currentIndex]
await fetch(`/api/revision/review/${card.id}`, {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ grade })
})
// Trigger slide-out animation
setIsSliding(true)
setTimeout(() => {
setIsFlipped(false)
setIsSliding(false)
setCurrentIndex(prev => prev + 1)
}, 300)
}
// Show completion message when all cards are reviewed
if (cards.length > 0 && currentIndex >= cards.length) {
return (
<div className="review-complete">
<h2>Session Complete</h2>
<p>You reviewed {cards.length} cards in {formatTime(sessionTime)}.</p>
</div>
)
}
if (cards.length === 0) return <div className="loading">Loading cards...</div>
const currentCard = cards[currentIndex]
return (
<div className="flashcard-review">
{/* Progress indicator and timer */}
<div className="review-header">
<span className="progress">Card {currentIndex + 1} of {cards.length}</span>
<span className="timer">{formatTime(sessionTime)}</span>
</div>
{/* Flashcard with flip animation */}
<div
className={`flashcard ${isFlipped ? 'flipped' : ''} ${isSliding ? 'slide-out' : ''}`}
>
<div className="flashcard-inner">
<div className="flashcard-front">
<p>{currentCard.front}</p>
</div>
<div className="flashcard-back">
<p>{currentCard.back}</p>
</div>
</div>
</div>
{/* Show Answer button or rating buttons */}
{!isFlipped ? (
<button className="show-answer-btn" onClick={() => setIsFlipped(true)}>
Show Answer
</button>
) : (
<div className="rating-buttons">
<button className="rating-again" onClick={() => handleRating(0)}>Again</button>
<button className="rating-hard" onClick={() => handleRating(2)}>Hard</button>
<button className="rating-good" onClick={() => handleRating(3)}>Good</button>
<button className="rating-easy" onClick={() => handleRating(5)}>Easy</button>
</div>
)}
</div>
)
}What does this code do?
This page fetches all due revision cards and displays them one at a time. The isFlipped state controls a CSS flip animation that reveals the answer on the back of the card.
When the student rates their recall (Again, Hard, Good, or Easy), the component sends the grade to your PATCH /revision/review/{card_id} endpoint. The SM-2 algorithm then recalculates interval and ease factor on the backend.
The isSliding state triggers a slide-out CSS animation before advancing to the next card, giving the student smooth visual feedback.
- Now add the CSS for the flip and slide animations. Create app/revision/review/review.css and add the following styles:
/* Flashcard flip animation using CSS transforms */
.flashcard {
width: 100%;
max-width: 500px;
height: 300px;
perspective: 1000px;
margin: 2rem auto;
}
.flashcard-inner {
position: relative;
width: 100%;
height: 100%;
transition: transform 0.6s;
transform-style: preserve-3d;
}
.flashcard.flipped .flashcard-inner {
transform: rotateY(180deg);
}
.flashcard-front,
.flashcard-back {
position: absolute;
width: 100%;
height: 100%;
backface-visibility: hidden;
display: flex;
align-items: center;
justify-content: center;
padding: 2rem;
border-radius: 12px;
background: #1a1a2e;
border: 1px solid #2a2a4a;
}
.flashcard-back {
transform: rotateY(180deg);
}
/* Slide-out animation when advancing cards */
.flashcard.slide-out {
animation: slideOut 0.3s forwards;
}
@keyframes slideOut {
to {
transform: translateX(-100%);
opacity: 0;
}
}
/* Rating buttons with color coding */
.rating-buttons {
display: flex;
gap: 1rem;
justify-content: center;
margin-top: 1rem;
}
.rating-again { background: #ef4444; color: white; }
.rating-hard { background: #f97316; color: white; }
.rating-good { background: #22c55e; color: white; }
.rating-easy { background: #3b82f6; color: white; }How does the flip animation work?
The perspective property on the container creates the 3D space. When the flipped class is applied, rotateY(180deg) spins the inner container to reveal the back face.
The backface-visibility: hidden property ensures only the front side or back side is visible at any given rotation angle.
- Save both app/revision/review/page.tsx and app/revision/review/review.css.
- Navigate to /revision/review in your browser. You should see a card with its front text, a "Show Answer" button, and a timer counting up in the header.
Card not flipping?
Make sure you imported the CSS file at the top of your page component. Check that the class names in your JSX match exactly with those in the CSS file.
If no cards appear at all, confirm that you have due cards in your database by calling GET /revision/due directly.
help me debug my flashcard flip animation
Build Quiz Mode with Groq distractor generation
Flashcard review tests free recall, but multiple-choice quizzes test recognition. Quiz Mode uses Groq to generate three plausible wrong answers (distractors) for each card, creating a challenging multiple-choice experience.
- Add the quiz endpoint to your FastAPI backend. In your revision routes file, add the following endpoint:
@router.post("/revision/quiz/{scope_type}/{scope_id}")
async def generate_quiz(scope_type: str, scope_id: str):
"""Select 10 due/random cards and generate multiple-choice distractors via Groq."""
async with aiosqlite.connect("e-hanatra.db") as db:
db.row_factory = aiosqlite.Row
# Fetch up to 10 due or random cards for the given scope
if scope_type == "document":
cursor = await db.execute(
"SELECT * FROM revision_cards WHERE document_id = ? ORDER BY due_date ASC LIMIT 10",
(scope_id,)
)
elif scope_type == "folder":
cursor = await db.execute(
"SELECT * FROM revision_cards WHERE folder_id = ? ORDER BY due_date ASC LIMIT 10",
(scope_id,)
)
else:
cursor = await db.execute(
"SELECT * FROM revision_cards ORDER BY due_date ASC LIMIT 10"
)
cards = await cursor.fetchall()
# Generate distractors for each card using Groq
quiz_items = []
for card in cards:
# Prompt Groq to create 3 plausible wrong answers
distractor_prompt = (
f"Given this flashcard question: '{card['front']}'\n"
f"The correct answer is: '{card['back']}'\n"
"Generate exactly 3 plausible but incorrect answers. "
"Return ONLY a JSON array of 3 strings, no explanation."
)
response = client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=[{"role": "user", "content": distractor_prompt}],
temperature=0.7
)
# Parse the distractors from Groq response
import json
try:
distractors = json.loads(response.choices[0].message.content)
except json.JSONDecodeError:
distractors = ["Option A", "Option B", "Option C"]
quiz_items.append({
"question": card["front"],
"correct_answer": card["back"],
"distractors": distractors[:3]
})
return quiz_itemsWhat does this endpoint do?
The endpoint selects up to 10 cards scoped to a document, folder, or all cards. For each card, it sends the question and correct answer to Groq, asking it to generate 3 plausible wrong answers.
The temperature=0.7 setting gives Groq enough creativity to produce varied distractors without being too random. The JSON parsing has a fallback in case Groq returns malformed output.
- Save routers/revision.py.
- Test the endpoint by sending a POST request to /revision/quiz/folder/{your-folder-id}. You should receive a JSON array of quiz items, each with a question, correct answer, and 3 distractors.
Getting empty distractors?
Ensure your Groq API key is configured correctly and the client object is initialized at the top of your routes file.
If you see the fallback values ("Option A", "Option B", "Option C"), Groq may be returning text that is not valid JSON. Check the raw response content in your terminal logs.
help me debug my Groq quiz distractor generation
- Now build the Quiz Mode frontend. Create app/revision/quiz/page.tsx with the following code:
'use client'
import { useState, useEffect, useRef } from 'react'
import { useSearchParams } from 'next/navigation'
// Shape of each quiz question from the backend
interface QuizItem {
question: string
correct_answer: string
distractors: string[]
}
export default function QuizMode() {
const searchParams = useSearchParams()
const scopeType = searchParams.get('scope') || 'global'
const scopeId = searchParams.get('id') || ''
const [quizItems, setQuizItems] = useState<QuizItem[]>([])
const [currentIndex, setCurrentIndex] = useState(0)
const [score, setScore] = useState(0)
const [selectedAnswer, setSelectedAnswer] = useState<string | null>(null)
const [showFeedback, setShowFeedback] = useState(false)
const [quizComplete, setQuizComplete] = useState(false)
const [shuffledOptions, setShuffledOptions] = useState<string[]>([])
const [startTime] = useState(Date.now())
const [elapsedTime, setElapsedTime] = useState(0)
// Fetch quiz items from the backend
useEffect(() => {
async function fetchQuiz() {
const res = await fetch(`/api/revision/quiz/${scopeType}/${scopeId}`, {
method: 'POST'
})
const data = await res.json()
setQuizItems(data)
}
fetchQuiz()
}, [scopeType, scopeId])
// Shuffle options whenever the current question changes
useEffect(() => {
if (quizItems.length > 0 && currentIndex < quizItems.length) {
const item = quizItems[currentIndex]
const options = [...item.distractors, item.correct_answer]
// Fisher-Yates shuffle for randomized option order
for (let i = options.length - 1; i > 0; i--) {
const j = Math.floor(Math.random() * (i + 1));
[options[i], options[j]] = [options[j], options[i]]
}
setShuffledOptions(options)
}
}, [currentIndex, quizItems])
// Handle answer selection with instant feedback
const handleAnswer = (answer: string) => {
setSelectedAnswer(answer)
setShowFeedback(true)
if (answer === quizItems[currentIndex].correct_answer) {
setScore(prev => prev + 1)
}
// Advance to next question after a short delay
setTimeout(() => {
setShowFeedback(false)
setSelectedAnswer(null)
if (currentIndex + 1 >= quizItems.length) {
setElapsedTime(Math.floor((Date.now() - startTime) / 1000))
setQuizComplete(true)
} else {
setCurrentIndex(prev => prev + 1)
}
}, 1500)
}
// Get CSS class for answer feedback coloring
const getOptionClass = (option: string) => {
if (!showFeedback) return 'quiz-option'
if (option === quizItems[currentIndex].correct_answer) return 'quiz-option correct'
if (option === selectedAnswer) return 'quiz-option incorrect'
return 'quiz-option'
}
// Show final score screen
if (quizComplete) {
const percentage = Math.round((score / quizItems.length) * 100)
const mins = Math.floor(elapsedTime / 60)
const secs = elapsedTime % 60
return (
<div className="quiz-complete">
<h2>Quiz Complete</h2>
<p className="quiz-score">{score}/{quizItems.length} ({percentage}%)</p>
<p className="quiz-time">Time: {mins}m {secs}s</p>
</div>
)
}
if (quizItems.length === 0) return <div className="loading">Generating quiz...</div>
return (
<div className="quiz-mode">
<div className="quiz-header">
<span>Question {currentIndex + 1} of {quizItems.length}</span>
<span>Score: {score}</span>
</div>
<div className="quiz-question">
<h3>{quizItems[currentIndex].question}</h3>
</div>
<div className="quiz-options">
{shuffledOptions.map((option, idx) => (
<button
key={idx}
className={getOptionClass(option)}
onClick={() => !showFeedback && handleAnswer(option)}
disabled={showFeedback}
>
{option}
</button>
))}
</div>
</div>
)
}How does the quiz feedback work?
When a student selects an answer, showFeedback becomes true. The getOptionClass function then applies green highlighting to the correct answer and red to any incorrect selection.
After 1.5 seconds, the quiz advances to the next question. This delay gives the student time to see which answer was correct before moving on.
- Save app/revision/quiz/page.tsx.
- Navigate to /revision/quiz?scope=folder&id={your-folder-id} in your browser. You should see a question with 4 shuffled options, and selecting one shows instant green or red feedback.
Options not shuffling?
Check that the useEffect that shuffles options has currentIndex and quizItems in its dependency array. Without those dependencies, the shuffle only runs once on mount.
help me debug my quiz mode option shuffling
Build the Revision Dashboard
The Revision Dashboard is the student's home base for spaced repetition. It shows how many cards are due today, tracks mastery progress per folder, counts daily review streaks, and displays a 7-day activity chart.
- Create app/revision/page.tsx as the main Revision Dashboard page:
'use client'
import { useState, useEffect } from 'react'
// Stats returned by the revision stats endpoint
interface RevisionStats {
total_cards: number
due_today: number
mastered_count: number
average_efactor: number
current_streak: number
}
// Folder-level mastery breakdown
interface FolderMastery {
folder_id: string
folder_name: string
total_cards: number
mastered_cards: number
mastery_percentage: number
}
// Daily review count for the 7-day chart
interface DailyReview {
date: string
count: number
}
export default function RevisionDashboard() {
const [stats, setStats] = useState<RevisionStats | null>(null)
const [folderMastery, setFolderMastery] = useState<FolderMastery[]>([])
const [weeklyData, setWeeklyData] = useState<DailyReview[]>([])
// Fetch all dashboard data on mount
useEffect(() => {
async function fetchDashboard() {
const [statsRes, foldersRes, weeklyRes] = await Promise.all([
fetch('/api/revision/stats'),
fetch('/api/revision/mastery-by-folder'),
fetch('/api/revision/weekly-activity')
])
setStats(await statsRes.json())
setFolderMastery(await foldersRes.json())
setWeeklyData(await weeklyRes.json())
}
fetchDashboard()
}, [])
if (!stats) return <div className="loading">Loading dashboard...</div>
// Find the max review count for scaling the bar chart
const maxReviews = Math.max(...weeklyData.map(d => d.count), 1)
return (
<div className="revision-dashboard">
{/* Due today section with Start Review CTA */}
<div className="due-today-card">
<h2>{stats.due_today} cards due today</h2>
<a href="/revision/review" className="start-review-btn">
Start Review
</a>
</div>
{/* Stats overview row */}
<div className="stats-row">
<div className="stat-item">
<span className="stat-value">{stats.current_streak}</span>
<span className="stat-label">Day Streak</span>
</div>
<div className="stat-item">
<span className="stat-value">{stats.mastered_count}</span>
<span className="stat-label">Mastered</span>
</div>
<div className="stat-item">
<span className="stat-value">{stats.total_cards}</span>
<span className="stat-label">Total Cards</span>
</div>
</div>
{/* Folder mastery breakdown with progress bars */}
<div className="folder-mastery">
<h3>Mastery by Folder</h3>
{folderMastery.map(folder => (
<div key={folder.folder_id} className="folder-progress">
<div className="folder-info">
<span>{folder.folder_name}</span>
<span>{folder.mastery_percentage}%</span>
</div>
<div className="progress-bar">
<div
className="progress-fill"
style={{ width: `${folder.mastery_percentage}%` }}
/>
</div>
{/* Generate Cards button per folder */}
<a
href={`/api/revision/generate/folder/${folder.folder_id}`}
className="generate-cards-btn"
>
Generate Cards
</a>
</div>
))}
</div>
{/* 7-day review activity bar chart */}
<div className="weekly-chart">
<h3>Reviews This Week</h3>
<div className="bar-chart">
{weeklyData.map(day => (
<div key={day.date} className="bar-column">
<div
className="bar"
style={{ height: `${(day.count / maxReviews) * 100}%` }}
/>
<span className="bar-label">{day.date.slice(-2)}</span>
</div>
))}
</div>
</div>
</div>
)
}What does the dashboard display?
- The due today card shows the count from GET /revision/stats and links directly to the review page.
- Folder mastery calculates percentage based on cards with an interval greater than 21 days (those are considered mastered).
- The streak counter tracks consecutive days where the student completed at least one review.
- The bar chart renders reviews per day for the last 7 days, scaled relative to the highest day.
- Save app/revision/page.tsx.
- Navigate to /revision in your browser. You should see the due card count with a "Start Review" button, a stats row showing streak and mastered counts, folder progress bars, and a bar chart at the bottom.
Dashboard showing zeros everywhere?
The dashboard relies on having revision cards in your database. Use the POST /revision/generate/{document_id} endpoint from the previous step to generate cards for at least one document first.
If the folder mastery section is empty, check that your cards have valid folder_id values in the revision_cards table.
help me debug my revision dashboard data
✔️ Awesome, I've got everything!
Great. Double check you have saved all your files before moving on.
ⓧ I'd like to double check the full code
Here is the quiz endpoint for your FastAPI backend:
@router.post("/revision/quiz/{scope_type}/{scope_id}")
async def generate_quiz(scope_type: str, scope_id: str):
"""Select 10 due/random cards and generate multiple-choice distractors via Groq."""
async with aiosqlite.connect("e-hanatra.db") as db:
db.row_factory = aiosqlite.Row
# Fetch up to 10 due or random cards for the given scope
if scope_type == "document":
cursor = await db.execute(
"SELECT * FROM revision_cards WHERE document_id = ? ORDER BY due_date ASC LIMIT 10",
(scope_id,)
)
elif scope_type == "folder":
cursor = await db.execute(
"SELECT * FROM revision_cards WHERE folder_id = ? ORDER BY due_date ASC LIMIT 10",
(scope_id,)
)
else:
cursor = await db.execute(
"SELECT * FROM revision_cards ORDER BY due_date ASC LIMIT 10"
)
cards = await cursor.fetchall()
# Generate distractors for each card using Groq
quiz_items = []
for card in cards:
# Prompt Groq to create 3 plausible wrong answers
distractor_prompt = (
f"Given this flashcard question: '{card['front']}'\n"
f"The correct answer is: '{card['back']}'\n"
"Generate exactly 3 plausible but incorrect answers. "
"Return ONLY a JSON array of 3 strings, no explanation."
)
response = client.chat.completions.create(
model="llama-3.3-70b-versatile",
messages=[{"role": "user", "content": distractor_prompt}],
temperature=0.7
)
# Parse the distractors from Groq response
import json
try:
distractors = json.loads(response.choices[0].message.content)
except json.JSONDecodeError:
distractors = ["Option A", "Option B", "Option C"]
quiz_items.append({
"question": card["front"],
"correct_answer": card["back"],
"distractors": distractors[:3]
})
return quiz_itemsHere is the Flashcard Review page:
'use client'
import { useState, useEffect, useRef } from 'react'
// Type definition for a revision card from the backend
interface RevisionCard {
id: string
front: string
back: string
difficulty: string
document_id: string
folder_id: string
}
export default function FlashcardReview() {
const [cards, setCards] = useState<RevisionCard[]>([])
const [currentIndex, setCurrentIndex] = useState(0)
const [isFlipped, setIsFlipped] = useState(false)
const [isSliding, setIsSliding] = useState(false)
const [sessionTime, setSessionTime] = useState(0)
const timerRef = useRef<NodeJS.Timeout | null>(null)
// Fetch due cards on mount
useEffect(() => {
async function fetchDueCards() {
const res = await fetch('/api/revision/due')
const data = await res.json()
setCards(data)
}
fetchDueCards()
}, [])
// Session timer that increments every second
useEffect(() => {
timerRef.current = setInterval(() => {
setSessionTime(prev => prev + 1)
}, 1000)
return () => {
if (timerRef.current) clearInterval(timerRef.current)
}
}, [])
// Format seconds into MM:SS display
const formatTime = (seconds: number) => {
const mins = Math.floor(seconds / 60)
const secs = seconds % 60
return `${mins.toString().padStart(2, '0')}:${secs.toString().padStart(2, '0')}`
}
// Submit rating to backend and advance to next card
const handleRating = async (grade: number) => {
const card = cards[currentIndex]
await fetch(`/api/revision/review/${card.id}`, {
method: 'PATCH',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ grade })
})
// Trigger slide-out animation
setIsSliding(true)
setTimeout(() => {
setIsFlipped(false)
setIsSliding(false)
setCurrentIndex(prev => prev + 1)
}, 300)
}
// Show completion message when all cards are reviewed
if (cards.length > 0 && currentIndex >= cards.length) {
return (
<div className="review-complete">
<h2>Session Complete</h2>
<p>You reviewed {cards.length} cards in {formatTime(sessionTime)}.</p>
</div>
)
}
if (cards.length === 0) return <div className="loading">Loading cards...</div>
const currentCard = cards[currentIndex]
return (
<div className="flashcard-review">
{/* Progress indicator and timer */}
<div className="review-header">
<span className="progress">Card {currentIndex + 1} of {cards.length}</span>
<span className="timer">{formatTime(sessionTime)}</span>
</div>
{/* Flashcard with flip animation */}
<div
className={`flashcard ${isFlipped ? 'flipped' : ''} ${isSliding ? 'slide-out' : ''}`}
>
<div className="flashcard-inner">
<div className="flashcard-front">
<p>{currentCard.front}</p>
</div>
<div className="flashcard-back">
<p>{currentCard.back}</p>
</div>
</div>
</div>
{/* Show Answer button or rating buttons */}
{!isFlipped ? (
<button className="show-answer-btn" onClick={() => setIsFlipped(true)}>
Show Answer
</button>
) : (
<div className="rating-buttons">
<button className="rating-again" onClick={() => handleRating(0)}>Again</button>
<button className="rating-hard" onClick={() => handleRating(2)}>Hard</button>
<button className="rating-good" onClick={() => handleRating(3)}>Good</button>
<button className="rating-easy" onClick={() => handleRating(5)}>Easy</button>
</div>
)}
</div>
)
}Here is the Quiz Mode page:
'use client'
import { useState, useEffect, useRef } from 'react'
import { useSearchParams } from 'next/navigation'
// Shape of each quiz question from the backend
interface QuizItem {
question: string
correct_answer: string
distractors: string[]
}
export default function QuizMode() {
const searchParams = useSearchParams()
const scopeType = searchParams.get('scope') || 'global'
const scopeId = searchParams.get('id') || ''
const [quizItems, setQuizItems] = useState<QuizItem[]>([])
const [currentIndex, setCurrentIndex] = useState(0)
const [score, setScore] = useState(0)
const [selectedAnswer, setSelectedAnswer] = useState<string | null>(null)
const [showFeedback, setShowFeedback] = useState(false)
const [quizComplete, setQuizComplete] = useState(false)
const [shuffledOptions, setShuffledOptions] = useState<string[]>([])
const [startTime] = useState(Date.now())
const [elapsedTime, setElapsedTime] = useState(0)
// Fetch quiz items from the backend
useEffect(() => {
async function fetchQuiz() {
const res = await fetch(`/api/revision/quiz/${scopeType}/${scopeId}`, {
method: 'POST'
})
const data = await res.json()
setQuizItems(data)
}
fetchQuiz()
}, [scopeType, scopeId])
// Shuffle options whenever the current question changes
useEffect(() => {
if (quizItems.length > 0 && currentIndex < quizItems.length) {
const item = quizItems[currentIndex]
const options = [...item.distractors, item.correct_answer]
// Fisher-Yates shuffle for randomized option order
for (let i = options.length - 1; i > 0; i--) {
const j = Math.floor(Math.random() * (i + 1));
[options[i], options[j]] = [options[j], options[i]]
}
setShuffledOptions(options)
}
}, [currentIndex, quizItems])
// Handle answer selection with instant feedback
const handleAnswer = (answer: string) => {
setSelectedAnswer(answer)
setShowFeedback(true)
if (answer === quizItems[currentIndex].correct_answer) {
setScore(prev => prev + 1)
}
// Advance to next question after a short delay
setTimeout(() => {
setShowFeedback(false)
setSelectedAnswer(null)
if (currentIndex + 1 >= quizItems.length) {
setElapsedTime(Math.floor((Date.now() - startTime) / 1000))
setQuizComplete(true)
} else {
setCurrentIndex(prev => prev + 1)
}
}, 1500)
}
// Get CSS class for answer feedback coloring
const getOptionClass = (option: string) => {
if (!showFeedback) return 'quiz-option'
if (option === quizItems[currentIndex].correct_answer) return 'quiz-option correct'
if (option === selectedAnswer) return 'quiz-option incorrect'
return 'quiz-option'
}
// Show final score screen
if (quizComplete) {
const percentage = Math.round((score / quizItems.length) * 100)
const mins = Math.floor(elapsedTime / 60)
const secs = elapsedTime % 60
return (
<div className="quiz-complete">
<h2>Quiz Complete</h2>
<p className="quiz-score">{score}/{quizItems.length} ({percentage}%)</p>
<p className="quiz-time">Time: {mins}m {secs}s</p>
</div>
)
}
if (quizItems.length === 0) return <div className="loading">Generating quiz...</div>
return (
<div className="quiz-mode">
<div className="quiz-header">
<span>Question {currentIndex + 1} of {quizItems.length}</span>
<span>Score: {score}</span>
</div>
<div className="quiz-question">
<h3>{quizItems[currentIndex].question}</h3>
</div>
<div className="quiz-options">
{shuffledOptions.map((option, idx) => (
<button
key={idx}
className={getOptionClass(option)}
onClick={() => !showFeedback && handleAnswer(option)}
disabled={showFeedback}
>
{option}
</button>
))}
</div>
</div>
)
}Here is the Revision Dashboard page:
'use client'
import { useState, useEffect } from 'react'
// Stats returned by the revision stats endpoint
interface RevisionStats {
total_cards: number
due_today: number
mastered_count: number
average_efactor: number
current_streak: number
}
// Folder-level mastery breakdown
interface FolderMastery {
folder_id: string
folder_name: string
total_cards: number
mastered_cards: number
mastery_percentage: number
}
// Daily review count for the 7-day chart
interface DailyReview {
date: string
count: number
}
export default function RevisionDashboard() {
const [stats, setStats] = useState<RevisionStats | null>(null)
const [folderMastery, setFolderMastery] = useState<FolderMastery[]>([])
const [weeklyData, setWeeklyData] = useState<DailyReview[]>([])
// Fetch all dashboard data on mount
useEffect(() => {
async function fetchDashboard() {
const [statsRes, foldersRes, weeklyRes] = await Promise.all([
fetch('/api/revision/stats'),
fetch('/api/revision/mastery-by-folder'),
fetch('/api/revision/weekly-activity')
])
setStats(await statsRes.json())
setFolderMastery(await foldersRes.json())
setWeeklyData(await weeklyRes.json())
}
fetchDashboard()
}, [])
if (!stats) return <div className="loading">Loading dashboard...</div>
// Find the max review count for scaling the bar chart
const maxReviews = Math.max(...weeklyData.map(d => d.count), 1)
return (
<div className="revision-dashboard">
{/* Due today section with Start Review CTA */}
<div className="due-today-card">
<h2>{stats.due_today} cards due today</h2>
<a href="/revision/review" className="start-review-btn">
Start Review
</a>
</div>
{/* Stats overview row */}
<div className="stats-row">
<div className="stat-item">
<span className="stat-value">{stats.current_streak}</span>
<span className="stat-label">Day Streak</span>
</div>
<div className="stat-item">
<span className="stat-value">{stats.mastered_count}</span>
<span className="stat-label">Mastered</span>
</div>
<div className="stat-item">
<span className="stat-value">{stats.total_cards}</span>
<span className="stat-label">Total Cards</span>
</div>
</div>
{/* Folder mastery breakdown with progress bars */}
<div className="folder-mastery">
<h3>Mastery by Folder</h3>
{folderMastery.map(folder => (
<div key={folder.folder_id} className="folder-progress">
<div className="folder-info">
<span>{folder.folder_name}</span>
<span>{folder.mastery_percentage}%</span>
</div>
<div className="progress-bar">
<div
className="progress-fill"
style={{ width: `${folder.mastery_percentage}%` }}
/>
</div>
{/* Generate Cards button per folder */}
<a
href={`/api/revision/generate/folder/${folder.folder_id}`}
className="generate-cards-btn"
>
Generate Cards
</a>
</div>
))}
</div>
{/* 7-day review activity bar chart */}
<div className="weekly-chart">
<h3>Reviews This Week</h3>
<div className="bar-chart">
{weeklyData.map(day => (
<div key={day.date} className="bar-column">
<div
className="bar"
style={{ height: `${(day.count / maxReviews) * 100}%` }}
/>
<span className="bar-label">{day.date.slice(-2)}</span>
</div>
))}
</div>
</div>
</div>
)
}Your revision system is now fully interactive. Students can review flashcards with spaced repetition, test themselves with AI-generated quizzes, and track their progress on the dashboard. Next up, you will add native Malagasy language support for transcription and understanding.
Add Malagasy Language Support
E-Hanatra's core learning features are built and working. But there is a critical gap. Your target users are Malagasy students who think, study, and take notes in Malagasy.
In this step, you will integrate OpenAI Whisper for Malagasy audio transcription, update all Groq prompts to understand and respond in Malagasy, and add full UI localization so students can use the app in their native language.
In this step, get ready to:
- Set up Whisper for Malagasy audio-to-text transcription.
- Update Groq system prompts for bilingual Malagasy and French support.
- Add UI localization with a language toggle and audio language selector.
Integrate Whisper for Malagasy transcription
OpenAI Whisper is an automatic speech recognition model that supports over 90 languages. Malagasy (language code mg) is included in Whisper's supported language list. The "medium" model (769M parameters) offers the best balance of accuracy and speed for non-English languages.
- Install OpenAI Whisper in your FastAPI environment by running this command:
pip install -U openai-whisperWhat does this install?
This installs the openai-whisper package which includes pre-trained speech recognition models. The -U flag ensures you get the latest version with all supported languages.
Installation taking too long?
Whisper has several dependencies including PyTorch. If you already have PyTorch installed, this should be faster. If not, the download may be several gigabytes.
help me troubleshoot my Whisper installation
- Create a new file called transcription.py in your FastAPI backend folder. This file handles the Whisper model loading and the transcription endpoint.
- Add the following code to transcription.py:
import whisper
import os
import uuid
import shutil
from fastapi import APIRouter, UploadFile, File, HTTPException
# Load the Whisper medium model at startup for Malagasy transcription
# Medium (769M params, ~5GB VRAM) is recommended for non-English languages
model = whisper.load_model("medium")
router = APIRouter()
@router.post("/transcribe")
async def transcribe_audio(file: UploadFile = File(...)):
"""Transcribe uploaded audio to Malagasy text using Whisper."""
# Validate file type
allowed_types = ["audio/wav", "audio/mpeg", "audio/mp4", "audio/x-m4a"]
if file.content_type not in allowed_types:
raise HTTPException(
status_code=400,
detail="Unsupported file type. Upload WAV, MP3, or M4A."
)
# Save uploaded file temporarily
temp_path = f"temp_{uuid.uuid4()}_{file.filename}"
try:
with open(temp_path, "wb") as buffer:
shutil.copyfileobj(file.file, buffer)
# Transcribe with Malagasy language specified
result = model.transcribe(temp_path, language="mg")
# Calculate audio duration from Whisper segments
duration = result["segments"][-1]["end"] if result["segments"] else 0.0
return {
"text": result["text"],
"language": "mg",
"duration_seconds": round(duration, 2)
}
finally:
# Clean up temp file
if os.path.exists(temp_path):
os.remove(temp_path)What does this code do?
The model loads once at module import time so it stays in memory between requests. This avoids reloading the 5GB model on every transcription call.
The endpoint saves the uploaded audio to a temp file, passes it to Whisper with language="mg" to force Malagasy recognition, and returns the transcribed text with duration. The temp file is always cleaned up, even if transcription fails.
- Save transcription.py.
- Register this router in your main FastAPI app file by adding the import and including the router.
# In your main app file, add this import
from transcription import router as transcription_router
# Include the transcription router
app.include_router(transcription_router)What about the HuggingFace fallback?
For even better Malagasy accuracy, the badrex/w2v-bert-2.0-malagasy-asr model on HuggingFace is fine-tuned specifically on 150 hours of Malagasy speech with a word error rate below 11.7%. You can integrate this via HuggingFace Transformers as an optional fallback if Whisper's general-purpose recognition is not accurate enough for your use case.
- Test the endpoint by starting your FastAPI server and sending a test audio file. You should receive a JSON response with text, language, and duration_seconds fields.
Seeing a CUDA or memory error?
The medium model requires approximately 5GB of VRAM. If your GPU does not have enough memory, Whisper will fall back to CPU automatically, but transcription will be slower. Ensure no other GPU-intensive processes are running.
help me fix Whisper model loading errors
Update Groq system prompts for Malagasy
Your Groq LLM calls power chat responses, concept extraction, and revision card generation. For these features to work with Malagasy content, you need to update the system prompts across all features.
- Locate the system prompts in your backend where Groq API calls are made (chat endpoints, concept extraction, and card generation).
- Add the following Malagasy instruction to each system prompt:
# Malagasy language instruction to append to all Groq system prompts
MALAGASY_INSTRUCTION = (
"You understand and can process content in Malagasy (Malagasy language). "
"When the input text is in Malagasy, generate responses, concepts, and "
"revision cards in Malagasy. When appropriate, provide bilingual output "
"(Malagasy + French). Always respond in the same language as the user's query."
)What does this prompt instruction do?
This instruction tells Groq's LLM to detect the input language and match it in the response. When a student asks a question in Malagasy, the chat responds in Malagasy. When concept extraction processes a Malagasy document, the concepts come back in Malagasy.
The bilingual clause handles academic content that mixes Malagasy and French (common in Madagascar's education system).
- Update your chat endpoint's system prompt to include the Malagasy instruction:
# Example: updating the chat system prompt
chat_system_prompt = (
"You are an intelligent study assistant. Answer questions based on the "
"provided context from the student's documents. Be clear and educational. "
+ MALAGASY_INSTRUCTION
)- Update your concept extraction prompt similarly:
# Example: updating the concept extraction prompt
extraction_system_prompt = (
"Extract key concepts from this text. Return JSON array: "
"[{name: string, definition: string, category: string, "
"related_to: [concept_names]}]. Categories: theory, definition, "
"formula, example, method. "
+ MALAGASY_INSTRUCTION
)- Update your revision card generation prompt as well:
# Example: updating the revision card generation prompt
card_generation_prompt = (
"Generate revision flashcards from this content. Return JSON array: "
"[{front: question_or_prompt, back: answer_or_explanation, "
"difficulty: easy|medium|hard, tags: [concept_names]}]. "
"Create 5-8 cards per logical section. Vary question types: "
"definitions, applications, comparisons. "
+ MALAGASY_INSTRUCTION
)Why not a separate Malagasy prompt?
Appending the instruction to existing prompts means the LLM handles language detection automatically. You do not need separate code paths for Malagasy versus French versus English. The same endpoints work for all three languages based on the input content.
- Save all modified prompt files.
- Test by sending a Malagasy text to your chat endpoint. The response should come back in Malagasy.
Groq responding in English despite Malagasy input?
Make sure the MALAGASY_INSTRUCTION string is concatenated to the system prompt, not the user message. System prompts set the behavior. Also verify your Groq model supports multilingual output. Llama-based models handle Malagasy with explicit system instructions.
help me debug why Groq is not responding in Malagasy
Add UI localization and language toggle
The backend now speaks Malagasy. The frontend needs to match. You will create locale files with translated UI strings and a language toggle that lets students switch between Malagasy, French, and English.
- Create a messages folder in your Next.js project root for locale files.
- Create messages/mg.json with Malagasy translations:
{
"nav": {
"documents": "Antontan-taratasy",
"chat": "Resaka",
"knowledgeMap": "Sarintany fahalalana",
"revision": "Famerenan-desona",
"dashboard": "Tabilao"
},
"actions": {
"upload": "Hampiditra rakitra",
"newFolder": "Lahatahiry vaovao",
"startReview": "Hanomboka famerenan-desona",
"generateCards": "Hamorona karatra",
"generateMap": "Hamorona sarintany"
},
"placeholders": {
"searchDocuments": "Hitady antontan-taratasy...",
"chatInput": "Manontania zavatra...",
"folderName": "Anaran'ny lahatahiry"
},
"labels": {
"dueToday": "Tokony haverina androany",
"mastered": "Efa hainao",
"streak": "Fitohy",
"language": "Fiteny"
}
}- Create messages/fr.json with French translations:
{
"nav": {
"documents": "Documents",
"chat": "Discussion",
"knowledgeMap": "Carte de connaissances",
"revision": "Révision",
"dashboard": "Tableau de bord"
},
"actions": {
"upload": "Télécharger un fichier",
"newFolder": "Nouveau dossier",
"startReview": "Commencer la révision",
"generateCards": "Générer des cartes",
"generateMap": "Générer la carte"
},
"placeholders": {
"searchDocuments": "Rechercher des documents...",
"chatInput": "Posez une question...",
"folderName": "Nom du dossier"
},
"labels": {
"dueToday": "À réviser aujourd'hui",
"mastered": "Maîtrisé",
"streak": "Série",
"language": "Langue"
}
}What are these locale files?
Each JSON file maps UI string keys to their translation. Your components reference keys like nav.documents instead of hardcoded text. Switching the active locale swaps all visible strings instantly.
- Now build the LanguageToggle component. Create components/LanguageToggle.tsx:
'use client';
import { useState, useEffect } from 'react';
// Supported languages for the UI
const LANGUAGES = [
{ code: 'mg', label: 'Malagasy' },
{ code: 'fr', label: 'Français' },
{ code: 'en', label: 'English' },
];
export default function LanguageToggle() {
// Initialize from localStorage or default to Malagasy
const [locale, setLocale] = useState('mg');
useEffect(() => {
const saved = localStorage.getItem('e-hanatra-locale');
if (saved) setLocale(saved);
}, []);
const handleChange = (newLocale: string) => {
setLocale(newLocale);
// Persist to localStorage so preference survives page reloads
localStorage.setItem('e-hanatra-locale', newLocale);
// Trigger app-wide locale update
window.dispatchEvent(new CustomEvent('locale-change', { detail: newLocale }));
};
return (
<div className="language-toggle">
{LANGUAGES.map((lang) => (
<button
key={lang.code}
onClick={() => handleChange(lang.code)}
className={locale === lang.code ? 'active' : ''}
>
{lang.label}
</button>
))}
</div>
);
}Why localStorage for language preference?
Persisting to localStorage means the student's language choice survives browser refreshes and returns. The custom event lets other components react to the change without a full page reload.
- Add the LanguageToggle to your app header. In your header component, import and render it:
// In your app header component, add the language toggle
import LanguageToggle from '@/components/LanguageToggle';
// Inside the header JSX, add alongside other header items
<LanguageToggle />- Finally, update the audio recording feature to include language selection before transcription. Create components/AudioLanguageSelector.tsx:
'use client';
import { useState } from 'react';
// Language options for audio transcription
const TRANSCRIPTION_LANGUAGES = [
{ code: 'mg', label: 'Malagasy' },
{ code: 'fr', label: 'Français' },
{ code: 'en', label: 'English' },
];
interface AudioLanguageSelectorProps {
onLanguageSelect: (code: string) => void;
selectedLanguage: string;
}
export default function AudioLanguageSelector({
onLanguageSelect,
selectedLanguage,
}: AudioLanguageSelectorProps) {
return (
<div className="audio-language-selector">
<label>Transcription language:</label>
<select
value={selectedLanguage}
onChange={(e) => onLanguageSelect(e.target.value)}
>
{TRANSCRIPTION_LANGUAGES.map((lang) => (
<option key={lang.code} value={lang.code}>
{lang.label}
</option>
))}
</select>
</div>
);
}What does this selector do?
This component appears above the audio recording button. The student picks their transcription language before recording. The selected language code is then sent to the POST /transcribe endpoint so Whisper uses the correct language model.
- Save all new component files.
- Refresh your browser. You should see the language toggle in the app header with three buttons (Malagasy, Français, English).
- Click each language button and verify the selection persists after refreshing the page.
Language toggle not persisting after refresh?
Open your browser DevTools and check Application > Local Storage. Look for the key e-hanatra-locale. If it is missing, make sure the handleChange function is being called on button click.
help me debug my language toggle localStorage persistence
✔️ Awesome, I've got everything!
Great. Double check you have saved all files: transcription.py, your updated prompt files, messages/mg.json, messages/fr.json, components/LanguageToggle.tsx, and components/AudioLanguageSelector.tsx.
ⓧ I'd like to double check the full code
Here is the full transcription endpoint:
import whisper
import os
import uuid
import shutil
from fastapi import APIRouter, UploadFile, File, HTTPException
# Load the Whisper medium model at startup for Malagasy transcription
# Medium (769M params, ~5GB VRAM) is recommended for non-English languages
model = whisper.load_model("medium")
router = APIRouter()
@router.post("/transcribe")
async def transcribe_audio(file: UploadFile = File(...)):
"""Transcribe uploaded audio to Malagasy text using Whisper."""
# Validate file type
allowed_types = ["audio/wav", "audio/mpeg", "audio/mp4", "audio/x-m4a"]
if file.content_type not in allowed_types:
raise HTTPException(
status_code=400,
detail="Unsupported file type. Upload WAV, MP3, or M4A."
)
# Save uploaded file temporarily
temp_path = f"temp_{uuid.uuid4()}_{file.filename}"
try:
with open(temp_path, "wb") as buffer:
shutil.copyfileobj(file.file, buffer)
# Transcribe with Malagasy language specified
result = model.transcribe(temp_path, language="mg")
# Calculate audio duration from Whisper segments
duration = result["segments"][-1]["end"] if result["segments"] else 0.0
return {
"text": result["text"],
"language": "mg",
"duration_seconds": round(duration, 2)
}
finally:
# Clean up temp file
if os.path.exists(temp_path):
os.remove(temp_path)Here is the Malagasy prompt instruction:
# Malagasy language instruction to append to all Groq system prompts
MALAGASY_INSTRUCTION = (
"You understand and can process content in Malagasy (Malagasy language). "
"When the input text is in Malagasy, generate responses, concepts, and "
"revision cards in Malagasy. When appropriate, provide bilingual output "
"(Malagasy + French). Always respond in the same language as the user's query."
)Here is the LanguageToggle component:
'use client';
import { useState, useEffect } from 'react';
// Supported languages for the UI
const LANGUAGES = [
{ code: 'mg', label: 'Malagasy' },
{ code: 'fr', label: 'Français' },
{ code: 'en', label: 'English' },
];
export default function LanguageToggle() {
// Initialize from localStorage or default to Malagasy
const [locale, setLocale] = useState('mg');
useEffect(() => {
const saved = localStorage.getItem('e-hanatra-locale');
if (saved) setLocale(saved);
}, []);
const handleChange = (newLocale: string) => {
setLocale(newLocale);
// Persist to localStorage so preference survives page reloads
localStorage.setItem('e-hanatra-locale', newLocale);
// Trigger app-wide locale update
window.dispatchEvent(new CustomEvent('locale-change', { detail: newLocale }));
};
return (
<div className="language-toggle">
{LANGUAGES.map((lang) => (
<button
key={lang.code}
onClick={() => handleChange(lang.code)}
className={locale === lang.code ? 'active' : ''}
>
{lang.label}
</button>
))}
</div>
);
}Here is the AudioLanguageSelector component:
'use client';
import { useState } from 'react';
// Language options for audio transcription
const TRANSCRIPTION_LANGUAGES = [
{ code: 'mg', label: 'Malagasy' },
{ code: 'fr', label: 'Français' },
{ code: 'en', label: 'English' },
];
interface AudioLanguageSelectorProps {
onLanguageSelect: (code: string) => void;
selectedLanguage: string;
}
export default function AudioLanguageSelector({
onLanguageSelect,
selectedLanguage,
}: AudioLanguageSelectorProps) {
return (
<div className="audio-language-selector">
<label>Transcription language:</label>
<select
value={selectedLanguage}
onChange={(e) => onLanguageSelect(e.target.value)}
>
{TRANSCRIPTION_LANGUAGES.map((lang) => (
<option key={lang.code} value={lang.code}>
{lang.label}
</option>
))}
</select>
</div>
);
}Your app now speaks Malagasy. Students can transcribe audio in their native language, get AI responses in Malagasy, and navigate the UI without switching to French or English. Next up, you will make this a full PWA so students can use E-Hanatra offline on their phones.
Set Up PWA and Offline Support
Your E-Hanatra app now supports Malagasy language across all features. Students can transcribe audio, chat, and generate revision cards in their native language.
The next challenge is accessibility. Students in Madagascar often have unreliable internet connections. A Progressive Web App turns your web app into something that feels native on mobile devices. It can be installed to the home screen, works offline, and caches critical content so students can review even without connectivity.
In this step, get ready to:
- Install Serwist and configure the service worker for offline fallback.
- Create a PWA manifest and integrate SerwistProvider into your layout.
- Implement smart caching for documents, chats, and offline revision ratings.
Install Serwist and create the service worker
Serwist is the maintained successor to next-pwa. It builds the service worker after Next.js finishes prerendering, so it can automatically precache all your routes. This approach is also bundler-agnostic, working with both webpack and Turbopack.
- Install the Serwist packages as dev dependencies by running this command in your Next.js project root:
npm i -D @serwist/next @serwist/cli serwist esbuild concurrentlyWhat are these packages?
- @serwist/next generates the CLI configuration needed to build the service worker for Next.js.
- @serwist/cli runs the actual service worker build step after Next.js prerendering completes.
- serwist provides the core service worker runtime (caching strategies, precaching, fallbacks).
- esbuild compiles your TypeScript service worker into JavaScript.
- concurrently lets you run the Serwist watcher alongside the Next.js dev server.
Installation failing?
Make sure you are running the command from the same directory as your package.json. Check that your Node version is 18 or higher, as Serwist requires it.
help me troubleshoot this Serwist installation error
- Create a serwist.config.js file in your project root (next to package.json) with the following configuration:
// @ts-check
import { serwist } from "@serwist/next/config";
// Configure Serwist to build the service worker
// swSrc points to your TypeScript service worker source
// swDest is where the compiled JS output goes
export default serwist({
swSrc: "app/sw.ts",
swDest: "public/sw.js",
});What does this config do?
This tells Serwist where to find your service worker source (app/sw.ts) and where to output the compiled version (public/sw.js). The @serwist/next/config helper generates the full CLI config needed by @serwist/cli under the hood.
- Create the service worker file at app/sw.ts with the following code:
/// <reference lib="esnext" />
/// <reference lib="webworker" />
import { defaultCache } from "@serwist/next/worker";
import type { PrecacheEntry, SerwistGlobalConfig } from "serwist";
import { Serwist } from "serwist";
// Declare the injection point for the precache manifest
// Serwist replaces self.__SW_MANIFEST with the actual entries at build time
declare global {
interface WorkerGlobalScope extends SerwistGlobalConfig {
__SW_MANIFEST: (PrecacheEntry | string)[] | undefined;
}
}
declare const self: ServiceWorkerGlobalScope;
// Instantiate Serwist with caching and offline fallback
const serwist = new Serwist({
precacheEntries: self.__SW_MANIFEST,
skipWaiting: true,
clientsClaim: true,
navigationPreload: true,
runtimeCaching: defaultCache,
fallbacks: {
entries: [
{
url: "/~offline",
matcher({ request }) {
return request.destination === "document";
},
},
],
},
});
serwist.addEventListeners();What does this service worker do?
- precacheEntries caches all prerendered pages at build time so they load instantly.
- skipWaiting and clientsClaim ensure the new service worker takes control immediately without waiting for a page refresh.
- navigationPreload speeds up navigation by starting network requests in parallel with service worker boot.
- runtimeCaching: defaultCache applies sensible caching strategies for static assets, images, and API calls.
- The fallbacks entry redirects to /~offline when the user navigates to a page without connectivity.
- Update your package.json build script to include the Serwist build step. Find this line in the scripts section:
"build": "next build"- Replace it with this updated build script that runs Serwist after Next.js finishes:
"build": "next build && serwist build"- Also add a dev script that runs both concurrently during development:
"dev": "concurrently -p none 'serwist build --watch' 'next dev'"- Add public/sw* to your .gitignore file so the compiled service worker output is not committed to version control:
# Serwist
public/sw*- Save serwist.config.js, app/sw.ts, package.json, and .gitignore.
- Confirm the files exist by running:
ls serwist.config.js app/sw.tsYou should see both files listed in the output.
File not found?
Double-check that serwist.config.js is in your project root (same level as package.json) and sw.ts is inside the app/ directory.
help me figure out where my Serwist config files should go
Create the PWA manifest and offline page
A web app manifest tells the browser how your app should behave when installed on a device. It defines the app name, colors, icons, and display mode. Next.js supports generating this manifest programmatically with a manifest.ts file.
- First, create the icon files. Make a public/icons directory:
mkdir -p public/icons- Add two PNG icon files: a 192x192 pixel icon and a 512x512 pixel icon inside public/icons/. Name them icon-192x192.png and icon-512x512.png. You can create simple placeholder icons or use your E-Hanatra logo.
Quick placeholder icons
If you do not have icons ready, create solid-color PNG files at the required sizes using any image editor or an online tool. You can replace them with proper branding later.
- Create app/manifest.ts with the following code:
import type { MetadataRoute } from "next";
// Define the PWA manifest for E-Hanatra
export default function manifest(): MetadataRoute.Manifest {
return {
name: "E-Hanatra",
short_name: "E-Hanatra",
description: "Votre Second Brain intelligent",
start_url: "/",
display: "standalone",
background_color: "#0f0f0f",
theme_color: "#6366f1",
icons: [
{
src: "/icons/icon-192x192.png",
sizes: "192x192",
type: "image/png",
purpose: "maskable",
},
{
src: "/icons/icon-512x512.png",
sizes: "512x512",
type: "image/png",
},
],
};
}What do these manifest fields mean?
- display: "standalone" makes the app run in its own window without browser chrome (address bar, tabs).
- background_color: "#0f0f0f" matches E-Hanatra's dark theme so the splash screen blends in.
- theme_color: "#6366f1" sets the indigo accent for the status bar on mobile devices.
- The 192x192 icon with purpose: "maskable" lets Android crop it into different shapes (circle, squircle). The 512x512 icon is for splash screens.
- Now add SerwistProvider to your root layout. Open app/layout.tsx and add the import at the top of your imports:
import { SerwistProvider } from "@serwist/next/react";- Wrap your layout's children with SerwistProvider inside the <body> tag, passing the service worker URL:
<body>
<SerwistProvider swUrl="/sw.js">
{/* Your existing layout content stays here */}
{children}
</SerwistProvider>
</body>What does SerwistProvider do?
SerwistProvider registers the compiled service worker (/sw.js) with the browser when the page loads. It handles the registration lifecycle for you, including waiting for the window load event.
- Create the offline fallback page. Make a new directory app/~offline/ and create a page.tsx file inside it:
mkdir -p "app/~offline"- Add the offline page content in app/~offline/page.tsx:
"use client";
import { useEffect, useState } from "react";
// Offline fallback page shown when user has no connectivity
export default function OfflinePage() {
const [cachedDocs, setCachedDocs] = useState<string[]>([]);
useEffect(() => {
// Retrieve cached document titles from localStorage
const stored = localStorage.getItem("cached_document_titles");
if (stored) {
setCachedDocs(JSON.parse(stored));
}
}, []);
return (
<div className="flex flex-col items-center justify-center min-h-screen bg-[#0f0f0f] text-white p-6">
<h1 className="text-2xl font-bold mb-4">Vous etes hors ligne</h1>
<p className="text-gray-400 mb-8 text-center">
Vous n'avez pas de connexion internet. Voici vos documents en cache.
</p>
{cachedDocs.length > 0 ? (
<ul className="space-y-2 w-full max-w-md">
{cachedDocs.map((title, index) => (
<li
key={index}
className="bg-gray-800 rounded-lg p-3 text-sm"
>
{title}
</li>
))}
</ul>
) : (
<p className="text-gray-500">
Aucun document en cache disponible.
</p>
)}
</div>
);
}Why the tilde in the route name?
The ~offline route uses a tilde prefix to avoid conflicting with normal app routes. The service worker's fallback config redirects here whenever a navigation request fails due to no connectivity.
- Save app/manifest.ts, app/layout.tsx, and app/~offline/page.tsx.
- Run the build to verify everything compiles:
npm run buildYou should see the Next.js build complete, followed by Serwist generating public/sw.js. The output will mention precached entries.
Build failing with Serwist error?
Make sure serwist.config.js uses ES module syntax (import/export). If your project uses CommonJS, either add "type": "module" to package.json or rename the config to serwist.config.mjs.
help me fix my Serwist build configuration
Implement smart caching and install prompt
The base service worker handles precaching and offline fallback. Now you need smarter caching that targets what students actually need offline: document previews, recent chat conversations, and queued revision card ratings.
- Create a caching utilities file at app/lib/cache-utils.ts that manages document and chat caching:
// Smart caching utilities for offline access
const DOCUMENT_CACHE = "document-previews-v1";
const CHAT_CACHE = "chat-conversations-v1";
const MAX_CACHED_CHATS = 5;
// Cache a document preview when the user first views it
export async function cacheDocumentPreview(
documentId: string,
title: string,
content: string
) {
const cache = await caches.open(DOCUMENT_CACHE);
const response = new Response(JSON.stringify({ title, content }), {
headers: { "Content-Type": "application/json" },
});
await cache.put(`/api/documents/${documentId}/preview`, response);
// Store title in localStorage for the offline page listing
const stored = JSON.parse(
localStorage.getItem("cached_document_titles") || "[]"
);
if (!stored.includes(title)) {
stored.push(title);
localStorage.setItem("cached_document_titles", JSON.stringify(stored));
}
}
// Cache a chat conversation (keep only the last 5)
export async function cacheChatConversation(
conversationId: string,
messages: unknown[]
) {
const cache = await caches.open(CHAT_CACHE);
// Get existing cached conversation keys
const keys = await cache.keys();
// If at the limit, remove the oldest cached conversation
if (keys.length >= MAX_CACHED_CHATS) {
await cache.delete(keys[0]);
}
const response = new Response(JSON.stringify(messages), {
headers: { "Content-Type": "application/json" },
});
await cache.put(`/api/chat/${conversationId}`, response);
}
// Retrieve a cached document preview
export async function getCachedDocument(documentId: string) {
const cache = await caches.open(DOCUMENT_CACHE);
const response = await cache.match(`/api/documents/${documentId}/preview`);
if (response) {
return response.json();
}
return null;
}
// Retrieve a cached chat conversation
export async function getCachedChat(conversationId: string) {
const cache = await caches.open(CHAT_CACHE);
const response = await cache.match(`/api/chat/${conversationId}`);
if (response) {
return response.json();
}
return null;
}What does this caching strategy achieve?
Document previews are cached on first view using the Cache Storage API. This means any document the student has opened at least once becomes available offline.
Chat conversations are capped at 5 entries. When the student opens a sixth conversation, the oldest one is evicted. This keeps storage usage reasonable on mobile devices while preserving the most recent study context.
- Create a Background Sync utility at app/lib/offline-sync.ts that queues revision card ratings made while offline:
// Background Sync for offline revision card ratings
const PENDING_REVIEWS_KEY = "pending_offline_reviews";
interface PendingReview {
cardId: string;
grade: number;
timestamp: number;
}
// Queue a review rating when offline
export function queueOfflineReview(cardId: string, grade: number) {
const pending: PendingReview[] = JSON.parse(
localStorage.getItem(PENDING_REVIEWS_KEY) || "[]"
);
pending.push({ cardId, grade, timestamp: Date.now() });
localStorage.setItem(PENDING_REVIEWS_KEY, JSON.stringify(pending));
// Request a Background Sync if the API is available
if ("serviceWorker" in navigator && "SyncManager" in window) {
navigator.serviceWorker.ready.then((registration) => {
(registration as any).sync.register("sync-reviews");
});
}
}
// Process all queued reviews when back online
export async function syncPendingReviews() {
const pending: PendingReview[] = JSON.parse(
localStorage.getItem(PENDING_REVIEWS_KEY) || "[]"
);
if (pending.length === 0) return;
const results = await Promise.allSettled(
pending.map((review) =>
fetch(`/api/revision/review/${review.cardId}`, {
method: "PATCH",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ grade: review.grade }),
})
)
);
// Remove successfully synced reviews from the queue
const remaining = pending.filter(
(_, index) => results[index].status === "rejected"
);
localStorage.setItem(PENDING_REVIEWS_KEY, JSON.stringify(remaining));
}
// Listen for online event to trigger sync
export function registerOnlineSync() {
window.addEventListener("online", () => {
syncPendingReviews();
});
}How does Background Sync work here?
When a student rates a revision card without internet, the rating is saved to localStorage. If the browser supports Background Sync, it registers a sync event. The browser then retries the request automatically when connectivity returns. As a fallback for browsers without Background Sync support, the online event listener triggers the sync manually.
- Create the install prompt banner component at app/components/InstallPrompt.tsx:
"use client";
import { useEffect, useState } from "react";
// PWA install prompt banner for mobile users
export default function InstallPrompt() {
const [deferredPrompt, setDeferredPrompt] = useState<any>(null);
const [showBanner, setShowBanner] = useState(false);
useEffect(() => {
// Track visit count to show prompt after 3+ visits
const visits = parseInt(localStorage.getItem("visit_count") || "0", 10) + 1;
localStorage.setItem("visit_count", String(visits));
// Only show the install banner after 3 visits
const handler = (e: Event) => {
e.preventDefault();
setDeferredPrompt(e);
if (visits >= 3) {
setShowBanner(true);
}
};
window.addEventListener("beforeinstallprompt", handler);
return () => window.removeEventListener("beforeinstallprompt", handler);
}, []);
const handleInstall = async () => {
if (!deferredPrompt) return;
deferredPrompt.prompt();
const { outcome } = await deferredPrompt.userChoice;
if (outcome === "accepted") {
setShowBanner(false);
}
setDeferredPrompt(null);
};
if (!showBanner) return null;
return (
<div className="fixed bottom-4 left-4 right-4 bg-indigo-600 text-white rounded-lg p-4 flex items-center justify-between shadow-lg z-50">
<div>
<p className="font-semibold">Installer E-Hanatra</p>
<p className="text-sm text-indigo-200">
Accedez a vos cours hors ligne.
</p>
</div>
<div className="flex gap-2">
<button
onClick={() => setShowBanner(false)}
className="text-sm text-indigo-200 hover:text-white"
>
Plus tard
</button>
<button
onClick={handleInstall}
className="bg-white text-indigo-600 px-4 py-2 rounded-md text-sm font-medium"
>
Installer
</button>
</div>
</div>
);
}Why wait for 3 visits?
Showing an install prompt on the first visit feels aggressive and leads to dismissals. By waiting until the student has returned at least three times, you are reaching engaged users who would genuinely benefit from the installed app experience.
- Add the InstallPrompt component to your root layout. In app/layout.tsx, import and place it inside the SerwistProvider:
import InstallPrompt from "./components/InstallPrompt";
// Inside your body/SerwistProvider:
<SerwistProvider swUrl="/sw.js">
{children}
<InstallPrompt />
</SerwistProvider>- Save all files: app/lib/cache-utils.ts, app/lib/offline-sync.ts, app/components/InstallPrompt.tsx, and app/layout.tsx.
- Run the build again to confirm everything compiles cleanly:
npm run buildYou should see a successful build with no TypeScript errors. The public/sw.js file should now include your /~offline route in its precache manifest.
Testing PWA on mobile
To test the install prompt and offline behavior, deploy to a server with HTTPS (service workers require a secure context). On Android, use Chrome DevTools remote debugging. On iOS, add to home screen from Safari 16.4 or later.
Service worker not registering?
Service workers only work over HTTPS (except on localhost). In development, use npm run dev which serves on localhost where service workers are allowed. Check the browser's Application panel in DevTools to see registration status.
help me debug my service worker registration
✔️ Awesome, I've got everything!
Great. Double check you have saved all files and your build completes without errors.
ⓧ I'd like to double check the full code
Here are the complete files you should have at this point:
// @ts-check
import { serwist } from "@serwist/next/config";
// Configure Serwist to build the service worker
// swSrc points to your TypeScript service worker source
// swDest is where the compiled JS output goes
export default serwist({
swSrc: "app/sw.ts",
swDest: "public/sw.js",
});/// <reference lib="esnext" />
/// <reference lib="webworker" />
import { defaultCache } from "@serwist/next/worker";
import type { PrecacheEntry, SerwistGlobalConfig } from "serwist";
import { Serwist } from "serwist";
// Declare the injection point for the precache manifest
// Serwist replaces self.__SW_MANIFEST with the actual entries at build time
declare global {
interface WorkerGlobalScope extends SerwistGlobalConfig {
__SW_MANIFEST: (PrecacheEntry | string)[] | undefined;
}
}
declare const self: ServiceWorkerGlobalScope;
// Instantiate Serwist with caching and offline fallback
const serwist = new Serwist({
precacheEntries: self.__SW_MANIFEST,
skipWaiting: true,
clientsClaim: true,
navigationPreload: true,
runtimeCaching: defaultCache,
fallbacks: {
entries: [
{
url: "/~offline",
matcher({ request }) {
return request.destination === "document";
},
},
],
},
});
serwist.addEventListeners();import type { MetadataRoute } from "next";
// Define the PWA manifest for E-Hanatra
export default function manifest(): MetadataRoute.Manifest {
return {
name: "E-Hanatra",
short_name: "E-Hanatra",
description: "Votre Second Brain intelligent",
start_url: "/",
display: "standalone",
background_color: "#0f0f0f",
theme_color: "#6366f1",
icons: [
{
src: "/icons/icon-192x192.png",
sizes: "192x192",
type: "image/png",
purpose: "maskable",
},
{
src: "/icons/icon-512x512.png",
sizes: "512x512",
type: "image/png",
},
],
};
}"use client";
import { useEffect, useState } from "react";
// Offline fallback page shown when user has no connectivity
export default function OfflinePage() {
const [cachedDocs, setCachedDocs] = useState<string[]>([]);
useEffect(() => {
// Retrieve cached document titles from localStorage
const stored = localStorage.getItem("cached_document_titles");
if (stored) {
setCachedDocs(JSON.parse(stored));
}
}, []);
return (
<div className="flex flex-col items-center justify-center min-h-screen bg-[#0f0f0f] text-white p-6">
<h1 className="text-2xl font-bold mb-4">Vous etes hors ligne</h1>
<p className="text-gray-400 mb-8 text-center">
Vous n'avez pas de connexion internet. Voici vos documents en cache.
</p>
{cachedDocs.length > 0 ? (
<ul className="space-y-2 w-full max-w-md">
{cachedDocs.map((title, index) => (
<li
key={index}
className="bg-gray-800 rounded-lg p-3 text-sm"
>
{title}
</li>
))}
</ul>
) : (
<p className="text-gray-500">
Aucun document en cache disponible.
</p>
)}
</div>
);
}// Smart caching utilities for offline access
const DOCUMENT_CACHE = "document-previews-v1";
const CHAT_CACHE = "chat-conversations-v1";
const MAX_CACHED_CHATS = 5;
// Cache a document preview when the user first views it
export async function cacheDocumentPreview(
documentId: string,
title: string,
content: string
) {
const cache = await caches.open(DOCUMENT_CACHE);
const response = new Response(JSON.stringify({ title, content }), {
headers: { "Content-Type": "application/json" },
});
await cache.put(`/api/documents/${documentId}/preview`, response);
// Store title in localStorage for the offline page listing
const stored = JSON.parse(
localStorage.getItem("cached_document_titles") || "[]"
);
if (!stored.includes(title)) {
stored.push(title);
localStorage.setItem("cached_document_titles", JSON.stringify(stored));
}
}
// Cache a chat conversation (keep only the last 5)
export async function cacheChatConversation(
conversationId: string,
messages: unknown[]
) {
const cache = await caches.open(CHAT_CACHE);
// Get existing cached conversation keys
const keys = await cache.keys();
// If at the limit, remove the oldest cached conversation
if (keys.length >= MAX_CACHED_CHATS) {
await cache.delete(keys[0]);
}
const response = new Response(JSON.stringify(messages), {
headers: { "Content-Type": "application/json" },
});
await cache.put(`/api/chat/${conversationId}`, response);
}
// Retrieve a cached document preview
export async function getCachedDocument(documentId: string) {
const cache = await caches.open(DOCUMENT_CACHE);
const response = await cache.match(`/api/documents/${documentId}/preview`);
if (response) {
return response.json();
}
return null;
}
// Retrieve a cached chat conversation
export async function getCachedChat(conversationId: string) {
const cache = await caches.open(CHAT_CACHE);
const response = await cache.match(`/api/chat/${conversationId}`);
if (response) {
return response.json();
}
return null;
}// Background Sync for offline revision card ratings
const PENDING_REVIEWS_KEY = "pending_offline_reviews";
interface PendingReview {
cardId: string;
grade: number;
timestamp: number;
}
// Queue a review rating when offline
export function queueOfflineReview(cardId: string, grade: number) {
const pending: PendingReview[] = JSON.parse(
localStorage.getItem(PENDING_REVIEWS_KEY) || "[]"
);
pending.push({ cardId, grade, timestamp: Date.now() });
localStorage.setItem(PENDING_REVIEWS_KEY, JSON.stringify(pending));
// Request a Background Sync if the API is available
if ("serviceWorker" in navigator && "SyncManager" in window) {
navigator.serviceWorker.ready.then((registration) => {
(registration as any).sync.register("sync-reviews");
});
}
}
// Process all queued reviews when back online
export async function syncPendingReviews() {
const pending: PendingReview[] = JSON.parse(
localStorage.getItem(PENDING_REVIEWS_KEY) || "[]"
);
if (pending.length === 0) return;
const results = await Promise.allSettled(
pending.map((review) =>
fetch(`/api/revision/review/${review.cardId}`, {
method: "PATCH",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ grade: review.grade }),
})
)
);
// Remove successfully synced reviews from the queue
const remaining = pending.filter(
(_, index) => results[index].status === "rejected"
);
localStorage.setItem(PENDING_REVIEWS_KEY, JSON.stringify(remaining));
}
// Listen for online event to trigger sync
export function registerOnlineSync() {
window.addEventListener("online", () => {
syncPendingReviews();
});
}"use client";
import { useEffect, useState } from "react";
// PWA install prompt banner for mobile users
export default function InstallPrompt() {
const [deferredPrompt, setDeferredPrompt] = useState<any>(null);
const [showBanner, setShowBanner] = useState(false);
useEffect(() => {
// Track visit count to show prompt after 3+ visits
const visits = parseInt(localStorage.getItem("visit_count") || "0", 10) + 1;
localStorage.setItem("visit_count", String(visits));
// Only show the install banner after 3 visits
const handler = (e: Event) => {
e.preventDefault();
setDeferredPrompt(e);
if (visits >= 3) {
setShowBanner(true);
}
};
window.addEventListener("beforeinstallprompt", handler);
return () => window.removeEventListener("beforeinstallprompt", handler);
}, []);
const handleInstall = async () => {
if (!deferredPrompt) return;
deferredPrompt.prompt();
const { outcome } = await deferredPrompt.userChoice;
if (outcome === "accepted") {
setShowBanner(false);
}
setDeferredPrompt(null);
};
if (!showBanner) return null;
return (
<div className="fixed bottom-4 left-4 right-4 bg-indigo-600 text-white rounded-lg p-4 flex items-center justify-between shadow-lg z-50">
<div>
<p className="font-semibold">Installer E-Hanatra</p>
<p className="text-sm text-indigo-200">
Accedez a vos cours hors ligne.
</p>
</div>
<div className="flex gap-2">
<button
onClick={() => setShowBanner(false)}
className="text-sm text-indigo-200 hover:text-white"
>
Plus tard
</button>
<button
onClick={handleInstall}
className="bg-white text-indigo-600 px-4 py-2 rounded-md text-sm font-medium"
>
Installer
</button>
</div>
</div>
);
}Your app is now a fully installable PWA with intelligent offline support. Next up, you will optimize the mobile experience with responsive layouts and touch interactions.
Optimize Mobile UX and Performance
Students use E-Hanatra on their phones between classes, on buses, and during study sessions. The desktop layout with a sidebar and hover-based interactions does not translate well to a 6-inch touchscreen.
In this step, you will overhaul the layout for mobile screens, add touch-friendly gestures to the Knowledge Map and revision cards, and optimize load performance with code splitting and virtualized scrolling.
In this step, get ready to:
- Implement a responsive layout with a bottom navigation bar, slide-over folder tree, and full-screen chat on mobile.
- Optimize touch interactions for the Knowledge Map and revision cards with swipe gestures.
- Improve performance with dynamic imports, skeleton loading, image optimization, and virtualized scrolling.
Implement the responsive layout overhaul
On screens narrower than 768px, the desktop sidebar becomes unusable. You will replace it with a bottom navigation bar, convert the folder tree into a full-screen slide-over panel, and make chat fill the entire viewport.
- Create a custom useMediaQuery hook by adding a new file hooks/useMediaQuery.ts in your Next.js app directory:
"use client";
import { useState, useEffect } from "react";
// Custom hook to detect screen size changes
export function useMediaQuery(query: string): boolean {
const [matches, setMatches] = useState(false);
useEffect(() => {
const mediaQuery = window.matchMedia(query);
setMatches(mediaQuery.matches);
// Update state when viewport changes
const handler = (event: MediaQueryListEvent) => {
setMatches(event.matches);
};
mediaQuery.addEventListener("change", handler);
return () => mediaQuery.removeEventListener("change", handler);
}, [query]);
return matches;
}What does this hook do?
The useMediaQuery hook wraps the browser's window.matchMedia API. It listens for viewport changes and returns a boolean indicating whether the query matches.
You will pass (max-width: 768px) to detect mobile screens and conditionally render the bottom nav instead of the sidebar.
- Create the BottomNav component in components/BottomNav.tsx:
"use client";
import { usePathname } from "next/navigation";
import Link from "next/link";
// Tab items for mobile bottom navigation
const tabs = [
{ label: "Documents", href: "/documents", icon: "📄" },
{ label: "Chat", href: "/chat", icon: "💬" },
{ label: "Map", href: "/knowledge-map", icon: "🗺️" },
{ label: "Revision", href: "/revision", icon: "🔄" },
{ label: "More", href: "/settings", icon: "⋯" },
];
export function BottomNav() {
const pathname = usePathname();
return (
<nav className="bottom-nav">
{tabs.map((tab) => (
<Link
key={tab.href}
href={tab.href}
className={`bottom-nav-tab ${
pathname.startsWith(tab.href) ? "active" : ""
}`}
>
<span className="bottom-nav-icon">{tab.icon}</span>
<span className="bottom-nav-label">{tab.label}</span>
</Link>
))}
</nav>
);
}
/* CSS to add to your global styles:
.bottom-nav {
display: none;
position: fixed;
bottom: 0;
left: 0;
right: 0;
background: #1a1a1a;
border-top: 1px solid #2a2a2a;
padding: 8px 0;
z-index: 1000;
}
@media (max-width: 768px) {
.bottom-nav {
display: flex;
justify-content: space-around;
align-items: center;
}
}
.bottom-nav-tab {
display: flex;
flex-direction: column;
align-items: center;
gap: 2px;
padding: 4px 12px;
text-decoration: none;
color: #888;
font-size: 10px;
}
.bottom-nav-tab.active {
color: #6366f1;
}
.bottom-nav-icon {
font-size: 20px;
}
*/What does this component do?
The BottomNav renders five tabs at the bottom of the screen: Documents, Chat, Map, Revision, and More. It uses next/navigation to highlight the active tab based on the current URL path.
The CSS media query ensures it only appears on screens narrower than 768px. On desktop, the existing sidebar remains visible.
- Create the slide-over panel component for the mobile folder tree in components/MobileSlideOver.tsx:
"use client";
import { useEffect } from "react";
interface MobileSlideOverProps {
isOpen: boolean;
onClose: () => void;
title: string;
children: React.ReactNode;
}
// Full-screen slide-over panel for mobile folder tree
export function MobileSlideOver({
isOpen,
onClose,
title,
children,
}: MobileSlideOverProps) {
// Prevent background scrolling when panel is open
useEffect(() => {
if (isOpen) {
document.body.style.overflow = "hidden";
} else {
document.body.style.overflow = "";
}
return () => {
document.body.style.overflow = "";
};
}, [isOpen]);
if (!isOpen) return null;
return (
<div className="slide-over-backdrop" onClick={onClose}>
<div
className="slide-over-panel"
onClick={(e) => e.stopPropagation()}
>
<div className="slide-over-header">
<h2>{title}</h2>
<button onClick={onClose} className="slide-over-close">
✕
</button>
</div>
<div className="slide-over-content">{children}</div>
</div>
</div>
);
}
/* CSS to add to your global styles:
.slide-over-backdrop {
position: fixed;
inset: 0;
background: rgba(0, 0, 0, 0.5);
z-index: 1100;
}
.slide-over-panel {
position: fixed;
top: 0;
left: 0;
bottom: 0;
width: 85vw;
max-width: 360px;
background: #1a1a1a;
overflow-y: auto;
animation: slideIn 0.2s ease-out;
}
@keyframes slideIn {
from { transform: translateX(-100%); }
to { transform: translateX(0); }
}
.slide-over-header {
display: flex;
justify-content: space-between;
align-items: center;
padding: 16px;
border-bottom: 1px solid #2a2a2a;
}
.slide-over-close {
background: none;
border: none;
color: #888;
font-size: 20px;
cursor: pointer;
}
.slide-over-content {
padding: 16px;
}
*/What does this component do?
On mobile, the folder tree can no longer live in a permanent sidebar. The MobileSlideOver component creates a full-screen overlay that slides in from the left when triggered from the Documents tab.
It locks background scrolling while open and dismisses when the user taps the backdrop or the close button.
- Update your root layout to conditionally render either the desktop sidebar or the mobile bottom nav. In your main layout file, integrate the useMediaQuery hook:
"use client";
import { useMediaQuery } from "@/hooks/useMediaQuery";
import { BottomNav } from "@/components/BottomNav";
// Inside your layout component
export function AppLayout({ children }: { children: React.ReactNode }) {
const isMobile = useMediaQuery("(max-width: 768px)");
return (
<div className="app-container">
{/* Desktop sidebar hidden on mobile */}
{!isMobile && <aside className="desktop-sidebar">{/* FolderTree */}</aside>}
{/* Main content area with bottom padding on mobile for nav bar */}
<main className={isMobile ? "main-content mobile" : "main-content"}>
{children}
</main>
{/* Mobile bottom nav visible only on small screens */}
{isMobile && <BottomNav />}
</div>
);
}
/* CSS additions:
.main-content.mobile {
padding-bottom: 72px; /* Space for bottom nav */
}
@media (max-width: 768px) {
.desktop-sidebar {
display: none;
}
}
*/Why conditionally render instead of CSS-only hide?
Using useMediaQuery to conditionally render components means the sidebar DOM nodes are not mounted at all on mobile. This reduces memory usage and avoids event listeners that would never be used.
The mobile layout adds bottom padding to main-content so content does not get hidden behind the fixed bottom navigation bar.
- Save all four new files: hooks/useMediaQuery.ts, components/BottomNav.tsx, components/MobileSlideOver.tsx, and components/AppLayout.tsx.
- Open your app in a browser and resize the window below 768px width. You should see the sidebar disappear and a bottom navigation bar appear with five tabs.
Bottom nav not appearing?
- Make sure the CSS for .bottom-nav is included in your global stylesheet. The display: none default and the media query override must both be present.
- Confirm AppLayout is actually rendered in your page tree. If your root layout does not use it, the hook will never fire.
- Check that you have "use client" at the top of each file that uses hooks. Server components cannot use useState or useEffect.
Still stuck? Help me debug why my mobile bottom navigation is not rendering.
Optimize touch interactions for mobile
Touch targets on mobile need to be at least 44x44px for comfortable tapping. The Knowledge Map nodes and revision card controls both need touch-specific adjustments.
- Update your ConceptNode component styles to enforce minimum 44x44px tap targets. Also add a Focus button using the useReactFlow hook's fitView method. Create or update components/KnowledgeMapControls.tsx:
"use client";
import { useCallback } from "react";
import { useReactFlow, Panel } from "@xyflow/react";
// Custom controls for Knowledge Map including Focus button
export function KnowledgeMapControls() {
const { fitView } = useReactFlow();
// Fit viewport to show all nodes
const handleFocus = useCallback(() => {
fitView({ padding: 0.2, duration: 300 });
}, [fitView]);
return (
<Panel position="top-right">
<button
onClick={handleFocus}
className="map-control-btn"
aria-label="Focus all nodes"
>
Focus
</button>
</Panel>
);
}
/* CSS additions for touch-friendly nodes:
.concept-node {
min-width: 44px;
min-height: 44px;
padding: 12px 16px;
touch-action: none;
cursor: pointer;
}
.map-control-btn {
min-width: 44px;
min-height: 44px;
padding: 8px 16px;
background: #6366f1;
color: white;
border: none;
border-radius: 6px;
font-size: 14px;
cursor: pointer;
}
*/What does fitView do here?
The fitView method from useReactFlow adjusts the viewport to fit all nodes into view. Calling it with no nodes argument means it targets every node on the canvas.
The padding: 0.2 adds breathing room around the edges. The duration: 300 creates a smooth animated transition. React Flow's built-in touch handlers already support pinch-to-zoom on mobile by default.
- Add the KnowledgeMapControls component inside your ReactFlow wrapper on the Knowledge Map page, alongside your existing MiniMap and Controls components.
- Now implement swipe gestures for the revision flashcard review. Create components/SwipeableCard.tsx:
"use client";
import { useState, useRef } from "react";
interface SwipeableCardProps {
children: React.ReactNode;
onSwipeRight: () => void; // Good (grade 3)
onSwipeLeft: () => void; // Again (grade 0)
onSwipeUp: () => void; // Easy (grade 5)
}
// Swipe gesture handler for revision cards
export function SwipeableCard({
children,
onSwipeRight,
onSwipeLeft,
onSwipeUp,
}: SwipeableCardProps) {
const [offset, setOffset] = useState({ x: 0, y: 0 });
const startPos = useRef({ x: 0, y: 0 });
const isDragging = useRef(false);
const SWIPE_THRESHOLD = 80;
const handlePointerDown = (e: React.PointerEvent) => {
isDragging.current = true;
startPos.current = { x: e.clientX, y: e.clientY };
(e.target as HTMLElement).setPointerCapture(e.pointerId);
};
const handlePointerMove = (e: React.PointerEvent) => {
if (!isDragging.current) return;
const dx = e.clientX - startPos.current.x;
const dy = e.clientY - startPos.current.y;
setOffset({ x: dx, y: dy });
};
const handlePointerUp = () => {
isDragging.current = false;
// Determine swipe direction based on distance
if (offset.x > SWIPE_THRESHOLD) {
onSwipeRight(); // Good
} else if (offset.x < -SWIPE_THRESHOLD) {
onSwipeLeft(); // Again
} else if (offset.y < -SWIPE_THRESHOLD) {
onSwipeUp(); // Easy
}
setOffset({ x: 0, y: 0 });
};
return (
<div
className="swipeable-card"
style={{
transform: `translate(${offset.x}px, ${offset.y}px)`,
transition: isDragging.current ? "none" : "transform 0.3s ease",
touchAction: "none",
}}
onPointerDown={handlePointerDown}
onPointerMove={handlePointerMove}
onPointerUp={handlePointerUp}
>
{children}
{/* Visual feedback indicators */}
{offset.x > 40 && <div className="swipe-indicator right">Good</div>}
{offset.x < -40 && <div className="swipe-indicator left">Again</div>}
{offset.y < -40 && <div className="swipe-indicator top">Easy</div>}
</div>
);
}
/* CSS additions:
.swipeable-card {
touch-action: none;
user-select: none;
cursor: grab;
}
.swipe-indicator {
position: absolute;
padding: 8px 16px;
border-radius: 8px;
font-weight: bold;
font-size: 14px;
}
.swipe-indicator.right {
top: 16px;
right: 16px;
background: #22c55e;
color: white;
}
.swipe-indicator.left {
top: 16px;
left: 16px;
background: #ef4444;
color: white;
}
.swipe-indicator.top {
top: 16px;
left: 50%;
transform: translateX(-50%);
background: #3b82f6;
color: white;
}
*/How do the swipe gestures map to ratings?
The swipe directions mirror the SM-2 rating buttons: swipe right triggers Good (grade 3), swipe left triggers Again (grade 0), and swipe up triggers Easy (grade 5). This gives mobile users a fast, thumb-friendly way to rate cards without needing to tap small buttons.
The component uses the Pointer Events API (onPointerDown, onPointerMove, onPointerUp) which unifies mouse and touch input. The touch-action: none CSS property prevents the browser from intercepting the gesture for scrolling.
- Save components/KnowledgeMapControls.tsx and components/SwipeableCard.tsx.
- Open the Knowledge Map page and tap the Focus button. All nodes should animate into view within the viewport.
- On the Flashcard Review page, wrap your card content with the SwipeableCard component. Drag a card to the right and verify the green "Good" indicator appears.
Swipe gesture not responding?
- Verify touch-action: none is applied to the card container. Without it, the browser may interpret the gesture as a scroll.
- Check that setPointerCapture is called on pointer down. This ensures the element continues receiving pointer events even if the finger moves outside it.
Help me debug why my swipe gestures are not being detected on mobile.
Add performance optimizations
Heavy components like the Knowledge Map (React Flow with dagre layout) and Revision Dashboard (charts and stats) should only load when needed. You will also add virtualized scrolling for long document and folder lists.
- Install react-window for virtualized list rendering by running:
npm install react-window- Create dynamic import wrappers for heavy page components. Add components/DynamicImports.tsx:
import dynamic from "next/dynamic";
// Lazy-load KnowledgeMap only when the route is visited
export const DynamicKnowledgeMap = dynamic(
() => import("@/components/KnowledgeMap").then((mod) => mod.KnowledgeMap),
{
loading: () => <KnowledgeMapSkeleton />,
ssr: false,
}
);
// Lazy-load RevisionDashboard to reduce initial bundle
export const DynamicRevisionDashboard = dynamic(
() => import("@/components/RevisionDashboard").then((mod) => mod.RevisionDashboard),
{
loading: () => <DashboardSkeleton />,
ssr: false,
}
);
// Skeleton for Knowledge Map while loading
function KnowledgeMapSkeleton() {
return (
<div className="skeleton-container">
<div className="skeleton-block" style={{ height: "60vh" }} />
</div>
);
}
// Skeleton for Revision Dashboard while loading
function DashboardSkeleton() {
return (
<div className="skeleton-container">
<div className="skeleton-block" style={{ height: 120 }} />
<div className="skeleton-block" style={{ height: 200 }} />
<div className="skeleton-block" style={{ height: 160 }} />
</div>
);
}
/* CSS additions:
.skeleton-container {
display: flex;
flex-direction: column;
gap: 16px;
padding: 24px;
}
.skeleton-block {
background: linear-gradient(90deg, #2a2a2a 25%, #3a3a3a 50%, #2a2a2a 75%);
background-size: 200% 100%;
animation: shimmer 1.5s infinite;
border-radius: 8px;
width: 100%;
}
@keyframes shimmer {
0% { background-position: 200% 0; }
100% { background-position: -200% 0; }
}
*/Why use dynamic imports here?
Next.js dynamic splits code at the component level. The Knowledge Map pulls in @xyflow/react and dagre, which together add significant weight to the JavaScript bundle.
By lazy-loading these components, the initial page load only downloads the code for the current route. The skeleton placeholders give users immediate visual feedback while the heavy components load in the background.
- Create a virtualized document list component using react-window. Add components/VirtualizedDocList.tsx:
"use client";
import { List, type RowComponentProps } from "react-window";
import Image from "next/image";
interface Document {
id: string;
title: string;
content_type: string;
created_at: string;
}
interface VirtualizedDocListProps {
documents: Document[];
onDocumentClick: (doc: Document) => void;
}
// Row renderer for virtualized document list
function DocumentRow({
index,
style,
documents,
onDocumentClick,
}: RowComponentProps<{ documents: Document[]; onDocumentClick: (doc: Document) => void }>) {
const doc = documents[index];
return (
<div style={style} className="doc-row" onClick={() => onDocumentClick(doc)}>
<div className="doc-row-icon">
{/* Use next/image for optimized WebP format */}
<Image
src={`/icons/${doc.content_type}.webp`}
alt={doc.content_type}
width={32}
height={32}
format="webp"
/>
</div>
<div className="doc-row-info">
<span className="doc-row-title">{doc.title}</span>
<span className="doc-row-date">
{new Date(doc.created_at).toLocaleDateString()}
</span>
</div>
</div>
);
}
// Virtualized list renders only visible document rows
export function VirtualizedDocList({
documents,
onDocumentClick,
}: VirtualizedDocListProps) {
return (
<List
rowCount={documents.length}
rowHeight={64}
rowComponent={DocumentRow}
rowProps={{ documents, onDocumentClick }}
style={{ height: "calc(100vh - 200px)" }}
/>
);
}
/* CSS additions:
.doc-row {
display: flex;
align-items: center;
gap: 12px;
padding: 12px 16px;
border-bottom: 1px solid #2a2a2a;
cursor: pointer;
}
.doc-row:hover {
background: #2a2a2a;
}
.doc-row-info {
display: flex;
flex-direction: column;
gap: 2px;
}
.doc-row-title {
color: #fff;
font-size: 14px;
}
.doc-row-date {
color: #888;
font-size: 12px;
}
*/Why virtualize the document list?
When a folder contains hundreds of documents, rendering every row into the DOM causes scroll jank on mobile devices. The List component from react-window only renders the rows visible in the viewport plus a small overscan buffer.
The next/image component automatically optimizes images to WebP format and lazy-loads them. Combined with virtualization, this keeps the First Contentful Paint fast even on 4G connections.
- Save components/DynamicImports.tsx and components/VirtualizedDocList.tsx.
- Replace your existing Knowledge Map page import with DynamicKnowledgeMap from components/DynamicImports. Do the same with DynamicRevisionDashboard for the revision dashboard page.
- Replace the document list rendering in your folder view with the VirtualizedDocList component.
- Run the Lighthouse performance audit in Chrome DevTools on your app. Target a Performance score above 85 and a First Contentful Paint under 1.5 seconds on a simulated 4G connection.
Measure before and after
Run a Lighthouse audit before making these changes, then again after. The bundle size reduction from dynamic imports and the rendering savings from virtualization should show a measurable improvement in both Performance score and FCP timing.
Seeing a blank list or hydration errors?
- The List component from react-window requires a defined height on its container. If height is 0, nothing renders. Make sure the parent has an explicit height.
- Dynamic imports with ssr: false can cause hydration mismatches if the server renders a different output than the client. This is expected. The skeleton loader handles the server render.
Help me fix my virtualized list or dynamic import issues.
Your app now delivers a polished mobile experience with thumb-friendly navigation, gestural card review, and fast load times. Next up, you will add user authentication to prepare E-Hanatra for multi-user deployment.
Build User Authentication — Backend
Your E-Hanatra platform has a full suite of learning features running in single-user mode. Folders, contextual chat, knowledge maps, and revision cards all work perfectly for one person.
To prepare for public deployment, you need user authentication so each student gets their own private workspace. In this step, you will add JWT-based authentication to the FastAPI backend, scope all existing data by user, and ensure complete isolation between users.
In this step, get ready to:
- Install auth dependencies and create a users table with Argon2 password hashing.
- Build a JWT-based authentication flow with register, login, and token refresh endpoints.
- Scope all existing endpoints and data by user_id for complete data isolation.
Install auth packages and create the users table
Before writing any authentication logic, you need two libraries. PyJWT handles encoding and decoding JSON Web Tokens. pwdlib with the Argon2 backend handles secure password hashing.
- Install PyJWT and pwdlib with Argon2 support by running:
pip install PyJWT "pwdlib[argon2]"What are these packages?
PyJWT is a Python library for encoding and decoding JSON Web Tokens per RFC 7519. It lets you create signed tokens that carry user identity claims.
pwdlib is a modern password hashing library that wraps the Argon2 algorithm. Argon2 is the winner of the Password Hashing Competition and is memory-hard, making brute-force attacks computationally expensive.
- Confirm the install succeeded by running:
pip list | grep -i "pyjwt\|pwdlib"You should see both PyJWT and pwdlib listed in the output.
Packages not showing up?
Make sure you are running pip install in the same virtual environment where your FastAPI server runs. If you are using a virtual environment, activate it first.
help me debug why my pip install is not working for PyJWT and pwdlib
Now create the users table and the authentication configuration. You will add these to a new auth.py file in your FastAPI project.
- Create a new file called auth.py in your FastAPI project directory by running:
touch auth.py- Add the auth configuration, password hashing setup, and users table schema to auth.py by pasting this code:
import jwt
from datetime import datetime, timedelta
from pwdlib import PasswordHash
from fastapi import Depends, HTTPException, status
from fastapi.security import OAuth2PasswordBearer
from pydantic import BaseModel, EmailStr
import aiosqlite
import uuid
# JWT configuration
SECRET_KEY = "your-secret-key-change-in-production"
ALGORITHM = "HS256"
ACCESS_TOKEN_EXPIRE_MINUTES = 1440 # 24 hours for development convenience
# Password hashing with Argon2 (recommended algorithm)
password_hash = PasswordHash.recommended()
# OAuth2 scheme for extracting tokens from Authorization header
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/auth/token")
# Pydantic models for request/response validation
class UserCreate(BaseModel):
email: EmailStr
username: str
password: str
display_name: str = ""
preferred_language: str = "fr"
class UserResponse(BaseModel):
id: str
email: str
username: str
display_name: str
preferred_language: str
class Token(BaseModel):
access_token: str
token_type: str = "bearer"
class ProfileUpdate(BaseModel):
display_name: str = None
preferred_language: str = None
# SQL to create the users table
CREATE_USERS_TABLE = """
CREATE TABLE IF NOT EXISTS users (
id TEXT PRIMARY KEY,
email TEXT UNIQUE NOT NULL,
username TEXT UNIQUE NOT NULL,
hashed_password TEXT NOT NULL,
display_name TEXT DEFAULT '',
preferred_language TEXT DEFAULT 'fr',
created_at TEXT DEFAULT CURRENT_TIMESTAMP
)
"""What does this code do?
- SECRET_KEY is the symmetric key used to sign and verify JWTs. You will replace this with an environment variable in production.
- PasswordHash.recommended() creates a hasher using Argon2 with secure defaults. It embeds the salt, algorithm parameters, and hash in a single string.
- OAuth2PasswordBearer(tokenUrl="/auth/token") tells FastAPI to expect a Bearer token in the Authorization header and points the OpenAPI docs to the login endpoint.
- ACCESS_TOKEN_EXPIRE_MINUTES = 1440 sets tokens to last 24 hours, convenient during development so you do not need to re-login constantly.
- Save auth.py.
Build the register, login, and token helper functions
With the configuration in place, you need functions that create tokens, register new users, and verify credentials. These form the core of the authentication flow.
- Add the token creation and user authentication functions below the existing code in auth.py:
# Create a signed JWT token with user_id in the 'sub' claim
def create_access_token(user_id: str, expires_delta: timedelta = None):
expire = datetime.utcnow() + (expires_delta or timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES))
payload = {"sub": user_id, "exp": expire}
token = jwt.encode(payload, SECRET_KEY, algorithm=ALGORITHM)
return token
# Dependency that extracts and validates the current user from the JWT
async def get_current_user(token: str = Depends(oauth2_scheme)):
credentials_exception = HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Could not validate credentials",
headers={"WWW-Authenticate": "Bearer"},
)
try:
# Decode the token and extract the user_id from 'sub' claim
payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
user_id: str = payload.get("sub")
if user_id is None:
raise credentials_exception
except jwt.ExpiredSignatureError:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Token has expired",
headers={"WWW-Authenticate": "Bearer"},
)
except jwt.InvalidTokenError:
raise credentials_exception
return user_idWhat does this code do?
create_access_token encodes the user's ID into the sub (subject) claim of a JWT, along with an expiration timestamp. The token is signed with HS256 using your secret key.
get_current_user is a FastAPI dependency. It extracts the Bearer token from the request, decodes it with jwt.decode, and returns the user_id. If the token is expired or invalid, it raises a 401 error.
- Now add the register and login route handlers below the helper functions in auth.py:
from fastapi import APIRouter
from fastapi.security import OAuth2PasswordRequestForm
auth_router = APIRouter(prefix="/auth", tags=["auth"])
@auth_router.post("/register", response_model=Token)
async def register(user_data: UserCreate):
async with aiosqlite.connect("e-hanatra.db") as db:
# Check if email already exists
cursor = await db.execute(
"SELECT id FROM users WHERE email = ?", (user_data.email,)
)
if await cursor.fetchone():
raise HTTPException(
status_code=status.HTTP_400_BAD_REQUEST,
detail="Email already registered"
)
# Hash the password with Argon2
hashed = password_hash.hash(user_data.password)
user_id = str(uuid.uuid4())
# Insert the new user
await db.execute(
"""INSERT INTO users (id, email, username, hashed_password, display_name, preferred_language)
VALUES (?, ?, ?, ?, ?, ?)""",
(user_id, user_data.email, user_data.username, hashed, user_data.display_name, user_data.preferred_language)
)
await db.commit()
# Return a JWT so the user is logged in immediately after registration
access_token = create_access_token(user_id)
return Token(access_token=access_token)
@auth_router.post("/token", response_model=Token)
async def login(form_data: OAuth2PasswordRequestForm = Depends()):
async with aiosqlite.connect("e-hanatra.db") as db:
# Look up user by username (OAuth2 form uses 'username' field)
cursor = await db.execute(
"SELECT id, hashed_password FROM users WHERE username = ? OR email = ?",
(form_data.username, form_data.username)
)
row = await cursor.fetchone()
if not row:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Incorrect username or password",
headers={"WWW-Authenticate": "Bearer"},
)
user_id, hashed_password = row
# Verify the password against the stored Argon2 hash
if not password_hash.verify(form_data.password, hashed_password):
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Incorrect username or password",
headers={"WWW-Authenticate": "Bearer"},
)
access_token = create_access_token(user_id)
return Token(access_token=access_token, token_type="bearer")
@auth_router.put("/profile")
async def update_profile(
profile: ProfileUpdate,
user_id: str = Depends(get_current_user)
):
async with aiosqlite.connect("e-hanatra.db") as db:
if profile.display_name is not None:
await db.execute(
"UPDATE users SET display_name = ? WHERE id = ?",
(profile.display_name, user_id)
)
if profile.preferred_language is not None:
await db.execute(
"UPDATE users SET preferred_language = ? WHERE id = ?",
(profile.preferred_language, user_id)
)
await db.commit()
return {"message": "Profile updated"}
@auth_router.post("/refresh", response_model=Token)
async def refresh_token(user_id: str = Depends(get_current_user)):
# Issue a fresh token with a new expiration
access_token = create_access_token(user_id)
return Token(access_token=access_token)What does this code do?
- POST /auth/register validates email uniqueness, hashes the password with Argon2, stores the new user, and returns a JWT immediately.
- POST /auth/token accepts the standard OAuth2 password form (username + password), verifies credentials, and returns an access token.
- PUT /auth/profile lets authenticated users update their display name or language preference.
- POST /auth/refresh issues a fresh token for users whose current token is still valid but approaching expiry.
- Save auth.py.
- Register the auth router in your main FastAPI app. In main.py, add this import and include statement:
from auth import auth_router, CREATE_USERS_TABLE
# Include auth routes
app.include_router(auth_router)- Also add the users table creation to your startup event in main.py:
# Inside your startup/lifespan function, add:
async with aiosqlite.connect("e-hanatra.db") as db:
await db.execute(CREATE_USERS_TABLE)
await db.commit()- Save main.py and restart your FastAPI server.
- Open your browser to http://localhost:8000/docs. You should see the new /auth/register, /auth/token, /auth/profile, and /auth/refresh endpoints listed under the "auth" tag.
Server won't start?
Check that you imported auth_router correctly and that auth.py is in the same directory as main.py. Also verify that aiosqlite is still installed in your environment.
help me fix my FastAPI auth import error
Scope all existing data by user_id and update ChromaDB metadata
The authentication endpoints are working, but existing data has no user association. You need to add a user_id column to every table and update ChromaDB chunk metadata so each user's documents, folders, concepts, and revision cards are completely isolated.
- Create a migration script to add user_id columns and assign existing data to a default admin user. Create a file called migrate_add_user_id.py with this code:
import asyncio
import aiosqlite
DEFAULT_USER_ID = "admin"
async def migrate():
async with aiosqlite.connect("e-hanatra.db") as db:
# Add user_id column to folders table
try:
await db.execute("ALTER TABLE folders ADD COLUMN user_id TEXT DEFAULT 'admin'")
except Exception:
pass # Column already exists
# Add user_id column to documents table
try:
await db.execute("ALTER TABLE documents ADD COLUMN user_id TEXT DEFAULT 'admin'")
except Exception:
pass
# Add user_id column to concepts table
try:
await db.execute("ALTER TABLE concepts ADD COLUMN user_id TEXT DEFAULT 'admin'")
except Exception:
pass
# Add user_id column to revision_cards table
try:
await db.execute("ALTER TABLE revision_cards ADD COLUMN user_id TEXT DEFAULT 'admin'")
except Exception:
pass
# Assign all existing data to the default admin user
await db.execute("UPDATE folders SET user_id = ? WHERE user_id IS NULL", (DEFAULT_USER_ID,))
await db.execute("UPDATE documents SET user_id = ? WHERE user_id IS NULL", (DEFAULT_USER_ID,))
await db.execute("UPDATE concepts SET user_id = ? WHERE user_id IS NULL", (DEFAULT_USER_ID,))
await db.execute("UPDATE revision_cards SET user_id = ? WHERE user_id IS NULL", (DEFAULT_USER_ID,))
await db.commit()
print("Migration complete: user_id added to all tables")
if __name__ == "__main__":
asyncio.run(migrate())What does this migration do?
SQLite supports ALTER TABLE ADD COLUMN but not IF NOT EXISTS on columns. The try/except blocks handle the case where you run the migration more than once.
All existing records get assigned to a default "admin" user so they remain accessible after the migration. New records created through authenticated endpoints will receive the actual user's ID.
- Run the migration by executing:
python migrate_add_user_id.pyYou should see "Migration complete: user_id added to all tables" in the output.
- Now update your existing endpoints to require authentication and filter by user_id. Add the get_current_user dependency to every route handler. Here is the pattern to apply across your existing endpoints in main.py:
from auth import get_current_user
from fastapi import Depends
# Example: update the folder creation endpoint
@app.post("/folders")
async def create_folder(folder_data: FolderCreate, user_id: str = Depends(get_current_user)):
async with aiosqlite.connect("e-hanatra.db") as db:
folder_id = str(uuid.uuid4())
await db.execute(
"""INSERT INTO folders (id, name, parent_id, user_id)
VALUES (?, ?, ?, ?)""",
(folder_id, folder_data.name, folder_data.parent_id, user_id)
)
await db.commit()
return {"id": folder_id, "name": folder_data.name}
# Example: update the folder tree endpoint to scope by user
@app.get("/folders/tree")
async def get_folder_tree(user_id: str = Depends(get_current_user)):
async with aiosqlite.connect("e-hanatra.db") as db:
cursor = await db.execute(
"SELECT id, name, parent_id FROM folders WHERE user_id = ?",
(user_id,)
)
folders = await cursor.fetchall()
# Build and return the tree structure
return build_tree(folders)How do I apply this to all endpoints?
The pattern is the same for every endpoint. Add user_id: str = Depends(get_current_user) as a parameter to each route handler. Then add WHERE user_id = ? to every SQL query that reads or writes data.
Apply this pattern to all folder, document, chat, knowledge map, and revision card endpoints. Every SELECT, INSERT, UPDATE, and DELETE statement should be scoped to the authenticated user.
- Update the ChromaDB ingestion to include user_id in chunk metadata. Find your document upload endpoint and update the metadata passed to ChromaDB:
# When adding chunks to ChromaDB, include user_id in metadata
collection.add(
documents=chunks,
metadatas=[{
"document_id": doc_id,
"folder_id": folder_id,
"document_title": title,
"user_id": user_id # Add user_id to every chunk
} for chunk in chunks],
ids=chunk_ids
)- Update the global chat scope to filter by user_id so users only search their own documents:
# Global chat scope - filter by user_id for data isolation
@app.post("/chat/global")
async def chat_global(query: ChatQuery, user_id: str = Depends(get_current_user)):
# Query ChromaDB with user_id filter for global scope
results = collection.query(
query_texts=[query.message],
n_results=10,
where={"user_id": user_id} # Ensures user only searches their own data
)
# Build context and send to Groq
context = "\n".join(results["documents"][0])
# ... rest of chat logicWhy filter ChromaDB by user_id?
Without user_id filtering on the global chat scope, a user's query could retrieve chunks from another user's documents. The where parameter in ChromaDB's query method applies metadata filtering before the similarity search runs, ensuring complete isolation.
Document-scoped and folder-scoped queries already isolate by document_id or folder_id. Since those resources belong to a specific user, they provide implicit isolation. The global scope is the only one that needs explicit user_id filtering.
- Write a script to backfill user_id on existing ChromaDB chunks. Add this to your migration or run it once:
# Backfill user_id metadata on existing ChromaDB chunks
# Run this once after the migration
import chromadb
client = chromadb.PersistentClient(path="./chroma_data")
collection = client.get_collection("documents")
# Get all existing documents and update their metadata
results = collection.get(include=["metadatas"])
for i, metadata in enumerate(results["metadatas"]):
if "user_id" not in metadata:
metadata["user_id"] = "admin" # Assign to default admin user
collection.update(
ids=[results["ids"][i]],
metadatas=[metadata]
)- Save all files and restart your FastAPI server.
- Test the full flow by opening http://localhost:8000/docs. Use the /auth/register endpoint to create a test user. Then use the Authorize button in the Swagger UI to test authenticated endpoints.
Getting 401 Unauthorized on existing endpoints?
This is expected. After adding get_current_user as a dependency, all endpoints now require a valid JWT in the Authorization header. Register a user first, then use the returned token.
In the Swagger UI, click the **Authorize** button at the top, enter your username and password, and click **Authorize**. The UI will automatically include the token on subsequent requests.
help me debug 401 errors after adding JWT auth to FastAPI
Your backend now has a complete authentication system. Every endpoint requires a valid JWT, and all data is scoped to the authenticated user. Next up, you will build the frontend login and registration pages to connect to these new endpoints.
Build User Authentication — Frontend
Your backend now handles user registration, login, and token-based access control. Every API endpoint requires a valid JWT in the Authorization header.
The missing piece is the frontend. Without login and register pages, route protection, and a token-forwarding mechanism, users cannot interact with the secured backend. In this step, you will build the entire authentication UI layer in Next.js so that the frontend and backend work together seamlessly.
In this step, get ready to:
- Build login and register pages with a Next.js API route that stores the JWT in an httpOnly cookie.
- Protect routes with middleware and add a user menu with logout functionality.
- Add personal space indicators and ensure all API calls carry the Authorization header.
Build login and register pages with AuthContext
The authentication flow works like this: the learner submits credentials on a login form, which calls a Next.js API route. That route proxies the request to FastAPI's POST /auth/token endpoint. On success, the API route stores the returned JWT in an httpOnly cookie and responds to the client.
Why an httpOnly cookie?
An httpOnly cookie cannot be read by client-side JavaScript. This protects the token from cross-site scripting (XSS) attacks. The Next.js API route acts as a proxy that receives the token from FastAPI and sets it securely on the browser.
- Create the login API route by adding a new file at app/api/auth/login/route.ts:
import { NextRequest, NextResponse } from 'next/server'
// Proxy login to FastAPI and store the JWT in an httpOnly cookie
export async function POST(request: NextRequest) {
const body = await request.json()
// Forward credentials to FastAPI's token endpoint
const res = await fetch('http://localhost:8000/auth/token', {
method: 'POST',
headers: { 'Content-Type': 'application/x-www-form-urlencoded' },
body: new URLSearchParams({
username: body.email,
password: body.password,
}),
})
if (!res.ok) {
const error = await res.json()
return NextResponse.json(
{ error: error.detail || 'Login failed' },
{ status: res.status }
)
}
const data = await res.json()
// Set the access token as an httpOnly cookie
const response = NextResponse.json({ success: true })
response.cookies.set({
name: 'access_token',
value: data.access_token,
httpOnly: true,
secure: process.env.NODE_ENV === 'production',
path: '/',
maxAge: 60 * 60 * 24, // 24 hours matching backend expiry
})
return response
}What does this code do?
- It receives the email and password from the client form as JSON.
- It forwards credentials to FastAPI using application/x-www-form-urlencoded format, which is what OAuth2PasswordRequestForm expects.
- On success, it stores access_token in an httpOnly cookie so the browser cannot access it via JavaScript.
- Save app/api/auth/login/route.ts.
- Create an AuthContext provider at lib/AuthContext.tsx that holds current user state and provides login, logout, and isAuthenticated helpers:
'use client'
import { createContext, useContext, useState, useEffect, ReactNode } from 'react'
import { useRouter } from 'next/navigation'
// Shape of the user object returned by the backend
interface User {
id: string
email: string
display_name: string
preferred_language: string
}
interface AuthContextType {
user: User | null
isAuthenticated: boolean
login: (email: string, password: string) => Promise<{ success: boolean; error?: string }>
logout: () => Promise<void>
refreshUser: () => Promise<void>
}
const AuthContext = createContext<AuthContextType | undefined>(undefined)
export function AuthProvider({ children }: { children: ReactNode }) {
const [user, setUser] = useState<User | null>(null)
const router = useRouter()
// Fetch the current user profile on mount
const refreshUser = async () => {
try {
const res = await fetch('/api/auth/me')
if (res.ok) {
const data = await res.json()
setUser(data)
} else {
setUser(null)
}
} catch {
setUser(null)
}
}
useEffect(() => {
refreshUser()
}, [])
// Login by calling the Next.js proxy route
const login = async (email: string, password: string) => {
const res = await fetch('/api/auth/login', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ email, password }),
})
if (res.ok) {
await refreshUser()
return { success: true }
}
const error = await res.json()
return { success: false, error: error.error || 'Login failed' }
}
// Logout clears the cookie and redirects to /login
const logout = async () => {
await fetch('/api/auth/logout', { method: 'POST' })
setUser(null)
router.push('/login')
}
return (
<AuthContext.Provider
value={{ user, isAuthenticated: !!user, login, logout, refreshUser }}
>
{children}
</AuthContext.Provider>
)
}
// Custom hook for consuming auth context
export function useAuth() {
const context = useContext(AuthContext)
if (!context) {
throw new Error('useAuth must be used within an AuthProvider')
}
return context
}What does this code do?
The AuthProvider wraps your app and manages the user session state. On mount, it calls /api/auth/me to check if a valid cookie exists. The login function posts to the proxy route, and logout clears the cookie and redirects.
The useAuth hook gives any component access to user state and auth actions without prop drilling.
- Save lib/AuthContext.tsx.
- Create the Login page at app/login/page.tsx:
'use client'
import { useState } from 'react'
import { useAuth } from '@/lib/AuthContext'
import { useRouter } from 'next/navigation'
import Link from 'next/link'
export default function LoginPage() {
const [email, setEmail] = useState('')
const [password, setPassword] = useState('')
const [error, setError] = useState('')
const [loading, setLoading] = useState(false)
const { login } = useAuth()
const router = useRouter()
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault()
setLoading(true)
setError('')
const result = await login(email, password)
if (result.success) {
router.push('/')
} else {
setError(result.error || 'Invalid credentials')
}
setLoading(false)
}
return (
<div className="auth-container">
<h1>Sign In to E-Hanatra</h1>
<form onSubmit={handleSubmit}>
<div className="form-group">
<label htmlFor="email">Email</label>
<input
id="email"
type="email"
value={email}
onChange={(e) => setEmail(e.target.value)}
required
placeholder="you@example.com"
/>
</div>
<div className="form-group">
<label htmlFor="password">Password</label>
<input
id="password"
type="password"
value={password}
onChange={(e) => setPassword(e.target.value)}
required
placeholder="Your password"
/>
</div>
{error && <p className="error-message">{error}</p>}
<button type="submit" disabled={loading}>
{loading ? 'Signing in...' : 'Sign In'}
</button>
</form>
<p className="auth-link">
Don't have an account? <Link href="/register">Create one</Link>
</p>
</div>
)
}What does this code do?
- The form captures email and password, then calls the login function from AuthContext.
- On success, it redirects to the home page. On failure, it displays the error message from the backend.
- A link at the bottom sends new users to the registration page.
- Save app/login/page.tsx.
- Create the Register page at app/register/page.tsx:
'use client'
import { useState } from 'react'
import { useAuth } from '@/lib/AuthContext'
import { useRouter } from 'next/navigation'
import Link from 'next/link'
export default function RegisterPage() {
const [email, setEmail] = useState('')
const [password, setPassword] = useState('')
const [displayName, setDisplayName] = useState('')
const [error, setError] = useState('')
const [loading, setLoading] = useState(false)
const { login } = useAuth()
const router = useRouter()
const handleSubmit = async (e: React.FormEvent) => {
e.preventDefault()
setLoading(true)
setError('')
// Register with FastAPI
const res = await fetch('/api/auth/register', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ email, password, display_name: displayName }),
})
if (!res.ok) {
const data = await res.json()
setError(data.error || 'Registration failed')
setLoading(false)
return
}
// Auto-login after successful registration
const loginResult = await login(email, password)
if (loginResult.success) {
router.push('/')
} else {
setError('Registered but could not sign in automatically.')
}
setLoading(false)
}
return (
<div className="auth-container">
<h1>Create Your Account</h1>
<form onSubmit={handleSubmit}>
<div className="form-group">
<label htmlFor="displayName">Display Name (optional)</label>
<input
id="displayName"
type="text"
value={displayName}
onChange={(e) => setDisplayName(e.target.value)}
placeholder="Your name"
/>
</div>
<div className="form-group">
<label htmlFor="email">Email</label>
<input
id="email"
type="email"
value={email}
onChange={(e) => setEmail(e.target.value)}
required
placeholder="you@example.com"
/>
</div>
<div className="form-group">
<label htmlFor="password">Password</label>
<input
id="password"
type="password"
value={password}
onChange={(e) => setPassword(e.target.value)}
required
placeholder="Choose a strong password"
/>
</div>
{error && <p className="error-message">{error}</p>}
<button type="submit" disabled={loading}>
{loading ? 'Creating account...' : 'Create Account'}
</button>
</form>
<p className="auth-link">
Already have an account? <Link href="/login">Sign in</Link>
</p>
</div>
)
}What does this code do?
- The form includes an optional display name field alongside email and password.
- On submit, it calls the /api/auth/register proxy route to create the account in FastAPI.
- After registration, it automatically logs the user in so they do not need to sign in again immediately.
- Save app/register/page.tsx.
The AuthProvider needs to wrap the entire app. This makes user state available to every page and component.
- In your root layout file (app/layout.tsx), import and wrap the children with AuthProvider:
import { AuthProvider } from '@/lib/AuthContext'
// Inside your RootLayout component, wrap children:
export default function RootLayout({ children }: { children: React.ReactNode }) {
return (
<html lang="fr">
<body>
<AuthProvider>
{children}
</AuthProvider>
</body>
</html>
)
}What does this code do?
By wrapping the entire app in AuthProvider, every page and component can call useAuth() to check login state, get the current user, or trigger logout.
- Save app/layout.tsx.
- Start your Next.js dev server and navigate to /login in your browser. You should see a sign-in form with email and password fields.
Don't see the login form?
- Make sure app/login/page.tsx is saved and the dev server is running.
- Check the terminal for TypeScript errors. The 'use client' directive must be the first line in the file.
- Confirm that lib/AuthContext.tsx exports AuthProvider and useAuth correctly.
help me debug my login page
Add route protection and user menu
Authenticated pages should not be accessible to unauthenticated visitors. Next.js middleware runs before every request, making it the ideal place to check for the auth cookie and redirect.
- Create middleware.ts in the root of your project (same level as app/):
import { NextResponse } from 'next/server'
import type { NextRequest } from 'next/server'
// Routes that do not require authentication
const publicRoutes = ['/login', '/register']
export function middleware(request: NextRequest) {
const { pathname } = request.nextUrl
// Allow public routes without checking auth
if (publicRoutes.some((route) => pathname.startsWith(route))) {
return NextResponse.next()
}
// Check for the auth cookie
const token = request.cookies.get('access_token')?.value
// Redirect to /login if no token is found
if (!token) {
return NextResponse.redirect(new URL('/login', request.url))
}
return NextResponse.next()
}
// Run middleware on all routes except static assets and API routes
export const config = {
matcher: ['/((?!api|_next/static|_next/image|.*\\.png$).*)'],
}What does this code do?
- It defines /login and /register as public routes that skip the auth check entirely.
- For all other routes, it reads the access_token cookie. If the cookie is missing, the user is redirected to /login.
- The config.matcher excludes API routes and static files from middleware processing.
- Save middleware.ts.
- Try navigating to / without being logged in. You should be redirected to /login.
Not getting redirected?
- Ensure middleware.ts is in the project root (next to app/), not inside the app/ directory.
- Clear browser cookies for localhost and reload the page.
- Check that the exported function is named middleware (not a default export with a different name).
help me debug my middleware redirect
Now you need a user menu in the header so logged-in users can see their name, toggle language, and log out.
- Create a UserMenu component at components/UserMenu.tsx:
'use client'
import { useState } from 'react'
import { useAuth } from '@/lib/AuthContext'
export default function UserMenu() {
const { user, logout } = useAuth()
const [open, setOpen] = useState(false)
if (!user) return null
// Toggle language preference between Malagasy and French
const toggleLanguage = async () => {
const newLang = user.preferred_language === 'mg' ? 'fr' : 'mg'
await fetch('/api/auth/profile', {
method: 'PUT',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ preferred_language: newLang }),
})
window.location.reload()
}
return (
<div className="user-menu">
<button
className="user-menu-trigger"
onClick={() => setOpen(!open)}
>
{user.display_name || user.email}
</button>
{open && (
<div className="user-menu-dropdown">
<button onClick={toggleLanguage}>
{user.preferred_language === 'mg' ? 'Passer en Francais' : 'Malagasy'}
</button>
<button onClick={logout}>Se deconnecter</button>
</div>
)}
</div>
)
}What does this code do?
- It shows the user's display name as a clickable button that toggles a dropdown menu.
- The language toggle calls PUT /api/auth/profile to update the user's preferred language on the backend.
- The logout button calls logout() from AuthContext, which clears the cookie and redirects to /login.
- Save components/UserMenu.tsx.
- Add UserMenu to your app header component so it appears in the top navigation area.
- Log in with a registered account. You should see your display name in the header with a dropdown offering language toggle and logout.
User menu not showing?
- Verify you are logged in. The component returns null if there is no user.
- Check that UserMenu is imported in a component that renders inside the AuthProvider wrapper.
help me debug why the user menu is not appearing
Add personal space indicators and fetch wrapper
Logged-in users should feel at home with a personalized dashboard. You also need to ensure every API call from the frontend includes the Authorization header automatically.
- Create an authenticated fetch wrapper at lib/api.ts that attaches the token to every request:
// Authenticated fetch wrapper for all FastAPI calls
const API_BASE = 'http://localhost:8000'
export async function apiFetch(
endpoint: string,
options: RequestInit = {}
): Promise<Response> {
// Read the token from the cookie via the Next.js proxy
const tokenRes = await fetch('/api/auth/token')
const { token } = await tokenRes.json()
const headers = new Headers(options.headers)
if (token) {
headers.set('Authorization', `Bearer ${token}`)
}
headers.set('Content-Type', 'application/json')
return fetch(`${API_BASE}${endpoint}`, {
...options,
headers,
})
}What does this code do?
The apiFetch function reads the current token from a server-side proxy endpoint and attaches it as a Bearer token in the Authorization header. Every component that calls the FastAPI backend uses this wrapper instead of raw fetch.
This centralizes auth logic so you never forget to include the token in an API call.
- Save lib/api.ts.
Next, add a personalized welcome message and resource counts to the dashboard.
- In your main dashboard page, add a welcome section that displays the user's name and their document and folder counts:
'use client'
import { useAuth } from '@/lib/AuthContext'
import { useEffect, useState } from 'react'
import { apiFetch } from '@/lib/api'
export default function DashboardWelcome() {
const { user } = useAuth()
const [docCount, setDocCount] = useState(0)
const [folderCount, setFolderCount] = useState(0)
useEffect(() => {
// Fetch document and folder counts for the current user
const loadCounts = async () => {
const docsRes = await apiFetch('/documents')
const foldersRes = await apiFetch('/folders/tree')
if (docsRes.ok) {
const docs = await docsRes.json()
setDocCount(Array.isArray(docs) ? docs.length : 0)
}
if (foldersRes.ok) {
const folders = await foldersRes.json()
setFolderCount(Array.isArray(folders) ? folders.length : 0)
}
}
loadCounts()
}, [])
if (!user) return null
return (
<div className="dashboard-welcome">
<h2>Welcome, {user.display_name || user.email}</h2>
<div className="user-stats">
<span>{docCount} documents</span>
<span>{folderCount} folders</span>
</div>
</div>
)
}What does this code do?
- It greets the user by name using user.display_name from AuthContext.
- It calls the documents and folders endpoints via apiFetch to show totals.
- These counts give students a quick overview of their knowledge base size.
- Save components/DashboardWelcome.tsx and import it into your main page.
- Log in and navigate to the dashboard. You should see "Welcome, {your name}" along with your document and folder counts.
Finally, build the My Profile page for account management.
- Create app/profile/page.tsx with sections for changing password, display name, preferred language, and a danger zone for account deletion:
'use client'
import { useState } from 'react'
import { useAuth } from '@/lib/AuthContext'
import { apiFetch } from '@/lib/api'
export default function ProfilePage() {
const { user, refreshUser, logout } = useAuth()
const [displayName, setDisplayName] = useState(user?.display_name || '')
const [language, setLanguage] = useState(user?.preferred_language || 'mg')
const [currentPassword, setCurrentPassword] = useState('')
const [newPassword, setNewPassword] = useState('')
const [message, setMessage] = useState('')
// Update display name and language
const handleUpdateProfile = async (e: React.FormEvent) => {
e.preventDefault()
const res = await apiFetch('/auth/profile', {
method: 'PUT',
body: JSON.stringify({ display_name: displayName, preferred_language: language }),
})
if (res.ok) {
await refreshUser()
setMessage('Profile updated successfully.')
} else {
setMessage('Failed to update profile.')
}
}
// Change password
const handleChangePassword = async (e: React.FormEvent) => {
e.preventDefault()
const res = await apiFetch('/auth/change-password', {
method: 'POST',
body: JSON.stringify({
current_password: currentPassword,
new_password: newPassword,
}),
})
if (res.ok) {
setMessage('Password changed successfully.')
setCurrentPassword('')
setNewPassword('')
} else {
setMessage('Failed to change password.')
}
}
// Delete account permanently
const handleDeleteAccount = async () => {
const confirmed = window.confirm(
'Are you sure? This will permanently delete your account and all data.'
)
if (!confirmed) return
const res = await apiFetch('/auth/delete-account', { method: 'DELETE' })
if (res.ok) {
await logout()
} else {
setMessage('Failed to delete account.')
}
}
if (!user) return null
return (
<div className="profile-page">
<h1>My Profile</h1>
{message && <p className="status-message">{message}</p>}
<section className="profile-section">
<h2>Account Settings</h2>
<form onSubmit={handleUpdateProfile}>
<div className="form-group">
<label htmlFor="displayName">Display Name</label>
<input
id="displayName"
type="text"
value={displayName}
onChange={(e) => setDisplayName(e.target.value)}
/>
</div>
<div className="form-group">
<label htmlFor="language">Preferred Language</label>
<select
id="language"
value={language}
onChange={(e) => setLanguage(e.target.value)}
>
<option value="mg">Malagasy</option>
<option value="fr">Francais</option>
<option value="en">English</option>
</select>
</div>
<button type="submit">Save Changes</button>
</form>
</section>
<section className="profile-section">
<h2>Change Password</h2>
<form onSubmit={handleChangePassword}>
<div className="form-group">
<label htmlFor="currentPassword">Current Password</label>
<input
id="currentPassword"
type="password"
value={currentPassword}
onChange={(e) => setCurrentPassword(e.target.value)}
required
/>
</div>
<div className="form-group">
<label htmlFor="newPassword">New Password</label>
<input
id="newPassword"
type="password"
value={newPassword}
onChange={(e) => setNewPassword(e.target.value)}
required
/>
</div>
<button type="submit">Change Password</button>
</form>
</section>
<section className="profile-section danger-zone">
<h2>Danger Zone</h2>
<p>Permanently delete your account and all associated data.</p>
<button className="danger-button" onClick={handleDeleteAccount}>
Delete My Account
</button>
</section>
</div>
)
}What does this code do?
- The Account Settings section lets users update their display name and preferred language via PUT /auth/profile.
- The Change Password section sends current and new passwords to a dedicated endpoint.
- The Danger Zone provides account deletion with a confirmation dialog. After deletion, it logs the user out.
- Save app/profile/page.tsx.
- Navigate to /profile in your browser while logged in. You should see Account Settings, Change Password, and Danger Zone sections.
Profile page shows nothing?
- The component returns null if user is null. Confirm you are logged in by checking that the access_token cookie exists in your browser dev tools.
- Make sure app/profile/page.tsx has the 'use client' directive at the top.
help me debug why my profile page is blank
Your frontend authentication layer is complete. Users can register, log in, and manage their accounts. The middleware keeps unauthorized visitors out. Next up, you will audit data isolation and polish the multi-user experience for production readiness.
Finalize Data Isolation and Production Hardening
Your authentication system is fully wired up. Users can register, log in, and access their personal data.
But there is a critical gap between "endpoints have user_id" and "no user can ever see another user's data." In this step, you will audit every data path with integration tests, build a welcoming onboarding experience for new users, and harden the API for production deployment.
In this step, get ready to:
- Write integration tests that prove two users cannot access each other's data.
- Add a post-registration onboarding flow with a welcome modal and default folder creation.
- Harden the API with rate limiting, CORS, request logging, and a deployment checklist.
Write data isolation integration tests
Adding a user_id column is necessary but not sufficient. A missing WHERE clause in even one query means user A could see user B's private notes, revision cards, or chat history.
Integration tests catch these leaks systematically. You will create two test users, populate data for each, and assert that cross-user access always fails.
- Create a new file called tests/test_data_isolation.py in your FastAPI project by running:
touch tests/test_data_isolation.py- Add the following test suite to tests/test_data_isolation.py:
import pytest
from httpx import AsyncClient, ASGITransport
from main import app
# Test configuration
BASE_URL = "http://test"
# Helper to register a user and get an auth token
async def create_user_and_login(client: AsyncClient, email: str, password: str):
# Register
await client.post("/auth/register", json={
"email": email,
"password": password,
"username": email.split("@")[0]
})
# Login
response = await client.post("/auth/token", data={
"username": email,
"password": password
})
token = response.json()["access_token"]
return {"Authorization": f"Bearer {token}"}
@pytest.mark.asyncio
async def test_folder_isolation():
"""User A cannot see User B's folders."""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url=BASE_URL) as client:
# Create two users
headers_a = await create_user_and_login(client, "usera@test.com", "pass123")
headers_b = await create_user_and_login(client, "userb@test.com", "pass456")
# User A creates a folder
resp = await client.post("/folders", json={"name": "Private A"}, headers=headers_a)
folder_a_id = resp.json()["id"]
# User B should NOT see User A's folder
resp = await client.get("/folders/tree", headers=headers_b)
folder_ids = [f["id"] for f in resp.json()]
assert folder_a_id not in folder_ids, "User B can see User A's folder!"
@pytest.mark.asyncio
async def test_document_isolation():
"""User A cannot access User B's documents."""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url=BASE_URL) as client:
headers_a = await create_user_and_login(client, "doc_a@test.com", "pass123")
headers_b = await create_user_and_login(client, "doc_b@test.com", "pass456")
# User A uploads a document (simulated)
resp = await client.post("/folders", json={"name": "Docs A"}, headers=headers_a)
folder_a_id = resp.json()["id"]
# User B tries to access User A's folder documents
resp = await client.get(f"/folders/{folder_a_id}/documents", headers=headers_b)
assert resp.status_code in [403, 404], "User B accessed User A's documents!"
@pytest.mark.asyncio
async def test_chat_history_isolation():
"""User A cannot read User B's chat history."""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url=BASE_URL) as client:
headers_a = await create_user_and_login(client, "chat_a@test.com", "pass123")
headers_b = await create_user_and_login(client, "chat_b@test.com", "pass456")
# User A starts a global chat
await client.post("/chat/global", json={"message": "Secret question"}, headers=headers_a)
# User B should not see User A's conversations
resp = await client.get("/chat/global", headers=headers_b)
messages = resp.json().get("messages", [])
for msg in messages:
assert "Secret question" not in msg.get("content", ""), "Chat leak detected!"
@pytest.mark.asyncio
async def test_revision_card_isolation():
"""User A cannot access User B's revision cards."""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url=BASE_URL) as client:
headers_a = await create_user_and_login(client, "rev_a@test.com", "pass123")
headers_b = await create_user_and_login(client, "rev_b@test.com", "pass456")
# Get User A's due cards
resp_a = await client.get("/revision/due", headers=headers_a)
# Get User B's due cards
resp_b = await client.get("/revision/due", headers=headers_b)
# Cards should be completely separate
cards_a = {c["id"] for c in resp_a.json()}
cards_b = {c["id"] for c in resp_b.json()}
assert cards_a.isdisjoint(cards_b), "Revision cards overlap between users!"
@pytest.mark.asyncio
async def test_concept_isolation():
"""User A cannot access User B's knowledge map concepts."""
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url=BASE_URL) as client:
headers_a = await create_user_and_login(client, "concept_a@test.com", "pass123")
headers_b = await create_user_and_login(client, "concept_b@test.com", "pass456")
# User A creates a folder for concepts
resp = await client.post("/folders", json={"name": "Concepts A"}, headers=headers_a)
folder_a_id = resp.json()["id"]
# User B tries to get User A's knowledge map
resp = await client.get(f"/knowledge-map/folder/{folder_a_id}", headers=headers_b)
assert resp.status_code in [403, 404], "User B accessed User A's concepts!"What do these tests verify?
Each test creates two isolated users and checks a specific resource type. The tests cover folders, documents, chat history, revision cards, and knowledge map concepts.
If any endpoint is missing its user_id filter, the assertion fails with a clear message identifying the leak. Run these tests after every endpoint change.
- Run the test suite to check for isolation leaks:
pytest tests/test_data_isolation.py -v- Review the output. Any failed assertion means a data leak exists in that endpoint.
- For each failure, locate the endpoint in your FastAPI code and add a WHERE user_id = ? clause to its database query.
- Re-run the tests until all pass.
Tests failing with import errors?
Make sure httpx is installed in your virtual environment. You also need pytest-asyncio for async test support.
help me debug import errors in my FastAPI integration tests
Add onboarding flow for new users
A blank slate is confusing. When a new user registers, they should immediately understand what E-Hanatra offers and have a sensible starting structure in place.
You will create a default "Mes Cours" root folder on registration and build a welcome modal that introduces the four core features.
- In your FastAPI auth module, update the registration endpoint to create a default folder. Find the POST /auth/register handler and add the folder creation logic after the user is inserted:
# Inside your POST /auth/register endpoint, after creating the user:
# Create default folder structure for the new user
async with aiosqlite.connect(DB_PATH) as db:
await db.execute(
"INSERT INTO folders (id, name, parent_id, user_id, created_at, updated_at) VALUES (?, ?, NULL, ?, ?, ?)",
(str(uuid4()), "Mes Cours", new_user_id, datetime.utcnow().isoformat(), datetime.utcnow().isoformat())
)
await db.commit()Why create a default folder?
"Mes Cours" (French for "My Courses") gives every new Malagasy student a meaningful starting point. Without it, the folder tree is empty and the user does not know where to begin uploading documents.
- Now build the frontend welcome modal. Create a new component file called OnboardingModal.tsx in your components directory:
'use client'
import { useState } from 'react'
interface OnboardingModalProps {
isOpen: boolean
onClose: () => void
userName: string
}
export default function OnboardingModal({ isOpen, onClose, userName }: OnboardingModalProps) {
const [step, setStep] = useState(0)
// Features to introduce during onboarding
const features = [
{
title: 'Organisation par dossiers',
description: 'Organisez vos cours dans des dossiers et sous-dossiers hierarchiques. Glissez-deposez pour reorganiser.'
},
{
title: 'Chat contextuel',
description: 'Posez des questions sur un document, un dossier entier, ou toute votre base de connaissances.'
},
{
title: 'Cartes de connaissances',
description: 'Visualisez les liens entre concepts avec des cartes interactives generees automatiquement.'
},
{
title: 'Revision intelligente',
description: 'Des flashcards avec repetition espacee (SM-2) pour memoriser efficacement.'
}
]
if (!isOpen) return null
return (
<div className="fixed inset-0 bg-black/60 flex items-center justify-center z-50">
<div className="bg-gray-900 border border-gray-700 rounded-xl p-8 max-w-md w-full mx-4">
{step === 0 ? (
// Welcome screen
<div className="text-center">
<h2 className="text-2xl font-bold text-white mb-4">
Bienvenue, {userName}!
</h2>
<p className="text-gray-300 mb-6">
E-Hanatra est votre Second Brain intelligent. Decouvrons ensemble ses fonctionnalites.
</p>
<button
onClick={() => setStep(1)}
className="bg-indigo-600 hover:bg-indigo-700 text-white px-6 py-2 rounded-lg"
>
Commencer
</button>
</div>
) : step <= features.length ? (
// Feature walkthrough
<div>
<p className="text-indigo-400 text-sm mb-2">
{step} / {features.length}
</p>
<h3 className="text-xl font-semibold text-white mb-3">
{features[step - 1].title}
</h3>
<p className="text-gray-300 mb-6">
{features[step - 1].description}
</p>
<div className="flex justify-between">
<button
onClick={() => setStep(step - 1)}
className="text-gray-400 hover:text-white"
>
Retour
</button>
<button
onClick={() => setStep(step + 1)}
className="bg-indigo-600 hover:bg-indigo-700 text-white px-6 py-2 rounded-lg"
>
{step === features.length ? 'Terminer' : 'Suivant'}
</button>
</div>
</div>
) : (
// Completion screen with document import option
<div className="text-center">
<h3 className="text-xl font-semibold text-white mb-3">
Vous etes pret!
</h3>
<p className="text-gray-300 mb-6">
Un dossier "Mes Cours" a ete cree pour vous. Commencez par y deposer vos premiers documents.
</p>
<button
onClick={onClose}
className="bg-indigo-600 hover:bg-indigo-700 text-white px-6 py-2 rounded-lg"
>
Commencer a utiliser E-Hanatra
</button>
</div>
)}
</div>
</div>
)
}What does this component do?
The modal walks new users through four screens: a welcome greeting, then one screen per core feature (folders, contextual chat, knowledge maps, revision cards). The final screen confirms their default folder is ready and invites them to upload documents.
You will trigger this modal from the registration success handler, passing the new user's display name. Store a flag in localStorage to avoid showing it again on subsequent visits.
- In your registration page or AuthContext, trigger the onboarding modal after a successful registration by checking for a hasSeenOnboarding flag in localStorage.
- Save OnboardingModal.tsx and your auth module changes.
- Test by registering a new user. You should see the welcome modal appear after successful registration.
Modal not appearing after registration?
Check that the isOpen prop is being set to true on successful registration. Also verify the hasSeenOnboarding localStorage key is not already set from a previous test.
help me debug why my onboarding modal is not showing after registration
Add rate limiting, CORS, and production hardening
An unprotected auth endpoint is an invitation for brute-force attacks. Rate limiting caps how many login attempts a single IP can make. CORS restricts which frontends can call your API. Request logging gives you visibility into production traffic.
- Install the rate limiting library by running:
pip install slowapiWhat is slowapi?
slowapi is a rate limiting library for FastAPI adapted from Flask-Limiter. It uses in-memory tracking by default and supports Redis for distributed setups. You attach limit decorators directly to endpoints.
- Create a new file called middleware.py in your FastAPI project root. Add rate limiting, CORS, and logging configuration:
import os
import time
import logging
from fastapi import FastAPI, Request
from fastapi.middleware.cors import CORSMiddleware
from slowapi import Limiter, _rate_limit_exceeded_handler
from slowapi.util import get_remote_address
from slowapi.errors import RateLimitExceeded
# Configure logging
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger("e-hanatra")
# Rate limiter setup (5 requests per minute per IP on auth endpoints)
limiter = Limiter(key_func=get_remote_address)
def configure_middleware(app: FastAPI):
"""Attach all production middleware to the app."""
# Rate limiting
app.state.limiter = limiter
app.add_exception_handler(RateLimitExceeded, _rate_limit_exceeded_handler)
# CORS configuration for production
allowed_origins = os.getenv("ALLOWED_ORIGINS", "http://localhost:3000").split(",")
app.add_middleware(
CORSMiddleware,
allow_origins=allowed_origins,
allow_credentials=True,
allow_methods=["GET", "POST", "PUT", "PATCH", "DELETE"],
allow_headers=["Authorization", "Content-Type"],
)
# Request logging middleware
@app.middleware("http")
async def log_requests(request: Request, call_next):
start_time = time.time()
response = await call_next(request)
duration = time.time() - start_time
logger.info(
f"{request.method} {request.url.path} "
f"status={response.status_code} "
f"duration={duration:.3f}s "
f"ip={request.client.host}"
)
return responseWhat does this configuration do?
- The limiter uses get_remote_address to identify each client by IP. You will apply the @limiter.limit("5/minute") decorator on auth endpoints.
- CORS reads allowed origins from the ALLOWED_ORIGINS environment variable. In production, this would be your deployed frontend URL.
- The logging middleware records every request's method, path, status code, duration, and client IP.
- Apply rate limiting to your auth endpoints. In your auth router file, add the @limiter.limit("5/minute") decorator to both POST /auth/register and POST /auth/token:
from middleware import limiter
from fastapi import Request
@router.post("/auth/register")
@limiter.limit("5/minute")
async def register(request: Request, ...):
# existing registration logic
...
@router.post("/auth/token")
@limiter.limit("5/minute")
async def login(request: Request, ...):
# existing login logic
...Why the request parameter matters
slowapi requires the endpoint to accept a request: Request parameter so it can extract the client IP. If your endpoints already have this parameter, no changes are needed beyond adding the decorator.
- Call configure_middleware(app) in your main app file where you create the FastAPI instance:
from fastapi import FastAPI
from middleware import configure_middleware
app = FastAPI()
configure_middleware(app)- Create a .env file in your project root with the required environment variables:
SECRET_KEY=your-secret-key-here
GROQ_API_KEY=your-groq-api-key-here
ALLOWED_ORIGINS=http://localhost:3000- Finally, create the deployment checklist. Create a file called deployment-checklist.md in your project root:
# E-Hanatra Deployment Checklist
## Environment Variables
- [ ] SECRET_KEY: Set a strong random string (min 32 characters)
- [ ] GROQ_API_KEY: Your Groq API key for LLM calls
- [ ] ALLOWED_ORIGINS: Comma-separated list of frontend URLs (e.g. https://e-hanatra.example.com)
## Database
- [ ] Back up e-hanatra.db before deploying
- [ ] Verify all tables have user_id column
- [ ] Run data isolation tests (pytest tests/test_data_isolation.py)
- [ ] Enable WAL mode for concurrent reads: PRAGMA journal_mode=WAL
## ChromaDB
- [ ] Configure persistence volume (mount ./chroma_data to persistent storage)
- [ ] Verify all chunks have user_id in metadata
- [ ] Test that global search filters by user_id
## Security
- [ ] Rate limiting active on /auth/register and /auth/token (5/min/IP)
- [ ] CORS restricted to production frontend origins only
- [ ] JWT token expiry set appropriately (currently 1440 min / 24 hours)
- [ ] No secrets in source code or Git history
## Performance
- [ ] Request logging enabled for monitoring
- [ ] SQLite WAL mode for concurrent access
- [ ] ChromaDB collection indexes verifiedWhat does this checklist cover?
The checklist ensures nothing is forgotten during deployment. Environment variables must be set on the production server. The database needs a backup strategy and WAL mode for handling multiple concurrent users. ChromaDB requires a persistent volume so vector data survives container restarts.
- Save all files: middleware.py, your updated auth.py, main.py, .env, and deployment-checklist.md.
- Test rate limiting by sending more than 5 login requests within one minute. You should receive a 429 status code on the 6th attempt.
Rate limiter not blocking requests?
Ensure the @limiter.limit decorator is placed below the route decorator (e.g. @router.post), not above it. Also confirm that app.state.limiter is set before the app starts handling requests.
help me debug why slowapi rate limiting is not working on my FastAPI auth endpoints
Your E-Hanatra backend is now production-ready. Data isolation is verified by tests, new users get a smooth onboarding experience, and the API is protected against brute-force attacks with proper CORS boundaries. The deployment checklist gives you a repeatable process for going live.
Secret mission
Add a Study Analytics Dashboard
Your learning platform is feature-complete. But how do students know if they're actually improving? In this secret mission, you'll build a Study Analytics dashboard with GitHub-style learning heatmaps, concept mastery radar charts, study time tracking, exportable PDF reports, and intelligent knowledge gap detection.
Clean Up Your Resources
Clean Up Your Resources
This project runs primarily on your local machine with free-tier services. The Groq API free tier covers development usage at no cost, SQLite is embedded with zero overhead, and Whisper runs locally. If you chose to deploy to a hosting platform on a paid tier, ongoing costs may apply.
Cost warning
If you used the Groq Whisper API instead of the local model, transcription costs approximately $0.006 per minute of audio. If you deployed to a paid hosting tier (beyond the free plans on Vercel or Railway), those services may bill monthly. Check your hosting dashboard to confirm.
Resources you used:
- SQLite database file (e-hanatra.db in the FastAPI project root).
- ChromaDB vector store (./chroma_data directory).
- Whisper model cache (~/.cache/whisper/ directory, approximately 5 GB for the medium model).
- Node.js dependencies (node_modules in the Next.js frontend).
- Python virtual environment and installed packages.
- Service worker cached assets in your browser.
- Groq API account and API keys.
Decide whether to keep your resources running, pause them to come back later, or delete them entirely.
Keep everything running
No action needed. Choose this if you are still actively building or plan to continue adding features to E-Hanatra.
- Your FastAPI server, database, and ChromaDB store remain intact and ready to use.
- The Whisper model stays cached for fast transcription on future uploads.
- Your Groq API key remains active on the free tier at no cost.
Pause - I'll come back to this later
Shut down running processes to free up memory and system resources. Your files and data stay intact for when you return.
- Stop the FastAPI development server (press Ctrl+C in the terminal where it is running).
- Stop the Next.js development server (press Ctrl+C in its terminal).
- Your e-hanatra.db and ./chroma_data remain on disk, ready to pick up where you left off.
- The Whisper model cache at ~/.cache/whisper/ persists so you do not need to re-download it next time.
Delete - I don't want to use this again
Remove all project resources and start fresh. This permanently deletes your data, uploaded documents, revision cards, and analytics history.
- Delete the SQLite database by running:
rm e-hanatra.db- Remove the ChromaDB vector store directory by running:
rm -rf ./chroma_data- Remove the Whisper model cache (frees approximately 5 GB) by running:
rm -rf ~/.cache/whisper/- Remove the Node.js dependencies by running:
rm -rf node_modules- If you used a Python virtual environment, deactivate and remove it by running:
deactivate
rm -rf venv- Clear the service worker and cached assets from your browser. Open DevTools, navigate to the Application tab, select Service Workers, and click Unregister. Then go to Storage and click Clear site data.
- To delete your Groq account, visit privacy.groq.com, click Deletion Request, fill out the form, and click Request review. This permanently removes all API keys, usage history, and account data.
That's a Wrap!
That's a Wrap!
Congratulations! You've transformed E-Hanatra from a V1 prototype into a complete intelligent learning platform ready for the Malagasy student market.
You've learned how to:
- Build a hierarchical folder system with drag-and-drop organization and contextual RAG chat scoped to single documents, entire folders, or the full knowledge base.
- Generate interactive Knowledge Maps using React Flow and power active recall with SM-2 spaced repetition revision cards and multiple-choice quiz mode.
- Ship a production-ready app with a PWA for offline access, native Malagasy language support via Whisper, and JWT-based multi-user authentication with complete data isolation.
- Build a Study Analytics dashboard with weekly learning heatmaps, concept mastery radar charts, and exportable PDF progress reports.
Ready to quiz yourself?