Build a Free RAG Chatbot with LangChain

Build a RAG chatbot that answers questions from your own PDF documents.

Introduction

30 Second Summary

When a new mother has a health question at 2 AM, she needs reliable answers immediately. AI chatbots can provide that support, but only if they draw from trusted medical documents instead of making things up.

In this project, you will build a free AI-powered chatbot called MaaCare AI that answers maternal healthcare questions by reading your own PDF documents. You will use retrieval-augmented generation to ground every response in real source material.

What You'll Build

A Python RAG pipeline connected to a Flask web app that loads PDFs, stores them as vector embeddings in a local database, and answers questions using Groq's free Llama 3.1 API.

By the end of this project, you'll have:

  • A PDF processing pipeline that chunks documents and generates searchable vector embeddings locally.
  • A ChromaDB vector database stored on your machine for instant document retrieval with zero cloud costs.
  • A Flask web application with conversational memory that maintains chat context across messages.
  • Secret Mission: Add source citations to every response so users can trace answers back to specific PDF pages.

Are there any prerequisites?

You should have basic Python knowledge and be familiar with pip and virtual environments. No prior experience with AI, LangChain, or vector databases is needed.

Before We Start

Before We Start...

Set Up Your Development Environment

Before you can build MaaCare AI, you need a Python environment with specialized libraries. LangChain orchestrates the retrieval pipeline, ChromaDB stores vector embeddings locally, and Groq provides free LLM inference.

In this step, you'll verify your Python installation, create the project folder structure, install all required packages, and obtain a free API key from Groq.

In this step, get ready to:
  • Install Python and create your project folder with a virtual environment.
  • Install all required Python packages for the RAG pipeline.
  • Get a free Groq API key and configure your environment files.
Install Python and create your project folder

The packages in this project require Python 3.10 or higher. Let's confirm you have the right version installed.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or Command Prompt (Windows) and press Enter to open it.
  • Check your Python version by running this command:

macOS

python3 --version

Windows

python --version

✔️ I see version 3.10 or higher

Your Python version is ready. Move on to creating your project folder below.

ⓧ I see an older version

Your system has an older Python that won't work with this project's packages. You need 3.10 or higher.

  • Go to python.org/downloads and download the latest Python installer for your operating system.
  • Run the installer. On Windows, check the box that says Add python.exe to PATH before clicking Install Now.
  • Close and reopen your terminal, then run the version check again.

macOS users with an older default Python

macOS may ship with an older Python. After installing a newer version from python.org, you may need to use python3.12 (or whichever version you installed) instead of python3 until you update your PATH.

ⓧ Command not found

Python is not installed on your machine yet. Let's fix that.

  • Go to python.org/downloads and download the latest Python installer for your operating system.
  • Run the installer. On Windows, check the box that says Add python.exe to PATH before clicking Install Now.
  • Close and reopen your terminal, then run the version check again to confirm it works.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir maacare-ai
cd maacare-ai
  • Create the subdirectories your project needs by running:
mkdir src templates static data
  • Confirm the folders exist by running:
ls

You should see data, src, static, and templates listed in the output.

Now you need a virtual environment to keep this project's packages isolated from your system Python.

  • Create and activate your virtual environment by running these commands:

macOS

python3 -m venv venv
source venv/bin/activate

Windows

python -m venv venv
venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

  • Make sure you ran the activation command (the second line), not just the creation command.
  • On Windows, if you get a permissions error, run Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned in PowerShell first, then try the activate command again.
  • Help me activate my Python virtual environment.
  • Open your project in VS Code. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Select the maacare-ai folder on your Desktop, then click Open.
Install project dependencies

Your project needs several Python packages. A requirements.txt file lists them all in one place so you can install everything with a single command.

  • In VS Code, create a new file by pressing Cmd+N (macOS) or Ctrl+N (Windows).
  • Save the file immediately with Cmd+S (macOS) or Ctrl+S (Windows). Name it requirements.txt and save it in the root of your maacare-ai folder.
  • Paste the following package list into requirements.txt:
flask
python-dotenv
langchain
langchain-classic
langchain-groq
langchain-chroma
langchain-huggingface[full]
langchain-text-splitters
langchain-community
pypdf

What are these packages?

  • flask powers the web interface for your chatbot.
  • langchain and langchain-classic provide the RAG pipeline framework.
  • langchain-groq connects your pipeline to Groq's free LLM API.
  • langchain-chroma integrates with your local ChromaDB vector database.
  • langchain-huggingface[full] includes sentence-transformers for generating embeddings locally.
  • pypdf reads and extracts text from PDF documents.
  • Save requirements.txt.
  • In your terminal (with (venv) still active), install all the packages by running:
pip install -r requirements.txt

This will take a few minutes

Some of these packages are large. ChromaDB includes onnxruntime, and langchain-huggingface[full] downloads sentence-transformers. Expect the installation to take 2-5 minutes depending on your internet speed.

When it finishes, you should see a line starting with Successfully installed followed by a list of package names.

Installation failed?

  • Make sure (venv) appears in your terminal prompt. If it doesn't, re-run the activation command from the previous substep.
  • If you see a build error mentioning C++ or compilation, try upgrading pip first with pip install --upgrade pip and then re-run the install command.
  • Help me fix my pip install error.

✔️ Awesome, I've got everything!

Great. Make sure you've saved requirements.txt before moving on.

ⓧ I'd like to double check the full code

Here is the complete requirements.txt file:

flask
python-dotenv
langchain
langchain-classic
langchain-groq
langchain-chroma
langchain-huggingface[full]
langchain-text-splitters
langchain-community
pypdf
Get your free Groq API key and create environment files

Groq provides free access to fast LLM inference with no credit card required. You'll create an API key and store it in a .env file so your Python code can use it securely.

  • Open your browser and go to console.groq.com.
  • Sign up for a free account. No credit card is required.
  • Once logged in, click API Keys in the left sidebar.
  • Click Create API Key.
  • Give your key a name like maacare-ai and click Submit.
  • Copy the API key that appears. This is the only time you'll see it, so keep it in your clipboard.

What is Groq?

Groq is an AI inference provider that runs large language models at extremely high speed. Their free tier gives you 14,400 requests per day and 500,000 tokens per day for the llama-3.1-8b-instant model. That's more than enough for this project.

  • Back in VS Code, create a new file and save it as .env in the root of your maacare-ai folder.
  • Paste the following into .env, replacing your-groq-api-key-here with the key you just copied:
GROQ_API_KEY=your-groq-api-key-here

Why a .env file?

Storing your API key in a .env file keeps it out of your source code. The python-dotenv package loads it into your app at runtime. This way you never accidentally share your key if you push code to GitHub.

  • Save .env.
  • Create one more file and save it as .gitignore in the root of your maacare-ai folder.
  • Paste the following into .gitignore:
.env
chroma_db/
__pycache__/
*.pyc
venv/

What does .gitignore do?

This file tells Git to ignore certain files and folders. Your .env contains secrets, chroma_db/ will hold your vector database (which can be rebuilt), and venv/ contains installed packages that anyone can recreate from requirements.txt.

  • Save .gitignore.

Check the Explorer sidebar in VS Code. You should see .env, .gitignore, and requirements.txt at the root level alongside your four folders.

✔️ Awesome, I've got everything!

Great. Double check you've saved all three files before moving on.

ⓧ I'd like to double check the full code

Here is the complete .gitignore file:

.env
chroma_db/
__pycache__/
*.pyc
venv/

And the complete .env file (with your own key replacing the placeholder):

GROQ_API_KEY=your-groq-api-key-here

Don't see your .env file in VS Code?

  • Files starting with a dot are hidden by default on some systems. In VS Code's Explorer sidebar, they should still appear. If not, make sure you saved the file with the exact name .env (including the leading dot).
  • On macOS, Finder hides dotfiles. Use VS Code's sidebar or run ls -a in your terminal to confirm the file exists.
  • Help me find my .env file.

Your development environment is fully set up. Next up, you'll build the PDF processing pipeline that turns documents into searchable vector embeddings.

Build the PDF Processing and Embedding Pipeline

Your chatbot needs a knowledge base to answer questions from. Feeding entire PDFs directly to an LLM is expensive and limited by context window size.

Instead, you'll build a pipeline that breaks documents into small, searchable pieces. This is the core of Retrieval-Augmented Generation (RAG). You'll load PDFs, split them into chunks, convert those chunks into vector embeddings, and store everything in a local ChromaDB database.

In this step, get ready to:
  • Create helper functions for PDF loading, text splitting, and embedding generation.
  • Build a script that indexes your documents into a local ChromaDB vector database.
  • Run the pipeline and verify your database is created on disk.
Create the helper functions for your RAG pipeline

Your RAG pipeline needs three core operations: loading PDFs from a folder, splitting text into manageable chunks, and generating vector embeddings. You'll put all three into a reusable helper module.

  • First, make the src/ folder into a Python package by creating an empty __init__.py file. Run this command in your terminal:
touch src/__init__.py
  • Confirm the file exists by running:
ls src/

You should see __init__.py in the output.

Why do you need __init__.py?

Python uses __init__.py files to mark directories as packages. Without this file, Python won't recognise src/ as a module and your from src.helper import ... statements will fail.

  • Create a new file called helper.py inside the src/ folder.
  • Add the imports and the PDF loading function by pasting this code into src/helper.py:
from langchain_community.document_loaders import PyPDFDirectoryLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings


def load_pdf(data_path):
    loader = PyPDFDirectoryLoader(data_path)
    documents = loader.load()
    return documents

What does this code do?

  • The three imports bring in classes from LangChain's ecosystem: a PDF loader, a text splitter, and an embedding model wrapper.
  • The load_pdf() function takes a directory path and uses PyPDFDirectoryLoader to find all PDF files in that folder. It returns a list of Document objects, one per page.
  • Add the text splitting and embedding functions below the load_pdf function:
def text_split(extracted_data):
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
    text_chunks = text_splitter.split_documents(extracted_data)
    return text_chunks


def download_hugging_face_embeddings():
    embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
    return embeddings

What does this code do?

  • The text_split() function takes the loaded documents and breaks them into 500-character chunks with a 50-character overlap. The overlap ensures no context is lost at chunk boundaries.
  • The download_hugging_face_embeddings() function initialises the sentence-transformers/all-MiniLM-L6-v2 model. This is a free, open-source embedding model that runs entirely on your CPU.
  • Save src/helper.py.

You won't see a change yet. The next substep uses these functions to build your database.

Seeing a red underline on the imports?

If your editor shows import errors, make sure your virtual environment is activated. Your IDE might be using a different Python interpreter. Check your editor's Python interpreter setting and point it to the venv folder in your project.

Help me fix import errors in my editor for LangChain packages.

✔️ Awesome, I've got everything!

Great. Double check you've saved src/helper.py before moving on.

ⓧ I'd like to double check the full code

from langchain_community.document_loaders import PyPDFDirectoryLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings


def load_pdf(data_path):
    loader = PyPDFDirectoryLoader(data_path)
    documents = loader.load()
    return documents


def text_split(extracted_data):
    text_splitter = RecursiveCharacterTextSplitter(chunk_size=500, chunk_overlap=50)
    text_chunks = text_splitter.split_documents(extracted_data)
    return text_chunks


def download_hugging_face_embeddings():
    embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
    return embeddings
Create the indexing script

Now you need a script that uses your helper functions to actually build the vector database. This script loads your PDFs, chunks them, generates embeddings, and stores everything in a local ChromaDB database folder on disk.

  • Create a new file called store_index.py in the root of your project folder (at the same level as src/).
  • Paste the following code into store_index.py:
from src.helper import load_pdf, text_split, download_hugging_face_embeddings
from langchain_chroma import Chroma

extracted_data = load_pdf("data/")
text_chunks = text_split(extracted_data)

embeddings = download_hugging_face_embeddings()

print("Building your free local database... Please wait.")
db = Chroma.from_documents(text_chunks, embeddings, persist_directory="./chroma_db")
print(f"Local Database successfully built! {len(text_chunks)} chunks indexed.")

What does this code do?

  • It imports the three helper functions you just created, plus the Chroma class from langchain_chroma.
  • It loads all PDFs from the data/ folder, splits them into chunks, and initialises the embedding model.
  • The Chroma.from_documents() call generates an embedding for each chunk and stores everything in the ./chroma_db folder on your disk. This data persists automatically.
  • Save store_index.py.

Getting a ModuleNotFoundError?

If you see an error about langchain_chroma not being found, your virtual environment might not be activated. Activate it and confirm langchain-chroma is installed by checking your requirements.txt matches what you installed.

Help me fix ModuleNotFoundError for langchain_chroma in my project.

✔️ Awesome, I've got everything!

Great. Make sure store_index.py is saved before running it in the next substep.

ⓧ I'd like to double check the full code

from src.helper import load_pdf, text_split, download_hugging_face_embeddings
from langchain_chroma import Chroma

extracted_data = load_pdf("data/")
text_chunks = text_split(extracted_data)

embeddings = download_hugging_face_embeddings()

print("Building your free local database... Please wait.")
db = Chroma.from_documents(text_chunks, embeddings, persist_directory="./chroma_db")
print(f"Local Database successfully built! {len(text_chunks)} chunks indexed.")
Add a PDF and build the vector index

Your pipeline is ready. Now you need a PDF to feed it. Any text-based PDF will work. Choose a PDF that contains text you'd like the chatbot to answer questions about.

  • Find any text-based PDF on your computer (a research paper, a textbook chapter, or an article you've downloaded).
  • Copy or move that PDF file into the data/ folder inside your project directory.

Pick a text-heavy PDF

PDFs that are mostly images (like scanned documents) won't produce good results. Choose a PDF with selectable, copy-paste-able text for the best output.

  • Confirm your PDF is in the data/ folder by running:
ls data/

You should see your PDF filename listed in the output.

  • Run the indexing script to build your vector database:
python store_index.py

What should I expect?

The first run downloads the all-MiniLM-L6-v2 embedding model (around 80MB). This may take a minute depending on your internet speed.

After the download completes, you should see two lines printed:

  • Building your free local database... Please wait.
  • Local Database successfully built! X chunks indexed. (where X is the number of chunks from your PDF).
  • Verify the database was created on disk by running:
ls chroma_db/

You should see database files (like chroma.sqlite3 and other files) listed in the output. This confirms your embeddings are persisted locally.

Script finishes but shows 0 chunks indexed?

This means no text was extracted from your PDF. Make sure the PDF contains selectable text (not scanned images). Try opening the PDF and copying text from it. If you can't select text, the PDF needs OCR processing which is outside the scope of this project. Use a different PDF with selectable text.

Also confirm your PDF file is directly inside data/ and not in a subfolder. PyPDFDirectoryLoader only scans the immediate directory by default.

Help me debug why store_index.py shows 0 chunks indexed.

Your local vector database is ready. Next up, you'll create the custom system prompt that gives MaaCare AI its caring, knowledgeable personality.

Create the Custom System Prompt

Your vector database is loaded with PDF knowledge and ready to serve relevant chunks on demand. Now you need to tell the LLM how to use those chunks when answering a question.

A system prompt defines your chatbot's personality, tone, and boundaries. In this step, you will write a prompt that turns a generic LLM into MaaCare AI, a culturally aware maternal healthcare companion.

In this step, get ready to:
  • Create the system prompt file with MaaCare AI's identity and guidelines.
  • Structure the prompt with a context placeholder that LangChain fills at runtime.
Create the system prompt file

The system prompt is the instruction set your LLM reads before every response. It controls tone, language, topic boundaries, and safety disclaimers.

  • Create a new file called prompt.py inside your src/ folder.
  • Paste the following code into src/prompt.py:
system_prompt = (
    "You are 'MaaCare AI', a supportive, motherly, and highly knowledgeable postpartum healthcare companion "
    "for new mothers in Bangladesh. Use the following pieces of retrieved context to answer the user's question.\n\n"
    "Guidelines:\n"
    "1. Respond in a comforting, empathetic tone using a blend of Bengali and English (Banglish) or pure Bengali "
    "depending on how the user communicates.\n"
    "2. Focus heavily on localized postpartum needs (nutrition using local Bangladeshi food like shol mach, daal, kalojira, etc.).\n"
    "3. You are an AI, not a doctor. If a user asks about serious symptoms, always say: 'Please consult your doctor immediately.'\n"
    "4. If you do not know the answer from the provided context, say so honestly.\n\n"
    "Context:\n{context}"
)

What does this prompt do?

This single string defines the entire personality of your chatbot. Here is what each guideline controls:

  • Guideline 1 sets the language style. The chatbot responds in Banglish or Bengali depending on how the user writes.
  • Guideline 2 grounds nutrition advice in locally available Bangladeshi foods rather than generic Western dietary recommendations.
  • Guideline 3 is a critical safety boundary. The chatbot must redirect serious medical questions to a real doctor.
  • Guideline 4 prevents hallucination. If the retrieved context does not contain the answer, the chatbot says so honestly instead of making something up.
  • Save src/prompt.py.
  • Confirm the file exists by running:
ls src/

You should see __init__.py, helper.py, and prompt.py listed in the output.

Don't see prompt.py in the list?

  • Make sure you created the file inside the src/ folder, not in the project root.
  • Check the filename is exactly prompt.py with no extra spaces or capital letters.

Why isn't my prompt.py file showing up in src/?

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before moving on.

ⓧ I'd like to double check the full code

Here is the complete src/prompt.py for comparison:

system_prompt = (
    "You are 'MaaCare AI', a supportive, motherly, and highly knowledgeable postpartum healthcare companion "
    "for new mothers in Bangladesh. Use the following pieces of retrieved context to answer the user's question.\n\n"
    "Guidelines:\n"
    "1. Respond in a comforting, empathetic tone using a blend of Bengali and English (Banglish) or pure Bengali "
    "depending on how the user communicates.\n"
    "2. Focus heavily on localized postpartum needs (nutrition using local Bangladeshi food like shol mach, daal, kalojira, etc.).\n"
    "3. You are an AI, not a doctor. If a user asks about serious symptoms, always say: 'Please consult your doctor immediately.'\n"
    "4. If you do not know the answer from the provided context, say so honestly.\n\n"
    "Context:\n{context}"
)
Understand the context placeholder

Notice the {context} at the end of the prompt string. This is not decorative. It is a template variable that LangChain replaces at runtime with the actual document chunks retrieved from your ChromaDB vector database.

How does {context} get filled?

When a user asks a question, LangChain's retrieval chain searches your vector database for the most relevant chunks. It then injects those chunks into the {context} slot before sending the complete prompt to the LLM.

This is the core of Retrieval-Augmented Generation. The LLM answers using your actual PDF content instead of relying solely on its training data.

Why this approach over fine-tuning?

Fine-tuning permanently changes a model's weights, which is expensive and time-consuming. RAG keeps the model unchanged and simply feeds it the right documents at query time.

This means you can update your chatbot's knowledge by adding new PDFs to the data/ folder and re-running the indexing script. No retraining required.

Your chatbot now has a personality, cultural awareness, and safety guardrails baked into a single prompt string. Next up, you will wire everything together into a Flask application that retrieves documents, injects them into this prompt, and streams answers back to the user.

Build the Conversational RAG Application

Your vector database is indexed and your system prompt is ready. Now you need to connect these components into a working application that accepts user questions, retrieves relevant document chunks, and generates responses.

In this step, you'll create the Flask web application that ties everything together. The app connects to your ChromaDB database, sends retrieved context to Groq's free Llama 3.1 API, and maintains conversation history across messages.

In this step, get ready to:
  • Connect Flask to your ChromaDB vector database and initialize the Groq LLM.
  • Build the RAG chain that combines document retrieval with LLM generation.
  • Add conversational memory and create the API routes.
Set up Flask and connect to your vector database

The core of a RAG application is the connection between your vector database (where document chunks live) and the LLM (which generates answers). You'll wire these together in a single file called app.py.

  • Create a new file called app.py in your project root (the same level as store_index.py).
  • Add the following imports, Flask initialization, and database connection code:
from flask import Flask, render_template, jsonify, request
from dotenv import load_dotenv
from src.helper import download_hugging_face_embeddings
from src.prompt import system_prompt
from langchain_chroma import Chroma
from langchain_groq import ChatGroq
from langchain_classic.chains import create_retrieval_chain
from langchain_classic.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.chat_history import BaseChatMessageHistory, InMemoryChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory

# Initialize Flask and load environment variables
app = Flask(__name__)
load_dotenv()

# Load the same embedding model used during indexing
embeddings = download_hugging_face_embeddings()
# Connect to the persisted ChromaDB using those embeddings
vector_store = Chroma(persist_directory="./chroma_db", embedding_function=embeddings)
# Create a retriever that fetches the top 3 most relevant chunks
retriever = vector_store.as_retriever(search_kwargs={"k": 3})

# Initialize the Groq LLM with Llama 3.1
llm = ChatGroq(model="llama-3.1-8b-instant", temperature=0.4)

What does this code do?

This block sets up the foundation of your RAG app. Here's what each section accomplishes:

  • load_dotenv() reads your .env file so the Groq API key is available as an environment variable.
  • download_hugging_face_embeddings() loads the same embedding model you used in store_index.py. Both steps must use the same model so vectors are compatible.
  • Chroma(persist_directory="./chroma_db") opens the database you already built. No re-indexing needed.
  • search_kwargs={"k": 3} tells the retriever to return the 3 most relevant chunks per query.
  • ChatGroq connects to Groq's free API. The temperature=0.4 keeps responses focused while allowing some conversational warmth.
  • Save app.py.

Seeing a ModuleNotFoundError?

  • Make sure your virtual environment is activated. You should see its name in your terminal prompt.
  • Verify all packages installed correctly by running pip install flask python-dotenv langchain langchain-classic langchain-groq langchain-chroma langchain-huggingface[full] langchain-text-splitters langchain-community pypdf again.

Help me fix a ModuleNotFoundError in my RAG app

Build the RAG chain

A RAG chain combines two steps into one pipeline: retrieving relevant documents and passing them to the LLM with your prompt. LangChain provides two helper functions that handle this wiring for you.

  • In app.py, add the following code below the llm = ChatGroq(...) line:
# Build the prompt template with system prompt, chat history, and user input
qa_prompt = ChatPromptTemplate.from_messages([
    ("system", system_prompt),
    MessagesPlaceholder("chat_history"),
    ("human", "{input}"),
])

# Chain 1: takes retrieved docs + prompt and sends to the LLM
question_answer_chain = create_stuff_documents_chain(llm, qa_prompt)
# Chain 2: retrieves docs first, then passes them to chain 1
rag_chain = create_retrieval_chain(retriever, question_answer_chain)

What does this code do?

  • ChatPromptTemplate.from_messages() builds a prompt that includes your system prompt (with the {context} placeholder), the conversation history, and the user's current question.
  • create_stuff_documents_chain takes the retrieved document chunks and "stuffs" them into the {context} variable in your prompt before sending everything to the LLM.
  • create_retrieval_chain wraps both steps into a single pipeline. When invoked, it first retrieves relevant chunks, then passes them to the question-answer chain.
  • Save app.py.

You won't see a change yet. The next chunk adds conversational memory and the Flask routes that make the chain accessible.

Confused about the two chains?

Think of it as an assembly line. The retrieval chain fetches raw materials (document chunks). The stuff-documents chain assembles them into a finished product (an LLM prompt with context). Together they form one end-to-end pipeline.

Explain how create_retrieval_chain and create_stuff_documents_chain work together in my RAG app

Add conversational memory and Flask routes

A chatbot without memory forgets everything after each message. Conversational memory lets the chatbot reference earlier parts of the conversation, so users can ask follow-up questions like "Tell me more about that."

  • In app.py, add the conversational memory setup below the rag_chain line:
# In-memory store for chat histories (one per session)
store = {}


def get_session_history(session_id: str) -> BaseChatMessageHistory:
    # Create a new history if this session hasn't been seen before
    if session_id not in store:
        store[session_id] = InMemoryChatMessageHistory()
    return store[session_id]


# Wrap the RAG chain with message history support
conversational_rag_chain = RunnableWithMessageHistory(
    rag_chain,
    get_session_history,
    input_messages_key="input",
    history_messages_key="chat_history",
    output_messages_key="answer",
)

What does this code do?

  • store is a Python dictionary that holds chat histories keyed by session ID. Each user session gets its own conversation thread.
  • get_session_history() retrieves an existing history or creates a fresh one. This is how the chatbot "remembers" past messages.
  • RunnableWithMessageHistory wraps your RAG chain so that every invocation automatically loads and saves chat history. The three key parameters map to the prompt's input, history placeholder, and output field names.
  • Save app.py.

Now add the Flask routes that serve the chat page and handle incoming messages. Add this code below the conversational_rag_chain block:

  • Add the Flask routes and the main entry point at the bottom of app.py:
# Route: serve the chat page
@app.route("/")
def index():
    return render_template("chat.html")


# Route: handle chat messages via POST
@app.route("/get", methods=["POST"])
def chat():
    msg = request.form["msg"]
    response = conversational_rag_chain.invoke(
        {"input": msg},
        config={"configurable": {"session_id": "mother_user_1"}},
    )
    return jsonify({"response": response["answer"]})


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080, debug=True)

What does this code do?

  • The / route serves the HTML chat page (you'll create this in the next step).
  • The /get route receives the user's message via POST, invokes the full conversational RAG chain, and returns the LLM's answer as JSON.
  • session_id="mother_user_1" ties all messages to one conversation thread. In a production app, you'd generate unique session IDs per user.
  • port=8080 runs the server on port 8080 so it doesn't conflict with other local services.
  • Save app.py.
  • Confirm the file exists by running:
ls app.py

You should see app.py listed in the output.

Don't see app.py?

  • Make sure you created the file in your project root folder (the same directory that contains your `src/` folder and `store_index.py`).
  • Check that you saved the file after adding code. Unsaved files won't appear with the expected content.

Help me find where I saved my app.py file

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

from flask import Flask, render_template, jsonify, request
from dotenv import load_dotenv
from src.helper import download_hugging_face_embeddings
from src.prompt import system_prompt
from langchain_chroma import Chroma
from langchain_groq import ChatGroq
from langchain_classic.chains import create_retrieval_chain
from langchain_classic.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.chat_history import BaseChatMessageHistory, InMemoryChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory

app = Flask(__name__)
load_dotenv()

embeddings = download_hugging_face_embeddings()
vector_store = Chroma(persist_directory="./chroma_db", embedding_function=embeddings)
retriever = vector_store.as_retriever(search_kwargs={"k": 3})

llm = ChatGroq(model="llama-3.1-8b-instant", temperature=0.4)

qa_prompt = ChatPromptTemplate.from_messages([
    ("system", system_prompt),
    MessagesPlaceholder("chat_history"),
    ("human", "{input}"),
])

question_answer_chain = create_stuff_documents_chain(llm, qa_prompt)
rag_chain = create_retrieval_chain(retriever, question_answer_chain)

store = {}


def get_session_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in store:
        store[session_id] = InMemoryChatMessageHistory()
    return store[session_id]


conversational_rag_chain = RunnableWithMessageHistory(
    rag_chain,
    get_session_history,
    input_messages_key="input",
    history_messages_key="chat_history",
    output_messages_key="answer",
)


@app.route("/")
def index():
    return render_template("chat.html")


@app.route("/get", methods=["POST"])
def chat():
    msg = request.form["msg"]
    response = conversational_rag_chain.invoke(
        {"input": msg},
        config={"configurable": {"session_id": "mother_user_1"}},
    )
    return jsonify({"response": response["answer"]})


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080, debug=True)

Your application logic is complete. Next up, you'll build the chat interface and test the chatbot with real questions from your PDF documents.

Create the Chat Interface and Test

Your RAG backend is fully wired up with Flask routes, a ChromaDB retriever, and conversational memory.

Now you need a way to actually talk to your chatbot. In this step, you'll create a web-based chat interface and test the full pipeline end to end.

In this step, get ready to:
  • Create a chat interface with HTML, CSS, and JavaScript.
  • Run the Flask app and test contextual responses from your PDFs.
  • Verify conversational memory by asking follow-up questions.
Create the chat interface

Flask looks for HTML files inside a folder called templates/ at the same level as your app.py. The render_template("chat.html") call in your app.py already points to this file.

  • Create a new file called chat.html inside the templates/ folder of your project.
  • Paste the following HTML and CSS to set up the page structure and styling:
<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>MaaCare AI</title>
    <style>
        * { margin: 0; padding: 0; box-sizing: border-box; }
        body { font-family: 'Segoe UI', sans-serif; background: #f0f4f8; height: 100vh; display: flex; flex-direction: column; }
        .header { background: #4a90d9; color: white; padding: 1rem; text-align: center; }
        .header h1 { font-size: 1.5rem; }
        .chat-container { flex: 1; overflow-y: auto; padding: 1rem; max-width: 800px; margin: 0 auto; width: 100%; }
        .message { margin-bottom: 1rem; padding: 0.75rem 1rem; border-radius: 1rem; max-width: 80%; word-wrap: break-word; }
        .user-msg { background: #4a90d9; color: white; margin-left: auto; border-bottom-right-radius: 0.25rem; }
        .bot-msg { background: white; color: #333; border-bottom-left-radius: 0.25rem; box-shadow: 0 1px 3px rgba(0,0,0,0.1); }
        .input-area { padding: 1rem; background: white; border-top: 1px solid #ddd; }
        .input-area form { display: flex; max-width: 800px; margin: 0 auto; gap: 0.5rem; }
        .input-area input { flex: 1; padding: 0.75rem; border: 1px solid #ddd; border-radius: 0.5rem; font-size: 1rem; }
        .input-area button { padding: 0.75rem 1.5rem; background: #4a90d9; color: white; border: none; border-radius: 0.5rem; cursor: pointer; font-size: 1rem; }
        .input-area button:hover { background: #357abd; }
    </style>
</head>
<body>
    <!-- Header bar with app name -->
    <div class="header">
        <h1>MaaCare AI - Maternal Healthcare Companion</h1>
    </div>
    <!-- Scrollable chat message area -->
    <div class="chat-container" id="chatbox">
        <div class="message bot-msg">Assalamu Alaikum! I am MaaCare AI, your postpartum healthcare companion. How can I help you today?</div>
    </div>
    <!-- Input form pinned to the bottom -->
    <div class="input-area">
        <form id="chat-form">
            <input type="text" id="user-input" placeholder="Ask me anything about maternal health..." autocomplete="off">
            <button type="submit">Send</button>
        </form>
    </div>

What does this code do?

The CSS creates a full-height page layout with a blue header, a scrollable message area in the middle, and a fixed input bar at the bottom.

The .user-msg class aligns messages to the right with a blue background, while .bot-msg aligns to the left with a white card style. The chatbox starts with a greeting message so the page is not empty on first load.

  • Now add the JavaScript that handles sending messages and displaying responses. Paste this code directly below the closing </div> of the input area (before any closing </body> tag):
    <script>
        // Grab references to the chatbox, form, and input elements
        const chatbox = document.getElementById("chatbox");
        const form = document.getElementById("chat-form");
        const input = document.getElementById("user-input");

        form.addEventListener("submit", async (e) => {
            e.preventDefault();
            const msg = input.value.trim();
            if (!msg) return;

            // Display the user's message in the chat
            chatbox.innerHTML += `<div class="message user-msg">${msg}</div>`;
            input.value = "";
            chatbox.scrollTop = chatbox.scrollHeight;

            // Send the message to the Flask backend
            const formData = new FormData();
            formData.append("msg", msg);

            try {
                const res = await fetch("/get", { method: "POST", body: formData });
                const data = await res.json();
                chatbox.innerHTML += `<div class="message bot-msg">${data.response}</div>`;
            } catch (err) {
                chatbox.innerHTML += `<div class="message bot-msg">Sorry, something went wrong. Please try again.</div>`;
            }
            chatbox.scrollTop = chatbox.scrollHeight;
        });
    </script>
</body>
</html>

What does this JavaScript do?

When the learner submits the form, the script prevents the default page reload and grabs the typed message.

It appends the user's message to the chatbox, then sends it as FormData to the /get endpoint. The response JSON contains an answer field that gets displayed as a bot message. If the request fails, a friendly error message appears instead.

  • Save templates/chat.html.

File not found when running later?

Make sure your file is at templates/chat.html relative to app.py. Flask only looks in the templates/ folder by default. If you accidentally saved it in src/ or the project root, Flask will not find it.

help me fix a template not found error in Flask

✔️ Awesome, I've got everything!

Great. Make sure you have saved templates/chat.html before moving on.

ⓧ I'd like to double check the full code

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>MaaCare AI</title>
    <style>
        * { margin: 0; padding: 0; box-sizing: border-box; }
        body { font-family: 'Segoe UI', sans-serif; background: #f0f4f8; height: 100vh; display: flex; flex-direction: column; }
        .header { background: #4a90d9; color: white; padding: 1rem; text-align: center; }
        .header h1 { font-size: 1.5rem; }
        .chat-container { flex: 1; overflow-y: auto; padding: 1rem; max-width: 800px; margin: 0 auto; width: 100%; }
        .message { margin-bottom: 1rem; padding: 0.75rem 1rem; border-radius: 1rem; max-width: 80%; word-wrap: break-word; }
        .user-msg { background: #4a90d9; color: white; margin-left: auto; border-bottom-right-radius: 0.25rem; }
        .bot-msg { background: white; color: #333; border-bottom-left-radius: 0.25rem; box-shadow: 0 1px 3px rgba(0,0,0,0.1); }
        .input-area { padding: 1rem; background: white; border-top: 1px solid #ddd; }
        .input-area form { display: flex; max-width: 800px; margin: 0 auto; gap: 0.5rem; }
        .input-area input { flex: 1; padding: 0.75rem; border: 1px solid #ddd; border-radius: 0.5rem; font-size: 1rem; }
        .input-area button { padding: 0.75rem 1.5rem; background: #4a90d9; color: white; border: none; border-radius: 0.5rem; cursor: pointer; font-size: 1rem; }
        .input-area button:hover { background: #357abd; }
    </style>
</head>
<body>
    <div class="header">
        <h1>MaaCare AI - Maternal Healthcare Companion</h1>
    </div>
    <div class="chat-container" id="chatbox">
        <div class="message bot-msg">Assalamu Alaikum! I am MaaCare AI, your postpartum healthcare companion. How can I help you today?</div>
    </div>
    <div class="input-area">
        <form id="chat-form">
            <input type="text" id="user-input" placeholder="Ask me anything about maternal health..." autocomplete="off">
            <button type="submit">Send</button>
        </form>
    </div>
    <script>
        const chatbox = document.getElementById("chatbox");
        const form = document.getElementById("chat-form");
        const input = document.getElementById("user-input");

        form.addEventListener("submit", async (e) => {
            e.preventDefault();
            const msg = input.value.trim();
            if (!msg) return;

            chatbox.innerHTML += `<div class="message user-msg">${msg}</div>`;
            input.value = "";
            chatbox.scrollTop = chatbox.scrollHeight;

            const formData = new FormData();
            formData.append("msg", msg);

            try {
                const res = await fetch("/get", { method: "POST", body: formData });
                const data = await res.json();
                chatbox.innerHTML += `<div class="message bot-msg">${data.response}</div>`;
            } catch (err) {
                chatbox.innerHTML += `<div class="message bot-msg">Sorry, something went wrong. Please try again.</div>`;
            }
            chatbox.scrollTop = chatbox.scrollHeight;
        });
    </script>
</body>
</html>
Run the Flask app and test

With the interface ready, it is time to start your Flask app and see if the full RAG pipeline works.

  • Start the Flask server by running this command in your terminal (make sure your virtual environment is active):
python app.py

What should I see?

The first time you run this, the sentence-transformers/all-MiniLM-L6-v2 embedding model loads into memory. This may take 10-20 seconds.

Once ready, you should see output indicating the server is running on http://0.0.0.0:8080.

  • Open your browser and navigate to http://localhost:8080.
  • You should see the MaaCare AI chat interface with a blue header and the initial greeting message.
  • Type a question related to your PDF content into the input field and click Send.

The chatbot should reply with a contextual answer drawn from the PDF documents you indexed earlier.

Seeing an error or no response?

  • If the page does not load, confirm that the terminal shows the server running on port 8080 without errors.
  • If you see an import error, make sure your virtual environment is active and all packages from requirements.txt are installed.
  • If the bot returns empty answers, verify that chroma_db/ exists and contains database files from running store_index.py in the previous step.

help me debug my Flask RAG chatbot that is not returning answers

Test conversational memory

A key feature of your chatbot is that it remembers what you said earlier in the conversation. This is powered by the RunnableWithMessageHistory chain you set up in the previous step.

  • In the same chat session, ask a follow-up question that references the chatbot's previous answer.
  • For example, if you asked about postpartum nutrition and got a response, try asking "Tell me more about that" or "What else should I know?"
  • Verify that the chatbot's response builds on the earlier context rather than asking you to repeat yourself.

How does the memory work?

Each message you send is stored in an InMemoryChatMessageHistory object keyed by session ID. When your follow-up arrives, the full conversation history is injected into the prompt via the MessagesPlaceholder("chat_history") slot.

This means the LLM sees both the retrieved document context and your previous messages, allowing it to give coherent multi-turn answers.

Bot not remembering your previous question?

Memory is session-based. If you refreshed the page between questions, the JavaScript clears the displayed chat but the server-side memory persists under session ID mother_user_1. Try asking your follow-up without refreshing.

If the bot truly ignores context, confirm that conversational_rag_chain (not rag_chain) is being invoked in your /get route.

help me debug conversational memory in my LangChain RAG chatbot

Your MaaCare AI chatbot is live and responding with context-aware answers from your own documents. Up next, you can push it further by adding source citations so users know exactly where each answer came from.

Secret mission

Add Source Citations to Responses

Your chatbot answers questions from PDFs, but users have no idea where the information comes from. Can you add source citations so every response shows exactly which PDF page the answer was pulled from?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely on your local machine using Groq's free tier, so there are no ongoing cloud costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Project folder on your Desktop (source code, chroma_db/ vector database, venv/ virtual environment, data/ PDFs).
  • Groq API key (managed at console.groq.com/keys).

Keep everything running

No action needed. Choose this if you plan to keep experimenting with MaaCare AI, adding more PDFs, or customizing the system prompt.

  • Your chroma_db/ folder stays on disk with your indexed documents ready to query.
  • Your Groq API key remains active on the free tier with no charges.
  • Your Groq account persists at no cost.

Pause - I'll come back to this later

Stop the running Flask server to free up your terminal and port 8080. Your files and database stay intact for next time.

  • In the terminal running your Flask app, press Ctrl+C to stop the server.
  • Your project folder, virtual environment, and chroma_db/ database remain on disk.
  • When you return, activate your virtual environment and run python app.py to start the chatbot again.

Delete - I don't want to use this again

Remove all project files from your machine and revoke your API key.

  • In the terminal running your Flask app, press Ctrl+C to stop the server.
  • Deactivate your virtual environment by running:
deactivate
  • Navigate to your Desktop and delete the entire project folder by running:

macOS

cd ~/Desktop
rm -rf maacare-ai

Windows

cd ~\Desktop
rmdir /s /q maacare-ai

Folder name may differ

Replace maacare-ai with whatever you named your project folder in Step 1.

  • Revoke your Groq API key. Open console.groq.com/keys in your browser.
  • Find the API key you created for this project and delete it from the list.

Nice Work!

Nice Work!

You just built a complete, zero-cost RAG chatbot from scratch. MaaCare AI can read your PDF documents, store them as searchable vector embeddings, and answer questions with context-aware responses powered by a free LLM API.

You've learned how to:

  • Built a RAG pipeline that loads PDFs, chunks text with RecursiveCharacterTextSplitter, and generates vector embeddings locally using HuggingFace.
  • Created a persistent ChromaDB vector database stored on disk and connected it to Groq's free Llama 3.1 API for fast inference.
  • Deployed a Flask web application with conversational memory that maintains chat context across messages using LangChain's RunnableWithMessageHistory.
  • Added source citations so users can trace every answer back to specific PDF filenames and page numbers.

Ready to quiz yourself?