Build a Production RAG Chatbot Backend

Introduction

30 Second Summary

When you visit a website and ask a chatbot a question, the best ones give you answers grounded in real content rather than making things up. Behind the scenes, they retrieve relevant information from a database and use it to generate accurate responses.

In this project, you will build Hugo AI, a production-ready chatbot backend that answers questions about an organization by retrieving content from a vector database and generating responses with an LLM. You will deploy it as a live API that a frontend team can integrate into their website.

What You'll Build

A FastAPI backend that uses Retrieval-Augmented Generation (RAG) to answer questions strictly about the AWS Student Builder Group, powered by Groq's Llama 3.3 70B model and Supabase pgvector for semantic search.

By the end of this project, you'll have:

• A complete RAG pipeline that embeds queries, retrieves relevant chunks from a vector database, and generates scoped responses with source attribution.
• A scope-locked system prompt with confidence thresholding that refuses off-topic queries and only responds when it has relevant context.
• A live production deployment on Render with environment variables, a health check endpoint, and session memory for multi-turn conversations.
• Secret Mission: Add a feedback logging endpoint that records thumbs-up/down ratings to track response quality over time.

Are there any prerequisites?

You should have basic Python knowledge and familiarity with REST APIs. You will also need a GitHub account. No prior experience with AI, vector databases, or LangChain is required.

Set Up Your Development Environment

Before you can build the RAG pipeline, you need the right foundation in place. That means a Python environment, the correct libraries, and API keys for the services Hugo AI depends on.

In this step, you will install Python, create your project folder, pull in all the dependencies, and set up your Groq and Supabase accounts. By the end, your machine will be fully ready to start writing the actual chatbot code.

In this step, get ready to:

• Install Python and create a virtual environment.
• Install all project dependencies with pinned versions.
• Set up Groq and Supabase accounts and store your API keys.

Install Python and create a virtual environment

This project requires Python 3.10 or higher because LangChain needs it. Let's confirm you have the right version, then create an isolated project folder.

• Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
• Type Terminal and press Enter to open it.
• Check your Python version by running this command:

macOS

python3 --version

Windows

python --version

✔️ I see version 3.10 or higher

You are all set. Move on to creating your project folder below.

ⓧ I see an older version

Your system has an older Python that will not work with this project's dependencies. You need to install Python 3.10 or higher alongside it.

• Go to python.org/downloads and download the latest Python 3.11 installer for your operating system.
• Run the installer. On Windows, check the Add Python to PATH box before clicking Install Now.
• On macOS, the installer adds the new version to /usr/local/bin/. Use python3.11 explicitly if python3 still resolves to the old version.
• Close and reopen your terminal, then rerun the version check to confirm you see 3.10 or higher.

ⓧ Command not found

Python is not installed yet. Let's install it now.

• Go to python.org/downloads and download the latest Python 3.11 installer for your operating system.
• Run the installer. On Windows, check the Add Python to PATH box before clicking Install Now.
• On macOS, the installer automatically adds Python to your shell path.
• Close and reopen your terminal, then run the version check again to confirm the install succeeded.

• Move to your Desktop by running this command:

cd ~/Desktop

• Create your project folder and move into it by running these commands:

mkdir hugo-ai
cd hugo-ai

• Create a virtual environment by running this command:

macOS

python3 -m venv venv

Windows

python -m venv venv

What is a virtual environment?

A virtual environment is an isolated folder that holds its own copy of Python and any packages you install. This keeps your project's dependencies separate from other projects on your machine.

• Activate the virtual environment by running:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

• Confirm it is active. You should see (venv) at the beginning of your terminal prompt.

Don't see (venv) in your prompt?

On Windows, you may need to allow script execution first. Run Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned in PowerShell, then try the activate command again.

On macOS, make sure you typed source venv/bin/activate with the correct path and no typos.

Why can't I activate my virtual environment?

Install project dependencies

Hugo AI depends on several libraries. You will pin every version in a requirements.txt file so the build is reproducible.

• Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
• Type VS Code (or Cursor) and press Enter to open it.
• Click File in the top menu bar.
• Click Open Folder.
• Navigate to Desktop, select the hugo-ai folder, then click Open.
• Create a new file called requirements.txt by clicking the new file icon in the Explorer sidebar.
• Paste the following dependency list into requirements.txt:

fastapi[standard]==0.136.3
langchain==1.3.4
langchain-groq==1.1.2
sentence-transformers==3.4.1
supabase==2.31.0
python-dotenv==1.1.0
gunicorn==23.0.0

What are these packages?

• fastapi[standard] is the web framework that will serve your chatbot API. The [standard] extra bundles uvicorn for running the server.
• langchain and langchain-groq provide the chain orchestration and the Groq LLM integration.
• sentence-transformers runs the embedding model locally to convert text into vectors.
• supabase is the Python client for your vector database.
• python-dotenv loads environment variables from a .env file.
• gunicorn is the production WSGI server you will use when deploying to Render.

• Save requirements.txt.
• Install all dependencies by running this command in your terminal (make sure (venv) is still showing):

pip install -r requirements.txt

This download may take a few minutes

The sentence-transformers package pulls in PyTorch, which is a large download (several hundred MB). Let it finish completely before moving on.

• Verify the install succeeded by running:

pip list

• You should see fastapi, langchain, sentence-transformers, and supabase listed in the output along with their version numbers.

Seeing an error during pip install?

If you see a permissions error, confirm that (venv) appears in your prompt. Installing without the virtual environment active requires admin privileges.

If a package fails to build, make sure you are running Python 3.10 or higher. Some wheels are not available for older versions.

Help me fix my pip install error

Set up accounts and API keys

Hugo AI talks to two external services: Groq for LLM inference and Supabase for the vector database. Both offer free tiers that cover this project with no credit card required.

• Open your browser and go to console.groq.com.
• Sign up for a free account (no credit card needed).
• Once signed in, navigate to API Keys in the left sidebar.
• Click Create API Key.
• Give the key a name (e.g. hugo-ai) and click Submit.
• Copy the key immediately and save it somewhere safe. You will not be able to see it again after you close the dialog.

What is Groq?

Groq provides ultra-fast LLM inference. You will use their hosted llama-3.3-70b-versatile model to generate Hugo's responses. The free tier gives you 30 requests per minute and 1,000 requests per day for this model.

• Open a new tab and go to supabase.com/dashboard.
• Sign up or sign in (you can use GitHub to authenticate).
• Click New Project.
• Enter a project name (e.g. hugo-ai).
• Set a database password and save it somewhere secure.
• Select a region close to you, then click Create new project.
• Wait for provisioning to complete (about 2 minutes).

Now collect your Supabase credentials.

• In the Supabase dashboard, click Settings in the left sidebar.
• Click API Keys.
• Copy your Project URL (starts with https://). This is your SUPABASE_URL.
• Find the service_role key under Legacy API Keys and copy it. This is your SUPABASE_SERVICE_KEY.

Why the service_role key?

The service_role key has full access to your database and bypasses Row Level Security. Hugo's backend is a trusted server, so it needs this elevated access to read and write to the documents table. Never expose this key in frontend code.

Now store these keys in a .env file so your Python code can load them securely.

• In VS Code, create a new file called .env in the root of your hugo-ai folder.
• Paste the following into .env and replace each placeholder with the real values you just copied:

GROQ_API_KEY=your-groq-api-key-here
SUPABASE_URL=your-supabase-project-url-here
SUPABASE_SERVICE_KEY=your-supabase-service-role-key-here

Never commit your .env file

This file contains live secrets. If you plan to push this project to GitHub, add .env to your .gitignore immediately.

• Create one more file called .env.example so collaborators know which keys are needed (without the real values):

GROQ_API_KEY=your-groq-api-key-here
SUPABASE_URL=your-supabase-project-url-here
SUPABASE_SERVICE_KEY=your-supabase-service-role-key-here

• Save both files.

Can't find the service_role key?

In the Supabase dashboard, make sure you are looking under Settings then API Keys. The service_role key appears in the Legacy API Keys tab. Click the eye icon to reveal the full value before copying.

Help me find my Supabase service_role key

✔️ Awesome, I've got everything!

Great. Double check you have saved both requirements.txt and .env.example.

ⓧ I'd like to double check the full code

Here is what your requirements.txt should look like:

fastapi[standard]==0.136.3
langchain==1.3.4
langchain-groq==1.1.2
sentence-transformers==3.4.1
supabase==2.31.0
python-dotenv==1.1.0
gunicorn==23.0.0

And here is .env.example:

GROQ_API_KEY=your-groq-api-key-here
SUPABASE_URL=your-supabase-project-url-here
SUPABASE_SERVICE_KEY=your-supabase-service-role-key-here

Your environment is fully set up. You have Python, an isolated virtual environment, all dependencies installed, and live API keys ready to go. Next up, you will configure the Supabase vector database and seed it with sample content so Hugo has something to retrieve.

Configure Supabase pgvector and Seed Sample Data

Your development environment is ready, and your API keys are stored. Now you need to give Hugo something to search through.

In this step, you will set up the vector database that stores your organization's content as searchable embeddings. Without this, Hugo has no knowledge to draw from when answering questions.

In this step, get ready to:

• Enable pgvector and create the documents table in Supabase.
• Create a similarity search function that ranks documents by relevance.
• Seed the database with sample website content using a Python script.

Enable pgvector and create the documents table

Hugo stores website content as vector embeddings. To support this, you need to enable the pgvector extension in your Supabase database and create a table to hold the content chunks alongside their embeddings.

• Return to your Supabase project dashboard in your browser.
• Click SQL Editor in the left sidebar.
• Click New Query to open a blank editor.
• Enable the pgvector extension and create the documents table by pasting the following SQL:

-- Enable the pgvector extension for vector operations
create extension vector with schema extensions;

-- Create the documents table to store website content and embeddings
create table documents (
  id uuid primary key default gen_random_uuid(),
  content text,
  embedding extensions.vector(384),
  page_source text,
  section_title text,
  content_type text,
  last_updated timestamptz default now()
);

• Click Run or press Cmd+Enter (macOS) or Ctrl+Enter (Windows) to execute the query.

What does this SQL do?

The first line enables pgvector, a Postgres extension that adds vector data types and similarity operators to your database.

The table stores each chunk of website content alongside its 384-dimensional embedding (matching the output size of the all-MiniLM-L6-v2 model). The page_source and section_title columns let Hugo cite where it found its answers.

• You should see a Success. No rows returned message at the bottom of the editor.
• Click Table Editor in the left sidebar to verify the documents table now appears in your table list.

Not seeing the documents table?

Make sure you selected the public schema in the Table Editor (the dropdown at the top of the table list). If you see an error about the vector extension already existing, that is fine. It means pgvector was already enabled on your project.

Help me debug my Supabase SQL query.

Create the match_documents RPC function

When Hugo receives a question, it needs to find the most relevant content chunks. This SQL function accepts a query embedding and returns documents ranked by cosine similarity, filtering out anything below a confidence threshold.

• Switch back to the SQL Editor in the left sidebar.
• Click New Query to open a fresh editor tab.
• Create the similarity search function by pasting this SQL:

-- Create a function that finds documents similar to a query embedding
create or replace function match_documents (
  query_embedding extensions.vector(384),
  match_threshold float,
  match_count int
)
returns setof documents
language sql
as $$
  select *
  from documents
  where documents.embedding <=> query_embedding < 1 - match_threshold
  order by documents.embedding <=> query_embedding asc
  limit least(match_count, 200);
$$;

• Click Run or press Cmd+Enter (macOS) or Ctrl+Enter (Windows) to execute.

What does this function do?

The <=> operator calculates the cosine distance between two vectors. A smaller distance means higher similarity.

The function converts the match_threshold (a similarity score like 0.78) into a distance filter (1 - 0.78 = 0.22). Only documents within that distance are returned, ranked from most to least similar.

• You should see a Success. No rows returned message confirming the function was created.

Seeing a type error?

If you see an error mentioning a type mismatch or undefined vector type, make sure the first query (enabling the pgvector extension) ran successfully. You can re-run it before trying this function again.

Help me fix my match_documents function error.

Seed sample content into the database

The table and function are ready, but Hugo still has no content to search through. You will write a Python script that generates embeddings for sample website content and inserts them into your documents table.

• Switch back to your code editor.
• Create a new file called seed_data.py in your project folder.
• Add the imports and Supabase client setup by pasting this code:

import os

from dotenv import load_dotenv
from sentence_transformers import SentenceTransformer
from supabase import create_client

load_dotenv()

SUPABASE_URL = os.environ["SUPABASE_URL"]
SUPABASE_SERVICE_KEY = os.environ["SUPABASE_SERVICE_KEY"]

# Initialize the Supabase client and embedding model
supabase = create_client(SUPABASE_URL, SUPABASE_SERVICE_KEY)
embedding_model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

What does this code do?

This loads your environment variables from the .env file so the script can connect to Supabase. The SentenceTransformer model (all-MiniLM-L6-v2) converts text into 384-dimensional vectors. It runs locally on your machine with no API calls needed.

• Below the client setup, add the sample content that represents your website pages:

SAMPLE_CONTENT = [
    {
        "content": "The AWS Student Builder Group (AWSCC-HUGO) is a community of students passionate about cloud computing and technology. We host events, build projects, and help members grow their technical skills.",
        "page_source": "/",
        "section_title": "About Us",
        "content_type": "landing_page",
    },
    {
        "content": "To apply, visit our Apply Now page. Applications are open during recruitment periods. You can apply for positions in departments like AI/ML, Web Development, Cloud, and Design. Requirements include being an enrolled student and having enthusiasm for technology.",
        "page_source": "/apply",
        "section_title": "How to Apply",
        "content_type": "apply",
    },
    {
        "content": "Cloud101 is our introductory workshop on AWS services. It covers EC2, S3, and Lambda basics. The next session is scheduled for July 2026. Registration opens one week before the event on our Events page.",
        "page_source": "/events",
        "section_title": "Cloud101 Workshop",
        "content_type": "event",
    },
    {
        "content": "The AI/ML Department builds intelligent applications using machine learning. Current projects include Hugo AI (a RAG-powered chatbot) and a computer vision attendance system. The department is led by the AI/ML Tech Lead.",
        "page_source": "/core-team",
        "section_title": "AI/ML Department",
        "content_type": "core_team",
    },
    {
        "content": "FAQ: What is AWSCC-HUGO? It is the AWS Cloud Club - Holy Angel University student chapter. Who can join? Any enrolled student at the university. Is there a membership fee? No, membership is completely free.",
        "page_source": "/faq",
        "section_title": "Frequently Asked Questions",
        "content_type": "faq",
    },
]

What is this content?

Each dictionary represents a chunk of the AWSCC-HUGO website. In production, you would scrape and chunk actual website pages. For now, these five samples cover the main sections Hugo needs to know about: the landing page, applications, events, team info, and FAQ.

• Below the sample content, add the seed function that generates embeddings and inserts them:

def seed():
    print("Generating embeddings and seeding data...")
    for item in SAMPLE_CONTENT:
        # Convert text content into a 384-dimensional vector
        embedding = embedding_model.encode(item["content"]).tolist()

        # Upsert into Supabase (insert or update if content already exists)
        supabase.table("documents").upsert(
            {
                "content": item["content"],
                "embedding": embedding,
                "page_source": item["page_source"],
                "section_title": item["section_title"],
                "content_type": item["content_type"],
            },
            on_conflict="content",
        ).execute()

        print(f"  Seeded: {item['section_title']}")

    print("Done! All sample content seeded.")


if __name__ == "__main__":
    seed()

What does the seed function do?

For each content chunk, the script encodes the text into a vector using the local embedding model. It then upserts (insert or update) the row into the documents table. The on_conflict="content" parameter means running the script multiple times will update existing rows rather than creating duplicates.

• Save seed_data.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before running it.

ⓧ I'd like to double check the full code

import os

from dotenv import load_dotenv
from sentence_transformers import SentenceTransformer
from supabase import create_client

load_dotenv()

SUPABASE_URL = os.environ["SUPABASE_URL"]
SUPABASE_SERVICE_KEY = os.environ["SUPABASE_SERVICE_KEY"]

supabase = create_client(SUPABASE_URL, SUPABASE_SERVICE_KEY)
embedding_model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

SAMPLE_CONTENT = [
    {
        "content": "The AWS Student Builder Group (AWSCC-HUGO) is a community of students passionate about cloud computing and technology. We host events, build projects, and help members grow their technical skills.",
        "page_source": "/",
        "section_title": "About Us",
        "content_type": "landing_page",
    },
    {
        "content": "To apply, visit our Apply Now page. Applications are open during recruitment periods. You can apply for positions in departments like AI/ML, Web Development, Cloud, and Design. Requirements include being an enrolled student and having enthusiasm for technology.",
        "page_source": "/apply",
        "section_title": "How to Apply",
        "content_type": "apply",
    },
    {
        "content": "Cloud101 is our introductory workshop on AWS services. It covers EC2, S3, and Lambda basics. The next session is scheduled for July 2026. Registration opens one week before the event on our Events page.",
        "page_source": "/events",
        "section_title": "Cloud101 Workshop",
        "content_type": "event",
    },
    {
        "content": "The AI/ML Department builds intelligent applications using machine learning. Current projects include Hugo AI (a RAG-powered chatbot) and a computer vision attendance system. The department is led by the AI/ML Tech Lead.",
        "page_source": "/core-team",
        "section_title": "AI/ML Department",
        "content_type": "core_team",
    },
    {
        "content": "FAQ: What is AWSCC-HUGO? It is the AWS Cloud Club - Holy Angel University student chapter. Who can join? Any enrolled student at the university. Is there a membership fee? No, membership is completely free.",
        "page_source": "/faq",
        "section_title": "Frequently Asked Questions",
        "content_type": "faq",
    },
]


def seed():
    print("Generating embeddings and seeding data...")
    for item in SAMPLE_CONTENT:
        embedding = embedding_model.encode(item["content"]).tolist()

        supabase.table("documents").upsert(
            {
                "content": item["content"],
                "embedding": embedding,
                "page_source": item["page_source"],
                "section_title": item["section_title"],
                "content_type": item["content_type"],
            },
            on_conflict="content",
        ).execute()

        print(f"  Seeded: {item['section_title']}")

    print("Done! All sample content seeded.")


if __name__ == "__main__":
    seed()

Now run the script to populate your database.

• Run the seed script in your terminal:

python seed_data.py

First run takes a moment

The first time you run this, the sentence-transformers library downloads the all-MiniLM-L6-v2 model (~90 MB). This only happens once. Subsequent runs use the cached model and complete in a few seconds.

• You should see this output in your terminal:

Generating embeddings and seeding data...
  Seeded: About Us
  Seeded: How to Apply
  Seeded: Cloud101 Workshop
  Seeded: AI/ML Department
  Seeded: Frequently Asked Questions
Done! All sample content seeded.

• Switch back to your Supabase dashboard.
• Click Table Editor in the left sidebar.
• Select the documents table. You should see 5 rows with content, embeddings (displayed as arrays of numbers), page sources, and section titles.

Script failing to connect?

Make sure your virtual environment is activated (you should see the environment name in your terminal prompt). Double check that your .env file contains the correct SUPABASE_URL and SUPABASE_SERVICE_KEY values from your Supabase project settings.

Help me debug my seed_data.py connection error.

Your vector database is loaded and ready to answer questions. Next up, you will build the RAG pipeline that takes a user's question, searches these embeddings, and generates a grounded response using Groq's LLM.

Build the RAG Pipeline with LangChain

Your vector database is seeded and ready to serve results. Now you need the code that actually uses it.

In this step, you will build the RAG pipeline that converts a user's question into a vector, retrieves matching content from Supabase, and decides whether the results are confident enough to send to the LLM.

In this step, get ready to:

• Initialize the embedding model and Groq LLM client.
• Build retrieval functions that query your Supabase vector store.
• Implement confidence thresholding to prevent low-quality answers.

Set up the embedding model and Groq LLM

The RAG pipeline needs two AI components: an embedding model to convert text into vectors, and an LLM to generate human-readable responses from retrieved context.

• Create a new file called rag_pipeline.py in your project folder.
• Add the imports and client initialization by pasting this code:

import os

from langchain_groq import ChatGroq
from sentence_transformers import SentenceTransformer
from supabase import create_client

from prompts import build_prompt, FALLBACK_RESPONSE

# Load environment variables for service connections
SUPABASE_URL = os.environ["SUPABASE_URL"]
SUPABASE_SERVICE_KEY = os.environ["SUPABASE_SERVICE_KEY"]
GROQ_API_KEY = os.environ["GROQ_API_KEY"]

# Initialize the Supabase client for vector lookups
supabase = create_client(SUPABASE_URL, SUPABASE_SERVICE_KEY)

# Load the embedding model (runs locally, no API calls needed)
embedding_model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

What does this code do?

This block sets up the three external connections your pipeline needs.

• The supabase client connects to your vector database where the seeded documents live.
• The SentenceTransformer model (all-MiniLM-L6-v2) converts text into 384-dimensional vectors. It runs entirely on your CPU with no API calls.
• The from prompts import line references a file you will create in the next step. The code will not run until that file exists.

• Now add the LLM client and matching constants directly below the embedding model line:

# Initialize the Groq LLM for response generation
llm = ChatGroq(
    model="llama-3.3-70b-versatile",
    api_key=GROQ_API_KEY,
    temperature=0.3,
    max_tokens=1024,
)

# Similarity thresholds for retrieval quality control
MATCH_THRESHOLD = 0.78
MATCH_COUNT = 5

What does this code do?

ChatGroq is LangChain's official integration with the Groq inference API. It sends your prompt to the llama-3.3-70b-versatile model and returns the generated text.

The temperature=0.3 keeps responses focused and factual. The two constants control retrieval: MATCH_THRESHOLD (0.78) sets the minimum similarity score a chunk must have to be considered relevant, and MATCH_COUNT (5) limits how many chunks to retrieve per query.

• Save rag_pipeline.py.

You will not be able to run this file yet because prompts.py does not exist. You will create it in the next step.

Seeing a red underline on the prompts import?

Your editor may show an error on from prompts import build_prompt, FALLBACK_RESPONSE because the file does not exist yet. This is expected and will resolve after the next step.

If you see errors on other import lines (langchain_groq, sentence_transformers, supabase), make sure your virtual environment is activated.

Help me fix import errors in my rag_pipeline.py file.

Build the retrieval function

With the models initialized, you need functions that convert a user's question into a vector and retrieve matching document chunks from Supabase.

• Add the following two functions below MATCH_COUNT = 5 in rag_pipeline.py:

def embed_query(query: str) -> list[float]:
    # Convert the user's question into a 384-dim vector
    embedding = embedding_model.encode(query)
    return embedding.tolist()


def retrieve_chunks(query_embedding: list[float]) -> list[dict]:
    # Call the Supabase match_documents RPC for cosine similarity search
    result = supabase.rpc(
        "match_documents",
        {
            "query_embedding": query_embedding,
            "match_threshold": MATCH_THRESHOLD,
            "match_count": MATCH_COUNT,
        },
    ).execute()

    return result.data if result.data else []

What does this code do?

These two functions form the retrieval half of your RAG pipeline.

• embed_query uses the local SentenceTransformer model to encode the user's question into a 384-dimensional vector. The .tolist() call converts the numpy array into a plain Python list that Supabase accepts.
• retrieve_chunks sends that vector to the match_documents RPC function you created in Supabase. It returns up to 5 document chunks ranked by cosine similarity, already filtered by the 0.78 threshold.

• Save rag_pipeline.py.

You will not see output yet. The next function will tie everything together.

Not sure where to add the functions?

These two functions go after the MATCH_COUNT = 5 line, separated by two blank lines from the constants above. Make sure they are at the top level of the file (not indented inside another function).

Help me figure out where to place the embed_query and retrieve_chunks functions.

Implement confidence thresholding

Retrieving chunks is not enough. If the retrieved content is only loosely related to the question, the LLM will still try to generate an answer, often hallucinating details.

Confidence thresholding solves this by checking similarity scores before calling the LLM. If no chunk scores high enough, Hugo returns a safe fallback response instead of guessing.

• Add the main orchestration function below retrieve_chunks in rag_pipeline.py:

def get_rag_response(query: str, session_history: list[dict]) -> dict:
    # Step 1: Embed the user query
    query_embedding = embed_query(query)
    # Step 2: Retrieve matching chunks from the vector store
    chunks = retrieve_chunks(query_embedding)

    # If no chunks matched at all, return the fallback immediately
    if not chunks:
        return {
            "response": FALLBACK_RESPONSE,
            "sources": [],
            "confidence": 0.0,
        }

    # Step 3: Calculate the highest similarity score among retrieved chunks
    max_similarity = max(
        1 - chunk.get("embedding_distance", 1.0) for chunk in chunks
    ) if chunks else 0.0

    # If even the best chunk is below the threshold, skip the LLM
    if max_similarity < MATCH_THRESHOLD:
        return {
            "response": FALLBACK_RESPONSE,
            "sources": [],
            "confidence": max_similarity,
        }

    # Step 4: Build the prompt with context and history, then call the LLM
    prompt = build_prompt(
        query=query,
        context_chunks=chunks,
        session_history=session_history,
    )

    response = llm.invoke(prompt)

    # Step 5: Extract source metadata for attribution
    sources = [
        {
            "page_source": chunk.get("page_source", "Unknown"),
            "section_title": chunk.get("section_title", "Unknown"),
        }
        for chunk in chunks
    ]

    return {
        "response": response.content,
        "sources": sources,
        "confidence": max_similarity,
    }

What does this code do?

This is the core orchestration function that the API endpoint will call. It has multiple exit points to prevent bad answers.

• First exit: if retrieve_chunks returns nothing, Hugo responds with the fallback message immediately. No LLM call, no cost.
• Second exit: if the best chunk's similarity score is below 0.78, Hugo still declines. This catches queries that get results but not relevant results.
• Happy path: only when retrieval succeeds AND confidence is high enough does Hugo call the LLM with build_prompt (from prompts.py) and return the generated response with source attribution.

Why skip the LLM call on low confidence?

Every LLM call costs tokens and adds latency. If the retrieved context is not relevant, the LLM will either hallucinate an answer or produce something vague. Skipping the call entirely saves resources and gives the user an honest redirect instead.

• Save rag_pipeline.py.

This file will not run on its own until you create prompts.py in the next step. That is where Hugo's system prompt and the build_prompt function live.

Confused by the embedding_distance calculation?

Supabase's pgvector returns a distance value (lower is better), but your threshold is a similarity value (higher is better). The formula 1 - embedding_distance converts distance to similarity so the comparison works correctly.

Help me understand how the similarity score calculation works in get_rag_response.

✔️ Awesome, I've got everything!

Great. Double check you have saved rag_pipeline.py and move on to the next step.

ⓧ I'd like to double check the full code

Here is the complete rag_pipeline.py file. Compare yours against it:

import os

from langchain_groq import ChatGroq
from sentence_transformers import SentenceTransformer
from supabase import create_client

from prompts import build_prompt, FALLBACK_RESPONSE

SUPABASE_URL = os.environ["SUPABASE_URL"]
SUPABASE_SERVICE_KEY = os.environ["SUPABASE_SERVICE_KEY"]
GROQ_API_KEY = os.environ["GROQ_API_KEY"]

supabase = create_client(SUPABASE_URL, SUPABASE_SERVICE_KEY)

embedding_model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

llm = ChatGroq(
    model="llama-3.3-70b-versatile",
    api_key=GROQ_API_KEY,
    temperature=0.3,
    max_tokens=1024,
)

MATCH_THRESHOLD = 0.78
MATCH_COUNT = 5


def embed_query(query: str) -> list[float]:
    embedding = embedding_model.encode(query)
    return embedding.tolist()


def retrieve_chunks(query_embedding: list[float]) -> list[dict]:
    result = supabase.rpc(
        "match_documents",
        {
            "query_embedding": query_embedding,
            "match_threshold": MATCH_THRESHOLD,
            "match_count": MATCH_COUNT,
        },
    ).execute()

    return result.data if result.data else []


def get_rag_response(query: str, session_history: list[dict]) -> dict:
    query_embedding = embed_query(query)
    chunks = retrieve_chunks(query_embedding)

    if not chunks:
        return {
            "response": FALLBACK_RESPONSE,
            "sources": [],
            "confidence": 0.0,
        }

    max_similarity = max(
        1 - chunk.get("embedding_distance", 1.0) for chunk in chunks
    ) if chunks else 0.0

    if max_similarity < MATCH_THRESHOLD:
        return {
            "response": FALLBACK_RESPONSE,
            "sources": [],
            "confidence": max_similarity,
        }

    prompt = build_prompt(
        query=query,
        context_chunks=chunks,
        session_history=session_history,
    )

    response = llm.invoke(prompt)

    sources = [
        {
            "page_source": chunk.get("page_source", "Unknown"),
            "section_title": chunk.get("section_title", "Unknown"),
        }
        for chunk in chunks
    ]

    return {
        "response": response.content,
        "sources": sources,
        "confidence": max_similarity,
    }

Your RAG pipeline is structured and ready to generate responses. Next up, you will write the system prompt that tells Hugo who it is, what it can answer, and how to handle off-topic questions.

Engineer the System Prompt and Scope Enforcement

Your RAG pipeline can retrieve relevant chunks and detect when confidence is too low. But the LLM still needs clear instructions about who it is, what topics it covers, and how to refuse off-topic requests.

In this step, you will write a system prompt that locks Hugo to AWSCC-HUGO topics, create a prompt assembly function that injects retrieved context and conversation history, and verify that off-topic queries get declined.

In this step, get ready to:

• Write a system prompt with scope rules and few-shot enforcement examples.
• Build a prompt assembly function that formats context and history into a single LLM input.
• Test scope enforcement locally to confirm Hugo declines off-topic queries.

Write the Hugo system prompt

A system prompt is the hidden instruction set that tells an LLM how to behave. Hugo's system prompt defines its identity, the topics it can discuss, and exactly how to decline anything else.

• Create a new file called prompts.py in your project folder (the same folder as rag_pipeline.py).
• Add the system prompt string by pasting the following code:

SYSTEM_PROMPT = """You are Hugo, the official AI assistant for the AWS Student Builder Group (AWSCC-HUGO). You ONLY answer questions about the organization, including:
- Membership and applications
- Events (upcoming and past)
- Projects (ongoing and finished)
- Core team and department structure
- FAQ content
- Merchandise and membership tiers

RULES:
1. ONLY use the provided context to answer questions. Never make up information.
2. If the context does not contain enough information to answer, say so clearly and suggest where on the website the user might find it.
3. NEVER answer questions unrelated to the AWS Student Builder Group. Politely decline and redirect.
4. Always cite which page/section your answer comes from.
5. Keep responses concise, friendly, and helpful.

EXAMPLES OF SCOPE ENFORCEMENT:
User: "Write me a Python script"
Hugo: "I'm Hugo, the AWS Student Builder Group assistant! I can only help with questions about our organization - like membership, events, projects, and team info. For programming help, I'd suggest checking out our Projects page to see what tech stacks our teams use!"

User: "What's the weather today?"
Hugo: "I appreciate the question, but I'm specifically designed to help with AWS Student Builder Group topics! I can tell you about upcoming events, how to apply, our projects, or anything else about the org. What would you like to know?"

CONTEXT FROM WEBSITE:
{context}

CONVERSATION HISTORY:
{history}
"""

What does this prompt do?

The prompt has four sections that work together to keep Hugo on-topic.

• The opening paragraph defines Hugo's identity and lists the exact topics it covers (membership, events, projects, team info, FAQ, merchandise).
• The RULES section enforces behavior. Rule 1 prevents hallucination. Rule 3 forces off-topic refusals.
• The EXAMPLES section gives the LLM concrete patterns to follow when declining requests. This is called few-shot prompting.
• The {context} and {history} placeholders are where retrieved chunks and conversation history get injected at runtime.

• Save prompts.py.
• Confirm the file exists by running:

ls prompts.py

You should see prompts.py listed in the output.

Don't see prompts.py?

Make sure you created the file in the same folder as rag_pipeline.py. Run ls to check which files are in your current directory. If rag_pipeline.py is not listed either, navigate to your project folder first.

Help me find my project folder and create prompts.py in the right location.

Next, add the fallback response that Hugo returns when no relevant context is found.

• In prompts.py, add the fallback response below the SYSTEM_PROMPT string by pasting this code:

FALLBACK_RESPONSE = (
    "I don't have enough information to answer that confidently. "
    "Please check the AWSCC-HUGO website directly, or try rephrasing your question. "
    "I can help with topics like membership, events, projects, and team structure!"
)

What is this fallback for?

When the confidence thresholding in rag_pipeline.py detects that no retrieved chunks are relevant enough, it skips the LLM call entirely and returns this canned response. This saves API calls and prevents Hugo from guessing.

• Save prompts.py.

You will not see a visible change yet. The next chunk completes the module with the prompt assembly function.

Build the prompt assembly function

The system prompt has placeholder slots for context and history. The build_prompt function fills those slots with the actual retrieved chunks and conversation turns, then appends the user's question.

• In prompts.py, add the build_prompt function below FALLBACK_RESPONSE by pasting this code:

def build_prompt(
    query: str, context_chunks: list[dict], session_history: list[dict]
) -> str:
    # Format each retrieved chunk with its source metadata
    context_text = "\n\n".join(
        f"[Source: {chunk.get('page_source', 'Unknown')} - {chunk.get('section_title', 'Unknown')}]\n{chunk.get('content', '')}"
        for chunk in context_chunks
    )

    # Format the last 5 conversation turns as User/Hugo pairs
    history_text = "\n".join(
        f"User: {turn['user']}\nHugo: {turn['assistant']}"
        for turn in session_history[-5:]
    )

    # Fill the system prompt placeholders with actual content
    full_prompt = SYSTEM_PROMPT.format(
        context=context_text if context_text else "No relevant context found.",
        history=history_text if history_text else "No previous conversation.",
    )

    return f"{full_prompt}\n\nUser: {query}\nHugo:"

What does this function do?

• It joins all retrieved chunks into a single text block, prefixing each with its source page and section title so the LLM can cite them.
• It formats the last 5 conversation turns (trimmed from the full session history) to give the LLM short-term memory.
• It fills the {context} and {history} placeholders in SYSTEM_PROMPT using Python's .format() method.
• The final return appends the user's question and ends with Hugo: so the LLM knows to respond as Hugo.

• Save prompts.py.
• Verify the module imports correctly by running:

python -c "from prompts import build_prompt, FALLBACK_RESPONSE; print('prompts.py OK')"

You should see prompts.py OK printed in your terminal.

Seeing a SyntaxError or ImportError?

Check that your triple-quoted string in SYSTEM_PROMPT closes with three double quotes on its own line. Also confirm that build_prompt is defined at the module level (no extra indentation).

Help me fix my prompts.py import error.

✔️ Awesome, I've got everything!

Great. Double check you have saved prompts.py and move on to testing.

ⓧ I'd like to double check the full code

SYSTEM_PROMPT = """You are Hugo, the official AI assistant for the AWS Student Builder Group (AWSCC-HUGO). You ONLY answer questions about the organization, including:
- Membership and applications
- Events (upcoming and past)
- Projects (ongoing and finished)
- Core team and department structure
- FAQ content
- Merchandise and membership tiers

RULES:
1. ONLY use the provided context to answer questions. Never make up information.
2. If the context does not contain enough information to answer, say so clearly and suggest where on the website the user might find it.
3. NEVER answer questions unrelated to the AWS Student Builder Group. Politely decline and redirect.
4. Always cite which page/section your answer comes from.
5. Keep responses concise, friendly, and helpful.

EXAMPLES OF SCOPE ENFORCEMENT:
User: "Write me a Python script"
Hugo: "I'm Hugo, the AWS Student Builder Group assistant! I can only help with questions about our organization - like membership, events, projects, and team info. For programming help, I'd suggest checking out our Projects page to see what tech stacks our teams use!"

User: "What's the weather today?"
Hugo: "I appreciate the question, but I'm specifically designed to help with AWS Student Builder Group topics! I can tell you about upcoming events, how to apply, our projects, or anything else about the org. What would you like to know?"

CONTEXT FROM WEBSITE:
{context}

CONVERSATION HISTORY:
{history}
"""

FALLBACK_RESPONSE = (
    "I don't have enough information to answer that confidently. "
    "Please check the AWSCC-HUGO website directly, or try rephrasing your question. "
    "I can help with topics like membership, events, projects, and team structure!"
)


def build_prompt(
    query: str, context_chunks: list[dict], session_history: list[dict]
) -> str:
    context_text = "\n\n".join(
        f"[Source: {chunk.get('page_source', 'Unknown')} - {chunk.get('section_title', 'Unknown')}]\n{chunk.get('content', '')}"
        for chunk in context_chunks
    )

    history_text = "\n".join(
        f"User: {turn['user']}\nHugo: {turn['assistant']}"
        for turn in session_history[-5:]
    )

    full_prompt = SYSTEM_PROMPT.format(
        context=context_text if context_text else "No relevant context found.",
        history=history_text if history_text else "No previous conversation.",
    )

    return f"{full_prompt}\n\nUser: {query}\nHugo:"

Test scope enforcement locally

With prompts.py complete, the full RAG pipeline can now run end-to-end. Time to verify that Hugo correctly declines off-topic queries.

• Test an off-topic query by running this command:

python -c "
from dotenv import load_dotenv
load_dotenv()
from rag_pipeline import get_rag_response
result = get_rag_response('Write me a Python script', [])
print('OFF-TOPIC:', result['response'][:80])
print('Confidence:', result['confidence'])
"

What should I see?

The output should print the beginning of the FALLBACK_RESPONSE string and a confidence score of 0.0 (or very low). This means the vector search found no relevant chunks for an off-topic query, so Hugo declined without even calling the LLM.

This is your first layer of defense. The confidence threshold catches clearly irrelevant queries before they reach the language model.

• Now test an on-topic query to verify Hugo responds with grounded content:

python -c "
from dotenv import load_dotenv
load_dotenv()
from rag_pipeline import get_rag_response
result = get_rag_response('How do I apply to AWSCC-HUGO?', [])
print('ON-TOPIC:', result['response'][:200])
print('Sources:', result['sources'])
print('Confidence:', result['confidence'])
"

You should see a response that references the application process, with sources pointing to the /apply page and a confidence score above 0.78.

Two layers of scope enforcement

Hugo has two defenses against off-topic questions. The confidence threshold (in rag_pipeline.py) catches queries that do not match any stored content. The system prompt (in prompts.py) handles edge cases where chunks partially match but the question is still out of scope.

Together, these ensure Hugo never answers questions outside AWSCC-HUGO topics.

Seeing a connection error or timeout?

The on-topic test calls both Supabase and the Groq API. Check that your .env file contains valid values for GROQ_API_KEY, SUPABASE_URL, and SUPABASE_SERVICE_KEY. Also confirm your Supabase project is not paused (it pauses after 7 days of inactivity on the free tier).

Help me debug my RAG pipeline connection issue.

Your system prompt and scope enforcement are locked in. Next up, you will add session memory so Hugo can handle follow-up questions naturally.

Add Session Memory and Source Attribution

Your RAG pipeline can now retrieve context and generate scoped responses. But every query starts from scratch. If a user asks "Tell me about Cloud101" and then follows up with "When is it?", Hugo has no idea what "it" refers to.

In this step, you will add session memory so Hugo can track multi-turn conversations. You will also review how the pipeline already attaches source attribution to every response, then test the full conversational flow end-to-end.

In this step, get ready to:

• Build a session memory class that stores conversation turns with automatic expiry.
• Review how source attribution provides page and section references in every response.
• Test multi-turn conversations to verify follow-up questions resolve correctly.

Implement session-scoped memory

Chatbots feel unnatural when every message is treated as a brand new conversation. Session memory solves this by storing recent conversation turns in a dictionary keyed by a unique session ID. Each session expires after 30 minutes of inactivity, preventing stale context from accumulating.

• Create a new file called session_memory.py in your project folder (the same folder as rag_pipeline.py).
• Add the following code to build the SessionMemory class:

import time


class SessionMemory:
    def __init__(self, max_turns: int = 10, timeout_minutes: int = 30):
        # Store all sessions in a dictionary keyed by session_id
        self.sessions: dict[str, dict] = {}
        self.max_turns = max_turns
        self.timeout_seconds = timeout_minutes * 60

    def _is_expired(self, session_id: str) -> bool:
        # A session is expired if it doesn't exist or hasn't been active recently
        if session_id not in self.sessions:
            return True
        last_active = self.sessions[session_id]["last_active"]
        return (time.time() - last_active) > self.timeout_seconds

    def get_history(self, session_id: str) -> list[dict]:
        # Return conversation history, or empty list if session expired
        if self._is_expired(session_id):
            self.sessions.pop(session_id, None)
            return []
        return self.sessions[session_id]["history"]

    def add_turn(self, session_id: str, user_message: str, assistant_message: str):
        # Create a fresh session if this one expired or is new
        if self._is_expired(session_id):
            self.sessions[session_id] = {"history": [], "last_active": time.time()}

        # Append the new conversation turn
        self.sessions[session_id]["history"].append(
            {"user": user_message, "assistant": assistant_message}
        )
        self.sessions[session_id]["last_active"] = time.time()

        # Trim history to max_turns to keep memory bounded
        if len(self.sessions[session_id]["history"]) > self.max_turns:
            self.sessions[session_id]["history"] = self.sessions[session_id]["history"][
                -self.max_turns :
            ]

What does this code do?

• self.sessions is a dictionary where each key is a session ID and each value holds the conversation history plus a timestamp of last activity.
• _is_expired() checks whether 30 minutes have passed since the session's last interaction. If so, the session is treated as gone.
• get_history() returns the list of past turns for an active session. If the session expired, it cleans up the entry and returns an empty list.
• add_turn() appends a new user/assistant pair and trims the history to the last 10 turns, preventing unbounded memory growth.

• Save session_memory.py.
• Confirm the file exists by running:

ls session_memory.py

You should see session_memory.py listed in the output.

File not showing up?

• Make sure you created the file in the same folder as rag_pipeline.py and prompts.py. Run ls to see all files in your current directory.
• Check that you saved the file (not just created it). The file should not be empty.

Why can't I find my session_memory.py file?

Why in-memory instead of Redis?

For a student organization website with low concurrent traffic, an in-memory dictionary keeps the architecture simple. If the service restarts, sessions reset. This is acceptable for a chatbot where losing a few conversation turns is not critical.

For higher-traffic production apps, you would swap this for Redis or a database-backed store. The interface stays the same.

✔️ Awesome, I've got everything!

Great. Make sure you have saved session_memory.py before moving on.

ⓧ I'd like to double check the full code

import time


class SessionMemory:
    def __init__(self, max_turns: int = 10, timeout_minutes: int = 30):
        self.sessions: dict[str, dict] = {}
        self.max_turns = max_turns
        self.timeout_seconds = timeout_minutes * 60

    def _is_expired(self, session_id: str) -> bool:
        if session_id not in self.sessions:
            return True
        last_active = self.sessions[session_id]["last_active"]
        return (time.time() - last_active) > self.timeout_seconds

    def get_history(self, session_id: str) -> list[dict]:
        if self._is_expired(session_id):
            self.sessions.pop(session_id, None)
            return []
        return self.sessions[session_id]["history"]

    def add_turn(self, session_id: str, user_message: str, assistant_message: str):
        if self._is_expired(session_id):
            self.sessions[session_id] = {"history": [], "last_active": time.time()}

        self.sessions[session_id]["history"].append(
            {"user": user_message, "assistant": assistant_message}
        )
        self.sessions[session_id]["last_active"] = time.time()

        if len(self.sessions[session_id]["history"]) > self.max_turns:
            self.sessions[session_id]["history"] = self.sessions[session_id]["history"][
                -self.max_turns :
            ]

Add source attribution to responses

A chatbot that answers questions without citing where it got the information is hard to trust. Source attribution tells the user exactly which page and section each answer came from, so they can verify the information themselves.

Your rag_pipeline.py already handles this. Open the file and find the sources list near the end of the get_rag_response function:

    sources = [
        {
            "page_source": chunk.get("page_source", "Unknown"),
            "section_title": chunk.get("section_title", "Unknown"),
        }
        for chunk in chunks
    ]

    return {
        "response": response.content,
        "sources": sources,
        "confidence": max_similarity,
    }

How does source attribution work?

Every chunk stored in Supabase has page_source and section_title metadata attached to it. When the pipeline retrieves matching chunks, it extracts these fields into a clean list of source objects.

The final response always includes three fields: the generated text, the list of sources (page URL + section name), and the confidence score. This structure makes it easy for a frontend to display clickable source links beneath Hugo's answer.

Why return confidence alongside sources?

The confidence score (highest similarity from the retrieved chunks) lets the frontend decide how to display the response. A high-confidence answer could appear normally. A borderline answer could show a disclaimer. This gives the web team flexibility without requiring backend changes.

Test follow-up questions

The real test of session memory is whether Hugo can handle follow-up questions that rely on earlier context. You will simulate a multi-turn conversation in your terminal to confirm everything connects.

• Open a Python interactive session by running:

python

• Run the following commands one by one to simulate a multi-turn conversation:

from dotenv import load_dotenv
load_dotenv()

from rag_pipeline import get_rag_response
from session_memory import SessionMemory

# Create a session memory instance
memory = SessionMemory()
session_id = "test-session"

# First question: ask about Cloud101
result = get_rag_response("Tell me about Cloud101", memory.get_history(session_id))
memory.add_turn(session_id, "Tell me about Cloud101", result["response"])
print("Q1 Response:", result["response"])
print("Q1 Sources:", result["sources"])
print("---")

# Follow-up: "When is it?" should resolve to Cloud101
result2 = get_rag_response("When is it?", memory.get_history(session_id))
memory.add_turn(session_id, "When is it?", result2["response"])
print("Q2 Response:", result2["response"])
print("Q2 Sources:", result2["sources"])

What should I see?

The first response should describe Cloud101 (the introductory AWS workshop). The second response should reference the Cloud101 event schedule (July 2026) even though you only asked "When is it?" without repeating the event name.

Both responses should include source objects pointing to /events and Cloud101 Workshop. This confirms that session memory is feeding conversation context back into the prompt.

• Exit the Python session by typing:

exit()

Follow-up answer doesn't mention Cloud101?

• Make sure you called memory.add_turn() after the first query. Without storing the first turn, the session history is empty for the second query.
• Check that you used the same session_id for both calls. Different IDs mean different sessions with no shared history.
• If you see an error about missing environment variables, confirm your .env file is in the same folder and that load_dotenv() ran successfully.

My follow-up questions aren't using session context. Help me debug.

Hugo can now hold a conversation and cite its sources. Next up, you will wrap this pipeline in a FastAPI server with proper request/response models so the web team can call it over HTTP.

Build the FastAPI Endpoints

Your RAG pipeline retrieves relevant chunks and generates scoped responses. Your session memory tracks conversations across turns.

Now you need to expose all of this through an API that the web team can call. In this step, you will create a FastAPI application with a /chat endpoint and a /health endpoint, then verify the full flow locally.

In this step, get ready to:

• Create a /chat POST endpoint with Pydantic request/response models.
• Add a /health GET endpoint and CORS middleware.
• Test the full RAG flow locally using Swagger UI and curl.

Create the /chat POST endpoint

The /chat endpoint is Hugo's core interface. It accepts a user message and session ID, runs the full RAG pipeline, stores the turn in memory, and returns a structured response with sources and a confidence score.

You will define the API contract with Pydantic models first, then wire the endpoint handler to everything you built in the previous steps.

• Create a new file called main.py in your hugo-ai project folder.
• Add the following imports and app setup code:

import os
import time
from contextlib import asynccontextmanager

from dotenv import load_dotenv
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel

from rag_pipeline import get_rag_response
from session_memory import SessionMemory

# Load environment variables from .env
load_dotenv()

# Initialize session memory with 10-turn limit and 30-minute timeout
session_memory = SessionMemory(max_turns=10, timeout_minutes=30)


# Lifespan context manager for startup/shutdown events
@asynccontextmanager
async def lifespan(app: FastAPI):
    yield


# Create the FastAPI application
app = FastAPI(title="Hugo AI", version="1.0.0", lifespan=lifespan)

What does this code do?

load_dotenv() reads your .env file so your API keys are available as environment variables.

SessionMemory is instantiated once at module level so all requests share the same session store. The lifespan context manager is where you would add startup or shutdown logic (loading models, closing connections). For now it simply yields.

• Below the app creation line, add the Pydantic models that define your API contract:

# Request model: what the frontend sends
class ChatRequest(BaseModel):
    message: str
    session_id: str


# Individual source reference in the response
class SourceItem(BaseModel):
    page_source: str
    section_title: str


# Response model: what the API returns
class ChatResponse(BaseModel):
    response: str
    sources: list[SourceItem]
    confidence: float

What do these models do?

ChatRequest defines the shape of incoming POST requests. The frontend must send a message string and a session_id string.

ChatResponse defines what Hugo sends back. Every response includes the generated text, a list of source references (page and section), and a confidence score indicating how relevant the retrieved context was.

• Below the models, add the /chat POST endpoint:

@app.post("/chat", response_model=ChatResponse)
def chat(request: ChatRequest):
    # Retrieve conversation history for this session
    history = session_memory.get_history(request.session_id)

    # Run the full RAG pipeline
    result = get_rag_response(
        query=request.message,
        session_history=history,
    )

    # Store this turn in session memory
    session_memory.add_turn(
        session_id=request.session_id,
        user_message=request.message,
        assistant_message=result["response"],
    )

    # Build source objects from the result
    sources = [
        SourceItem(page_source=s["page_source"], section_title=s["section_title"])
        for s in result["sources"]
    ]

    return ChatResponse(
        response=result["response"],
        sources=sources,
        confidence=result["confidence"],
    )

What does the /chat endpoint do?

This handler orchestrates the full flow. It fetches the session's conversation history, passes the user's message to get_rag_response (which handles embedding, retrieval, thresholding, and LLM generation), then stores the new turn for follow-up context.

The response_model=ChatResponse parameter tells FastAPI to validate the output and generate automatic API documentation from the model schema.

• Save main.py.

Seeing import errors?

Make sure your virtual environment is activated. The imports from rag_pipeline import get_rag_response and from session_memory import SessionMemory rely on rag_pipeline.py and session_memory.py being in the same directory as main.py.

help me debug import errors in my FastAPI project

Add the /health GET endpoint and CORS middleware

A health check endpoint lets monitoring tools (and Render's deploy process) confirm the service is running. CORS middleware allows the frontend website to call your API from a different domain without the browser blocking it.

• In main.py, find this line:

app = FastAPI(title="Hugo AI", version="1.0.0", lifespan=lifespan)

• Add the CORS middleware directly below it:

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

What does CORS middleware do?

When the AWSCC-HUGO website (running on one domain) calls your API (running on a different domain), the browser blocks the request by default. CORS middleware adds headers that tell the browser "this API accepts requests from other origins."

Setting allow_origins=["*"] allows any frontend to call the API. In production, you would narrow this to the specific frontend domain.

• Now find the /chat endpoint (the line @app.post("/chat", response_model=ChatResponse)) and add the /health endpoint directly above it:

@app.get("/health")
def health_check():
    return {"status": "healthy", "version": "1.0.0", "service": "hugo-ai"}

What does the /health endpoint do?

This returns a simple JSON object confirming the service is alive. Render uses this to verify successful deployments. You can also use it for uptime monitoring tools that periodically ping the endpoint.

• Save main.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before moving on.

ⓧ I'd like to double check the full code

import os
import time
from contextlib import asynccontextmanager

from dotenv import load_dotenv
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
from pydantic import BaseModel

from rag_pipeline import get_rag_response
from session_memory import SessionMemory

load_dotenv()

session_memory = SessionMemory(max_turns=10, timeout_minutes=30)


@asynccontextmanager
async def lifespan(app: FastAPI):
    yield


app = FastAPI(title="Hugo AI", version="1.0.0", lifespan=lifespan)

app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)


class ChatRequest(BaseModel):
    message: str
    session_id: str


class SourceItem(BaseModel):
    page_source: str
    section_title: str


class ChatResponse(BaseModel):
    response: str
    sources: list[SourceItem]
    confidence: float


@app.get("/health")
def health_check():
    return {"status": "healthy", "version": "1.0.0", "service": "hugo-ai"}


@app.post("/chat", response_model=ChatResponse)
def chat(request: ChatRequest):
    history = session_memory.get_history(request.session_id)

    result = get_rag_response(
        query=request.message,
        session_history=history,
    )

    session_memory.add_turn(
        session_id=request.session_id,
        user_message=request.message,
        assistant_message=result["response"],
    )

    sources = [
        SourceItem(page_source=s["page_source"], section_title=s["section_title"])
        for s in result["sources"]
    ]

    return ChatResponse(
        response=result["response"],
        sources=sources,
        confidence=result["confidence"],
    )

Test the API locally

Before deploying, you need to confirm the full pipeline works end-to-end on your machine. FastAPI includes a built-in Swagger UI that lets you test endpoints interactively in your browser.

• Start the FastAPI development server by running this command:

uvicorn main:app --reload

What does this command do?

uvicorn is the ASGI server that runs your FastAPI app. main:app tells it to look for the app object in main.py. The --reload flag restarts the server automatically when you save code changes.

• You should see output indicating the server is running on http://127.0.0.1:8000.
• Open your browser and navigate to http://127.0.0.1:8000/docs to access the Swagger UI.

What is the Swagger UI?

FastAPI auto-generates interactive documentation at /docs. You can see all your endpoints, their request/response schemas, and test them directly from the browser without writing any client code.

• In the Swagger UI, click the GET /health endpoint.
• Click Try it out, then click Execute.
• You should see a JSON response with "status": "healthy" in the response body.

Now test the full RAG flow with a chat request.

• Open a new terminal window (keep the server running in the first one).
• Send a test chat request by running this command:

curl -X POST http://127.0.0.1:8000/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "How do I apply?", "session_id": "test-session-1"}'

• You should see a JSON response with a response field containing Hugo's answer about applying to the organization, a sources array with page and section references, and a confidence score above 0.78.

Test scope enforcement too

Try an off-topic request like "message": "Write me a Python script". Hugo should decline politely and redirect to organization topics. If the confidence score is below the threshold, you will get the fallback response instead of an LLM-generated answer.

Server not starting?

Make sure your virtual environment is activated and all dependencies from requirements.txt are installed. Check that your .env file is in the same directory as main.py with valid values for GROQ_API_KEY, SUPABASE_URL, and SUPABASE_SERVICE_KEY.

If you see a connection error from Supabase, confirm your Supabase project has not paused due to inactivity. Visit your Supabase dashboard and check the project status.

help me debug why my FastAPI uvicorn server won't start

Your API is working locally. Next up, you will deploy it to Render so the web team can integrate it into the live website.

Deploy to Render

Your chatbot works locally. Now it needs a home on the internet where the web team can actually call it.

In this step, you will prepare your project for production deployment, push it to GitHub, and deploy it to Render as a live web service. By the end, anyone with your URL can hit the API and get grounded, scoped responses from Hugo.

In this step, get ready to:

• Create deployment configuration files and push your project to GitHub.
• Deploy a Render web service from your GitHub repo with environment variables configured.
• Verify the live API responds correctly in production.

Prepare deployment files and push to GitHub

Render needs two things to deploy your app: a start command and your code on GitHub. You will create a Procfile that tells Render how to start your server, and a render.yaml blueprint that defines the full service configuration.

• Create a file called Procfile (no file extension) in your project root with the following content:

web: gunicorn main:app --workers 1 --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:$PORT

What does this Procfile do?

The Procfile tells Render which command to run when starting your service. It launches Gunicorn (a production-grade Python WSGI server) with a single Uvicorn worker that serves your FastAPI app.

The $PORT variable is automatically set by Render at runtime, so your server binds to the correct port.

• Save Procfile.
• Create a file called render.yaml in your project root with the following content:

services:
  - type: web
    name: hugo-ai
    runtime: python
    plan: starter
    buildCommand: pip install -r requirements.txt
    startCommand: gunicorn main:app --workers 1 --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:$PORT
    envVars:
      - key: GROQ_API_KEY
        sync: false
      - key: SUPABASE_URL
        sync: false
      - key: SUPABASE_SERVICE_KEY
        sync: false
      - key: PYTHON_VERSION
        value: "3.11.9"

What does this render.yaml do?

This file is a Render Blueprint. It defines your entire service configuration as code so deployments are reproducible.

• plan: starter selects the Starter instance ($7/month, 512 MB RAM). This plan never spins down, which means zero cold starts for production traffic.
• sync: false on the secret keys tells Render not to store actual values in code. You will enter them in the dashboard instead.
• PYTHON_VERSION pins the runtime to 3.11.9 so builds are consistent.

• Save render.yaml.
• Confirm both new files exist by running:

ls Procfile render.yaml

You should see both filenames listed in the output.

Don't see one of the files?

Make sure Procfile has no file extension (not Procfile.txt). Some editors add extensions automatically. Rename it if needed.

help me troubleshoot my missing Procfile or render.yaml

✔️ Awesome, I've got everything!

Great. Make sure both files are saved before moving on.

ⓧ I'd like to double check the full code

Here is the complete Procfile:

web: gunicorn main:app --workers 1 --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:$PORT

And the complete render.yaml:

services:
  - type: web
    name: hugo-ai
    runtime: python
    plan: starter
    buildCommand: pip install -r requirements.txt
    startCommand: gunicorn main:app --workers 1 --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:$PORT
    envVars:
      - key: GROQ_API_KEY
        sync: false
      - key: SUPABASE_URL
        sync: false
      - key: SUPABASE_SERVICE_KEY
        sync: false
      - key: PYTHON_VERSION
        value: "3.11.9"

Now you need to push your project to GitHub so Render can access it. First, create a .gitignore file to prevent your secrets and local files from being committed.

• Create a .gitignore file in your project root with the following content:

.env
__pycache__/
venv/
*.pyc

Why a .gitignore?

The .gitignore file tells Git which files to exclude from your repository. Your .env file contains API keys that must never be committed to a public repo.

• Save .gitignore.
• Go to github.com and create a new repository. Name it hugo-ai. Leave it empty (no README, no .gitignore template).
• Initialize your local repo and push by running these commands:

git init
git add .
git commit -m "Initial commit: Hugo AI RAG backend"
git branch -M main
git remote add origin https://github.com/[[GITHUB_USERNAME="your-github-username"]]/hugo-ai.git
git push -u origin main

What do these commands do?

• git init creates a new Git repository in your project folder.
• git add . stages all files (except those listed in .gitignore) for commit.
• git push -u origin main uploads your code to GitHub where Render can pull it.

• Refresh your GitHub repository page in the browser. You should see all your project files listed.

Push failed with authentication error?

GitHub no longer accepts password authentication. You need a Personal Access Token or SSH key. Go to GitHub Settings then Developer settings then Personal access tokens to generate one. Use it as your password when prompted.

help me authenticate with GitHub for pushing my repository

Deploy on Render

With your code on GitHub, you can now deploy it as a live web service on Render. The Starter plan costs $7/month and keeps your service running 24/7 with no cold starts.

• Go to render.com and sign up for an account (or sign in if you already have one).
• In the Render Dashboard, click + New in the top-right corner.
• Select Web Service.
• Connect your GitHub account if prompted. Render will redirect you to GitHub to authorize access, then redirect back.
• Find your hugo-ai repository in the list and click Connect.

The rest of the service creation form appears. Fill in the following fields:

• Name: hugo-ai.
• Language: Python.
• Branch: main.
• Build Command: pip install -r requirements.txt.
• Start Command: gunicorn main:app --workers 1 --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:$PORT.

• For the instance type, select Starter ($7/month).
• Scroll down to the Advanced section to add environment variables.
• Click Add Environment Variable three times to add the following keys and values:

• Key: GROQ_API_KEY / Value: paste your Groq API key.
• Key: SUPABASE_URL / Value: paste your Supabase project URL.
• Key: SUPABASE_SERVICE_KEY / Value: paste your Supabase service role key.

• Click Create Web Service.

Render kicks off the first build. You can watch the build logs on the service's Events page. The first build takes a few minutes because it downloads the sentence-transformers model (~90 MB).

Why Starter instead of Free?

Render's free tier spins down your service after 15 minutes of inactivity, causing a ~60-second cold start on the next request. For a chatbot that the web team integrates into a live website, this delay is unacceptable.

The Starter plan ($7/month, 512 MB RAM, 0.5 CPU) keeps your service running continuously. No spin-down, no cold starts.

Build failing?

Check the build logs on the Events page. Common causes:

• A missing file in your repo. Make sure requirements.txt, main.py, rag_pipeline.py, prompts.py, and session_memory.py were all pushed to GitHub.
• A typo in the build or start command. Compare against the values listed above.

help me debug my Render build failure for Hugo AI

Verify the live deployment

Once the deploy finishes, Render gives your service a public URL. Time to confirm the full RAG pipeline works in production.

• Copy your service URL from the Render dashboard. It looks like https://hugo-ai-xxxx.onrender.com.
• Test the health endpoint by running:

curl [[RENDER_URL="https://your-service-name.onrender.com"]]/health

You should see this response:

{"status": "healthy", "version": "1.0.0", "service": "hugo-ai"}

Getting a connection error?

The first deploy can take 3-5 minutes to finish. Check the Events page in your Render dashboard and wait for the deploy status to show Live.

help me troubleshoot my Render deployment not responding

• Test the full RAG pipeline by sending a chat request:

curl -X POST [[RENDER_URL="https://your-service-name.onrender.com"]]/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "How do I apply to AWSCC-HUGO?", "session_id": "test-001"}'

You should see a JSON response with Hugo's answer about the application process, a list of sources, and a confidence score above 0.78.

What should the response look like?

A successful response includes three fields: response (Hugo's answer citing the Apply page), sources (a list with page_source and section_title), and confidence (a float above 0.78).

This proves the full pipeline is working: embedding, vector retrieval, context injection, and LLM generation.

Getting an empty response or low confidence?

Double-check that your SUPABASE_URL and SUPABASE_SERVICE_KEY environment variables are correct in the Render dashboard. A wrong key means the retrieval step fails silently and returns no chunks.

help me debug why Hugo AI returns empty responses in production

Your RAG chatbot is live in production. The web team can now integrate this API into the AWSCC-HUGO website. Next up, you will add a feedback system to track response quality over time.

Secret mission

Add a Feedback Logging Endpoint

Hugo can answer questions, but how do you know if the answers are actually good? Build a feedback system that lets users rate responses with thumbs up or thumbs down, then query aggregated stats to track quality over time.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. The Render Starter plan costs $7/month, so leaving it active will incur charges.

Cost warning

Your Render Starter web service costs $7/month and will continue billing until you downgrade or delete it. Supabase and Groq are on free tiers with no ongoing charges.

Resources you used:

• Render Starter web service ($7/month for the hugo-ai service).
• Supabase project (free tier, pauses after 7 days of inactivity).
• Groq API key (free tier, no charges).
• Local project folder with Python virtual environment and source files.

Keep everything running

No action needed. Choose this if you plan to hand the API URL to your web team for integration or continue developing Hugo AI.

• Your Render service stays live and responsive at your .onrender.com URL. Cost: $7/month.
• Your Supabase project stays active on the free tier. Ping it at least once every 7 days to prevent it from pausing automatically.
• Your Groq API key remains active on the free tier with no charges.

Pause - I'll come back to this later

Shut down the paid service to stop billing, but keep everything intact for easy resumption later.

• Open the Render Dashboard and click on your hugo-ai service.
• Change the instance type from Starter to Free. This stops the $7/month charge.
• Your service will now spin down after 15 minutes of inactivity and take about 60 seconds to wake up on the next request. This is not suitable for production, but it preserves your deployment for free.
• Your Supabase project and Groq key need no changes. Both remain on free tiers.

Delete - I don't want to use this again

Remove all project resources and start fresh. This is permanent.

• Delete the Render service: open the Render Dashboard, click on your hugo-ai service, go to Settings, scroll to the bottom of the page, and click Delete Web Service. Type the service name to confirm.
• Delete the Supabase project: open your Supabase Dashboard, select your project, go to Project Settings, and click Delete project in the Danger Zone section. This permanently removes your database and all data.
• Revoke the Groq API key: open the GroqCloud Console, navigate to the API Keys page, and delete the key you created for this project.
• Delete your local project folder by running this command:

macOS

rm -rf ~/Desktop/hugo-ai

Windows

Remove-Item -Recurse -Force ~\Desktop\hugo-ai

Nice Work!

Nice Work!

Nice work! You just built and deployed a production RAG chatbot backend from scratch.

Hugo AI is live, grounded in real organization content, and ready for the web team to integrate into the AWSCC-HUGO website.

You learned how to:

• Build a complete RAG pipeline with vector embeddings, semantic retrieval from Supabase pgvector, and LLM generation via Groq.
• Engineer a scope-locked system prompt with confidence thresholding that only responds when relevant context exists and refuses off-topic queries.
• Deploy a live FastAPI backend on Render with session memory, source attribution, and environment configuration.
• Added a feedback logging system with thumbs-up/down ratings stored in Supabase for continuous response quality tracking.

Ready to quiz yourself?