Build a RAG Chatbot with Groq and Supabase

Build a RAG chatbot backend with vector search, FastAPI, and Groq.

Introduction

30 Second Summary

Ever asked a chatbot a question and gotten a completely made-up answer? Most AI chatbots hallucinate because they generate responses from their training data alone, with no way to check against real information.

In this project, you will build a RAG chatbot backend that only answers questions using content you control. You will store your content as vector embeddings in a database, retrieve the most relevant pieces when a user asks a question, and generate grounded responses using a large language model.

What You'll Build

A context-aware chatbot API that retrieves relevant content from a Supabase vector database and generates grounded responses using Groq's Llama 3.3 70B model, declining off-topic requests.

By the end of this project, you'll have:

  • A Supabase database with pgvector enabled, storing embedded content chunks and a semantic search function.
  • A Python ingestion script that chunks content, generates vector embeddings with all-MiniLM-L6-v2, and upserts to your database.
  • A FastAPI backend with a RAG pipeline and scope enforcement, deployed live on Render's free tier.
  • Secret Mission: Add session-based conversation memory so users can ask follow-up questions naturally.

Are there any prerequisites?

You should be comfortable with basic Python (functions, dictionaries, f-strings) and have a general familiarity with REST APIs. No prior experience with AI, vector databases, or deployment is needed.

Set Up Your Development Environment

Before you can build the RAG chatbot, you need a local development environment with the right tools and accounts in place.

In this step, you will install Python, create your project folder with a virtual environment, and sign up for the two cloud services that power the chatbot: Supabase for the vector database and Groq for ultra-fast LLM inference.

In this step, get ready to:
  • Install Python 3.11+ and verify it works.
  • Create a project folder with a virtual environment and install all dependencies.
  • Create Supabase and Groq accounts and configure your API keys.
Check your Python version

This project requires Python 3.11 or higher. Let's verify which version you have installed.

  • Open your terminal by pressing Cmd+Space (macOS) or the Windows key (Windows), typing Terminal (macOS) or Command Prompt (Windows), and pressing Enter.
  • Check your Python version by running this command:

macOS

python3 --version

Windows

python --version

✔️ I see version 3.11 or higher

You should see output like Python 3.11.x or higher. You are good to go.

ⓧ I see an older version

If you see a version lower than 3.11 (for example 3.9 or 3.10), you need to install a newer version. On macOS, the system Python is often outdated.

  • Download the latest Python 3.11+ installer from python.org/downloads.
  • Run the installer and follow the prompts.
  • After installation, close and reopen your terminal, then run the version check again.

macOS users with Homebrew

If you use Homebrew, you can run:

brew install python@3.11

After installing, use python3.11 --version to confirm the new version is available.

ⓧ Command not found

Python is not installed on your system yet.

  • Go to python.org/downloads and download the latest Python 3.11+ installer for your operating system.
  • Run the installer. On Windows, check the box that says Add Python to PATH before clicking Install Now.
  • After installation, close and reopen your terminal, then run the version check again.
Create your project folder and install dependencies

Your project needs its own folder and a virtual environment to keep dependencies isolated from your system Python.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir rag-chatbot
cd rag-chatbot
  • Create a virtual environment by running this command:

macOS

python3 -m venv venv

Windows

python -m venv venv

What is a virtual environment?

A virtual environment is an isolated Python installation inside your project folder. It keeps this project's packages separate from your system Python, so different projects can use different package versions without conflict.

  • Activate the virtual environment by running:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

Don't see (venv) in your prompt?

On Windows, you may need to allow script execution first. Run Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned then try activating again.

On macOS, make sure you typed source before the path. Without it, the command does nothing.

Help me activate my virtual environment

  • Create the requirements.txt file that lists all project dependencies. In your terminal, run:

macOS

cat > requirements.txt << 'EOF'
fastapi==0.115.12
uvicorn==0.34.2
sentence-transformers==3.4.1
supabase==2.13.0
httpx==0.28.1
python-dotenv==1.1.0
EOF

Windows

Create a new file called requirements.txt in your rag-chatbot folder using Notepad or any text editor, and paste this content:

fastapi==0.115.12
uvicorn==0.34.2
sentence-transformers==3.4.1
supabase==2.13.0
httpx==0.28.1
python-dotenv==1.1.0

Save the file inside the rag-chatbot folder.

What are these packages?

  • fastapi is the web framework for your chatbot API.
  • uvicorn is the server that runs your FastAPI app.
  • sentence-transformers generates vector embeddings from text locally.
  • supabase is the Python client for your vector database.
  • httpx makes HTTP requests to the Groq API.
  • python-dotenv loads environment variables from your .env file.
  • Install all the dependencies by running:
pip install -r requirements.txt

This may take a few minutes

The sentence-transformers package includes PyTorch, which is a large download. Expect 2-5 minutes depending on your internet speed.

Once complete, you should see a message ending with Successfully installed followed by a list of packages.

Seeing installation errors?

Make sure (venv) is visible in your prompt. If not, re-run the activate command from above.

If you see permission errors, you may be installing to the system Python instead of the virtual environment.

Help me fix my pip install errors

✔️ Awesome, I've got everything!

Great. Double check you have requirements.txt saved in your rag-chatbot folder.

ⓧ I'd like to double check the full code

Here is what your requirements.txt should contain:

fastapi==0.115.12
uvicorn==0.34.2
sentence-transformers==3.4.1
supabase==2.13.0
httpx==0.28.1
python-dotenv==1.1.0
Create your Supabase and Groq accounts

Your chatbot needs two external services. Supabase provides the vector database that stores your content, and Groq provides the LLM that generates responses.

  • Go to supabase.com and click Start your project.
  • Sign up with your GitHub account or email.
  • Once logged in, click New project.
  • Enter a project name (for example, rag-chatbot).
  • Set a database password. Save this somewhere safe.
  • Select a region close to you.
  • Click Create new project. Wait for the project to finish provisioning.

Now grab your Supabase credentials.

  • In your Supabase dashboard, open the Connect dialog at the top of your project page.
  • Copy the Project URL value. You will need this shortly.
  • Go to Settings in the left sidebar, then click API Keys.
  • Copy the anon key from the Legacy API Keys tab. This is your public key for client-side access.

What is the anon key?

The anon key is a public API key that identifies your project. It is safe to use in client-facing code because access is still controlled by Row Level Security policies on your tables.

Now create your Groq account for LLM access.

  • Go to console.groq.com and sign up with your Google or GitHub account.
  • Once logged in, click API Keys in the left sidebar.
  • Click Create API Key.
  • Give it a name like rag-chatbot and click Submit.
  • Copy the API key immediately. It will not be shown again.

Copy your Groq key now

Groq only shows your API key once at creation time. If you close the dialog without copying it, you will need to create a new key.

Having trouble creating a Groq API key?

Some users have reported issues with API key creation due to ad blockers or VPN software. Try disabling any ad blockers, turning off VPN software, or using an incognito browser window.

Help me create a Groq API key

Now store your credentials in a .env file so your code can access them securely.

  • Create the .env file in your rag-chatbot folder. In your terminal, run:

macOS

cat > .env << 'EOF'
SUPABASE_URL=your-supabase-url-here
SUPABASE_KEY=your-supabase-anon-key-here
GROQ_API_KEY=your-groq-api-key-here
EOF

Windows

Create a new file called .env in your rag-chatbot folder using Notepad or any text editor, and paste this content:

SUPABASE_URL=your-supabase-url-here
SUPABASE_KEY=your-supabase-anon-key-here
GROQ_API_KEY=your-groq-api-key-here

Save the file inside the rag-chatbot folder.

  • Replace your-supabase-url-here with the Project URL you copied from Supabase.
  • Replace your-supabase-anon-key-here with the anon key you copied from Supabase.
  • Replace your-groq-api-key-here with the API key you copied from Groq.

Keep your .env file private

Never commit your .env file to Git or share it publicly. It contains your private API keys. If you use Git, add .env to your .gitignore file.

  • Save your .env file.
  • Verify it exists by running:

macOS

cat .env

Windows

type .env

You should see your three environment variables printed to the terminal with the values you entered.

✔️ Awesome, I've got everything!

Great. Double check you have saved .env in your rag-chatbot folder with all three keys filled in.

ⓧ I'd like to double check the full code

Here is what your .env should look like (with your actual values in place of the placeholders):

SUPABASE_URL=your-supabase-url-here
SUPABASE_KEY=your-supabase-anon-key-here
GROQ_API_KEY=your-groq-api-key-here

Your development environment is ready. Next up, you will set up Supabase with the pgvector extension and create the database structure for semantic search.

Set Up Supabase with pgvector

Your chatbot needs a place to store content and find the most relevant pieces when someone asks a question.

In this step, you will set up a Supabase database with pgvector, a PostgreSQL extension that lets you store and search vector embeddings. This is what powers the semantic search behind your RAG pipeline.

In this step, get ready to:
  • Enable the pgvector extension in your Supabase project.
  • Create a documents table to store embedded content chunks.
  • Create a match_documents function for semantic similarity search.
Enable the pgvector extension

Standard PostgreSQL does not understand vector embeddings. The pgvector extension adds a vector data type and distance operators so your database can store and compare embeddings natively.

  • Return to your Supabase project dashboard in your browser.
  • Click SQL Editor in the left sidebar.
  • Click New Query to open a blank editor.
  • Enable the vector extension by pasting the following SQL and clicking Run (or pressing Cmd+Enter on macOS, Ctrl+Enter on Windows):
-- Enable the vector extension
create extension if not exists vector
with schema extensions;

What does this SQL do?

This enables the pgvector extension under the extensions schema in your Supabase project.

Once enabled, your database gains a vector data type that can hold arrays of floats (embeddings) and operators for measuring distance between them.

  • Confirm you see a Success. No rows returned message in the results panel below the editor.

Seeing an error?

Make sure you pasted the SQL exactly as shown. A missing semicolon at the end can cause a syntax error.

If you see a permissions error, confirm you are the project owner. Only project owners can enable extensions.

Why is my Supabase extension failing to enable?

Create the documents table

Your chatbot needs a table to hold the content chunks it searches through. Each row stores a piece of text, some metadata about where it came from, and a 384-dimensional embedding vector.

  • Clear the SQL Editor and paste the following SQL to create the table and a search index:
-- Create the documents table
create table documents (
  id bigint primary key generated always as identity,
  content text not null,
  metadata jsonb not null default '{}'::jsonb,
  embedding extensions.vector(384)
);

-- Create an index for faster similarity search
create index on documents
using hnsw (embedding extensions.vector_cosine_ops);

What does this SQL do?

  • id is an auto-incrementing primary key. You never need to set it manually.
  • content holds the actual text chunk that the chatbot can retrieve and reference.
  • metadata stores JSON about where the content came from (page name, section title). This lets the chatbot cite its sources.
  • embedding is a 384-dimensional vector column matching the output size of the all-MiniLM-L6-v2 model you will use later.
  • The HNSW index speeds up similarity searches by building an approximate nearest-neighbor graph instead of scanning every row.
  • Click Run.
  • Confirm you see Success. No rows returned.
  • Click Table Editor in the left sidebar. You should see the documents table listed with its four columns.

Don't see the documents table?

Make sure you ran both statements together (the create table and create index). If you only ran one, go back to the SQL Editor and run the other.

If you see an error about extensions.vector not existing, the pgvector extension was not enabled successfully in the previous substep. Re-run the extension SQL first.

Help me fix my Supabase documents table creation

Create the match_documents function

When a user asks a question, you need a fast way to find the most relevant content chunks. A PostgreSQL function handles this inside the database itself, returning results ranked by cosine similarity.

  • Switch back to the SQL Editor in the left sidebar.
  • Click New Query to open a fresh editor tab.
  • Create the similarity search function by pasting this SQL and clicking Run:
-- Create the match_documents function
create or replace function match_documents (
  query_embedding extensions.vector(384),
  match_threshold float,
  match_count int
)
returns table (
  id bigint,
  content text,
  metadata jsonb,
  similarity float
)
language sql stable
as $$
  select
    documents.id,
    documents.content,
    documents.metadata,
    1 - (documents.embedding <=> query_embedding) as similarity
  from documents
  where 1 - (documents.embedding <=> query_embedding) > match_threshold
  order by (documents.embedding <=> query_embedding) asc
  limit match_count;
$$;

What does this function do?

  • query_embedding is the 384-dimensional vector of the user's question. Your Python code will generate this before calling the function.
  • match_threshold filters out chunks that are not similar enough. Only results above this score are returned.
  • match_count limits how many results come back (top K).
  • The <=> operator calculates cosine distance. The function converts it to similarity with 1 - distance so higher scores mean more relevant matches.
  • Confirm you see Success. No rows returned.

Function creation failed?

Double-check that you pasted the entire block including the closing $$; at the end. Missing that delimiter causes a syntax error.

If you see an error mentioning documents not existing, run the table creation SQL from the previous substep first.

Help me debug my match_documents function in Supabase

Now save a local copy of all the SQL you just ran. This makes it easy to recreate the database setup if needed.

  • In your code editor, create a new file called setup.sql in your project folder.
  • Paste the following combined SQL into setup.sql:
-- Enable the vector extension
create extension if not exists vector
with schema extensions;

-- Create the documents table
create table documents (
  id bigint primary key generated always as identity,
  content text not null,
  metadata jsonb not null default '{}'::jsonb,
  embedding extensions.vector(384)
);

-- Create an index for faster similarity search
create index on documents
using hnsw (embedding extensions.vector_cosine_ops);

-- Create the match_documents function
create or replace function match_documents (
  query_embedding extensions.vector(384),
  match_threshold float,
  match_count int
)
returns table (
  id bigint,
  content text,
  metadata jsonb,
  similarity float
)
language sql stable
as $$
  select
    documents.id,
    documents.content,
    documents.metadata,
    1 - (documents.embedding <=> query_embedding) as similarity
  from documents
  where 1 - (documents.embedding <=> query_embedding) > match_threshold
  order by (documents.embedding <=> query_embedding) asc
  limit match_count;
$$;
  • Save setup.sql.

✔️ Awesome, I've got everything!

Great. Double check you have saved setup.sql in your project folder and that all three SQL statements ran successfully in your Supabase SQL Editor.

ⓧ I'd like to double check the full code

Here is the complete setup.sql file for reference:

-- Enable the vector extension
create extension if not exists vector
with schema extensions;

-- Create the documents table
create table documents (
  id bigint primary key generated always as identity,
  content text not null,
  metadata jsonb not null default '{}'::jsonb,
  embedding extensions.vector(384)
);

-- Create an index for faster similarity search
create index on documents
using hnsw (embedding extensions.vector_cosine_ops);

-- Create the match_documents function
create or replace function match_documents (
  query_embedding extensions.vector(384),
  match_threshold float,
  match_count int
)
returns table (
  id bigint,
  content text,
  metadata jsonb,
  similarity float
)
language sql stable
as $$
  select
    documents.id,
    documents.content,
    documents.metadata,
    1 - (documents.embedding <=> query_embedding) as similarity
  from documents
  where 1 - (documents.embedding <=> query_embedding) > match_threshold
  order by (documents.embedding <=> query_embedding) asc
  limit match_count;
$$;

Your database is ready to store and search vector embeddings. Next up, you will build the ingestion pipeline that fills this table with embedded content chunks.

Build the Content Ingestion Pipeline

Your Supabase database is ready to store vectors, but it has no content yet. Before the chatbot can answer questions, it needs a knowledge base to draw from.

In this step, you'll build an ingestion pipeline that takes raw text content, splits it into chunks, converts each chunk into a 384-dimensional vector embedding, and stores everything in your documents table. This is how RAG systems learn new information.

In this step, get ready to:
  • Write a Python script with sample content representing website pages.
  • Generate vector embeddings for each content chunk using sentence-transformers.
  • Upsert the embedded chunks into your Supabase database and verify the data.
Define the sample content for your chatbot

A RAG chatbot can only answer questions about content it has ingested. You'll create a Python script with sample content that represents pages from a fictional AWS Cloud Club website.

  • Create a new file called ingest.py in your project folder.
  • Add the imports and client setup by pasting this code into ingest.py:
import os
from dotenv import load_dotenv
from sentence_transformers import SentenceTransformer
from supabase import create_client

load_dotenv()

supabase = create_client(os.getenv("SUPABASE_URL"), os.getenv("SUPABASE_KEY"))
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

What does this code do?

load_dotenv() reads your .env file so your Supabase credentials are available as environment variables.

create_client connects to your Supabase project using those credentials. SentenceTransformer loads the all-MiniLM-L6-v2 model, which converts text into 384-dimensional vectors that capture semantic meaning.

What is a vector embedding?

A vector embedding is a list of numbers that represents the meaning of a piece of text. Similar text produces similar number patterns, which lets you find relevant content by comparing vectors instead of matching exact keywords.

  • Below the model setup line, add the sample content that your chatbot will learn from:
# Sample content representing website pages
CONTENT = [
    {
        "content": "The AWS Cloud Club is a student-led community focused on cloud computing, AI/ML, and software development. We host workshops, hackathons, and study groups throughout the academic year.",
        "metadata": {"source": "landing_page", "section": "About Us"}
    },
    {
        "content": "To apply for membership, visit our Apply Now page and fill out the application form. Applications are open at the start of each semester. You need to be a currently enrolled student.",
        "metadata": {"source": "apply_now", "section": "How to Apply"}
    },
    {
        "content": "Our Cloud Engineering department builds and maintains cloud infrastructure projects. The AI/ML department focuses on machine learning research and applications. The Web Development department handles our online presence.",
        "metadata": {"source": "core_team", "section": "Departments"}
    },
    {
        "content": "Cloud 101 Workshop is our beginner-friendly introduction to AWS services. It covers EC2, S3, and Lambda basics. The next session is scheduled for the first week of each month.",
        "metadata": {"source": "events", "section": "Cloud 101"}
    },
    {
        "content": "The AI Study Group meets every Wednesday at 6 PM in the Computer Science building, Room 204. We cover topics from basic neural networks to advanced LLM applications.",
        "metadata": {"source": "events", "section": "AI Study Group"}
    },
    {
        "content": "Open positions include Cloud Engineering Lead, AI/ML Research Assistant, and Web Developer. All positions require a minimum commitment of 5 hours per week.",
        "metadata": {"source": "apply_now", "section": "Open Positions"}
    },
    {
        "content": "Our FAQ: Membership is free for all enrolled students. You do not need prior cloud experience to join. We provide all learning resources and AWS credits for projects.",
        "metadata": {"source": "faq", "section": "General FAQ"}
    },
]

What does this content represent?

Each dictionary has two fields: content (the actual text the chatbot will search through) and metadata (the source page and section title for attribution).

In a production system, you would scrape real web pages or load documents from a CMS. For this project, the hardcoded content lets you focus on the RAG pipeline itself.

Add the chunking and ingestion functions

Large documents need to be split into smaller chunks before embedding. This ensures each vector represents a focused piece of information rather than an entire page of mixed topics.

  • Below the CONTENT list, add the chunking function:
def chunk_content(text, chunk_size=500, overlap=50):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks

Why use overlapping chunks?

The overlap parameter (50 characters) ensures that sentences sitting at chunk boundaries aren't cut off and lost. If a key phrase spans two chunks, the overlap preserves it in at least one of them.

With chunk_size=500, our sample content (each under 500 characters) produces one chunk per document. Longer documents would be split into multiple overlapping pieces.

  • Below chunk_content, add the main ingestion function and the script entry point:
def ingest():
    """Embed and upsert all content into Supabase."""
    print("Starting ingestion...")

    for i, item in enumerate(CONTENT):
        chunks = chunk_content(item["content"])

        for j, chunk in enumerate(chunks):
            embedding = model.encode(chunk).tolist()

            record = {
                "content": chunk,
                "metadata": item["metadata"],
                "embedding": embedding,
            }

            supabase.table("documents").upsert(record).execute()
            print(f"  Ingested chunk {j+1} from {item['metadata']['source']}/{item['metadata']['section']}")

    print(f"\nIngestion complete! {len(CONTENT)} documents processed.")


if __name__ == "__main__":
    ingest()

What does the ingestion function do?

  • model.encode(chunk).tolist() converts the text chunk into a 384-number list (the vector embedding).
  • The record dictionary bundles the original text, its metadata, and the embedding together.
  • supabase.table("documents").upsert(record).execute() inserts the record into your documents table (or updates it if it already exists).
  • Save ingest.py.

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before running it in the next substep.

ⓧ I'd like to double check the full code

import os
from dotenv import load_dotenv
from sentence_transformers import SentenceTransformer
from supabase import create_client

load_dotenv()

supabase = create_client(os.getenv("SUPABASE_URL"), os.getenv("SUPABASE_KEY"))
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

# Sample content representing website pages
CONTENT = [
    {
        "content": "The AWS Cloud Club is a student-led community focused on cloud computing, AI/ML, and software development. We host workshops, hackathons, and study groups throughout the academic year.",
        "metadata": {"source": "landing_page", "section": "About Us"}
    },
    {
        "content": "To apply for membership, visit our Apply Now page and fill out the application form. Applications are open at the start of each semester. You need to be a currently enrolled student.",
        "metadata": {"source": "apply_now", "section": "How to Apply"}
    },
    {
        "content": "Our Cloud Engineering department builds and maintains cloud infrastructure projects. The AI/ML department focuses on machine learning research and applications. The Web Development department handles our online presence.",
        "metadata": {"source": "core_team", "section": "Departments"}
    },
    {
        "content": "Cloud 101 Workshop is our beginner-friendly introduction to AWS services. It covers EC2, S3, and Lambda basics. The next session is scheduled for the first week of each month.",
        "metadata": {"source": "events", "section": "Cloud 101"}
    },
    {
        "content": "The AI Study Group meets every Wednesday at 6 PM in the Computer Science building, Room 204. We cover topics from basic neural networks to advanced LLM applications.",
        "metadata": {"source": "events", "section": "AI Study Group"}
    },
    {
        "content": "Open positions include Cloud Engineering Lead, AI/ML Research Assistant, and Web Developer. All positions require a minimum commitment of 5 hours per week.",
        "metadata": {"source": "apply_now", "section": "Open Positions"}
    },
    {
        "content": "Our FAQ: Membership is free for all enrolled students. You do not need prior cloud experience to join. We provide all learning resources and AWS credits for projects.",
        "metadata": {"source": "faq", "section": "General FAQ"}
    },
]


def chunk_content(text, chunk_size=500, overlap=50):
    """Split text into overlapping chunks."""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunks.append(text[start:end])
        start = end - overlap
    return chunks


def ingest():
    """Embed and upsert all content into Supabase."""
    print("Starting ingestion...")

    for i, item in enumerate(CONTENT):
        chunks = chunk_content(item["content"])

        for j, chunk in enumerate(chunks):
            embedding = model.encode(chunk).tolist()

            record = {
                "content": chunk,
                "metadata": item["metadata"],
                "embedding": embedding,
            }

            supabase.table("documents").upsert(record).execute()
            print(f"  Ingested chunk {j+1} from {item['metadata']['source']}/{item['metadata']['section']}")

    print(f"\nIngestion complete! {len(CONTENT)} documents processed.")


if __name__ == "__main__":
    ingest()
Run the ingestion script

With the script complete, it's time to embed your content and store it in Supabase.

First-time model download

The first time you run this script, sentence-transformers downloads the all-MiniLM-L6-v2 model from Hugging Face (approximately 80 MB). This is a one-time download that gets cached locally for future runs.

  • Run the ingestion script in your terminal:
python ingest.py
  • Wait for the script to finish. You should see output listing each ingested chunk, ending with:
Starting ingestion...
  Ingested chunk 1 from landing_page/About Us
  Ingested chunk 1 from apply_now/How to Apply
  Ingested chunk 1 from core_team/Departments
  Ingested chunk 1 from events/Cloud 101
  Ingested chunk 1 from events/AI Study Group
  Ingested chunk 1 from apply_now/Open Positions
  Ingested chunk 1 from faq/General FAQ

Ingestion complete! 7 documents processed.
  • Verify the data in Supabase by switching to your Supabase dashboard.
  • Click **Table Editor** in the left sidebar.
  • Select the documents table.
  • You should see 7 rows, each with content, metadata, and a long embedding array.

Script not completing?

  • If you see a connection error mentioning Hugging Face, check your internet connection. The model must download on first run.
  • If you see an authentication error from Supabase, double-check that your SUPABASE_URL and SUPABASE_KEY values in .env are correct. Copy them fresh from your Supabase project settings.
  • If you see a 404 error on the table, make sure you created the documents table in the previous step using the SQL Editor.

Still stuck? Help me debug my ingest.py script. I'm getting an error when trying to embed content and upsert it to Supabase.

Your knowledge base is populated and ready for retrieval. Next up, you'll build the FastAPI endpoint that queries these embeddings and generates answers using Groq.

Build the RAG Pipeline and FastAPI Endpoint

Your Supabase database is loaded with embedded content chunks. Now it's time to build the API that ties the entire RAG pipeline together.

In this step, you'll create a FastAPI backend that takes a user's question, embeds it, retrieves relevant chunks from Supabase, injects them into a system prompt, and sends everything to Groq's LLM for a grounded answer.

In this step, get ready to:
  • Set up the FastAPI app with configuration, system prompt, and request/response models.
  • Write the retrieval and generation functions that power the RAG flow.
  • Wire up the /chat endpoint with confidence thresholding and test locally.
Set up the FastAPI app foundation

The app needs to initialize the same embedding model used during ingestion, connect to Supabase, and define constants for the Groq API.

  • Create a new file called main.py in your project folder.
  • Add the imports, app initialization, and API constants by pasting this code:
import os
from dotenv import load_dotenv
from fastapi import FastAPI
from pydantic import BaseModel
from sentence_transformers import SentenceTransformer
from supabase import create_client
import httpx

load_dotenv()

# Initialize the FastAPI app
app = FastAPI(title="Hugo AI - RAG Chatbot")

# Connect to Supabase and load the embedding model
supabase = create_client(os.getenv("SUPABASE_URL"), os.getenv("SUPABASE_KEY"))
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

# Groq API configuration
GROQ_API_KEY = os.getenv("GROQ_API_KEY")
GROQ_API_URL = "https://api.groq.com/openai/v1/chat/completions"
SIMILARITY_THRESHOLD = 0.3
TOP_K = 5

What does this code do?

  • FastAPI(title="Hugo AI - RAG Chatbot") creates the web application with a title that shows in the auto-generated docs.
  • The SentenceTransformer model is the same all-MiniLM-L6-v2 used during ingestion. Queries must be embedded with the same model to get meaningful similarity scores.
  • SIMILARITY_THRESHOLD = 0.3 is the minimum cosine similarity score a chunk needs to be considered relevant. TOP_K = 5 limits retrieval to the 5 most similar chunks.

Next, you need the system prompt that instructs the LLM to stay grounded in the retrieved context, and the fallback message for when no relevant content is found.

  • Add the system prompt, fallback response, and request/response models below the constants in main.py:
SYSTEM_PROMPT = """You are Hugo, the AWS Cloud Club assistant. You ONLY answer questions about the organization using the context provided below.

Rules:
1. Only use information from the provided context to answer questions.
2. If the context does not contain relevant information, say: "I don't have information about that. Please check our website directly or ask a club officer."
3. Never make up information or answer questions outside the scope of the organization.
4. Be friendly, concise, and helpful.
5. When possible, mention which page or section the information comes from.

Context:
{context}
"""

FALLBACK_RESPONSE = "I don't have information about that topic. I can only answer questions about the AWS Cloud Club \u2014 membership, events, projects, positions, and our team. Is there something specific about the club I can help you with?"


# Request and response schemas
class ChatRequest(BaseModel):
    message: str


class ChatResponse(BaseModel):
    response: str
    sources: list[dict]
    confidence: float

What does this code do?

SYSTEM_PROMPT defines the LLM's personality and rules. The {context} placeholder gets filled with retrieved chunks at runtime. This is the core of prompt engineering for RAG.

FALLBACK_RESPONSE is returned when no relevant chunks are found. This prevents the LLM from hallucinating answers about topics outside your content.

ChatRequest and ChatResponse use Pydantic to validate incoming requests and structure responses. The response includes the answer text, source metadata, and a confidence score.

  • Save main.py.

Seeing a syntax error in the prompt string?

Make sure you're using triple quotes (""") for the SYSTEM_PROMPT string. Single or double quotes will break on the multi-line content.

help me fix a syntax error in my multi-line Python string

Write the retrieval and generation functions

These two functions are the heart of the RAG pipeline. The first embeds the user's question and finds matching content. The second sends that content to the LLM.

  • Add the retrieve_context() function below your model definitions in main.py:
def retrieve_context(query: str) -> tuple[str, list[dict], float]:
    """Embed query and retrieve relevant documents from Supabase."""
    # Convert the user's question into a 384-dimensional vector
    query_embedding = model.encode(query).tolist()

    # Call the match_documents function we created in Supabase
    result = supabase.rpc("match_documents", {
        "query_embedding": query_embedding,
        "match_threshold": SIMILARITY_THRESHOLD,
        "match_count": TOP_K,
    }).execute()

    if not result.data:
        return "", [], 0.0

    # Collect matched content and track the highest similarity score
    context_parts = []
    sources = []
    max_similarity = 0.0

    for doc in result.data:
        context_parts.append(doc["content"])
        sources.append(doc["metadata"])
        if doc["similarity"] > max_similarity:
            max_similarity = doc["similarity"]

    context = "\n\n".join(context_parts)
    return context, sources, max_similarity

What does this code do?

  • model.encode(query).tolist() turns the user's question into a 384-dimensional vector using the same model that embedded your content.
  • supabase.rpc("match_documents", ...) calls the PostgreSQL function you created earlier. It returns chunks ranked by cosine similarity.
  • The function returns three things: the combined context string, a list of source metadata (so the chatbot can cite where info came from), and the highest similarity score (used for confidence thresholding).
  • Add the generate_response() function below retrieve_context() in main.py:
def generate_response(context: str, user_message: str) -> str:
    """Call Groq API with context-injected prompt."""
    # Build the messages array with system prompt and user question
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT.format(context=context)},
        {"role": "user", "content": user_message},
    ]

    # Send the request to Groq's OpenAI-compatible endpoint
    with httpx.Client() as client:
        response = client.post(
            GROQ_API_URL,
            headers={
                "Authorization": f"Bearer {GROQ_API_KEY}",
                "Content-Type": "application/json",
            },
            json={
                "model": "llama-3.3-70b-versatile",
                "messages": messages,
                "temperature": 0.3,
                "max_tokens": 1024,
            },
            timeout=30.0,
        )
        response.raise_for_status()

    return response.json()["choices"][0]["message"]["content"]

What does this code do?

SYSTEM_PROMPT.format(context=context) injects the retrieved chunks into the prompt's {context} placeholder. The LLM sees the relevant content right in its instructions.

The Groq API uses the same format as OpenAI's chat completions endpoint. temperature: 0.3 keeps responses factual and consistent. httpx handles the HTTP call with a 30-second timeout.

Why httpx instead of the requests library?

httpx is the recommended HTTP client for FastAPI projects. It supports both sync and async patterns, which makes it easier to upgrade to async endpoints later without rewriting your HTTP calls.

  • Save main.py.

Import errors when saving?

Make sure your virtual environment is active. If you see a red underline under httpx or sentence_transformers, your editor might not be using the correct Python interpreter. Check that it points to the one inside your virtual environment.

help me fix import errors in my Python virtual environment

Wire up the /chat endpoint and test locally

The endpoint ties everything together. It calls retrieve_context() first, checks if the confidence is high enough, and only calls the LLM if relevant content was found.

  • Add the /chat and /health endpoints at the bottom of main.py:
@app.post("/chat", response_model=ChatResponse)
def chat(request: ChatRequest):
    """Main chat endpoint implementing the RAG pipeline."""
    # Step 1: Retrieve relevant context from Supabase
    context, sources, confidence = retrieve_context(request.message)

    # Step 2: If no relevant content found, return fallback without calling LLM
    if not context or confidence < SIMILARITY_THRESHOLD:
        return ChatResponse(
            response=FALLBACK_RESPONSE,
            sources=[],
            confidence=0.0,
        )

    # Step 3: Generate a grounded response using the retrieved context
    response_text = generate_response(context, request.message)

    return ChatResponse(
        response=response_text,
        sources=sources,
        confidence=round(confidence, 3),
    )


@app.get("/health")
def health():
    return {"status": "healthy", "model": "llama-3.3-70b-versatile"}

What is confidence thresholding?

The if not context or confidence < SIMILARITY_THRESHOLD check is a retrieval gate. If no chunks score above 0.3 similarity, the chatbot knows it has nothing useful to say. It returns the fallback response immediately without wasting an LLM call.

This prevents hallucination on off-topic questions. If someone asks about the weather, no chunks will match, so the chatbot politely declines instead of making something up.

  • Save main.py.
  • Start the FastAPI development server by running:
uvicorn main:app --reload

What does this command do?

  • uvicorn is the ASGI server that runs your FastAPI app.
  • main:app tells uvicorn to look for the app object inside main.py.
  • --reload automatically restarts the server when you save changes to your code.

The first startup will take a moment while the SentenceTransformer model loads into memory.

  • Open your browser and navigate to http://127.0.0.1:8000/docs to see the interactive API documentation.
  • Click the POST /chat endpoint, then click Try it out.
  • Replace the request body with this JSON and click Execute:
{"message": "How do I join the club?"}

You should see a JSON response with a grounded answer about membership, a list of sources, and a confidence score above 0.3.

  • Test with an off-topic question to verify scope enforcement:
{"message": "What is the capital of France?"}

This time you should see the fallback response with an empty sources list and a confidence of 0.0.

Server won't start or crashes on startup?

  • If you see ModuleNotFoundError, make sure your virtual environment is active before running the command.
  • If you see an error about SUPABASE_URL or GROQ_API_KEY being None, check that your .env file is in the same directory where you're running the command.
  • If port 8000 is already in use, stop whatever is running on that port first.

help me debug why my FastAPI server won't start

✔️ Awesome, I've got everything!

Great. Double check you've saved main.py and your server is running without errors.

ⓧ I'd like to double check the full code

Here is the complete main.py file at this point:

import os
from dotenv import load_dotenv
from fastapi import FastAPI
from pydantic import BaseModel
from sentence_transformers import SentenceTransformer
from supabase import create_client
import httpx

load_dotenv()

app = FastAPI(title="Hugo AI - RAG Chatbot")

supabase = create_client(os.getenv("SUPABASE_URL"), os.getenv("SUPABASE_KEY"))
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

GROQ_API_KEY = os.getenv("GROQ_API_KEY")
GROQ_API_URL = "https://api.groq.com/openai/v1/chat/completions"
SIMILARITY_THRESHOLD = 0.3
TOP_K = 5

SYSTEM_PROMPT = """You are Hugo, the AWS Cloud Club assistant. You ONLY answer questions about the organization using the context provided below.

Rules:
1. Only use information from the provided context to answer questions.
2. If the context does not contain relevant information, say: "I don't have information about that. Please check our website directly or ask a club officer."
3. Never make up information or answer questions outside the scope of the organization.
4. Be friendly, concise, and helpful.
5. When possible, mention which page or section the information comes from.

Context:
{context}
"""

FALLBACK_RESPONSE = "I don't have information about that topic. I can only answer questions about the AWS Cloud Club \u2014 membership, events, projects, positions, and our team. Is there something specific about the club I can help you with?"


class ChatRequest(BaseModel):
    message: str


class ChatResponse(BaseModel):
    response: str
    sources: list[dict]
    confidence: float


def retrieve_context(query: str) -> tuple[str, list[dict], float]:
    """Embed query and retrieve relevant documents from Supabase."""
    query_embedding = model.encode(query).tolist()

    result = supabase.rpc("match_documents", {
        "query_embedding": query_embedding,
        "match_threshold": SIMILARITY_THRESHOLD,
        "match_count": TOP_K,
    }).execute()

    if not result.data:
        return "", [], 0.0

    context_parts = []
    sources = []
    max_similarity = 0.0

    for doc in result.data:
        context_parts.append(doc["content"])
        sources.append(doc["metadata"])
        if doc["similarity"] > max_similarity:
            max_similarity = doc["similarity"]

    context = "\n\n".join(context_parts)
    return context, sources, max_similarity


def generate_response(context: str, user_message: str) -> str:
    """Call Groq API with context-injected prompt."""
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT.format(context=context)},
        {"role": "user", "content": user_message},
    ]

    with httpx.Client() as client:
        response = client.post(
            GROQ_API_URL,
            headers={
                "Authorization": f"Bearer {GROQ_API_KEY}",
                "Content-Type": "application/json",
            },
            json={
                "model": "llama-3.3-70b-versatile",
                "messages": messages,
                "temperature": 0.3,
                "max_tokens": 1024,
            },
            timeout=30.0,
        )
        response.raise_for_status()

    return response.json()["choices"][0]["message"]["content"]


@app.post("/chat", response_model=ChatResponse)
def chat(request: ChatRequest):
    """Main chat endpoint implementing the RAG pipeline."""
    context, sources, confidence = retrieve_context(request.message)

    if not context or confidence < SIMILARITY_THRESHOLD:
        return ChatResponse(
            response=FALLBACK_RESPONSE,
            sources=[],
            confidence=0.0,
        )

    response_text = generate_response(context, request.message)

    return ChatResponse(
        response=response_text,
        sources=sources,
        confidence=round(confidence, 3),
    )


@app.get("/health")
def health():
    return {"status": "healthy", "model": "llama-3.3-70b-versatile"}

Your RAG pipeline is working end-to-end. Next up, you'll refine the scope enforcement and test edge cases to make sure the chatbot stays on-topic consistently.

Implement Scope Enforcement and System Prompt

Your chatbot can retrieve relevant content and generate responses. But without guardrails, an LLM will happily answer questions about anything, even topics completely outside your embedded content.

In this step, you'll review the scope enforcement system already built into your code and verify it works by testing with both on-topic and off-topic queries. The goal is to confirm Hugo only answers from your embedded content and politely declines everything else.

In this step, get ready to:
  • Understand how the system prompt restricts Hugo to only answer from provided context.
  • Review the retrieval gate that blocks off-topic queries before they reach the LLM.
  • Test with on-topic and off-topic queries to verify scope enforcement.
Review the system prompt that restricts Hugo to provided context

A well-designed system prompt is your first line of defense against hallucination. It tells the LLM exactly what it can and cannot do.

  • Open main.py and find the SYSTEM_PROMPT variable near the top of the file.
SYSTEM_PROMPT = """You are Hugo, the AWS Cloud Club assistant. You ONLY answer questions about the organization using the context provided below.

Rules:
1. Only use information from the provided context to answer questions.
2. If the context does not contain relevant information, say: "I don't have information about that. Please check our website directly or ask a club officer."
3. Never make up information or answer questions outside the scope of the organization.
4. Be friendly, concise, and helpful.
5. When possible, mention which page or section the information comes from.

Context:
{context}
"""

What does each rule do?

  • Rule 1 restricts Hugo to ONLY the retrieved chunks injected into {context}. No prior training knowledge allowed.
  • Rule 2 gives Hugo a specific fallback phrase to use when context is insufficient. This prevents vague or made-up responses.
  • Rule 3 explicitly bans hallucination and out-of-scope answers.
  • Rule 4 sets the tone for responses.
  • Rule 5 encourages source attribution so users know where the information came from.

Why five rules instead of one?

LLMs follow multi-rule prompts more reliably than a single vague instruction like "only answer about the club." Each rule addresses a distinct failure mode: using outside knowledge, vague deflections, fabrication, tone drift, and missing attribution.

Understand the retrieval gate that blocks off-topic queries

The system prompt handles in-scope queries well, but it relies on the LLM following instructions perfectly. A retrieval gate adds a hard-coded safety layer that intercepts off-topic queries before they ever reach the LLM.

  • In main.py, find the chat function and locate this block:
    if not context or confidence < SIMILARITY_THRESHOLD:
        return ChatResponse(
            response=FALLBACK_RESPONSE,
            sources=[],
            confidence=0.0,
        )

How does the retrieval gate work?

When a user asks a question, retrieve_context() embeds the query and searches your Supabase vector database for matching chunks. If nothing matches above the SIMILARITY_THRESHOLD of 0.3, the function returns empty context with a confidence of 0.0.

The chat function checks for this and immediately returns FALLBACK_RESPONSE without ever calling the Groq API. This saves API tokens and guarantees the chatbot cannot hallucinate on off-topic questions.

  • Also find the FALLBACK_RESPONSE variable defined near the top of the file:
FALLBACK_RESPONSE = "I don't have information about that topic. I can only answer questions about the AWS Cloud Club \u2014 membership, events, projects, positions, and our team. Is there something specific about the club I can help you with?"

Why a retrieval gate instead of relying on the system prompt alone?

System prompts are soft constraints. LLMs can be coaxed into ignoring them through clever phrasing. The retrieval gate is a hard constraint in your application code. If the vector search finds nothing relevant, the code short-circuits the response before the LLM is ever called.

This two-layer approach (code gate + prompt rules) is the standard pattern in production RAG systems.

Test with on-topic and off-topic queries

Time to prove the scope enforcement works. You will send two queries to Hugo: one that matches your embedded content and one that is completely unrelated.

  • Start your server by running this command:
uvicorn main:app --reload

You should see the server start at http://127.0.0.1:8000.

  • Open a second terminal window. Test an on-topic query by running:
curl -X POST http://127.0.0.1:8000/chat -H "Content-Type: application/json" -d '{"message": "How do I join?"}'

What should I see?

You should see a JSON response with three fields: response, sources, and confidence. The response should mention visiting the Apply Now page and filling out the application form. The sources array should include entries with apply_now as the source. The confidence value should be above 0.3.

  • Now test an off-topic query by running:
curl -X POST http://127.0.0.1:8000/chat -H "Content-Type: application/json" -d '{"message": "What is the capital of France?"}'

You should see the fallback response with an empty sources array and a confidence of 0.0.

Why does the off-topic query return confidence 0.0?

The question about France has no semantic similarity to your embedded content about the AWS Cloud Club. The match_documents function finds nothing above the 0.3 threshold, so retrieve_context() returns empty results. The retrieval gate catches this and returns the fallback without calling Groq.

Not seeing the expected responses?

  • Make sure your .env file has valid SUPABASE_URL, SUPABASE_KEY, and GROQ_API_KEY values.
  • Confirm you ran ingest.py successfully in the previous step. The on-topic query needs embedded content to match against.
  • If the on-topic query returns the fallback response, your Supabase database may be empty. Re-run the ingestion script.

Help me debug my RAG chatbot scope enforcement

✔️ Awesome, I've got everything!

Great. Make sure you have saved main.py and confirmed both test queries return the expected results.

ⓧ I'd like to double check the full code

Here is the complete main.py file for reference:

import os
from dotenv import load_dotenv
from fastapi import FastAPI
from pydantic import BaseModel
from sentence_transformers import SentenceTransformer
from supabase import create_client
import httpx

load_dotenv()

app = FastAPI(title="Hugo AI - RAG Chatbot")

supabase = create_client(os.getenv("SUPABASE_URL"), os.getenv("SUPABASE_KEY"))
model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

GROQ_API_KEY = os.getenv("GROQ_API_KEY")
GROQ_API_URL = "https://api.groq.com/openai/v1/chat/completions"
SIMILARITY_THRESHOLD = 0.3
TOP_K = 5

SYSTEM_PROMPT = """You are Hugo, the AWS Cloud Club assistant. You ONLY answer questions about the organization using the context provided below.

Rules:
1. Only use information from the provided context to answer questions.
2. If the context does not contain relevant information, say: "I don't have information about that. Please check our website directly or ask a club officer."
3. Never make up information or answer questions outside the scope of the organization.
4. Be friendly, concise, and helpful.
5. When possible, mention which page or section the information comes from.

Context:
{context}
"""

FALLBACK_RESPONSE = "I don't have information about that topic. I can only answer questions about the AWS Cloud Club \u2014 membership, events, projects, positions, and our team. Is there something specific about the club I can help you with?"


class ChatRequest(BaseModel):
    message: str


class ChatResponse(BaseModel):
    response: str
    sources: list[dict]
    confidence: float


def retrieve_context(query: str) -> tuple[str, list[dict], float]:
    """Embed query and retrieve relevant documents from Supabase."""
    query_embedding = model.encode(query).tolist()

    result = supabase.rpc("match_documents", {
        "query_embedding": query_embedding,
        "match_threshold": SIMILARITY_THRESHOLD,
        "match_count": TOP_K,
    }).execute()

    if not result.data:
        return "", [], 0.0

    context_parts = []
    sources = []
    max_similarity = 0.0

    for doc in result.data:
        context_parts.append(doc["content"])
        sources.append(doc["metadata"])
        if doc["similarity"] > max_similarity:
            max_similarity = doc["similarity"]

    context = "\n\n".join(context_parts)
    return context, sources, max_similarity


def generate_response(context: str, user_message: str) -> str:
    """Call Groq API with context-injected prompt."""
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT.format(context=context)},
        {"role": "user", "content": user_message},
    ]

    with httpx.Client() as client:
        response = client.post(
            GROQ_API_URL,
            headers={
                "Authorization": f"Bearer {GROQ_API_KEY}",
                "Content-Type": "application/json",
            },
            json={
                "model": "llama-3.3-70b-versatile",
                "messages": messages,
                "temperature": 0.3,
                "max_tokens": 1024,
            },
            timeout=30.0,
        )
        response.raise_for_status()

    return response.json()["choices"][0]["message"]["content"]


@app.post("/chat", response_model=ChatResponse)
def chat(request: ChatRequest):
    """Main chat endpoint implementing the RAG pipeline."""
    context, sources, confidence = retrieve_context(request.message)

    if not context or confidence < SIMILARITY_THRESHOLD:
        return ChatResponse(
            response=FALLBACK_RESPONSE,
            sources=[],
            confidence=0.0,
        )

    response_text = generate_response(context, request.message)

    return ChatResponse(
        response=response_text,
        sources=sources,
        confidence=round(confidence, 3),
    )


@app.get("/health")
def health():
    return {"status": "healthy", "model": "llama-3.3-70b-versatile"}

Your scope enforcement is verified. Hugo answers club-related questions with grounded responses and politely declines everything else without wasting API calls. Next up, you will deploy this to Render so it is accessible as a live API.

Deploy to Render

Your RAG chatbot works locally. Now it needs a public URL so anyone can send it questions over the internet.

In this step, you'll push your project to GitHub and deploy it on Render's free tier. Render pulls your code from GitHub, installs dependencies, and runs your FastAPI server on a public URL.

In this step, get ready to:
  • Prepare your project for deployment with a .gitignore file.
  • Push your code to a GitHub repository.
  • Deploy your FastAPI app as a free Render web service and test it live.
Create a .gitignore file

Your .env file contains API keys that must never be pushed to a public repository. A .gitignore file tells Git which files to skip.

  • Create a new file called .gitignore in your project folder and add the following content:
# Environment variables (contains secrets)
.env

# Python virtual environment
venv/

# Python cache
__pycache__/

# Model cache (downloaded at runtime)
.cache/

Why exclude these files?

The .env file holds your Supabase and Groq keys. The venv/ folder contains installed packages that Render will reinstall from requirements.txt. The __pycache__/ folder holds compiled bytecode that Python regenerates automatically.

  • Save .gitignore.
Push your project to GitHub

Render deploys code directly from a GitHub repository. Every time you push to your linked branch, Render automatically rebuilds and redeploys.

  • Create a new repository on GitHub. Go to github.com/new in your browser.
  • Name the repository hugo-ai-chatbot (or any name you prefer).
  • Leave it set to Public.
  • Do not check any of the initialization options (no README, no .gitignore, no license).
  • Click Create repository.

GitHub shows you the commands to push an existing project. Switch back to your terminal.

  • Initialize Git and push your code by running these commands:
git init
git add .
git commit -m "Initial commit - RAG chatbot backend"
git branch -M main
git remote add origin https://github.com/[[GITHUB_USERNAME="your-github-username"]]/[[REPO_NAME="hugo-ai-chatbot"]].git
git push -u origin main

What do these commands do?

  • git init creates a new Git repository in your project folder.
  • git add . stages all files (except those in .gitignore).
  • git push -u origin main uploads your code to GitHub.
  • Refresh your GitHub repository page in the browser. You should see main.py, ingest.py, requirements.txt, and setup.sql listed. You should not see .env.

Don't see your files on GitHub?

  • If git push asks for authentication, GitHub requires a personal access token instead of a password. Generate one at github.com/settings/tokens and use it as your password.
  • If you see .env in the repository, your .gitignore was not saved before running git add .. Delete the repository, save the .gitignore, and repeat the Git commands.

Help me push my project to GitHub.

Deploy to Render and test your live API

Render connects to your GitHub repository, installs your dependencies, and starts your FastAPI server on a public URL. The free tier gives you 512 MB RAM and 750 instance hours per month.

  • Go to render.com and sign up for a free account (you can sign up with GitHub for fastest setup).
  • In the Render Dashboard, click + New in the top-right corner.
  • Select Web Service.
  • Under the source section, select Git Provider and connect your GitHub account if prompted. Authorize Render to access your repositories.
  • Find and select your hugo-ai-chatbot repository, then click Connect.
  • Fill in the service settings:
  • Name: hugo-ai (this becomes part of your public URL).
  • Language: Python 3.
  • Branch: main.
  • Build Command: pip install -r requirements.txt.
  • Start Command: uvicorn main:app --host 0.0.0.0 --port 10000.
  • Under instance type, choose Free.
  • Expand the Advanced section and add three environment variables:
  • SUPABASE_URL = your Supabase project URL (from your .env file).
  • SUPABASE_KEY = your Supabase anon key.
  • GROQ_API_KEY = your Groq API key.
  • Click Create Web Service.

What happens during the deploy?

Render clones your repository, runs pip install -r requirements.txt to install your dependencies (including downloading the sentence-transformers model), then starts your server with the start command. The first deploy takes 3-5 minutes because of the model download.

Wait for the deploy to complete. The status on your service's Events page changes to Live when the build succeeds.

Once the service is live, Render gives you a public URL (shown at the top of your service's page). It looks like https://hugo-ai.onrender.com or similar.

  • Test the health endpoint by opening this URL in your browser:
https://[[RENDER_URL="your-service-name"]].onrender.com/health
  • You should see this JSON response:
{"status": "healthy", "model": "llama-3.3-70b-versatile"}

Now test the chat endpoint with a real query.

  • Run this command in your terminal to send an on-topic question:
curl -X POST https://[[RENDER_URL="your-service-name"]].onrender.com/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "When does the AI Study Group meet?"}'
  • You should see a JSON response with a response field answering the question, a sources array with metadata about where the answer came from, and a confidence score above 0.3.
  • Test scope enforcement by sending an off-topic question:
curl -X POST https://[[RENDER_URL="your-service-name"]].onrender.com/chat \
  -H "Content-Type: application/json" \
  -d '{"message": "What is the capital of France?"}'
  • You should see the fallback response: the chatbot politely declines because the question is outside its scope.

Getting a timeout or 502 error?

  • Render's free tier spins down after 15 minutes without traffic. The first request after a cold start takes about a minute to respond while the service wakes up. Wait and try again.
  • Check your service's Logs tab in the Render Dashboard for error messages. A missing environment variable is the most common cause of startup failures.
  • Verify all three environment variables (SUPABASE_URL, SUPABASE_KEY, GROQ_API_KEY) are set in the Render Dashboard under Environment.

Help me troubleshoot my Render deployment.

Your RAG chatbot is live on the internet. Anyone with the URL can ask questions about your content and get grounded, source-attributed answers. Next up, you'll take on the Secret Mission and add conversation memory so your chatbot can handle follow-up questions.

Secret mission

Add Conversation Memory

Your chatbot answers individual questions well, but it forgets everything between requests. Real conversations have context. Can you add session-based memory so users can ask follow-up questions like "Tell me more about that" or "When is it?" without repeating themselves?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. Both Supabase and Render are on free tiers, so there are no ongoing costs. However, both services have idle behavior that may pause or spin down your project.

Resources you used:

  • Supabase project (free tier, with pgvector-enabled database and documents table).
  • Render web service (free tier, hosting your FastAPI chatbot).
  • Local project folder on your Desktop with main.py, ingest.py, requirements.txt, setup.sql, and .env.
  • GitHub repository connected to Render.
  • Groq API key (free tier, stored in Render environment variables).

Keep everything running

No action needed. Choose this if you are still actively building or testing your chatbot.

  • Your Supabase project will pause automatically after 1 week of inactivity. You can restore it from the Supabase dashboard when you are ready to use it again.
  • Your Render service spins down after 15 minutes without inbound traffic. The next request takes about a minute to spin back up.
  • Your Groq API key remains active with free tier rate limits.

Pause - I'll come back to this later

Both services auto-pause on the free tier. No manual action is required.

  • Supabase pauses your project after 1 week without activity. Your data stays intact.
  • Render spins down your service after 15 minutes without traffic. Your code and configuration stay intact.
  • When you come back, send a request to your Render URL and restore your Supabase project from the dashboard.

Delete - I don't want to use this again

Remove all project resources and start fresh. This cannot be undone.

  • Delete your Render web service: open the Render Dashboard, click on your service, select the Settings tab, scroll to the bottom of the page, and click Delete Web Service.
  • Delete your Supabase project: open the Supabase Dashboard, select your project, click Project Settings in the sidebar, go to the General page, scroll to the bottom, and click Delete project.
  • Delete your local project folder by running this command:

macOS

rm -rf ~/Desktop/rag-chatbot

Windows

Remove-Item -Recurse -Force ~\Desktop\rag-chatbot
  • Optionally delete your GitHub repository: go to your repository on GitHub, click Settings, scroll to the bottom under Danger Zone, and click Delete this repository.
  • Optionally revoke your Groq API key: go to console.groq.com, open API Keys, and delete the key you created for this project.

Nice Work!

Nice Work!

You just built and deployed a production-style RAG chatbot backend. This is the same architecture behind enterprise knowledge assistants, customer support bots, and internal AI tools.

Here's what you learned:

  • Built a vector database with pgvector on Supabase and a content ingestion pipeline using sentence-transformers embeddings.
  • Implemented the full RAG pipeline: embed the user query, retrieve relevant context via semantic search, and generate grounded responses with Groq's Llama 3.3 70B model.
  • Enforced chatbot scope with a retrieval gate and system prompt design, then deployed the live API on Render's free tier.
  • Added session-based conversation memory so users can ask follow-up questions naturally across multi-turn conversations.

Ready to quiz yourself?