RAG Pipeline for Healthcare IoT Docs

Build an accuracy-measured RAG pipeline for medical device documentation.

Introduction

30 Second Summary

Healthcare devices like heart monitors and infusion pumps generate life-critical data. When a nurse asks "what's the alarm threshold for this device?", the answer needs to be 100% accurate with zero room for guessing.

In this project, you will build a RAG (Retrieval-Augmented Generation) pipeline that ingests healthcare IoT device documentation and answers technical questions with provable accuracy, measured by quantitative evaluation metrics.

What You'll Build

A Python-based RAG pipeline that ingests simulated medical device manuals into a local ChromaDB vector database, retrieves relevant information using semantic search, and generates accurate answers using Groq's Llama 3.3 70B model.

By the end of this project, you'll have:

  • A structure-aware ingestion pipeline that chunks healthcare documentation while preserving safety-critical information boundaries, stored in a metadata-enriched vector store with filtering by device type, safety classification, and document version.
  • A citation-grounded query engine powered by Groq that refuses to hallucinate and explicitly states when it lacks sufficient information.
  • A RAGAS evaluation harness that measures your pipeline's faithfulness, context recall, and factual correctness with quantitative scores.
  • Secret Mission: Add hybrid search to improve retrieval of exact medical device codes and alarm thresholds.

Are there any prerequisites?

You'll need basic Python knowledge and familiarity with installing packages via pip. No prior experience with RAG, LlamaIndex, or vector databases is required.

Before We Start

Healthcare IoT devices generate critical documentation that clinicians depend on for patient safety. When someone asks "what's the bradycardia alarm threshold?", the answer must be precise and traceable back to the source document.

In this step, you'll set up your development environment with all the tools needed to build a RAG pipeline that retrieves and answers questions from healthcare device documentation.

In this step, get ready to:
  • Install Python and create a project environment.
  • Install required packages and configure API access.
  • Verify everything works together.
Install Python 3.10+ and create your project environment

LlamaIndex requires Python 3.10 or higher. Let's confirm you have the right version installed.

  • Open your terminal. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or Command Prompt (Windows) and press Enter to open it.
  • Check your Python version by running:
python3 --version

✔️ I see version 3.10 or higher

You should see output like Python 3.10.x or higher. You're all set to continue.

ⓧ I see an older version

If you see a version below 3.10 (such as 3.9 or 3.8), you need to install a newer version. macOS often ships with an older Python by default.

  • Visit python.org/downloads and download Python 3.10 or newer.
  • Run the installer with default settings.
  • After installation, close and reopen your terminal.
  • Run python3 --version again to confirm you now see 3.10 or higher.

macOS version shadowing

macOS sometimes resolves python3 to an older system Python even after installing a newer version. If this happens, try running python3.10 --version or python3.11 --version directly to use the specific binary you installed.

ⓧ Command not found

Python is not installed on your system. Let's install it now.

  • Visit python.org/downloads and download Python 3.10 or newer.
  • Run the installer. On Windows, check the box that says Add Python to PATH before clicking Install Now.
  • Close and reopen your terminal after installation.
  • Run python3 --version (macOS/Linux) or python --version (Windows) to confirm installation.

Now let's create your project folder. First, move to your Desktop so you can easily find your project later.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir healthcare-rag-pipeline
cd healthcare-rag-pipeline
  • Create a virtual environment to keep your packages isolated by running:
python3 -m venv venv

What is a virtual environment?

A virtual environment is an isolated Python installation inside your project folder. Packages you install here won't interfere with other Python projects on your computer.

  • Activate the virtual environment:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate
  • Confirm the virtual environment is active. You should see (venv) at the start of your terminal prompt.

Don't see (venv) in your prompt?

Make sure you ran the activation command for your operating system. On Windows, if you see an error about execution policies, run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned in PowerShell and try again.

Help me activate my Python virtual environment.

Install all required packages and configure Groq API access

Your pipeline uses several libraries: LlamaIndex for the RAG framework, ChromaDB for the vector database, and RAGAS for accuracy evaluation.

  • Create a requirements.txt file inside your healthcare-rag-pipeline folder. This file lists every package your project needs with exact version pins.
  • Open the file in your text editor and paste the following contents:
llama-index==0.14.22
llama-index-llms-openai-like==0.3.5
llama-index-embeddings-huggingface==0.5.2
llama-index-vector-stores-chroma==0.4.1
chromadb==1.5.9
ragas==0.4.3
langchain-groq==0.3.3
python-dotenv==1.1.0

What do these packages do?

  • llama-index is the core RAG framework that handles document loading, chunking, indexing, and querying.
  • llama-index-llms-openai-like lets LlamaIndex talk to any OpenAI-compatible API, including Groq.
  • llama-index-embeddings-huggingface provides free, local embedding models for converting text to vectors.
  • llama-index-vector-stores-chroma connects LlamaIndex to ChromaDB for persistent vector storage.
  • chromadb is the vector database that stores and retrieves your document chunks.
  • ragas evaluates your RAG pipeline's accuracy with metrics like Faithfulness and Context Recall.
  • langchain-groq wraps the Groq LLM for use as a RAGAS evaluator.
  • python-dotenv loads environment variables from a .env file so you don't hardcode secrets.
  • Save requirements.txt.
  • Install all the packages by running:
pip install -r requirements.txt

This may take a few minutes

The HuggingFace embedding model and ChromaDB have several dependencies. You'll see a lot of download progress bars. Wait until you see the final "Successfully installed" message before continuing.

Seeing installation errors?

If you see errors about missing compilers or build tools, make sure your virtual environment is activated (you should see (venv) in your prompt). Also confirm you're running Python 3.10 or higher inside the venv.

Help me fix my pip install errors for the healthcare RAG project.

Now let's get your Groq API key. Groq provides a free tier with no credit card required.

  • Open your browser and go to console.groq.com.
  • Sign up with your email address or GitHub account.
  • Once logged in, navigate to API Keys in the left sidebar.
  • Click Create API Key.
  • Give your key a name (e.g. healthcare-rag) and copy the generated key immediately. You won't be able to see it again.

Why Groq?

Groq provides an OpenAI-compatible API endpoint powered by custom LPU hardware. It runs Llama 3.3 70B at very high speeds on a generous free tier. No credit card is required.

  • Create a .env file in your healthcare-rag-pipeline folder. This file stores your API key as an environment variable.
  • Open the file and paste the following, replacing the placeholder with your actual key:
GROQ_API_KEY=your-groq-api-key-here
  • Save .env.

Keep your API key safe

Never commit your .env file to version control. If you use Git, add .env to your .gitignore file.

✔️ Awesome, I've got everything!

Great. Double check you've saved both requirements.txt and .env in your healthcare-rag-pipeline folder.

ⓧ I'd like to double check the full code

Here are both files for comparison:

llama-index==0.14.22
llama-index-llms-openai-like==0.3.5
llama-index-embeddings-huggingface==0.5.2
llama-index-vector-stores-chroma==0.4.1
chromadb==1.5.9
ragas==0.4.3
langchain-groq==0.3.3
python-dotenv==1.1.0
GROQ_API_KEY=your-groq-api-key-here
Verify the installation

Let's confirm that all libraries import correctly and that your Groq connection works.

  • Run the following verification script in your terminal:
python3 -c "
import os
from dotenv import load_dotenv
load_dotenv()

# Test core imports
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader, Settings
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.huggingface import HuggingFaceEmbedding
import chromadb
from ragas.metrics import Faithfulness

print('All imports successful!')

# Test Groq connectivity
api_key = os.getenv('GROQ_API_KEY')
if api_key and api_key != 'your-groq-api-key-here':
    print(f'GROQ_API_KEY loaded (starts with {api_key[:8]}...)')
else:
    print('WARNING: GROQ_API_KEY not set or still placeholder')

print('Environment ready!')
"
  • You should see:
All imports successful!
GROQ_API_KEY loaded (starts with gsk_...)
Environment ready!

Seeing import errors?

If you see ModuleNotFoundError, confirm your virtual environment is activated (look for (venv) in your prompt) and that pip install -r requirements.txt completed without errors.

If your API key shows as the placeholder, make sure you replaced your-groq-api-key-here in .env with the actual key you copied from the Groq console.

Help me fix my import errors in the healthcare RAG project.

Your development environment is ready. Next up, you'll create the healthcare IoT document corpus that your RAG pipeline will ingest and retrieve answers from.

Create the Healthcare IoT Document Corpus

Your development environment is ready. Now you need data for your RAG pipeline to ingest.

In this step, you will create a set of simulated healthcare IoT device manuals that mirror the precision and structure of real medical device documentation. These documents contain exact alarm codes, numeric thresholds, and safety classifications that your pipeline must retrieve accurately.

In this step, get ready to:
  • Create a data directory with three Markdown device documentation files.
  • Structure documents with safety classifications, alarm codes, and network protocols.
  • Define the metadata mapping for document traceability.
Create the data directory and patient monitor documentation

Healthcare IoT devices like patient monitors generate critical data that clinicians depend on. Your RAG pipeline needs realistic documentation to prove it can retrieve safety-critical information without hallucinating.

  • Create the data/ directory inside your project folder by running this command:
mkdir data
  • Confirm the directory was created by running:
ls

You should see data listed alongside your existing files like requirements.txt and .env.

  • Create a new file called patient_monitor.md inside the data/ directory. Paste the following content into it:
# CardioWatch Pro X200 - Patient Monitor

## Device Overview
- **Device Type**: Multi-parameter Patient Monitor
- **Safety Classification**: IEC 62304 Class C (highest safety)
- **Firmware Version**: 4.2.1
- **Connectivity**: Wi-Fi 802.11ac, Bluetooth 5.0, HL7 FHIR R4

## Vital Signs Monitoring

### Heart Rate Monitoring
- Measurement Range: 30-300 BPM
- Accuracy: +/- 1 BPM
- Alarm Thresholds:
  - Bradycardia (HR-BRADY-001): Below 50 BPM
  - Tachycardia (HR-TACHY-001): Above 120 BPM
  - Asystole (HR-ASYS-001): No detected heartbeat for 4 seconds

### Blood Oxygen (SpO2)
- Measurement Range: 0-100%
- Accuracy: +/- 2% (70-100% range)
- Alarm Thresholds:
  - Hypoxemia (SPO2-LOW-001): Below 90%
  - Critical Hypoxemia (SPO2-CRIT-001): Below 85%

### Blood Pressure (NIBP)
- Systolic Range: 40-270 mmHg
- Diastolic Range: 20-200 mmHg
- Accuracy: +/- 3 mmHg
- Measurement Interval: Configurable 1-60 minutes

## Network Security
- All data transmitted using TLS 1.3 encryption
- Device authentication via X.509 certificates
- RBAC (Role-Based Access Control) with 4 levels: Admin, Clinician, Nurse, Technician
- Audit logging of all access events per HIPAA requirements
- Network segmentation required: medical device VLAN isolation mandatory

## Data Transmission Protocol
- Primary: HL7 FHIR R4 over HTTPS
- Fallback: HL7 v2.x over MLLP
- Data refresh rate: Real-time (1 second intervals for ECG)
- Store-and-forward capability: 72 hours of local storage if network unavailable

What does this document cover?

This simulates a multi-parameter patient monitor's technical documentation. It includes exact alarm codes (like HR-BRADY-001) with precise numeric thresholds, which your RAG pipeline must retrieve character-for-character.

The network security section covers TLS 1.3, X.509 certificates, and VLAN isolation. These are the kinds of specific technical details that a healthcare RAG system must return accurately.

  • Save data/patient_monitor.md.

File not saving in the data folder?

Make sure you created the file inside the data/ directory, not in the project root. If using VS Code, right-click the data folder in the Explorer sidebar and select New File.

Help me create files inside a subdirectory in my code editor.

Create the infusion pump and blood pressure monitor documentation

The remaining two documents cover devices with different safety profiles. The infusion pump is Class C (highest safety) with drug dosage interlocks. The blood pressure monitor is Class B with calibration requirements.

  • Create a new file called infusion_pump.md inside the data/ directory. Paste the following content into it:
# MedFlow Precision 3000 - Smart Infusion Pump

## Device Overview
- **Device Type**: Volumetric Infusion Pump
- **Safety Classification**: IEC 62304 Class C (highest safety)
- **Firmware Version**: 6.1.0
- **Connectivity**: Wi-Fi 802.11ac, Ethernet 10/100, HL7 FHIR R4

## Dosage and Flow Control

### Flow Rate
- Range: 0.1 - 1200 mL/hr
- Accuracy: +/- 5% of set rate
- Resolution: 0.1 mL/hr (for rates below 100 mL/hr)

### Drug Library
- Pre-programmed limits for 500+ medications
- Hard limits: Cannot be overridden by user
- Soft limits: Require clinician confirmation to override
- Auto-programming via EMR integration (Cerner, Epic)

### Safety Interlocks
- Air-in-line detection (AIIL-001): Ultrasonic sensor, threshold 50 microliters
- Occlusion detection (OCC-001): Upstream and downstream, threshold 300-900 mmHg configurable
- Door-open detection (DOOR-001): Immediate pause of infusion
- Free-flow protection: Anti-free-flow clamp engages on door open
- Over-pressure alarm (PRES-HIGH-001): Triggers at 1000 mmHg

## Network Security
- WPA3-Enterprise authentication required
- 802.1X port-based access control
- Encrypted drug library updates via signed firmware packages
- Intrusion detection: anomalous traffic patterns trigger PUMP-SEC-001 alert
- Device certificates rotate every 90 days

## Cybersecurity Compliance
- FDA premarket cybersecurity guidance (2023) compliant
- NIST Cybersecurity Framework aligned
- Software Bill of Materials (SBOM) maintained
- Vulnerability disclosure program active
- Patch cycle: Critical within 48 hours, routine monthly

What does this document cover?

This infusion pump document includes safety interlocks with specific alarm codes (AIIL-001, OCC-001, DOOR-001) and exact thresholds. It also covers cybersecurity compliance details like SBOM maintenance and patch cycles.

These precise identifiers are exactly what semantic search can struggle with. Later, you will test whether your pipeline can retrieve these codes accurately.

  • Save data/infusion_pump.md.
  • Create a new file called blood_pressure_monitor.md inside the data/ directory. Paste the following content into it:
# VitalLink BP Connect - Connected Blood Pressure Monitor

## Device Overview
- **Device Type**: Connected Oscillometric Blood Pressure Monitor
- **Safety Classification**: IEC 62304 Class B
- **Firmware Version**: 2.3.4
- **Connectivity**: Bluetooth Low Energy 5.2, Wi-Fi 802.11n

## Measurement Specifications

### Accuracy Standards
- Validated per ISO 81060-2:2018
- Systolic accuracy: +/- 3 mmHg (mean error), SD < 8 mmHg
- Diastolic accuracy: +/- 3 mmHg (mean error), SD < 8 mmHg
- Clinical validation: 85 subjects, 255 measurements per protocol

### Measurement Range
- Systolic: 60-260 mmHg
- Diastolic: 30-200 mmHg
- Pulse rate: 40-200 BPM
- Cuff pressure range: 0-299 mmHg

### Calibration
- Factory calibrated with NIST-traceable reference
- Recalibration interval: Every 2 years or 10,000 measurements
- Self-test on power-up: pressure sensor zero-point verification
- Calibration alarm (CAL-DUE-001): Triggers 30 days before due date

## Data Transmission
- Protocol: Bluetooth Health Device Profile (HDP)
- Data format: IEEE 11073-10407 (Blood Pressure)
- Transmission frequency: After each measurement (immediate) or batch upload (configurable)
- Cloud sync: End-to-end encrypted via AES-256-GCM
- Patient data stored locally: Last 200 measurements

## Privacy and Security
- Pairing: Secure Simple Pairing with numeric comparison
- Data at rest: AES-256 encrypted local storage
- HIPAA compliant data handling
- User authentication: PIN code (4-6 digits) required before measurement
- Remote wipe capability via cloud management portal

What does this document cover?

This blood pressure monitor is classified as IEC 62304 Class B (lower safety class than the other two devices). It includes calibration requirements and ISO validation standards that your pipeline will need to distinguish from the Class C devices.

The calibration alarm code CAL-DUE-001 and AES-256-GCM encryption details are the type of precise values that healthcare staff would query.

  • Save data/blood_pressure_monitor.md.
  • Verify all three files exist by running:
ls data/

You should see blood_pressure_monitor.md, infusion_pump.md, and patient_monitor.md listed.

Missing one or more files?

Double-check that each file is saved inside the data/ subdirectory (not the project root). The filenames must match exactly: patient_monitor.md, infusion_pump.md, blood_pressure_monitor.md.

Help me check why my files aren't appearing in the data directory.

✔️ Awesome, I've got everything!

Great. Make sure all three files are saved before moving on.

ⓧ I'd like to double check the full code

Here are all three files for comparison.

# CardioWatch Pro X200 - Patient Monitor

## Device Overview
- **Device Type**: Multi-parameter Patient Monitor
- **Safety Classification**: IEC 62304 Class C (highest safety)
- **Firmware Version**: 4.2.1
- **Connectivity**: Wi-Fi 802.11ac, Bluetooth 5.0, HL7 FHIR R4

## Vital Signs Monitoring

### Heart Rate Monitoring
- Measurement Range: 30-300 BPM
- Accuracy: +/- 1 BPM
- Alarm Thresholds:
  - Bradycardia (HR-BRADY-001): Below 50 BPM
  - Tachycardia (HR-TACHY-001): Above 120 BPM
  - Asystole (HR-ASYS-001): No detected heartbeat for 4 seconds

### Blood Oxygen (SpO2)
- Measurement Range: 0-100%
- Accuracy: +/- 2% (70-100% range)
- Alarm Thresholds:
  - Hypoxemia (SPO2-LOW-001): Below 90%
  - Critical Hypoxemia (SPO2-CRIT-001): Below 85%

### Blood Pressure (NIBP)
- Systolic Range: 40-270 mmHg
- Diastolic Range: 20-200 mmHg
- Accuracy: +/- 3 mmHg
- Measurement Interval: Configurable 1-60 minutes

## Network Security
- All data transmitted using TLS 1.3 encryption
- Device authentication via X.509 certificates
- RBAC (Role-Based Access Control) with 4 levels: Admin, Clinician, Nurse, Technician
- Audit logging of all access events per HIPAA requirements
- Network segmentation required: medical device VLAN isolation mandatory

## Data Transmission Protocol
- Primary: HL7 FHIR R4 over HTTPS
- Fallback: HL7 v2.x over MLLP
- Data refresh rate: Real-time (1 second intervals for ECG)
- Store-and-forward capability: 72 hours of local storage if network unavailable
# MedFlow Precision 3000 - Smart Infusion Pump

## Device Overview
- **Device Type**: Volumetric Infusion Pump
- **Safety Classification**: IEC 62304 Class C (highest safety)
- **Firmware Version**: 6.1.0
- **Connectivity**: Wi-Fi 802.11ac, Ethernet 10/100, HL7 FHIR R4

## Dosage and Flow Control

### Flow Rate
- Range: 0.1 - 1200 mL/hr
- Accuracy: +/- 5% of set rate
- Resolution: 0.1 mL/hr (for rates below 100 mL/hr)

### Drug Library
- Pre-programmed limits for 500+ medications
- Hard limits: Cannot be overridden by user
- Soft limits: Require clinician confirmation to override
- Auto-programming via EMR integration (Cerner, Epic)

### Safety Interlocks
- Air-in-line detection (AIIL-001): Ultrasonic sensor, threshold 50 microliters
- Occlusion detection (OCC-001): Upstream and downstream, threshold 300-900 mmHg configurable
- Door-open detection (DOOR-001): Immediate pause of infusion
- Free-flow protection: Anti-free-flow clamp engages on door open
- Over-pressure alarm (PRES-HIGH-001): Triggers at 1000 mmHg

## Network Security
- WPA3-Enterprise authentication required
- 802.1X port-based access control
- Encrypted drug library updates via signed firmware packages
- Intrusion detection: anomalous traffic patterns trigger PUMP-SEC-001 alert
- Device certificates rotate every 90 days

## Cybersecurity Compliance
- FDA premarket cybersecurity guidance (2023) compliant
- NIST Cybersecurity Framework aligned
- Software Bill of Materials (SBOM) maintained
- Vulnerability disclosure program active
- Patch cycle: Critical within 48 hours, routine monthly
# VitalLink BP Connect - Connected Blood Pressure Monitor

## Device Overview
- **Device Type**: Connected Oscillometric Blood Pressure Monitor
- **Safety Classification**: IEC 62304 Class B
- **Firmware Version**: 2.3.4
- **Connectivity**: Bluetooth Low Energy 5.2, Wi-Fi 802.11n

## Measurement Specifications

### Accuracy Standards
- Validated per ISO 81060-2:2018
- Systolic accuracy: +/- 3 mmHg (mean error), SD < 8 mmHg
- Diastolic accuracy: +/- 3 mmHg (mean error), SD < 8 mmHg
- Clinical validation: 85 subjects, 255 measurements per protocol

### Measurement Range
- Systolic: 60-260 mmHg
- Diastolic: 30-200 mmHg
- Pulse rate: 40-200 BPM
- Cuff pressure range: 0-299 mmHg

### Calibration
- Factory calibrated with NIST-traceable reference
- Recalibration interval: Every 2 years or 10,000 measurements
- Self-test on power-up: pressure sensor zero-point verification
- Calibration alarm (CAL-DUE-001): Triggers 30 days before due date

## Data Transmission
- Protocol: Bluetooth Health Device Profile (HDP)
- Data format: IEEE 11073-10407 (Blood Pressure)
- Transmission frequency: After each measurement (immediate) or batch upload (configurable)
- Cloud sync: End-to-end encrypted via AES-256-GCM
- Patient data stored locally: Last 200 measurements

## Privacy and Security
- Pairing: Secure Simple Pairing with numeric comparison
- Data at rest: AES-256 encrypted local storage
- HIPAA compliant data handling
- User authentication: PIN code (4-6 digits) required before measurement
- Remote wipe capability via cloud management portal
Define the metadata mapping for document traceability

Healthcare systems require traceability. Every piece of retrieved information must link back to a specific device, firmware version, and safety classification. This prevents cross-contamination of information between devices.

Your RAG pipeline will use a metadata mapping to associate each document with structured attributes. Here is the mapping you will implement in the next step when you build the ingestion pipeline:

# Metadata for each document
DOCUMENT_METADATA = {
    "patient_monitor.md": {
        "device_type": "patient_monitor",
        "device_name": "CardioWatch Pro X200",
        "safety_class": "IEC 62304 Class C",
        "firmware_version": "4.2.1",
    },
    "infusion_pump.md": {
        "device_type": "infusion_pump",
        "device_name": "MedFlow Precision 3000",
        "safety_class": "IEC 62304 Class C",
        "firmware_version": "6.1.0",
    },
    "blood_pressure_monitor.md": {
        "device_type": "blood_pressure_monitor",
        "device_name": "VitalLink BP Connect",
        "safety_class": "IEC 62304 Class B",
        "firmware_version": "2.3.4",
    },
}

Why attach metadata to documents?

When a clinician asks about alarm thresholds, the system needs to specify which device the answer applies to. Without metadata filtering, your pipeline might return infusion pump thresholds when the user asked about the patient monitor.

Each entry maps a filename to four attributes: device_type for filtering queries by device category, device_name for human-readable citations, safety_class for regulatory context, and firmware_version for version traceability.

You will code this in the next step

You do not need to save this code yet. This mapping will be part of create_rag_pipeline.py which you build in the next step. For now, understand the structure so the ingestion pipeline makes sense when you get there.

Your document corpus is ready. Next up, you will build the ingestion pipeline that chunks these documents, attaches the metadata, and stores everything in a ChromaDB vector database for retrieval.

Build the Ingestion Pipeline with Structure-Aware Chunking

Your healthcare IoT documents are ready. Now you need a pipeline that can load them, split them intelligently, and store them in a searchable vector database.

In this step, you will build the ingestion pipeline that transforms raw Markdown files into vector embeddings stored in ChromaDB. The pipeline preserves safety-critical boundaries during chunking and tags every chunk with metadata for traceability.

In this step, get ready to:
  • Configure LlamaIndex with Groq's Llama 3.3 70B model and local HuggingFace embeddings.
  • Define a metadata mapping that tags each document with its device type and safety class.
  • Ingest all three device manuals into ChromaDB with structure-aware chunking.
Create the pipeline configuration file

Before writing the ingestion logic, you need to configure the tools your pipeline depends on. This includes the LLM connection (Groq), the embedding model (HuggingFace), and the metadata mapping that links each file to its device information.

  • Create a new file called create_rag_pipeline.py in your project root folder.
  • Add the following imports, settings, and metadata configuration:
import os
import chromadb
from dotenv import load_dotenv
from llama_index.core import (
    SimpleDirectoryReader,
    VectorStoreIndex,
    StorageContext,
    Settings,
)
from llama_index.core.node_parser import SentenceSplitter
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

load_dotenv()

# Configure LlamaIndex settings with Groq API
Settings.llm = OpenAILike(
    model="llama-3.3-70b-versatile",
    api_base="https://api.groq.com/openai/v1",
    api_key=os.getenv("GROQ_API_KEY"),
    temperature=0.1,
    is_chat_model=True,
)
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
Settings.chunk_size = 512
Settings.chunk_overlap = 50

# Metadata for each document
DOCUMENT_METADATA = {
    "patient_monitor.md": {
        "device_type": "patient_monitor",
        "device_name": "CardioWatch Pro X200",
        "safety_class": "IEC 62304 Class C",
        "firmware_version": "4.2.1",
    },
    "infusion_pump.md": {
        "device_type": "infusion_pump",
        "device_name": "MedFlow Precision 3000",
        "safety_class": "IEC 62304 Class C",
        "firmware_version": "6.1.0",
    },
    "blood_pressure_monitor.md": {
        "device_type": "blood_pressure_monitor",
        "device_name": "VitalLink BP Connect",
        "safety_class": "IEC 62304 Class B",
        "firmware_version": "2.3.4",
    },
}

What does this code do?

  • Settings.llm connects LlamaIndex to Groq's Llama 3.3 70B model using the OpenAI-compatible API format. The low temperature=0.1 keeps responses factual and grounded.
  • Settings.embed_model uses HuggingFace's BAAI/bge-small-en-v1.5 model to convert text into vector embeddings locally. No API calls needed for embeddings.
  • Settings.chunk_size and Settings.chunk_overlap set global defaults for how documents get split into smaller pieces.
  • DOCUMENT_METADATA maps each filename to its device type, safety classification, and firmware version. Every chunk from that file inherits this metadata for filtering later.
  • Save create_rag_pipeline.py.

Seeing an import error?

  • Make sure your virtual environment is activated. You should see its name in parentheses at the start of your terminal prompt.
  • Verify all packages installed correctly by running pip list and checking for llama-index, chromadb, and llama-index-embeddings-huggingface.

Help me debug why my LlamaIndex imports are failing.

Understand the chunking strategy

Healthcare documents contain safety-critical information where context matters. A naive split could separate an alarm code from its threshold value, making retrieval unreliable.

Why 512 tokens with 50 token overlap?

A chunk size of 512 tokens is large enough to hold a complete subsection (like all the alarm thresholds for heart rate monitoring) without truncation. The 50-token overlap means adjacent chunks share boundary text, so a query about a threshold still finds the right chunk even if the split happened nearby.

The SentenceSplitter uses a paragraph_separator of "\n\n" to prefer splitting between paragraphs. This respects the natural section boundaries in your Markdown files, keeping related specifications together inside a single chunk.

Ingest documents into ChromaDB

Now you will write the function that ties everything together. It loads your three device manuals, attaches the metadata you defined, splits them with the structure-aware chunker, and stores the resulting vectors in a persistent ChromaDB database.

  • In create_rag_pipeline.py, add the following ingest_documents() function below the DOCUMENT_METADATA dictionary:
def ingest_documents():
    """Load documents and ingest into ChromaDB with metadata."""
    print("Loading documents from data/ directory...")
    documents = SimpleDirectoryReader("data").load_data()

    # Attach metadata to each document based on filename
    for doc in documents:
        filename = os.path.basename(doc.metadata.get("file_name", ""))
        if filename in DOCUMENT_METADATA:
            doc.metadata.update(DOCUMENT_METADATA[filename])

    print(f"Loaded {len(documents)} documents")

    # Set up structure-aware chunking
    node_parser = SentenceSplitter(
        chunk_size=512,
        chunk_overlap=50,
        paragraph_separator="\n\n",
    )

    # Set up ChromaDB persistent client
    chroma_client = chromadb.PersistentClient(path="./chroma_db")
    chroma_collection = chroma_client.get_or_create_collection(
        name="healthcare_iot_docs"
    )
    vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
    storage_context = StorageContext.from_defaults(vector_store=vector_store)

    # Create index with chunked documents
    print("Creating vector index with structure-aware chunking...")
    index = VectorStoreIndex.from_documents(
        documents,
        storage_context=storage_context,
        transformations=[node_parser],
    )

    print("Ingestion complete! Documents stored in ./chroma_db")
    return index

What does this function do?

  • SimpleDirectoryReader("data").load_data() reads every file in your data/ folder and creates a LlamaIndex Document object from each one.
  • The for loop matches each document's filename against DOCUMENT_METADATA and attaches device type, safety class, and firmware version as metadata fields.
  • SentenceSplitter splits documents into chunks at paragraph boundaries, keeping each chunk under 512 tokens with 50 tokens of overlap between adjacent chunks.
  • chromadb.PersistentClient(path="./chroma_db") creates a database on disk that survives between script runs. Your embeddings persist without re-processing.
  • VectorStoreIndex.from_documents() runs the full pipeline: chunk the documents, generate embeddings using the HuggingFace model, and store everything in ChromaDB.
  • Save create_rag_pipeline.py.
  • Run the ingestion pipeline by running this command:
python create_rag_pipeline.py

You should see output showing the documents loading, then a message confirming ingestion is complete. The first run downloads the HuggingFace embedding model (about 130MB), so it may take a minute.

What should I see?

Your terminal should display something like:

Loading documents from data/ directory...
Loaded 3 documents
Creating vector index with structure-aware chunking...
Ingestion complete! Documents stored in ./chroma_db

You should also see a new chroma_db/ folder appear in your project directory. This is your persistent vector database.

Seeing an API key error?

  • Check that your .env file contains GROQ_API_KEY=your-actual-key with no spaces around the equals sign.
  • Make sure load_dotenv() is at the top of your file (after imports). It reads the .env file into your environment.
  • If you see a connection timeout, the ingestion step does not call Groq. The embeddings are generated locally. The error may be from a later query step. Confirm the chroma_db/ folder was created successfully.

Help me fix my Groq API key or ChromaDB connection issue.

✔️ Awesome, I've got everything!

Great. Double check you have saved create_rag_pipeline.py and that the chroma_db/ folder exists in your project directory.

ⓧ I'd like to double check the full code

Here is the complete create_rag_pipeline.py file up to this point:

import os
import chromadb
from dotenv import load_dotenv
from llama_index.core import (
    SimpleDirectoryReader,
    VectorStoreIndex,
    StorageContext,
    Settings,
)
from llama_index.core.node_parser import SentenceSplitter
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

load_dotenv()

# Configure LlamaIndex settings with Groq API
Settings.llm = OpenAILike(
    model="llama-3.3-70b-versatile",
    api_base="https://api.groq.com/openai/v1",
    api_key=os.getenv("GROQ_API_KEY"),
    temperature=0.1,
    is_chat_model=True,
)
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
Settings.chunk_size = 512
Settings.chunk_overlap = 50

# Metadata for each document
DOCUMENT_METADATA = {
    "patient_monitor.md": {
        "device_type": "patient_monitor",
        "device_name": "CardioWatch Pro X200",
        "safety_class": "IEC 62304 Class C",
        "firmware_version": "4.2.1",
    },
    "infusion_pump.md": {
        "device_type": "infusion_pump",
        "device_name": "MedFlow Precision 3000",
        "safety_class": "IEC 62304 Class C",
        "firmware_version": "6.1.0",
    },
    "blood_pressure_monitor.md": {
        "device_type": "blood_pressure_monitor",
        "device_name": "VitalLink BP Connect",
        "safety_class": "IEC 62304 Class B",
        "firmware_version": "2.3.4",
    },
}


def ingest_documents():
    """Load documents and ingest into ChromaDB with metadata."""
    print("Loading documents from data/ directory...")
    documents = SimpleDirectoryReader("data").load_data()

    # Attach metadata to each document based on filename
    for doc in documents:
        filename = os.path.basename(doc.metadata.get("file_name", ""))
        if filename in DOCUMENT_METADATA:
            doc.metadata.update(DOCUMENT_METADATA[filename])

    print(f"Loaded {len(documents)} documents")

    # Set up structure-aware chunking
    node_parser = SentenceSplitter(
        chunk_size=512,
        chunk_overlap=50,
        paragraph_separator="\n\n",
    )

    # Set up ChromaDB persistent client
    chroma_client = chromadb.PersistentClient(path="./chroma_db")
    chroma_collection = chroma_client.get_or_create_collection(
        name="healthcare_iot_docs"
    )
    vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
    storage_context = StorageContext.from_defaults(vector_store=vector_store)

    # Create index with chunked documents
    print("Creating vector index with structure-aware chunking...")
    index = VectorStoreIndex.from_documents(
        documents,
        storage_context=storage_context,
        transformations=[node_parser],
    )

    print("Ingestion complete! Documents stored in ./chroma_db")
    return index

Your documents are now embedded and stored in a persistent vector database. Next up, you will build a query engine that searches these vectors and generates grounded answers using Groq's Llama 3.3 70B model.

Build the Query Engine with Citation Grounding

Your vector store is loaded with chunked healthcare documentation and rich metadata. Now it's time to build the component that actually answers questions.

A RAG query engine retrieves the most relevant chunks from your vector store and passes them to the LLM as context. The LLM then generates an answer grounded only in that retrieved evidence. In healthcare, this grounding is critical because fabricated alarm thresholds or made-up safety protocols could lead to dangerous decisions.

In this step, get ready to:
  • Connect the VectorStoreIndex to a retrieval-powered query engine.
  • Write a system prompt that enforces citation grounding and hallucination refusal.
  • Test the pipeline with three queries including one out-of-scope question.
Understand how the query engine connects to your index

Before writing new code, take a moment to understand the architecture you already have in place.

  • Switch back to create_rag_pipeline.py in your editor.
  • Notice that your ingest_documents() function already returns a VectorStoreIndex connected to your ChromaDB collection.
  • Notice that Settings.llm at the top of the file already configures Groq's Llama 3.3 70B model via the OpenAI-compatible endpoint.

How does the query engine work?

When you call index.as_query_engine(), LlamaIndex combines three pieces: the vector store (for retrieval), the embedding model (to vectorize your question), and the LLM (to generate a response from retrieved chunks). The similarity_top_k parameter controls how many chunks are passed to the LLM as context.

Add the query function with a citation-grounding prompt

The system prompt is what separates a trustworthy healthcare assistant from a general chatbot. It instructs the LLM to only answer from retrieved context and to refuse when information is insufficient.

  • In create_rag_pipeline.py, scroll to the end of the ingest_documents() function.
  • Add the following query_pipeline() function below it:
def query_pipeline(index, query_text):
    """Query the RAG pipeline with citation grounding."""
    query_engine = index.as_query_engine(
        similarity_top_k=5,
        system_prompt=(
            "You are a healthcare IoT technical assistant. "
            "Answer questions ONLY based on the provided context from device documentation. "
            "Always cite the specific device name and section when providing information. "
            "If the context does not contain enough information to answer the question, "
            "explicitly state: 'I don't have enough information in the available documentation to answer this question.' "
            "Never make up specifications, alarm codes, or safety thresholds."
        ),
    )
    response = query_engine.query(query_text)
    return response

What does this code do?

  • similarity_top_k=5 retrieves the 5 most semantically similar chunks from ChromaDB for every query. More chunks means more context, but too many can dilute relevance.
  • The system_prompt enforces three constraints: answer only from context, always cite the device name, and explicitly refuse when information is missing.
  • The refusal instruction ("I don't have enough information") is critical for healthcare. A fabricated alarm threshold could lead a clinician to ignore a real emergency.

Why a system prompt instead of fine-tuning?

Fine-tuning changes a model's weights permanently and requires extensive training data. A system prompt gives you the same behavioral guardrails with zero training cost and instant iteration. For healthcare retrieval, prompt-based constraints are the standard first line of defense against hallucination.

  • Save create_rag_pipeline.py.

You won't see a change yet. The next chunk adds the test code that actually runs this function.

Run the pipeline with test queries

Now you'll wire everything together with a main block that ingests documents and fires off three carefully chosen test queries.

  • Add this __main__ block at the very bottom of create_rag_pipeline.py:
if __name__ == "__main__":
    # Ingest documents
    index = ingest_documents()

    # Test queries
    test_queries = [
        "What is the bradycardia alarm threshold for the patient monitor?",
        "What network security protocols does the infusion pump use?",
        "What is the battery life of the patient monitor?",  # Not in docs - tests hallucination resistance
    ]

    print("\n" + "=" * 60)
    print("TESTING RAG PIPELINE")
    print("=" * 60)

    for query in test_queries:
        print(f"\nQuery: {query}")
        print("-" * 40)
        response = query_pipeline(index, query)
        print(f"Answer: {response}")
        print()

What do these three test queries check?

  • Query 1 (bradycardia alarm threshold) tests exact numeric recall. The correct answer is below 50 BPM with alarm code HR-BRADY-001.
  • Query 2 (infusion pump network security) tests retrieval of multi-fact security information like WPA3-Enterprise, 802.1X, and signed firmware packages.
  • Query 3 (patient monitor battery life) tests hallucination resistance. Battery life is NOT in the documentation, so the model should refuse to answer.
  • Save create_rag_pipeline.py.
  • Run the pipeline by executing this command in your terminal:
python create_rag_pipeline.py

The first run will download the embedding model (about 130 MB) if you haven't already. After ingestion completes, you should see three query-answer pairs printed to the terminal.

What should I see?

For query 1, the answer should reference the CardioWatch Pro X200, cite the Heart Rate Monitoring section, and state the threshold of below 50 BPM with alarm code HR-BRADY-001.

For query 2, the answer should cite the MedFlow Precision 3000's Network Security section and mention WPA3-Enterprise, 802.1X, and signed firmware packages.

For query 3, the answer should contain language like "I don't have enough information in the available documentation to answer this question." This proves your hallucination guardrail is working.

Seeing an API error or empty responses?

  • Check that your GROQ_API_KEY environment variable is set correctly in your .env file. The key should start with gsk_.
  • If you see a rate limit error, wait 60 seconds and try again. Groq's free tier has per-minute request limits.
  • If the embedding model download stalls, check your internet connection and re-run the script.

Help me debug my RAG pipeline query errors.

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

Here is the complete create_rag_pipeline.py file for comparison:

import os
import chromadb
from dotenv import load_dotenv
from llama_index.core import (
    SimpleDirectoryReader,
    VectorStoreIndex,
    StorageContext,
    Settings,
)
from llama_index.core.node_parser import SentenceSplitter
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

load_dotenv()

# Configure LlamaIndex settings with Groq API
Settings.llm = OpenAILike(
    model="llama-3.3-70b-versatile",
    api_base="https://api.groq.com/openai/v1",
    api_key=os.getenv("GROQ_API_KEY"),
    temperature=0.1,
    is_chat_model=True,
)
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")
Settings.chunk_size = 512
Settings.chunk_overlap = 50

# Metadata for each document
DOCUMENT_METADATA = {
    "patient_monitor.md": {
        "device_type": "patient_monitor",
        "device_name": "CardioWatch Pro X200",
        "safety_class": "IEC 62304 Class C",
        "firmware_version": "4.2.1",
    },
    "infusion_pump.md": {
        "device_type": "infusion_pump",
        "device_name": "MedFlow Precision 3000",
        "safety_class": "IEC 62304 Class C",
        "firmware_version": "6.1.0",
    },
    "blood_pressure_monitor.md": {
        "device_type": "blood_pressure_monitor",
        "device_name": "VitalLink BP Connect",
        "safety_class": "IEC 62304 Class B",
        "firmware_version": "2.3.4",
    },
}


def ingest_documents():
    """Load documents and ingest into ChromaDB with metadata."""
    print("Loading documents from data/ directory...")
    documents = SimpleDirectoryReader("data").load_data()

    # Attach metadata to each document based on filename
    for doc in documents:
        filename = os.path.basename(doc.metadata.get("file_name", ""))
        if filename in DOCUMENT_METADATA:
            doc.metadata.update(DOCUMENT_METADATA[filename])

    print(f"Loaded {len(documents)} documents")

    # Set up structure-aware chunking
    node_parser = SentenceSplitter(
        chunk_size=512,
        chunk_overlap=50,
        paragraph_separator="\n\n",
    )

    # Set up ChromaDB persistent client
    chroma_client = chromadb.PersistentClient(path="./chroma_db")
    chroma_collection = chroma_client.get_or_create_collection(
        name="healthcare_iot_docs"
    )
    vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
    storage_context = StorageContext.from_defaults(vector_store=vector_store)

    # Create index with chunked documents
    print("Creating vector index with structure-aware chunking...")
    index = VectorStoreIndex.from_documents(
        documents,
        storage_context=storage_context,
        transformations=[node_parser],
    )

    print("Ingestion complete! Documents stored in ./chroma_db")
    return index


def query_pipeline(index, query_text):
    """Query the RAG pipeline with citation grounding."""
    query_engine = index.as_query_engine(
        similarity_top_k=5,
        system_prompt=(
            "You are a healthcare IoT technical assistant. "
            "Answer questions ONLY based on the provided context from device documentation. "
            "Always cite the specific device name and section when providing information. "
            "If the context does not contain enough information to answer the question, "
            "explicitly state: 'I don't have enough information in the available documentation to answer this question.' "
            "Never make up specifications, alarm codes, or safety thresholds."
        ),
    )
    response = query_engine.query(query_text)
    return response


if __name__ == "__main__":
    # Ingest documents
    index = ingest_documents()

    # Test queries
    test_queries = [
        "What is the bradycardia alarm threshold for the patient monitor?",
        "What network security protocols does the infusion pump use?",
        "What is the battery life of the patient monitor?",  # Not in docs - tests hallucination resistance
    ]

    print("\n" + "=" * 60)
    print("TESTING RAG PIPELINE")
    print("=" * 60)

    for query in test_queries:
        print(f"\nQuery: {query}")
        print("-" * 40)
        response = query_pipeline(index, query)
        print(f"Answer: {response}")
        print()

Your query engine is working and grounding answers in real documentation. Next up, you'll put this pipeline through a rigorous accuracy evaluation using RAGAS metrics to get quantitative proof of how faithful and correct your system is.

Evaluate Accuracy with RAGAS

Your RAG pipeline is built, ingesting documents and returning cited answers. But how do you know those answers are actually correct?

In healthcare, "it seems to work" is not good enough. You need quantitative proof that the system stays grounded in its source documents and never fabricates information. RAGAS is an evaluation framework that scores your pipeline on metrics like faithfulness and factual correctness, giving you hard numbers instead of guesswork.

In this step, get ready to:
  • Create an evaluation dataset with ground truth answers covering different query types.
  • Run RAGAS metrics to measure faithfulness, context recall, and factual correctness.
  • Analyze scores and implement one improvement to your pipeline.
Create an evaluation dataset with ground truth answers

To measure RAG accuracy, you need a dataset of questions paired with known-correct answers. Each pair acts as a test case: you ask the pipeline the question, then compare its response against the ground truth.

  • Create a new file called evaluate_accuracy.py in your project folder.
  • Add the imports, settings configuration, and the load_index() function by pasting this code:
import os
from dotenv import load_dotenv
from ragas import evaluate, EvaluationDataset
from ragas.metrics import LLMContextRecall, Faithfulness, FactualCorrectness
from ragas.llms import LangchainLLMWrapper
from langchain_groq import ChatGroq
import chromadb
from llama_index.core import VectorStoreIndex, Settings
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

load_dotenv()

# Configure LlamaIndex settings with Groq
Settings.llm = OpenAILike(
    model="llama-3.3-70b-versatile",
    api_base="https://api.groq.com/openai/v1",
    api_key=os.getenv("GROQ_API_KEY"),
    temperature=0.1,
    is_chat_model=True,
)
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")


def load_index():
    """Load existing ChromaDB index."""
    chroma_client = chromadb.PersistentClient(path="./chroma_db")
    chroma_collection = chroma_client.get_collection(name="healthcare_iot_docs")
    vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
    index = VectorStoreIndex.from_vector_store(vector_store)
    return index

What does this code do?

  • The imports bring in RAGAS evaluation tools alongside the LlamaIndex and ChromaDB libraries you already used.
  • langchain_groq provides a LangChain-compatible wrapper for Groq, which RAGAS needs as its evaluator LLM.
  • load_index() reconnects to your existing ChromaDB collection without re-ingesting the documents.
  • The Settings block mirrors your pipeline configuration so queries use the same LLM and embeddings.
  • Save evaluate_accuracy.py.

Import errors?

Make sure your virtual environment is activated. The ragas and langchain-groq packages were installed in Step 1.

Confirm you have ragas==0.4.3 and langchain-groq==0.3.3 in your requirements.txt.

help me fix import errors in my evaluate_accuracy.py file

Now add the function that builds your evaluation dataset. Each entry pairs a question with a ground truth reference answer drawn directly from the device documentation.

  • Below the load_index() function, add the create_evaluation_dataset() function:
def create_evaluation_dataset(index):
    """Create evaluation dataset by querying the RAG pipeline."""
    eval_questions = [
        {
            "query": "What is the bradycardia alarm threshold and its alarm code?",
            "reference": "The bradycardia alarm threshold is below 50 BPM with alarm code HR-BRADY-001.",
        },
        {
            "query": "What encryption is used for data transmission on the patient monitor?",
            "reference": "The patient monitor uses TLS 1.3 encryption for all data transmitted.",
        },
        {
            "query": "What is the air-in-line detection threshold for the infusion pump?",
            "reference": "The air-in-line detection threshold is 50 microliters, using an ultrasonic sensor with alarm code AIIL-001.",
        },
        {
            "query": "What accuracy standard is the blood pressure monitor validated against?",
            "reference": "The VitalLink BP Connect is validated per ISO 81060-2:2018 with systolic and diastolic accuracy of +/- 3 mmHg mean error and SD less than 8 mmHg.",
        },
        {
            "query": "How often must the infusion pump device certificates rotate?",
            "reference": "Device certificates on the MedFlow Precision 3000 infusion pump rotate every 90 days.",
        },
    ]

    query_engine = index.as_query_engine(
        similarity_top_k=5,
        system_prompt=(
            "You are a healthcare IoT technical assistant. "
            "Answer questions ONLY based on the provided context from device documentation. "
            "Always cite the specific device name and section when providing information. "
            "If the context does not contain enough information to answer the question, "
            "explicitly state: 'I don't have enough information in the available documentation to answer this question.' "
            "Never make up specifications, alarm codes, or safety thresholds."
        ),
    )

    dataset = []
    for item in eval_questions:
        response = query_engine.query(item["query"])
        source_nodes = response.source_nodes
        retrieved_contexts = [node.get_content() for node in source_nodes]

        dataset.append(
            {
                "user_input": item["query"],
                "retrieved_contexts": retrieved_contexts,
                "response": str(response),
                "reference": item["reference"],
            }
        )
        print(f"Evaluated: {item['query'][:50]}...")

    return dataset

What does this function do?

This function runs each evaluation question through your RAG pipeline and captures everything RAGAS needs to score it:

  • user_input: the question asked.
  • retrieved_contexts: the actual document chunks the retriever found.
  • response: the LLM's generated answer.
  • reference: your ground truth answer from the original documentation.

The five questions cover different query types: exact numeric recall (alarm threshold), protocol lookup (encryption), safety interlock details, regulatory standards, and temporal facts (certificate rotation).

  • Save evaluate_accuracy.py.

Not sure where to add this function?

Place it directly below the load_index() function, at the same indentation level (no nesting). It should start at column 0, just like load_index().

help me place the create_evaluation_dataset function correctly

Run the RAGAS evaluation

With your evaluation dataset defined, you need a function that feeds it to RAGAS and computes the accuracy scores. RAGAS uses a separate evaluator LLM to judge your pipeline's outputs.

  • Below the create_evaluation_dataset() function, add the run_evaluation() function and the __main__ block:
def run_evaluation(dataset):
    """Run RAGAS evaluation on the dataset."""
    evaluation_dataset = EvaluationDataset.from_list(dataset)

    # Use Groq's fast Llama 3.1 8B as the evaluator LLM
    evaluator_llm = LangchainLLMWrapper(
        ChatGroq(model="llama-3.1-8b-instant", temperature=0.0)
    )

    print("\nRunning RAGAS evaluation...")
    result = evaluate(
        dataset=evaluation_dataset,
        metrics=[LLMContextRecall(), Faithfulness(), FactualCorrectness()],
        llm=evaluator_llm,
    )

    print("\n" + "=" * 60)
    print("RAGAS EVALUATION RESULTS")
    print("=" * 60)
    print(f"Context Recall:      {result['context_recall']:.4f}")
    print(f"Faithfulness:        {result['faithfulness']:.4f}")
    print(f"Factual Correctness: {result['factual_correctness']:.4f}")
    print("=" * 60)
    print("\nScore Interpretation:")
    print("  0.9+   = Excellent")
    print("  0.8-0.9 = Good")
    print("  0.7-0.8 = Acceptable")
    print("  Below 0.7 = Needs improvement")

    return result


if __name__ == "__main__":
    print("Loading existing index...")
    index = load_index()

    print("Creating evaluation dataset (querying RAG pipeline)...")
    dataset = create_evaluation_dataset(index)

    results = run_evaluation(dataset)

What does this code do?

  • EvaluationDataset.from_list() converts your list of dictionaries into the format RAGAS expects.
  • The evaluator LLM (llama-3.1-8b-instant via Groq) is a separate model that judges your pipeline's answers. Using a fast, small model keeps evaluation quick.
  • LLMContextRecall measures whether retrieval found all relevant information.
  • Faithfulness checks if the answer stays grounded in retrieved context (no hallucination).
  • FactualCorrectness compares the answer against your ground truth reference.
  • The __main__ block ties everything together: load the index, build the evaluation dataset, then run scoring.
  • Save evaluate_accuracy.py.
  • Run the evaluation script by running this command:
python evaluate_accuracy.py

The script queries your pipeline 5 times, then runs RAGAS scoring. This takes 1-2 minutes depending on Groq API response times.

You should see output showing each question being evaluated, followed by a results table with three scores between 0 and 1.

Seeing rate limit errors from Groq?

The free Groq tier has rate limits. If you see errors, wait 60 seconds and run the script again. The evaluation makes multiple API calls in sequence, which can sometimes exceed the per-minute token limit.

help me fix rate limit errors when running my RAGAS evaluation with Groq

Analyze results and identify improvements

Your RAGAS scores tell you exactly where your pipeline is strong and where it needs work. Each metric reveals a different dimension of accuracy.

How do I interpret my scores?

  • Context Recall below 0.8: your retriever is missing relevant chunks. The chunking strategy might be splitting important information across boundaries.
  • Faithfulness below 0.8: the LLM is adding information not present in the retrieved context. Your system prompt may need stronger grounding instructions.
  • Factual Correctness below 0.8: answers don't match ground truth. This could mean retrieval is finding the wrong chunks, or the LLM is misinterpreting the context.

For healthcare applications, Faithfulness is the most critical metric. A system that hallucinates medical specifications is dangerous.

One common improvement for technical documentation is increasing chunk overlap. When safety-critical information spans a section boundary, higher overlap ensures both chunks contain the full context.

  • Open create_rag_pipeline.py and find this line in the Settings configuration:
Settings.chunk_overlap = 50
  • Change 50 to 100 to increase the overlap between chunks:
Settings.chunk_overlap = 100
  • Also find the matching parameter in the SentenceSplitter inside ingest_documents():
    node_parser = SentenceSplitter(
        chunk_size=512,
        chunk_overlap=50,
        paragraph_separator="\n\n",
    )
  • Change chunk_overlap=50 to chunk_overlap=100 here as well:
    node_parser = SentenceSplitter(
        chunk_size=512,
        chunk_overlap=100,
        paragraph_separator="\n\n",
    )
  • Save create_rag_pipeline.py, then re-run the ingestion pipeline to rebuild the index with the new overlap:
python create_rag_pipeline.py
  • After ingestion completes, run the evaluation again to compare scores:
python evaluate_accuracy.py

Compare your new scores against the previous run. You may see improvements in Context Recall as more context is preserved at chunk boundaries.

What if my scores didn't improve?

Not every change improves every metric. If chunk overlap didn't help, your original chunking was already capturing the right boundaries. Other improvements to try include adding section headings to chunk text for better semantic matching, or increasing similarity_top_k to retrieve more context per query.

  • Set Settings.chunk_overlap back to 50 in create_rag_pipeline.py (both in Settings and in the SentenceSplitter) and save the file.

Keep a record of your experiments

In production RAG systems, teams log every chunking configuration alongside its RAGAS scores. This lets you make data-driven decisions about retrieval tuning rather than guessing.

✔️ Awesome, I've got everything!

Great. Double check you've saved evaluate_accuracy.py and that create_rag_pipeline.py is back to its original chunk_overlap=50 values.

ⓧ I'd like to double check the full code

Here is the complete evaluate_accuracy.py file:

import os
from dotenv import load_dotenv
from ragas import evaluate, EvaluationDataset
from ragas.metrics import LLMContextRecall, Faithfulness, FactualCorrectness
from ragas.llms import LangchainLLMWrapper
from langchain_groq import ChatGroq
import chromadb
from llama_index.core import VectorStoreIndex, Settings
from llama_index.vector_stores.chroma import ChromaVectorStore
from llama_index.llms.openai_like import OpenAILike
from llama_index.embeddings.huggingface import HuggingFaceEmbedding

load_dotenv()

Settings.llm = OpenAILike(
    model="llama-3.3-70b-versatile",
    api_base="https://api.groq.com/openai/v1",
    api_key=os.getenv("GROQ_API_KEY"),
    temperature=0.1,
    is_chat_model=True,
)
Settings.embed_model = HuggingFaceEmbedding(model_name="BAAI/bge-small-en-v1.5")


def load_index():
    chroma_client = chromadb.PersistentClient(path="./chroma_db")
    chroma_collection = chroma_client.get_collection(name="healthcare_iot_docs")
    vector_store = ChromaVectorStore(chroma_collection=chroma_collection)
    index = VectorStoreIndex.from_vector_store(vector_store)
    return index


def create_evaluation_dataset(index):
    eval_questions = [
        {
            "query": "What is the bradycardia alarm threshold and its alarm code?",
            "reference": "The bradycardia alarm threshold is below 50 BPM with alarm code HR-BRADY-001.",
        },
        {
            "query": "What encryption is used for data transmission on the patient monitor?",
            "reference": "The patient monitor uses TLS 1.3 encryption for all data transmitted.",
        },
        {
            "query": "What is the air-in-line detection threshold for the infusion pump?",
            "reference": "The air-in-line detection threshold is 50 microliters, using an ultrasonic sensor with alarm code AIIL-001.",
        },
        {
            "query": "What accuracy standard is the blood pressure monitor validated against?",
            "reference": "The VitalLink BP Connect is validated per ISO 81060-2:2018 with systolic and diastolic accuracy of +/- 3 mmHg mean error and SD less than 8 mmHg.",
        },
        {
            "query": "How often must the infusion pump device certificates rotate?",
            "reference": "Device certificates on the MedFlow Precision 3000 infusion pump rotate every 90 days.",
        },
    ]

    query_engine = index.as_query_engine(
        similarity_top_k=5,
        system_prompt=(
            "You are a healthcare IoT technical assistant. "
            "Answer questions ONLY based on the provided context from device documentation. "
            "Always cite the specific device name and section when providing information. "
            "If the context does not contain enough information to answer the question, "
            "explicitly state: 'I don't have enough information in the available documentation to answer this question.' "
            "Never make up specifications, alarm codes, or safety thresholds."
        ),
    )

    dataset = []
    for item in eval_questions:
        response = query_engine.query(item["query"])
        source_nodes = response.source_nodes
        retrieved_contexts = [node.get_content() for node in source_nodes]
        dataset.append(
            {
                "user_input": item["query"],
                "retrieved_contexts": retrieved_contexts,
                "response": str(response),
                "reference": item["reference"],
            }
        )
        print(f"Evaluated: {item['query'][:50]}...")
    return dataset


def run_evaluation(dataset):
    evaluation_dataset = EvaluationDataset.from_list(dataset)
    evaluator_llm = LangchainLLMWrapper(
        ChatGroq(model="llama-3.1-8b-instant", temperature=0.0)
    )
    print("\nRunning RAGAS evaluation...")
    result = evaluate(
        dataset=evaluation_dataset,
        metrics=[LLMContextRecall(), Faithfulness(), FactualCorrectness()],
        llm=evaluator_llm,
    )
    print("\n" + "=" * 60)
    print("RAGAS EVALUATION RESULTS")
    print("=" * 60)
    print(f"Context Recall:      {result['context_recall']:.4f}")
    print(f"Faithfulness:        {result['faithfulness']:.4f}")
    print(f"Factual Correctness: {result['factual_correctness']:.4f}")
    print("=" * 60)
    print("\nScore Interpretation:")
    print("  0.9+   = Excellent")
    print("  0.8-0.9 = Good")
    print("  0.7-0.8 = Acceptable")
    print("  Below 0.7 = Needs improvement")
    return result


if __name__ == "__main__":
    print("Loading existing index...")
    index = load_index()
    print("Creating evaluation dataset (querying RAG pipeline)...")
    dataset = create_evaluation_dataset(index)
    results = run_evaluation(dataset)

You now have quantitative proof of your pipeline's accuracy. Your RAGAS scores provide a baseline you can track as you tune retrieval strategies, adjust prompts, or swap models. Next up, you'll take on a Secret Mission to push your retrieval even further with hybrid search.

Secret mission

Add Hybrid Search for Medical Device Codes

Healthcare IoT documentation contains exact codes like HR-BRADY-001, AIIL-001, and protocol identifiers like HL7 FHIR R4. Pure semantic search can miss these precise strings because embeddings focus on meaning, not exact character matches. Can you add a query fusion strategy that improves recall for exact medical device codes and alarm identifiers?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely on your local machine with a free Groq API tier, so there are no ongoing costs.

Resources you used:

  • Project folder on your Desktop (containing create_rag_pipeline.py, evaluate_accuracy.py, create_rag_pipeline_hybrid.py, data/ directory, requirements.txt, .env, virtual environment, and ChromaDB stores).

Keep everything running

No action needed. Choose this if you plan to keep experimenting with your RAG pipeline or extending it with new documents and queries.

  • Your project folder stays on your Desktop with all scripts, data files, and ChromaDB vector stores intact.
  • Your virtual environment remains ready to activate whenever you want to run the pipeline again.
  • Your free Groq API key remains active in your account at console.groq.com. There is no charge for keeping it.

Pause - I'll come back to this later

Deactivate your virtual environment to free up your terminal session. Your files and ChromaDB stores remain on disk for when you return.

  • Deactivate your virtual environment by running:
deactivate
  • When you come back, navigate to your project folder and reactivate the virtual environment with source venv/bin/activate (macOS/Linux) or venv\Scripts\activate (Windows).
  • Your ChromaDB stores at ./chroma_db and ./chroma_db_hybrid persist between sessions. You do not need to re-ingest your documents.

Delete - I don't want to use this again

Remove all project files, ChromaDB vector stores, and the virtual environment from your machine.

  • Deactivate your virtual environment first (if it is still active) by running:
deactivate
  • Navigate to your Desktop and remove the project folder by running:
cd ~/Desktop
rm -rf [[PROJECT_FOLDER="your project folder name"]]

On Windows, use this command instead:

cd ~/Desktop
rmdir /s /q [[PROJECT_FOLDER="your project folder name"]]
  • This removes everything: your Python scripts, the data/ directory, both ChromaDB stores (./chroma_db and ./chroma_db_hybrid), your virtual environment, and the .env file containing your API key.
  • Your free Groq account at console.groq.com remains active but costs nothing. You can leave it as-is or revoke your API key from the Groq dashboard if you prefer.

Nice Work!

Nice Work!

You just built a production-grade RAG pipeline for healthcare IoT documentation. Your system ingests medical device manuals, retrieves relevant information with metadata traceability, and proves its accuracy with quantitative evaluation scores.

You've learned how to:

  • Build a complete RAG ingestion pipeline with structure-aware chunking and a metadata-enriched ChromaDB vector store for healthcare document traceability.
  • Implement a citation-grounded query engine powered by Groq (Llama 3.3 70B) that refuses to hallucinate and explicitly states when it lacks information.
  • Measure accuracy quantitatively using RAGAS evaluation metrics (Faithfulness, Context Recall, Factual Correctness) to prove your system returns grounded answers.
  • Added hybrid search using QueryFusionRetriever to handle exact medical device codes and alarm identifiers that semantic search alone can miss.

Ready to quiz yourself?