Build a CUDA and vLLM Retrieval Core

Introduction

30 Second Summary

Useful technical notes can still leave you scrolling through dozens of pages for one trustworthy passage. A single promising search result gives you little evidence that the next question will work as well.

In this project, you will turn your own CUDA and vLLM documents into a local retrieval layer that returns ranked evidence with stable citations. You will compare dense retrieval with cross-encoder reranking to measure quality against latency.

What You'll Build

You'll ask a real technical question and see five ranked excerpts with stable citations plus a benchmark that reveals which retrieval configuration works best for your evidence.

By the end of this project, you'll have:

• A searchable technical knowledge base that turns your PDF, Markdown, or text files into evidence you can query. Each result carries a stable source citation.
• A side-by-side retrieval check that shows how dense search ranks your hardest question. The reranked run lets you inspect whether the evidence order improves.
• A reproducible benchmark report that reveals the quality cost of faster retrieval. Pinned inputs plus corpus and evaluation hashes make the comparison traceable.
• Secret Mission: Re-index the same corpus with a 500-character chunking profile. Compare its retrieval quality and latency with the original profile.

Do I need any experience before starting?

You'll need basic Python plus 6 to 10 locally available CUDA or vLLM files. This project assumes Visual Studio Code, Docker Desktop, and an NVIDIA GPU are available on your Windows machine. Scanned image-only PDFs need OCR before they can supply text to this workflow.

Before We Start

A useful benchmark starts with technical evidence that reflects a retrieval problem you actually care about. Your representative corpus needs 6 to 10 digitally born CUDA or vLLM files in PDF, Markdown, or text format with enough material for at least 20 chunks.

Set Up the Reproducible Local Stack

A trustworthy benchmark needs a runtime you can recreate. Dependency drift or temporary vector storage would make later results difficult to trace.

This step creates a pinned Python environment for the retrieval code. You will also start a persistent local Qdrant service through Docker Desktop.

In this step, get ready to:

• Prepare an active Python virtual environment with pinned dependencies.
• Start a persistent local Qdrant service with Docker Compose.
• Verify the Qdrant dashboard and Python dependency imports.

Verify Python and create the environment

The folder containing data/corpus becomes your project workspace when you open it in Visual Studio Code. Its integrated terminal starts inside that folder.

• Press the Windows key to open system search.
• Type Visual Studio Code into the search field.
• Press Enter to launch Visual Studio Code.
• Select File from the top menu.
• Select Open Folder....
• Choose the folder that contains your existing data/corpus directory.
• Select Select Folder.
• Approve the workspace if Visual Studio Code asks whether you trust the files.

Why Open the Parent Folder?

Visual Studio Code treats the opened folder as your workspace. Every relative path in this project now starts from the same location.

• Select View from the top menu.
• Select Terminal.
• Select PowerShell from the terminal profile menu if another shell opens.

You will see a PowerShell prompt inside the project folder. Commands you run here create files beside data.

• Check which Python version the terminal resolves by running this command:

python --version

Why Check the Version?

The pinned packages support Python 3.10 through 3.12. The version check prevents an incompatible interpreter from creating your virtual environment.

✔️ I see a supported Python version

Your Python runtime is within the supported range. It can create this project's isolated environment.

ⓧ I see an unsupported version

Your current Python command resolves to a version outside the supported range. Install Python 3.12 so this project uses the target runtime.

• Open the official Python releases for Windows page.
• Find the Python 3.12 release entry.
• Download the Windows installer (64-bit).
• Run the downloaded installer.
• Select the installer option that adds Python to PATH.
• Complete the installation.
• Restart Visual Studio Code so its terminal reads the updated environment.
• Confirm the updated Python version by running this command:

python --version

What Does This Confirm?

This command shows the interpreter that your new PowerShell session resolves. Continue when it reports Python 3.10 through 3.12.

ⓧ Command not found

PowerShell cannot find a Python installation through its current environment. Install Python 3.12 from the official Windows release page.

• Open the official Python releases for Windows page.
• Find the Python 3.12 release entry.
• Download the Windows installer (64-bit).
• Run the downloaded installer.
• Select the installer option that adds Python to PATH.
• Complete the installation.
• Restart Visual Studio Code so its terminal reads the updated environment.
• Confirm that PowerShell can find Python by running this command:

python --version

What Does This Confirm?

This command asks the newly installed interpreter to report its version. Continue when it reports Python 3.10 through 3.12.

Still Seeing the Wrong Python Version?

Close every integrated terminal after installing Python. Open a fresh PowerShell terminal so Visual Studio Code reloads PATH.

If several Python versions are installed, confirm that the supported installation is available through the official Python launcher.

Ask for help with your specific version output:

A virtual environment keeps this project's packages separate from other Python projects. Activating it makes the local .venv interpreter the default for the current terminal.

• Create and activate .venv by running these commands:

python -m venv .venv
.\.venv\Scripts\Activate.ps1

What Do These Commands Do?

• The first command creates an isolated Python environment inside .venv.
• The second command activates that environment for the current PowerShell session.

You will see (.venv) at the start of the PowerShell prompt. That prefix confirms that package installs now stay inside this project.

Activation Did Not Change the Prompt?

Confirm that .venv appears in the Visual Studio Code Explorer. Run the activation command from the folder that contains it.

If PowerShell blocks the activation script, use this prompt to diagnose the local policy safely:

The dependency file pins every direct package used by this retrieval layer. These pins make a later benchmark traceable to one known software stack.

• Right-click the opened project folder in the Explorer sidebar.
• Choose the option for creating a new file.
• Name the file requirements.txt.
• Paste the following pinned dependencies into requirements.txt.

langchain-core==1.6.4
langchain-text-splitters==1.1.2
pypdf==6.19.0
qdrant-client==1.19.1
sentence-transformers==6.1.0

What Does This File Control?

LangChain Core provides the document abstraction. LangChain Text Splitters provides deterministic chunking.

pypdf extracts embedded PDF text. The Qdrant client connects Python to the local vector service.

Sentence Transformers provides the embedding model and reranker used later. Exact package versions reduce dependency drift between benchmark runs.

• Save requirements.txt in Visual Studio Code.

You will see requirements.txt beside the data directory in the Explorer sidebar.

✔️ Awesome, I've got everything!

Your dependency file is saved with all five pinned packages.

ⓧ I'd like to double check the full code

langchain-core==1.6.4
langchain-text-splitters==1.1.2
pypdf==6.19.0
qdrant-client==1.19.1
sentence-transformers==6.1.0

Package installation can take several minutes while Python resolves larger dependencies. Let the command finish before moving to Qdrant.

• Install the pinned dependencies into the active environment by running:

python -m pip install -r requirements.txt

What Does This Command Do?

Python runs the copy of pip attached to the active interpreter. The installer reads every exact version from requirements.txt.

Using the active interpreter keeps the dependency set inside .venv.

The terminal eventually returns to the (.venv) prompt without reporting an installation failure. Your isolated Python stack is now populated.

Dependency Installation Failed?

Confirm that the prompt begins with (.venv). Check that every line in requirements.txt matches the double-check tab.

Use this prompt to diagnose the package that failed: Share the non-secret terminal output with the NextWork community if the installation still fails.

Start pinned Qdrant with persistent storage

Qdrant stores the vectors created by your future ingestion runs. A named Docker volume keeps those vectors available when the container stops.

• Right-click the opened project folder in the Explorer sidebar.
• Choose the option for creating a new file.
• Name the file compose.yaml.
• Paste the following service definition into compose.yaml.

services:
  qdrant:
    image: qdrant/qdrant:v1.19.1
    ports:
      - "6333:6333"
      - "6334:6334"
    volumes:
      - qdrant_storage:/qdrant/storage

volumes:
  qdrant_storage:

What Does This Configuration Do?

• The qdrant/qdrant:v1.19.1 image pins the Qdrant server release.
• Port 6333 exposes the REST API and local dashboard.
• Port 6334 exposes the gRPC endpoint.
• The qdrant_storage volume persists collection data under the container's storage path.

• Save compose.yaml in Visual Studio Code.

You will see compose.yaml beside requirements.txt in the Explorer sidebar.

✔️ Awesome, I've got everything!

Your Compose file now defines the pinned Qdrant service and its persistent volume.

ⓧ I'd like to double check the full code

services:
  qdrant:
    image: qdrant/qdrant:v1.19.1
    ports:
      - "6333:6333"
      - "6334:6334"
    volumes:
      - qdrant_storage:/qdrant/storage

volumes:
  qdrant_storage:

Docker Desktop can take a minute to prepare its local engine after launch. A short wait here is normal.

• Press the Windows key to open system search.
• Type Docker Desktop into the search field.
• Press Enter to launch Docker Desktop.
• Wait until Docker Desktop reports that its engine is running.

The first start may download the pinned Qdrant image before creating the container. Keep Docker Desktop running while you use the retrieval project.

• Create the Qdrant container in the background by running:

docker compose up --detach

What Does This Command Do?

Docker Compose reads compose.yaml from the current project folder. It creates the Qdrant container plus the project network.

The --detach flag leaves the service running in the background. Your PowerShell terminal remains available for later commands.

The terminal returns to the prompt after the service starts. Qdrant continues running through Docker Desktop.

Qdrant Did Not Start?

Confirm that Docker Desktop has finished starting. Check that compose.yaml is saved in the folder shown by the Visual Studio Code Explorer.

Use this prompt to diagnose the Compose output: Share the non-secret output with the NextWork community if the container still does not start.

Before you open the dashboard, do you expect the local Qdrant service to answer on port 6333?

• Paste http://localhost:6333/dashboard into your browser address bar.

You will see the local Qdrant dashboard load. That page proves the container is reachable through the exposed REST port.

Dashboard Did Not Load?

Wait for the image download to finish if this is the first Qdrant start. Keep Docker Desktop running while you refresh the dashboard.

Use this prompt to check the container and port configuration:

Verify the Python dependencies

The environment variable RAG_DEVICE can force a specific compute device. Leaving it unset lets Sentence Transformers select an available accelerator or fall back to the CPU.

• Leave RAG_DEVICE unset in the active PowerShell terminal.

Before you run the final check, which message would prove that every direct Python dependency imports successfully?

• Import the five project dependencies by running:

python -c "import langchain_core, langchain_text_splitters, qdrant_client, sentence_transformers, pypdf; print('dependencies ready')"

What Does This Check Prove?

Python imports each direct dependency from the active .venv environment. The final print runs only after every import succeeds.

You will see dependencies ready in PowerShell. That output confirms the pinned Python stack is ready for ingestion.

Import Check Did Not Finish?

Confirm that the prompt still begins with (.venv). An import failure usually identifies the package that needs attention.

Use this prompt to diagnose the failing dependency without changing the pinned versions:

Your pinned environment now imports successfully. Next, you will turn your selected technical corpus into searchable chunks with stable citation metadata.

Ingest and Search Your Corpus

Your local stack is ready. However, your Qdrant service still needs searchable evidence from the technical files you selected.

This step turns those files into deterministic chunks with stable citation metadata. You will then use dense retrieval to surface five ranked excerpts for a real CUDA or vLLM question.

In this step, get ready to:

• Confirm that your selected technical files form a representative corpus.
• Build deterministic ingestion with content-addressed Qdrant storage.
• Run a dense search that returns five citation-ready excerpts.

Curate your technical corpus

A useful retrieval test needs enough variety to expose weak rankings. Your existing data/corpus folder should contain a small mix of documentation, papers, and notes that covers questions you genuinely care about.

• Review the files under data/corpus in the Visual Studio Code Explorer sidebar.
• Confirm that the folder contains between 6 and 10 files.
• Confirm that the file extensions are .md, .txt, or .pdf.
• Confirm that every PDF contains selectable text.
• Replace any image-only scanned PDF with a digitally born or OCRed copy.

Why exclude scanned PDFs?

The pypdf library extracts text already embedded in a PDF. It does not perform OCR on pages that contain only images.

Selectable text gives the ingestion pipeline content it can split, embed, and cite.

Your corpus is ready when the Explorer sidebar shows 6 to 10 supported files. The ingestion output will confirm whether they produce the target of at least 20 chunks.

Build deterministic ingestion

Deterministic ingestion gives the same source material the same chunk identities. The pipeline combines file hashes, character offsets, content hashes, and UUID5 identifiers so later benchmark results can point back to stable evidence.

The collection name also includes a hash of the corpus and chunking configuration. A changed corpus or chunking profile therefore produces a separate Qdrant collection instead of silently overwriting the previous experiment.

• Create rag_core.py beside compose.yaml by using the file-creation icon next to your project folder in the Explorer sidebar.

You should see rag_core.py listed at the same level as requirements.txt and compose.yaml.

• Select the second tab below.
• Copy the complete rag_core.py reference file.

✔️ Awesome, I've got everything!

Your rag_core.py file already matches the complete retrieval core. Continue with the ingestion run below.

ⓧ I'd like to double check the full code

from __future__ import annotations

import argparse
import hashlib
import json
import os
import platform
import sys
import uuid
from dataclasses import asdict, dataclass
from importlib.metadata import version
from pathlib import Path
from time import perf_counter
from typing import Any

from langchain_core.documents import Document
from langchain_text_splitters import RecursiveCharacterTextSplitter
from pypdf import PdfReader
from qdrant_client import QdrantClient, models
from sentence_transformers import CrossEncoder, SentenceTransformer

QDRANT_URL = os.getenv("QDRANT_URL", "http://localhost:6333")
RAG_DEVICE = os.getenv("RAG_DEVICE") or None
CORPUS_DIR = Path("data/corpus")
ARTIFACT_DIR = Path("artifacts")
INDEX_MANIFEST_PATH = ARTIFACT_DIR / "index_manifest.json"

EMBEDDING_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
EMBEDDING_REVISION = "3746fd5f4cfd46ae64fc781df53e7cbb7849eb62"
RERANKER_MODEL = "cross-encoder/ms-marco-MiniLM-L6-v2"
RERANKER_REVISION = "c5ee24cb16019beea0893ab7796b1df96625c6b8"

DEFAULT_CHUNK_SIZE = 800
DEFAULT_CHUNK_OVERLAP = 120
DENSE_TOP_K = 5
RERANK_CANDIDATES = 20
RERANK_TOP_K = 5
SUPPORTED_SUFFIXES = {".md", ".txt", ".pdf"}
DIRECT_PACKAGES = (
    "langchain-core",
    "langchain-text-splitters",
    "pypdf",
    "qdrant-client",
    "sentence-transformers",
)


@dataclass(frozen=True)
class RetrievedChunk:
    rank: int
    chunk_id: str
    source_id: str
    page: int | None
    start_index: int
    citation: str
    text: str
    dense_score: float
    rerank_score: float | None


@dataclass(frozen=True)
class RetrievalResponse:
    query: str
    configuration: str
    latency_ms: float
    results: list[RetrievedChunk]


def sha256_bytes(value: bytes) -> str:
    return hashlib.sha256(value).hexdigest()


def collect_files(corpus_dir: Path) -> list[Path]:
    files = sorted(
        path
        for path in corpus_dir.rglob("*")
        if path.is_file() and path.suffix.lower() in SUPPORTED_SUFFIXES
    )
    if not files:
        raise ValueError(
            f"No PDF, Markdown, or text files found under {corpus_dir.resolve()}"
        )
    return files


def snapshot_corpus(files: list[Path], corpus_dir: Path) -> dict[str, Any]:
    entries = []
    for path in files:
        entries.append(
            {
                "source_id": path.relative_to(corpus_dir).as_posix(),
                "sha256": sha256_bytes(path.read_bytes()),
                "bytes": path.stat().st_size,
            }
        )
    digest = sha256_bytes(
        json.dumps(entries, sort_keys=True, separators=(",", ":")).encode("utf-8")
    )
    return {"sha256": digest, "files": entries}


def load_documents(corpus_dir: Path) -> tuple[list[Document], dict[str, Any]]:
    files = collect_files(corpus_dir)
    corpus_snapshot = snapshot_corpus(files, corpus_dir)
    documents: list[Document] = []

    for path in files:
        source_id = path.relative_to(corpus_dir).as_posix()
        suffix = path.suffix.lower()
        if suffix == ".pdf":
            reader = PdfReader(str(path))
            for page_number, page in enumerate(reader.pages, start=1):
                text = page.extract_text() or ""
                if text.strip():
                    documents.append(
                        Document(
                            page_content=text,
                            metadata={
                                "source_id": source_id,
                                "source_type": "pdf",
                                "page": page_number,
                            },
                        )
                    )
        else:
            text = path.read_text(encoding="utf-8", errors="replace")
            if text.strip():
                documents.append(
                    Document(
                        page_content=text,
                        metadata={
                            "source_id": source_id,
                            "source_type": suffix.lstrip("."),
                            "page": None,
                        },
                    )
                )

    if not documents:
        raise ValueError("Files were found, but no extractable text was produced.")
    return documents, corpus_snapshot


def split_documents(
    documents: list[Document], chunk_size: int, chunk_overlap: int
) -> list[Document]:
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=chunk_overlap,
        add_start_index=True,
        separators=["\n## ", "\n### ", "\n\n", "\n", " ", ""],
    )
    chunks = splitter.split_documents(documents)

    for chunk in chunks:
        source_id = str(chunk.metadata["source_id"])
        page = chunk.metadata.get("page")
        start_index = int(chunk.metadata.get("start_index", 0))
        content_hash = sha256_bytes(chunk.page_content.encode("utf-8"))
        stable_key = f"{source_id}|{page}|{start_index}|{content_hash}"
        chunk.metadata["chunk_id"] = str(
            uuid.uuid5(uuid.NAMESPACE_URL, stable_key)
        )
        chunk.metadata["content_sha256"] = content_hash
    return chunks


def collection_name(
    corpus_sha256: str, chunk_size: int, chunk_overlap: int
) -> str:
    configuration = {
        "corpus_sha256": corpus_sha256,
        "embedding_model": EMBEDDING_MODEL,
        "embedding_revision": EMBEDDING_REVISION,
        "chunk_size": chunk_size,
        "chunk_overlap": chunk_overlap,
    }
    digest = sha256_bytes(
        json.dumps(configuration, sort_keys=True).encode("utf-8")
    )
    return f"technical_kb_{digest[:12]}"


def package_versions() -> dict[str, str]:
    return {package: version(package) for package in DIRECT_PACKAGES}


def build_embedder() -> SentenceTransformer:
    return SentenceTransformer(
        EMBEDDING_MODEL,
        revision=EMBEDDING_REVISION,
        device=RAG_DEVICE,
    )


def build_reranker() -> CrossEncoder:
    return CrossEncoder(
        RERANKER_MODEL,
        revision=RERANKER_REVISION,
        device=RAG_DEVICE,
    )


def write_json(path: Path, payload: dict[str, Any]) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(
        json.dumps(payload, indent=2, sort_keys=True),
        encoding="utf-8",
    )


def ingest(
    corpus_dir: Path = CORPUS_DIR,
    chunk_size: int = DEFAULT_CHUNK_SIZE,
    chunk_overlap: int = DEFAULT_CHUNK_OVERLAP,
    force: bool = False,
) -> dict[str, Any]:
    documents, corpus_snapshot = load_documents(corpus_dir)
    chunks = split_documents(documents, chunk_size, chunk_overlap)
    embedder = build_embedder()
    vectors = embedder.encode_document(
        [chunk.page_content for chunk in chunks],
        batch_size=32,
        show_progress_bar=True,
        normalize_embeddings=True,
    )
    vector_size = int(vectors.shape[1])
    target_collection = collection_name(
        corpus_snapshot["sha256"], chunk_size, chunk_overlap
    )

    client = QdrantClient(url=QDRANT_URL)
    exists = client.collection_exists(collection_name=target_collection)
    if exists and force:
        client.delete_collection(collection_name=target_collection)
        exists = False

    if not exists:
        client.create_collection(
            collection_name=target_collection,
            vectors_config=models.VectorParams(
                size=vector_size,
                distance=models.Distance.COSINE,
            ),
        )
        points = [
            models.PointStruct(
                id=str(chunk.metadata["chunk_id"]),
                vector=vector.tolist(),
                payload={
                    "chunk_id": str(chunk.metadata["chunk_id"]),
                    "source_id": str(chunk.metadata["source_id"]),
                    "source_type": str(chunk.metadata["source_type"]),
                    "page": chunk.metadata.get("page"),
                    "start_index": int(chunk.metadata.get("start_index", 0)),
                    "content_sha256": str(chunk.metadata["content_sha256"]),
                    "text": chunk.page_content,
                },
            )
            for chunk, vector in zip(chunks, vectors, strict=True)
        ]
        for start in range(0, len(points), 64):
            client.upsert(
                collection_name=target_collection,
                points=points[start : start + 64],
                wait=True,
            )
        action = "created"
    else:
        action = "reused"

    manifest = {
        "collection_name": target_collection,
        "qdrant_url": QDRANT_URL,
        "corpus": corpus_snapshot,
        "chunking": {
            "chunk_size": chunk_size,
            "chunk_overlap": chunk_overlap,
            "length_unit": "characters",
            "chunk_count": len(chunks),
        },
        "embedding": {
            "model_id": EMBEDDING_MODEL,
            "revision": EMBEDDING_REVISION,
            "normalize_embeddings": True,
            "vector_size": vector_size,
        },
        "reranker": {
            "model_id": RERANKER_MODEL,
            "revision": RERANKER_REVISION,
        },
        "requested_device": RAG_DEVICE or "auto",
        "python": sys.version,
        "platform": platform.platform(),
        "packages": package_versions(),
    }
    write_json(INDEX_MANIFEST_PATH, manifest)
    print(
        f"Collection {target_collection} {action}: "
        f"{len(chunks)} chunks from {len(corpus_snapshot['files'])} files"
    )
    print(f"Manifest: {INDEX_MANIFEST_PATH}")
    return manifest


def load_index_manifest(path: Path = INDEX_MANIFEST_PATH) -> dict[str, Any]:
    if not path.exists():
        raise FileNotFoundError("Run `python rag_core.py ingest --force` first.")
    return json.loads(path.read_text(encoding="utf-8"))


def make_citation(source_id: str, page: int | None, start_index: int) -> str:
    if page is not None:
        return f"[{source_id}#page={page}]"
    return f"[{source_id}#char={start_index}]"


class TechnicalRetriever:
    def __init__(
        self,
        include_reranker: bool = True,
        manifest_path: Path = INDEX_MANIFEST_PATH,
    ) -> None:
        self.manifest = load_index_manifest(manifest_path)
        self.collection_name = str(self.manifest["collection_name"])
        self.client = QdrantClient(url=QDRANT_URL)
        if not self.client.collection_exists(collection_name=self.collection_name):
            raise RuntimeError(
                f"Qdrant collection {self.collection_name} does not exist. Run ingest."
            )
        self.embedder = build_embedder()
        self.reranker = build_reranker() if include_reranker else None

    def retrieve(
        self,
        query: str,
        configuration: str = "dense",
    ) -> RetrievalResponse:
        if configuration not in {"dense", "dense_reranked"}:
            raise ValueError("configuration must be dense or dense_reranked")
        if configuration == "dense_reranked" and self.reranker is None:
            raise RuntimeError("This retriever was created without a reranker.")

        started = perf_counter()
        query_vector = self.embedder.encode_query(
            query,
            normalize_embeddings=True,
        )
        candidate_limit = (
            RERANK_CANDIDATES
            if configuration == "dense_reranked"
            else DENSE_TOP_K
        )
        response = self.client.query_points(
            collection_name=self.collection_name,
            query=query_vector.tolist(),
            with_payload=True,
            limit=candidate_limit,
        )

        candidates: list[dict[str, Any]] = []
        for point in response.points:
            payload = dict(point.payload or {})
            candidates.append(
                {
                    "chunk_id": str(payload["chunk_id"]),
                    "source_id": str(payload["source_id"]),
                    "page": (
                        int(payload["page"])
                        if payload.get("page") is not None
                        else None
                    ),
                    "start_index": int(payload.get("start_index", 0)),
                    "text": str(payload["text"]),
                    "dense_score": float(point.score),
                    "rerank_score": None,
                }
            )

        if configuration == "dense_reranked":
            pairs = [(query, candidate["text"]) for candidate in candidates]
            scores = self.reranker.predict(
                pairs,
                batch_size=32,
                show_progress_bar=False,
            )
            for candidate, score in zip(candidates, scores, strict=True):
                candidate["rerank_score"] = float(score)
            candidates.sort(
                key=lambda candidate: float(candidate["rerank_score"]),
                reverse=True,
            )
            candidates = candidates[:RERANK_TOP_K]

        results = []
        for rank, candidate in enumerate(candidates, start=1):
            results.append(
                RetrievedChunk(
                    rank=rank,
                    chunk_id=candidate["chunk_id"],
                    source_id=candidate["source_id"],
                    page=candidate["page"],
                    start_index=candidate["start_index"],
                    citation=make_citation(
                        candidate["source_id"],
                        candidate["page"],
                        candidate["start_index"],
                    ),
                    text=candidate["text"],
                    dense_score=candidate["dense_score"],
                    rerank_score=candidate["rerank_score"],
                )
            )

        return RetrievalResponse(
            query=query,
            configuration=configuration,
            latency_ms=round((perf_counter() - started) * 1000, 3),
            results=results,
        )


def main() -> None:
    parser = argparse.ArgumentParser(description="Technical RAG retrieval core")
    subparsers = parser.add_subparsers(dest="command", required=True)

    ingest_parser = subparsers.add_parser("ingest")
    ingest_parser.add_argument("--force", action="store_true")
    ingest_parser.add_argument("--chunk-size", type=int, default=DEFAULT_CHUNK_SIZE)
    ingest_parser.add_argument(
        "--chunk-overlap", type=int, default=DEFAULT_CHUNK_OVERLAP
    )

    search_parser = subparsers.add_parser("search")
    search_parser.add_argument("question")
    search_parser.add_argument(
        "--config",
        choices=("dense", "dense_reranked"),
        default="dense",
    )

    args = parser.parse_args()
    if args.command == "ingest":
        ingest(
            chunk_size=args.chunk_size,
            chunk_overlap=args.chunk_overlap,
            force=args.force,
        )
    else:
        retriever = TechnicalRetriever(
            include_reranker=args.config == "dense_reranked"
        )
        response = retriever.retrieve(args.question, args.config)
        print(json.dumps(asdict(response), indent=2))


if __name__ == "__main__":
    main()

How is the retrieval core organized?

• The file loaders preserve relative source paths and PDF page numbers.
• The RecursiveCharacterTextSplitter adds character start indexes before each chunk receives a content hash and stable UUID5 identifier.
• The ingestion path uses Sentence Transformers to create normalized vectors before storing them with citation metadata.
• The TechnicalRetriever class gives later project layers one stable retrieval interface.

• Paste the copied code into rag_core.py.
• Save rag_core.py with Ctrl+S.

The first ingestion is the slowest because it downloads the public embedding model before processing your corpus. A few minutes is normal when the model is not already cached.

Before you run the ingestion, what collection detail do you expect to change if the corpus or chunking values change?

• Build the content-addressed collection by running this command in the activated PowerShell terminal:

python rag_core.py ingest --force

What happens during ingestion?

• The loader hashes every supported source file before extracting its text and metadata.
• The splitter uses a chunk size of 800 characters with an overlap of 120 characters.
• The embedding model uses revision 3746fd5f4cfd46ae64fc781df53e7cbb7849eb62 with normalized vectors.
• The manifest captures the collection configuration and direct package versions as JSON.

You should see a collection name beginning with technical_kb_. The same output reports the chunk count and points to artifacts/index_manifest.json.

• Switch back to the Qdrant dashboard from the previous step.
• Confirm that a collection beginning with technical_kb_ is listed.

That is the ingestion backbone working. Your corpus now has stable chunk identities, normalized vectors, and a manifest that ties the index to its exact inputs.

Ingestion did not create a collection?

Confirm that the activated PowerShell terminal is still inside the project folder. Confirm that Qdrant remains reachable at http://localhost:6333.

If the output mentions missing files or missing extractable text, check that data/corpus contains supported files with readable text.

If the process stops while loading the model, confirm that your internet connection can access the public model repository. The automatic device setting can use CPU when CUDA is unavailable.

Help me diagnose why the ingestion command did not create my Qdrant collection.

Run your first dense search

A dense search embeds your question into the same vector space as the chunks. Qdrant ranks nearby vectors by cosine similarity before the retrieval core returns the five highest-scoring results.

Stable citations connect each excerpt to a source page or character offset. This lets you inspect the evidence before any generation layer is allowed to use it.

• Choose one technical question whose answer you already know from your corpus.
• Replace your real technical question in the command below with that question.

Before you run the search, which source do you expect to appear in the top five?

• Search the collection with the dense configuration by running this command:

python rag_core.py search "your real technical question" --config dense

How do you read the results?

• The rank field shows each excerpt's position from 1 through 5.
• The source_id field identifies the original file relative to data/corpus.
• The page and start_index fields locate the evidence inside its source.
• The dense_score field records vector similarity while citation provides a reusable source reference.

You should see five objects in the results array. Each object includes an excerpt plus its source_id, page or start_index, chunk_id, dense_score, and citation.

• Inspect the five excerpts in rank order.
• Confirm that the expected source appears among the results.
• Compare the returned excerpt with the answer you already know.

Search results missing or irrelevant?

Confirm that the ingestion output reported at least 20 chunks. A smaller corpus gives the ranking less evidence to choose from.

Check that your question uses the same technical vocabulary as the source material. Then inspect whether the expected document contains extractable text.

If the collection cannot be found, rerun the ingestion command before searching. This recreates the manifest that tells TechnicalRetriever which collection to query.

Help me understand why my dense search did not return the expected source.

Your retrieval core can now turn local CUDA and vLLM material into ranked, citation-ready evidence. Next, you will build an evaluation set that tests this dense configuration across six real technical questions.

Build the Retrieval Evaluation Set

Your citation-ready Qdrant index can retrieve five ranked excerpts for a real question. One successful search cannot tell you whether retrieval quality is dependable.

Now you will turn real corpus questions into a fixed evaluation set. A repeatable benchmark gives every later retrieval configuration the same evidence-backed test.

In this step, get ready to:

• Ground six technical questions in evidence from your corpus.
• Implement deterministic retrieval metrics in benchmark.py.
• Generate a dense retrieval baseline from the fixed evaluation set.

Author six evidence-backed questions

An evaluation case connects a realistic question to the source material that should answer it. The evidence excerpt proves why each listed source belongs in the expected result.

What makes an evaluation case defensible?

Each line in the JSON Lines file represents one retrieval case. Keep every object on one physical line so the benchmark can load it independently.

• The id field gives the case a unique label.
• The question field contains a real CUDA or vLLM question.
• The task_type field identifies the kind of retrieval task.
• The relevant_sources field contains one or more exact source_id paths from your dense search output.
• The evidence field contains short excerpts that support the expected answer.

• Switch back to Visual Studio Code from earlier.
• Expand the data folder in the Explorer sidebar.
• Create eval.jsonl inside the data folder.
• Return to the dense search output from the previous step.
• Select a real question whose answer is supported by one or more returned excerpts.
• Copy each supporting source_id exactly as it appears in the search output.
• Copy a short supporting excerpt into the case evidence.

Use Evidence You Can Defend

Choose sources because their excerpts answer the question. A high dense score alone does not make a source relevant.

Keep the questions specific enough that you can judge whether the expected evidence appears in the top five.

• Add one case for an exact configuration lookup.
• Add one case for a compatibility constraint.
• Add one case for a technical procedure.
• Add one case for a failure diagnosis.
• Add one case for an architecture explanation.
• Add one two-source comparison case with two exact source IDs.

Your six cases now cover different ways people search technical documentation. Their fixed order keeps future benchmark runs comparable.

• Keep each JSON object on one non-empty line.
• Save data/eval.jsonl.
• Confirm the file contains at least six non-empty lines.

Does a Case Lack Clear Evidence?

Rewrite broad questions so one or two corpus excerpts can support a concrete answer. Remove any source path that you selected only because it ranked highly.

Check every path against the exact source_id shown by dense search.

Help me review an evaluation case that lacks clear source evidence.

Implement deterministic retrieval metrics

The Python benchmark replays the cases in their saved order. It records retrieval quality beside the latency observed for each question.

What will the benchmark measure?

• Hit Rate@5 records whether at least one expected source appears in the first five results.
• MRR@5 rewards an expected source for appearing closer to rank one.
• Relevant-source recall@5 measures how much of the expected source set appears in the first five results.
• Retrieval latency records the time spent retrieving each question after a warm-up request.

• Create benchmark.py beside rag_core.py from the Explorer sidebar.
• Add the imports plus default paths by pasting this code:

from __future__ import annotations

import argparse
import hashlib
import json
import statistics
from dataclasses import asdict
from pathlib import Path
from typing import Any

from rag_core import TechnicalRetriever

DEFAULT_EVAL_PATH = Path("data/eval.jsonl")
DEFAULT_OUTPUT_PATH = Path("artifacts/benchmark_report.json")

What does this code set up?

• The standard-library imports provide command parsing, hashing, JSON handling, statistics, dataclass conversion, paths, and type hints.
• TechnicalRetriever reuses the retrieval interface you built earlier.
• DEFAULT_EVAL_PATH points to your fixed evaluation cases.
• DEFAULT_OUTPUT_PATH provides the default report destination.

• Save benchmark.py.
• Confirm benchmark.py appears beside rag_core.py in the Explorer sidebar.

Do the Imports Show a Warning?

Confirm the editor is using the activated .venv interpreter. Check that the import from rag_core.py matches the file name exactly.

Help me diagnose the imports in benchmark.py.

• Add evaluation case loading below the default paths by pasting this function:

def load_cases(path: Path) -> list[dict[str, Any]]:
    cases = [
        json.loads(line)
        for line in path.read_text(encoding="utf-8").splitlines()
        if line.strip()
    ]
    required = {"id", "question", "task_type", "relevant_sources", "evidence"}
    for case in cases:
        missing = required - case.keys()
        if missing:
            raise ValueError(f"Case {case.get('id', '<unknown>')} is missing {missing}")
        if not case["relevant_sources"] or not case["evidence"]:
            raise ValueError(
                f"Case {case['id']} needs relevant_sources and evidence."
            )
    if len(cases) < 6:
        raise ValueError("Create at least six real evaluation cases.")
    return cases

How does case validation work?

• load_cases() parses every non-empty line as an independent JSON object.
• The required set prevents incomplete cases from entering the benchmark.
• Empty source or evidence values cause the benchmark to stop with a focused message.
• The final check enforces the six-case minimum.

• Save benchmark.py.
• Confirm load_cases() appears below the default paths.

Seeing JSON Editing Problems?

Check that every line in data/eval.jsonl contains one complete JSON object. Remove blank fragments caused by wrapping an object across physical lines.

Help me find malformed JSON in my evaluation file.

• Add percentile selection below load_cases() by pasting this function:

def percentile(values: list[float], percentile_value: float) -> float:
    ordered = sorted(values)
    index = round((len(ordered) - 1) * percentile_value)
    return ordered[index]

Why calculate percentiles?

A sorted latency list lets the function select a representative position for p50 or p95. Percentiles expose typical latency plus slower tail behavior.

• Save benchmark.py.
• Confirm percentile() appears below load_cases().

Is the Percentile Function Misaligned?

Keep the three function-body lines indented beneath percentile(). Align the function declaration with load_cases().

Help me correct indentation in the percentile function.

• Start the configuration evaluator below percentile() by pasting this code:

def evaluate_configuration(
    retriever: TechnicalRetriever,
    cases: list[dict[str, Any]],
    configuration: str,
) -> dict[str, Any]:
    retriever.retrieve(cases[0]["question"], configuration)
    details = []

    for case in cases:
        response = retriever.retrieve(case["question"], configuration)
        expected = set(case["relevant_sources"])
        retrieved_sources = [result.source_id for result in response.results]
        relevant_ranks = [
            rank
            for rank, source_id in enumerate(retrieved_sources, start=1)
            if source_id in expected
        ]

How does evaluation begin?

• The first retrieval warms the model before measured case runs begin.
• Each case uses the same retrieval interface and selected configuration.
• The expected source paths become a set for direct membership checks.
• relevant_ranks records where expected sources appear in the returned top five.

• Save benchmark.py.
• Confirm the warm-up call appears before the measured case loop.

Does the Loop Indentation Look Wrong?

Keep the retrieval lines inside the for case in cases loop. Keep the list-comprehension lines aligned beneath relevant_ranks.

Help me correct the evaluation loop indentation.

• Continue the case loop directly below relevant_ranks by pasting this code:

        unique_relevant = expected.intersection(retrieved_sources)
        details.append(
            {
                "id": case["id"],
                "task_type": case["task_type"],
                "question": case["question"],
                "expected_sources": sorted(expected),
                "retrieved_sources": retrieved_sources,
                "hit_at_5": 1.0 if relevant_ranks else 0.0,
                "reciprocal_rank_at_5": (
                    1.0 / relevant_ranks[0] if relevant_ranks else 0.0
                ),
                "relevant_source_recall_at_5": len(unique_relevant) / len(expected),
                "latency_ms": response.latency_ms,
                "results": [asdict(result) for result in response.results],
            }
        )

What does each case record?

• The intersection counts distinct expected sources found in the retrieved results.
• hit_at_5 becomes one when any expected source appears.
• reciprocal_rank_at_5 uses the first relevant rank.
• The case detail retains latency plus complete citation-ready results for later inspection.

• Save benchmark.py.
• Confirm the appended block remains inside the case loop.

Are Case Details Outside the Loop?

Align unique_relevant with the other statements inside the loop. Keep the dictionary contents indented beneath details.append().

Help me align the per-case metric block.

• Complete evaluate_configuration() directly below the case loop by pasting this code:

    latencies = [detail["latency_ms"] for detail in details]
    count = len(details)
    return {
        "summary": {
            "questions": count,
            "hit_rate_at_5": sum(item["hit_at_5"] for item in details) / count,
            "mrr_at_5": sum(
                item["reciprocal_rank_at_5"] for item in details
            )
            / count,
            "mean_relevant_source_recall_at_5": sum(
                item["relevant_source_recall_at_5"] for item in details
            )
            / count,
            "mean_latency_ms": statistics.fmean(latencies),
            "p50_latency_ms": percentile(latencies, 0.50),
            "p95_latency_ms": percentile(latencies, 0.95),
        },
        "cases": details,
    }

How are aggregate metrics produced?

• The summary averages quality measurements across the fixed case count.
• statistics.fmean() calculates mean retrieval latency.
• The percentile helper calculates p50 plus p95 latency from the per-case values.
• The return value keeps the aggregate summary beside every detailed case.

• Save benchmark.py.
• Confirm the aggregate return block aligns with the start of the case loop.

Is the Summary Nested Inside One Case?

Align latencies with the for case in cases statement. This placement aggregates all completed case details.

Help me place the aggregate summary after the case loop.

• Start the command-line entry point below evaluate_configuration() by pasting this code:

def main() -> None:
    parser = argparse.ArgumentParser(description="Reproducible retrieval benchmark")
    parser.add_argument("--eval", type=Path, default=DEFAULT_EVAL_PATH)
    parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT_PATH)
    parser.add_argument(
        "--configs",
        nargs="+",
        choices=("dense", "dense_reranked"),
        default=["dense", "dense_reranked"],
    )
    args = parser.parse_args()

    cases = load_cases(args.eval)
    retriever = TechnicalRetriever(
        include_reranker="dense_reranked" in args.configs
    )
    results = {
        configuration: evaluate_configuration(retriever, cases, configuration)
        for configuration in args.configs
    }

How does the CLI select a run?

• The parser accepts an evaluation path, an output path, and one or more retrieval configurations.
• The selected configuration list determines whether the retriever loads the reranker.
• The results comprehension evaluates every requested configuration against the same ordered cases.

• Save benchmark.py.
• Confirm main() appears below the evaluation function.

Are the CLI Arguments Missing?

Keep each parser.add_argument() call inside main(). Check that the configuration choices match the names accepted by TechnicalRetriever.

Help me review the benchmark command-line parser.

• Add comparison status handling inside main() by pasting this code:

    comparison_status = (
        "complete" if set(args.configs) == {"dense", "dense_reranked"}
        else "missing_second_configuration"
    )
    comparison = None
    if comparison_status == "complete":
        dense = results["dense"]["summary"]
        reranked = results["dense_reranked"]["summary"]
        comparison = {
            "mrr_at_5_delta": reranked["mrr_at_5"] - dense["mrr_at_5"],
            "source_recall_at_5_delta": (
                reranked["mean_relevant_source_recall_at_5"]
                - dense["mean_relevant_source_recall_at_5"]
            ),
            "p95_latency_ms_delta": (
                reranked["p95_latency_ms"] - dense["p95_latency_ms"]
            ),
        }

When can the report compare results?

The status becomes complete only when both named configurations have results. A single-configuration run leaves comparison empty because no delta can be calculated.

• Save benchmark.py.
• Confirm the three delta calculations remain inside the complete-status condition.

Do the Delta Lines Show Indentation Issues?

Keep the dense and reranked summaries inside the status condition. Align all three delta keys inside the comparison dictionary.

Help me correct the comparison block.

• Build the report payload below the comparison block by pasting this code:

    manifest = retriever.manifest
    report = {
        "comparison_status": comparison_status,
        "evaluation_sha256": hashlib.sha256(args.eval.read_bytes()).hexdigest(),
        "evaluation_cases": len(cases),
        "index_manifest": manifest,
        "configurations": results,
        "comparison": comparison,
        "generation_metrics": {
            "status": "not_measured_in_retrieval_only_project",
            "answer_faithfulness": None,
            "input_tokens": None,
            "output_tokens": None,
            "cost_usd": None,
        },
    }

Why store this report context?

• The evaluation SHA-256 changes when the exact bytes in data/eval.jsonl change.
• The index manifest carries the corpus, chunking, model, device, platform, and package context into the report.
• The generation fields stay explicitly unmeasured because this project evaluates retrieval only.

• Save benchmark.py.
• Confirm generation_metrics remains inside the top-level report dictionary.

Is the Report Dictionary Unbalanced?

Match every opening brace with a closing brace. Keep the generation_metrics dictionary nested inside report.

Help me check the report dictionary structure.

• Complete the report writer directly below the payload by pasting this code:

    args.output.parent.mkdir(parents=True, exist_ok=True)
    args.output.write_text(
        json.dumps(report, indent=2, sort_keys=True),
        encoding="utf-8",
    )
    print(
        json.dumps(
            {
                "comparison_status": comparison_status,
                "summary": {
                    name: result["summary"] for name, result in results.items()
                },
                "output": str(args.output),
            },
            indent=2,
        )
    )


if __name__ == "__main__":
    main()

How does the benchmark finish?

• The output folder is created when it does not already exist.
• The complete report is written as sorted, indented JSON.
• A smaller summary is printed in the terminal for the immediate check.
• The final guard runs main() when you launch the file as a script.

• Save benchmark.py.
• Confirm the file ends with the call to main().

Does the File End Inside main()?

Keep the final condition aligned at the left edge of the file. Indent only the main() call beneath it.

Help me correct the end of benchmark.py.

✔️ Awesome, I've got everything!

Great. Save benchmark.py before running the baseline.

ⓧ I'd like to double check the full code

from __future__ import annotations

import argparse
import hashlib
import json
import statistics
from dataclasses import asdict
from pathlib import Path
from typing import Any

from rag_core import TechnicalRetriever

DEFAULT_EVAL_PATH = Path("data/eval.jsonl")
DEFAULT_OUTPUT_PATH = Path("artifacts/benchmark_report.json")


def load_cases(path: Path) -> list[dict[str, Any]]:
    cases = [
        json.loads(line)
        for line in path.read_text(encoding="utf-8").splitlines()
        if line.strip()
    ]
    required = {"id", "question", "task_type", "relevant_sources", "evidence"}
    for case in cases:
        missing = required - case.keys()
        if missing:
            raise ValueError(f"Case {case.get('id', '<unknown>')} is missing {missing}")
        if not case["relevant_sources"] or not case["evidence"]:
            raise ValueError(
                f"Case {case['id']} needs relevant_sources and evidence."
            )
    if len(cases) < 6:
        raise ValueError("Create at least six real evaluation cases.")
    return cases


def percentile(values: list[float], percentile_value: float) -> float:
    ordered = sorted(values)
    index = round((len(ordered) - 1) * percentile_value)
    return ordered[index]


def evaluate_configuration(
    retriever: TechnicalRetriever,
    cases: list[dict[str, Any]],
    configuration: str,
) -> dict[str, Any]:
    retriever.retrieve(cases[0]["question"], configuration)
    details = []

    for case in cases:
        response = retriever.retrieve(case["question"], configuration)
        expected = set(case["relevant_sources"])
        retrieved_sources = [result.source_id for result in response.results]
        relevant_ranks = [
            rank
            for rank, source_id in enumerate(retrieved_sources, start=1)
            if source_id in expected
        ]
        unique_relevant = expected.intersection(retrieved_sources)
        details.append(
            {
                "id": case["id"],
                "task_type": case["task_type"],
                "question": case["question"],
                "expected_sources": sorted(expected),
                "retrieved_sources": retrieved_sources,
                "hit_at_5": 1.0 if relevant_ranks else 0.0,
                "reciprocal_rank_at_5": (
                    1.0 / relevant_ranks[0] if relevant_ranks else 0.0
                ),
                "relevant_source_recall_at_5": len(unique_relevant) / len(expected),
                "latency_ms": response.latency_ms,
                "results": [asdict(result) for result in response.results],
            }
        )

    latencies = [detail["latency_ms"] for detail in details]
    count = len(details)
    return {
        "summary": {
            "questions": count,
            "hit_rate_at_5": sum(item["hit_at_5"] for item in details) / count,
            "mrr_at_5": sum(
                item["reciprocal_rank_at_5"] for item in details
            )
            / count,
            "mean_relevant_source_recall_at_5": sum(
                item["relevant_source_recall_at_5"] for item in details
            )
            / count,
            "mean_latency_ms": statistics.fmean(latencies),
            "p50_latency_ms": percentile(latencies, 0.50),
            "p95_latency_ms": percentile(latencies, 0.95),
        },
        "cases": details,
    }


def main() -> None:
    parser = argparse.ArgumentParser(description="Reproducible retrieval benchmark")
    parser.add_argument("--eval", type=Path, default=DEFAULT_EVAL_PATH)
    parser.add_argument("--output", type=Path, default=DEFAULT_OUTPUT_PATH)
    parser.add_argument(
        "--configs",
        nargs="+",
        choices=("dense", "dense_reranked"),
        default=["dense", "dense_reranked"],
    )
    args = parser.parse_args()

    cases = load_cases(args.eval)
    retriever = TechnicalRetriever(
        include_reranker="dense_reranked" in args.configs
    )
    results = {
        configuration: evaluate_configuration(retriever, cases, configuration)
        for configuration in args.configs
    }

    comparison_status = (
        "complete" if set(args.configs) == {"dense", "dense_reranked"}
        else "missing_second_configuration"
    )
    comparison = None
    if comparison_status == "complete":
        dense = results["dense"]["summary"]
        reranked = results["dense_reranked"]["summary"]
        comparison = {
            "mrr_at_5_delta": reranked["mrr_at_5"] - dense["mrr_at_5"],
            "source_recall_at_5_delta": (
                reranked["mean_relevant_source_recall_at_5"]
                - dense["mean_relevant_source_recall_at_5"]
            ),
            "p95_latency_ms_delta": (
                reranked["p95_latency_ms"] - dense["p95_latency_ms"]
            ),
        }

    manifest = retriever.manifest
    report = {
        "comparison_status": comparison_status,
        "evaluation_sha256": hashlib.sha256(args.eval.read_bytes()).hexdigest(),
        "evaluation_cases": len(cases),
        "index_manifest": manifest,
        "configurations": results,
        "comparison": comparison,
        "generation_metrics": {
            "status": "not_measured_in_retrieval_only_project",
            "answer_faithfulness": None,
            "input_tokens": None,
            "output_tokens": None,
            "cost_usd": None,
        },
    }
    args.output.parent.mkdir(parents=True, exist_ok=True)
    args.output.write_text(
        json.dumps(report, indent=2, sort_keys=True),
        encoding="utf-8",
    )
    print(
        json.dumps(
            {
                "comparison_status": comparison_status,
                "summary": {
                    name: result["summary"] for name, result in results.items()
                },
                "output": str(args.output),
            },
            indent=2,
        )
    )


if __name__ == "__main__":
    main()

Expose the dense-only limitation

Your fixed cases can now produce a repeatable dense retrieval baseline. The warm-up request keeps model startup outside the recorded query latencies.

Before you run the benchmark, do you think one configuration can justify a production retrieval choice?

• Return to the activated .venv in PowerShell.
• Run the dense-only benchmark with this command:

python benchmark.py --configs dense --output artifacts/baseline_report.json

PowerShell prints a JSON summary containing dense metrics plus the output path. You will see "comparison_status": "missing_second_configuration" in that summary.

This Missing Comparison Is Intentional

The report measures all six questions through dense retrieval. It has no second result set for calculating quality or latency deltas.

This baseline exposes the production decision gap that a controlled comparison must close.

• Switch back to Visual Studio Code.
• Select artifacts/baseline_report.json in the Explorer sidebar.
• Confirm evaluation_cases is 6 or higher.
• Confirm comparison is null.
• Confirm comparison_status is missing_second_configuration.
• Confirm the dense configuration contains aggregate metrics plus per-case results.
• Confirm the report contains the index manifest plus the evaluation SHA-256 digest.

Did the Baseline Run Stop Early?

A case-loading failure points to malformed JSON, a missing required field, empty evidence, or fewer than six cases. Correct the named case in data/eval.jsonl before running the benchmark again.

A retrieval failure usually means local Qdrant is unavailable or the collection recorded in artifacts/index_manifest.json cannot be found.

Help me diagnose why the dense-only benchmark failed.

Your benchmark now turns six real questions into reproducible quality and latency measurements. Next, you will add another ranking configuration so the report can make an evidence-based comparison.

Add Reranking and Citation Results

Your dense-only baseline measures retrieval quality. It also exposes the missing comparison that blocks a production decision.

A second configuration uses cross-encoder reranking to rescore a broader candidate pool. The retriever must preserve one stable response contract so later application layers can switch configurations without changing their integration.

In this step, get ready to:

• Confirm the stable retrieval response contract.
• Trace the dense retrieval and reranking paths.
• Compare one hard question across both configurations.

Confirm the stable retrieval contract

A retrieval contract defines the fields every search returns. Both configurations use the same dataclasses so downstream code receives the same shape.

• Locate the RetrievedChunk dataclass in rag_core.py.
• Confirm that RetrievedChunk contains rank, chunk_id, source_id, page, start_index, citation, text, dense_score, and rerank_score.
• Locate the RetrievalResponse dataclass in the same file.
• Confirm that RetrievalResponse contains query, configuration, latency_ms, and results.

Why Keep One Response Contract?

The optional rerank_score lets dense results use None while reranked results carry a second score. Future generation or API code can consume either configuration through RetrievalResponse.

Stable citation fields keep the evidence traceable even when reranking changes its position.

Trace the reranking path

Dense mode asks Qdrant for the five closest vectors. Reranked mode retrieves 20 candidates before Sentence Transformers scores each query-passage pair.

The final sort uses the cross-encoder score. The best five candidates still become the same RetrievedChunk objects.

• Locate TechnicalRetriever.retrieve() in rag_core.py.
• Confirm that configuration accepts dense or dense_reranked.
• Trace the dense_reranked branch from candidate retrieval through pair scoring.
• Confirm that the branch sorts by rerank_score before returning five results.

How Does Reranking Work?

• The DENSE_TOP_K value keeps five results for dense retrieval.
• The RERANK_CANDIDATES value broadens reranked retrieval to 20 candidates.
• The RERANK_TOP_K value returns the best five rescored candidates.
• The RERANKER_REVISION value pins cross-encoder/ms-marco-MiniLM-L6-v2 to c5ee24cb16019beea0893ab7796b1df96625c6b8.

✔️ Awesome, I've got everything!

Your existing rag_core.py contains the stable contract and both retrieval paths.

ⓧ I'd like to double check the full code

from __future__ import annotations

import argparse
import hashlib
import json
import os
import platform
import sys
import uuid
from dataclasses import asdict, dataclass
from importlib.metadata import version
from pathlib import Path
from time import perf_counter
from typing import Any

from langchain_core.documents import Document
from langchain_text_splitters import RecursiveCharacterTextSplitter
from pypdf import PdfReader
from qdrant_client import QdrantClient, models
from sentence_transformers import CrossEncoder, SentenceTransformer

QDRANT_URL = os.getenv("QDRANT_URL", "http://localhost:6333")
RAG_DEVICE = os.getenv("RAG_DEVICE") or None
CORPUS_DIR = Path("data/corpus")
ARTIFACT_DIR = Path("artifacts")
INDEX_MANIFEST_PATH = ARTIFACT_DIR / "index_manifest.json"

EMBEDDING_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
EMBEDDING_REVISION = "3746fd5f4cfd46ae64fc781df53e7cbb7849eb62"
RERANKER_MODEL = "cross-encoder/ms-marco-MiniLM-L6-v2"
RERANKER_REVISION = "c5ee24cb16019beea0893ab7796b1df96625c6b8"

DEFAULT_CHUNK_SIZE = 800
DEFAULT_CHUNK_OVERLAP = 120
DENSE_TOP_K = 5
RERANK_CANDIDATES = 20
RERANK_TOP_K = 5
SUPPORTED_SUFFIXES = {".md", ".txt", ".pdf"}
DIRECT_PACKAGES = (
    "langchain-core",
    "langchain-text-splitters",
    "pypdf",
    "qdrant-client",
    "sentence-transformers",
)


@dataclass(frozen=True)
class RetrievedChunk:
    rank: int
    chunk_id: str
    source_id: str
    page: int | None
    start_index: int
    citation: str
    text: str
    dense_score: float
    rerank_score: float | None


@dataclass(frozen=True)
class RetrievalResponse:
    query: str
    configuration: str
    latency_ms: float
    results: list[RetrievedChunk]


def sha256_bytes(value: bytes) -> str:
    return hashlib.sha256(value).hexdigest()


def collect_files(corpus_dir: Path) -> list[Path]:
    files = sorted(
        path
        for path in corpus_dir.rglob("*")
        if path.is_file() and path.suffix.lower() in SUPPORTED_SUFFIXES
    )
    if not files:
        raise ValueError(
            f"No PDF, Markdown, or text files found under {corpus_dir.resolve()}"
        )
    return files


def snapshot_corpus(files: list[Path], corpus_dir: Path) -> dict[str, Any]:
    entries = []
    for path in files:
        entries.append(
            {
                "source_id": path.relative_to(corpus_dir).as_posix(),
                "sha256": sha256_bytes(path.read_bytes()),
                "bytes": path.stat().st_size,
            }
        )
    digest = sha256_bytes(
        json.dumps(entries, sort_keys=True, separators=(",", ":")).encode("utf-8")
    )
    return {"sha256": digest, "files": entries}


def load_documents(corpus_dir: Path) -> tuple[list[Document], dict[str, Any]]:
    files = collect_files(corpus_dir)
    corpus_snapshot = snapshot_corpus(files, corpus_dir)
    documents: list[Document] = []

    for path in files:
        source_id = path.relative_to(corpus_dir).as_posix()
        suffix = path.suffix.lower()
        if suffix == ".pdf":
            reader = PdfReader(str(path))
            for page_number, page in enumerate(reader.pages, start=1):
                text = page.extract_text() or ""
                if text.strip():
                    documents.append(
                        Document(
                            page_content=text,
                            metadata={
                                "source_id": source_id,
                                "source_type": "pdf",
                                "page": page_number,
                            },
                        )
                    )
        else:
            text = path.read_text(encoding="utf-8", errors="replace")
            if text.strip():
                documents.append(
                    Document(
                        page_content=text,
                        metadata={
                            "source_id": source_id,
                            "source_type": suffix.lstrip("."),
                            "page": None,
                        },
                    )
                )

    if not documents:
        raise ValueError("Files were found, but no extractable text was produced.")
    return documents, corpus_snapshot


def split_documents(
    documents: list[Document], chunk_size: int, chunk_overlap: int
) -> list[Document]:
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=chunk_overlap,
        add_start_index=True,
        separators=["\n## ", "\n### ", "\n\n", "\n", " ", ""],
    )
    chunks = splitter.split_documents(documents)

    for chunk in chunks:
        source_id = str(chunk.metadata["source_id"])
        page = chunk.metadata.get("page")
        start_index = int(chunk.metadata.get("start_index", 0))
        content_hash = sha256_bytes(chunk.page_content.encode("utf-8"))
        stable_key = f"{source_id}|{page}|{start_index}|{content_hash}"
        chunk.metadata["chunk_id"] = str(
            uuid.uuid5(uuid.NAMESPACE_URL, stable_key)
        )
        chunk.metadata["content_sha256"] = content_hash
    return chunks


def collection_name(
    corpus_sha256: str, chunk_size: int, chunk_overlap: int
) -> str:
    configuration = {
        "corpus_sha256": corpus_sha256,
        "embedding_model": EMBEDDING_MODEL,
        "embedding_revision": EMBEDDING_REVISION,
        "chunk_size": chunk_size,
        "chunk_overlap": chunk_overlap,
    }
    digest = sha256_bytes(
        json.dumps(configuration, sort_keys=True).encode("utf-8")
    )
    return f"technical_kb_{digest[:12]}"


def package_versions() -> dict[str, str]:
    return {package: version(package) for package in DIRECT_PACKAGES}


def build_embedder() -> SentenceTransformer:
    return SentenceTransformer(
        EMBEDDING_MODEL,
        revision=EMBEDDING_REVISION,
        device=RAG_DEVICE,
    )


def build_reranker() -> CrossEncoder:
    return CrossEncoder(
        RERANKER_MODEL,
        revision=RERANKER_REVISION,
        device=RAG_DEVICE,
    )


def write_json(path: Path, payload: dict[str, Any]) -> None:
    path.parent.mkdir(parents=True, exist_ok=True)
    path.write_text(
        json.dumps(payload, indent=2, sort_keys=True),
        encoding="utf-8",
    )


def ingest(
    corpus_dir: Path = CORPUS_DIR,
    chunk_size: int = DEFAULT_CHUNK_SIZE,
    chunk_overlap: int = DEFAULT_CHUNK_OVERLAP,
    force: bool = False,
) -> dict[str, Any]:
    documents, corpus_snapshot = load_documents(corpus_dir)
    chunks = split_documents(documents, chunk_size, chunk_overlap)
    embedder = build_embedder()
    vectors = embedder.encode_document(
        [chunk.page_content for chunk in chunks],
        batch_size=32,
        show_progress_bar=True,
        normalize_embeddings=True,
    )
    vector_size = int(vectors.shape[1])
    target_collection = collection_name(
        corpus_snapshot["sha256"], chunk_size, chunk_overlap
    )

    client = QdrantClient(url=QDRANT_URL)
    exists = client.collection_exists(collection_name=target_collection)
    if exists and force:
        client.delete_collection(collection_name=target_collection)
        exists = False

    if not exists:
        client.create_collection(
            collection_name=target_collection,
            vectors_config=models.VectorParams(
                size=vector_size,
                distance=models.Distance.COSINE,
            ),
        )
        points = [
            models.PointStruct(
                id=str(chunk.metadata["chunk_id"]),
                vector=vector.tolist(),
                payload={
                    "chunk_id": str(chunk.metadata["chunk_id"]),
                    "source_id": str(chunk.metadata["source_id"]),
                    "source_type": str(chunk.metadata["source_type"]),
                    "page": chunk.metadata.get("page"),
                    "start_index": int(chunk.metadata.get("start_index", 0)),
                    "content_sha256": str(chunk.metadata["content_sha256"]),
                    "text": chunk.page_content,
                },
            )
            for chunk, vector in zip(chunks, vectors, strict=True)
        ]
        for start in range(0, len(points), 64):
            client.upsert(
                collection_name=target_collection,
                points=points[start : start + 64],
                wait=True,
            )
        action = "created"
    else:
        action = "reused"

    manifest = {
        "collection_name": target_collection,
        "qdrant_url": QDRANT_URL,
        "corpus": corpus_snapshot,
        "chunking": {
            "chunk_size": chunk_size,
            "chunk_overlap": chunk_overlap,
            "length_unit": "characters",
            "chunk_count": len(chunks),
        },
        "embedding": {
            "model_id": EMBEDDING_MODEL,
            "revision": EMBEDDING_REVISION,
            "normalize_embeddings": True,
            "vector_size": vector_size,
        },
        "reranker": {
            "model_id": RERANKER_MODEL,
            "revision": RERANKER_REVISION,
        },
        "requested_device": RAG_DEVICE or "auto",
        "python": sys.version,
        "platform": platform.platform(),
        "packages": package_versions(),
    }
    write_json(INDEX_MANIFEST_PATH, manifest)
    print(
        f"Collection {target_collection} {action}: "
        f"{len(chunks)} chunks from {len(corpus_snapshot['files'])} files"
    )
    print(f"Manifest: {INDEX_MANIFEST_PATH}")
    return manifest


def load_index_manifest(path: Path = INDEX_MANIFEST_PATH) -> dict[str, Any]:
    if not path.exists():
        raise FileNotFoundError("Run `python rag_core.py ingest --force` first.")
    return json.loads(path.read_text(encoding="utf-8"))


def make_citation(source_id: str, page: int | None, start_index: int) -> str:
    if page is not None:
        return f"[{source_id}#page={page}]"
    return f"[{source_id}#char={start_index}]"


class TechnicalRetriever:
    def __init__(
        self,
        include_reranker: bool = True,
        manifest_path: Path = INDEX_MANIFEST_PATH,
    ) -> None:
        self.manifest = load_index_manifest(manifest_path)
        self.collection_name = str(self.manifest["collection_name"])
        self.client = QdrantClient(url=QDRANT_URL)
        if not self.client.collection_exists(collection_name=self.collection_name):
            raise RuntimeError(
                f"Qdrant collection {self.collection_name} does not exist. Run ingest."
            )
        self.embedder = build_embedder()
        self.reranker = build_reranker() if include_reranker else None

    def retrieve(
        self,
        query: str,
        configuration: str = "dense",
    ) -> RetrievalResponse:
        if configuration not in {"dense", "dense_reranked"}:
            raise ValueError("configuration must be dense or dense_reranked")
        if configuration == "dense_reranked" and self.reranker is None:
            raise RuntimeError("This retriever was created without a reranker.")

        started = perf_counter()
        query_vector = self.embedder.encode_query(
            query,
            normalize_embeddings=True,
        )
        candidate_limit = (
            RERANK_CANDIDATES
            if configuration == "dense_reranked"
            else DENSE_TOP_K
        )
        response = self.client.query_points(
            collection_name=self.collection_name,
            query=query_vector.tolist(),
            with_payload=True,
            limit=candidate_limit,
        )

        candidates: list[dict[str, Any]] = []
        for point in response.points:
            payload = dict(point.payload or {})
            candidates.append(
                {
                    "chunk_id": str(payload["chunk_id"]),
                    "source_id": str(payload["source_id"]),
                    "page": (
                        int(payload["page"])
                        if payload.get("page") is not None
                        else None
                    ),
                    "start_index": int(payload.get("start_index", 0)),
                    "text": str(payload["text"]),
                    "dense_score": float(point.score),
                    "rerank_score": None,
                }
            )

        if configuration == "dense_reranked":
            pairs = [(query, candidate["text"]) for candidate in candidates]
            scores = self.reranker.predict(
                pairs,
                batch_size=32,
                show_progress_bar=False,
            )
            for candidate, score in zip(candidates, scores, strict=True):
                candidate["rerank_score"] = float(score)
            candidates.sort(
                key=lambda candidate: float(candidate["rerank_score"]),
                reverse=True,
            )
            candidates = candidates[:RERANK_TOP_K]

        results = []
        for rank, candidate in enumerate(candidates, start=1):
            results.append(
                RetrievedChunk(
                    rank=rank,
                    chunk_id=candidate["chunk_id"],
                    source_id=candidate["source_id"],
                    page=candidate["page"],
                    start_index=candidate["start_index"],
                    citation=make_citation(
                        candidate["source_id"],
                        candidate["page"],
                        candidate["start_index"],
                    ),
                    text=candidate["text"],
                    dense_score=candidate["dense_score"],
                    rerank_score=candidate["rerank_score"],
                )
            )

        return RetrievalResponse(
            query=query,
            configuration=configuration,
            latency_ms=round((perf_counter() - started) * 1000, 3),
            results=results,
        )


def main() -> None:
    parser = argparse.ArgumentParser(description="Technical RAG retrieval core")
    subparsers = parser.add_subparsers(dest="command", required=True)

    ingest_parser = subparsers.add_parser("ingest")
    ingest_parser.add_argument("--force", action="store_true")
    ingest_parser.add_argument("--chunk-size", type=int, default=DEFAULT_CHUNK_SIZE)
    ingest_parser.add_argument(
        "--chunk-overlap", type=int, default=DEFAULT_CHUNK_OVERLAP
    )

    search_parser = subparsers.add_parser("search")
    search_parser.add_argument("question")
    search_parser.add_argument(
        "--config",
        choices=("dense", "dense_reranked"),
        default="dense",
    )

    args = parser.parse_args()
    if args.command == "ingest":
        ingest(
            chunk_size=args.chunk_size,
            chunk_overlap=args.chunk_overlap,
            force=args.force,
        )
    else:
        retriever = TechnicalRetriever(
            include_reranker=args.config == "dense_reranked"
        )
        response = retriever.retrieve(args.question, args.config)
        print(json.dumps(asdict(response), indent=2))


if __name__ == "__main__":
    main()

What Does This File Contain?

This reference combines deterministic ingestion with both retrieval configurations. It also preserves the citation metadata required by later generation and API layers.

Compare a hard question

A higher rerank score matters when stronger evidence moves closer to rank one. Comparing the same question keeps the evidence need fixed while the retrieval strategy changes.

• Identify the hardest question in data/eval.jsonl.
• Locate the matching dense result in artifacts/baseline_report.json.
• Keep its source ordering visible for comparison.

Before you run the reranked search, which expected source do you think will move closest to rank one?

• Replace your hardest evaluation question in the command below with the exact question from data/eval.jsonl.
• Run the reranked search by using the edited command:

python rag_core.py search "your hardest evaluation question" --config dense_reranked

What Does This Search Test?

• The dense_reranked configuration retrieves 20 dense candidates before rescoring them.
• The cross-encoder scores the question beside every candidate passage.
• The command returns five results through the existing response contract.

The first reranked search can take longer while the public model files are downloaded. Once it finishes, you'll see a JSON response containing five ranked results.

Each result includes its citation fields, excerpt, dense score, and rerank score. The top-level response includes the total latency_ms value.

• Compare the reranked source_id order with the dense order in artifacts/baseline_report.json.
• Check whether the expected citation moved closer to rank one.
• Compare the returned excerpts with the evidence in data/eval.jsonl.
• Compare the reranked latency_ms with the dense latency for the same case.

Reranked Search Did Not Finish?

Confirm that Qdrant is still running through the Compose stack from earlier. The retriever needs the original content-addressed collection before it can fetch candidates.

A first reranked run may pause while the model files download. Keep the terminal open while that download completes.

Check that the question remains enclosed in quotation marks if it contains spaces.

Help me diagnose my reranked retrieval command.

You now have one retrieval interface that can return dense or reranked evidence with stable citations. The quality-versus-latency tradeoff is visible for a real question from your corpus.

Your second retrieval configuration is ready. Next, you'll run both configurations across the unchanged evaluation set and freeze the production comparison.

Compare and Freeze Configurations

Both retrieval paths now share one response contract. You can compare dense retrieval with cross-encoder reranking without changing how downstream code reads the results.

One successful question cannot support a production decision. This step uses a fixed retrieval evaluation to measure quality against latency across your real technical questions.

In this step, get ready to:

• Run both retrieval configurations against the unchanged evaluation set.
• Confirm the report freezes every input needed to reproduce the comparison.
• Record which configuration fits your future API target.

Run the controlled comparison

A controlled comparison changes only the retrieval configuration. The corpus, evaluation cases, question order, and active Qdrant collection stay fixed.

Before you run this, do you expect reranking to improve MRR@5 enough to justify its added p95 latency?

• Run the controlled comparison from the PowerShell terminal where .venv is active by running:

python benchmark.py --configs dense dense_reranked --output artifacts/benchmark_report.json

What Does This Benchmark Freeze?

• The --configs option evaluates dense and dense_reranked against the same cases.
• The benchmark warms each retrieval path before recording query latency.
• The --output option stores the full comparison in artifacts/benchmark_report.json.

The terminal prints a JSON summary containing "comparison_status": "complete" plus summaries for both configurations. It also names artifacts/benchmark_report.json as the output.

You now have the evidence the dense-only baseline was missing. Both retrieval designs have been tested against the same questions.

Benchmark Stopped Early?

• Confirm Qdrant from earlier still shows as running in Docker Desktop.
• Check that the virtual environment from earlier is still active in your PowerShell terminal.
• Read the final traceback line to identify whether the failure came from the evaluation file, model loading, or Qdrant connection.

Help me diagnose the benchmark failure.

Inspect the frozen report

A reproducible result needs enough context to explain where every number came from. The experiment manifest preserves those inputs beside the metrics.

A SHA-256 digest fingerprints the exact corpus and evaluation bytes. A changed file produces a different digest on a future run.

• Switch back to Visual Studio Code from earlier.
• Expand the artifacts folder in the Explorer sidebar.
• Select benchmark_report.json.
• Find the comparison_status field near the top of the report.
• Scroll through the configurations object.
• Locate the evaluation_sha256 field.
• Inspect the index_manifest object.
• Inspect the generation_metrics object.

You should see dense and dense_reranked summaries with case-level results. Each summary includes hit_rate_at_5, mrr_at_5, mean_relevant_source_recall_at_5, mean_latency_ms, p50_latency_ms, and p95_latency_ms.

The index_manifest retains the collection name, corpus hashes, chunking values, model revisions, requested device, platform, Python version, and package versions. The generation_metrics status remains not_measured_in_retrieval_only_project.

Why Freeze the Inputs?

Quality scores only support a comparison when both configurations use the same evidence. The corpus hash and evaluation hash expose any changed input.

The model revisions, chunking values, package versions, and platform details explain how the index was produced. Latency can still vary across hardware or system load.

Make the production decision

Quality gains matter when they improve the questions you care about. The added p95 latency must fit the response-time target for your future API.

How to Read the Trade-Off

• A positive mrr_at_5_delta means expected sources moved closer to the top on average.
• A positive source_recall_at_5_delta means the top five captured more of the sources required by each question.
• The p95_latency_ms_delta measures the added latency near the slow end of your six-or-more-case run.

• Compare mrr_at_5_delta with the dense-only result.
• Compare source_recall_at_5_delta with the evidence needs in your hardest questions.
• Compare p95_latency_ms_delta with your future API latency target.
• Review the per-case results for questions where the source ordering changed.
• Select the configuration that provides the best useful quality within your latency target.

Before the final check, which frozen fields would expose a changed corpus or evaluation set on a future run?

• Return to the top of artifacts/benchmark_report.json.
• Confirm comparison_status has the value complete.
• Confirm evaluation_sha256 contains a digest.
• Inspect the corpus object inside index_manifest.
• Confirm answer_faithfulness, input_tokens, output_tokens, and cost_usd remain null.

You will see a complete comparison beside the evaluation digest. Inside index_manifest, the corpus object contains its own sha256 value plus per-file hashes.

The generation fields stay unmeasured because this report benchmarks retrieval only. Your chosen configuration is now supported by measured quality and p95 latency.

That is the production decision locked in. Your retrieval baseline now records the evidence, trade-off, and inputs needed for a repeatable future comparison.

Secret mission

Secret Mission: Benchmark a Second Chunking Profile

Re-index the same CUDA/vLLM corpus with a 500-character chunking profile. Benchmark the unchanged evaluation set to decide whether the quality shift earns its latency cost.

Clean Up Your Resources

Clean Up Your Resources

Choose whether to keep your local retrieval stack active, pause it for later, or remove its indexed data. This project runs locally through Docker Compose with no paid cloud resources.

Resources you used:

• A running Qdrant 1.19.1 service container.
• A Docker Compose project network for the local service.
• The named qdrant_storage volume holding both content-addressed collections.

What Stays on Disk?

All local project files remain on disk under every option below. This includes .venv, data/corpus, data/eval.jsonl, the Python files, plus the reports in artifacts.

Your local Hugging Face model cache also remains available. You can reuse the downloaded models without fetching them again.

Keep everything running

No action is needed. Choose this if you plan to connect a generation layer or API to the retrieval core soon.

• Keep the Qdrant service available at http://localhost:6333.
• Preserve artifacts/benchmark_report.json as the original retrieval baseline.
• Preserve artifacts/benchmark_chunks_500.json as the second chunking-profile comparison.
• Reuse both indexed collections in later generation experiments.

Pause - I'll come back to this later

Stop the running service to free local memory. Your named volume keeps both indexed collections ready for a later restart.

• Stop Qdrant without deleting its data by running this command:

docker compose stop

What Does This Command Do?

Docker Compose stops the running Qdrant service. The qdrant_storage volume remains attached to the project.

• Refresh http://localhost:6333/dashboard to verify that Qdrant has stopped.

Your browser can no longer reach the dashboard. This confirms that the container is paused while its indexed data remains stored.

Qdrant Still Running?

• Make sure PowerShell is still in the project folder containing compose.yaml.
• Make sure Docker Desktop is running before you retry the command.
• Help me troubleshoot why Docker Compose cannot stop my local Qdrant service.

Delete - I don't want to use this again

Remove every Docker Compose resource created for this project. Choose this only if you no longer need either indexed collection.

Both Indexed Collections Will Be Deleted

Removing qdrant_storage is irreversible. Both indexed collections disappear with it.

Your source documents remain safe on disk. Your benchmark reports remain safe on disk.

• Remove the Compose containers, network, and named Qdrant volume by running this command:

docker compose down --volumes

What Does This Command Remove?

Docker Compose removes the Qdrant container. It also removes the project network.

The --volumes flag removes the named qdrant_storage volume. Both content-addressed collections are deleted with that volume.

• Refresh http://localhost:6333/dashboard to verify that the local service is gone.

Your browser can no longer reach the Qdrant dashboard. The corpus, code, evaluation set, reports, virtual environment, and model cache remain on disk.

Cleanup Command Failed?

• Make sure PowerShell is in the project folder containing compose.yaml.
• Make sure Docker Desktop is running before you retry the cleanup.
• Help me troubleshoot why Docker Compose cannot remove my Qdrant stack.

Nice Work!

Nice Work!

That is a major milestone. You have built a benchmarked retrieval core that can surface ranked evidence from your own CUDA and vLLM knowledge base.

You've learned how to:

• Build a deterministic ingestion path that converts technical documents into stable chunks with source metadata.
• Return citation-ready evidence through one retrieval interface that supports dense retrieval plus cross-encoder reranking.
• Produce a manifest-backed benchmark that connects retrieval quality plus latency to frozen experiment inputs.
• Secret Mission: Re-index the same corpus with chunk size 500 plus overlap 75. Choose the profile to carry into generation from its measured quality-versus-latency tradeoff.

Your retrieval decisions now rest on reproducible evidence. This baseline is ready for later generation, observability, API, or deployment layers.

Ready to quiz yourself?