Billion-Scale Vector Search on Kubernetes

Deploy Cassandra 5.0 on Kubernetes and build a telecom AI search system.

Introduction

30 Second Summary

When you call your phone company about a dropped signal, someone on the other end is searching through thousands of troubleshooting documents to find the one that matches your problem. At massive scale, companies need to find the right document out of billions in milliseconds.

In this project, you will deploy a Cassandra 5.0 database on a local Kubernetes cluster, embed telecom support documents using a local AI model, and build a vector search system that answers natural-language questions by finding the most semantically relevant documents.

What You'll Build

You will type a question like "why is my 5G connection dropping?" and watch your system instantly return the most relevant troubleshooting documents, filtered by category, with measured speed comparisons proving why your optimizations matter.

By the end of this project, you'll have:

  • A running vector search system where you type a natural-language question and get back the most relevant telecom troubleshooting docs ranked by semantic similarity.
  • A hybrid retrieval pipeline that combines AI-powered similarity search with structured category filters so irrelevant results never appear.
  • A measured performance comparison showing the speed difference between naive cosine search and optimized dot-product search with HNSW-graph-powered indexing.
  • Secret Mission: Implement partition-aware data modeling to simulate multi-tenant telecom search where each region's data stays isolated for faster queries.

Are there any prerequisites?

You need Docker Desktop installed on your machine (with WSL2 on Windows), Python 3.12, and VS Code. Step 1 walks you through enabling Kubernetes in Docker Desktop and installing the Python packages.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters for your deployment skills.

Set Up Kubernetes and Python Environment

Your project needs two foundations: a running Kubernetes cluster to host your database, and Python packages to generate embeddings and communicate with Cassandra.

Docker Desktop includes a built-in single-node Kubernetes cluster you can activate with a single checkbox. This eliminates hours of cloud setup and gives you a production-grade orchestrator running locally.

In this step, get ready to:
  • Enable Kubernetes in Docker Desktop and confirm the cluster is running.
  • Install the Python packages for database connectivity and embedding generation.
Enable Kubernetes in Docker Desktop

Docker Desktop ships with a single-node Kubernetes cluster that runs alongside your containers. You just need to turn it on.

  • Press the Windows key to open your search bar.
  • Type Docker Desktop and press Enter to open it.
  • Click the gear icon in the top-right corner to open Settings.
  • Select Kubernetes in the left sidebar.
  • Check the Enable Kubernetes checkbox.
  • Click Apply & Restart.
  • Wait for the green indicator beside Kubernetes running to appear in the bottom-left of the Docker Desktop window.

What did enabling Kubernetes do?

Docker Desktop launched a single-node Kubernetes cluster on your machine. This cluster has its own networking, DNS, and container scheduling.

It also installed kubectl, the command-line tool you will use to manage the cluster. The cluster behaves identically to a production multi-node setup, just on one machine.

Kubernetes stuck on Starting?

The first time you enable Kubernetes, Docker downloads several container images. This can take 2-5 minutes depending on your internet speed.

If it stays on Starting for more than 5 minutes, click Reset Kubernetes Cluster in the same settings panel, then click Apply & Restart again.

Help me troubleshoot Kubernetes not starting in Docker Desktop

Verify Kubernetes and install Python packages

Now that the green indicator is visible, let's confirm the cluster responds to commands and then install your project dependencies.

  • Press the Windows key to open your search bar.
  • Type PowerShell and press Enter to open it.

Before you run the next command, what do you expect to see if Kubernetes started successfully?

  • Confirm your Kubernetes cluster is live by running this command:
kubectl get nodes

You should see a table with columns NAME, STATUS, ROLES, AGE, and VERSION. The node named docker-desktop appears with Ready status.

✔️ I see docker-desktop in Ready status

Your cluster is live and accepting commands. The single node named docker-desktop is where you will deploy Cassandra in the next step.

ⓧ I see an error or NotReady status

If the STATUS column shows NotReady, Kubernetes is still initializing. Wait 30 seconds and run the command again.

If you see a connection refused error, make sure Docker Desktop is running and the green Kubernetes indicator is visible.

If your terminal says kubectl is not recognized, close and reopen PowerShell so it picks up the path Docker Desktop added during Kubernetes setup.

Switch kubectl context

If you have used other Kubernetes clusters before, your context might point elsewhere. Run the following to switch back to Docker Desktop:

kubectl config use-context docker-desktop
  • Install the Python packages for this project by running this command:
pip install cassandra-driver==3.30.0 sentence-transformers==5.6.0

This download may take a minute or two. You will see progress bars as pip fetches each package and its dependencies.

What are these packages?

  • cassandra-driver 3.30.0 is the official Python driver for Apache Cassandra. It handles connecting, executing CQL queries, and serializing vector data.
  • sentence-transformers 5.6.0 runs AI embedding models locally on your CPU. You will use it to convert telecom documents into 384-dimensional vectors for search.

pip install failed?

If you see a pip is not recognized error, try running python -m pip install instead. This uses the pip module directly from your Python installation.

If you see build errors related to sentence-transformers, make sure you are running Python 3.12 by checking with:

python --version

Help me fix a pip install failure for cassandra-driver or sentence-transformers

  • Confirm the packages installed successfully by running:
pip show cassandra-driver sentence-transformers

You should see version 3.30.0 for cassandra-driver and 5.6.0 for sentence-transformers in the output.

Your Kubernetes cluster is live and your Python tools are ready. Next up, you will deploy a real Cassandra 5.0 database onto your cluster and watch it come online.

Deploy Cassandra 5.0 on Kubernetes

Your Kubernetes cluster is running and your Python packages are installed. Now it is time to deploy something real on that cluster.

Running a database in a container is straightforward. Running it on Kubernetes with health checks, stable networking, and proper resource limits is a deployment skill that transfers to any production system. In this step, you will deploy Cassandra 5.0 on your local cluster and verify it accepts queries.

In this step, get ready to:
  • Write a Kubernetes manifest defining a headless Service and StatefulSet for Cassandra.
  • Deploy the manifest and watch the pod reach Ready status.
  • Connect to Cassandra with CQL and confirm the cluster is live.
Write a Kubernetes manifest

Cassandra needs stable network identity and ordered startup. A StatefulSet provides both, giving each pod a predictable DNS name like cassandra-0. A headless Service (one with no cluster IP) makes that DNS name reachable inside the cluster.

  • Move to your Desktop and create a project folder by running these commands:
cd ~/Desktop
mkdir telecom-search
cd telecom-search

Why ~/Desktop?

Creating the folder on your Desktop means you can always find it easily. PowerShell resolves ~ to your home directory, so ~/Desktop points to your Windows Desktop folder.

  • Open VS Code in this folder by running:
code .

VS Code opens with the telecom-search folder in the Explorer sidebar.

  • In VS Code, create a new file by pressing Ctrl+N.
  • Save it immediately with Ctrl+S and name it cassandra.yaml in the telecom-search folder.
  • Paste the following Service definition into cassandra.yaml:
apiVersion: v1
kind: Service
metadata:
  name: cassandra
  labels:
    app: cassandra
spec:
  ports:
    - port: 9042
      name: cql
  clusterIP: None
  selector:
    app: cassandra

What does this Service do?

This defines a headless Service (clusterIP: None). Instead of load-balancing traffic to random pods, it gives each pod a stable DNS address like cassandra-0.cassandra.default.svc.cluster.local.

Port 9042 is the standard CQL port that Cassandra listens on for client connections.

  • Save cassandra.yaml.
  • Switch back to your PowerShell terminal and apply the manifest to confirm the Service definition is valid:
kubectl apply -f cassandra.yaml

You should see service/cassandra created in the output.

Seeing 'error: no objects passed to apply'?

This means the file is empty or not saved. Make sure you saved cassandra.yaml in VS Code before running the command. Verify you are in the telecom-search folder by checking that ls shows cassandra.yaml.

Help me troubleshoot why kubectl apply cannot find my YAML file.

Now you will add the StatefulSet definition below the Service. This tells Kubernetes how to run the Cassandra container with resource limits and health monitoring.

  • In cassandra.yaml, place your cursor at the end of the file (after the last line of the Service) and paste the following StatefulSet definition:
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: cassandra
spec:
  serviceName: cassandra
  replicas: 1
  selector:
    matchLabels:
      app: cassandra
  template:
    metadata:
      labels:
        app: cassandra
    spec:
      containers:
        - name: cassandra
          image: cassandra:5.0.8
          ports:
            - containerPort: 9042
              name: cql
          resources:
            requests:
              memory: "1Gi"
              cpu: "500m"
            limits:
              memory: "2Gi"
              cpu: "1000m"

What does this section define?

  • The --- separator marks a new Kubernetes resource in the same file.
  • serviceName: cassandra links the StatefulSet to the headless Service you created above, enabling stable DNS.
  • The cassandra:5.0.8 image is the official Cassandra 5.0 container with native vector search support.
  • Resource requests and limits prevent Cassandra from consuming all memory on your machine.
  • Directly below the last line you just pasted (the cpu: "1000m" line), paste the health check and environment configuration:
          readinessProbe:
            exec:
              command:
                - cqlsh
                - -e
                - "DESCRIBE CLUSTER"
            initialDelaySeconds: 60
            periodSeconds: 10
            timeoutSeconds: 5
          env:
            - name: CASSANDRA_CLUSTER_NAME
              value: "TelecomSearch"
            - name: CASSANDRA_DC
              value: "dc1"
            - name: CASSANDRA_ENDPOINT_SNITCH
              value: "GossipingPropertyFileSnitch"
            - name: MAX_HEAP_SIZE
              value: "512M"
            - name: HEAP_NEWSIZE
              value: "128M"

What do the probe and env vars do?

  • The readinessProbe runs cqlsh -e "DESCRIBE CLUSTER" every 10 seconds. Kubernetes only routes traffic to the pod once this command succeeds.
  • initialDelaySeconds: 60 gives Cassandra a full minute to start before health checks begin.
  • CASSANDRA_CLUSTER_NAME names this cluster TelecomSearch. The heap settings cap JVM memory so it fits inside the 2Gi container limit.
  • Save cassandra.yaml.

Check your indentation

YAML is indentation-sensitive. The readinessProbe and env keys must be at the same indent level as resources and ports above them (10 spaces from the left margin). If anything is misaligned, Kubernetes will reject the file.

✔️ Awesome, I've got everything!

Great. Make sure you have saved cassandra.yaml before continuing.

ⓧ I'd like to double check the full code

Your cassandra.yaml should contain all three chunks you pasted, in this order:

  • The Service definition at the top (starting with apiVersion: v1, ending with app: cassandra under selector).
  • The --- separator followed by the StatefulSet header (ending at cpu: "1000m" under limits).
  • The readinessProbe and env block (starting with readinessProbe:, ending at value: "128M" for HEAP_NEWSIZE).

Scroll up and confirm each chunk is present and properly indented. The file should be about 53 lines total.

Apply and monitor the deployment

With the full manifest saved, you will apply it to your cluster and watch Cassandra start up. The pod needs about 60-90 seconds to initialize because Cassandra loads its storage engine before the readiness probe passes.

  • Apply the complete manifest by running:
kubectl apply -f cassandra.yaml

You should see service/cassandra unchanged (already created earlier) and statefulset.apps/cassandra created.

  • Watch the pod start up by running:
kubectl get pods -w

What does the -w flag do?

The -w flag watches for changes and prints status updates as they happen. You will see cassandra-0 transition from Pending to ContainerCreating to Running, and finally show 1/1 under READY.

  • Wait until you see cassandra-0 show 1/1 under READY and Running under STATUS. This takes about 60-90 seconds.
  • Press Ctrl+C to stop watching once the pod is Ready.

Pod stuck in Pending or CrashLoopBackOff?

  • If the pod stays in Pending, Docker Desktop may not have enough memory allocated. Open Docker Desktop, go to Settings then Resources, and ensure at least 4GB of RAM is assigned.
  • If the pod shows CrashLoopBackOff, check the logs with kubectl logs cassandra-0. The most common cause is insufficient memory for the JVM heap.

Help me fix my Cassandra pod that is not reaching Ready status on Docker Desktop Kubernetes.

Connect and validate CQL access

The pod is running, but you need to confirm Cassandra is actually accepting CQL queries. You will exec into the pod and run a cluster describe command.

Before you run the next command, what do you think you will see? The readiness probe already runs DESCRIBE CLUSTER internally. Do you expect the cluster name to match the environment variable you set?

  • Verify Cassandra is live by running:
kubectl exec -it cassandra-0 -- cqlsh -e "DESCRIBE CLUSTER"

You should see output showing the cluster name TelecomSearch and the partitioner (Murmur3Partitioner). This confirms Cassandra 5.0 is fully operational and accepting CQL queries.

Seeing 'Connection refused'?

If you get a connection error, the pod may not be fully ready yet. Wait another 30 seconds and try again. Confirm the pod shows 1/1 Ready with kubectl get pods before retrying.

Help me connect to my Cassandra pod using cqlsh.

Your Python scripts in the next step will connect to Cassandra from outside the pod. To make port 9042 accessible on localhost, you need to set up port forwarding.

  • Open a new PowerShell window (keep your current one open).
  • Start port forwarding by running this command in the new window:
kubectl port-forward cassandra-0 9042:9042

Why a separate window?

Port forwarding runs continuously and blocks the terminal. Keeping it in a dedicated window means your main terminal stays free for running Python scripts. The forwarding stays active as long as this window is open.

  • You should see Forwarding from 127.0.0.1:9042 -> 9042 printed. Leave this window running.

Port already in use?

If you see an error about port 9042 being in use, another process is occupying it. Close any other database or service using that port and retry the command.

Help me free up port 9042 so I can port-forward to my Cassandra pod.

Cassandra 5.0 is running on your Kubernetes cluster with health monitoring, resource limits, and stable networking. Next, you will create the vector-enabled schema and load telecom documents with AI-generated embeddings.

Build the Telecom Knowledge Base

Your Cassandra cluster is running and accepting connections. Now it needs data.

A vector database without data is just an expensive empty box. In this step, you will create the schema that includes Cassandra 5.0's native VECTOR<FLOAT, 384> data type, generate 384-dimensional embeddings from telecom troubleshooting documents using a local AI model, and load everything into the database.

In this step, get ready to:
  • Create a Python script that builds the keyspace and vector-enabled table schema.
  • Generate normalized embeddings from 12 telecom documents and ingest them into Cassandra.
  • Verify the full knowledge base is queryable with a row count check.
Create the keyspace and vector-enabled table

Before you can store embeddings, Cassandra needs a keyspace (its equivalent of a database) and a table with a VECTOR<FLOAT, 384> column. You will write a Python script that waits for Cassandra to be reachable, then creates both.

  • Create a new file called setup_schema.py in your project folder and add the connection-wait logic by pasting this code:
from cassandra.cluster import Cluster
from cassandra.policies import RoundRobinPolicy
import time


def wait_for_cassandra(host="127.0.0.1", port=9042, timeout=120):
    """Wait for Cassandra to be ready."""
    start = time.time()
    while time.time() - start < timeout:
        try:
            cluster = Cluster([host], port=port, load_balancing_policy=RoundRobinPolicy())
            session = cluster.connect()
            session.shutdown()
            cluster.shutdown()
            print("Cassandra is ready!")
            return True
        except Exception:
            print("Waiting for Cassandra...")
            time.sleep(5)
    raise TimeoutError("Cassandra did not become ready in time")

What does this code do?

The wait_for_cassandra function repeatedly tries to open a connection to Cassandra on 127.0.0.1:9042. If the connection fails, it waits 5 seconds and retries for up to 120 seconds total.

This is necessary because even after the pod shows Ready, the CQL interface may still be initializing. The function ensures your schema commands never hit a half-started database.

  • Below the wait_for_cassandra function in the same file, add the schema creation function and main block:
def create_schema():
    """Create keyspace and vector-enabled table."""
    cluster = Cluster(["127.0.0.1"], port=9042, load_balancing_policy=RoundRobinPolicy())
    session = cluster.connect()

    session.execute("""
        CREATE KEYSPACE IF NOT EXISTS telecom
        WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1}
    """)

    session.set_keyspace("telecom")

    session.execute("""
        CREATE TABLE IF NOT EXISTS knowledge_base (
            doc_id UUID PRIMARY KEY,
            category TEXT,
            title TEXT,
            content TEXT,
            embedding VECTOR<FLOAT, 384>
        )
    """)

    print("Schema created: telecom.knowledge_base")
    session.shutdown()
    cluster.shutdown()


if __name__ == "__main__":
    wait_for_cassandra()
    create_schema()

What does this code do?

The create_schema function does three things in sequence. It creates the telecom keyspace with simple single-node replication.

It then creates the knowledge_base table with a VECTOR<FLOAT, 384> column. This is Cassandra 5.0's native vector data type that stores 384-dimensional floating-point embeddings directly in each row.

What is VECTOR<FLOAT, 384>?

This is a native CQL data type introduced in Cassandra 5.0. It stores a fixed-length array of 384 floating-point numbers directly in the row. The 384 matches the output dimension of the all-MiniLM-L6-v2 embedding model you will use next.

  • Save setup_schema.py and run it to create the schema:
python setup_schema.py

You should see output like this:

Waiting for Cassandra...
Cassandra is ready!
Schema created: telecom.knowledge_base

Seeing a connection refused error?

Make sure your port-forward from the previous step is still running. If the terminal session closed, re-run the port-forward command to map localhost:9042 to the Cassandra pod.

If the script times out after 120 seconds, verify the pod is still healthy with kubectl get pods and confirm cassandra-0 shows 1/1 Running.

help me debug a Cassandra connection refused error in my Python script

✔️ Awesome, I've got everything!

Great. Double check you have saved setup_schema.py and confirmed the schema output before moving on.

ⓧ I'd like to double check the full code

Your setup_schema.py file should contain two chunks in order:

  • The imports (Cluster, RoundRobinPolicy, time) followed by the wait_for_cassandra function.
  • The create_schema function followed by the if __name__ == "__main__" block that calls both functions.

Scroll up and confirm each chunk is present in your file, in that order.

Generate embeddings and ingest documents

With the schema in place, you need to fill the table with real telecom support documents. Each document gets converted into a 384-dimensional vector using a local AI model, then stored alongside its text and category metadata.

  • Create a new file called ingest_data.py and add the imports, document data, and normalization utility:
import uuid
import numpy as np
from cassandra.cluster import Cluster
from cassandra.policies import RoundRobinPolicy
from sentence_transformers import SentenceTransformer


TELECOM_DOCS = [
    {"category": "network", "title": "5G Signal Dropping", "content": "If your 5G signal keeps dropping, check if you are in a fringe coverage area. 5G uses higher frequencies that have shorter range. Move closer to a window or open area. If the problem persists, reset your network settings."},
    {"category": "network", "title": "Slow Internet Speed", "content": "Slow internet speeds can be caused by network congestion during peak hours, distance from the cell tower, or too many connected devices. Run a speed test and compare with your plan limits."},
    {"category": "network", "title": "No Service Error", "content": "A no service error usually means your device cannot connect to any cell tower. Check if airplane mode is off, restart your device, and verify your SIM card is properly seated."},
    {"category": "network", "title": "WiFi Calling Issues", "content": "WiFi calling requires a stable broadband connection with at least 1 Mbps upload speed. Ensure your router firmware is updated and QoS settings prioritize voice traffic."},
    {"category": "billing", "title": "Unexpected Charges", "content": "Unexpected charges may come from premium SMS services, international roaming, or data overages. Check your usage details in the app. You can set spending alerts to avoid surprises."},
    {"category": "billing", "title": "Plan Upgrade Cost", "content": "Upgrading your plan takes effect at the next billing cycle. Prorated charges apply if you upgrade mid-cycle. The new rate includes additional data and international minutes."},
    {"category": "billing", "title": "Payment Failed", "content": "If your payment failed, verify your card details and expiration date. We retry failed payments after 3 days. If the retry fails, your service may be suspended until payment is received."},
    {"category": "billing", "title": "Refund Process", "content": "Refunds for billing errors are processed within 5-7 business days. Submit a dispute through the app or call customer service with your account number and the charge in question."},
    {"category": "device", "title": "Battery Draining Fast", "content": "Rapid battery drain is often caused by background app refresh, high screen brightness, or poor signal forcing the radio to work harder. Enable battery saver mode and check which apps consume the most power."},
    {"category": "device", "title": "Phone Overheating", "content": "Overheating can occur during heavy use, charging, or in direct sunlight. Close background apps, remove the case while charging, and avoid using the phone while it charges."},
    {"category": "device", "title": "SIM Not Detected", "content": "If your SIM is not detected, power off the device, remove and reinsert the SIM card. Check for physical damage to the SIM. If the issue persists, request a SIM replacement at any store."},
    {"category": "device", "title": "Software Update Failed", "content": "Failed software updates require at least 50% battery and 2GB free storage. Connect to WiFi and retry. If the update continues to fail, perform a factory reset after backing up your data."},
]


def normalize_vector(vec):
    """L2 normalize a vector for dot-product similarity."""
    norm = np.linalg.norm(vec)
    if norm == 0:
        return vec
    return (vec / norm).tolist()

Why normalize vectors?

L2 normalization scales every vector to length 1. When all vectors have the same length, dot product and cosine similarity produce identical rankings. The difference is that dot product skips the expensive division step that cosine requires.

By normalizing at insert time, you pay a tiny cost once per document. Every future search query then benefits from the faster dot-product math.

  • Below the normalize_vector function in the same file, add the ingestion function:
def ingest():
    """Generate embeddings and insert into Cassandra."""
    print("Loading embedding model...")
    model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")

    cluster = Cluster(["127.0.0.1"], port=9042, load_balancing_policy=RoundRobinPolicy())
    session = cluster.connect("telecom")

    insert_stmt = session.prepare("""
        INSERT INTO knowledge_base (doc_id, category, title, content, embedding)
        VALUES (?, ?, ?, ?, ?)
    """)

    print("Generating embeddings and ingesting documents...")
    for doc in TELECOM_DOCS:
        text_to_embed = f"{doc['title']}. {doc['content']}"
        embedding = model.encode(text_to_embed)
        normalized = normalize_vector(embedding)
        session.execute(insert_stmt, [
            uuid.uuid4(),
            doc["category"],
            doc["title"],
            doc["content"],
            normalized,
        ])
        print(f"  Ingested: {doc['title']} [{doc['category']}]")

    print(f"\nDone! Ingested {len(TELECOM_DOCS)} documents.")
    session.shutdown()
    cluster.shutdown()

What does this code do?

The ingest function loads the all-MiniLM-L6-v2 model locally (an 80 MB download on first run). It then connects to the telecom keyspace and prepares an INSERT statement.

For each document, it combines the title and content into a single string, generates a 384-dimensional embedding, normalizes it to unit length, and inserts the full row using a prepared statement. Prepared statements let Cassandra parse the CQL once and reuse the execution plan for all 12 inserts.

  • At the bottom of the file, add the main block that triggers the ingestion:
if __name__ == "__main__":
    ingest()

What is sentence-transformers/all-MiniLM-L6-v2?

It is a lightweight sentence-transformers model with 22.7 million parameters. It maps any text up to 128 tokens into a 384-dimensional dense vector that captures semantic meaning.

The model runs entirely on your CPU. No GPU or cloud API needed. It produces the same quality embeddings as cloud services for this use case, just from your local machine.

  • Save ingest_data.py and run it to embed and load all 12 documents:
python ingest_data.py

The first run downloads the model (~80 MB). After that, you should see each document ingested one by one:

Loading embedding model...
Generating embeddings and ingesting documents...
  Ingested: 5G Signal Dropping [network]
  Ingested: Slow Internet Speed [network]
  Ingested: No Service Error [network]
  Ingested: WiFi Calling Issues [network]
  Ingested: Unexpected Charges [billing]
  Ingested: Plan Upgrade Cost [billing]
  Ingested: Payment Failed [billing]
  Ingested: Refund Process [billing]
  Ingested: Battery Draining Fast [device]
  Ingested: Phone Overheating [device]
  Ingested: SIM Not Detected [device]
  Ingested: Software Update Failed [device]

Done! Ingested 12 documents.

Model download hanging or failing?

The first run requires an internet connection to download the model from Hugging Face. If the download stalls, check your network connection and retry.

If you see a NoHostAvailable error, your port-forward to Cassandra has dropped. Restart it before re-running the script.

help me debug a sentence-transformers model download or Cassandra connection issue in my ingest script

✔️ Awesome, I've got everything!

Great. Double check you have saved ingest_data.py and confirmed all 12 documents were ingested.

ⓧ I'd like to double check the full code

Your ingest_data.py file should contain three chunks in order:

  • The imports, the TELECOM_DOCS list (12 dictionary entries covering network, billing, and device categories), and the normalize_vector function.
  • The ingest function that loads the model, prepares the INSERT, and loops through all docs.
  • The if __name__ == "__main__" block at the bottom that calls ingest().

Scroll up and confirm each chunk is present in your file, in that order.

Before you run the verification query, what do you think SELECT COUNT(*) will return given that you ingested 12 documents?

  • Verify your knowledge base is fully loaded by running this CQL query through the pod:
kubectl exec -it cassandra-0 -- cqlsh -e "SELECT COUNT(*) FROM telecom.knowledge_base;"

You should see the count return 12, confirming all your telecom documents are stored with their embeddings.

Your knowledge base is loaded and ready. Next up, you will query it and discover why naive vector search falls short when you need precise, category-filtered results.

Run Naive Vector Search

Your knowledge base has 12 telecom documents embedded and stored in Cassandra. The next question is: when a customer asks "why is my 5G dropping?", how do you find the most relevant document out of those 12 (or billions, at scale)?

The simplest approach is to create a vector similarity index and search across everything with no filters. In this step, you will try exactly that and observe where it falls short.

In this step, get ready to:
  • Create a cosine similarity SAI index on the embedding column.
  • Run an unfiltered ANN search and observe the relevance problem.
Create the search script with a cosine index

A Storage-Attached Index (SAI) tells Cassandra how to organize your vectors for fast lookup. You will start with cosine similarity, the most common default.

  • Create a new file called search_naive.py in your project folder.
  • Add the imports, normalize helper, and the first half of the search function that creates the cosine index:
import time
import numpy as np
from cassandra.cluster import Cluster
from cassandra.policies import RoundRobinPolicy
from sentence_transformers import SentenceTransformer

def normalize_vector(vec):
    """L2 normalize a vector for dot-product similarity."""
    norm = np.linalg.norm(vec)
    if norm == 0:
        return vec
    return (vec / norm).tolist()

def naive_search():
    """Run a naive cosine ANN search without category filtering."""
    model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
    cluster = Cluster(["127.0.0.1"], port=9042, load_balancing_policy=RoundRobinPolicy())
    session = cluster.connect("telecom")

    # Create cosine similarity index
    print("Creating COSINE similarity index...")
    session.execute("""
        CREATE CUSTOM INDEX IF NOT EXISTS idx_kb_embedding_cosine
        ON knowledge_base (embedding)
        USING 'StorageAttachedIndex'
        WITH OPTIONS = {'similarity_function': 'COSINE'}
    """)
    time.sleep(2)

What does this code do?

The CREATE CUSTOM INDEX statement builds a Storage-Attached Index on the embedding column using cosine as the similarity function. This tells Cassandra to build an HNSW-style graph structure so queries can find approximate nearest neighbors without scanning every row.

The time.sleep(2) gives the index a moment to become queryable after creation.

Add the ANN query and run the search

Now you need the code that embeds a user question, sends it to Cassandra as an Approximate Nearest Neighbor query, and prints the top 5 results with their similarity scores.

  • Add the following code directly below the time.sleep(2) line in search_naive.py:
    # Search query
    query_text = "My 5G signal keeps dropping and I lose connection"
    query_embedding = normalize_vector(model.encode(query_text))

    print(f"\nQuery: '{query_text}'")
    print("=" * 60)
    print("NAIVE SEARCH (cosine, no filtering)")
    print("=" * 60)

    start_time = time.time()
    results = session.execute(f"""
        SELECT title, category, content,
               similarity_cosine(embedding, {query_embedding}) AS score
        FROM knowledge_base
        ORDER BY embedding ANN OF {query_embedding}
        LIMIT 5
    """)
    elapsed = (time.time() - start_time) * 1000

    for row in results:
        relevance = "RELEVANT" if row.category == "network" else "IRRELEVANT"
        print(f"  [{relevance}] [{row.category}] {row.title} (score: {row.score:.4f})")

    print(f"\nExecution time: {elapsed:.1f}ms")
    print("\nPROBLEM: Notice irrelevant billing/device docs appear in results!")
    session.shutdown()
    cluster.shutdown()

if __name__ == "__main__":
    naive_search()

What does this query do?

The ORDER BY embedding ANN OF [...] clause tells Cassandra to use the HNSW graph to find the 5 closest vectors to your query embedding. This gives you log(N) lookups instead of scanning every row linearly.

The similarity_cosine() function returns a score between 0 and 1 for each result so you can see how close each match is. Notice there is no WHERE clause filtering by category. Every document in the table is a candidate.

  • Save search_naive.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved the file with both chunks combined.

ⓧ I'd like to double check the full code

Your search_naive.py is built from the two code chunks above. Scroll up and confirm your file contains, in order:

  • Chunk 1: the imports, normalize_vector function, and the start of naive_search through time.sleep(2).
  • Chunk 2: the query embedding, the ANN query execution, result printing loop, shutdown calls, and the if __name__ block at the bottom.

The second chunk continues directly inside the same function. It is indented at the same level as the code above it.

Before you run this, think about what you expect. You are searching for "5G signal dropping" across a table with network, billing, and device docs. Do you think only network docs will come back, or will you see results from all categories?

  • Run the naive search script:
python search_naive.py

You will see 5 results printed, each tagged as RELEVANT or IRRELEVANT. Billing and device documents appear in your results even though you asked about a network problem.

Why do irrelevant docs appear?

Without a WHERE clause, the ANN query searches every vector in the table. A billing doc about "payment failed" might mention "service" or "suspended", which gives it a non-zero cosine similarity to your 5G question.

At 12 documents this is annoying. At a billion documents, returning noise from unrelated categories makes the system useless. This is the gap you will close in the next step.

Seeing a connection error?

Make sure your port-forward to Cassandra is still active in another terminal. If it timed out, restart it with kubectl port-forward cassandra-0 9042:9042 and then re-run the script.

If you see an error about the index already existing, that is fine. The IF NOT EXISTS clause handles re-runs safely.

Still stuck? Help me debug my naive search script.

You have now seen the problem firsthand. A pure vector search returns results ranked by similarity, but it has no concept of category boundaries. Next, you will apply two optimization tricks that fix both the relevance and speed problems in a single query.

Apply Optimized Hybrid Search

In the previous step, you saw the naive search in action. It returned billing and device docs when you asked about a 5G network issue, and it used cosine similarity which calculates an extra normalization step on every comparison.

Now you will fix both problems in a single script. You will switch the similarity function to DOT_PRODUCT (approximately 50% faster than cosine on normalized vectors) and add a Storage-Attached Index on the category column so that Cassandra can combine vector search with metadata filtering in a single query pass.

In this step, get ready to:
  • Rebuild the vector index with DOT_PRODUCT similarity for faster comparisons.
  • Run a hybrid ANN + CQL filter query that returns only relevant network docs.
Rebuild the index with DOT_PRODUCT

Your embeddings are already L2-normalized, which means their magnitude is always 1. When vectors are normalized, dot product produces the same ranking as cosine similarity but skips the per-comparison normalization math.

  • Create a new file called search_optimized.py in VS Code. Start with the imports, helper function, and function setup by pasting:
import time
import numpy as np
from cassandra.cluster import Cluster
from cassandra.policies import RoundRobinPolicy
from sentence_transformers import SentenceTransformer


def normalize_vector(vec):
    """L2 normalize a vector for dot-product similarity."""
    norm = np.linalg.norm(vec)
    if norm == 0:
        return vec
    return (vec / norm).tolist()


def optimized_search():
    """Run optimized DOT_PRODUCT hybrid search with category filtering."""
    model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
    cluster = Cluster(["127.0.0.1"], port=9042, load_balancing_policy=RoundRobinPolicy())
    session = cluster.connect("telecom")

What does this code do?

This is the same connection setup as your naive search script. The function connects to the telecom keyspace and loads the same embedding model so the query vector dimensions match the stored embeddings.

  • Still inside the optimized_search() function, add the index-rebuilding logic that drops the old cosine index and creates two new SAI indexes:
    # Drop old cosine index and create DOT_PRODUCT index
    print("Rebuilding index with DOT_PRODUCT (50% faster on normalized vectors)...")
    session.execute("DROP INDEX IF EXISTS idx_kb_embedding_cosine")
    time.sleep(1)

    session.execute("""
        CREATE CUSTOM INDEX IF NOT EXISTS idx_kb_embedding_dot
        ON knowledge_base (embedding)
        USING 'StorageAttachedIndex'
        WITH OPTIONS = {'similarity_function': 'DOT_PRODUCT'}
    """)

    # Create SAI index on category for hybrid filtering
    print("Creating SAI index on category column for hybrid queries...")
    session.execute("""
        CREATE CUSTOM INDEX IF NOT EXISTS idx_kb_category
        ON knowledge_base (category)
        USING 'StorageAttachedIndex'
    """)
    time.sleep(2)

    # Search query
    query_text = "My 5G signal keeps dropping and I lose connection"
    query_embedding = normalize_vector(model.encode(query_text))

    print(f"\nQuery: '{query_text}'")
    print("=" * 60)
    print("OPTIMIZED SEARCH (dot_product + hybrid filter)")
    print("=" * 60)

What's happening here?

This code makes three key changes to the database indexes:

  • DROP INDEX IF EXISTS idx_kb_embedding_cosine removes the cosine index you created in the naive search step.
  • The new idx_kb_embedding_dot index uses DOT_PRODUCT similarity, which skips the normalization math since your vectors are already unit-length.
  • The idx_kb_category index on the category column enables Cassandra to filter rows by category during the ANN scan rather than after it.

Why DOT_PRODUCT instead of COSINE?

Cassandra's documentation states that dot product is 50% faster than cosine similarity on normalized embeddings. Since you normalized every vector before insertion in the ingestion step, DOT_PRODUCT gives identical rankings with less computation.

  • Save search_optimized.py.
Run a hybrid ANN + filter query

The real power of SAI is that multiple indexes compose together. Your vector index and your category index can both participate in a single query, so Cassandra filters by category and ranks by vector similarity in one pass.

  • Still inside the optimized_search() function, add the hybrid query execution and results printing directly below the print statements you just added:
    start_time = time.time()
    results = session.execute(f"""
        SELECT title, category, content,
               similarity_dot_product(embedding, {query_embedding}) AS score
        FROM knowledge_base
        WHERE category = 'network'
        ORDER BY embedding ANN OF {query_embedding}
        LIMIT 5
    """)
    elapsed = (time.time() - start_time) * 1000

    for row in results:
        print(f"  [RELEVANT] [{row.category}] {row.title} (score: {row.score:.4f})")

    print(f"\nExecution time: {elapsed:.1f}ms")
    print("\nFIX APPLIED:")
    print("  1. DOT_PRODUCT on normalized vectors = ~50% faster than cosine")
    print("  2. SAI index on 'category' = hybrid filter in single query pass")
    print("  3. Only relevant network docs returned, zero noise")

    session.shutdown()
    cluster.shutdown()


if __name__ == "__main__":
    optimized_search()

What does this query do?

This is a hybrid query. The WHERE category = 'network' clause uses the SAI category index to narrow results to network docs only. The ORDER BY embedding ANN OF [...] clause uses the vector index to rank those filtered docs by semantic similarity.

Both indexes work together in a single pass. Cassandra does not scan all rows, then filter. It prunes irrelevant partitions before the ANN graph traversal even begins.

  • Save search_optimized.py.

Before you run this, what do you expect to see differently compared to the naive search output? Think about which categories will appear and whether the execution time will change.

  • Run the optimized search script in PowerShell:
python search_optimized.py

You should see output where every result is tagged [RELEVANT] and [network]. No billing or device docs appear. The execution time should be printed alongside the fix summary confirming the three improvements applied.

Seeing an index error or no results?

  • If you see an error about the index not existing, the time.sleep(2) may not be long enough for your machine. Increase it to time.sleep(5) and run again.
  • If you get a connection error, confirm your port-forward is still active in the other terminal.
  • If you see zero results, verify your ingest_data.py ran successfully in the earlier step by checking the row count with kubectl exec -it cassandra-0 -- cqlsh -e "SELECT COUNT(*) FROM telecom.knowledge_base;".

Help me debug my optimized search query

✔️ Awesome, I've got everything!

Great. Double check you have saved search_optimized.py and confirmed that only network-category results appear in your output.

ⓧ I'd like to double check the full code

Your search_optimized.py should contain these three sections in order:

  • The imports and normalize_vector helper function (same as the first code block above).
  • The optimized_search() function body with index drops, DOT_PRODUCT and category index creation, query text setup, and header prints (same as the second code block above).
  • The hybrid query execution, results loop, fix summary prints, session shutdown, and the if __name__ == "__main__" block (same as the third code block above).

Scroll up and confirm each section is present and in order. Then save and re-run.

Your knowledge base now uses the same search pattern that powers billion-scale vector databases in production. The combination of DOT_PRODUCT on normalized vectors and SAI-backed hybrid filtering means Cassandra can traverse its HNSW-style graph in logarithmic time while skipping entire categories of irrelevant data. Next up, a Secret Mission that takes this even further with partition-aware multi-tenant search.

Secret mission

Partition-Aware Multi-Tenant Search

In production telecom deployments, each region (APAC, EMEA, Americas) has its own data partition. Queries that include the partition key skip the expensive scatter-gather phase entirely, routing straight to the single node that holds the data. Can you redesign the data model to combine partition isolation with vector search for maximum speed?

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Cassandra 5.0 pod running on Kubernetes (the cassandra StatefulSet and headless Service, using 1-2GB RAM from Docker Desktop).
  • Python packages (cassandra-driver and sentence-transformers).
  • sentence-transformers model cache (~500MB on disk at ~/.cache/huggingface/hub on macOS/Linux or %USERPROFILE%\.cache\huggingface\hub on Windows).
  • Project files (cassandra.yaml, setup_schema.py, ingest_data.py, search_naive.py, search_optimized.py, search_partitioned.py).

Keep everything running

No action needed. Choose this if you are still actively experimenting with your vector search queries or planning to extend the knowledge base.

  • Docker Desktop and Kubernetes stay enabled. The Cassandra pod persists until you explicitly delete it.
  • Your Python packages and model cache remain installed and ready to use.
  • The Cassandra pod uses 1-2GB of RAM from your Docker Desktop memory allocation while running.

Pause - I'll come back to this later

Shut down the Cassandra pod to free up RAM while keeping your data and configuration intact. You can resume later without re-deploying.

  • Scale the StatefulSet down to zero replicas by running:
  • When you are ready to resume, scale back up by running:
kubectl scale statefulset cassandra --replicas=0

This stops the Cassandra pod without deleting your data. To resume later:

kubectl scale statefulset cassandra --replicas=1

Wait about 60 seconds for Cassandra to start back up, then your data and indexes will be ready.

Delete - I don't want to use this again

Remove all project resources and start fresh. This deletes your Kubernetes pod, Python packages, model cache, and project files.

  • Delete the Cassandra pod and headless Service from Kubernetes by running:
kubectl delete -f cassandra.yaml
  • Uninstall the Python packages by running:
pip uninstall cassandra-driver sentence-transformers -y
  • Delete the sentence-transformers model cache. On macOS/Linux:
rm -rf ~/.cache/huggingface/hub/models--sentence-transformers--all-MiniLM-L6-v2

On Windows (PowerShell):

Remove-Item -Recurse -Force "$env:USERPROFILE\.cache\huggingface\hub\models--sentence-transformers--all-MiniLM-L6-v2"
  • Delete your project files. On macOS/Linux:
rm cassandra.yaml setup_schema.py ingest_data.py search_naive.py search_optimized.py search_partitioned.py

On Windows (PowerShell):

Remove-Item cassandra.yaml, setup_schema.py, ingest_data.py, search_naive.py, search_optimized.py, search_partitioned.py

Nice Work!

Nice Work!

You just built a production-ready vector search system from scratch on Kubernetes. Your telecom knowledge base can answer natural-language questions in milliseconds by combining semantic understanding with structured metadata filtering.

You've learned how to:

  • Deploy Cassandra 5.0 on a local Kubernetes cluster using a StatefulSet with health checks and stable networking.
  • Measure the performance gap between naive cosine similarity search and optimized DOT_PRODUCT hybrid retrieval on normalized embeddings.
  • Implement hybrid queries combining ANN vector search with CQL metadata filters in a single SAI-backed Cassandra pass.
  • Secret Mission: Design partition-aware data models for multi-tenant isolation, eliminating cluster-wide scatter-gather by scoping queries to a single partition key.

Ready to quiz yourself?