Rerank Search Results with Voyage

Use Voyage AI to rerank keyword results and compare ranking quality.

Introduction

30 Second Summary

A search result can contain the answer and still waste your time when that answer sits halfway down the list. The order matters as much as finding the document.

In this project, you will build a Python script that searches 30 to 50 of your own short documents. You will use a Voyage AI reranker to reorder the strongest 20 candidates.

What You'll Build

You'll watch a buried answer rise to first place after asking a question about your own documents.

By the end of this project, you'll have:

  • A before-and-after ranking that shows the correct document moving above weaker keyword matches.
  • A side-by-side model comparison that reveals where rerank-3-lite and rerank-3 disagree.
  • A CSV comparison that records the ranking shift caused by your plain-English instruction.
  • Secret Mission: An optional challenge to push your skills further.

Are there any prerequisites?

You need Python 3 on your computer. You also need a Voyage AI account plus 30 to 50 short documents.

Before We Start

Your reranker script needs a secure way to reach Voyage AI. It also needs a local Python environment that can load the official client.

You will activate your Voyage account first. You will finish with an empty project folder where the API key is available without appearing in a source file.

In this step, get ready to:
  • Activate your Voyage account with its free token allowance.
  • Protect a new API key in the VOYAGE_API_KEY environment variable.
  • Prepare the local Python workspace for the Voyage client.
Create your Voyage account and API key

A Voyage account gives your script permission to use the reranking service. An API key identifies that account when the script sends a request.

  • Visit the official Voyage AI website to start the account creation flow.
  • Complete the account creation flow.
  • Sign in to reach the Voyage dashboard.

Your Voyage dashboard is now the control centre for your API access.

Your new key is a live credential, so the dashboard reveals it for secure storage. Keep the key private throughout this project.

  • Open the API keys section in the Voyage dashboard.
  • Click Create new secret key.
  • Copy the new key when the dashboard reveals it.
  • Store the key in a secure local credential manager.
  • Dismiss the key reveal after the key is stored.

Why keep the key private?

The key authorizes requests against your Voyage account. Anyone who obtains it can use your token allowance.

You will place the key in an environment variable later in this step. This keeps the credential outside the empty project folder.

  • Locate the reranker token allowance in the dashboard to answer how many free tokens remain.

A new account shows an allowance of 200 million free reranker tokens. This project uses only a small fraction of that allowance.

What does the allowance cover?

Voyage measures reranker usage from the query plus the candidate documents it processes. The first 200 million tokens are free for the account.

The dashboard balance gives you a visible checkpoint before the script starts making requests.

Prepare Python and your project folder

Python runs the search script. The pip package installer adds the Voyage client to that Python environment.

Start by checking both tools in a terminal. Choose the tab for your operating system.

macOS

  • Press Cmd+Space to open Spotlight.
  • Type Terminal into Spotlight.
  • Press Enter to open Terminal.
  • Check Python and pip by running these commands:
python3 --version
python3 -m pip --version

What do these checks prove?

  • The first command confirms that the Python interpreter is available.
  • The second command confirms that pip belongs to the same Python installation.

✔️ I see Python and pip versions

Your local Python tools are ready. Both commands should show version information without an error.

ⓧ Python or pip is missing

The missing command means the official Python installation needs to be added before you continue.

  • Download the latest stable macOS installer from the official Python downloads page.
  • Run the downloaded installer.
  • Keep the installer defaults that include pip.
  • Close Terminal after the installation finishes.
  • Reopen Terminal through Spotlight.
  • Repeat the two version checks shown above.
Still missing Python or pip?

A terminal that stayed open during installation may still be using its old command path. Close every Terminal window before trying the checks again.

If pip remains unavailable, rerun the official installer with its default components enabled.

Help me troubleshoot my Python installation.

Windows

  • Press the Windows key to open search.
  • Type PowerShell into search.
  • Press Enter to open PowerShell.
  • Check Python and pip by running these commands:
py --version
py -m pip --version

What do these checks prove?

  • The first command confirms that the Python launcher can find an installed interpreter.
  • The second command confirms that pip belongs to that Python installation.

✔️ I see Python and pip versions

Your local Python tools are ready. Both commands should show version information without an error.

ⓧ Python or pip is missing

The missing command means the official Python installation needs to be added before you continue.

  • Download Python through the official Windows downloads page.
  • Run the downloaded installer.
  • Keep the installer defaults that include pip.
  • Close PowerShell after the installation finishes.
  • Reopen PowerShell through Windows search.
  • Repeat the two version checks shown above.
Still missing Python or pip?

A PowerShell window that stayed open during installation may still be using its old command path. Close every PowerShell window before trying the checks again.

If pip remains unavailable, rerun the official installer with its default components enabled.

Help me troubleshoot my Python installation.

With Python available, you can give the project a predictable home on your Desktop.

  • Create the empty reranker-project folder by running these commands:
cd ~/Desktop
mkdir reranker-project
cd reranker-project
pwd

What do these commands do?

  • The first command moves the terminal to your Desktop.
  • The second command creates the reranker-project folder.
  • The third command moves the terminal into that folder.
  • The final command prints the current location as a visible check.

You should see a path ending in /Desktop/reranker-project on macOS. Windows shows a path ending in \Desktop\reranker-project.

Project folder command failed?

Confirm that your terminal started in your user account. A restricted terminal may not be able to reach your Desktop.

If the folder already exists, move into the existing reranker-project folder before continuing.

Help me reach my project folder.

Install the Voyage SDK and protect your key

The Voyage AI Python SDK gives your script a Python interface to the service. The VOYAGE_API_KEY environment variable supplies its credential.

This credential stays in the current terminal session. Your empty project folder receives no secret file that could enter source control.

macOS

  • Install the latest Voyage Python package by running this command:
python3 -m pip install -U voyageai

What does this command install?

Python runs pip as a module so the package reaches the interpreter you checked earlier. The -U option upgrades an older copy when one is already installed.

You will see package download and installation output. The terminal returns to its prompt when installation finishes.

Package installation failed?

Confirm that the terminal still shows a path ending in reranker-project. Check that the earlier pip version command still succeeds.

Help me install the Voyage package.

  • Retrieve the key from your secure local credential manager.
  • Replace <your secret key> with the key value in this command:
export VOYAGE_API_KEY="<your secret key>"

Why use an environment variable?

The export command makes the key available to programs launched from this Terminal session. It does not create a credential file inside reranker-project.

Keep this Terminal window open for the next steps. Closing it removes the session value.

Before you run the final check, do you expect the terminal to confirm the setup or report a missing key?

  • Verify the package import and environment variable by running this command:
python3 -c "import os
import voyageai
print('Voyage setup ready' if os.getenv('VOYAGE_API_KEY') else 'VOYAGE_API_KEY is missing')"

What does this check prove?

Importing voyageai confirms that the package is available to Python. Reading VOYAGE_API_KEY confirms that the current Terminal session can access your credential.

You should see Voyage setup ready. That confirms the local package and credential are ready for your script.

Setup check did not pass?

A missing-key result usually means the placeholder was not replaced before the export command ran. Repeat the export command with the actual key in this same Terminal window.

An import error means the package reached a different Python installation. Repeat the installation command with python3.

Help me finish the Voyage setup check.

Windows

  • Install the latest Voyage Python package by running this command:
py -m pip install -U voyageai

What does this command install?

The Python launcher runs pip against the interpreter you checked earlier. The -U option upgrades an older copy when one is already installed.

You will see package download and installation output. PowerShell returns to its prompt when installation finishes.

Package installation failed?

Confirm that PowerShell still shows a path ending in reranker-project. Check that the earlier pip version command still succeeds.

Help me install the Voyage package.

  • Retrieve the key from your secure local credential manager.
  • Replace <your secret key> with the key value in this command:
$Env:VOYAGE_API_KEY = "<your secret key>"

Why use an environment variable?

The assignment makes the key available to programs launched from this PowerShell session. It does not create a credential file inside reranker-project.

Keep this PowerShell window open for the next steps. Closing it removes the session value.

Before you run the final check, do you expect PowerShell to confirm the setup or report a missing key?

  • Verify the package import and environment variable by running this command:
py -c "import os
import voyageai
print('Voyage setup ready' if os.getenv('VOYAGE_API_KEY') else 'VOYAGE_API_KEY is missing')"

What does this check prove?

Importing voyageai confirms that the package is available to Python. Reading VOYAGE_API_KEY confirms that the current PowerShell session can access your credential.

You should see Voyage setup ready. That confirms the local package and credential are ready for your script.

Setup check did not pass?

A missing-key result usually means the placeholder was not replaced before the assignment ran. Repeat the assignment with the actual key in this same PowerShell window.

An import error means the package reached a different Python installation. Repeat the installation command with py.

Help me finish the Voyage setup check.

✔️ Awesome, I've got everything!

Your project terminal is ready. Keep it open so the next step can use the package and environment variable.

ⓧ I'd like to double check the full setup

Compare your setup with this final state before moving on.

  • Python 3 is installed and available from the project terminal.
  • The voyageai package is installed and available from the project terminal.
  • Your Voyage account is active.
  • A valid API key is available through the local VOYAGE_API_KEY environment variable.
  • The Voyage dashboard shows the available free token allowance.
  • The terminal is open inside the empty reranker-project folder.

Your Voyage account and local Python environment are ready. Next up, you will load your documents and expose the weakness in a keyword-only ranking.

Build a Keyword Search Baseline

Your Voyage AI account is ready. Python is available in your project terminal.

A reranker needs candidates from an initial search. This step builds a deliberately weak keyword search baseline that leaves the correct document below the first five results.

In this step, get ready to:
  • Populate a searchable corpus with 30 to 50 short documents.
  • Build a script that selects the top 20 candidates by keyword overlap.
  • Print five results with the correct answer ranked below them.
Prepare your corpus and question

A corpus is the collection of documents that your search examines. Short notes make each result easy to inspect.

Your test question needs one known answer in the corpus. Distractor documents should repeat more of the question's exact words than the correct document.

  • Create a folder named docs inside the open project folder using your editor or file manager's new-folder control.
  • Copy 30 to 50 learner-owned .txt documents into docs/.
  • Confirm that one document contains the answer to a question you can ask about the corpus.
  • Confirm that several other documents repeat important words from that question.

You should now see between 30 and 50 text documents inside docs/.

  • Record the question you want to test: your question about the notes.
  • Record the file that contains the correct answer: the file containing the correct answer.
Load and rank the documents

The script first reads every .txt file into memory. It keeps each filename beside its text so every result remains traceable.

  • Create rerank_notes.py beside docs/ using your editor's new-file control.
  • Paste this document-loading code into rerank_notes.py:
from pathlib import Path
import re

# Point the script at the learner-owned search corpus.
DOCS_DIR = Path("docs")
QUESTION = """[[QUESTION="your question about the notes"]]"""
TOP_CANDIDATES = 20
TOP_RESULTS = 5

def load_documents():
    # Keep each filename with its text so ranked results stay traceable.
    return [
        {"name": path.name, "text": path.read_text(encoding="utf-8")}
        for path in sorted(DOCS_DIR.glob("*.txt"))
    ]

documents = load_documents()

# Stop early if the corpus falls outside the planned size.
if not 30 <= len(documents) <= 50:
    raise ValueError("Add 30 to 50 .txt documents to docs/.")

print(f"Loaded {len(documents)} documents.")

What does this code do?

  • The DOCS_DIR value points to your searchable notes.
  • The QUESTION value stores the question that drives the ranking.
  • The load_documents() function reads each text file with UTF-8 encoding.
  • The document count check keeps the corpus between 30 and 50 files.
  • Save rerank_notes.py.
  • Check that the script loads your corpus by running this command:
python rerank_notes.py

What does this command do?

The command asks Python to execute rerank_notes.py. The script reads the text documents from docs/.

You should see a loaded count between 30 and 50.

Documents not loading?

  • Check that docs/ sits beside rerank_notes.py.
  • Check that every searchable document uses the .txt extension.
  • Convert any unreadable document into a plain UTF-8 text file.

Help me fix my document loader.

That confirms the script can see your complete corpus. The next chunk gives every document a keyword score.

A keyword score counts the unique words shared by the question and each document. Documents with more exact overlap move higher even when a lower result contains the stronger answer.

  • Add this ranking code below the existing print statement in rerank_notes.py:
def rank_by_keywords(documents, question):
    # Normalize the question into unique terms for simple overlap scoring.
    question_terms = set(re.findall(r"\b\w+\b", question.lower()))

    for document in documents:
        # Count the unique question terms found in this document.
        document_terms = set(re.findall(r"\b\w+\b", document["text"].lower()))
        document["keyword_score"] = len(question_terms & document_terms)

    # Put documents with the highest overlap first.
    return sorted(
        documents,
        key=lambda document: document["keyword_score"],
        reverse=True,
    )

ranked_documents = rank_by_keywords(documents, QUESTION)
candidates = ranked_documents[:TOP_CANDIDATES]
print(f"Selected {len(candidates)} keyword candidates.")

How does the ranking work?

  • The regular expression extracts lowercase words from the question and document.
  • A set keeps each word once so repeated terms do not inflate the score.
  • The keyword_score value records how many question terms appear in each document.
  • The final slice keeps the first 20 documents for later reranking.
  • Save rerank_notes.py.
  • Confirm that the script selects 20 candidates by running this command:
python rerank_notes.py

Why run the script again?

This run executes the new scoring function after loading the corpus. It confirms that 20 keyword-ranked documents are ready for the next stage.

You should see the loaded document count followed by Selected 20 keyword candidates.

Candidate count missing?

  • Check that the ranking code sits below the document-loading code.
  • Match the indentation inside rank_by_keywords() to the example.
  • Save rerank_notes.py before running it again.

Help me debug the keyword ranking function.

Print the baseline ranking

The first five results make the weakness of the baseline easy to inspect. Each result shows its rank and filename.

The output also includes the keyword score. A short text preview helps you judge whether the document answers the question.

  • Add this output loop below the candidate-selection code in rerank_notes.py:
print(f"\nQuestion: {QUESTION}")
print("Top five keyword results:")

for rank, document in enumerate(candidates[:TOP_RESULTS], start=1):
    # Collapse whitespace so every result has a compact preview.
    preview = " ".join(document["text"].split())[:180]
    print(f"\n{rank}. {document['name']} | score={document['keyword_score']}")
    print(preview)

What does this output show?

  • The first line repeats your question to give the ranking context.
  • The loop prints only the first five candidates.
  • The filename identifies the source of each result.
  • The preview exposes enough text to judge each result.
  • Save rerank_notes.py.

Before you run the script, predict whether exact word overlap puts the correct-answer document in the first five.

  • Reveal the baseline ranking by running this command:
python rerank_notes.py

What does this run reveal?

This run prints the complete keyword baseline. Compare its five filenames with the answer document you recorded earlier.

You should see five numbered results with filenames and keyword scores. Each result should include a short text preview.

✔️ The answer is below the top five

You found the intended shortfall. Exact word overlap promoted documents that echoed the question while the useful answer stayed buried.

Good work. Your baseline now gives the reranker a meaningful ranking problem to solve.

ⓧ The answer is in the top five

This question gives keyword search an easy win. A better test uses wording that appears across several distractor documents.

  • Replace the text assigned to QUESTION with another question about your corpus.
  • Use important terms that appear in several distractor documents.
  • Save rerank_notes.py.

Before you rerun the script, predict which distractor filenames the revised wording promotes.

  • Test the revised question by running this command:
python rerank_notes.py

Why rerun the baseline?

The new run scores every document against the revised question. Adjust the question until the correct-answer document is absent from the first five results.

Answer still ranking too highly?

Choose a question containing terms that appear in several unrelated notes. Keep the correct answer phrased with synonyms or more specific language.

Help me design a useful baseline question.

✔️ Awesome, I've got everything!

Your corpus is loaded. Double check that you saved rerank_notes.py before moving on.

ⓧ I'd like to double check the full code

Compare your script with this complete version. Your selected question appears in the QUESTION value.

from pathlib import Path
import re

# Point the script at the learner-owned search corpus.
DOCS_DIR = Path("docs")
QUESTION = """[[QUESTION="your question about the notes"]]"""
TOP_CANDIDATES = 20
TOP_RESULTS = 5

def load_documents():
    # Keep each filename with its text so ranked results stay traceable.
    return [
        {"name": path.name, "text": path.read_text(encoding="utf-8")}
        for path in sorted(DOCS_DIR.glob("*.txt"))
    ]

documents = load_documents()

# Stop early if the corpus falls outside the planned size.
if not 30 <= len(documents) <= 50:
    raise ValueError("Add 30 to 50 .txt documents to docs/.")

print(f"Loaded {len(documents)} documents.")

def rank_by_keywords(documents, question):
    # Normalize the question into unique terms for simple overlap scoring.
    question_terms = set(re.findall(r"\b\w+\b", question.lower()))

    for document in documents:
        # Count the unique question terms found in this document.
        document_terms = set(re.findall(r"\b\w+\b", document["text"].lower()))
        document["keyword_score"] = len(question_terms & document_terms)

    # Put documents with the highest overlap first.
    return sorted(
        documents,
        key=lambda document: document["keyword_score"],
        reverse=True,
    )

ranked_documents = rank_by_keywords(documents, QUESTION)
candidates = ranked_documents[:TOP_CANDIDATES]
print(f"Selected {len(candidates)} keyword candidates.")

print(f"\nQuestion: {QUESTION}")
print("Top five keyword results:")

for rank, document in enumerate(candidates[:TOP_RESULTS], start=1):
    # Collapse whitespace so every result has a compact preview.
    preview = " ".join(document["text"].split())[:180]
    print(f"\n{rank}. {document['name']} | score={document['keyword_score']}")
    print(preview)

Your keyword baseline now exposes the ranking problem. Next up, you will send its 20 candidates to rerank-3-lite and see whether the correct document moves upward.

Rerank the Top Candidates

Your keyword search has exposed the gap in first-pass retrieval. The answer is present among the 20 candidates. Simple term overlap still leaves it below the first five.

A reranker reads your question beside each candidate to judge semantic relevance. In this step, you will send the candidates to Voyage AI using rerank-3-lite. You will then print the reordered documents with their relevance scores.

In this step, get ready to:
  • Create a Voyage client that reads your existing VOYAGE_API_KEY environment variable.
  • Send QUESTION plus the 20 keyword candidates to rerank-3-lite in one request.
  • Print the reordered documents with their relevance scores.
Add the Voyage reranking pass

The Voyage client automatically reads VOYAGE_API_KEY from your terminal environment. The reranking request keeps your existing keyword results as its candidate set.

  • In rerank_notes.py, move to the end of the existing loop that prints the five keyword-ranked results.
  • Add the reranking pass by copying this code below that loop:
import voyageai

# Read the API key from the existing VOYAGE_API_KEY environment variable
client = voyageai.Client()

# Send the 20 keyword candidates in one reranking request
candidate_texts = [candidate["text"] for candidate in top_candidates]
reranking = client.rerank(
    QUESTION,
    candidate_texts,
    model="rerank-3-lite",
)

# Show the new order with a comparable score for each document
print("\nRerank-3-lite results:")
for rank, result in enumerate(reranking.results, start=1):
    print(f"\n{rank}. Relevance score: {result.relevance_score:.4f}")
    print(result.document)

How does the reranking pass work?

  • The voyageai.Client() call creates an authenticated client from the environment variable you configured earlier.
  • The candidate_texts list extracts the text from each item in top_candidates.
  • The client.rerank() call compares QUESTION with all 20 candidates in one batch.
  • The result loop prints each document in descending relevance order. Its relevance_score helps you compare candidates within this run.
  • Save rerank_notes.py.
  • Confirm the new section contains model="rerank-3-lite" below the keyword output loop.

✔️ Awesome, I've got everything!

Your reranking block is in place. Your saved script is ready to make its first reranking request.

ⓧ I'd like to double check the full code

Compare the new section at the bottom of rerank_notes.py with this reference. Keep your existing document loader plus keyword search above it.

import voyageai

# Read the API key from the existing VOYAGE_API_KEY environment variable
client = voyageai.Client()

# Send the 20 keyword candidates in one reranking request
candidate_texts = [candidate["text"] for candidate in top_candidates]
reranking = client.rerank(
    QUESTION,
    candidate_texts,
    model="rerank-3-lite",
)

# Show the new order with a comparable score for each document
print("\nRerank-3-lite results:")
for rank, result in enumerate(reranking.results, start=1):
    print(f"\n{rank}. Relevance score: {result.relevance_score:.4f}")
    print(result.document)
Test the reranked search

The next run reveals whether semantic relevance can rescue the answer that keyword overlap buried. Before you run it, which document do you expect the reranker to promote above its keyword position? Keep that guess in mind.

  • Run the updated script from the project terminal with this command:
python3 rerank_notes.py

You will first see the five keyword-ranked results. Under Rerank-3-lite results:, you will see all 20 candidates in a new order. Each candidate has a relevance score.

That ranking shift is your first reranker win. The answer-bearing document now appears above its original keyword position with a score beside it.

Missing the reranked results?

  • If the request stops before the reranked heading, continue using the project terminal where VOYAGE_API_KEY is available.
  • If Python points to candidate["text"], confirm each item in top_candidates contains the document text under the text key.
  • If the answer remains low, confirm your selected QUESTION matches information that appears in one of the 20 candidates.

Help me diagnose my reranking output.

Your keyword candidates now pass through a semantic reranker before reaching the screen. Next, you will run the same candidates through rerank-3 to compare where the two models disagree.

Compare Both Reranker Models

Your rerank-3-lite results already show how a reranker can reorder weak keyword candidates. One ranking alone cannot reveal whether the full model makes different relevance judgments.

This comparison holds QUESTION constant across the same 20 candidates. A side-by-side table then exposes where the two Voyage AI models agree or disagree.

In this step, get ready to:
  • Send the same 20 candidates to rerank-3.
  • Display both model rankings side by side.
  • Identify positions where the model order differs.
Run the full reranker on the same candidates

A fair model comparison keeps every input fixed. The second request changes only the model so any ranking shift comes from that model's relevance judgment.

  • In rerank_notes.py, find the existing block that contains model="rerank-3-lite".
  • Place your cursor after the loop that prints the lite model results.
  • Add the full model request with the following code:
# Rerank the same candidate set with Voyage's full model
full_ranking = client.rerank(
    QUESTION,
    top_candidates,
    model="rerank-3",
)

# Print the full model's order and scores
print("\n=== rerank-3 results ===")
for rank, result in enumerate(full_ranking.results, start=1):
    print(f"{rank:>2}. {result.relevance_score:.4f} | {result.document[:80]}")

What does this code do?

  • The full_ranking variable stores the response from the full model.
  • The request reuses QUESTION and top_candidates so the inputs stay fixed.
  • The loop prints each document beside its relevance_score value.
  • Save rerank_notes.py.
  • Return to the project terminal from earlier.
  • Execute the updated script by running this command:
python3 rerank_notes.py

What should you see?

You should see a new rerank-3 section below the lite results. It lists the full model's document order with one relevance score per result.

Missing the full model results?

  • Confirm that the new request sits after top_candidates is created.
  • Check that the model value is exactly rerank-3.
  • Run the script from the same terminal that already has VOYAGE_API_KEY available.

Ask for help with the exact output you received: Help me debug why my rerank-3 request is not printing results.

Strong progress. Your script now evaluates both reranker models against the same evidence.

Display the rankings side by side

Separate result lists make disagreements hard to spot. A shared table aligns equal rank positions so each difference becomes visible in one row.

  • In rerank_notes.py, place your cursor after the loop that prints the full model results.
  • Add the side-by-side comparison with the following code:
# Shorten each document so both rankings fit in one row
def preview(document, width=34):
    return " ".join(document.split())[:width]

# Print matching rank positions in one table
print("\n=== Side-by-side comparison ===")
print("Rank | rerank-3-lite                      | rerank-3")
print("-" * 83)
for rank, (lite_result, full_result) in enumerate(
    zip(lite_ranking.results, full_ranking.results),
    start=1,
):
    lite_text = preview(lite_result.document)
    full_text = preview(full_result.document)
    print(
        f"{rank:>4} | {lite_text:<34} {lite_result.relevance_score:.4f} | "
        f"{full_text:<34} {full_result.relevance_score:.4f}"
    )

How does the comparison work?

  • The preview() function shortens each document so the table stays readable.
  • The zip() call pairs results that occupy the same rank position.
  • Each row shows both document previews with their model-specific scores.
  • A disagreement appears when the two document previews differ in the same row.
  • Save rerank_notes.py.

Before you run the final comparison, do you expect both models to produce the same top five order?

  • Generate the side-by-side table by running this command:
python3 rerank_notes.py

What should you see?

You should see a table with rerank-3-lite on the left and rerank-3 on the right. Every row shows a rank position with both relevance scores.

Matching previews show agreement at that position. Different previews expose a ranking disagreement.

Table missing or misaligned?

  • Confirm that both lite_ranking and full_ranking are created before the comparison loop.
  • Check the closing parenthesis around the zip() call.
  • Widen the terminal panel if long document previews wrap onto another line.

Share the error or malformed row for targeted help: Help me fix my side-by-side reranker comparison table.

  • Compare the document previews across the first five rows.
  • Find the first rank where the two previews differ.
  • Review all 20 positions if the first five rows match.
  • Identify which document each model places at that rank.

✔️ Awesome, I've got everything!

Great. Double check that rerank_notes.py is saved before you continue.

ⓧ I'd like to double check the full code

Compare your script with this cumulative version. Keep your existing question in QUESTION.

from pathlib import Path
import os
import re

import voyageai

# Keep the question focused on an answer inside your documents
QUESTION = "[[QUESTION=\"your question about the documents\"]]"

# Load every learner-owned text document from the docs folder
documents = [
    {
        "name": path.name,
        "text": path.read_text(encoding="utf-8"),
    }
    for path in sorted(Path("docs").glob("*.txt"))
]

# Count how many distinct question words appear in each document
def keyword_score(document):
    question_words = set(re.findall(r"\w+", QUESTION.lower()))
    document_words = set(re.findall(r"\w+", document.lower()))
    return len(question_words & document_words)

# Rank documents by keyword overlap and keep the strongest candidates
keyword_ranking = sorted(
    documents,
    key=lambda document: keyword_score(document["text"]),
    reverse=True,
)
top_candidates = [document["text"] for document in keyword_ranking[:20]]

# Show the first-pass keyword ranking
print("=== Keyword results ===")
for rank, document in enumerate(keyword_ranking[:5], start=1):
    print(
        f"{rank:>2}. {keyword_score(document['text'])} matches | "
        f"{document['name']} | {document['text'][:80]}"
    )

# Create a client from the API key stored in the environment
client = voyageai.Client(api_key=os.environ["VOYAGE_API_KEY"])

# Rerank the keyword candidates with the lite model
lite_ranking = client.rerank(
    QUESTION,
    top_candidates,
    model="rerank-3-lite",
)

# Print the lite model's order and scores
print("\n=== rerank-3-lite results ===")
for rank, result in enumerate(lite_ranking.results, start=1):
    print(f"{rank:>2}. {result.relevance_score:.4f} | {result.document[:80]}")

# Rerank the same candidate set with Voyage's full model
full_ranking = client.rerank(
    QUESTION,
    top_candidates,
    model="rerank-3",
)

# Print the full model's order and scores
print("\n=== rerank-3 results ===")
for rank, result in enumerate(full_ranking.results, start=1):
    print(f"{rank:>2}. {result.relevance_score:.4f} | {result.document[:80]}")

# Shorten each document so both rankings fit in one row
def preview(document, width=34):
    return " ".join(document.split())[:width]

# Print matching rank positions in one table
print("\n=== Side-by-side comparison ===")
print("Rank | rerank-3-lite                      | rerank-3")
print("-" * 83)
for rank, (lite_result, full_result) in enumerate(
    zip(lite_ranking.results, full_ranking.results),
    start=1,
):
    lite_text = preview(lite_result.document)
    full_text = preview(full_result.document)
    print(
        f"{rank:>4} | {lite_text:<34} {lite_result.relevance_score:.4f} | "
        f"{full_text:<34} {full_result.relevance_score:.4f}"
    )

You can now see how the lite and full rerankers judge the same candidates. Next up, you will add a plain-English instruction and watch relevance shift toward your preferred evidence.

Steer the Ranking with Instructions

Your Voyage AI comparison now reveals where rerank-3-lite and rerank-3 disagree. Both models still decide relevance from the question alone.

A reranker can also account for qualities such as recency or source authority. In this step, you will add a plain-English instruction to steer both models toward the documents you value.

In this step, get ready to:
  • Choose a ranking preference that your documents can support.
  • Send the preference to both reranking models with the original question.
  • Compare the original ranks with the instructed ranks.
Choose a useful instruction

Instruction following lets you define what relevance means for a particular search. The best instruction describes a distinction that already exists in your candidates.

  • Review the original comparison in the terminal to identify a meaningful difference between your documents.
  • Choose a preference that the text can support through dates, source names, document types, or another visible clue.
  • Record the preference you want both models to follow: prefer recent or official documents.

How does an instruction affect relevance?

The instruction joins your existing question inside the reranking query. Each model then scores the same 20 candidates against that combined intent.

A relevance score reflects how well one candidate matches that intent. Treat the score as ranking evidence within this search.

Add the instructed reranking pass

A fair comparison keeps the candidates fixed. Only the query changes when you attach RERANK_INSTRUCTION to QUESTION.

  • Switch back to rerank_notes.py from earlier.
  • Find the lines that print the original lite and full comparison.
  • Paste the following code directly below that comparison:
# Define the source preference that should influence relevance
RERANK_INSTRUCTION = "[[RERANK_INSTRUCTION="prefer recent or official documents"]]"

def print_instruction_shift(model_name, original_result):
    # Combine the original question with the ranking preference
    instructed_query = f"{QUESTION}\nInstruction: {RERANK_INSTRUCTION}"

    # Score the same candidates so the query is the only change
    instructed_result = client.rerank(
        query=instructed_query,
        documents=candidate_documents,
        model=model_name,
        top_k=20,
    )

    # Match each candidate to its position in the original ranking
    original_ranks = {item.index: rank for rank, item in enumerate(original_result.results, 1)}
    print(f"\n{model_name} instruction shift")
    print("New | Old | Shift | Score | Document")
    for instructed_rank, item in enumerate(instructed_result.results[:5], 1):
        original_rank = original_ranks[item.index]
        shift = original_rank - instructed_rank
        preview = item.document.replace("\n", " ")[:70]
        print(f"{instructed_rank:>3} | {original_rank:>3} | {shift:+5d} | {item.relevance_score:.4f} | {preview}")
    return instructed_result

# Compare both instructed rankings with their original model results
instructed_lite = print_instruction_shift("rerank-3-lite", lite_reranking)
instructed_full = print_instruction_shift("rerank-3", full_reranking)

What does this code do?

  • The instructed_query value combines the original question with your source preference.
  • Each client.rerank() call receives the same 20 candidate documents used in the original comparison.
  • The original_ranks mapping finds where each document appeared before the instruction.
  • A positive shift value means the instruction moved that document closer to rank 1.
  • Save rerank_notes.py.

Before you run the script, pause on one prediction: the document you expect your instruction to move closest to rank 1.

  • Run the updated ranking comparison with this command:
python rerank_notes.py

You will see a new instruction-shift section for each model. Each row shows the rank change plus the instructed relevance score.

Look for a non-zero shift among the top results. That movement proves the instruction changed how the model interpreted relevance.

Do all the shifts show zero?

Use an instruction tied to evidence that appears in several candidates. A preference for recent documents needs dates or recency language in those documents.

If the new sections do not print, confirm the added code sits after lite_reranking and full_reranking are created.

Help me diagnose why my reranking instruction is not changing the results.

✔️ Awesome, I've got everything!

Your script now compares the original rankings with instruction-steered results. Double check that you saved rerank_notes.py before moving on.

ⓧ I'd like to double check the full code

Compare your cumulative rerank_notes.py with this reference. Keep your existing question inside QUESTION.

from pathlib import Path
import re

import voyageai

# Keep the question selected for your document collection
QUESTION = "[[QUESTION="your existing question"]]"
DOCS_DIR = Path("docs")


def tokenize(text):
    # Normalize text into words for the first-pass keyword score
    return re.findall(r"\b\w+\b", text.lower())


def keyword_score(text):
    # Count how many query terms appear in one document
    query_terms = set(tokenize(QUESTION))
    document_terms = tokenize(text)
    return sum(document_terms.count(term) for term in query_terms)


# Load every learner-owned document from the docs folder
document_records = [
    {"name": path.name, "text": path.read_text(encoding="utf-8")}
    for path in sorted(DOCS_DIR.iterdir())
    if path.is_file()
]

# Rank all documents with simple keyword matching
ranked_records = sorted(
    document_records,
    key=lambda record: keyword_score(record["text"]),
    reverse=True,
)
top_candidates = ranked_records[:20]
candidate_documents = [
    f"{record['name']}\n{record['text']}" for record in top_candidates
]

# Show the weak first-pass ranking before semantic reranking
print("Keyword search top five")
for rank, record in enumerate(top_candidates[:5], 1):
    preview = record["text"].replace("\n", " ")[:70]
    print(f"{rank:>2}. {record['name']} | {preview}")

# Create a client that reads VOYAGE_API_KEY from the environment
client = voyageai.Client()

# Rerank the same candidates with the lite and full models
lite_reranking = client.rerank(
    query=QUESTION,
    documents=candidate_documents,
    model="rerank-3-lite",
    top_k=20,
)
full_reranking = client.rerank(
    query=QUESTION,
    documents=candidate_documents,
    model="rerank-3",
    top_k=20,
)


def print_side_by_side(lite_result, full_result):
    # Display both model orders with their relevance scores
    print("\nOriginal reranked comparison")
    print("Rank | rerank-3-lite | rerank-3")
    for rank, (lite_item, full_item) in enumerate(
        zip(lite_result.results[:5], full_result.results[:5]),
        1,
    ):
        lite_preview = lite_item.document.replace("\n", " ")[:35]
        full_preview = full_item.document.replace("\n", " ")[:35]
        print(
            f"{rank:>4} | {lite_item.relevance_score:.4f} {lite_preview} | "
            f"{full_item.relevance_score:.4f} {full_preview}"
        )


# Print the original model comparison before adding instructions
print_side_by_side(lite_reranking, full_reranking)

# Define the source preference that should influence relevance
RERANK_INSTRUCTION = "[[RERANK_INSTRUCTION="prefer recent or official documents"]]"


def print_instruction_shift(model_name, original_result):
    # Combine the original question with the ranking preference
    instructed_query = f"{QUESTION}\nInstruction: {RERANK_INSTRUCTION}"

    # Score the same candidates so the query is the only change
    instructed_result = client.rerank(
        query=instructed_query,
        documents=candidate_documents,
        model=model_name,
        top_k=20,
    )

    # Match each candidate to its position in the original ranking
    original_ranks = {item.index: rank for rank, item in enumerate(original_result.results, 1)}
    print(f"\n{model_name} instruction shift")
    print("New | Old | Shift | Score | Document")
    for instructed_rank, item in enumerate(instructed_result.results[:5], 1):
        original_rank = original_ranks[item.index]
        shift = original_rank - instructed_rank
        preview = item.document.replace("\n", " ")[:70]
        print(f"{instructed_rank:>3} | {original_rank:>3} | {shift:+5d} | {item.relevance_score:.4f} | {preview}")
    return instructed_result


# Compare both instructed rankings with their original model results
instructed_lite = print_instruction_shift("rerank-3-lite", lite_reranking)
instructed_full = print_instruction_shift("rerank-3", full_reranking)

You did it. Your search can now treat a source preference as part of relevance.

Next, you will save the keyword rank plus both model ranks to a CSV for a spreadsheet-ready comparison.

Export the Ranking Comparison

Your instructed reranking query now shifts the candidate order. You can compare the two model rankings in your terminal.

Terminal output is temporary. Exporting the results to a CSV gives you a record that you can filter or share.

In this step, get ready to:
  • Add a CSV export to rerank_notes.py.
  • Generate ranking_comparison.csv with one row for each candidate.
  • Open the ranking comparison in a spreadsheet.
Write the CSV export

Each CSV row connects one document to its position in all three rankings. Candidate indexes let you match reordered model results back to the original keyword order.

  • Open rerank_notes.py in the same editor you used earlier.
  • Add Python's built-in CSV module beside the existing imports.
  • Place the export logic after the code that produces both instructed reranking results.
  • Configure the export for ranking_comparison.csv with columns in this order:
  • Use query, document, keyword rank, lite rank, full rank.
  • Build a one-based rank lookup from the lite result indexes.
  • Build a one-based rank lookup from the full result indexes.
  • Write one row for every candidate in keyword order.

How the CSV preserves each ranking

  • The query column stores the instructed query sent to both rerankers.
  • The document column stores the complete candidate text.
  • The keyword rank column stores the candidate's original position from 1 through 20.
  • The lite rank column stores the candidate's position in the lite result.
  • The full rank column stores the candidate's position in the full result.
  • Save rerank_notes.py.
Generate and inspect the CSV

Running the updated script should now create a file beside rerank_notes.py. The file should contain the same 20 candidates used in the model comparison.

Before you rerun the script, do you expect the CSV to contain 5 candidates or all 20 candidates?

  • Return to the project terminal from earlier.
  • Rerun rerank_notes.py with the same Python command you used in the previous step.
  • Select ranking_comparison.csv in your editor's file sidebar.
  • Count the rows below the header.

You should see 20 data rows beneath the header. Your ranking comparison now survives after the terminal output scrolls away.

CSV file missing or empty?

  • Confirm that you saved rerank_notes.py before rerunning it.
  • Check that the export logic comes after both instructed reranking results have been created.
  • Check that the row-writing loop uses all 20 keyword candidates.

Ask for help with the result you see: Help me debug why rerank_notes.py does not create a populated ranking_comparison.csv file.

Open the comparison in a spreadsheet

A spreadsheet places the three rank columns beside each document. Differences between the model orders become visible row by row.

Before you inspect the rows, do you expect the lite and full rerankers to assign every document the same position?

  • Use your editor's file sidebar context menu to reveal ranking_comparison.csv in Finder or File Explorer.
  • Double-click ranking_comparison.csv to open it in your default spreadsheet app.
  • Locate the row containing the document that best answers your question.
  • Compare its keyword rank with its two model ranks.
  • Find one row where lite rank differs from full rank.

You should see the five expected column headings above 20 document rows. The rank columns should contain positions from 1 through 20.

How to read rank disagreements

A smaller rank number means the model placed that document closer to the top. Different lite and full values show where the two rerankers disagree.

You have turned a scrolling terminal experiment into a comparison you can inspect in one view. The CSV now captures the query plus every document's position across all three rankings.

Secret mission

Stress-Test Your Reranker

Run the same corpus through a second question. Tighten one ranking instruction. Reveal whether both rerankers respond the same way.

Clean Up Your Resources

Clean Up Your Resources

Your local files have no ongoing cost. Choose whether to keep your work, pause by closing the open tools, or delete the project resources.

Resources you used:

  • Your active Voyage AI API key in the Voyage dashboard.
  • The current terminal session with the VOYAGE_API_KEY environment variable available.
  • Your local project folder containing docs/, rerank_notes.py, and ranking_comparison.csv.

Keep everything running

No action is needed. Choose this if you plan to rerun the ranking comparison or experiment with more questions.

  • Keep the project folder in its current location.
  • Leave the Voyage API key active while you continue using the script.
  • Keep the API key private.

Pause - I'll come back to this later

Close the open tools to pause your work. Your documents, script, and CSV stay on disk.

  • Close the spreadsheet containing ranking_comparison.csv.
  • Close the current terminal session.
  • Leave the project folder in its current location.

No ranking process continues running after the script finishes. The Voyage API key remains active until you delete it from the dashboard.

Delete - I don't want to use this again

Remove the Voyage API key before deleting the local project. This prevents the unused credential from authorizing future requests.

  • Return to the Voyage dashboard from earlier.
  • Locate the API key you created for this project.
  • Use the key's delete control to remove it.
  • Confirm the API key no longer appears in the dashboard.

That credential can no longer authorize requests. The value in your current terminal session is now unusable.

  • Close the spreadsheet containing ranking_comparison.csv.
  • Close the current terminal session.

Deleting the local project folder is permanent. Your documents, script, and comparison CSV disappear with it.

macOS

Use Finder to remove the project folder from your Mac.

  • Press Cmd+Space to open macOS search.
  • Type Finder into the search field.
  • Press Enter to open Finder.
  • Return to the location containing the project folder from earlier.
  • Select the folder containing docs/, rerank_notes.py, and ranking_comparison.csv.
  • Drag the selected project folder to Trash.
  • Confirm the project folder no longer appears in its original location.

Windows

Use File Explorer to remove the project folder from your Windows computer.

  • Press the Windows key to open Windows search.
  • Type File Explorer into the search field.
  • Press Enter to open File Explorer.
  • Return to the location containing the project folder from earlier.
  • Select the folder containing docs/, rerank_notes.py, and ranking_comparison.csv.
  • Press Delete to move the selected project folder to the Recycle Bin.
  • Confirm the project folder no longer appears in its original location.

Nice Work!

Nice Work!

Strong finish! You built a Python search evaluation workflow that compares keyword ranking with Voyage AI reranking across your own documents.

What you learned:

  • Built a keyword search baseline over 30 to 50 documents. It exposed cases where the correct answer appeared below weaker results.
  • Reranked the top 20 candidates with rerank-3-lite plus rerank-3. You compared their relevance scores plus ranking decisions.
  • Steered relevance with a plain-English instruction. You exported the keyword rank plus both reranker ranks to ranking_comparison.csv for spreadsheet review.
  • Completed an optional Secret Mission to push your reranking skills further.

Ready to quiz yourself?