Route AI Requests by Confidence

Build a router that sends low-confidence requests to a chat model.

Introduction

30 Second Summary

Support queues often contain messages that only need sorting. Sending every message to a full chat model spends time and money on prose that nobody needs.

In this project, you will use OpenRouter to compare Jev, a decision model, with a normal chat model on the same support messages. You will build a confidence-based router that sends uncertain messages to the chat model while handling confident messages with canned answers.

What You'll Build

You will run a batch of support messages and watch a routing table reveal each category, confidence score, chosen path, response time, and cost.

By the end of this project, you'll have:

A side-by-side model comparison that shows how typed scores differ from a written chat response.
A scored batch of twenty messages where every message has a category and confidence score.
A confidence-based router that reserves written replies for uncertain messages and reports the cost of each route.
An optional Secret Mission to push your routing skills further.

Are there any prerequisites?

Have these ready before you begin:

Use an OpenRouter account with a small credit balance.
Prepare twenty short support messages from real examples or messages written for this exercise.

Before We Start

Your routing experiment compares model outputs from the same support messages. That comparison depends on a local place to build the script.

A terminal runs the script. A code editor gives you a place to write it.

The model requests need internet access. This step checks all three basics before any API work begins.

In this step, get ready to:

Confirm that a terminal is available.
Confirm that a code editor is available.
Verify that your internet connection can reach OpenRouter.

Confirm your terminal

The terminal provides the command prompt that runs your local script. A visible prompt confirms that the application is ready for later commands.

Use your operating system's application search to launch its terminal.
Confirm that the terminal window displays a command prompt.

One tool is ready. Your terminal can now accept local commands.

Can't Launch a Terminal?

Use your operating system's help search to find its built-in terminal application.
Restart your device if the terminal opens to a blank window.

Ask for help with your specific device:

Confirm your code editor

The code editor holds the support-message routing script. Its start screen confirms that you can create local files when the build begins.

Use your operating system's application search to launch the code editor you plan to use.
Confirm that the editor reaches its start screen without an installation prompt.

Your local workspace is taking shape. The editor is ready for the routing script.

Don't Have a Code Editor?

Visit the official website for the code editor you want to use.
Follow its installation guide for your operating system.
Launch the editor again through application search.

Get help choosing or installing an editor:

Check your internet connection

The script later sends model requests through OpenRouter. Loading its website confirms that your connection can reach the service.

Use your operating system's application search to launch your web browser.
Visit the OpenRouter website.

Before you refresh, do you expect the page to load again without an offline message?

Refresh the OpenRouter website.

You should see the website load again. That confirms your internet connection can reach the service.

Website Not Loading?

Confirm that your device's network connection is enabled.
Try loading a second trusted webpage.
Restart your browser if other webpages load successfully.

Troubleshoot the connection with guided help:

Your local workspace is ready. Next, you'll use OpenRouter to send a support message to the decision model.

Test a Decision Model

Support teams often need a category or urgency score before anyone writes a reply. A decision model produces those bounded values with confidence information that code can use.

In this step, you will connect to OpenRouter and test Jev 1.14 on one support message. The result gives your future router a typed category plus numeric scores.

In this step, get ready to:

Store an OpenRouter API key in an environment variable.
Create a local script that sends one support message to a decision model.
Inspect the category, scores, and confidence in the response.

Configure your OpenRouter key

An API key identifies your requests to OpenRouter. The OPENROUTER_API_KEY environment variable keeps that credential outside your Python file.

Keep the key private

API keys deserve extra care because they authorize billable requests. Keeping this key in an environment variable prevents it from being saved inside your routing script.

Visit the OpenRouter API keys page in your browser.
Sign in to your OpenRouter account.
Create one API key for this project.
Copy the key when OpenRouter shows it.

Your terminal can hold the key for its current session. The script reads the value when it sends the request.

Select your terminal platform below.

macOS

Store your copied key in OPENROUTER_API_KEY by replacing sk-or-v1-YOUR-API-KEY before running this command:

export OPENROUTER_API_KEY="sk-or-v1-YOUR-API-KEY"

What does this command do?

This command assigns your key to OPENROUTER_API_KEY in the current terminal session. Your script can authenticate without containing the credential.

Windows

Store your copied key in OPENROUTER_API_KEY by replacing sk-or-v1-YOUR-API-KEY before running this PowerShell command:

$env:OPENROUTER_API_KEY="sk-or-v1-YOUR-API-KEY"

What does this command do?

This command assigns your key to OPENROUTER_API_KEY in the current PowerShell session. Your script can authenticate without containing the credential.

The sensitive setup is complete. Your key now stays separate from the code that uses it.

Create the decision script

The Requests library sends the support message to the Decisions API. Your script defines a category question plus an urgency scale before making that request.

Install the Requests library by running this command:

python -m pip install requests

What does this command do?

The python command starts your Python installation. The pip module adds requests so the script can make an HTTP request.

Read the final terminal line to confirm Requests installed or was already available.

Requests installation failed?

Select the same Python interpreter that your code editor uses.
Repeat the installation command in that terminal.

If the installation still fails, help me troubleshoot my Requests installation.

Create test_openrouter.py in the folder shown by your terminal prompt by pasting the code below:

import json
import os
import requests
# Store a realistic ticket that needs classification.
message = "I was charged twice for the same order and need the extra charge refunded."
# Define the typed decisions the routing workflow needs.
questions = {
    "category": {
        "type": "choice",
        "instructions": "Which support category best fits this message?",
        "criteria": {"billing": "Charges, invoices, or refunds.", "account": "Login or account access.", "product": "Product features or compatibility."},
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this support message?",
        "criteria": ["Can wait", "Needs attention soon", "Needs immediate attention"],
    },
}

# Send the state and questions to the selected decision model.
response = requests.post(
    "https://openrouter.ai/api/alpha/decisions",
    headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}", "Content-Type": "application/json"},
    json={"model": "typesafe/jev-1.14", "state": {"ticket": message}, "questions": questions},
    timeout=30,
)
response.raise_for_status()

# Print every typed answer with its numeric values.
print(json.dumps(response.json()["answers"], indent=2))

What does this code do?

The message variable holds one billing-related support request.
The category question asks the model to select one defined option.
The urgency question asks the model to place the message on an ordered scale.
The final print displays the typed answers plus their probabilities and confidence values.

Save test_openrouter.py.

Run your first decision

Before you run the script, which category do you expect the decision model to select?

Submit the sample support message by running this command:

python test_openrouter.py

What should I see?

You will see a category object with the choice type. It includes the selected category plus numeric probabilities and confidence.

You will also see an urgency object with the score type. Its numeric score places the message on the ordered urgency scale.

Decision request failed?

Check that OPENROUTER_API_KEY is set in the same terminal that runs the script.
Replace the example key placeholder with your real OpenRouter key.
Confirm that your OpenRouter account can make billed requests.

If the request still fails, help me debug my OpenRouter decision request.

You did it. One support message now produces a typed category plus numeric signals that your code can inspect.

✔️ Awesome, I've got everything!

Your saved script now contains the complete first decision-model test.

ⓧ I'd like to double check the full code

import json
import os
import requests
# Store a realistic ticket that needs classification.
message = "I was charged twice for the same order and need the extra charge refunded."
# Define the typed decisions the routing workflow needs.
questions = {
    "category": {
        "type": "choice",
        "instructions": "Which support category best fits this message?",
        "criteria": {"billing": "Charges, invoices, or refunds.", "account": "Login or account access.", "product": "Product features or compatibility."},
    },
    "urgency": {
        "type": "score",
        "instructions": "How urgent is this support message?",
        "criteria": ["Can wait", "Needs attention soon", "Needs immediate attention"],
    },
}

# Send the state and questions to the selected decision model.
response = requests.post(
    "https://openrouter.ai/api/alpha/decisions",
    headers={"Authorization": f"Bearer {os.environ['OPENROUTER_API_KEY']}", "Content-Type": "application/json"},
    json={"model": "typesafe/jev-1.14", "state": {"ticket": message}, "questions": questions},
    timeout=30,
)
response.raise_for_status()

# Print every typed answer with its numeric values.
print(json.dumps(response.json()["answers"], indent=2))

How does this reference help?

This reference combines the sample message, typed questions, API request, and printed response in one file. It should match your saved test_openrouter.py.

Your decision model can now classify one support message without writing a reply. Next, you will send the same message to a chat model and compare the two output shapes.

Compare the Chat Model

Your decision model has already turned one support message into typed output. You can see its category plus numeric scores plus confidence.

A real support workflow also needs a written reply when a message needs deeper handling. This step sends the same message to a chat model through OpenRouter so you can compare both output shapes.

In this step, get ready to:

Send the existing sample support message to a selected chat model.
Display the decision output beside the written response.
Explain why each output shape suits a different job.

Send the same message to the chat model

The existing decision request gives you the baseline. Reusing its sample message ensures both models receive identical input.

Switch back to the support routing script from earlier.
Extend the script with a second request whose selected chat model receives the existing sample support message before its written reply prints below the current decision output.
Save the support routing script.
Rerun the script in the terminal from earlier using the same command.

You'll see the familiar typed decision result followed by a written reply to the same support message.

No Written Reply?

Confirm the second request reuses the environment-backed API configuration from earlier.
Check that the second request uses the existing sample support message.

Ask for help with the second model request.

Place both outputs side by side

A vertical stack can hide the contrast between the results. A side-by-side layout makes each model's role visible in one terminal view.

Format the output as two labeled sides that place the complete decision result opposite the written reply.
Save the support routing script.
Rerun the script in the terminal from earlier using the same command.

You'll see the typed category plus numeric scores plus confidence on the decision side. You'll see a complete written reply on the chat side.

Comparison Wrapping Incorrectly?

Increase the terminal window width if the two sides wrap onto separate lines.
Confirm each label prints beside its corresponding model result.

Ask for help formatting the comparison.

Run the final comparison

Before you run the final check, which side do you expect software to use directly as a routing signal?

Rerun the updated script in the terminal from earlier using the same command.

Comparison Checkpoint

The decision model's structured typed output and the chat model's natural-language written reply for the same sample message are displayed side by side.

Compare how quickly you can identify the category from the decision side.
Read the chat side as if you were the customer receiving it.
Confirm both sides use the same sample support message.

What Does This Comparison Prove?

The decision model returns values your code can compare directly. Its confidence becomes a routing signal.

The chat model returns language a customer can read. This side handles expression when a case needs a written response.

You made the contrast visible. The same support message now produces a routing signal beside a written reply.

Next up, you will run twenty messages through the decision model to map its category results plus confidence patterns.

Score Twenty Support Messages

Your earlier comparison made the difference visible. A decision model returns typed scores while a chat model writes a reply.

One message cannot reveal how confidence changes across different requests. A batch of twenty messages gives you enough results to spot patterns before choosing a routing threshold.

In this step, get ready to:

Create a batch of twenty support messages.
Score every message with the decision model.
Print a table containing each message, category, and confidence.

Build the message batch

A useful confidence table needs messages with different wording and levels of detail. Your batch can use real support messages or examples you write for this exercise.

Switch back to the support routing script in the code editor from earlier.
Leave the existing sample comparison unchanged.
Add exactly twenty short support messages as one batch input.
Save the support routing script.
Count the messages in the batch.

You should count exactly twenty messages. This set becomes the input for your decision score table.

Score every message

Batch scoring reuses the decision-model request that already works. Each response contributes one structured row to the final table.

What should the batch preserve?

The existing sample comparison still displays the decision model beside the chat model.
Each message in the new batch goes only to the decision model.
Each batch result becomes one row with message, category, and confidence columns.

Update the support routing script so its batch flow matches the three requirements above.
Save the support routing script.

Before you run the script, do you expect every batch message to produce one complete table row?

Return to the terminal from earlier.
Run the support routing script with the same command you used for the sample comparison.

Nice work: your twenty decisions are now visible in one table. You should see one row per message beneath the message, category, and confidence headings.

Table missing rows?

Check that the batch contains twenty entries. Confirm that your decision-model request runs once for each entry.

Make sure each result is added to the table inside the batch loop. A row added after the loop captures only one result.

Ask for help with the batch logic if the row count still falls short.

Verify the decision score table

Treat each confidence value as a model score that supports comparison across messages. Rows can share a category while carrying different confidence values.

Before you inspect the full table, do you think the confidence values will be tightly grouped or spread across a wider range?

Count the data rows beneath the table header.
Compare the row count with the twenty messages in your batch.
Scan the category column for blank cells.
Scan the confidence column for blank cells.
Find the message with the highest confidence.
Find the message with the lowest confidence.

You should find twenty complete rows. Every row represents one support message with a category and confidence from the decision model.

Your batch now exposes how confidence changes from message to message. Next up, you will choose a threshold and use it to decide which requests need a written response.

Route Messages by Confidence

Each of your twenty support messages now has a category plus a confidence value. Those scores can become routing decisions.

A confidence threshold creates the boundary. Messages below it use the selected chat model for a written reply. Messages at or above it receive a category-appropriate canned answer.

In this step, get ready to:

Choose a confidence threshold from the score distribution.
Route uncertain messages to the selected chat model.
Print the selected route plus its response for every message.

Choose a confidence threshold

Confidence measures how strongly the decision model supports its result. The threshold turns that signal into a consistent routing rule.

Return to the twenty-row score table from earlier.
Find the lowest confidence in the table.
Find the highest confidence in the table.
Choose a threshold greater than the lowest confidence.
Keep the threshold at or below the highest confidence.
Record your chosen value here: your confidence threshold.

This boundary leaves at least one message on each route. You can now compare written replies with canned answers in the same run.

What does the threshold control?

A confidence below the threshold triggers the chat route. This path generates a written response for an uncertain message.

A confidence equal to the threshold uses the canned route. Any higher value uses the same route.

Build the routing branches

The decision model still processes every support message first. Its confidence determines whether the script spends another model call on a written reply.

How does the router decide?

Every message first goes to the decision model. This preserves a category for every table row. It also preserves confidence.

A score below the threshold calls the selected chat model. Its written reply becomes the response.

A score at or above the threshold selects the canned answer for that category. The chat model is skipped.

Each completed row stores five fields. They are the support message, category, confidence, selected route, and response.

Switch back to the support routing script from earlier in your code editor.
Replace the current table-row logic with a routing block that uses your confidence threshold. Include one canned answer for every category in your existing table. Call the selected chat model only when confidence falls below the threshold. Store the selected route plus its response in each row.
Save the support routing script.

Before you run the script, which messages do you expect to use the chat route?

Rerun the support routing script with the same terminal command you used earlier.

You should see a twenty-row table with category, confidence, route, and response information. Both routing paths should appear.

Count the rows in the routing table.
Locate a row with confidence below your threshold.
Confirm that the low-confidence row uses the chat route.
Locate a row with confidence at or above your threshold.
Confirm that the higher-confidence row uses the canned route.
Read several canned responses to confirm that each one matches its row's category.

You have the routing loop working. Every support message now receives a response through a path chosen from its confidence.

Only seeing one route?

Compare your threshold with the lowest confidence in the table.
Compare your threshold with the highest confidence in the table.
Confirm that values below the threshold use the chat route.
Confirm that values equal to the threshold use the canned route.

If the routes still look wrong, ask for help checking the threshold condition.

✔️ Awesome, I've got everything!

Your saved script prints twenty routed messages. The table shows which path produced every response.

ⓧ I'd like to double check the full code

Confirm that the script retains all twenty support messages.
Confirm that the script retains the earlier sample comparison.
Confirm that one chosen confidence threshold controls the routing decision.
Confirm that every message still calls the selected decision model.
Confirm that confidence below the threshold calls the selected chat model.
Confirm that confidence at or above the threshold selects a category-appropriate canned answer.
Confirm that every table row contains the message, category, confidence, route, and response.

Your router now reserves written generation for uncertain messages. Next, you will measure the cost plus response time of each path.

Measure Cost and Response Time

Your twenty-message router now uses a decision model to choose either a canned answer or a reply from a chat model. That gives you the routing behavior you need to measure.

Cost exposes the price of each choice. Response time exposes the delay across each OpenRouter route. Measuring both across the full batch gives you evidence for the threshold you chose.

In this step, get ready to:

Record the cost of every routed message.
Measure the response time of each route.
Calculate total cost plus total response time for the batch.

Record each route's cost

Every message incurs a decision-model charge. A low-confidence message also incurs a chat-model charge. The row cost must include every model call used to produce that response.

Where does the cost come from?

OpenRouter reports the charge for each completed model request inside usage.cost. The value arrives with the response usage data.

Use the decision-model charge as the cost of a canned route. Use the sum of both model charges for a chat route.

Return to the support routing script in your code editor.
Update the existing row-building logic so its printed cost sums usage.cost from every model response used by that route.
Save the support routing script.

Before you rerun the script, consider which route will carry two model charges.

Rerun the support routing script with the same terminal command you used earlier.

You'll see a cost value on every row. Low-confidence rows include the charges from both model calls. Good progress. Your table now exposes the spend created by each routing decision.

Cost values missing?

Confirm that the decision-model response contributes usage.cost to every row.
Confirm that the chat-model response contributes usage.cost only on the low-confidence route.
Keep enough decimal places in the table to distinguish small request charges.

Help me debug the missing cost values in my routing table.

Measure each route's response time

A route's response time starts immediately before its decision-model request. It ends after the canned answer or chat-model reply is ready. This captures the complete wait experienced by that message.

Wrap each message's complete route in a monotonic elapsed-time measurement so the printed row shows its response time.
Save the support routing script.

Before you rerun the script, consider how the extra model call could affect low-confidence rows.

Rerun the support routing script with the same terminal command you used earlier.

You'll see a response-time value beside the cost on every row. You can now compare the complete canned route with the complete chat route.

Response times look incomplete?

Place the timer start immediately before the decision-model request.
Place the timer stop after the selected route has produced its final response.
Use the elapsed duration for the current row instead of a shared running duration.

Help me correct the response-time measurement in my routing script.

Build the batch summary

Individual rows reveal which messages are expensive or slow. Batch totals reveal the impact of your confidence threshold across all twenty messages.

Add a summary beneath the table that prints the sum of all row costs plus the sum of all row response times.
Save the support routing script.

Before you rerun the script, predict which total will be affected most by low-confidence messages.

Rerun the support routing script with the same terminal command you used earlier.

You'll see twenty rows with the message, category, confidence, route, response, cost, and response time. Beneath the table, you'll see total cost plus total response time for the complete batch.

Totals do not match the rows?

Calculate both totals after all twenty rows have been collected.
Sum the stored row values instead of reading the final row twice.
Reset both totals before each complete batch run.

Help me debug the totals in my routing table.

✔️ Awesome, I've got everything!

Great. Your saved routing script now measures each route and summarizes the complete batch.

ⓧ I'd like to double check the full code

The completed support routing script has the following cumulative behavior.

The script contains all twenty support messages.
The sample decision-model comparison remains available.
Every support message is sent to the selected decision model.
The chosen confidence threshold still selects either a canned route or a chat-model route.
Every row includes the support message, category, confidence, selected route, and response.
Every row includes its complete route cost plus its complete route response time.
The output includes total cost beneath the twenty rows.
The output includes total response time beneath the twenty rows.

Your routing table now exposes the cost plus response-time trade-off behind every decision. Next up, you'll move the confidence threshold and count how many messages shift to the expensive route.

Tune the Confidence Threshold

The routing table from the previous step shows how the decision model handled twenty support messages. It also shows the work passed to the chat model.

The confidence threshold controls that split. Testing values on both sides of your current cutoff reveals how sensitive the route count is to one policy choice.

In this step, get ready to:

Test a lower confidence threshold.
Test a higher confidence threshold.
Choose a final threshold from the measured route shifts.

Test a lower threshold

A lower cutoff allows more messages to use canned responses. Anchoring the new value to a confidence score from your table makes the test measurable.

Return to the twenty-row routing table in the terminal from earlier.
Count the rows that used the chat-model route.
Record that count as baseline chat count.
Select one row that used the chat-model route.
Choose a lower threshold just below that row's confidence.
Record the new value as your lower threshold.

The selected row gives your experiment a clear target. Crossing its confidence value should move it away from the written-response route.

Switch back to the local support-message routing script in your code editor.
Find the setting that holds the chosen confidence threshold.
Record its current value as your baseline threshold.
Replace the current value with your lower threshold.
Save the local support-message routing script.

Before you rerun the script, do you expect the chat-model count to rise or fall?

This rerun uses account credit for the existing twenty-message batch, which keeps the experiment bounded.

Return to the terminal from earlier.
Rerun the same script command you used in the previous step.

You should see a fresh twenty-row table with fewer chat-model routes when the threshold crosses the selected confidence. The targeted message should now take the canned route if its fresh confidence remains above the new cutoff.

Count the rows that used the chat-model route.
Record the result as lower-threshold chat count.
Record the displayed total cost as lower-threshold total cost.
Record the displayed total response time as lower-threshold total response time.

No routes changed?

The new cutoff must cross a confidence value to change a route. Fresh model scores can also vary between runs.

Choose a lower threshold beneath another confidence value from a chat-routed row.
Rerun the local support-message routing script.
Compare the new chat-model count with baseline chat count.

Help me understand why the lower threshold did not change my route count.

Your first controlled comparison is complete. The lower cutoff now has a measured route count plus batch totals.

Test a higher threshold

A higher cutoff sends more uncertain messages to the written-response route. This second run measures the cost of giving the chat model more work.

Return to the routing table produced with your baseline threshold.
Select one row that used the canned route.
Choose a higher threshold just above that row's confidence.
Record the new value as your higher threshold.

This value tests the opposite side of the routing trade-off. The selected canned response now sits below your proposed cutoff.

Switch back to the local support-message routing script in your code editor.
Replace the lower test value with your higher threshold.
Save the local support-message routing script.

Before you rerun the script, do you expect more messages to receive written replies?

Return to the terminal from earlier.
Rerun the same script command from the previous test.

You should see another twenty-row table with more chat-model routes when the new cutoff crosses the selected confidence. The targeted message should now receive a written reply if its fresh confidence remains below the higher cutoff.

Count the rows that used the chat-model route.
Record the result as higher-threshold chat count.
Record the displayed total cost as higher-threshold total cost.
Record the displayed total response time as higher-threshold total response time.

Higher count unchanged?

A cutoff between the same two confidence values preserves the route split. The higher value must cross a confidence score from a canned-response row.

Choose a threshold above another confidence value from a canned-response row.
Rerun the local support-message routing script.
Compare the new result with lower-threshold chat count.

Help me choose a higher threshold that changes at least one route.

You now have both sides of the experiment. The second run shows how extra chat coverage changes the batch totals.

Choose the final threshold

A useful final cutoff reflects the route shifts across the full batch. The lowest confidence categories also show where canned answers carry the most risk.

Calculate number of shifted messages by subtracting lower-threshold chat count from higher-threshold chat count.
Compare lower-threshold total cost with higher-threshold total cost.
Compare lower-threshold total response time with higher-threshold total response time.
Review the category labels attached to the lowest confidence scores across all three runs.
Identify any category that never reached your baseline threshold.
Identify any category that repeatedly appeared in low-confidence rows.
Record the findings as your low-confidence categories.

What does the threshold control?

A lower threshold gives canned responses more work. A higher threshold sends more uncertain cases to the chat model.

Batch cost plus response time reveal the operational trade-off. Repeatedly low-confidence categories reveal where written replies provide the strongest safety net.

Choose the threshold that gives these twenty messages the best routing balance.
Record the chosen value as your final threshold.
Switch back to the local support-message routing script in your code editor.
Rewrite the threshold setting with your final threshold plus a comment that records number of shifted messages.
Include your low-confidence categories in that comment.
Save the local support-message routing script.

Before the final run, which messages do you expect to take the chat-model route at your final threshold?

Return to the terminal from earlier.
Rerun the same script command with the final threshold saved.

What should I see?

You should see twenty support-message rows.
You should see a decision-model category for every message.
You should see a confidence value for every message.
You should see the selected route for every message.
You should see a canned answer or written reply for every message.
You should see a cost for every message.
You should see a response time for every message.
You should see total cost beneath the table.
You should see total response time beneath the table.
You should still see the earlier sample comparison.

Final route count looks wrong?

Confirm the saved setting matches your final threshold.
Check that confidence values below the threshold still use the chat-model route.
Check that confidence values at the threshold still use the canned route.

Help me debug why my final confidence threshold produces an unexpected route count.

That is the routing policy backed by evidence. Your final threshold now has route counts plus cost data from the complete twenty-message batch.

Secret mission

Design a Category-Aware Escalation Rule

Your global threshold gives every support category the same cutoff. In this mission, you will audit a low-confidence category and design a stricter escalation rule using evidence from your routing table.

Clean Up Your Resources

Your routing script runs locally. It only uses OpenRouter credits when you run it and send model requests.

Resources you used:

The local support-message routing script containing your twenty messages.
The final routing table containing your threshold results.
The OPENROUTER_API_KEY environment variable.
The corresponding API key in OpenRouter.
Any OpenRouter credit spend alerts created during the project.

Keep everything running

No action is needed. Choose this if you want to keep testing confidence thresholds or adding support messages.

Keep the local support-message routing script in its current location.
Keep the final routing table as a record of your chosen confidence threshold.
Keep the low-confidence category notes for future routing improvements.
Leave the API configuration in place only if you plan to run the script again.

Pause - I'll come back to this later

Shut down any active script process to stop new requests. Your files remain available for your next threshold experiment.

Stop an active routing-script run by pressing Ctrl+C in the terminal.
Close the terminal session after the process stops.
Keep the local support-message routing script in your code editor workspace.
Keep the final routing table with your chosen threshold notes.

You're safely paused. The script cannot send more model requests while it is stopped.

Delete - I don't want to use this again

Deletion is permanent. Your final routing table disappears with the local script.

macOS

Return to the Finder folder containing the local support-message routing script.
Move the local support-message routing script to Trash.
Empty Trash to remove the script from your Mac.

Windows

Return to the File Explorer folder containing the local support-message routing script.
Move the local support-message routing script to the Recycle Bin.
Empty the Recycle Bin to remove the script from your computer.

Remove OPENROUTER_API_KEY from the environment configuration where you stored it.
Delete the project API key from the OpenRouter page where you created it.
Remove any leftover credit spend alerts from the OpenRouter area where you configured them.

That's the cleanup complete. The project API key can no longer authorize new model requests.

Nice Work!

You did it! Your OpenRouter support-message router uses a cheap decision model to sort every request before a chat model writes the low-confidence replies.

You've learned how to:

Compare structured decisions containing categories and confidence scores with written replies from the same support message.
Route twenty support messages using a documented confidence threshold. High-confidence messages receive canned answers. Low-confidence messages receive chat-model replies.
Measure cost and response time for every route. Your final table totals both measurements. Higher and lower threshold runs reveal how many messages move to the expensive path.
Secret Mission: Take on an optional challenge that pushes your confidence-based routing skills further.

Ready to quiz yourself?