Cost Out an AI Model's Thinking Mode

Call an AI model with reasoning on and off, compare tokens and cost.

Introduction

30 Second Summary

You have probably seen a chatbot pause and "think" before answering. That pause is not free, and someone is paying for every extra word of it.

In this project, you will send the same prompt to one model twice, once with its reasoning switched on and once with it switched off. You will measure exactly what the thinking cost you in tokens, in money, and in answer quality.

What You'll Build

You will finish with two drawings open side by side in your browser and a table on your terminal that tells you which one cost more to make.

By the end of this project, you'll have:

  • A script that calls one model on OpenRouter twice with different reasoning settings, so you can rerun the comparison against any model you are curious about.
  • Two SVG files you open in your browser and compare, showing you what the model drew when it was thinking and when it was not.
  • A printed token and cost table you can check against your OpenRouter activity page, so the numbers are yours and not someone else's blog post.
  • Secret Mission: Run the same comparison across a second model on OpenRouter and add it to your table, so you can see whether thinking is worth paying for on more than one model.

Are there any prerequisites?

You need an OpenRouter account with a small amount of credit on it, plus any language that can make an HTTP POST.

There is nothing to install and no model to download. The model runs on someone else's hardware.

Before We Start

Reasoning modes get sold as a straight upgrade. The only honest way to find out what one costs you is to measure the same prompt twice, once with thinking on and once with it off.

The model in this project has 770 billion parameters, so running it on your laptop is off the table. OpenRouter serves it over a normal HTTP API, so a key and a script are all you need.

In this step, get ready to:
  • Create an OpenRouter account and add credits to it.
  • Generate an API key and store it somewhere safe.
  • Read the published per-token input and output prices for your model.
Create your OpenRouter account

OpenRouter bills per token, so an empty account cannot run a single call. Adding credits asks for card details, and the handful of short calls in this project costs small change.

  • Open https://openrouter.ai/ in your browser.
  • Sign up for an account using whichever sign-in option you prefer.
  • Sign in if you are not signed in automatically.
  • Add credits to your account so your calls have something to spend against.
  • Check the credit balance shown in your account.

You should see the amount you just added sitting as your balance.

Why pay before writing any code?

Every model on the service is priced per token, and the balance is what a request draws down. No balance means the call fails before the model ever sees your prompt.

The upside is that the spend is measurable. By the end of the project you read the exact figure for each call off your own activity page.

Generate your API key

Your script has to prove it is you on every single call. An API key is that proof, a long string that starts with sk-or- and stands in for your account.

Your key is shown once

The page shows you the full key at the moment it is created and never again. Copy it before you close that dialog, because the only fix afterwards is deleting the key and making a new one.

Keep the value out of screenshots, chat messages and your code. Anyone holding it can spend your credits.

  • Open the keys page at https://openrouter.ai/settings/keys.
  • Click the button on that page that creates a new key.
  • Name the key reasoning-cost so you can recognise it later.
  • Copy the key into your password manager or another safe place.
  • Close the dialog showing the key.
  • Find your new key in the list on the keys page.

You should see it listed under the name you gave it, with most of the value hidden behind dots.

Closed the dialog before copying?

The key cannot be shown again, so delete the one you just made and create a second key with the same name.

If the new key still will not appear in the list, why isn't my new OpenRouter key showing up?

Read the model's per-token price

Every call in this project bills against two separate rates. Input tokens are what you send, and output tokens are what the model writes back.

Before you open the page, which of those two do you think carries the higher price per token?

  • Open the model page at https://openrouter.ai/tencent/hy4-preview.
  • Record the model slug shown on the page, in provider/model form: your model slug.
  • Record the published input price: your input price per million tokens.
  • Record the published output price: your output price per million tokens.

On most models the output rate is the higher of the two, and a reasoning trace is output. That gap is exactly where thinking mode's price comes from.

Model page not loading?

Preview models get renamed or pulled without warning, so a saved link can go stale. Browse the catalogue at https://openrouter.ai/models and search for the model there.

Always take the slug from the page you land on rather than typing it from memory. If nothing close comes up, help me pick a replacement model with a reasoning switch.

You now have three things written down: the slug, the input price and the output price. Those are the numbers your script will be checked against.

Your account is funded, your key is safe, and you know what each token costs. Next up, you'll write the script that spends them.

Call the Model With Reasoning On

You have an OpenRouter key and you have read the model's published per-token price off its page. A price per million tokens tells you nothing until you know how many tokens one real answer actually burns.

Reasoning models spend tokens thinking before they answer, and you pay for every one of those tokens. This step sends your first request with the reasoning setting left at its default, so you can watch that hidden thinking arrive alongside the answer.

In this step, get ready to:
  • Check Python is ready and create your project folder.
  • Write a script that asks your model to draw something as an SVG.
  • Print the reply, the reasoning trace, and the raw response in your terminal.
Check Python and set up your project folder

OpenRouter is a plain HTTP API, so anything that can send a POST request will do. Python with one small library is the shortest route from your terminal to a reply.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Check which Python you have by running this command:
python3 --version

✔️ I see a version number

Python is ready. Carry on below to make your project folder.

ⓧ Command not found

  • On Windows, run python --version instead. The Windows installer registers python, not python3.
  • If that also fails, open python.org/downloads in your browser.
  • Click the download button for your operating system.
  • Run the installer and accept the defaults.
  • Close your terminal, reopen it, then run the version check again.

Windows uses a shorter command

Every command in this step starts with python3. On Windows, type python instead each time.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir reasoning-cost
cd reasoning-cost
  • Install the one library your script needs by running this command:
python3 -m pip install requests
  • Read the last line of the output. You should see a line beginning with Successfully installed, or a note that the requirement is already satisfied.

Install failed?

  • A permissions error usually means Python is installed system-wide. Add --user to the end of the install command and run it again.
  • A network timeout usually means a proxy or firewall is in the way. Retry on a different connection.
  • Still stuck? Why won't pip install requests on my machine?
Point the script at your model and your key
  • Create your script file by running this command:
touch compare_reasoning.py
  • Confirm the file exists by running:
ls
  • You should see compare_reasoning.py listed in the output.
  • Open compare_reasoning.py in your text editor.

Your key authenticates every request, so the script needs to reach it. Pasting it into the file means one careless commit leaks it, so hand it over through the terminal instead.

macOS

  • Load your key into this terminal session by running this command, swapping in the key you generated:
export OPENROUTER_API_KEY=your-openrouter-key-here

Windows

  • Load your key into this PowerShell session by running this command, swapping in the key you generated:
$env:OPENROUTER_API_KEY="your-openrouter-key-here"
  • In compare_reasoning.py, write the script's setup by copying in this code:
import json
import os

import requests

# Read the key from the environment so it never sits inside this file.
API_KEY = os.environ["OPENROUTER_API_KEY"]

# The slug shown on the model page you opened in the last step.
MODEL = "[[MODEL_SLUG="the model slug from the model page"]]"

# An SVG is a drawing written as text, so you can see how hard the model tried.
PROMPT = "Draw a pelican riding a bicycle as an SVG. Reply with the SVG markup only."

print(f"Calling {MODEL} with reasoning left at its default...")

What does this code do?

  • os.environ pulls the key out of the terminal session, so the file stays safe to share.
  • MODEL holds the exact slug printed on the model page. Preview models get renamed and retired, so copy it rather than typing it from memory.
  • PROMPT asks for a drawing because a picture makes the quality difference obvious. A paragraph of prose would not.
  • Save compare_reasoning.py.
  • Back in your terminal, run the script:
python3 compare_reasoning.py
  • You should see one line naming your model, ending in with reasoning left at its default....

Seeing a KeyError instead?

  • A KeyError means the terminal never received your key. Run the export command again in the same terminal window you are running the script from.
  • Opening a new terminal tab wipes the variable. Set it again after every new tab.
  • Still stuck? Why can't my Python script read my environment variable?
Send the request and print the reasoning trace

OpenRouter takes one POST request and hands back one JSON document. Leaving the reasoning setting out of the request is what keeps it at its default, which for this model means thinking is on.

  • Add the request to the bottom of compare_reasoning.py by copying in this code:
# One chat completion request, with the reasoning setting left alone.
response = requests.post(
    url="https://openrouter.ai/api/v1/chat/completions",
    headers={
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json",
    },
    data=json.dumps({
        "model": MODEL,
        "messages": [{"role": "user", "content": PROMPT}],
    }),
)
print("HTTP status:", response.status_code)

What does this request do?

  • The Authorization header carries your key as a bearer token. Without it every call comes back rejected.
  • messages is a list because the API expects a whole conversation. Yours is one turn long.
  • There is no reasoning field anywhere in this body. That absence is the point. You are measuring the default.
  • Save compare_reasoning.py.
  • Run the script again:
python3 compare_reasoning.py
  • You should see a new line reading HTTP status: 200 after a pause of a few seconds.

That pause is the model thinking. You are paying for it, and you cannot see any of it yet.

Not seeing 200?

  • A 401 means the key was rejected. Check you pasted the whole key, with no trailing space.
  • A 402 means the account has no credits. Add credits in your OpenRouter dashboard, then run the script again.
  • A 404 usually means the slug is wrong. Copy it again from the model page, character for character.

The answer and the thinking come back in two separate fields. A script that grabs only content silently throws the reasoning away, which is exactly the thing you are here to measure.

  • Add the printing to the bottom of compare_reasoning.py by copying in this code:
# The whole reply, including token counts, arrives as one JSON document.
result = response.json()
message = result["choices"][0]["message"]

# The thinking lives in its own field, separate from the answer.
print("\n--- REASONING TRACE ---")
print(message.get("reasoning"))

print("\n--- REPLY ---")
print(message["content"])

# The raw document carries the token counts you will price up later.
print("\n--- RAW RESPONSE ---")
print(json.dumps(result, indent=2))

Why the trace and the reply print separately

OpenRouter returns reasoning tokens in the reasoning field of each message, and the final answer in content.

Both are billed as output tokens. Printing the raw document too means the token counts are already on screen when you need them.

  • Save compare_reasoning.py.

Before you run this, make a guess. Do you think the reasoning trace will be shorter or longer than the drawing the model finally produces?

  • Run the script and find out:
python3 compare_reasoning.py
  • You should see three labelled sections scroll past: a block of the model talking itself through the drawing under --- REASONING TRACE ---, then <svg markup under --- REPLY ---, then the full JSON.

Reasoning trace printing as None?

  • Scroll into the raw JSON and look inside message to see which fields the provider actually returned.
  • Some providers return the thinking as reasoning_details instead. If that is what you see, print that key rather than reasoning.
  • Still stuck? Why is my OpenRouter reasoning field empty?

✔️ Awesome, I've got everything!

Great, double check you've saved compare_reasoning.py.

ⓧ I'd like to double check the full code

Scroll up and confirm all three blocks are in your file, in this order:

  • The setup block: the imports, API_KEY, MODEL, PROMPT, and the opening print.
  • The request block: requests.post and the HTTP status: print.
  • The printing block: result, message, and the three labelled sections.

You have watched the model think out loud and paid for the privilege. Next up, you'll send the same prompt with reasoning switched off and see what disappears.

Turn Reasoning Off and Resend

Your script already gets the model to think out loud. You watched the trace scroll past, then the answer arrive.

A trace on its own tells you nothing about what that thinking cost you. The only way to price it is to run the identical prompt with the thinking switched off.

That is all this step does. Same model, same prompt, one flag flipped.

In this step, get ready to:
  • Wrap your request in a function you can call more than once.
  • Send the same prompt a second time with reasoning turned off.
  • Compare the two replies in your terminal.
Wrap your request in a function

Right now your script runs top to bottom and sends exactly one request. Sending a second one means either pasting the whole block again, or giving it a name you can call twice.

  • Open compare_reasoning.py and put your cursor just above the lines that send your request.
  • Give your request a reusable name by pasting in this function:
def send_request(reasoning=None):
    # The parts that never change between calls: same model, same prompt
    payload = {
        "model": MODEL,
        "messages": [{"role": "user", "content": PROMPT}],
    }
    # Only add the reasoning field when asked, so the default stays untouched
    if reasoning is not None:
        payload["reasoning"] = reasoning
    response = requests.post(
        "https://openrouter.ai/api/v1/chat/completions",
        headers={"Authorization": f"Bearer {API_KEY}"},
        json=payload,
    )
    # Hand the whole parsed reply back so the caller can read any part of it
    return response.json()

What does this function do?

  • payload holds the parts that stay identical across both calls, using the MODEL and PROMPT variables you already defined at the top of the file.
  • reasoning=None leaves the reasoning field out of the body entirely, so the model falls back to its own default.
  • Passing a dictionary instead adds that field to the body, which is how you take control of the setting.
  • response.json() returns the whole parsed JSON reply, so the caller can read the text, the trace, or the token counts.
  • Scroll down to the lines that post your request and parse the reply, just below your new function.
  • Find this code:
response = requests.post(
    "https://openrouter.ai/api/v1/chat/completions",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={"model": MODEL, "messages": [{"role": "user", "content": PROMPT}]},
)
data = response.json()
  • Delete every line from the requests.post( call down to the line that parses the reply into a dictionary.
  • Replace them with this single call, so that part of the file now looks like this:
data = send_request()
  • Save compare_reasoning.py.
  • Confirm nothing broke by running the script the same way you ran it last step:
python3 compare_reasoning.py

You get the same reply text, the same reasoning trace and the same raw response as before. The function is doing the work now.

Seeing a NameError?

  • Check that def send_request sits above the line that calls it. Python reads the file top to bottom.
  • Check that MODEL, PROMPT and API_KEY match the names you used at the top of your file.
  • Check that every line inside the function is indented by four spaces.

Why is my Python script raising a NameError after I moved code into a function?

Send the same prompt with reasoning switched off

OpenRouter accepts a reasoning object on the request body. Setting enabled to false inside it tells the model to answer without thinking first.

  • Scroll to the very bottom of compare_reasoning.py, below your existing print lines.
  • Send the second call by pasting this code there:
print("=== reasoning off ===")
# Same model and same prompt, with reasoning explicitly switched off
off = send_request({"enabled": False})
off_message = off["choices"][0]["message"]
print(off_message["content"])
# Prints None when the model sends back no reasoning trace
print(off_message.get("reasoning"))
print(json.dumps(off, indent=2))

What does this code do?

  • send_request({"enabled": False}) adds a reasoning field to the body with enabled set to false.
  • The === reasoning off === line splits your terminal output in two. Everything above it came from the first call.
  • off_message.get("reasoning") uses .get rather than square brackets, so a missing trace prints None instead of crashing the script.
  • json.dumps(off, indent=2) prints the full raw reply, which is where the token counts live for the step after next.
  • Save compare_reasoning.py.
  • Before you run it, decide which of the two replies you expect to be longer, and what you expect the second call's trace line to print.
  • Send both calls by running:
python3 compare_reasoning.py

The first call prints its reply and its trace as before. Below === reasoning off === you get a shorter reply and a bare None where the trace used to be.

That None is your proof. The same model answered the same question without thinking about it first.

Second call came back with an error instead of a reply?

  • The reasoning setting is model specific, so a model that has no thinking mode can reject the field rather than ignoring it. Check the raw reply your script printed for an error object in place of choices.
  • Check that you passed False with a capital F, since Python is case-sensitive and lowercase false is not defined.
  • Check that your account still has credits, because an empty balance fails the second call even when the first one succeeded.

Help me read the error my OpenRouter request came back with.

✔️ Awesome, I've got everything!

Great, double check you've saved compare_reasoning.py.

ⓧ I'd like to double check the full code

Scroll up through compare_reasoning.py and confirm these four pieces are present, in this order:

  • Your original imports and the API_KEY, MODEL and PROMPT variables, unchanged.
  • The send_request function you pasted in first.
  • The single line data = send_request(), where your old request block used to be, followed by your original print lines.
  • The === reasoning off === block at the very bottom of the file.

Two replies from one model, one thinking and one not. Next up, you'll save each one as an .svg file and open both drawings side by side.

Save and Render Both Drawings

You now have two replies from the same model, one with reasoning on and one with it off. Both are sitting in your terminal as raw markup, scrolled past and gone the next time you run the script.

Markup is hard to judge by eye. In this step you'll pull the SVG out of each reply, save it to its own file, and let your browser draw both so the difference is something you can actually look at.

In this step, get ready to:
  • Pull the SVG markup out of each reply.
  • Save each drawing as its own .svg file.
  • Open both files in your browser and see how they differ.
Save each reply as its own SVG file

The model's reply is a block of text with drawing markup somewhere inside it, usually surrounded by chatter. A browser only renders the markup, so the job is to find where it starts and where it stops.

  • Switch back to compare_reasoning.py in your editor.
  • Add a helper that finds the markup and writes it to a file by pasting this at the bottom of the file:
def save_svg(reply_text, filename):
    # Find where the SVG markup starts and ends inside the reply
    start = reply_text.find("<svg")
    end = reply_text.find("</svg>")
    if start == -1 or end == -1:
        print("No SVG markup found for " + filename)
        return

    # Keep everything from the opening tag to the end of the closing tag
    svg = reply_text[start:end + len("</svg>")]
    with open(filename, "w") as f:
        f.write(svg)

    print("Wrote " + filename + " with " + str(len(svg)) + " characters")

# Save each reply to its own file
save_svg(reasoning_on_text, "reasoning-on.svg")
save_svg(reasoning_off_text, "reasoning-off.svg")

What does this code do?

  • find("<svg") locates where the drawing begins inside the reply, ignoring anything the model said before it.
  • The if guard prints a warning instead of writing an empty file when a reply came back with no markup at all.
  • The two calls at the bottom run the same helper twice, so each reply lands in its own file with a name that says which setting produced it.
  • Swap reasoning_on_text and reasoning_off_text for whatever your two reply-text variables are called.
  • Save compare_reasoning.py.
  • Send both calls again and write the files by running this command:
python3 compare_reasoning.py
  • Look at the last two lines of the output. You should see Wrote reasoning-on.svg and Wrote reasoning-off.svg, each with a character count.
  • Confirm both files landed on disk by running:
ls

The listing now includes reasoning-on.svg and reasoning-off.svg next to your script.

Seeing "No SVG markup found"?

  • Check that you passed the variable holding the reply text, not the whole response object. find only works on a string.
  • Scroll up to the printed reply. If the model answered in words with no markup, run the script again so it gets another go at the prompt.
  • Still stuck? Why is my script not finding the SVG in the reply?

✔️ Awesome, I've got everything!

Great, double check you've saved compare_reasoning.py before moving on.

ⓧ I'd like to double check the full code

Scroll through compare_reasoning.py and confirm each piece is present, in this order:

  • The code that loads your API key.
  • The model name and the prompt asking for a drawing.
  • The first request, with the reasoning setting left at its default.
  • The second request, with reasoning explicitly turned off.
  • The save_svg helper you just added.
  • The two save_svg calls at the very bottom.
Open both drawings in your browser

Both files are still just text until something renders them. Your browser is the fastest renderer you already have.

Before you open them, make a call: which drawing do you expect to have more parts in it, the reasoning-on one or the reasoning-off one?

macOS

  • Open both drawings at once by running this command:
open reasoning-on.svg reasoning-off.svg

Windows

  • Open the reasoning-on drawing by running this command:
start reasoning-on.svg
  • Open the reasoning-off drawing by running this command:
start reasoning-off.svg

Both drawings appear, each in its own tab or window. Flick between them and you'll see the same subject drawn two different ways.

Opened in something other than a browser?

Your machine may hand .svg files to an image viewer or an editor instead.

  • Drag each file from your file browser onto an open browser window to render it.
  • Seeing a blank page instead of a drawing? Why is my SVG file rendering blank in the browser?
Compare the two drawings side by side
  • Drag one browser tab out into its own window.
  • Place the two windows side by side on your screen.
  • Count the distinct shapes in each drawing.
  • Check whether any labels or text appear in one drawing and not the other.
  • Look at whether the parts sit in sensible proportion in each drawing.

Whichever version came out richer, you're looking at the difference reasoning made on a single prompt. That difference is the thing you're about to put a price on.

You can see the difference now. Next up, you'll put a number on it by reading the token count and cost off each reply.

Print the Cost Comparison Table

Your two drawings are open side by side in the browser, and the difference between thinking and not thinking is right there on screen.

Better output is only half the question. The other half is what that thinking cost you.

Every reply from OpenRouter carries the token counts and the price of that call in the same JSON your script already parses. You will pull those numbers out and lay both calls side by side.

In this step, get ready to:
  • Read the token counts and cost OpenRouter returns for each call.
  • Print both calls as a side by side table.
  • Match your table against your OpenRouter activity page.
Read the usage block OpenRouter sends back

Every reply carries a usage object alongside the text you already use. It holds the token counts and the price OpenRouter charged for that one call.

The code below uses on_response and off_response for the two replies your script already stores. Swap in your own names if you called them something else.

  • Return to compare_reasoning.py in your editor.
  • Print the usage block for both calls by adding these lines at the very bottom of the file, below the code that writes reasoning-off.svg:
# The usage block OpenRouter returns alongside every reply
print("reasoning on:", on_response["usage"])
print("reasoning off:", off_response["usage"])

What's in the usage block?

  • prompt_tokens counts everything you sent, including your drawing instructions.
  • completion_tokens counts everything the model wrote back.
  • completion_tokens_details holds reasoning_tokens, the thinking the model did before it answered.
  • cost is what OpenRouter charged your account in credits for that single call.

Before you run this, which of the two calls do you think burned more tokens, and by how much?

  • Save compare_reasoning.py.
  • Send both calls again and print their usage blocks by running this command in your terminal:
python3 compare_reasoning.py

You should see two lines print after the script finishes, each one a dictionary of token counts ending in a cost figure.

Seeing a KeyError?

  • Check that on_response and off_response match the names your script uses for the two replies.
  • On Windows, run python compare_reasoning.py if python3 isn't recognised.

Why is my Python script raising a KeyError?

Print the two calls as a table

Two raw dictionaries make you hunt for the numbers that differ. A table puts the same fields on the same line so the gap jumps straight out.

  • In compare_reasoning.py, find the three lines you just added at the bottom:
# The usage block OpenRouter returns alongside every reply
print("reasoning on:", on_response["usage"])
print("reasoning off:", off_response["usage"])
  • Turn those raw dictionaries into a table by replacing all three lines with this code:
def usage_row(label, response):
    # Pull the token counts and the price out of one reply
    usage = response["usage"]
    # reasoning_tokens only appears when the model actually thought
    details = usage.get("completion_tokens_details") or {}
    return {
        "label": label,
        "prompt": usage["prompt_tokens"],
        "completion": usage["completion_tokens"],
        "reasoning": details.get("reasoning_tokens", 0),
        "cost": usage.get("cost", 0),
    }

rows = [usage_row("reasoning on", on_response), usage_row("reasoning off", off_response)]

# Fixed column widths keep the numbers lined up under their headings
print(f"{'call':<14}{'prompt':>8}{'completion':>12}{'reasoning':>11}{'cost':>12}")
for row in rows:
    print(f"{row['label']:<14}{row['prompt']:>8}{row['completion']:>12}{row['reasoning']:>11}{row['cost']:>12.6f}")

What does this code do?

  • usage_row turns one reply into a single row of five numbers.
  • details.get("reasoning_tokens", 0) returns 0 for the reasoning-off call, where that field never arrives.
  • The widths inside each f-string hold every column under its own heading.

Before you run this, how much more do you think the thinking call cost? Twice as much, or ten times?

  • Save compare_reasoning.py.
  • Print your comparison table by running this command:
python3 compare_reasoning.py

You should see a five column table with one row for each call.

The reasoning-on row carries a reasoning count above zero. The reasoning-off row shows 0, and the gap between the two cost figures is the price of thinking.

Columns not lining up?

  • Check that both print lines use the same five widths, otherwise the headings sit over the wrong numbers.
  • Check that usage_row sits above the rows line, since Python reads the file top to bottom.

Why is my formatted table printing out of alignment?

✔️ Awesome, I've got everything!

Great, double check you've saved compare_reasoning.py.

ⓧ I'd like to double check the full code

Scroll up and confirm each block is present in compare_reasoning.py, in this order:

  • Everything from the earlier steps, ending with the code that writes reasoning-off.svg.
  • The usage_row function.
  • The rows line that builds one row per call.
  • The heading print line and the for loop that prints each row.
Match your table against the OpenRouter activity page

Your table is your own arithmetic, run on your own machine. OpenRouter recorded the same two calls on its side, and the two records should agree.

Every run of the script is two more calls. If you ran it three times, the dashboard holds six entries and the newest two are the ones in your table.

Before you open it, do you expect the dashboard's figures to match your table exactly, or to be rounded off?

  • Open the Activity dashboard at openrouter.ai/activity in your browser.
  • Click through a chart or table row to drill down to the underlying request logs.
  • Find the two newest entries, one for each call your script just made.
  • Compare their token counts and cost against the two rows your table printed.

The two entries on the dashboard carry the same numbers your script printed. Your table is measuring what OpenRouter actually billed you, not an estimate.

Don't see your two calls?

  • Refresh the page after a few seconds, since entries take a moment to land.
  • Check you are signed in to the same OpenRouter account that owns the API key your script uses.

Why don't my API calls show up on my OpenRouter activity page?

Secret mission

Sweep Three Prompts for the Thinking Premium

One drawing told you what thinking cost. It did not tell you whether thinking was worth it. Run the same on and off comparison across three prompts of rising difficulty, then read the premium off a single table.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. Nothing in this project runs in the background, so your only ongoing exposure is your OpenRouter API key, which can spend credits every time something calls it.

Resources you used:

  • Your OpenRouter API key, still active and able to spend credits.
  • compare_reasoning.py, the script that sends both calls.
  • reasoning-on.svg and reasoning-off.svg, the two saved drawings.

Keep everything running

No action needed. Choose this if you're still actively comparing models and want to keep running your script.

  • Leave your API key active so you can rerun compare_reasoning.py against another model.
  • Check your OpenRouter activity page after each rerun, because every call adds another charge.

Your OpenRouter account stays open either way. The account itself costs nothing to hold.

Pause - I'll come back to this later

Delete the key but keep the files. Choose this if you want the script for later without leaving a live credential lying around.

  • Open the API keys page in your OpenRouter dashboard, the same list where your key appeared after you generated it.
  • Find the key you used for this project in that list.
  • Delete that key from the list.
  • Keep compare_reasoning.py, reasoning-on.svg and reasoning-off.svg on disk.

When you come back, generate a fresh key and load it into compare_reasoning.py the same way you did the first one.

Delete - I don't want to use this again

Remove the key and the project files. Choose this if you don't plan to run the comparison again.

  • Open the API keys page in your OpenRouter dashboard, the same list where your key appeared after you generated it.
  • Find the key you used for this project in that list.
  • Delete that key from the list.
  • In the terminal you used to run your script, confirm you're in the folder holding your project files by running this command:
ls

You should see compare_reasoning.py, reasoning-on.svg and reasoning-off.svg listed.

  • Delete the three files on macOS or Linux by running this command:
rm compare_reasoning.py reasoning-on.svg reasoning-off.svg
  • Delete the three files on Windows by running this command in PowerShell instead:
Remove-Item compare_reasoning.py, reasoning-on.svg, reasoning-off.svg
  • Run ls one more time.

None of the three files appear in the output. Everything this project wrote to disk is gone.

Your OpenRouter account stays open. Holding the account costs nothing once the key is deleted.

Nice Work!

Nice Work

Brilliant work. You just turned a vague claim about reasoning models into two SVG drawings, a token table, and a receipt on your OpenRouter activity page that backs up every number.

You've learned how to:

  • Call a hosted model over a plain HTTP API with your own key, so a 770B parameter model runs for you without a single GPU on your desk.
  • Flip a model's reasoning setting between on and off, then compare the two replies as reasoning-on.svg and reasoning-off.svg side by side in your browser.
  • Read token usage and cost straight off each response and print a comparison table, then match those figures against the OpenRouter activity page.
  • Secret Mission: You took on the optional challenge and pushed compare_reasoning.py further than the main path required.

The best part is that compare_reasoning.py isn't tied to one model. Swap the model slug and you have a repeatable way to ask whether thinking was worth paying for.

Ready to quiz yourself?