Extract Chart Data into a Spreadsheet

Send a chart screenshot to an AI model and extract the values into a CSV.

Introduction

30 Second Summary

You open a report or a slide deck and there it is: a chart with exactly the numbers you need, locked inside a picture. Nobody publishes the spreadsheet behind it.

In this project, you will build a script that sends a chart screenshot to a multimodal model and gets the underlying values back as structured data. You will write those values into a CSV you can open in a spreadsheet.

What You'll Build

You will drop a screenshot of a chart onto your script, watch numbers come back, and open a spreadsheet holding the data that used to live only as pixels.

By the end of this project, you'll have:

  • A working extractor script you point at any chart image, which returns the values as JSON you could paste straight into code.
  • A CSV file holding the numbers from three different charts, which you can open in a spreadsheet and check row by row against the original images.
  • A documented failure case: a chart the model reads wrong, a note on exactly what it got wrong, and the one prompt line you added to try to fix it.
  • Secret Mission: An optional challenge that extends your extractor beyond the main path.

Are there any prerequisites?

You need a laptop, an OpenRouter account with an API key, and a small credit balance. The model page lists $0.15 input and $0.47 output per million tokens, so running a handful of charts costs a fraction of a cent.

Before We Start

Before you touch a single tool, lock in what you're about to build and why it matters to you.

Send a Chart Screenshot With a Plain Prompt

Your OpenRouter key is live and you have credits sitting behind it. Now you get to spend a fraction of a cent finding out what it can do.

A chart in a slide deck is just pixels. The numbers are visible to you and invisible to your spreadsheet.

In this step you write a short script that hands a chart screenshot to qwen3.8-flash and asks it a plain question. Then you look hard at what comes back.

In this step, get ready to:
  • Set up a project folder with a chart screenshot to test against.
  • Install the client library and load your API key into your terminal session.
  • Send the chart to the model with a plain-language question and read the reply.
Set up your project folder and test chart

The script you are about to write runs on Python, so the first job is confirming Python is on your machine.

  • Check whether Python is installed by running this command:
python3 --version

On Windows, run py --version instead.

✔️ I see a version number

Python is ready. Carry on below to build your project folder.

ⓧ Command not found

Python is not installed yet. Install it, then come back and run the version check again.

  • Open the official Python downloads page.
  • Download the installer for your operating system.
  • Run the installer and accept the default options.
  • Close your terminal and open a fresh one, so it picks up the new install.
  • Run the version check again to confirm it worked.

Still seeing command not found?

The installer usually offers a checkbox to add Python to your system path. If you skipped it, rerun the installer and tick it.

My terminal still cannot find Python after installing it. What should I check?

macOS

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir chart-extractor
cd chart-extractor
  • Create the empty script file by running this command:
touch extract_chart.py

Windows

  • Move to your Desktop by running this command:
cd ~\Desktop
  • Create your project folder and move into it by running these commands:
mkdir chart-extractor
cd chart-extractor
  • Create the empty script file by running this command:
New-Item extract_chart.py
  • Find a bar chart or line chart with clear axis labels in a report, a slide deck, or a news article.
  • Press Cmd+Shift+4 (macOS) or Win+Shift+S (Windows) to start a screenshot.
  • Drag a box around the chart to capture it.
  • Rename the saved image to chart1.png.
  • Move chart1.png into the chart-extractor folder on your Desktop.
  • Confirm both files are in place by running this command:
ls

You should see chart1.png and extract_chart.py listed.

Pick a chart you can check by eye

A chart with five to ten bars and readable axis labels is ideal for a first run. You need to be able to verify the model's answer yourself.

Crop tightly around the plot area so the axis labels stay sharp. A blurry label is a number the model has to guess at.

Your files are not showing up?

Screenshots usually land on the Desktop rather than inside a folder, so check whether your image is still sitting one level up.

Why does my terminal not list the files I just created?

Install the OpenRouter client and load your key

Installing packages straight into your system Python causes conflicts across projects. A virtual environment keeps this project's packages in their own box.

macOS

  • Create and switch on your virtual environment by running these commands:
python3 -m venv venv
source venv/bin/activate

Windows

  • Create and switch on your virtual environment by running these commands:
py -m venv venv
venv\Scripts\activate

From here on, use python wherever the commands below say python3.

Your terminal prompt now starts with (venv). That prefix is how you know the environment is active.

  • Install the client library by running this command:
pip install openai

The last line of the output confirms the install and names the version of openai that was fetched.

Why an OpenAI package talks to OpenRouter

OpenRouter serves an OpenAI-compatible HTTP API. Pointing the openai package at a different base address is all it takes to reach a completely different catalogue of models.

That buys you one request format for every model you might try. Swapping models later is a one-line change.

Your key belongs in your terminal session, not in the script file. That way it never ends up in a file you might share.

macOS

  • Load your key into this terminal session by running this command, swapping your-api-key-here for the key from your dashboard:
export OPENROUTER_API_KEY="your-api-key-here"

Windows

  • Load your key into this terminal session by running this command, swapping your-api-key-here for the key from your dashboard:
$env:OPENROUTER_API_KEY="your-api-key-here"
  • Confirm the key landed by running this command:
python3 -c "import os; print('Key loaded:', bool(os.environ.get('OPENROUTER_API_KEY')))"

You should see Key loaded: True. This prints only whether a key is present, never the key itself.

Seeing False instead of True?

The key lives in one terminal session only. If you opened a new tab or window after setting it, run the export command again in that window.

A stray space around the equals sign also stops the variable being set, so check the command matches exactly.

Why is my environment variable not sticking in my terminal?

Ask the model what the chart shows
  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Select your chart-extractor folder, then click Open.
  • Open extract_chart.py from the Explorer sidebar.
  • Connect to OpenRouter and load your chart into memory by pasting this into the empty extract_chart.py:
import base64
import os
from openai import OpenAI

# Point the OpenAI client at OpenRouter instead of OpenAI
client = OpenAI(
    base_url="https://openrouter.ai/api/v1",
    api_key=os.environ["OPENROUTER_API_KEY"],
)

# Read the chart image and turn its bytes into base64 text
with open("chart1.png", "rb") as image_file:
    image_data = base64.b64encode(image_file.read()).decode("utf-8")

print("Encoded image length:", len(image_data))

What does this code do?

  • base_url sends every request to OpenRouter, so the same library reaches a different set of models.
  • os.environ pulls your key out of the terminal session, which keeps it out of the file.
  • base64.b64encode turns the raw image bytes into plain text, because a JSON request body cannot carry binary.
  • Save extract_chart.py.
  • Check that your chart loads by running this command in your terminal:
python3 extract_chart.py

You should see Encoded image length: followed by a number in the tens of thousands. That number is your chart, rewritten as text.

Python cannot find your chart?

The script looks for chart1.png in the folder you run it from. Run ls and confirm the image is listed alongside the script.

If the file is a JPEG rather than a PNG, rename it so the extension matches what the script opens.

My script cannot find my image file. How do I debug this?

  • In extract_chart.py, find this line at the bottom of the file:
print("Encoded image length:", len(image_data))
  • Replace that line with the request below, which sends the image and a plain-language question to the model:
# Wrap the base64 text in a data URL so the API reads it as a PNG
data_url = f"data:image/png;base64,{image_data}"

# Send the chart and a plain-language question in a single message
response = client.chat.completions.create(
    model="qwen/qwen3.8-flash",
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "What are the values shown in this chart?"},
                {"type": "image_url", "image_url": {"url": data_url}},
            ],
        }
    ],
)

# Print the model's reply exactly as it arrives
print(response.choices[0].message.content)

What's happening here?

  • model names which of OpenRouter's models handles the request.
  • content carries two parts in one message: your question as text, and the chart as an image.
  • The data:image/png;base64, prefix tells the API the long string that follows is a PNG, not an address to fetch.
  • response.choices[0].message.content is the model's reply, untouched.

Before you run this, what shape do you think the answer arrives in? Something you could drop straight into a spreadsheet, or something else?

  • Save extract_chart.py.
  • Send your chart to the model by running this command:
python3 extract_chart.py

After a few seconds you get paragraphs of English describing your chart. The numbers are in there, wrapped in sentences and softened with words like roughly and approximately.

Why this reply is hard to use

The model read your chart correctly. That part works, and it is the hard part.

The problem is the packaging. To get these numbers into a spreadsheet you would have to write code that hunts through prose, and the prose changes shape every single run.

This is the expected result, not a mistake on your part. You asked an open question, so you got an open answer.

The request came back with an error?

An authentication error means the key in your terminal session does not match the one on your dashboard. Run the export command again with a fresh copy of the key.

A billing error means your credit balance has not landed yet. Check the balance on your OpenRouter dashboard before retrying.

My OpenRouter request is returning an error. Help me read it.

✔️ Awesome, I've got everything!

Great, double check you've saved extract_chart.py and move on.

ⓧ I'd like to double check the full code

Scroll up and confirm your file contains these two blocks, in this order:

  • The imports, the client setup, and the with open("chart1.png", "rb") block that builds image_data.
  • The data_url line, the client.chat.completions.create call, and the final print.
  • Nothing else. The old Encoded image length line should be gone.

The model can read your chart. It just answers in the shape you asked for, and you asked for a conversation. Next up, you rewrite the prompt to pin down the exact shape you want back.

Get a Structured JSON Response

Last step you got an answer out of the model, and it came back as a paragraph of prose. Reading it yourself was easy. Getting a script to read it was not.

A spreadsheet needs rows and columns, not sentences. In this step you tell the model the exact shape you want back, so the reply arrives as JSON your script can parse.

In this step, get ready to:
  • Rewrite your prompt so it pins down the exact JSON shape you want back.
  • Run the same chart image again and read a machine-readable reply.
  • Parse that reply and print each label and value from your script.
Rewrite your prompt to pin down the JSON shape

Models answer the question you actually asked. A loose question invites a paragraph, which is exactly what came back.

  • Return to extract_chart.py in your editor.
  • Find the line that sets the prompt text:
prompt = "What are the values shown in this chart?"
  • Replace that line with this prompt, which spells out the exact shape you want back:
prompt = """Read the values from this chart.
Reply with JSON only, with no explanation, in exactly this shape:
{"data": [{"label": "string", "value": number}]}
One object per bar or point, in the order they appear."""

Why spell out the shape?

Showing the model a literal example of the structure gives it a target to copy. It has to fill in your keys rather than invent its own wording.

Naming the fields also fixes what each number means. A label is always the axis text and a value is always the number, in every reply.

  • Save extract_chart.py.
Run the same chart through your new prompt

Same image, same model, different question. Before you run it, what do you think comes back this time?

  • Send the same chart through the new prompt by running this command:
python3 extract_chart.py

You should see a block of JSON instead of a sentence. Each entry in the data list carries a label read off the axis and the number beside it.

Still seeing a sentence?

Check that you saved extract_chart.py before rerunning. An unsaved edit sends the old prompt straight back out.

Check that the whole prompt sits between the opening and closing """ markers. A missing closing marker swallows the rest of your instructions.

Why is my model still replying in prose instead of JSON?

Parse the reply into values your script can use

The reply is still just text that happens to look like JSON. Your script cannot loop over it until you turn it into real Python data.

  • At the top of extract_chart.py, add this import below the imports already there:
import json
  • Find the line at the bottom of the file that prints the model's reply:
print(reply)
  • Turn that reply into data you can loop over by adding this code directly below that line:
# Turn the model's JSON text into a Python dictionary
chart = json.loads(reply)

# Print one line per label and value pair the model found
for point in chart["data"]:
    print(point["label"], point["value"])

What does this code do?

  • json.loads() reads the model's text and hands back a Python dictionary.
  • chart["data"] is the list of points you asked for in your prompt.
  • The loop prints one pair per line, so you can check each number against the image.

Before you run it, how many printed lines do you expect to see?

  • Save extract_chart.py.
  • See your parsed values by running this command:
python3 extract_chart.py

Below the raw JSON, you now get one clean line per bar or point. That is one line for every label on your chart.

Script stopping with a decoding error?

Look at the raw reply printed just above the error. Models sometimes wrap their JSON in a code fence, and those extra characters break the parser.

Add a line to your prompt telling the model to reply with raw JSON and no code fences, then save and rerun.

Help me read this JSON parsing error in my Python script.

✔️ Awesome, I've got everything!

Great. Double check you have saved extract_chart.py before moving on.

ⓧ I'd like to double check the full code

Scroll up and confirm each piece is in your file, in this order:

  • import json sits with the other imports at the top.
  • The prompt spells out the {"data": [{"label": "string", "value": number}]} shape inside triple quotes.
  • print(reply) is still there, near the bottom.
  • json.loads(reply) sits directly below it and stores the result in chart.
  • The loop over chart["data"] closes out the file.

Your script now returns data instead of sentences. Next up, you'll push three different charts through it and collect every result in one CSV.

Extract Multiple Charts Into One CSV

Your prompt now pulls clean JSON back out of a single chart. One chart is a demo.

A real report has dozens of charts, and the numbers only become useful once they sit together somewhere you can sort and total them. In this step you point the script at three different charts and collect every extracted row in one CSV file.

In this step, get ready to:
  • Capture two more chart screenshots to test the script on.
  • Loop the script over all three images in a single run.
  • Collect every extracted row in one results.csv and check it against the images.
Capture two more chart screenshots

One chart tells you the script works on one chart. Three charts of different shapes tell you where it holds up and where it starts to wobble.

  • Find a second chart with a different shape to chart1.png, such as a line chart in a news article.
  • Capture it the same way you captured chart1.png earlier.
  • Save it as chart2.png in the same folder as extract_chart.py.
  • Capture a third chart, such as a pie chart or a scatter plot.
  • Save the third chart as chart3.png in that same folder.
  • Confirm both screenshots landed next to your script by running this command:
ls

You should see chart1.png, chart2.png, chart3.png and extract_chart.py listed together.

Don't see your new screenshots?

Your screenshot tool probably saved to its own default location instead of your project folder. Move the two files across, then run the command again.

Check the file names too. A file saved as Chart2.png will not match the lowercase name your script asks for.

My screenshots aren't showing up in my project folder

Loop the script over all three images

Right now your script names one file and stops. A loop hands the same request one image after another, so you never touch the filename again.

  • In extract_chart.py, add these lines directly above the code that reads your image file:
# Every chart image the script should work through
IMAGE_PATHS = ["chart1.png", "chart2.png", "chart3.png"]

for image_path in IMAGE_PATHS:
    print("Extracting", image_path)
  • Select every line below the print line and press Tab to indent it by four spaces.
  • Swap the hardcoded chart1.png filename inside the loop for image_path.

What does this loop do?

  • IMAGE_PATHS holds the three file names as a plain Python list.
  • for image_path in IMAGE_PATHS: runs everything indented beneath it once per file.
  • Indenting your existing code under the loop means one request goes to OpenRouter for each chart instead of one in total.
  • Save extract_chart.py.
  • Run all three charts through the model by running this command:
python3 extract_chart.py

You should see an Extracting line for each of your three images, each one followed by the values qwen3.8-flash pulled out of that chart.

Seeing an IndentationError?

Python is telling you some lines below the for line are still sitting at the left margin. Every line that belongs to the loop needs the same four spaces of indent.

On Windows, try python extract_chart.py if python3 is not recognised.

Why is my Python loop throwing an indentation error?

Append every row to results.csv

Terminal output scrolls away the moment you close the window. A CSV file keeps the numbers, opens in any spreadsheet, and lets you put all three charts side by side.

  • In extract_chart.py, add this directly below your existing import lines at the top of the file:
import csv
import os

CSV_PATH = "results.csv"

# Create the file with a header row the first time the script runs
if not os.path.exists(CSV_PATH):
    with open(CSV_PATH, "w", newline="") as f:
        csv.writer(f).writerow(["source_image", "label", "value"])

What does this code do?

  • CSV_PATH keeps the output file name in one place, so the rest of the script never spells it out again.
  • os.path.exists(CSV_PATH) checks whether the file is already there.
  • The header row only gets written when the file is missing, so rerunning the script never buries a second header in the middle of your data.
  • Save extract_chart.py.
  • Create the file by running your script again:
python3 extract_chart.py
  • Show what is inside the new file by running this command:
cat results.csv

You should see exactly one line: source_image,label,value. The columns are ready and waiting for rows.

  • Fill those columns by adding this inside the loop, directly below the line where you parse the JSON reply:
    # Save one row per label/value pair, tagged with the image it came from
    with open(CSV_PATH, "a", newline="") as f:
        writer = csv.writer(f)
        for item in chart_data["data"]:
            writer.writerow([image_path, item["label"], item["value"]])

What does this code do?

  • The a in open(CSV_PATH, "a", newline="") means append, so each chart adds to the file instead of wiping it.
  • chart_data["data"] is the list of label and value pairs from your parsed reply. If you named your parsed variable something else, swap that name in here.
  • Writing image_path into every row is what lets you tell three charts apart once they share a file.
  • Save extract_chart.py.

Before you run this, how many rows do you expect in results.csv? Count the bars or points across your three charts and make a guess.

  • Run all three charts into the file by running this command:
python3 extract_chart.py
  • Show the filled file by running this command:
cat results.csv

Below the header you should now see one row per data point, each one starting with chart1.png, chart2.png or chart3.png.

No rows below the header?

A NameError usually means your parsed reply is not called chart_data. Use whatever name you gave it when you parsed the JSON.

A KeyError means the reply came back in a different shape. Print the parsed reply and check whether the pairs really sit under a data key.

Why isn't my script writing rows to the CSV?

✔️ Awesome, I've got everything!

Great, double check you've saved extract_chart.py.

ⓧ I'd like to double check the full code

Scroll up and confirm all three additions from this step are in your file, in this order:

  • The csv and os imports, CSV_PATH, and the header row block, all near the top of the file.
  • The IMAGE_PATHS list and the for line, with your original request code indented beneath them.
  • The append block that writes one row per label and value pair, indented inside the loop.
  • Open results.csv in your spreadsheet application by double clicking it in Finder (macOS) or File Explorer (Windows).
  • Open chart1.png next to the spreadsheet and compare its rows against the chart by eye.
  • Repeat that comparison for chart2.png.
  • Repeat it once more for chart3.png.

Most labels and values should line up with what you can read off the images. Any that drift are worth remembering, because that is what the next step digs into.

Three charts, one file, numbers you can sort and total. Next up, you'll hand the script a chart it gets wrong and find out exactly where it breaks.

Secret mission

Find the Chart That Breaks It

Your extractor has only met charts it can read. Feed it a stacked bar or a log-scale chart, watch it hand back confident nonsense, then add one line to your prompt and see whether it recovers. The note you write about that failure is the part worth keeping.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. Nothing here runs in the background, so your only ongoing exposure is the OpenRouter key sitting live on your account.

Resources you used:

  • The OpenRouter API key you created on your account dashboard.
  • Your project folder holding extract_chart.py, results.csv and failure_notes.md.
  • The chart screenshots chart1.png, chart2.png, chart3.png and chart4_failing.png.

Keep everything running

No action needed. Choose this if you're still actively running charts through your extractor.

  • Leave your API key active so the script keeps authenticating.
  • Keep failure_notes.md somewhere you'll find it again.
  • Leave any unused credit on your account for your next batch of charts.

Your notes outlast the model

Model IDs change fast, and this family moves faster than most. The record of which chart type your prompt misread is the part that stays useful when you swap the model out.

Pause - I'll come back to this later

Shut off anything that could spend credit while keeping your work intact. Choose this if you want to return to the extractor later.

  • Sign in to OpenRouter and open the API keys page you used to create your key in Step 1.
  • Delete the key you created for this project.
  • Leave your project folder and all its files untouched on disk.

Creating a fresh key takes seconds, so pausing costs you nothing but a paste into extract_chart.py when you return.

Delete - I don't want to use this again

Remove everything this project created and start fresh. Choose this if you don't plan to come back.

  • Sign in to OpenRouter and open the API keys page you used to create your key in Step 1.
  • Delete the key you created for this project.
  • Remove your project folder and every file inside it by running this command:
rm -rf [[PROJECT_FOLDER="path to your project folder"]]
  • On Windows, remove the same folder by running this command instead:
rmdir /s /q [[PROJECT_FOLDER="path to your project folder"]]

That takes the script, the screenshots, the CSV and the notes with it.

Your remaining credit stays put

Deleting the key doesn't remove the balance you topped up. The credit stays on your OpenRouter account, and with no live key there's nothing that can spend it.

Nice Work!

Nice Work

Nice work! You've built a script that turns a chart screenshot into structured rows inside results.csv.

You also pushed it until it broke and wrote down why. That note is the part most people skip.

You've learned how to:

  • Turn a chart screenshot into numbers by sending the image to qwen3.8-flash through OpenRouter.
  • Pin a prompt to an exact JSON shape so the reply comes back parseable instead of prose.
  • Find the chart that fools the model, then record the exact failure in failure_notes.md before trying a fix.
  • Secret Mission: An optional challenge that takes your extractor further.

Ready to quiz yourself?