Ask Questions About Screen Recordings

Turn a screen recording into a clickable, timestamped AI-generated timeline.

Introduction

30 Second Summary

Screen recordings often hold the answer to what happened. Finding one moment can mean scrubbing through the whole clip.

In this project, you will send a short screen recording to a hosted vision model through OpenRouter. You will turn its answer into structured JSON and display the events as a clickable timeline beside the video.

What You'll Build

You will click a timestamped event beside your recording to jump straight to the matching moment.

By the end of this project, you'll have:

  • A working model connection that returns a text answer in your terminal.
  • A visual question-answering workflow that describes what appears in a screenshot.
  • A clickable recording timeline that jumps to the matching moment for each structured event label.
  • Secret Mission: An optional challenge to push your video analysis skills further.

Are there any prerequisites?

You'll need an OpenRouter account with an API key. A screen recorder plus any HTTP client completes your setup.

Before We Start

Your screen-recording project eventually sends media to OpenRouter. Before that happens, your laptop needs a dependable local toolkit.

One missing command can stop the workflow before the first request leaves your machine. In this step, you will confirm the local tools that support every later check.

In this step, get ready to:
  • Confirm that Python 3 runs from your terminal.
  • Check that your code editor is ready.
  • Test a screen recorder with a short privacy-safe clip.
Confirm Python 3 in your terminal

Python runs the scripts in this project. The terminal gives you a direct way to prove that the interpreter is available.

macOS

  • Press Cmd+Space to open macOS search.
  • Search for the terminal application.
  • Launch the terminal result.
  • Check your Python interpreter by running this command:
python3 --version

What does this command do?

  • The python3 command invokes the installed Python interpreter on macOS.
  • The --version option prints the installed version.

You should see Python followed by a version number that begins with 3.

Windows

  • Press the Windows key to open Windows search.
  • Search for the terminal application.
  • Launch the terminal result.
  • Check your Python interpreter by running this command:
py --version

What does this command do?

  • The py command invokes the installed Python interpreter on Windows.
  • The --version option prints the installed version.

You should see Python followed by a version number that begins with 3.

  • Choose the outcome below that matches your terminal.

✔️ Python is available

That check gives your project a working Python interpreter. Your terminal is ready to run the scripts you build later.

ⓧ Python is unavailable

  • Visit the official Python downloads page.
  • Choose the installer for your operating system.
  • Run the downloaded installer.
  • Approve any installation permission request from your operating system.
  • Return to the terminal window from earlier.
  • Repeat the version check for your operating system.

Python Still Unavailable?

Restart your terminal after installation so it can detect the new command.

Check that you downloaded the installer for your operating system.

Help me get Python 3 working in my terminal.

Confirm your editing and browsing tools

Your code editor is where you will write the script and the timeline page. Your browser gives you a place to inspect saved output later.

  • Press Cmd+Space on macOS or the Windows key on Windows to open search.
  • Search for the code editor you plan to use.
  • Launch the editor from the search results.
  • Confirm that the editor can display a blank workspace.
  • Return to the browser displaying this project.
  • Open the official Python downloads page in a new tab.

You should now have an editor workspace open. Your browser should display the Python download options in its new tab.

No Code Editor Available?

Use the plain-text editor included with your operating system. Confirm that it can save files without rich-text formatting.

Help me choose a code editor for this project.

Test your screen recorder

A screen recording can capture notifications or personal tabs. A blank desktop keeps this test safe.

  • Minimize any windows that show private information.
  • Use your operating system's search to find your screen-recording tool.
  • Launch the screen-recording tool from the search results.
  • Choose a screen or window capture mode.
  • Start a test recording.
  • Move your pointer in a small circle for five seconds.
  • Stop the recording.

Before playback, do you expect the pointer to trace the same small circle?

  • Play the test recording.

You should see the pointer trace the circle you made. The duration should be about five seconds.

✔️ The Test Clip Plays

Your recorder is ready for the real clip. You have proved that it can capture movement within the one-minute limit.

ⓧ I Cannot Record or Play the Clip

  • Confirm that the recorder has permission to capture your screen.
  • Choose a screen-recording tool that supports screen or window capture.
  • Install the tool from its official source if your laptop has no recorder.
  • Repeat the five-second test.

Recorder Still Not Working?

Screen-capture permissions can block recording even when the tool opens correctly. Review your operating system's privacy settings for the recorder.

Help me set up screen recording on my laptop.

  • Delete the five-second test clip.

Your local toolkit is ready. Next up, you will create an OpenRouter key and send a text-only request.

Send Your First Model Request

Your screen recording will eventually travel to a hosted vision model. The connection needs a safe identity before any pixels leave your laptop.

In this step, you will create an OpenRouter API key. You will store it in an environment variable before sending a text-only request from Python.

In this step, get ready to:
  • Generate an OpenRouter API key.
  • Store the key outside your Python code.
  • Send a text-only request from analyze_recording.py.
Create your OpenRouter API key

An API key acts like a password for your requests. OpenRouter shows its plaintext value when the key is created.

  • In your browser from earlier, go to the OpenRouter API Keys page.
  • Complete the sign-up or login flow to reach the API Keys page.
  • Create a new API key named screen-recording-project.
  • Copy the key when OpenRouter displays it.

That is the sensitive part done. You now have a credential ready for your script.

Store the key outside your code

An environment variable keeps the key in your terminal session. Your script can read the value without placing the secret inside analyze_recording.py.

  • Use the tab for your operating system to store the key in your current terminal session.

macOS

  • Replace your-api-key-here with the key you copied.
  • Store the key in your terminal session by running this command:
export OPENROUTER_API_KEY="your-api-key-here"

What does this command do?

  • The export command makes the key available to programs started from this terminal.
  • The value remains available for the current terminal session.

Windows

  • Replace your-api-key-here with the key you copied.
  • Store the key in your PowerShell session by running this command:
$Env:OPENROUTER_API_KEY = "your-api-key-here"

What does this command do?

  • The $Env: prefix stores the key as an environment variable in PowerShell.
  • The value remains available for the current PowerShell session.
  • Keep this terminal session open for the rest of the step.

Key not available to Python?

  • Use the same terminal session where you stored OPENROUTER_API_KEY.
  • Check that you replaced your-api-key-here with the complete key.
  • Help me check why Python cannot read my OpenRouter environment variable.
Send the text-only request

The first request uses text only. This isolates the connection before you add screenshots or video in later steps.

  • Build the request by pasting this code into a new file in your code editor:
import json
import os
import urllib.request

# Read the key from the terminal so it stays out of the source file
api_key = os.environ["OPENROUTER_API_KEY"]

# Define the model and the text-only question
payload = {
    "model": "qwen/qwen2.5-vl-32b-instruct:free",
    "messages": [
        {"role": "user", "content": "Reply with one short sentence confirming you can read this message."}
    ],
}

# Build an authenticated JSON request
request = urllib.request.Request(
    "https://openrouter.ai/api/v1/chat/completions",
    data=json.dumps(payload).encode("utf-8"),
    headers={
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json",
    },
    method="POST",
)

# Send the request, read the JSON response, and print the assistant reply
with urllib.request.urlopen(request) as response:
    data = json.load(response)
print(data["choices"][0]["message"]["content"])

What does this script do?

  • The os.environ lookup reads your key from the current terminal session.
  • The payload selects qwen/qwen2.5-vl-32b-instruct:free.
  • The messages list contains one text-only user prompt.
  • The request sends JSON to https://openrouter.ai/api/v1/chat/completions with your key in the authorization header.
  • The final line extracts data["choices"][0]["message"]["content"] so the terminal displays only the assistant reply.
  • Save the new file as analyze_recording.py on your Desktop.

✔️ Awesome, I've got everything!

Great. Double-check that analyze_recording.py is saved on your Desktop.

ⓧ I'd like to double check the full code

Compare your saved file with this complete version.

import json
import os
import urllib.request

# Read the key from the terminal so it stays out of the source file
api_key = os.environ["OPENROUTER_API_KEY"]

# Define the model and the text-only question
payload = {
    "model": "qwen/qwen2.5-vl-32b-instruct:free",
    "messages": [
        {"role": "user", "content": "Reply with one short sentence confirming you can read this message."}
    ],
}

# Build an authenticated JSON request
request = urllib.request.Request(
    "https://openrouter.ai/api/v1/chat/completions",
    data=json.dumps(payload).encode("utf-8"),
    headers={
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json",
    },
    method="POST",
)

# Send the request, read the JSON response, and print the assistant reply
with urllib.request.urlopen(request) as response:
    data = json.load(response)
print(data["choices"][0]["message"]["content"])

Before you run the script, do you expect a short confirmation or the full raw response?

macOS

  • Move to your Desktop and send the request by running these commands:
cd ~/Desktop
python3 analyze_recording.py

What should I see?

  • The first command selects your Desktop.
  • The second command runs analyze_recording.py.
  • You should see one short sentence confirming that the model read your message.

Windows

  • Move to your Desktop and send the request by running these commands:
Set-Location -Path ~\Desktop
py analyze_recording.py

What should I see?

  • The first command selects your Desktop.
  • The second command runs analyze_recording.py.
  • You should see one short sentence confirming that the model read your message.

That first reply proves the connection works from end to end. Your laptop can now reach the model without placing the key inside the script.

Request not returning a reply?

  • Check that you ran the script in the same terminal session where you stored the key.
  • Generate a new key if OpenRouter rejects the credential.
  • Wait briefly before retrying if the free model route is busy.
  • Help me troubleshoot my first OpenRouter request.

Your first hosted model request is working. Next up, you will replace the text-only input with a screenshot.

Ask About a Screenshot

Your first request proved that OpenRouter can reach the model from your script. A text reply cannot show whether the model can inspect pixels.

This step adds a multimodal prompt containing a question plus an image. Your script will encode a local screenshot so the hosted model can describe what appears on screen.

In this step, get ready to:
  • Capture a local screenshot with a clear interface to inspect.
  • Encode the screenshot as a base64 data URL.
  • Send the screenshot to the vision model and read its description.
Capture a screenshot

A screenshot gives you a result that you can check with your own eyes. Pick a screen with several visible details so the model has something useful to describe.

Choose a Safe Screen

A screenshot can contain names or account details. A neutral app window keeps this test free of private information.

  • Return to an app window that contains no credentials or private information.
  • Arrange the window so one clear screen state is visible.
  • Use your computer's screenshot control to capture that window.
  • Save the capture as screenshot.png in the folder that already contains analyze_recording.py.
  • Return to the code editor from earlier.
  • Confirm that screenshot.png appears beside analyze_recording.py in the file list.

You now have a local image that can reveal whether the model understands the same screen details you see.

Teach the script to package an image

A local image must be converted into text-safe base64 data before it can travel inside a JSON request. The build_content() helper keeps the original text-only path while adding an image path.

  • In analyze_recording.py, select the existing import lines.
  • Replace that selection with the code below:
# Load modules for image encoding, environment access, and HTTP requests.
import base64
import os
import requests

def build_content(prompt, image_path=None):
    # Keep text-only requests compatible with the original script.
    if image_path is None:
        return prompt

    # Convert the local PNG bytes into an API-ready data URL.
    with open(image_path, "rb") as image_file:
        encoded_image = base64.b64encode(image_file.read()).decode("utf-8")

    data_url = f"data:image/png;base64,{encoded_image}"

    # Put the question first so the model knows what to inspect.
    return [
        {"type": "text", "text": prompt},
        {"type": "image_url", "image_url": {"url": data_url}},
    ]

What Does This Code Do?

  • The image_path parameter decides which request shape the helper returns.
  • Calling build_content() without an image returns the original plain prompt.
  • Providing screenshot.png converts its binary bytes into base64 text.
  • The returned content array holds the question first. The image data URL follows it.
  • Save analyze_recording.py.
  • Confirm that the original text-only request still runs by using this command:
python3 analyze_recording.py

What Should I See?

You should see another assistant reply without a Python traceback. This confirms that the helper preserved the existing text-only path.

Seeing a Python Error?

  • Check that each line inside build_content() uses consistent indentation.
  • Confirm that the original request code still sits below the new helper.
  • Compare the three import names with the code block above.
  • Ask for help with the helper function.
Send the screenshot question

A multimodal message represents its content as typed parts. One part carries your question. The second part carries the base64 data URL for screenshot.png.

  • In analyze_recording.py, select everything below build_content().
  • Replace that selection with the code below:
# Keep the API destination, model, question, and local image in one place.
API_URL = "https://openrouter.ai/api/v1/chat/completions"
MODEL = "qwen/qwen2.5-vl-32b-instruct:free"
QUESTION = "What appears on this screen? Describe the visible interface and text."
IMAGE_PATH = "screenshot.png"

# Read the stored key without placing the secret in source code.
api_key = os.environ["OPENROUTER_API_KEY"]

# Send the question and encoded screenshot as one user message.
payload = {
    "model": MODEL,
    "messages": [
        {"role": "user", "content": build_content(QUESTION, IMAGE_PATH)}
    ],
}
response = requests.post(
    API_URL,
    headers={"Authorization": f"Bearer {api_key}"},
    json=payload,
)
response.raise_for_status()

# Print only the assistant's description from the response.
result = response.json()
print(result["choices"][0]["message"]["content"])

How Does the Screenshot Reach the Model?

  • The QUESTION value tells the model which details to report.
  • The IMAGE_PATH value points to the local screenshot beside the script.
  • The messages array combines the question with the encoded image.
  • The final line extracts the assistant's description from the API response.
  • Save analyze_recording.py.

Before you run it, do you expect the reply to mention details that only appear inside your screenshot?

  • Send the screenshot question by running this command:
python3 analyze_recording.py

What Should I See?

You should see a description of the interface in screenshot.png. The response should mention visible controls or text from the captured screen.

That is your first visual result. Your script can now turn a local screen into a readable description.

No Screenshot Description?

  • Confirm that screenshot.png sits beside analyze_recording.py.
  • Check that the saved image uses the .png file extension.
  • Confirm that IMAGE_PATH contains the exact filename screenshot.png.
  • Ask for help with the multimodal request.

✔️ Awesome, I've got everything!

Great. Double-check that you saved analyze_recording.py beside screenshot.png.

ⓧ I'd like to double check the full code

Compare your complete analyze_recording.py file with this version.

# Load modules for image encoding, environment access, and HTTP requests.
import base64
import os
import requests

def build_content(prompt, image_path=None):
    # Keep text-only requests compatible with the original script.
    if image_path is None:
        return prompt

    # Convert the local PNG bytes into an API-ready data URL.
    with open(image_path, "rb") as image_file:
        encoded_image = base64.b64encode(image_file.read()).decode("utf-8")

    data_url = f"data:image/png;base64,{encoded_image}"

    # Put the question first so the model knows what to inspect.
    return [
        {"type": "text", "text": prompt},
        {"type": "image_url", "image_url": {"url": data_url}},
    ]

# Keep the API destination, model, question, and local image in one place.
API_URL = "https://openrouter.ai/api/v1/chat/completions"
MODEL = "qwen/qwen2.5-vl-32b-instruct:free"
QUESTION = "What appears on this screen? Describe the visible interface and text."
IMAGE_PATH = "screenshot.png"

# Read the stored key without placing the secret in source code.
api_key = os.environ["OPENROUTER_API_KEY"]

# Send the question and encoded screenshot as one user message.
payload = {
    "model": MODEL,
    "messages": [
        {"role": "user", "content": build_content(QUESTION, IMAGE_PATH)}
    ],
}
response = requests.post(
    API_URL,
    headers={"Authorization": f"Bearer {api_key}"},
    json=payload,
)
response.raise_for_status()

# Print only the assistant's description from the response.
result = response.json()
print(result["choices"][0]["message"]["content"])

Your script can now inspect one still image through the same API connection. Next up, you will send a short screen recording and ask the model to list what happens over time.

Analyze a Screen Recording

Your screenshot test proved that the vision model can describe one frozen moment from your screen. A recording adds a sequence of visible actions.

In this step, you will capture a short app session. You will send it through OpenRouter with the question list what happens so the terminal can show the events in order.

In this step, get ready to:
  • Record a screen clip under one minute that shows you using an app.
  • Extend analyze_recording.py with video input.
  • Send the recording with the question list what happens.
Record a short clip

Screen recordings can capture notifications or private data. A clean clip keeps the test focused while protecting your privacy.

  • Switch back to the screen-recording tool from earlier.
  • Choose an app session that contains no private information.
  • Start your screen recording.
  • Use the app for less than one minute.
  • Stop your screen recording.
  • Save the clip as recording.mp4 in the same folder as analyze_recording.py.
  • Check that folder in your code editor.

You should see recording.mp4 beside screenshot.png and analyze_recording.py.

Extend the script for video

The screenshot path already turns local media into a Base64 data URL. Video follows the same pattern with an MP4 data URL.

OpenRouter expects the video inside a video_url content item. The API sends that item beside the text question in one user message.

  • Switch back to analyze_recording.py in your code editor.
  • Add a third input mode that reads recording.mp4 as binary data.
  • Encode the video bytes with Base64.
  • Build the MP4 data URL by placing data:video/mp4;base64, before the encoded text.
  • Place the completed data URL inside a video_url content item.
  • Set the video mode text prompt to list what happens.
  • Select the new video mode as the active input.
  • Save analyze_recording.py.

How Does the Video Path Work?

Base64 converts the local video bytes into text that fits inside the JSON request. The MP4 prefix tells the receiving model how to interpret those bytes.

The text item supplies the question. The video_url item supplies the recording that the question refers to.

✔️ My video mode is ready

Your script now has a video path beside the existing text-only path and screenshot path.

ⓧ I'd like to double check the full code

  • Confirm the script still reads the locally stored API key.
  • Confirm the existing text-only path remains available.
  • Confirm the existing screenshot.png path remains available.
  • Confirm the new path reads recording.mp4 as binary data.
  • Confirm the new path converts the recording into a Base64 MP4 data URL.
  • Confirm the video request includes the prompt list what happens.
  • Confirm the returned assistant message still prints in the terminal.
Inspect the recording response

This request tests whether the model can follow an app session across multiple frames. The returned list should reflect the sequence in your clip.

Before you run it, do you think the model will list the actions in the same order as your recording?

  • Return to the terminal from earlier.
  • Run analyze_recording.py with the same Python command you used earlier.

You should see a returned list of events that follows the visible order of your screen recording.

Video Request Not Returning a List?

  • Confirm recording.mp4 sits in the same folder as analyze_recording.py.
  • Confirm the recording uses the .mp4 file extension.
  • Confirm the clip lasts less than one minute.
  • Pause before another attempt if OpenRouter reports a rate-limit problem.
  • Help me troubleshoot why my OpenRouter video request is not returning an event list.

That is the key jump completed. Your script can now turn a moving app session into a readable event list.

Your recording now comes back as readable events. Next, you will make those events predictable by saving each one as structured JSON.

Save a Timestamped Timeline

Your recording already produces a readable list of events. A web page needs a predictable structure before it can turn those events into clickable timestamps.

You will tighten the existing OpenRouter video prompt so every event follows the same shape.

Python will validate the reply as JSON before saving it to timeline.json.

In this step, get ready to:
  • Request one timestamped label for each video event.
  • Validate the model response before saving it.
  • Open the saved timeline to confirm its structure.
Request a structured event array

The previous prompt allowed the model to choose its own response format. A precise structure gives every event a numeric time plus a short description.

  • Switch back to analyze_recording.py in your code editor.
  • Replace the video question with one instruction that requests only valid JSON as an array. Require one object per event. Require a numeric timestamp in seconds. Require a string label. Exclude surrounding prose plus code fences.
  • Save analyze_recording.py.

Before you run the script, do you think the reply will begin with an array or include extra explanation?

  • Run analyze_recording.py using the same terminal command from the previous step.

You should see an array of event objects in the terminal. Each object should contain a numeric timestamp plus a string label.

Why Use One Object per Event?

Each object keeps one timestamp connected to one label. The next step can render every object as a separate timeline entry.

Seeing Extra Text Around the Array?

  • Strengthen the prompt so it requests only the JSON array.
  • Remove any wording that invites the model to explain its answer.
  • Run the script again after saving the updated prompt.
  • Help me make my video-analysis prompt return only valid JSON.
Validate and save the response

A response can look structured while still containing invalid JSON syntax. Parsing catches that problem before the result reaches your file.

  • Add Python's JSON handling to analyze_recording.py with json.loads() to validate the assistant message before json.dump() writes the parsed array to timeline.json.
  • Save analyze_recording.py.

The script uploads recording.mp4 again. Expect this run to take about as long as your previous video request.

Before you run the script, which file do you expect to appear after the response passes validation?

  • Run analyze_recording.py again using the same terminal command.

You should see timeline.json appear beside your existing project files. Its presence proves the model reply passed JSON validation.

Timeline File Missing?

  • Check that json.loads() receives the assistant message content.
  • Check that json.dump() runs only after parsing succeeds.
  • Inspect the terminal response for surrounding explanation or code fences.
  • Help me find why timeline.json was not created.
Confirm the timeline shape

Valid JSON still needs the expected outer array plus the correct fields. This final check confirms the saved data is ready for the timeline page.

Before you open the file, what structure do you expect to surround the event objects?

  • Select timeline.json from your code editor's file sidebar.
  • Confirm the top level is an array.
  • Check that every event has a numeric timestamp value.
  • Check that every event has a string label value.

That is the structured-data step complete. Your recording now has a saved timeline that another program can read.

✔️ Awesome, I've got everything!

Great. Your script now validates the video analysis before saving a timestamped event array to timeline.json.

ⓧ I'd like to double check the full code

Compare your project against this end state:

  • Confirm analyze_recording.py sends recording.mp4 through the existing OpenRouter video request.
  • Confirm the prompt requests a JSON array containing one object per event.
  • Confirm every event requires a numeric timestamp plus a string label.
  • Confirm the script parses the returned JSON before writing it to timeline.json.

Your video events are now structured and saved. Next up, you will place those labels beside the recording and make each timestamp clickable.

Build an Interactive Timeline

You now have valid JSON containing timestamped events from your recording. A data file still makes it awkward to compare each label with the footage.

In this step, you will turn that data into an interactive HTML page. A small amount of JavaScript makes each timestamp jump the video to its matching moment.

In this step, get ready to:
  • Create an HTML page that places the recording beside a timeline.
  • Render the events from timeline.json as timestamp controls.
  • Test two timestamp jumps against the recording.
Create the video layout

The page needs a video player plus a dedicated space for the model's event labels. A flexible layout keeps both parts visible for quick comparisons.

  • Use your code editor's new-file control to create index.html in the same local folder as recording.mp4.
  • Build the player layout in index.html by pasting this code:
<!doctype html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Screen Recording Timeline</title>
  <style>
    /* Keep the recording beside its event controls */
    main { display: flex; gap: 2rem; align-items: flex-start; }
    video { width: 60%; }
    #timeline button { display: block; margin-bottom: 0.75rem; }
  </style>
</head>
<body>
  <h1>Screen Recording Timeline</h1>
  <main>
    <!-- Load the recording analyzed by the model -->
    <video id="recording" controls src="recording.mp4"></video>
    <section>
      <h2>Timeline</h2>
      <div id="timeline">Loading timeline...</div>
    </section>
  </main>
</body>
</html>

What does this page do?

  • The video element loads recording.mp4 with the browser's playback controls.
  • The timeline container reserves a space for the event labels from timeline.json.
  • The flexible layout places the recording beside its timeline.
  • Save index.html.
  • Return to the terminal from earlier.
  • Serve the folder containing your project files with a local HTTP server by running this command:
python -m http.server
  • Leave the terminal running while you test the page.
  • Switch back to the browser tab from earlier.
  • Enter http://localhost:8000/ in the address bar.

You'll see the Screen Recording Timeline heading with your video beside a timeline area. Your first visual slice is live. The recording now has a page built around it.

Page or video not loading?

  • Confirm index.html is in the same folder as recording.mp4.
  • Make sure the terminal command is running from that same folder.
  • Use the same Python executable that worked in the earlier steps if the server command is unavailable.
  • Help me troubleshoot my local HTML page and video.
Load the timeline data

The Fetch API reads timeline.json through the local server. The script turns every event object into a clickable button.

  • In index.html, find the closing </main> line.
  • Add this script below </main> before the closing </body> line:
<script>
  const video = document.getElementById("recording");
  const timeline = document.getElementById("timeline");

  // Load the model events and turn each one into a timeline control
  fetch("timeline.json")
    .then((response) => response.json())
    .then((events) => {
      timeline.textContent = "";
      events.forEach((event) => {
        const button = document.createElement("button");
        button.textContent = `${event.timestamp}s: ${event.label}`;

        // Seek to this event before starting or resuming playback
        button.addEventListener("click", () => {
          video.currentTime = event.timestamp;
          video.play();
        });

        timeline.appendChild(button);
      });
    });
</script>

What does this script do?

  • The video and timeline constants connect the script to the matching elements on the page.
  • The fetch() call loads the event array from timeline.json.
  • Each button displays the event's numeric timestamp with its label.
  • A click sets currentTime to the event timestamp. The play() call starts or resumes the recording from that position.
  • Save index.html.
  • Refresh http://localhost:8000/ in your browser.

You'll see the loading message replaced by one timestamped button per event in timeline.json. That is the data bridge working. Your model response is now a usable interface.

Timeline still says loading?

  • Confirm timeline.json is in the same folder as index.html.
  • Check that the terminal running the local server remains active.
  • Compare the opening and closing braces in your script with the code above.
  • Help me find why my timeline buttons are not rendering.

✔️ Awesome, I've got everything!

Great. Double-check that index.html is saved beside recording.mp4 and timeline.json.

ⓧ I'd like to double check the full code

Compare your saved index.html with this complete file.

<!doctype html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <title>Screen Recording Timeline</title>
  <style>
    /* Keep the recording beside its event controls */
    main { display: flex; gap: 2rem; align-items: flex-start; }
    video { width: 60%; }
    #timeline button { display: block; margin-bottom: 0.75rem; }
  </style>
</head>
<body>
  <h1>Screen Recording Timeline</h1>
  <main>
    <!-- Load the recording analyzed by the model -->
    <video id="recording" controls src="recording.mp4"></video>
    <section>
      <h2>Timeline</h2>
      <div id="timeline">Loading timeline...</div>
    </section>
  </main>
  <script>
    const video = document.getElementById("recording");
    const timeline = document.getElementById("timeline");

    // Load the model events and turn each one into a timeline control
    fetch("timeline.json")
      .then((response) => response.json())
      .then((events) => {
        timeline.textContent = "";
        events.forEach((event) => {
          const button = document.createElement("button");
          button.textContent = `${event.timestamp}s: ${event.label}`;

          // Seek to this event before starting or resuming playback
          button.addEventListener("click", () => {
            video.currentTime = event.timestamp;
            video.play();
          });

          timeline.appendChild(button);
        });
      });
  </script>
</body>
</html>
Test two timestamp jumps

Model-generated timestamps are estimates. Spot checking two clear events tells you whether the timeline matches what actually happened in the recording.

Before you click the first timestamp, where do you expect the video playhead to land?

  • Choose a label that describes a clear action near the beginning of the recording.
  • Click that label's timestamp.

You'll see the video jump to the event's timestamp. Playback starts or resumes from that point.

  • Choose a second label that describes a clear action later in the recording.
  • Click the second label's timestamp.

You'll see the video jump to the second event. Compare the visible action with the label beside the player.

Does a timestamp miss the event?

  • Pause the recording at the event's actual starting point.
  • Read the player's current time.
  • Update the corresponding numeric timestamp in timeline.json.
  • Refresh the page before testing that label again.
  • Help me correct an inaccurate timeline timestamp.

You have turned an unstructured screen recording into a clickable timeline. Each tested label now leads straight to the moment it describes.

Secret mission

Calibrate Every Timeline Timestamp

Audit every model-generated timestamp against your recording. Correct any drift in the timeline data. Prove that every label lands on the event it describes.

Clean Up Your Resources

Clean Up Your Resources

The project leaves no cloud resource running, so keeping its local files creates no ongoing cost. Your clip and screenshot may contain private details, so choose whether to keep them, pause for later, or delete them entirely.

Resources you used:

  • Input files recording.mp4 and screenshot.png.
  • Analysis script analyze_recording.py and generated timeline timeline.json.
  • Timeline page index.html.
  • Local OPENROUTER_API_KEY entry used by the analysis script.

Keep everything running

No action is needed. Choose this if you want to ask more questions about the recording or refine the timeline later.

  • Keep recording.mp4, timeline.json, and index.html together so the timeline page can load both files.
  • Keep analyze_recording.py if you want to analyze another short recording.
  • Review recording.mp4 and screenshot.png before sharing the project files with anyone.
  • Keep the local OPENROUTER_API_KEY entry only if you plan to make more model requests.

Pause - I'll come back to this later

Close the local page to free up your current workspace. There is no background server or cloud process to stop.

  • Close the browser tab displaying index.html.
  • Close the terminal when you have finished using the analysis script.
  • Keep screenshot.png, recording.mp4, analyze_recording.py, timeline.json, and index.html in their current folder.
  • Resume later by opening index.html in your browser.

Delete - I don't want to use this again

Remove the local project files if you want to start fresh. Deleting the clip and screenshot becomes permanent after you empty the Trash or Recycle Bin, so save another copy first if you want to preserve them.

  • Close the browser tab displaying index.html.

macOS

  • In Finder, return to the folder containing the project files.
  • Select screenshot.png, recording.mp4, analyze_recording.py, timeline.json, and index.html.
  • Move the selected files to the Trash.
  • Empty the Trash to remove the files permanently.
  • Confirm the five filenames no longer appear in their original folder.

Windows

  • In File Explorer, return to the folder containing the project files.
  • Select screenshot.png, recording.mp4, analyze_recording.py, timeline.json, and index.html.
  • Press Delete to move the selected files to the Recycle Bin.
  • Empty the Recycle Bin to remove the files permanently.
  • Confirm the five filenames no longer appear in their original folder.

Remove your local credential:

  • Return to the local file or shell setting where you stored OPENROUTER_API_KEY earlier.
  • Remove the OPENROUTER_API_KEY entry from that storage location.
  • Save the local configuration file if you edited one.
  • Close the terminal to end its current shell session.
  • Confirm the OPENROUTER_API_KEY entry no longer appears in the same local storage location.

Nice Work!

Nice Work!

You did it! You built a screen-recording analyzer that turns a short video into an interactive timeline using a hosted vision model through OpenRouter.

You've learned how to:

  • Send text, images, and video to a hosted vision model through OpenRouter and read its responses in your terminal.
  • Turn an unstructured screen recording into structured JSON containing numeric timestamps and event labels.
  • Build an interactive HTML timeline that plays recording.mp4 beside the events from timeline.json. You also verified two timestamps by jumping to their expected moments.
  • Extended the project through an optional Secret Mission to push your video-analysis skills further.

Ready to quiz yourself?