HN Job Scraper Actor with Apify MCP

Build a serverless HN job scraper and wire it into an AI agent via MCP.

Introduction

30 Second Summary

Job hunting means scrolling through hundreds of forum posts and manually checking each one for the right keywords, stack, and location. What if an AI agent could do that filtering for you in seconds?

In this project, you will build a serverless job-scraping tool called an Apify Actor that fetches the latest Hacker News "Who's Hiring" thread and returns structured job postings. You'll then connect it to an AI agent in Cursor so it can run your scraper and filter results using plain English.

What You'll Build

You'll type "Find me remote Go backend jobs from the latest HN hiring thread" into Cursor, and the AI agent will discover your Actor, run it, receive 50+ job posts, and return a filtered summary of the ones that match your criteria.

By the end of this project, you'll have:

A deployed Actor you can trigger from any AI agent to get a structured list of fresh job postings from Hacker News.
A live MCP connection between Cursor and the Apify platform so the AI agent can discover and call your Actor on demand.
A working AI agent workflow where you ask a natural-language question and get filtered, summarized job results back.
Secret Mission: Add keyword pre-filtering to your Actor so it returns only relevant posts before the AI agent even sees them.

Are there any prerequisites?

You should be comfortable with basic Python (async/await, HTTP requests). No prior Apify or MCP experience is needed.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

Set Up Apify CLI and Scaffold Your Actor

Before you can write any Actor code, you need the Apify development environment on your machine. The Apify CLI bridges your local terminal to the Apify cloud platform.

It handles everything from scaffolding new projects to running them locally to deploying them. Without it, there is no way to create, test, or deploy Actors.

In this step, get ready to:

Install and verify the Apify CLI.
Create a free Apify account and log in via the CLI.
Scaffold a Python Actor project from the starter template.

Install the Apify CLI

The Apify CLI is a Node.js package you install globally with npm. Since you already have Node.js v22.22.1 installed, this is a one-liner.

Install the Apify CLI globally by running this command:

npm install -g apify-cli

What does this command do?

The -g flag installs the package globally, making the apify command available from any directory in your terminal. This is the CLI you will use throughout this project to create, run, and deploy your Actor.

Verify the installation by running:

apify --version

✔️ I see a version number

You should see something like apify-cli/x.x.x printed in your terminal. The Apify CLI is installed and ready to use.

ⓧ Command not found

If your terminal says the command was not found, your Node.js global bin folder may not be in your system PATH.

Close and reopen your terminal, then try apify --version again.
If it still fails, run npm list -g to confirm the package installed. If you see apify-cli in the list, your PATH needs updating.

Ask for help: Why can't my terminal find the apify command after I installed it?

Create your Apify account and log in

The CLI needs to authenticate with an Apify account so it can deploy Actors to the cloud later. You will create a free account and then connect it to your local CLI.

Open your browser and navigate to the Apify sign-up page:
https://console.apify.com/sign-up
Click Sign up using your email, Google, or GitHub account.
If you signed up with email, check your inbox for a verification email and click the link to verify.

Is this free?

The Apify free plan costs $0 per month and includes $5 of platform usage. No credit card is required. This project uses a tiny fraction of that allowance.

Once your account is confirmed, connect it to your CLI.

apify login

What will I see?

The CLI will prompt you to choose a login method. Select the option to sign in via the Apify Console in your browser. Your browser will open a sign-in page where you authenticate.

After you approve, the CLI saves your credentials to ~/.apify/auth.json and prints a success confirmation. All future CLI commands will use these stored credentials.

Login not completing?

Make sure you finished the account sign-up and verified your email before running apify login.
If the browser flow hangs, try the token method instead. Go to your Apify Console, navigate to Settings > Integrations, copy your API token, and run apify login -t your-api-token-here.

Need more help? Help me get apify login working

Scaffold your Actor project

With the CLI authenticated, you can now generate a complete Apify Actor project from a template. The python-start template gives you a ready-to-run Python project with the correct file structure.

Move to your Desktop by running this command:

cd ~/Desktop

Create your Actor project using the Python starter template by running:

apify create hn-jobs-scraper --template python-start

What does this command do?

This creates a new hn-jobs-scraper folder on your Desktop, downloads the Python starter template into it, creates a virtual environment at .venv, and installs the Actor dependencies automatically.

You should see output ending with a success message confirming the Actor was created.

Move into your new project folder by running:

cd hn-jobs-scraper

Before you check the project structure, what files do you expect to see inside an Apify Python Actor project?

Confirm the project files exist by running:

ls

You should see .actor, .venv, requirements.txt, and src in the output.

Verify the source code folder contains the main Actor file by running:

ls src

You should see main.py and __main__.py listed. These are the two files that make up your Actor's code.

Missing files or directories?

Make sure you ran cd hn-jobs-scraper before running ls. If you are still in ~/Desktop, you will see your Desktop files instead of the project.
If the hn-jobs-scraper folder does not exist, re-run the apify create command from ~/Desktop and check for error output.

Still stuck? Help me find my scaffolded Actor files

Your Apify environment is ready and you have a scaffolded project waiting for real scraping logic. Next up, you will replace the template code with a job scraper that pulls live data from Hacker News.

Build Your Job Scraper and Run It Locally

Your Actor project is scaffolded, but the template code just logs a greeting. You need real scraping logic that talks to the Hacker News API and returns structured job postings.

In this step, you will replace the template code with a scraper that automatically finds the latest "Who's Hiring" thread, fetches each top-level comment, strips the HTML, and saves each post as structured data. Running it locally first means you can see real results before deploying anything to the cloud.

In this step, get ready to:

Update your project dependencies and entry point.
Define the Actor's input schema so it accepts a thread ID and max post count.
Write the HN scraper logic and run it locally to verify structured output.

Update your dependencies and entry point

Your Actor needs two Python packages to work. The apify SDK handles the Actor lifecycle and data storage. The httpx library provides async HTTP requests for calling the HN API.

Open requirements.txt in Cursor and replace its contents with the following:

apify
httpx

What does this file do?

The requirements.txt file tells the Apify platform (and your local environment) which Python packages to install. The apify package provides the Actor SDK. The httpx package is an async-friendly HTTP client you will use to call the Hacker News API.

Install the updated dependencies by activating the virtual environment and running:

source .venv/bin/activate
pip install -r requirements.txt

You should see pip downloading and installing httpx and its dependencies. The apify package may already be installed from the template scaffold.

Seeing a permission error or command not found?

Make sure you are inside your Actor project folder (the one created by apify create) and that the .venv folder exists there. On Windows, use .venv\Scripts\activate instead of source .venv/bin/activate.

help me fix my virtual environment activation

Next, your Actor needs an entry point file so apify run knows how to start it. The template may already include this file, but you need to make sure its contents match.

Open src/__main__.py in Cursor (create it if it does not exist) and set its contents to:

import asyncio

from .main import main

if __name__ == '__main__':
    asyncio.run(main())

What does this entry point do?

When you run apify run, the CLI executes python -m src under the hood. Python looks for src/__main__.py and runs it. This file imports your main function and kicks it off with asyncio.run().

Confirm both files are saved by running:

ls src/

You should see __main__.py and main.py listed in the output.

Define the Actor's input schema

The input schema tells Apify (and any AI agent) what parameters your Actor accepts. Your scraper needs two optional inputs: a specific thread ID and a maximum number of posts to fetch.

Open .actor/input_schema.json in Cursor and replace its contents with the following:

{
    "title": "HN Who's Hiring Scraper",
    "type": "object",
    "schemaVersion": 1,
    "properties": {
        "threadId": {
            "title": "Thread ID",
            "type": "integer",
            "description": "Hacker News thread ID to scrape. If not provided, the Actor will automatically find the latest 'Who is hiring?' thread.",
            "editor": "number"
        },
        "maxPosts": {
            "title": "Max Posts",
            "type": "integer",
            "description": "Maximum number of job posts to fetch from the thread.",
            "default": 50,
            "editor": "number"
        }
    }
}

What does this schema define?

threadId is an optional integer. If omitted, the Actor will search for the latest "Who is hiring?" thread automatically.
maxPosts caps how many comments to fetch. It defaults to 50 so you get a meaningful sample without hitting every single post.
The editor field tells Apify Console how to render the input field in the UI.

Save .actor/input_schema.json.

Write the scraper logic

Now for the core of your Actor. You will replace the template src/main.py with logic that finds the latest HN "Who's Hiring" thread, fetches each top-level comment, strips HTML tags, and pushes structured data to the Actor's dataset.

Open src/main.py in Cursor. Delete all existing template code and paste the following imports and thread-search logic:

import re
from html import unescape

import httpx
from apify import Actor


async def main() -> None:
    async with Actor:
        actor_input = await Actor.get_input() or {}
        thread_id = actor_input.get('threadId')
        max_posts = actor_input.get('maxPosts', 50)

        async with httpx.AsyncClient(timeout=30.0) as client:
            if not thread_id:
                Actor.log.info('No threadId provided, searching for latest "Who is hiring?" thread...')
                search_url = 'https://hn.algolia.com/api/v1/search?query=%22Ask+HN%3A+Who+is+hiring%22&tags=ask_hn&hitsPerPage=1'
                resp = await client.get(search_url)
                resp.raise_for_status()
                hits = resp.json().get('hits', [])
                if not hits:
                    Actor.log.error('Could not find a "Who is hiring?" thread.')
                    return
                thread_id = int(hits[0]['objectID'])
                Actor.log.info(f'Found thread: {hits[0]["title"]} (ID: {thread_id})')

What does this code do?

async with Actor initializes the Apify runtime (logging, storage, environment variables).
Actor.get_input() reads the JSON input you defined in the schema.
If no threadId is provided, the scraper queries the Algolia HN Search API to find the most recent "Ask HN: Who is hiring?" thread. This makes the Actor smart enough to always grab the latest monthly thread.

Directly below the code you just pasted (still inside the async with httpx.AsyncClient block), add the thread-fetching and comment-iteration logic:

            thread_url = f'https://hacker-news.firebaseio.com/v0/item/{thread_id}.json'
            resp = await client.get(thread_url)
            resp.raise_for_status()
            thread = resp.json()

            if not thread or 'kids' not in thread:
                Actor.log.error(f'Thread {thread_id} has no comments.')
                return

            thread_title = thread.get('title', 'Unknown')
            comment_ids = thread['kids'][:max_posts]
            Actor.log.info(f'Fetching {len(comment_ids)} job posts from "{thread_title}"...')

            for comment_id in comment_ids:
                comment_url = f'https://hacker-news.firebaseio.com/v0/item/{comment_id}.json'
                resp = await client.get(comment_url)
                if resp.status_code != 200:
                    continue
                comment = resp.json()
                if not comment or comment.get('deleted') or comment.get('dead'):
                    continue

                text = comment.get('text', '')
                clean_text = strip_html(text)

                await Actor.push_data({
                    'text': clean_text,
                    'author': comment.get('by', 'unknown'),
                    'timestamp': comment.get('time', 0),
                    'hnUrl': f'https://news.ycombinator.com/item?id={comment_id}',
                    'threadTitle': thread_title,
                })

            Actor.log.info('Done! Scraping complete.')

What does this section do?

It fetches the full thread object from the official HN Firebase API using the thread ID.
The kids field contains the IDs of all top-level comments (job postings). It slices to max_posts to respect the limit.
For each comment, it fetches the full item, skips deleted or dead posts, strips HTML, and pushes a clean data object to the Actor's default dataset via Actor.push_data().

Finally, add the strip_html helper function at the bottom of the file (outside the main function, with no indentation):


def strip_html(html_text: str) -> str:
    """Remove HTML tags and decode entities."""
    text = re.sub(r'<p>', '\n\n', html_text)
    text = re.sub(r'<br\s*/?>', '\n', text)
    text = re.sub(r'<[^>]+>', '', text)
    text = unescape(text)
    return text.strip()

Why do you need to strip HTML?

HN comments are stored as raw HTML. Tags like <p> and <a href="..."> would clutter your dataset and confuse any AI agent reading the results. This helper converts HTML into clean, readable plain text.

Save src/main.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved src/main.py with all three sections together in one file.

ⓧ I'd like to double check the full code

Here is the complete src/main.py file. Scroll through and confirm yours matches:

Chunk 1 (lines 1-25): imports, main() function start, input reading, and Algolia search for the latest thread.
Chunk 2 (lines 26-57): thread fetch via Firebase API, comment iteration loop, HTML stripping, and Actor.push_data() calls.
Chunk 3 (lines 59-65): the strip_html() helper function at the bottom.

Make sure there is a blank line between the end of main() and the start of strip_html(). Confirm all three chunks are present and in order.

Run the Actor locally and verify output

Everything is in place. Before you run it, think about what you expect to happen. The Actor should fetch the latest "Who's Hiring" thread and print structured job postings.

Before you run this, what do you think will appear in the terminal? Will it succeed on the first try, or could something go wrong?
Run the Actor locally by executing:

apify run

You should see log lines from your Actor, including the thread title it found and a message like Fetching 50 job posts from "Ask HN: Who is hiring?" followed by Done! Scraping complete..

Confirm the scraped data was saved by running:

ls storage/datasets/default/

You should see numbered JSON files (like 000000001.json), one per job posting. Each file contains your structured data.

Peek at one of the results by running:

cat storage/datasets/default/000000001.json

You should see a JSON object with text, author, timestamp, hnUrl, and threadTitle fields filled with real job posting data.

Actor failed or no files appeared?

If you see a ModuleNotFoundError, make sure you activated the virtual environment (source .venv/bin/activate) and ran pip install -r requirements.txt before apify run.
If you see a network timeout error, check your internet connection. The Actor needs to reach hn.algolia.com and hacker-news.firebaseio.com.
If the storage/datasets/default/ folder is empty, check the terminal logs for errors. A common cause is a typo in the Algolia search URL or a missing await keyword.

help me debug why my Apify Actor run produced no output

Your scraper works locally and produces real job data. Next up, you will deploy it to the Apify cloud so it becomes a permanent, API-accessible tool.

Deploy to Apify and Hit the Data Wall

Your Actor works on your machine, but that is where it stays. Nobody else can call it, no AI agent can discover it, and the data disappears the moment you close the terminal.

In this step you will deploy the Actor to the Apify platform so it runs in the cloud with a persistent dataset. You will also see firsthand why raw scraping output is painful to read without an AI agent to help.

In this step, get ready to:

Push your Actor to the Apify cloud with a single command.
Run the Actor from Apify Console and produce a full dataset.
Browse 50+ raw job posts and experience the data wall.

Push your Actor to the Apify platform

Deploying uploads your source code and builds it in the cloud. Once deployed, your Actor gets its own API endpoint, a dataset store, and a public URL on the Apify platform.

Deploy your Actor to Apify by running this command:

apify push

The CLI will stream build logs and finish with a success message. Before you continue, what do you expect to see at the end of the output?

You should see a line that says the Actor was deployed to Apify cloud and built there, along with a link to the Actor detail page in Apify Console.

What does this command do?

The apify push command reads your .actor/actor.json file to identify the Actor name and version. It uploads your source files to the Apify platform and triggers a Docker build in the cloud.

Once the build finishes, your Actor is live and callable via the Apify API or Console.

Seeing a build error?

Make sure you are running the command from your Actor project directory (the folder containing .actor/).

If you see an authentication error, run apify login again to refresh your credentials.

help me fix my apify push build error

Run the Actor from Apify Console

Now that your Actor lives in the cloud, you can trigger a run from the web console and watch it scrape Hacker News in real time.

Open Apify Console in your browser.
In the left sidebar, click Development, then My Actors.
Click your Actor's name to open its detail page.
Switch to the Input tab. Leave all fields at their defaults (no Thread ID, Max Posts at 50).
Click Start to launch the run.

The Actor will start running in the cloud. You can watch the logs stream in real time on the run page.

Wait until you see a log line that says Done! Scraping complete. and the run status shows SUCCEEDED.

What should I see?

The logs will show the Actor searching for the latest "Who is hiring?" thread, finding it, then fetching 50 job posts one by one. The run typically finishes in under 30 seconds.

Run status shows FAILED?

Click the Log tab on the run page to see the error details. The most common cause is a missing dependency. Check that requirements.txt contains both apify and httpx.

If the dependency is missing, add it to requirements.txt, then redeploy with apify push and start a new run.

help me debug my failed Actor run

Browse the dataset and hit the data wall

Your Actor finished and the data lives in the cloud now. Time to see what 50 raw job posts actually look like when you try to read them yourself.

On the run page, click the Output tab.
Scroll through the items in the dataset.

Each item is a wall of unformatted text. Company names, tech stacks, locations, and application links are all mashed together with no structure. Try to answer this question just by scrolling: which of these 50 posts are remote Go backend roles?

You cannot. The text is too dense, too inconsistent, and there are too many posts to skim manually. This is the data wall.

Why is this a problem?

Raw job postings on Hacker News have no standard format. One post might lead with the company name, another with the tech stack, another with the location. There are no tags, no filters, and no search.

You built a scraper that reliably collects this data. The missing piece is something that can reason about natural language text and answer questions like "which of these are remote Go roles?" That is exactly what an AI agent does.

Your Actor is deployed and producing real data in the cloud. The problem is clear: you have 50+ posts and no fast way to filter them. Next up, you will connect an AI agent to this Actor so it can call your scraper and reason about the results for you.

Connect Cursor to the Apify MCP Server

Your Actor is deployed and producing data, but scrolling through 50+ raw job posts to find the ones you care about is painful. The fix is letting an AI agent call your Actor directly, receive the results, and reason about them for you.

The Model Context Protocol (MCP) is what makes this possible. It gives AI agents a standard way to discover and call external tools. In this step, you will connect Cursor to the Apify MCP server so the agent can find and run your Actor on demand.

In this step, get ready to:

Create the MCP configuration file in your project.
Authorize the connection via the Apify OAuth flow.
Verify that the AI agent can see Apify tools.

Create the MCP configuration file

Cursor looks for a file called .cursor/mcp.json in your project root to know which MCP servers to connect to. You will create this file with the Apify MCP server URL.

Create the .cursor directory in your project by running this command in your terminal:

mkdir -p .cursor

Confirm the directory was created by running:

ls -a

You should see .cursor listed alongside your existing .actor and src directories.

Create the mcp.json file inside the .cursor directory with the Apify MCP server configuration by running:

cat > .cursor/mcp.json << 'EOF'
{
    "mcpServers": {
        "apify": {
            "url": "https://mcp.apify.com/"
        }
    }
}
EOF

What does this configuration do?

The mcpServers object tells Cursor which external tool servers to connect to. The url field points to Apify's hosted MCP server at https://mcp.apify.com/.

When Cursor connects for the first time, it uses OAuth to authenticate. This means it opens your browser for you to sign in to Apify and authorize the connection, keeping your API token secure without hardcoding it in the file.

Verify the file contents are correct by running:

cat .cursor/mcp.json

You should see the JSON with https://mcp.apify.com/ as the URL.

File not created?

Make sure you are running the commands from your Actor project's root directory (the folder that contains src/ and .actor/). If you see an error about the heredoc syntax, make sure you are using a Bash-compatible shell.

help me create the .cursor/mcp.json file

Authorize the connection and restart Cursor

Cursor reads the MCP config on startup. You need to restart it so it picks up the new .cursor/mcp.json file.

Close Cursor completely (quit the application, not just close the window).
Reopen Cursor and open your Actor project folder again.
Open Cursor's MCP settings by going to Preferences > Cursor Settings > Tools & MCPs.
Scroll down to find apify in the MCP server list.
Click Connect next to the Apify MCP server entry.

Cursor opens a browser tab for the Apify OAuth flow. Since you already have an Apify account from earlier, you should see a permissions screen.

Review the permissions and click Allow access in your browser.
Return to Cursor. The apify entry in Tools & MCPs should now show as connected.

Browser doesn't open or OAuth fails?

If the browser does not open automatically, look for an OAuth URL in Cursor's output panel and paste it into your browser manually.

If the MCP server still shows as disconnected, try toggling it off and on in the Tools & MCPs page, then click Connect again.

As a fallback, you can authenticate with your Apify API token directly. Go to Apify Console > Settings > Integrations, copy your API token, and set it as an environment variable before starting Cursor:

export APIFY_TOKEN=your-api-token-here

help me connect Cursor to the Apify MCP server

Verify the MCP tools are available

The MCP connection gives the AI agent access to a set of tools for interacting with Apify. You will confirm the agent can see them by asking it directly.

Open a new Agent chat in Cursor by pressing Cmd+Shift+I (macOS) or Ctrl+Shift+I (Windows).

Before you ask the agent anything, what do you expect it to know about? If the MCP connection is working, the agent should be able to list the Apify tools it has access to.

Type the following message in the agent chat:

What Apify tools are available?

The agent responds with a list of available tools. You should see tools including search-actors and call-actor in the response. These are the tools the agent will use in the next step to find and run your Actor.

What are these MCP tools?

The Apify MCP server exposes a set of tools that any connected AI agent can call. search-actors lets the agent search Apify's catalog by name or keyword. call-actor lets the agent run any Actor and retrieve its output.

Together, these tools let the agent discover your deployed Actor, inspect its input schema, run it with parameters, and then reason about the results.

Agent doesn't list any Apify tools?

Go back to Preferences > Cursor Settings > Tools & MCPs and confirm the Apify server shows a green connected status. If it shows disconnected, click Connect again and complete the OAuth flow.

If the workspace-level config is not detected, try moving your .cursor/mcp.json to the global location at ~/.cursor/mcp.json instead.

help me get Cursor to recognize my Apify MCP server

✔️ Awesome, I've got everything!

Great. Double check you have saved your .cursor/mcp.json file and that the Apify MCP server shows as connected in Cursor's Tools & MCPs settings.

ⓧ I'd like to double check the full code

Here is the complete .cursor/mcp.json file:

{
    "mcpServers": {
        "apify": {
            "url": "https://mcp.apify.com/"
        }
    }
}

Your AI agent now has a direct line to Apify. In the next step, you will put this to work by asking the agent to find your Actor, run it, and filter the results using natural language.

Let the AI Agent Find Your Next Job

Remember the data wall from Step 3? You had 50+ raw job posts dumped into a dataset, and scrolling through them manually was painful. That problem is still unsolved.

The fix is right here. Your Actor is deployed, the MCP server is connected, and now you'll let Cursor's AI agent do the heavy lifting. You'll ask a natural-language question and watch the agent discover your Actor, run it, and reason about the results.

In this step, get ready to:

Ask the agent to discover your Actor by name using Apify MCP tools.
Ask the agent to run your Actor and filter results for specific roles.
Observe the full agent tool chain from discovery to filtered summary.

Ask the agent to discover your Actor

The Apify MCP server exposes tools like search-actors and call-actor that the AI agent can call on your behalf. First, you'll confirm the agent can find your specific Actor among all available ones.

In Cursor, open a new Agent chat by pressing Cmd+L (macOS) or Ctrl+L (Windows).
Make sure the chat mode is set to Agent (not Ask or Edit) using the dropdown at the top of the chat panel.
Type this prompt into the chat:

Search for my Apify Actor called "hn-whos-hiring-scraper" using the Apify tools available to you. Tell me what you find.

Press Enter to send the message.

The agent should respond by calling the search-actors tool. You'll see it find your Actor and report its name, description, and input schema details.

What is happening behind the scenes?

When you send a message to the agent, it reads your prompt and decides which MCP tools to call. The search-actors tool queries the Apify platform for Actors matching your search term.

The agent then uses fetch-actor-details to inspect the Actor's input schema. This tells it what parameters your Actor accepts, like threadId and maxPosts.

Agent not finding your Actor?

If the agent says it cannot find any Apify tools, the MCP connection may have dropped. Go to Cursor Settings and check the MCP section to confirm the Apify server shows a green status.

If the agent can see Apify tools but cannot find your specific Actor, try using your Apify username in the search (e.g., "search for username/hn-whos-hiring-scraper").

Help me troubleshoot why the Cursor agent can't find my Apify Actor via MCP.

Ask the agent to run your Actor and filter results

Now that the agent knows your Actor exists and understands its input schema, you can ask it to actually run the Actor and do something useful with the results.

In the same agent chat, type this follow-up prompt:

Run my HN jobs scraper and find me remote Go or Rust backend roles. Show me the matching posts with their HN links.

Press Enter to send.

Before you look at the response, think about what the agent needs to do. How many tool calls do you think it takes to go from your plain-English question to filtered job results?

Watch the agent's response stream in. It chains multiple tool calls: call-actor to trigger your Actor with default inputs, then get-actor-output to retrieve the dataset items. Finally, it reads through 50+ job posts and picks out the ones mentioning Go, Rust, remote, or backend.

Why does the agent filter results, not the Actor?

Your Actor returns all job posts without filtering. The AI agent reads the full dataset and uses its language understanding to find relevant posts. This is flexible because you can ask for "remote Go roles" one minute and "ML engineer positions in Europe" the next without changing any code.

The tradeoff: for large threads (500+ posts), sending everything to the agent uses more tokens. The Secret Mission addresses this by adding keyword pre-filtering at the Actor level.

Agent run taking a long time?

The Actor needs to fetch up to 50 individual comments from the Hacker News API. This can take 15-30 seconds. The agent will wait for the run to complete before retrieving results.

If the agent reports an error about the Actor run failing, check your Actor's run log in Apify Console to see the specific error.

Help me figure out why my Actor run is failing when called from the Cursor agent.

Observe the full agent tool chain

The agent's response is the proof that the full loop works. Look at what just happened in a single chat message.

Scroll through the agent's response and identify each tool call it made. You should see a chain like this:

Discovery: the agent used search-actors or fetch-actor-details to locate your Actor and read its input schema.
Execution: the agent used call-actor to trigger your Actor with the default maxPosts of 50.
Retrieval: the agent used get-actor-output to pull the scraped job posts from the dataset.
Reasoning: the agent read through the posts and selected ones mentioning Go, Rust, remote, or backend. It presented a filtered summary with HN links.

You should see a handful of matching job posts, each with the company or author name, a brief description of the role, and a direct hnUrl link back to the original Hacker News comment.

What if no posts match?

The latest "Who's Hiring" thread may not have Go or Rust backend roles posted yet. If the agent reports no matches, try a broader query like "find me any remote engineering roles" to confirm the full chain works. The tool integration is the achievement here, not the specific filter results.

Try different queries

Now that the loop works, experiment with different prompts. Try "find ML or data roles in Europe" or "show me any YC companies hiring." The agent uses the same tool chain each time but reasons differently about the results.

Secret mission

Add Keyword Pre-Filtering to Your Actor

Your Actor dumps every job post to the agent, which then filters them. For large threads with 500+ posts, that wastes tokens and slows the response. Can you make the Actor smarter by filtering at the source?

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs on Apify's free tier with $5/month included platform credits, so your deployed Actor only costs compute units when it actually runs.

Resources you used:

Local project directory on your Desktop (Actor source code, input schema, MCP config).
Deployed Apify Actor (cloud-hosted on the Apify platform).
Apify CLI (installed globally via npm).
Apify account (free tier, no credit card required).

Keep everything running

No action needed. Choose this if you plan to keep developing your Actor or want the AI agent to keep calling it on demand.

Your Actor sits idle on the Apify platform at zero cost until someone runs it.
The MCP connection in Cursor stays active so the AI agent can call your Actor anytime.
The Apify CLI remains available for future projects.

Pause - I'll come back to this later

Shut down running processes and disconnect the MCP server to free up resources, but keep your files and deployed Actor intact.

Remove the MCP connection by deleting the .cursor/mcp.json file from your project folder. This disconnects Cursor from Apify but does not delete your Actor.
Log out of the Apify CLI by running:

apify logout

Your deployed Actor remains on the platform and can be reconnected later by logging back in with apify login.

Delete - I don't want to use this again

Remove all project resources and start fresh. This permanently deletes your Actor from the Apify platform and removes local files.

Delete your deployed Actor from the Apify platform by running:

apify actors rm [[ACTOR_ID="your-username~my-first-actor"]] -y

How do I find my Actor ID?

Your Actor ID uses the format your-username~my-first-actor. You can find it by running:

apify actors ls

Log out of the Apify CLI:

apify logout

Uninstall the Apify CLI:

npm uninstall -g apify-cli

Delete the local project folder. On macOS or Linux:

cd ~/Desktop
rm -rf my-first-actor

On Windows:

cd ~\Desktop
rmdir /s /q my-first-actor

To delete your Apify account entirely, open Apify Console and click Settings in the side menu. Use the account deletion option on the Account page. This is optional if you want to keep the account for future projects.

Nice Work!

You just built a serverless job-scraping tool from scratch and plugged it directly into an AI agent workflow. That is a production-grade integration pattern used by teams building real AI-powered products.

You've learned how to:

Build a Python Apify Actor from scratch that scrapes Hacker News "Who's Hiring" threads into structured job postings.
Deploy an Actor to the Apify platform as a cloud-hosted serverless microservice with a public API and dataset store.
Connect Cursor to the Apify MCP server and demonstrate the full agent loop: discovery, schema inspection, execution, and natural-language reasoning over results.
Secret Mission: Add keyword pre-filtering to your Actor so it returns only relevant posts before the AI agent sees them, reducing token waste for large threads.

Ready to quiz yourself?