Build a Book Price Scraper with Apify

Scrape book titles, prices, and ratings and deploy to the cloud.

Introduction

30 Second Summary

Every online store, job board, and review site holds data you could use. Instead of copying it by hand, web scrapers do the work for you in seconds.

In this project, you will build a web scraper that automatically extracts book titles, prices, and star ratings from a live website. You'll deploy it to the cloud as a serverless Actor on Apify, ready to run on demand without your computer needing to be on.

What You'll Build

You'll run a single command, watch 20 books appear as structured data in your terminal, then open the Apify Console and trigger the same scraper from the cloud with one click.

By the end of this project, you'll have:

  • A working scraper you can run with one command that pulls 20 books with their titles, prices, and star ratings into clean JSON.
  • A cloud-deployed Actor on the Apify platform you can trigger from anywhere with one click, no local machine required.
  • An exportable dataset you can download as CSV or JSON and open in any spreadsheet app.
  • Secret Mission: Add multi-page crawling so your Actor follows pagination links and scrapes the entire 1000-book catalog.

Are there any prerequisites?

You'll need basic JavaScript knowledge (variables, functions, loops) and Node.js v22 or higher installed on your machine. You'll also need a free Apify account, which you can create with Google sign-in and no credit card.

Before We Start

Before We Start...

Before we dive in, let's lock in what you're about to build and why it matters. This is the moment to commit to the goal so every tool and command in the steps ahead has a clear purpose.

Set Up the Apify CLI

The Apify CLI is your bridge between local code and the Apify cloud. It lets you create, test, and deploy web scrapers from your terminal without manually clicking through a browser interface.

In this step, you'll install the CLI, authenticate with your Apify account, and scaffold a new Actor project ready for your book-scraping code.

In this step, get ready to:
  • Install the Apify CLI globally using npm.
  • Log in to your Apify account from the terminal.
  • Create a new Actor project with the CheerioCrawler template.
Install the Apify CLI

The Apify CLI is an npm package you install globally so it's available from any directory on your machine.

  • Open your terminal by pressing Cmd+Space (macOS) or the Windows key (Windows), typing Terminal, and pressing Enter.
  • Install the Apify CLI globally by running this command:
npm install -g apify-cli

What does this command do?

The -g flag tells npm to install the package globally. This means you can run apify commands from any folder on your computer, not just inside a specific project.

  • Verify the installation succeeded by running:
apify --version

You should see a version number printed in your terminal.

✔️ I see a version number

The Apify CLI is installed and ready to use. Move on to the next substep.

ⓧ Command not found

If your terminal says the apify command is not found, try these fixes:

  • Close and reopen your terminal, then run apify --version again. Some terminals need a restart to pick up new global packages.
  • Make sure Node.js v22 or higher is installed by running node --version. The Apify CLI requires Node.js version 22 or higher.
  • On macOS, if you get a permissions error, try running the install with sudo:
sudo npm install -g apify-cli
Log in to your Apify account

Before you can deploy anything to the cloud, the CLI needs to know which Apify account to use. Logging in saves your credentials locally so every future command is authenticated.

  • Start the login process by running:
apify login

What prompts will you see?

The CLI will present two sign-in options. Choose Sign in via the Apify Console in your browser (the recommended option). Your browser will open an Apify authentication page.

Once you approve the sign-in in your browser, switch back to your terminal. You'll see a confirmation message that you're logged in.

  • Select the browser sign-in option when prompted.
  • Approve the authentication in your browser when the Apify page opens.
  • Return to your terminal and confirm you see a logged-in confirmation message.

Not seeing the browser open?

If the browser doesn't open automatically, look for a URL in your terminal output. Copy and paste it into your browser manually.

If login still fails, you can use the alternative method: run apify login again and choose the API token option. Find your token at console.apify.com under Settings > Integrations.

Help me troubleshoot my Apify CLI login

Create your Actor project

Now you'll scaffold a new Actor project using a template that already has Crawlee and the CheerioCrawler class configured. This gives you a working starting point with all dependencies ready.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your Actor project by running:
apify create book-scraper

What prompts will the CLI show?

The CLI will walk you through a few choices:

  • When prompted for a programming language, choose JavaScript.
  • When prompted for a template, choose Crawlee + CheerioCrawler.

The CLI will create a book-scraper directory, download the template files, and install all npm dependencies. This takes a few seconds.

  • Wait for the installation to complete. You'll see a success message in your terminal.
  • Navigate into the new project folder and open it in VS Code by running these commands:
cd book-scraper
code .
  • Confirm you see the project files in the VS Code Explorer sidebar. You should see a src folder, a .actor folder, a package.json, and a storage folder.

VS Code not opening from the terminal?

If code . doesn't work, you may need to add VS Code to your system PATH. Open VS Code manually, press Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows), type Shell Command: Install 'code' command in PATH, and select it. Then try code . again.

Alternatively, open VS Code manually, click File in the top menu bar, click Open Folder, and select the book-scraper folder on your Desktop.

Help me open my project in VS Code

Your development environment is locked and loaded. Next up, you'll run the default template to see web scraping in action before writing any custom code.

Run the Default Template

Your Actor project is scaffolded and ready to go. Before writing any custom scraping code, you need to see what the template already does out of the box.

Running the default template now gives you a mental model of the full scraping cycle. You will see how logs appear in the terminal, where scraped data lands on disk, and what each file in the project controls.

In this step, get ready to:
  • Run the default Actor locally and watch the crawl logs.
  • Inspect the scraped data in the output folder.
  • Map out the project structure so you know where to make changes next.
Run the Actor locally

The template came pre-wired with a working scraper that targets a sample website. One command kicks off the entire crawl and saves the results to your local disk.

Before you run it, what do you think a web scraper's terminal output looks like? Take a guess at what you will see.

  • Run the default Actor by entering this command in your terminal:
apify run

Watch the logs scroll by. You will see system info printed first, then lines showing URLs being fetched and data being extracted. The run completes in a few seconds.

What does this command do?

The apify run command executes your Actor locally with simulated Apify environment variables. It reads input from storage/key_value_stores/default/INPUT.json and stores scraped data in the local storage directory.

The default template uses CheerioCrawler to fetch web pages and extract headings. You will see log lines like INFO System info followed by extracted heading entries.

Seeing an error when you run?

  • If you see a "module not found" error, make sure you are inside the book-scraper folder where package.json lives. The template installed dependencies during apify create, but they only resolve from the project root.
  • If you see a network-related error, check your internet connection. The crawler needs to reach external URLs to scrape them.

Why is my apify run failing with an error?

Inspect the output
  • In VS Code's Explorer sidebar, expand the storage folder, then datasets, then default.
  • Click on one of the numbered JSON files (for example, 000000001.json) to open it.

Each file contains data the template extracted from one scraped page. You will see page titles and headings from the default crawl target.

Why are there multiple JSON files?

Each call to pushData inside the crawler stores one record as a separate file in the Apify dataset. When you deploy to the cloud later, these records merge into a single exportable dataset.

Map out the project structure

Now that you have seen data flowing through the Actor, here is how the project files map to each part of the process.

What does each file do?

  • src/main.js is where all the scraping logic lives. This is the file you will edit in the next step.
  • .actor/ contains configuration files (actor.json and input_schema.json) that define how your Actor runs on the Apify platform.
  • storage/key_value_stores/default/INPUT.json is the input your Actor reads when it starts. It contains the start URLs to crawl.
  • package.json lists the project dependencies: apify (the SDK) and crawlee (the crawling library).

You have a working scraper running locally and you know where every piece lives. Next up, you will swap out the default logic and point your Actor at a real target to scrape book data.

Write Your Book Scraper

You ran the default template and saw how an Actor scrapes a website, processes pages, and stores data in JSON files. Now it's time to make the scraper yours.

The default template targets a generic website and extracts page headings. You need it to target books.toscrape.com and extract the specific data you care about: titles, prices, and star ratings. This is where you learn how Cheerio selectors work to pluck data out of HTML.

In this step, get ready to:
  • Point your Actor's input at the Books to Scrape URL.
  • Write a CheerioCrawler that extracts book titles, prices, and ratings.
  • Run the scraper locally and confirm 20 book objects appear in the dataset.
Update the input to target your scraping URL

Your Actor reads its start URLs from storage/key_value_stores/default/INPUT.json. Right now it points at the default template's target site.

  • In VS Code, open storage/key_value_stores/default/INPUT.json.
  • Select all the existing content and replace it with this:
{
    "startUrls": [
        { "url": "https://books.toscrape.com/catalogue/page-1.html" }
    ]
}

What does this file do?

When an Actor starts, it calls Actor.getInput() to read this JSON. The startUrls array tells the crawler which page to visit first.

You're pointing it at page 1 of Books to Scrape's catalog, which lists 20 books per page.

  • Save INPUT.json.
Replace the scraping logic in src/main.js

The default template's crawling logic lives in src/main.js. You'll replace all of it with code that uses Cheerio's CSS selectors to pull book data from the page's HTML.

  • Open src/main.js in VS Code.
  • Select all the existing code (Cmd+A on macOS or Ctrl+A on Windows) and delete it.
  • Paste the following setup code at the top of the now-empty file:
// Import the Apify SDK and CheerioCrawler from Crawlee
import { Actor } from 'apify';
import { CheerioCrawler } from 'crawlee';

// Initialize the Actor and read the start URLs from input
await Actor.init();

const input = await Actor.getInput();
const startUrls = input.startUrls;

What does this code do?

  • Actor.init() boots up the Apify environment so your code can use datasets, key-value stores, and other platform features.
  • Actor.getInput() reads the JSON you just saved in INPUT.json and returns it as a JavaScript object.
  • startUrls now holds the array of URLs the crawler will visit.
  • Directly below the setup code, paste the crawler definition:
const crawler = new CheerioCrawler({
    async requestHandler({ request, $, pushData, log }) {
        log.info(`Scraping ${request.url}...`);

        const books = [];

        // Select each book card and extract title, price, rating
        $('article.product_pod').each((index, element) => {
            const title = $(element).find('h3 a').attr('title');
            const price = $(element).find('.price_color').text();
            const ratingClass = $(element).find('.star-rating').attr('class');
            const rating = ratingClass ? ratingClass.replace('star-rating ', '') : 'Unknown';

            books.push({
                title,
                price,
                rating,
            });
        });

        log.info(`Found ${books.length} books on ${request.url}`);

        for (const book of books) {
            await pushData(book);
        }
    },
});

await crawler.run(startUrls);

await Actor.exit();

What does this code do?

  • CheerioCrawler fetches each URL and parses the HTML into a Cheerio $ object you can query with CSS selectors.
  • $('article.product_pod') selects every book card on the page. Each card contains the title, price, and star rating.
  • $(element).find('h3 a').attr('title') grabs the full book title from the link's title attribute (the link text is truncated, but the attribute has the full name).
  • The rating is stored as a CSS class like star-rating Three, so you strip the prefix to get just the word.
  • pushData(book) saves each book object to the Actor's dataset, which creates one JSON file per item.
  • Actor.exit() tells the platform the Actor has finished cleanly.
  • Save src/main.js.

✔️ Awesome, I've got everything!

Great. Double check you've saved src/main.js and let's test it.

ⓧ I'd like to double check the full code

Your src/main.js should have two sections in order:

  • The setup block (imports, Actor.init(), getInput, startUrls) from the first code block above.
  • The crawler definition (CheerioCrawler, the requestHandler, crawler.run, Actor.exit) from the second code block above.

Scroll up to both code blocks and confirm each section is present in your file, in that order, with no leftover template code.

Test your scraper locally

Before running, you need to clear out the old dataset files left over from the template's previous run. Otherwise your results will be mixed with stale data.

  • Delete the old dataset files by running this command:
rm -rf storage/datasets/default/*
  • Confirm the folder is empty by running:
ls storage/datasets/default/

You should see no output, meaning the folder is empty and ready for fresh data.

Before you run the scraper, think about what you expect to happen. How many books are on a single page of Books to Scrape? How many JSON files should appear?

  • Run your Actor by executing:
apify run

You should see log output showing the crawler starting, fetching the Books to Scrape URL, and finding 20 books. The run completes in a few seconds.

  • Check the results by running:
ls storage/datasets/default/

You should see 20 numbered JSON files (one per book).

  • Open any one of the JSON files in VS Code to confirm it contains title, price, and rating fields.

Not seeing 20 files?

  • Make sure you saved both INPUT.json and src/main.js before running.
  • Check that the URL in INPUT.json is exactly https://books.toscrape.com/catalogue/page-1.html with no extra spaces.
  • If you see a syntax error, compare your src/main.js against the two code blocks above. A missing comma or bracket is the most common cause.

Help me debug my Apify Actor. I'm not seeing 20 JSON files after running apify run.

Your scraper works locally and produces clean, structured data from a live website. Next up, you'll deploy this Actor to Apify's cloud so it can run on demand from anywhere.

Deploy to Apify Cloud

Your scraper works locally and extracts 20 books from Books to Scrape. That proves the code is correct.

The real power of Apify is running scrapers in the cloud, on demand, without your computer needing to be on. In this step, you'll deploy your local script to the Apify platform where it becomes a serverless Actor accessible from anywhere.

In this step, get ready to:
  • Push your Actor to the Apify cloud.
  • Run the Actor from the Apify Console.
  • Export your scraped data as CSV or JSON.
Push your Actor to the cloud

The apify push command packages your entire project and uploads it to the Apify platform. Once there, it builds automatically and becomes available to run from anywhere.

  • Deploy your Actor to the Apify platform by running this command:
apify push

You'll see build logs scrolling in your terminal as Apify uploads and builds your Actor. At the end, you'll see a URL pointing to your Actor on the platform.

Seeing an authentication error?

Your login session may have expired. Run apify login again to re-authenticate, then retry apify push.

Also make sure your terminal is inside the book-scraper directory. The command needs to find your project files to upload them.

Help me troubleshoot this

Run the Actor in Apify Console

Your Actor is now live on the platform. Time to run it in the cloud and confirm it produces the same 20 books you saw locally.

  • Open your browser and go to console.apify.com.
  • Click Development in the left sidebar.
  • Click My Actors.
  • Find your book-scraper Actor and click on it.
  • Go to the Input tab. Confirm the Start URL is set to https://books.toscrape.com/catalogue/page-1.html.

Before you click Start, what do you think the cloud run will produce? Will it behave differently from your local run?

  • Click Start to begin the cloud run.
  • Watch the run logs. You'll see the same log messages you saw locally: the crawler starting, fetching the URL, and finding 20 books.

The run completes in a few seconds. The status changes to SUCCEEDED once done.

What just happened?

Your scraper ran on Apify's servers, not your machine. It used the same code and input you developed locally, but executed entirely in the cloud. You could close your laptop and it would still run.

This is what makes Actors powerful. You can schedule them, trigger them via API, or share them with others.

Run status shows FAILED?

Click on the run to check the logs. The most common cause is a mismatch between your local input and what the cloud version expects.

Make sure your INPUT.json was included in the push. If the Start URL field on the Input tab is empty, paste https://books.toscrape.com/catalogue/page-1.html into it and click Start again.

Help me figure out why my run failed

Export your scraped data

The scraped book data is stored in an Apify dataset. You can export it in multiple formats directly from the Console.

  • Once the run status shows SUCCEEDED, go to the Output tab.
  • You'll see your 20 book records listed with their titles, prices, and ratings.
  • Click Export and choose CSV or JSON as the format.
  • Your file downloads. Open it to confirm the data matches what you saw locally.

What can I do with this data?

The CSV format opens directly in spreadsheet apps like Google Sheets or Excel. The JSON format is useful for feeding data into other programs or APIs.

On the free plan, dataset results are retained for 7 days. After that, you'd need to re-run the Actor to regenerate them.

Your scraper runs from anywhere now

You can share the Actor URL with others, trigger it via API, or set up a schedule to re-scrape automatically. All without touching your local machine again.

Secret mission

Crawl All 50 Pages

Your scraper grabs 20 books from a single page. The real catalog has 1000 books spread across 50 pages. Can you teach your crawler to follow pagination links and scrape them all?

Clean Up Your Resources

Clean Up Your Resources

This project uses Apify's free tier with no subscription cost or credit card requirement. Your Actor only consumes compute credits when you actively trigger a run, so there are no ongoing charges while it sits idle.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Actor named book-scraper deployed on the Apify platform.
  • Dataset results stored on Apify (retained for 7 days on the free plan).
  • Local book-scraper project folder on your Desktop.
  • Apify CLI installed globally via npm.

Keep everything running

No action needed. Choose this if you're still actively building or want your Actor ready for future runs.

  • Your Actor stays deployed and ready to run whenever you trigger it.
  • You only consume compute credits when you actually run the Actor. On the free plan, you get $5 in monthly credits.
  • Dataset results are retained for 7 days on the free plan, then automatically deleted.
  • The Apify CLI remains installed on your machine for future projects.

Pause - I'll come back to this later

No infrastructure to shut down. Actors only run when you trigger them, so there is nothing to pause on the cloud side.

  • Your Actor remains deployed but idle. It uses zero credits until you run it again.
  • Your local project files stay in the book-scraper folder on your Desktop.
  • If you want to free up disk space, delete the storage/datasets/default/ folder (the ~1000 JSON files). You can regenerate them later with apify run.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow each item below to fully clean up.

  • Delete your Actor from the Apify Console:
  • Delete your local project folder. On macOS/Linux, run:
cd ~/Desktop
rm -rf book-scraper

On Windows, run:

cd ~\Desktop
Remove-Item -Recurse -Force book-scraper
  • Uninstall the Apify CLI globally by running:
npm uninstall -g apify-cli

That's a Wrap!

That's a Wrap!

Nice work! You just built and deployed a fully functional web scraper that extracts structured book data from a live website and stores it in the cloud.

You've learned how to:

  • Write a web scraper using CheerioCrawler and Cheerio selectors to extract titles, prices, and star ratings from a live website.
  • Deploy a serverless Apify Actor to the cloud that you can trigger on demand from anywhere, without your computer running.
  • Export scraped datasets as JSON or CSV, ready to open in a spreadsheet or feed into another tool.
  • Secret Mission: Added multi-page pagination crawling with enqueueLinks to scrape all 1000 books across 50 pages.

Ready to quiz yourself?