Deploy a Scraper on Every Push

Connect an Apify scraper to Git and rebuild it with every push.

Introduction

30 Second Summary

A code change feels finished only when the live version reflects it. Manual rebuilds make that last handoff easy to miss.

In this project, you will create an Apify Actor backed by its own Git repository. A small scraper change will prove the deploy-on-push loop by triggering a fresh cloud build.

What You'll Build

A single repository push will produce a fresh cloud build whose next run returns the dataset field you just changed.

By the end of this project, you'll have:

A cloud scraper that turns your chosen URL into rows in a dataset table.
A repository-backed project whose starter scraper code is ready for your changes.
A visible deploy-on-push loop where each commit creates a fresh build before the next run uses your update.
Secret Mission: An optional challenge to push your skills further.

Are there any prerequisites?

You need an Apify account plus an account with a supported Git provider. The project runs in your browser with no local installation.

Before We Start

The deploy-on-push loop needs two online homes.

The Apify Console runs your Actor. GitHub holds the repository that triggers each rebuild.

This step gets your browser ready. It also confirms both accounts can sign in.

That preparation keeps account setup from interrupting your first deployment.

In this step, get ready to:

Verify your browser can reach the Apify Console.
Create or access your Apify account.
Prepare a GitHub account for repository access.

Confirm your browser is ready

The whole workflow runs in browser tabs. A successful page load proves your browser can reach the service.

Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
Type the name of your installed web browser into the search field.
Press Enter to open the browser.
Open the official Apify Console page in your browser.

You should see an Apify page without a browser connection warning.

Can't reach the Apify Console?

Check that another website loads in the same browser.
Disable any browser extension that blocks the page.
Try the page in a private browser window.

Still stuck? Help me troubleshoot access to the Apify Console

Create or access your Apify account

The Apify Console is the cloud workspace where you create the scraper. It also shows every build log and run log later in the project.

Your password stays inside the official account form. Never paste it into this project.

✔️ I already have an Apify account

Visit the official Apify sign-in page.
Enter the email address linked to your Apify account.
Enter your Apify password.
Complete any authentication check requested by your account.

You should arrive inside the Apify Console under your account.

ⓧ I need to create an Apify account

A new account needs email verification before you can use the Apify Console.

Visit the official Apify sign-up page.
Enter your email address in the account form.
Create a unique password for your Apify account.
Submit the sign-up form.
Open the verification message in your email inbox.
Select the verification link in the message.
Return to your Apify Console browser tab.

You should arrive inside the Apify Console under your new account.

You're in. The Apify Console is now your cloud workspace for the scraper.

Can't access your Apify account?

Check your inbox for the Apify verification message.
Check your spam folder for the verification message.
Use the account recovery option on the sign-in page if your password is rejected.

Still stuck? Help me access my Apify account

Prepare your GitHub account

The repository provider is the second half of the deployment loop. Apify requests permission to create a repository there in the next step.

GitHub requires a verified email address for basic repository tasks. Finishing verification now keeps the authorization flow moving.

✔️ I can sign in to GitHub

Open the official GitHub sign-in page in a second browser tab.
Enter your GitHub username or email address.
Enter your GitHub password.
Complete two-factor authentication if your account requests it.

Your GitHub home page should load under your account.

ⓧ I need a GitHub account

The sign-up flow collects your account details before verifying your email address.

Open the official GitHub sign-up page in a second browser tab.
Enter your email address.
Create a unique password for your GitHub account.
Choose a GitHub username.
Submit the sign-up form.
Open the verification message in your email inbox.
Select the verification link in the message.
Return to your GitHub browser tab.

Your verified GitHub account should open under your chosen username.

That's both sides ready. Apify can run your cloud program while GitHub holds its source code.

Can't finish GitHub setup?

Check that you completed the email verification step.
Check your spam folder for the GitHub verification message.
Use the account recovery option if GitHub rejects your password.

Still stuck? Help me prepare my GitHub account

Before you check, do you expect both account tabs to stay signed in when you switch between them?

Switch to the Apify Console tab.
Confirm that the console navigation is visible.
Switch to the GitHub tab.
Confirm that GitHub shows your signed-in account controls.

You should see the Apify Console in one tab. You should see your signed-in GitHub account in the other.

Your browser and accounts are ready. Next up, you'll choose your scraper type and see which Actor template Apify matches to it.

Choose Your Actor Template

Every later push depends on choosing the right starting point. The Apify Console uses your workload choice to narrow the Actor templates.

In this step, you'll choose a JavaScript web scraper. You'll review the matched template before preparing it for repository setup.

In this step, get ready to:

Open the Actor creation flow.
Match a JavaScript web-scraper template.
Prepare push-to-deploy-scraper for its Git connection.

Open the Actor creation flow

The creation flow asks a few questions before it suggests any starter code. Your answers define the kind of cloud program you want to create.

Go to the official Actor creation page in your browser.
Sign in to Apify Console with your existing account.

You'll see the first question, What do you want to build?.

Match the web scraper template

Apify matches templates using the job type first. Your language choice then narrows the result to compatible starter code.

Select Web scraper under What do you want to build?.
Select JavaScript under Which language do you prefer?.

You'll reach Here's your best match. The screen shows templates matched to your two choices.

Review the description in the matched template preview.
Confirm that the preview describes a JavaScript web scraper.

Why review the match?

The matched template gives you working scraper code aligned with JavaScript. Reviewing it now confirms that your future repository starts with scraper code.

Don't see a web-scraper match?

Return to What do you want to build?.
Select Web scraper.
Return to Which language do you prefer?.
Select JavaScript.

Still stuck? Help me find the matching JavaScript web-scraper template in Apify Console. You can also ask the Apify community.

Prepare the Actor for Git

The next stage decides where the source code lives. This project pauses before provider authorization so you can verify the Actor configuration first.

Select the matched template shown in its preview.

You'll reach Where do you want to host your code? with a Git option.

Enter push-to-deploy-scraper in the Actor naming field.

Before you check the setup, what name should appear with the matched JavaScript scraper?

Confirm that the Actor name reads push-to-deploy-scraper.
Confirm that the matched template uses JavaScript.

You'll see push-to-deploy-scraper paired with the JavaScript web-scraper template. The code-hosting question remains unanswered.

Leave Git unselected.

No provider authorization window should open. No repository should exist yet.

Your Actor setup

Created in the Apify Console from a JavaScript web-scraper template. Git repository connection has not yet been authorized or created.

You have the starting point locked in: push-to-deploy-scraper now carries the matched JavaScript scraper template. Next, you'll authorize your Git provider so Apify can create the repository.

Connect Your Git Repository

The matched JavaScript template gives your Apify Actor a starting point. Its source still needs a repository connection before a push can trigger a cloud build.

In this step, you connect the Apify Console to your Git provider through OAuth. That connection turns every future push into an automatic Actor build.

In this step, get ready to:

Choose the Git provider that hosts your repository.
Authorize Apify to create the repository in your provider account.
Verify the starter code and automatic build connection.

Choose where your code lives

A Git repository gives the template a source of truth. Apify can watch that source for pushed commits.

Switch back to the matched template preview in Apify Console.
Continue from the template preview to Where do you want to host your code?.
Select Git.
Select the provider for your existing account from GitHub, GitLab, or Bitbucket.

Why Git for this Actor?

Hosting the source with Git adds version history. Each commit becomes a checkpoint you can review or restore.

The same connection gives Apify a build trigger. A push becomes a new Actor build without separate CI configuration.

Authorize your provider account

Connecting accounts can feel sensitive. Your provider shows the requested permissions before you approve them.

Follow the provider redirect from Apify Console.
Sign in to your Git provider if prompted.
Review the permissions requested by Apify.
Approve the authorization request.

Your browser returns to the Actor creation flow after the provider approves the connection.

Select the account or organization that should own the repository.
Confirm that the repository name shown in the flow is push-to-deploy-scraper.
Complete the Git connection using the final confirmation shown in the flow.

Apify creates a private repository named push-to-deploy-scraper. It pushes the JavaScript web-scraper starter template into that repository.

The connection also configures Apify to build the Actor whenever a commit reaches the repository.

Authorization did not complete?

Return to the provider authorization page if repository creation fails.
Grant the repository access requested by Apify.
Ask your organization owner to approve the integration if your organization restricts connected applications.

Still stuck? Help me troubleshoot the Apify Git provider authorization flow. You can also ask in the official Apify and Crawlee Discord.

That is the account handoff complete. Your scraper source now has a repository under your control.

Open and verify the repository

A repository is useful once the expected source reaches its default branch. This check proves that Apify created more than an empty repository.

Before you open it, what evidence would show that Apify pushed the web-scraper template?

Open the new repository using the link shown in Apify Console.
Confirm that the repository name is push-to-deploy-scraper.
Inspect the file list on the repository's default branch.

You should see multiple JavaScript starter files on the default branch. The commit history should contain the template pushed by Apify.

Before you check the Actor, where do you expect its source to point?

Switch back to the push-to-deploy-scraper Actor in Apify Console.
Select the Source tab.
Confirm that the source type shows Git repository.
Confirm that the linked repository is push-to-deploy-scraper.

You should see Git repository as the Actor source. The linked repository should be push-to-deploy-scraper.

That is the deploy-on-push loop wired up. Every pushed commit now triggers an automatic build of your Actor.

Repository connection not showing?

Check that you returned to the push-to-deploy-scraper Actor.
Refresh the Source tab if the connection is still processing.
Repeat the provider connection if the source type still does not show Git repository.

Still stuck? Help me verify that my Apify Actor is connected to the correct Git repository. You can also ask in the official Apify and Crawlee Discord.

Your repository now owns the scraper source. Next, you will watch Apify turn that starter commit into the Actor's first cloud build.

Verify the First Build

Your repository now holds the starter scraper. Apify still needs to package that source into a runnable Actor build.

In this step, you'll follow the initial automatic build to completion. You'll inspect its log to confirm that the starter template produced a runnable cloud build.

In this step, get ready to:

Locate the automatic build triggered by the starter-template push.
Read the build log and confirm the build finished successfully.

Find the automatic build

A Git-triggered build turns each pushed commit into a new Actor version. The starter-template push has already triggered this first build.

Switch back to the Apify Console tab from earlier.
Select Builds in the navigation for push-to-deploy-scraper.

You should see an initial build associated with the starter-template commit.

Check the newest build's current status.

Expect the first build to take a few minutes while Apify packages the template. An in-progress status means the build is still working.

Wait for the status to reach a final state if the build is still in progress.

Inspect the build log

The build log records how Apify turns the repository source into a runnable cloud package. Reading it gives you evidence that the connection handled the starter code cleanly.

Select the newest build entry to open its details.

You should see the build details alongside a chronological log.

Scroll through the build log to look for any failed stage.
Continue to the final entries in the log.

Before you check the final result, do you think the repository connection produced a runnable build on its first attempt?

Check the build's final status at the top of its detail page.

You should see the initial build marked as successful. The build log should finish without a failed stage.

Great work. push-to-deploy-scraper now has a current build that Apify can run.

Why This Build Matters

The build log proves that Apify received the starter-template commit. The successful final state confirms that the Actor package is ready to run.

The repository connection remains active. Each future push triggers another automatic build.

Build Not Finishing Successfully?

Refresh the build detail page once if its status has stopped updating.
Read the earliest failed stage in the log to find the first reported cause.
Confirm that the default branch in the repository tab from earlier still contains the starter template.

Still stuck? Help me diagnose my failed initial Apify Actor build

Your starter code is now a runnable cloud build. Next, you'll give push-to-deploy-scraper a URL. Its first run will turn a webpage into dataset rows.

Run Your Web Scraper

The successful build from the previous step proves your Apify Actor is ready to run. A build alone has not scraped a page.

Now you will supply a webpage to the current build. You will follow that execution from the run log to the Dataset rows it creates.

In this step, get ready to:

Provide the Actor with a public webpage you choose.
Read the run log to confirm the scrape completed.
Inspect the collected rows in the default Dataset.

Choose a page and start the run

An Actor run combines the latest successful build with the input you provide. The JavaScript starter reads one page.

Each heading becomes a row in the run's default Dataset. Choose a page with several visible headings so the result is easy to verify.

Switch back to the Apify Console browser tab showing the build log for push-to-deploy-scraper.
Select the Input tab.
Visit a publicly accessible page with several visible headings in a separate browser tab.
Copy its full URL from the browser address bar.
Return to the Input tab in the Apify Console.
Replace the value in URL of the page with the URL you copied.

Before you start the run, do you expect the scraper to find one heading or several headings on your chosen page?

Select Start to run the current Actor build.

The Console opens the new run. Its log begins updating as the Actor fetches your chosen page.

This one-page run can remain active for a few seconds while the page is fetched. A short pause here is normal.

Read the run log

The run log records what the Actor does during execution. Its final entries confirm whether the current build completed the scrape cleanly.

Wait for the run to finish.
Select the Log tab.
Scroll to the final log entries.

You should see that the run completed. The final entries should contain no failure details.

Did the run fail?

Return to the Input tab.
Replace the current value with the complete address of a publicly accessible page.
Select Start again.

Some websites block automated requests. Help me diagnose why my Apify Actor could not scrape the URL I entered.

Inspect the dataset rows

The run log proves the program executed. The default Dataset shows the structured data it collected.

Before you open the Dataset, how many rows do you expect based on the headings you saw on your chosen page?

Select the Storage tab on the run page.
Select Dataset under the run's storage options.
Select Overview to display the results as a table.

You should see rows in a table with Level and Text columns. Each row represents one heading collected from your chosen page.

Is the Dataset empty?

Some pages create their visible content inside the browser after the initial HTML loads. The starter scraper needs headings in the HTML returned by the page.

Return to the Input tab.
Replace URL of the page with the address of another public page that contains visible headings.
Select Start to run the scraper again.

Still seeing no rows? Help me find out why my Apify Dataset is empty.

You have a working cloud scraper now. Your current build turned a live webpage into structured Dataset rows.

Your manual run proves the starter code works in the cloud. Next, you will change what the scraper collects and push that commit so Apify builds the new version automatically.

Push a Scraper Update

Your first Apify Actor run proved that the scraper can collect rows from a URL. The repository connection now has to prove that a code change reaches the running Actor.

You will rename one collected output field in the connected repository. A push through Git should trigger a fresh build without a workflow file or manual deployment.

In this step, get ready to:

Change one collected output field in the repository.
Confirm the push creates a successful Apify build.
Run the latest build against the same URL.

Change one collected field

Each row in your dataset is a JSON object. Each property key becomes a dataset column.

A renamed key gives you a clear marker for the updated scraper. Keeping its value expression intact preserves the data being collected.

Return to the initial dataset from your earlier run.
Choose one column whose name clearly appears in the collected rows.
Record its current name here: your current dataset field.
Set a unique new name using letters only here: your changed dataset field.
Switch back to the repository push-to-deploy-scraper from earlier.
Search the repository source for your current dataset field.
Open the source result where that name appears as a property key in the collected row.
Enter the browser editor for that file.

Change the key, keep the value

The property key controls the column name in the dataset. The existing value expression controls what the scraper reads from the page.

Renaming the key creates an observable output change without altering the scraper's extraction behavior.

Rename only the selected property key to your changed dataset field.
Review the diff to confirm that one property key changed.
Submit the commit directly to the displayed default branch with the message Change scraper output field.

Good work. Your repository now shows the scraper change on the branch that Apify watches.

A browser commit to the default branch is already pushed. Apify can now react to it.

Can't find the field in source?

Search for another column name from the initial dataset.
Choose a source match inside the object that creates a collected row.
Confirm the commit landed on the repository's default branch.

Still stuck? Help me locate and rename one dataset field in my Apify JavaScript scraper

Watch the push trigger a build

The repository now holds the update. The next proof is a new cloud build created from that commit.

Before you check the build history, predict whether Apify has already noticed the commit.

A cloud build can remain in progress for a few minutes. Apify is preparing the new runnable version during that wait.

Switch back to the Apify Console from earlier.
Return to the Actor push-to-deploy-scraper.
Open the Actor's build section.
Refresh the build list until a new entry appears for your recent commit.
Open the new build's log.
Wait for the build to reach a successful status.

You should see a successful build created after the scraper update. Its source details should point to the commit you just pushed.

That is the key proof. Your push produced a fresh runnable build without a manual deployment.

Don't see a new successful build?

Confirm the update appears on the same default branch connected to the Actor.
Refresh the Actor's build list after confirming the commit.
Use the first failure shown in the build log to inspect the renamed property key.

Need a hand? Help me diagnose why my pushed commit did not create a successful Apify build

Run the latest build

A successful build proves that the code deployed. A fresh dataset proves that the changed scraper is now running.

Before you start the run, predict which field name the new rows should contain.

Return to the Actor's run page.
Select latest in the Build option.
Reuse the same learner-chosen URL from the earlier successful run.
Click Start.
Select the Log tab after the run begins.

You should see the run finish successfully against the latest build. The log confirms that the updated Actor completed its scrape.

Open the Storage tab for the completed run.
Select Dataset.

You should see rows from the same URL with a column named your changed dataset field. The previous field name should no longer identify that collected value.

Still seeing the old field?

Refresh the dataset after the run finishes.
Confirm the Build option was set to latest.
Start one more run from the current build if the previous run used an older build.

Need help tracing the result? Help me find why my updated Apify dataset still shows the old field

You have closed the deployment loop. A repository commit now rebuilds your Actor automatically and changes the rows produced by its next run.

Secret mission

Trace a Change from Commit to Dataset

Your deploy-on-push loop works. Trace the changed scraper field from the repository through the successful Apify build to the dataset that proves it ran.

Clean Up Your Resources

The small template runs in this project fit within Apify's free plan. Choose whether to keep the connection active, leave the Actor idle, or delete the cloud resources before future runs consume more plan credits.

Resources you used:

The Apify Actor push-to-deploy-scraper. It contains the current build plus the build history and run history.
Two default datasets containing the rows collected by the initial and updated scraper runs.
The Git repository push-to-deploy-scraper under your provider account.
The Git provider authorization that lets Apify build the Actor after each push.

Keep everything running

No action is needed. Choose this if you want to keep the scraper ready for more deploy-on-push experiments.

Keep the Actor connected if you want future pushes to trigger builds.
Start the Actor only when you want to run another scrape.
Review your Apify plan credits before running larger scraping jobs.
Keep the repository as the source history for your scraper.

Pause - I'll come back to this later

Leave the Actor idle to preserve your build history and datasets. An idle Actor does not need another run until you return.

Avoid starting a new Actor run.
Avoid pushing commits to the default branch because each push triggers another build.
Keep both default datasets for comparing the original output with the changed field.
Start a new Actor run when you return.
Push a new commit when you want Apify to create a fresh build.

Delete - I don't want to use this again

Remove all project resources and start fresh. Deletion is permanent, so save any code or results you want to keep first.

The two datasets hold the rows collected during your test runs. Delete them before the Actor so they remain easy to locate.

Return to the updated run results from earlier in the Apify Console.
Use the default dataset's deletion control.
Return to the run history for push-to-deploy-scraper.
Select the initial completed run.
Return to its default dataset.
Use the dataset's deletion control.

The Actor holds the hosted scraper plus its build and run history. Deleting it also removes the automatic build connection for this Actor.

Return to the Actor push-to-deploy-scraper in the Apify Console.
Use the Actor's deletion control.
Confirm the permanent deletion.

Your Git repository belongs to your provider account. It remains available after the Actor is deleted.

Switch to the repository push-to-deploy-scraper in your Git provider.
Use the repository settings to permanently delete it.
Complete the provider's confirmation prompt.

The Git provider authorization can remain active after the project resources are gone. Revoke it if you authorized Apify only for this test.

Switch to the authorized applications area in your Git provider account.
Select the Apify connection.
Revoke its authorization.

Nice Work!

You did it! You built a cloud scraper as an Apify Actor.

Each push to its connected Git repository now triggers a fresh Actor build.

You've learned how to:

Create a cloud-hosted scraper as an Apify Actor. Run it on demand against a URL you choose.
Inspect the build log to confirm the starter template deployed cleanly. Read the run log to trace the scraper's cloud execution.
Edit a collected output field in the repository. Push the commit to trigger an automatic build. Confirm the updated dataset includes the changed field.
Secret Mission: Take on an optional challenge to push your deploy-on-push skills further.

Ready to quiz yourself?