Re-Embed Only Changed Pages

Build a crawler that embeds only changed documentation pages.

Introduction

30 Second Summary

A tiny edit to one help page should trigger a tiny update. Reprocessing every untouched page turns that change into an expensive nightly habit.

In this project, you will build a pipeline that crawls a documentation site into a searchable vector collection. Each later run uses a content hash to embed changed pages while skipping unchanged ones.

What You'll Build

Your finished automation shows each untouched page being skipped in the run log while a plain-English search returns the new wording from the page you edited.

By the end of this project, you'll have:

A searchable docs collection that answers a plain-English question with the matching source page.
An incremental update run that skips unchanged pages while embedding the page whose text changed.
Side-by-side credit totals that reveal the savings from processing one changed page instead of the full site.
An optional Secret Mission to push your skills further.

Are there any prerequisites?

This project requires free-tier accounts with Apify, Qdrant, and n8n. You do not need a local toolchain because every step runs in the browser.

Before We Start

A scheduled crawler depends on several services being able to reach each other. A missing login or credential would stop the first crawl before it produces a dataset.

This step prepares Apify, n8n, Qdrant, plus OpenAI. You also choose a small documentation site whose source you can edit.

In this step, get ready to:

Select a small documentation site that you can edit.
Access the four service accounts used by the pipeline.
Prepare the credentials required for later connections.

Choose an editable documentation site

The later comparison depends on changing one source page between crawls. Your source therefore needs public pages that the crawler can read plus content that you control.

Choose a documentation site with a small set of public pages.
Confirm that you can edit at least one page on the site.
Open the documentation homepage in your browser.
Record the homepage URL here: https://your-docs-site.example.

You should see the documentation homepage load without requiring a private login. This proves Apify can reach the same public source in the next step.

Access the four service accounts

Each platform handles one part of the pipeline. Apify crawls the pages. n8n coordinates the process. Qdrant stores the vectors. OpenAI creates the embeddings.

Access your Apify account through Apify Console.
Access your automation workspace through the n8n Cloud setup page.
Access your vector database account through Qdrant Cloud.
Access your embedding provider account through the OpenAI Platform.

Good progress. You now have a dashboard for every service that moves the documentation from a web page into searchable vectors.

Can't access one of the services?

Complete any account verification message sent to your email address.
Return to the official service link after completing verification.
Use help me resolve this account access problem if a dashboard still does not load.

Prepare credentials and connection details

An API key lets one service make authenticated requests to another. The pipeline needs crawler access plus vector database access plus embedding access.

Protect Your Credentials

Creating credentials is fiddly because some secret values appear only once. A password manager keeps them available without placing them in project notes or screenshots.

The OpenAI Platform may ask you to configure billing before a key can make requests. Your account shows the current terms before you approve anything.

Create an Apify token from the official API & Integrations page.
Store the Apify token in your password manager.

The API & Integrations page should now list the token entry. Keep the secret value outside any screenshots.

Create a Qdrant Free cluster named your Qdrant cluster name by following the official cluster guide.
Store the generated Qdrant API key in your password manager.
Record the cluster endpoint here: https://your-cluster.qdrant.io:6333.

Qdrant shows the generated key once. Provisioning can take some time while the cluster moves into its ready state.

Create an OpenAI API key using the key creation control on the official API keys page.
Store the OpenAI API key in your password manager.
Create an Apify account credential in your n8n workspace using the stored Apify token.

Your n8n credentials list should now contain the Apify connection. Your Qdrant endpoint plus both remaining keys are ready for the workflow steps that use them.

Verify your project access

A complete setup has one visible success signal in every service. This final check catches missing access before the crawler starts.

Before you check, consider which dashboard is most likely to show a missing connection.

Return to Apify Console from earlier to confirm that the token entry is listed.
Return to your n8n workspace to confirm that the Apify account credential is listed.
Return to Qdrant Cloud to confirm that the cluster shows a ready state.
Return to the OpenAI API keys page to confirm that the key entry is listed.
Return to https://your-docs-site.example to confirm that the documentation homepage still loads publicly.

That's the setup complete. Your crawler platform plus automation platform plus vector database plus embedding provider are accessible for the selected documentation site.

✔️ Everything is ready

Crawler platform, automation platform, vector database, and embedding provider access configured for the selected documentation site.

ⓧ I'd like to double check the full setup

Confirm that your project setup includes every item below:

An accessible Apify account with a stored API token.
An accessible n8n workspace with an Apify account credential.
A ready Qdrant cluster with its endpoint recorded.
An OpenAI API key stored securely outside the project.
An editable documentation site with a publicly accessible homepage.

Your services can now work together. Next up, you'll crawl the selected documentation site and inspect the captured page text.

Crawl the Documentation Site

The source site is selected, but its pages still live outside your pipeline. Your later search workflow needs page-level records it can process.

Apify's Website Content Crawler turns those pages into a dataset. In this step, you'll run it once.

You'll then inspect the captured page addresses and extracted text.

In this step, get ready to:

Set the selected documentation site as the crawler's starting point.
Complete a crawl that stores one record per captured page.
Inspect the captured page URLs and extracted text.

Run the first crawl

A starting URL gives the crawler its entry point. The crawler follows linked pages beneath that address.

This crawl consumes a small amount of platform usage. Your selected documentation site keeps the run contained.

Return to the Apify Console from earlier.
Use the Actor search to find Website Content Crawler.
Select Website Content Crawler by Apify.
Select the Input tab.
Paste your selected documentation site URL into Start URLs.
Click Start.

How Does the Crawl Stay Focused?

A starting URL aimed at the documentation section keeps unrelated pages outside the crawl.

Each captured page becomes a separate dataset record. That structure gives the later automation one stable unit to process.

A small documentation site can take a few minutes to crawl. A quiet run status during that wait is normal.

Wait for the run status to show that the crawl completed.
Confirm the completed run reports at least one stored item.

That's the first crawl complete. Your source now exists as page-level records that you can inspect.

Inspect the crawl dataset

The item count proves that the crawler stored data. The fields inside each record reveal whether that data contains useful page content.

Before you open the first record, which page address and text do you expect to recognize?

Select the Output tab.
Open the first dataset record.
Locate the url field.
Confirm the url belongs to your selected documentation site.
Locate the text field.
Read the first few lines in text.
Confirm those lines contain a recognizable page heading or body copy.
Inspect a second record to confirm it represents another captured page.

You should see one dataset record for each captured page. Every inspected record should contain its page address in url plus readable page content in text.

Why Check the Text Now?

This inspection catches a crawl that wandered into an unrelated section. It also reveals records filled with navigation instead of article copy.

Clean page-level records give the later automation a reliable source.

Only Seeing One Page or Empty Text?

Confirm Start URLs points to the top of the documentation section.
Check that the linked documentation pages sit beneath the starting URL path.
Confirm the source pages are publicly accessible when the text field is empty.
Help me diagnose the crawl results.

Your documentation is now captured as page-level records. Next, you'll prepare a searchable collection for those records.

Embed the First Crawl

The crawler has turned your documentation site into page records you can inspect. An embedding turns each page into numbers that semantic search can compare.

You'll use n8n to send every page through your embedding provider. Qdrant stores one point per page.

In this step, get ready to:

Pull every record from the initial Apify crawl into the workflow.
Generate one vector per page with your existing embedding provider.
Upsert those vectors into Qdrant with URL plus page text metadata.

Connect the crawl dataset

The dataset is your source of truth for this first load. Its total gives you the point count that Qdrant must reach.

Return to the Apify dataset from earlier.
Record its total item count here: your initial crawl page count.
Create a workflow for the initial embedding load in the n8n workspace.
Add a dataset input step that uses your existing Apify connection.
Select the completed dataset from earlier as the input.
Configure the dataset input step to return every record.
Run the dataset input step.

You should see one workflow item per crawled page. Each item should contain a page URL plus extracted page text.

That connection is live. Your crawl now flows into the automation one page at a time.

Why capture the page count?

Matching counts proves that every source page reached the collection once. This baseline makes later content changes measurable.

Missing pages from the workflow?

Confirm that the dataset input points to the completed crawl from earlier.
Check that the input step returns the complete dataset instead of a limited preview.
Ask for help with the dataset connection.

Generate one embedding per page

Your embedding provider converts each page's extracted text into a vector. Keeping one input page tied to one output vector preserves the count you recorded.

Add an embedding step after the dataset input step.
Select your existing embedding provider connection.
Map the extracted page text into the embedding input.
Configure the step to produce one output item for each incoming page.
Run the embedding step.

You should see the same number of output items as input items. Each output should include a vector generated from that page's text.

That is the transformation working. Every crawled page now has a numerical representation ready for storage.

What does the embedding step preserve?

Vectors give Qdrant a form it can compare by meaning. The original URL stays attached as metadata.

The page text also stays attached as metadata. A later search result can show the matching source content.

Embedding fewer pages than expected?

Confirm that the embedding step receives every item from the dataset input.
Check that extracted page text is mapped as the embedding input for each item.
Ask for help tracing the missing items.

Store one vector point per page

An upsert creates a point for a new identifier. It replaces the existing point when that identifier returns.

A stable identifier derived from the page URL gives each source page a consistent home in the collection.

Add a Qdrant destination step after the embedding step.
Select your existing Qdrant connection.
Test the destination connection.

You should see a successful connection result. The destination can now reach your vector database.

Enter a name for the new collection here: your vector collection name.
Enable automatic collection creation for the first successful write.

The destination settings should now show the collection name with automatic creation enabled.

Map the embedding output into the point vector.
Derive a stable point identifier from the page URL.
Preview the first mapped point.

The preview should show one vector. It should also show an identifier derived from that page's URL.

Map the page URL into the point metadata.
Map the extracted page text into the point metadata.

Why keep both metadata fields?

The URL identifies where the page came from. The text lets later searches return the content that matched the question.

A multi-page run can take a little while because each page calls the embedding provider.

Predict whether the final point count will match your initial crawl page count.
Execute the workflow from the dataset input through the Qdrant destination.
Wait until every workflow item finishes.

You should see one successful write per input page. The first successful write also creates the collection.

Return to the Qdrant collection named your vector collection name.
Read the collection's point count.
Compare the point count with your initial crawl page count.

The two counts should match exactly. That's your first full load complete: every crawled page now has one vector point with its URL plus page text.

Point count does not match?

Confirm that the collection was empty before this initial workflow run.
Check that every page URL produces a distinct stable point identifier.
Ask for help comparing the workflow items with the collection points.

Your collection now has a complete vector baseline. Next, you'll ask it a plain-English question to see which page it retrieves.

Search the Collection

Your first crawl is now stored as one vector point per page in Qdrant. The matching point count proves that every captured page reached the collection.

Now you need evidence that those vectors can lead you back to useful source material. You will use n8n to run a similarity search from a plain-English question.

In this step, get ready to:

Choose a question with an answer in the initial crawl dataset.
Configure a direct search against the existing vector collection.
Review the matched page URL and page text.

Choose a question with a known answer

A useful search test starts with an answer you can verify. Choosing a known source page gives you a clear way to judge the match.

Switch back to the initial crawl dataset from earlier.
Find a captured page with a distinctive topic in its page text.

You now have a source page that can act as the expected match.

Phrase a plain-English question that the selected page can answer.
Record the question here: your plain-English question.

Configure a direct similarity search

The search node uses an embedding model to convert your question into a vector. Qdrant compares that vector with the stored page vectors to find the closest match.

Why query the collection directly?

The Get Many mode runs retrieval without adding an AI agent. This keeps the check focused on whether the stored vectors return the right source page.

The raw result keeps the matched URL visible. It also keeps the source text and similarity score visible.

Switch back to the initial embedding automation in n8n.
Use the node picker on the n8n canvas to add a Qdrant Vector Store node.

You should now see the new Qdrant Vector Store node on the canvas.

Set the Operation Mode field to Get Many.

The node now shows the fields needed for a direct search, including Prompt and Limit.

Select your existing Qdrant connection in the credential selector.
Select the collection from earlier in the Qdrant collection name field.

Seeing the existing collection in the node confirms that n8n can reach your stored vectors.

Connect the same embedding model node used by the initial embedding path to the search node's embedding input.
Enter your plain-English question in the Prompt field.

Using the same embedding model places the question in the same vector space as the stored pages.

Set the Limit field to 1.

Run the search and inspect the match

Before you run the search, which page do you expect Qdrant to rank first?

Run the Qdrant Vector Store node from the n8n canvas.

You should see one result containing a similarity score. The result should also include page text plus metadata containing the source URL.

Expand the first item in the node output panel.
Review the matched page URL in the result metadata.
Read the returned page text in the result.
Compare the matched page with the source page in the initial crawl dataset.

The matched URL should point to a page that answers your question. Its returned text should contain the relevant source material.

No useful match?

Confirm that the search node points to the collection populated by the initial crawl.
Confirm that the search node uses the same embedding model as the initial embedding path.
Check that your question can be answered from the text captured in the initial crawl dataset.
Ask for help with the search configuration: Help me diagnose my similarity search.

That closes the retrieval loop. Your plain-English question now leads back to a page from the crawl.

Next, you will edit one source page and run the crawler again so there is a real change to track.

Update and Re-Crawl a Page

Your vector collection already answers questions from the first crawl. Now you need a second version of the source so you can see what changed.

An incremental pipeline depends on comparing two snapshots of the same pages. You will edit one page before running the website crawler again.

In this step, get ready to:

Edit one page on the source documentation site.
Run the crawler again to capture the updated page text.
Compare the matching page record across both datasets.

Edit one source page

A controlled edit gives you a known difference between the two crawls. A distinctive sentence makes that difference easy to find.

In the initial crawl dataset from earlier, choose one page that you can edit on the source documentation site.
Record its page URL here: URL of the page you edited.
Visit URL of the page you edited in your browser.

You should see the current public version of the page. This gives you a baseline for the edit.

In your documentation editor, open the editing view for that page.
Add This page was updated for the incremental crawl test. to the end of one existing paragraph.
Make the edited page live through the publishing workflow you use for the documentation site.
Refresh URL of the page you edited in your browser.

You should see the new sentence on the public page. That controlled change is now ready for the crawler to capture.

Don't see the edited sentence?

Confirm that your documentation editor shows the latest saved version of the page.
Repeat the publishing workflow if the edit is still waiting to go live.
Reload the public page after the publishing workflow finishes.
Help me find why my documentation page is still showing its old text.

Run the second crawl

The second crawl creates a fresh dataset for comparison. The existing vector collection remains your original baseline.

Before you start the second crawl, consider whether the record count or the page text will reveal your edit.

Switch back to the crawler run page from earlier.
Start a new run without changing the existing crawl configuration.

When the run finishes, you should see a second completed crawl with its own dataset. You now have the updated snapshot needed for comparison.

Open the dataset for the second completed crawl.
Use the page URL field to locate URL of the page you edited.

The matching record should contain This page was updated for the incremental crawl test. in its extracted page text.

Is the updated text missing?

Confirm that the public page contains the new sentence.
Check that the second crawl started after the page became public.
Confirm that the record URL matches URL of the page you edited.
Help me work out why the second crawl captured the old page text.

Compare both crawl datasets

A side-by-side comparison connects the same URL across two points in time. This isolates the edited content from every unchanged page.

Before you compare the records, decide which parts should stay identical. Consider which part should differ.

Keep the second crawl dataset open in one browser window.
Switch back to the initial crawl dataset from earlier in another browser window.
Arrange the two browser windows side by side with your computer's window controls.
Locate URL of the page you edited in the initial dataset.
Locate URL of the page you edited in the second dataset.
Compare the total record count shown for each dataset.

You should see the same page URL in both records. Only the second record should contain the new sentence.

The total record counts should match because you edited an existing page. That isolates one content change without adding another page.

What does this comparison prove?

Each crawl stores a complete snapshot of the documentation site. The matching URL connects the original page text to its updated version.

The automation still needs a way to identify that single changed record. Your side-by-side evidence defines the exact job for the next step.

Your two datasets now reveal one controlled content change. Next, you will make the automation detect that difference so unchanged pages can be skipped.

Skip Unchanged Pages

Your second crawl is ready. You can see the edited page beside the first run.

The existing automation still sends every record through embedding. A scheduled run would repeat work for identical text.

A content hash gives each page a fingerprint based on its extracted text. This step adds a gate that lets only changed pages reach the existing embedding path.

In this step, get ready to:

Define how each page is recognised across crawls.
Build a hash gate that bypasses unchanged pages.
Verify that only the edited page reaches the existing embedding-to-upsert path.

Choose the comparison fields

The page URL gives the automation a stable identity across crawler runs. The extracted page text provides the signal that reveals a content change.

Switch back to the side-by-side crawl datasets from earlier.
Locate one page that you did not edit in both datasets.
Compare the page URL across those records.
Compare the extracted text across those records.
Locate the edited page in both datasets.
Compare the page URL across the edited records.
Identify the wording that changed in the extracted text.

The page URL should stay fixed in both examples. The extracted text should differ only for the edited page.

Build the hash gate

The same extracted text produces the same hash when the hashing method stays consistent. A text change produces a new fingerprint for the comparison.

The page URL finds the existing point in your vector database. The automation can then compare the current fingerprint with the stored fingerprint for that point.

Why hash the extracted text?

A hash reduces the page text to a compact value that is quick to compare. Matching values let the automation bypass repeated embedding work.

Use only the extracted page text as the hash input. Crawl timestamps or request details can change even when the page content stays identical.

Switch back to the embedding automation from earlier.
Create the current content hash by adding a processing step immediately after the crawler output.
Set the processing input to the extracted page text.
Execute that processing step for one unchanged record from the second crawl dataset.

You should see a content hash in the processing output. That value becomes the current fingerprint for the page.

Add a vector-point lookup after the current hash step.
Configure the lookup to find the existing point through its page URL.
Execute this path for the same unchanged record.

You should see the existing point return with its stored page URL. Its stored page text should match the unchanged crawler record.

Add a fallback that computes a hash from stored page text when current hash metadata is missing.
Save the fallback hash to the existing point metadata without using the embedding provider.
Execute the path again for the unchanged record.

You should see equal fingerprints for the current text plus the stored text. The fallback also prepares older points for future incremental runs.

Add a comparison immediately after the lookup path.
Route matching hashes around the existing embedding-to-upsert path.
Execute the path for the unchanged record.

You should see the unchanged record take the skip route. It should never reach the embedding provider.

Route a missing or different hash into the existing embedding-to-upsert path.
Configure the upsert to retain the current content hash in the point metadata.
Execute this path for the edited page record.

You should see the edited page reach the embedding provider. Its existing vector point should update with the new page text plus the current content hash.

Run the incremental update

The full second crawl now tests both routes in one execution. Its log provides the evidence that the automation is doing less work.

Set the matching-hash route to record a skipped outcome in the run log.
Set the changed-hash route to record a processed outcome after vector upsert.

Before you start the full run, how many pages do you expect to reach the existing embedding-to-upsert path?

Run the updated automation against the second crawl dataset.

You should see every unchanged page recorded as skipped. The edited page alone should be recorded as processed.

Review the completed run log.
Confirm that the edited page is the only record sent to the embedding provider.
Confirm that the edited page continues through vector upsert.
Inspect the metadata for one unchanged point in the vector collection.
Confirm that the unchanged point now retains its current content hash.
Inspect the point for the edited page.
Confirm that the edited point retains the new page text plus its current content hash.

That is the efficiency win you were aiming for. Your automation now processes the content delta while preserving one vector point per page.

Seeing every page reach embedding?

Check that the current hash uses only the extracted page text.
Confirm that the current record plus the stored point use the same page URL.
Verify that matching hashes connect to the skip route.
Review the fallback if older points have no stored hash metadata.
Help me debug my incremental embedding routes.

✔️ My incremental run is complete

Strong work. Your completed log proves that unchanged pages bypass embedding while the edited page updates its existing vector point.

ⓧ I'd like to double check the automation

Your incremental automation should match this end state:

Every crawled page receives a content hash computed from its extracted page text.
Each page URL identifies the existing vector point for comparison.
A missing stored hash is backfilled from the stored page text without using the embedding provider.
Matching hashes bypass the existing embedding-to-upsert path.
A missing or different hash enters the existing embedding-to-upsert path.
Every vector point retains its page URL plus its page text plus its current content hash.
The edited page point contains the wording from the second crawl dataset.
The completed log shows unchanged pages skipped.
The completed log shows only the edited page processed.

Your automation now avoids repeated embedding work for unchanged pages. Next, you will place the full-run credit use beside the incremental-run credit use.

Compare Embedding Costs

Your automation now skips unchanged pages. Its log shows that only the edited page reached the embedding step.

The last question is how much that behavior changes credit usage. You will compare the full run with the incremental run using the credits recorded for each execution.

In this step, get ready to:

Record the embedding credit cost from each completed run.
Calculate the credit difference between the runs.
Verify the completed comparison against the two run records.

Record both run costs

A useful comparison needs matching units. The embedding credit value attached to each execution gives both runs the same basis.

Return to the initial full-run execution from earlier to find its embedding credit usage.
Record the full-run value here: full-run embedding credit cost.
Return to the incremental run log from earlier to find its embedding credit usage.
Record the incremental-run value here: incremental-run embedding credit cost.
Confirm that both values use the same credit unit.

Your two comparison figures are ready. Each value now maps to a specific execution.

Can't Match a Cost to a Run?

Match each automation execution to its corresponding crawler-run timestamp.
Copy the credit value attached specifically to the embedding step.
Help me match the runs to their costs.

Calculate the credit difference

The credit difference measures how many embedding credits the incremental run avoided for this updated crawl.

Before you calculate the difference, do you expect it to sit closer to the full-run cost or closer to zero?

Subtract incremental-run embedding credit cost from full-run embedding credit cost.
Record the result here: embedding credit difference.
Review the completed comparison below.

Your Cost Comparison

Full-run embedding credit cost: full-run embedding credit cost. Incremental-run embedding credit cost: incremental-run embedding credit cost. Credit difference: embedding credit difference.

You should see the incremental-run cost below the full-run cost. The difference should match the subtraction you just completed.

Does the Difference Look Wrong?

Recalculate with the incremental-run cost subtracted from the full-run cost.
Confirm that the incremental log shows only the edited page reaching the embedding step.
Help me check my embedding credit comparison.

You have now turned selective embedding into a measurable credit result.

Secret mission

Remove Vectors for Deleted Pages

A page can vanish from a crawl while its old vector remains searchable. Extend your automation to detect the missing URL. Delete its stale point without another embedding. Restore the page to prove the collection resynchronizes.

Clean Up Your Resources

Your small-site runs use the free-tier allowances planned for this project. An active crawl schedule can still spend account credits after you finish.

You now have three options. You can keep the pipeline active, pause scheduled runs, or remove the project data.

Cost warning

A recurring crawl keeps using crawler credits. Its embedding run can also use provider credits.

Choose Pause or Delete unless you plan to keep testing the pipeline.

Resources you used:

Crawler dataset storage for both recorded runs.
Incremental embedding automation with its content hash comparison.
Stored run history from the completed incremental run.
A vector collection containing one current point per crawled page.
Any recurring crawl or automation schedule enabled for this project.

Keep everything running

No action is needed. This option fits continued testing with the selected documentation site.

Your automation remains ready to compare future crawl datasets with the hashes stored in the vector collection.
Your vector collection keeps the latest page text from the incremental run.
Your run history keeps the evidence that unchanged pages were skipped.
Review your service credit balances before leaving any recurring schedule active.

Pause - I'll come back to this later

Pausing preserves the reusable automation. It prevents unattended runs from consuming more credits.

Return to the automation platform from earlier.
Select the incremental embedding automation from earlier.
Disable any trigger that runs the automation on a schedule.
Return to the crawler platform from earlier.
Select the crawler configuration for the documentation site from earlier.
Disable any active crawl schedule.
Confirm the automation platform shows no future scheduled run.
Confirm the crawler platform shows no future scheduled crawl.

Delete - I don't want to use this again

Complete removal clears the project data from each service. Your source documentation site stays available.

Deletion is permanent

Deleting these resources removes your stored vectors, crawl datasets, workflow, and run history. The page edit on your documentation site remains.

Automation platform:

Return to the automation platform from earlier.
Select the stored run history for the incremental embedding automation.
Use the run-history deletion control to remove the completed incremental run log.
Select the incremental embedding automation from earlier.
Disable any active scheduled trigger.
Use the automation deletion control to remove the workflow.
Confirm the workflow no longer appears in your automation list.
Confirm the incremental run no longer appears in the stored run history.

Crawler datasets:

Return to the crawler platform from earlier.
Select the initial crawl dataset for the documentation site.
Use the dataset deletion control to remove the initial crawl dataset.
Select the second crawl dataset for the documentation site.
Use the dataset deletion control to remove the second crawl dataset.
Confirm both datasets no longer appear in the crawler dataset storage.

Crawler schedule:

Select the crawler configuration for the documentation site from earlier.
Disable any active crawl schedule.
Confirm the crawler platform shows no future scheduled crawl.

Vector database:

Return to the vector database from earlier.
Select the vector collection created for this project.
Use the collection deletion control to remove the vector collection.
Confirm the vector collection no longer appears in the collection list.

Nice Work!

Outstanding work! You built an incremental documentation pipeline that processes only changed pages. It skips unchanged content so scheduled runs spend fewer embedding credits.

You've learned how to:

Use a website crawler to capture documentation pages as inspectable dataset records. Compare the initial dataset with the second dataset to pinpoint the edited page.
Create a searchable vector collection with one point per page. Run a plain-English similarity search to retrieve the best-matching page record.
Use content hashes to identify changed pages while skipping unchanged pages. Compare the full-run cost with the incremental-run cost to record the credit difference.
Secret Mission: Took on an optional challenge to push the incremental update workflow further.

Ready to quiz yourself?