Build a Paginated Quote Collector

Scrape paginated quotes with Python, export CSV data, and rank authors.

Introduction

30 Second Summary

Lists on the web often continue across a row of numbered pages. Reading only the first page leaves much of the information out of sight.

In this project, you will build a command-line Quote Collector with Python, Requests, and Beautiful Soup. It will follow every current page on Quotes to Scrape to produce a CSV archive with a five-author frequency report.

What You'll Build

Your finished collector shows the quote count growing as it processes each page of the archive.

By the end of this project, you'll have:

  • A page-by-page collector that prints its progress as it follows each Next link through the complete current archive.
  • A structured CSV dataset where every row includes the quote text, author, tags, and source page URL.
  • A ranked author report that shows the five writers represented most often in the archive.
  • Secret Mission: A configurable tag filter that creates quotes_love.csv with only quotes tagged love.

Are there any prerequisites?

You need basic Python knowledge on a Windows computer. You also need Python 3.10 or later plus Visual Studio Code.

Before We Start

This is your moment to lock in what the quote collector captures. You will also identify why a single page cannot represent a paginated archive.

Set Up a Reproducible Project

A reliable collector needs a compatible Python interpreter. It also needs the same package versions every time you recreate the project.

A project-specific virtual environment keeps these packages isolated. Visual Studio Code connects that environment to the files you run.

In this step, get ready to:
  • Confirm a supported Python interpreter.
  • Create an isolated environment with pinned dependencies.
  • Configure VS Code to use the project interpreter.
Confirm the Python and VS Code tools

The pinned Requests release requires Python 3.10 or later. Checking the interpreter first prevents confusing package installation failures.

  • Press the Windows key to open Windows search.
  • Type Visual Studio Code.
  • Press Enter to open the application.
  • Open an integrated terminal from the VS Code top menu.
  • Check your Python 3 version by running this command:
py -3 --version

What does this command check?

  • The py command uses the Python launcher on Windows.
  • The -3 option targets Python 3.
  • The version output reveals whether the selected interpreter supports the pinned dependencies.

✔️ I see a supported version

Your interpreter reports Python 3.10 or later. That clears the first compatibility check.

ⓧ I see an older version

The installed interpreter cannot run the pinned Requests release. Update Python before creating the project environment.

Windows may request permission while the installer makes system changes.

  • Visit the official Python Windows releases page.
  • Select the current stable Python 3 release, version 3.14.7.
  • Download the Windows installer from the release page.
  • Run the downloaded installer.
  • Complete the installation.
  • Restart VS Code.
  • Open a new integrated terminal from the VS Code top menu.
  • Repeat the version check by running this command:
py -3 --version

What should I see?

The command now reports Python 3.10 or later. The newer interpreter can run the required Requests release.

Still seeing the older version?

Restart Windows so the Python launcher can detect the new installation. Run the version check again after signing back in.

If the older version remains selected, help me identify why the Windows Python launcher still uses it.

ⓧ Command not found

The Python launcher is unavailable. Installing the current stable Python 3 release provides the supported interpreter this project needs.

Windows may request permission while the installer makes system changes.

  • Visit the official Python Windows releases page.
  • Select the current stable Python 3 release, version 3.14.7.
  • Download the Windows installer from the release page.
  • Run the downloaded installer.
  • Complete the installation.
  • Restart VS Code.
  • Open a new integrated terminal from the VS Code top menu.
  • Repeat the version check by running this command:
py -3 --version

What should I see?

The command now reports Python 3.10 or later. The Python launcher is ready for the project setup.

Python launcher still unavailable?

Restart Windows before repeating the check. This gives the system a fresh chance to detect the installation.

If the command remains unavailable, help me diagnose my Windows Python installation.

The Microsoft Python extension adds Python language support to VS Code. It also provides the commands for selecting an interpreter and running Python files.

  • Select the extensions icon in the VS Code Activity Bar.
  • Search for Microsoft Python extension.
  • Select the result published by Microsoft.
  • Confirm that the extension is installed.
  • Use the extension page's installation control if it is missing.
  • Open VS Code's command search from the top menu.
  • Enter Python: Select Interpreter.
  • Select the interpreter that matches the supported version you confirmed.

VS Code now has Python support. It also knows which compatible interpreter to use while the project environment is created.

Can't select the interpreter?

Restart VS Code if the interpreter command is missing after installation. Confirm that the extension result is published by Microsoft.

If no supported interpreter appears, help me connect my Python installation to VS Code.

Create the environment and dependency file

The project folder gives every file a predictable home. The dependency file records the exact library versions needed to rebuild the environment.

  • Use the VS Code top menu to open the folder picker.
  • Select your Desktop in the folder picker.
  • Create a folder named paginated-quote-collector.
  • Select the paginated-quote-collector folder.
  • Confirm the folder selection.

The VS Code Explorer sidebar now shows paginated-quote-collector as the open folder.

  • Select the New File icon in the Explorer toolbar.
  • Enter requirements.txt as the file name.
  • Add the dependency pins to requirements.txt by pasting this content:
requests==2.34.2
beautifulsoup4==4.15.0

What does this file control?

  • The requests==2.34.2 line pins the HTTP library used by the collector.
  • The beautifulsoup4==4.15.0 line pins the HTML parsing library.
  • The exact pins make future environment installations use the same dependency versions.
  • Save requirements.txt.
  • Confirm that requirements.txt appears inside paginated-quote-collector in the Explorer sidebar.

Dependency file missing or incorrect?

Confirm that the file is inside the paginated-quote-collector folder. Check that its name ends with .txt.

Make sure each package pin has its own line. Help me verify my requirements file.

✔️ Awesome, I've got everything!

Great. Double-check that you saved requirements.txt inside paginated-quote-collector.

ⓧ I'd like to double check the full code

requests==2.34.2
beautifulsoup4==4.15.0

The .venv folder isolates this project's installed packages from other Python projects on your computer.

  • Open a new integrated terminal from the VS Code top menu.
  • Create the virtual environment inside paginated-quote-collector by running this command:
py -m venv .venv

What does this command create?

  • The Python launcher runs the built-in environment creation module.
  • The .venv argument gives the isolated environment its folder name.
  • The environment receives its own Python interpreter and package location.
  • Confirm that the Explorer sidebar now lists a .venv folder inside paginated-quote-collector.

Virtual environment folder not appearing?

Confirm that the terminal is inside the paginated-quote-collector folder. Repeat the creation command after the terminal prompt shows that folder.

If the command reports a Python problem, confirm the version check still succeeds. Help me create the project virtual environment.

  • Activate the new environment by running this command:
.venv\Scripts\activate

What does activation change?

Activation connects the terminal session to the interpreter inside .venv. Package installations in this terminal now belong to the project environment.

The terminal prompt now indicates that the virtual environment is active.

Activation blocked in PowerShell?

Use the terminal profile menu in VS Code to create a Command Prompt terminal. Run the activation command again in that terminal.

If activation still fails, confirm that the .venv folder exists. Help me activate this Windows virtual environment.

  • Open VS Code's command search from the top menu.
  • Enter Python: Select Interpreter.
  • Select the interpreter marked as belonging to the .venv folder.

VS Code is now connected to the project-specific interpreter. Python files in this folder use the isolated environment when you run them.

Install the pinned packages

The Requests library sends the collector's web requests. Beautiful Soup turns the returned HTML into searchable elements.

Installing from requirements.txt gives the environment Requests 2.34.2. A separate pin gives it Beautiful Soup 4.15.0.

Before you run the installation, which two pinned versions do you expect the terminal to confirm?

  • Install the pinned dependencies into the active environment by running this command:
py -m pip install -r requirements.txt

What does this command install?

  • The Python launcher runs pip as a module.
  • The -r option tells pip to read package requirements from a file.
  • The final argument points the installer to requirements.txt.

The terminal completes without an error. It confirms Requests 2.34.2 as installed or already available.

It also confirms Beautiful Soup 4.15.0 as installed or already available.

Package installation not completing?

Confirm that the terminal is inside paginated-quote-collector. Check that the virtual environment still appears active in the prompt.

A missing requirements file means the terminal is using the wrong folder. Help me troubleshoot this dependency installation.

Your reproducible Python workspace is ready. Next, you will fetch the first quote page and turn its HTML into structured records.

Collect the First Page

The isolated Python environment from the previous step is ready. The collector can now turn a live page from Quotes to Scrape into reusable records.

An HTTP GET request gives Requests one page of markup. Beautiful Soup turns that markup into searchable HTML so you can test how much of the archive one response contains.

In this step, get ready to:
  • Fetch the verified starting page with response validation.
  • Extract text, author, tags, and source URLs into structured dictionaries.
  • Run the collector to measure what one response captures.
Fetch and parse the starting page

The collector validates the response before parsing it. A 10-second timeout prevents the request from waiting indefinitely.

  • Create quote_collector.py inside the paginated-quote-collector folder using the VS Code file sidebar.

You will see quote_collector.py in the file sidebar. VS Code also opens it in a new editor tab.

  • Add the request foundation to quote_collector.py by pasting this code:
import requests
from bs4 import BeautifulSoup

START_URL = "https://quotes.toscrape.com/page/1/"
HEADERS = {"User-Agent": "PaginatedQuoteCollector/1.0 (learning project)"}


def main():
    response = requests.get(START_URL, headers=HEADERS, timeout=10)
    response.raise_for_status()
    soup = BeautifulSoup(response.content, "html.parser")
    quotes = []

    print(f"Collected {len(quotes)} quotes.")


if __name__ == "__main__":
    main()

What does this foundation do?

  • The requests import provides the page-request tools.
  • The BeautifulSoup import provides the HTML parser.
  • START_URL identifies the first page of the quote archive.
  • HEADERS gives the request a descriptive User-Agent value.
  • requests.get() sends the request with the header plus the timeout.
  • raise_for_status() stops the script when the server reports an HTTP error.
  • response.content supplies the response bytes to the html.parser parser.
  • Save quote_collector.py.
  • Click Run Python File in the editor title area.

The terminal prints Collected 0 quotes. because the record list is still empty. Your first live request now completes successfully.

First request not completing?

  • Confirm that VS Code still has the project interpreter from .venv selected.
  • Check that the computer can reach the starting page through its internet connection.
  • Compare the imports plus the URL with the code block above.

Get help diagnosing the request setup.

Extract structured quote records

Each div.quote card groups one quote with its author plus its tags. CSS selectors let the parser read those values from each card.

  • Select the existing main() function in quote_collector.py from def main(): through print(f"Collected {len(quotes)} quotes.").
  • Replace the selected function by pasting this code:
def main():
    response = requests.get(START_URL, headers=HEADERS, timeout=10)
    response.raise_for_status()
    soup = BeautifulSoup(response.content, "html.parser")
    quotes = []

    for card in soup.select("div.quote"):
        tag_names = [
            tag.get_text(strip=True)
            for tag in card.select("div.tags a.tag")
        ]
        quote = {
            "text": card.select_one("span.text").get_text(strip=True),
            "author": card.select_one("small.author").get_text(strip=True),
            "tags": ", ".join(tag_names),
            "source_url": START_URL,
        }
        quotes.append(quote)
        print(f"{quote['text']} | {quote['author']}")

    print(f"Collected {len(quotes)} quotes.")

How are the records built?

  • soup.select("div.quote") returns the quote cards found on the page.
  • card.select("div.tags a.tag") finds every tag link inside the current card.
  • get_text(strip=True) removes surrounding whitespace from each extracted value.
  • quote stores the text, author, tags, and source page in one dictionary.
  • quotes.append(quote) preserves each finished dictionary in the collection.
  • The two print statements expose every quote-author pair plus the final record count.
  • Save quote_collector.py.

Before you run the collector, do you think one response contains the site's full quote archive?

  • Click Run Python File in the editor title area.

You will see ten quote-author lines. The final line reads Collected 10 quotes..

You have turned the first live page into ten structured records.

Missing quotes or authors?

  • Confirm that the loop selects div.quote.
  • Check that the quote selector is span.text.
  • Check that the author selector is small.author.

Get help comparing the parser with the live quote markup.

✔️ Awesome, I've got everything!

Great. Double-check that you saved quote_collector.py with the complete first-page collector.

ⓧ I'd like to double check the full code

import requests
from bs4 import BeautifulSoup

START_URL = "https://quotes.toscrape.com/page/1/"
HEADERS = {"User-Agent": "PaginatedQuoteCollector/1.0 (learning project)"}


def main():
    response = requests.get(START_URL, headers=HEADERS, timeout=10)
    response.raise_for_status()
    soup = BeautifulSoup(response.content, "html.parser")
    quotes = []

    for card in soup.select("div.quote"):
        tag_names = [
            tag.get_text(strip=True)
            for tag in card.select("div.tags a.tag")
        ]
        quote = {
            "text": card.select_one("span.text").get_text(strip=True),
            "author": card.select_one("small.author").get_text(strip=True),
            "tags": ", ".join(tag_names),
            "source_url": START_URL,
        }
        quotes.append(quote)
        print(f"{quote['text']} | {quote['author']}")

    print(f"Collected {len(quotes)} quotes.")


if __name__ == "__main__":
    main()
Confirm the pagination limit

The terminal count describes the current response. The page navigation answers whether more quote records remain elsewhere.

Before you inspect the page, do you expect its bottom navigation to show another page?

You will see a Next link after the tenth quote. That link points beyond the response your script processed.

  • Leave the Next link unselected.
  • Switch back to VS Code.

The terminal still reports ten records because the script sent one request. The visible li.next a link proves that the current collection is incomplete.

Your first page is now a structured dataset with a visible boundary. Next, you will follow each relative pagination link until the archive ends.

Follow Every Next Link

Your first-page collector proved that the page markup can become structured records. Its 10-record result also exposed the pagination gap.

A pagination loop follows each Next link until the final page has no match. This turns one successful request into the complete current archive.

In this step, get ready to:
  • Separate page fetching from quote parsing.
  • Follow each relative Next link until pagination ends.
  • Handle request failures before incomplete records can continue.
Separate fetching from parsing

Two helper functions give fetching and parsing one responsibility each. The pagination loop can reuse those helpers for every page.

  • Replace the import block at the top of quote_collector.py with this code:
import time
from urllib.parse import urljoin

import requests
from bs4 import BeautifulSoup

What do these imports support?

  • The time module provides the delay between page requests.
  • The urljoin function converts each relative Next URL into an absolute URL.
  • Requests continues to handle each HTTP request.
  • Beautiful Soup continues to parse the returned page markup.
  • Save quote_collector.py.
  • Click Run Python File in the top-right corner of VS Code.

You should still see the 10 first-page quote-author records. This confirms that the expanded imports have kept the existing collector runnable.

Seeing an import problem?

Compare the import order with the code above. Check that urljoin comes from urllib.parse.

Confirm that VS Code still shows the selected .venv interpreter. Help me troubleshoot these Python imports.

  • Add fetch_soup() below the HEADERS setting by pasting this code:
def fetch_soup(url):
    response = requests.get(url, headers=HEADERS, timeout=10)
    response.raise_for_status()
    return BeautifulSoup(response.content, "html.parser")

What does the fetching helper do?

  • The url parameter lets the collector request a different page on each loop.
  • The existing headers remain attached to every request.
  • The 10-second timeout prevents a request from waiting indefinitely.
  • The status check stops unsuccessful responses before their content reaches the parser.
  • Save quote_collector.py.
  • Click Run Python File again.

You should see the same 10 records without a syntax error. The new helper now sits safely beside the working first-page logic.

Does the file stop immediately?

Check that fetch_soup() starts at the left edge of the file. Keep each line inside the function indented by four spaces.

Compare the parentheses around requests.get() with the code above. Help me fix the fetch_soup function.

  • Add parse_quotes() below fetch_soup() by pasting this code:
def parse_quotes(soup, page_url):
    quotes = []

    for card in soup.select("div.quote"):
        tag_names = [
            tag.get_text(strip=True)
            for tag in card.select("div.tags a.tag")
        ]
        quotes.append(
            {
                "text": card.select_one("span.text").get_text(strip=True),
                "author": card.select_one("small.author").get_text(strip=True),
                "tags": ", ".join(tag_names),
                "source_url": page_url,
            }
        )

    return quotes

What does the parsing helper return?

  • The soup parameter holds the parsed markup for one page.
  • The page_url parameter records the source page for each quote.
  • The selectors extract text, author names, plus complete tag values from each quote card.
  • The returned list gives the pagination loop one batch of structured records.
  • Save quote_collector.py.
  • Click Run Python File again.

You should still see all 10 first-page records. The file remains runnable after separating the reusable parsing logic.

Seeing a parsing syntax problem?

Check that the closing brackets for tag_names align with its opening bracket. Confirm that the quote dictionary remains inside quotes.append().

Keep return quotes aligned with the for loop. Help me inspect the parse_quotes indentation.

Build the pagination loop

Each page provides a relative URL through li.next a. The final page ends the loop because that selector has no match.

  • Add collect_quotes() above the existing main() function by pasting this code:
def collect_quotes():
    current_url = START_URL
    all_quotes = []
    page_number = 1

    while current_url:
        soup = fetch_soup(current_url)
        page_quotes = parse_quotes(soup, current_url)
        all_quotes.extend(page_quotes)

        print(
            f"Page {page_number}: collected {len(page_quotes)} quotes "
            f"({len(all_quotes)} total)"
        )

        next_link = soup.select_one("li.next a")
        current_url = (
            urljoin(current_url, next_link.get("href", ""))
            if next_link
            else None
        )

        if current_url:
            time.sleep(0.5)

        page_number += 1

    return all_quotes

How does the loop reach every page?

  • The current_url variable starts with START_URL.
  • The page_quotes list holds records from the current page.
  • The all_quotes list grows after every parsed page.
  • The next_link value controls whether another loop begins.
  • The 0.5 second pause spaces out requests while another page remains.
  • Save quote_collector.py.
  • Click Run Python File to confirm the existing entry point still runs.

You should see the familiar 10-record result. This confirms that the file still runs after adding the complete loop.

Does the new loop break the run?

Check that collect_quotes() sits above the existing main() function. Confirm that return all_quotes aligns with the while statement.

Check the indentation around the conditional urljoin() expression. Help me debug the pagination loop.

Handle failures and verify progress

Repeated requests can fail because of network or server problems. The entry point catches request failures before incomplete records can move further through the workflow.

  • Find the existing def main(): block in quote_collector.py.
  • Select the entire block through print(f"Collected {len(quotes)} quotes.").
  • Replace the selection with this error-handling entry point:
def main():
    try:
        quotes = collect_quotes()
    except requests.RequestException as error:
        print(f"Collection failed: {error}")
        return

    print(f"\nCollected {len(quotes)} quotes.")

How does the entry point protect the run?

  • The try block starts the complete collection workflow.
  • The except block catches any requests.RequestException raised during collection.
  • The captured error value gives the terminal a clear failure message.
  • The early return prevents an incomplete quote list from continuing.
  • The final message reports the complete count only after every request succeeds.
  • Save quote_collector.py.

Before you run the collector, predict whether the total will stay at 10 or grow beyond it.

  • Click Run Python File in the top-right corner of VS Code.

You should see page-by-page progress in the terminal. The cumulative total grows beyond 10 before the collector stops after the final page.

That is the pagination barrier cleared. Your collector now reaches the complete current quote archive without requesting another URL after the final page.

Does collection stop before the final page?

Check that next_link uses the exact selector li.next a. Confirm that current_url receives the result from urljoin() when a link exists.

A request failure prints a collection message instead of a final count. Help me diagnose this incomplete pagination run.

✔️ Awesome, I've got everything!

Great work. Double-check that you saved quote_collector.py after the complete paginated run.

ⓧ I'd like to double check the full code

import time
from urllib.parse import urljoin

import requests
from bs4 import BeautifulSoup

START_URL = "https://quotes.toscrape.com/page/1/"
HEADERS = {"User-Agent": "PaginatedQuoteCollector/1.0 (learning project)"}


def fetch_soup(url):
    response = requests.get(url, headers=HEADERS, timeout=10)
    response.raise_for_status()
    return BeautifulSoup(response.content, "html.parser")


def parse_quotes(soup, page_url):
    quotes = []

    for card in soup.select("div.quote"):
        tag_names = [
            tag.get_text(strip=True)
            for tag in card.select("div.tags a.tag")
        ]
        quotes.append(
            {
                "text": card.select_one("span.text").get_text(strip=True),
                "author": card.select_one("small.author").get_text(strip=True),
                "tags": ", ".join(tag_names),
                "source_url": page_url,
            }
        )

    return quotes


def collect_quotes():
    current_url = START_URL
    all_quotes = []
    page_number = 1

    while current_url:
        soup = fetch_soup(current_url)
        page_quotes = parse_quotes(soup, current_url)
        all_quotes.extend(page_quotes)

        print(
            f"Page {page_number}: collected {len(page_quotes)} quotes "
            f"({len(all_quotes)} total)"
        )

        next_link = soup.select_one("li.next a")
        current_url = (
            urljoin(current_url, next_link.get("href", ""))
            if next_link
            else None
        )

        if current_url:
            time.sleep(0.5)

        page_number += 1

    return all_quotes


def main():
    try:
        quotes = collect_quotes()
    except requests.RequestException as error:
        print(f"Collection failed: {error}")
        return

    print(f"\nCollected {len(quotes)} quotes.")


if __name__ == "__main__":
    main()

What should match?

Compare the imports, helper functions, pagination loop, plus error-handling entry point with your file. Keep the existing final main() call at the bottom.

Your collector now holds the complete archive in memory. Next, you will save those records to quotes.csv and rank the most frequently represented authors.

Export and Rank the Archive

Your pagination loop now returns the complete quote archive. However, that archive disappears when the script ends.

A CSV file gives the records a portable home. A ranked author report makes the archive easy to inspect.

In this step, get ready to:
  • Export the complete archive to quotes.csv.
  • Build a top-five author ranking with consistent tie handling.
  • Inspect records from the beginning and end of the saved dataset.
Write a clean CSV file

Python's csv.DictWriter maps each quote dictionary to a row. A fixed field order keeps every record aligned with the same four columns.

  • In quote_collector.py, replace the opening standard-library import group with this code:
import csv
import time
from urllib.parse import urljoin

What does this import add?

  • The csv module provides the dictionary-based writer used for the export.
  • The existing time and urljoin imports continue supporting request pacing and pagination.
  • Add the output filename directly below START_URL by pasting this line:
OUTPUT_FILE = "quotes.csv"

Why define the filename once?

The OUTPUT_FILE setting gives the export one reusable destination. The save message can reference the same value.

  • Save quote_collector.py.

The collector pauses for 0.5 seconds between pages, so let each progress line arrive before judging the run.

  • Click Run Python File in the top-right corner of VS Code.

You will see the usual page progress followed by the complete collected count. This confirms that the new import and output setting load successfully.

Does the script stop near the new settings?

Check that import csv is the first line of the file. Confirm that OUTPUT_FILE sits directly below START_URL.

Make sure both lines match the snippets exactly. Help me check the new CSV import and output setting.

  • In quote_collector.py, locate the return all_quotes line at the end of collect_quotes().
  • Add the CSV-writing function below collect_quotes() by pasting this code:
def save_csv(quotes, filename=OUTPUT_FILE):
    fieldnames = ["text", "author", "tags", "source_url"]

    with open(filename, "w", newline="", encoding="utf-8") as csv_file:
        writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
        writer.writeheader()
        writer.writerows(quotes)

What does this function do?

  • The fieldnames list fixes the column order as quote text, author, tags, and source page URL.
  • The newline="" setting lets the CSV writer handle row endings correctly.
  • The encoding="utf-8" setting preserves punctuation and characters from the collected quotes.
  • The writer creates the header before writing every dictionary in quotes.
  • In main(), locate the current collected-count statement:
print(f"\nCollected {len(quotes)} quotes.")

What does the current line prove?

This line confirms the archive size in the terminal. It does not preserve the records after the process exits.

  • Replace the current collected-count statement with this export code:
save_csv(quotes)
print(f"\nSaved {len(quotes)} quotes to {OUTPUT_FILE}.")

How does the export join the workflow?

  • The save_csv(quotes) call writes the complete in-memory archive to disk.
  • The following message reports the saved record count and destination filename.
  • Save quote_collector.py.

Before you run the collector, do you expect quotes.csv to contain one page or the complete archive?

  • Click Run Python File.

You will see every page processed before the saved-count message appears. The file list on the left will now include quotes.csv.

  • Select quotes.csv from the file list on the left.
  • Confirm the first row contains text, author, tags, and source_url.

Your archive now survives beyond the terminal run. Each collected dictionary has become a reusable CSV row.

Can't find the CSV file?

Confirm that the terminal reached the saved-count message. A request failure returns from main() before the export begins.

Check that save_csv(quotes) appears after the request exception block. Help me troubleshoot why quotes.csv was not created.

Count and rank the authors

An author-count dictionary turns repeated names into totals. The ranking then places larger totals first while author names settle ties alphabetically.

  • In quote_collector.py, locate the end of save_csv().
  • Add the ranking function below save_csv() by pasting this code:
def print_top_authors(quotes, limit=5):
    author_counts = {}

    for quote in quotes:
        author = quote["author"]
        author_counts[author] = author_counts.get(author, 0) + 1

    ranked_authors = sorted(
        author_counts.items(),
        key=lambda item: (-item[1], item[0]),
    )[:limit]

    print(f"\nTop {limit} authors:")
    for author, count in ranked_authors:
        print(f"{count} | {author}")

How does the ranking work?

  • The author_counts dictionary stores one total for each author.
  • The get(author, 0) + 1 expression increments the current author's total.
  • The sort key uses a negative count to place higher totals first.
  • The second sort value orders equal totals by author name before [:limit] keeps five results.
  • In main(), add the ranking call directly below the saved-count message by pasting this line:
print_top_authors(quotes)

Why call the ranking here?

The call receives the same complete archive that was written to quotes.csv. No additional page requests are needed for the report.

  • Save quote_collector.py.

Before you run it, which author do you expect to appear near the top of the complete archive?

  • Click Run Python File.

After the saved-count message, you will see Top 5 authors: followed by five author-count pairs. Each pair uses the format count | author.

The archive now explains itself at a glance. Its most frequently represented authors appear in a stable order.

Missing the author report?

Check that print_top_authors(quotes) sits inside main(). It should align with the save_csv(quotes) call.

Confirm that the function name matches print_top_authors everywhere. Help me diagnose a missing author report.

Inspect the finished workflow

Your finished file should now match the cumulative version below. The final run checks the export and analysis together.

✔️ Awesome, I've got everything!

Great. Double-check that you saved quote_collector.py.

ⓧ I'd like to double check the full code

import csv
import time
from urllib.parse import urljoin

import requests
from bs4 import BeautifulSoup

START_URL = "https://quotes.toscrape.com/page/1/"
OUTPUT_FILE = "quotes.csv"
HEADERS = {"User-Agent": "PaginatedQuoteCollector/1.0 (learning project)"}


def fetch_soup(url):
    response = requests.get(url, headers=HEADERS, timeout=10)
    response.raise_for_status()
    return BeautifulSoup(response.content, "html.parser")


def parse_quotes(soup, page_url):
    quotes = []

    for card in soup.select("div.quote"):
        tag_names = [
            tag.get_text(strip=True)
            for tag in card.select("div.tags a.tag")
        ]
        quotes.append(
            {
                "text": card.select_one("span.text").get_text(strip=True),
                "author": card.select_one("small.author").get_text(strip=True),
                "tags": ", ".join(tag_names),
                "source_url": page_url,
            }
        )

    return quotes


def collect_quotes():
    current_url = START_URL
    all_quotes = []
    page_number = 1

    while current_url:
        soup = fetch_soup(current_url)
        page_quotes = parse_quotes(soup, current_url)
        all_quotes.extend(page_quotes)

        print(
            f"Page {page_number}: collected {len(page_quotes)} quotes "
            f"({len(all_quotes)} total)"
        )

        next_link = soup.select_one("li.next a")
        current_url = (
            urljoin(current_url, next_link.get("href", ""))
            if next_link
            else None
        )

        if current_url:
            time.sleep(0.5)

        page_number += 1

    return all_quotes


def save_csv(quotes, filename=OUTPUT_FILE):
    fieldnames = ["text", "author", "tags", "source_url"]

    with open(filename, "w", newline="", encoding="utf-8") as csv_file:
        writer = csv.DictWriter(csv_file, fieldnames=fieldnames)
        writer.writeheader()
        writer.writerows(quotes)


def print_top_authors(quotes, limit=5):
    author_counts = {}

    for quote in quotes:
        author = quote["author"]
        author_counts[author] = author_counts.get(author, 0) + 1

    ranked_authors = sorted(
        author_counts.items(),
        key=lambda item: (-item[1], item[0]),
    )[:limit]

    print(f"\nTop {limit} authors:")
    for author, count in ranked_authors:
        print(f"{count} | {author}")


def main():
    try:
        quotes = collect_quotes()
    except requests.RequestException as error:
        print(f"Collection failed: {error}")
        return

    save_csv(quotes)
    print(f"\nSaved {len(quotes)} quotes to {OUTPUT_FILE}.")
    print_top_authors(quotes)


if __name__ == "__main__":
    main()
  • Save quote_collector.py.

Before you run the finished collector, which terminal messages and file details will prove the entire workflow succeeded?

  • Click Run Python File.

You will see page-by-page progress followed by the saved record count. The terminal will then display five ranked author-count pairs.

  • Select quotes.csv from the file list on the left.
  • Check that the first line reads text,author,tags,source_url.
  • Inspect the first few data rows.
  • Scroll to the final row.

You should see quote records at both ends of the file. The final record remains aligned under the same four columns.

  • Return to the terminal from the completed run.

The saved-count message and five-author report remain visible beneath the page progress. These results prove that collection, export, and analysis completed in one workflow.

Does the CSV look incomplete?

  • Confirm the terminal reaches its saved-count message before inspecting the file.
  • Close the current quotes.csv tab if it shows older contents.
  • Select quotes.csv again from the file list on the left.

A request failure prevents partial results from being exported. Help me verify whether the complete archive reached quotes.csv.

You have the complete pipeline working with a reusable archive plus a ranked author report.

Secret mission

Create a Tag-Specific Quote Collection

Turn the complete quote archive into a focused collection for one configurable tag. You will export quotes tagged with love to a second CSV file without repeating the crawl.

Clean Up Your Resources

Clean Up Your Resources

Choose whether to keep your local project available, pause your work session, or delete the project entirely. This project runs on your Windows computer, so its files create no ongoing service costs.

Resources you used:

  • A local project folder named paginated-quote-collector containing quote_collector.py and requirements.txt.
  • A project-specific virtual environment in .venv containing the pinned Python packages.
  • Two generated CSV datasets named quotes.csv and quotes_love.csv.

Keep everything running

Keeping the folder preserves the collector for future experiments. No action is needed while you still want to inspect the datasets or change the tag filter.

  • Leave the paginated-quote-collector folder and its contents unchanged.
  • Keep quotes.csv as your complete archive.
  • Keep quotes_love.csv as your focused tag collection.

Pause - I'll come back to this later

Pausing ends your local work session while preserving every project file. Choose this when you want to free up memory before returning later.

  • Return to the terminal panel in VS Code.
  • Select the trash can in the terminal panel to end the active terminal session.
  • Close VS Code.

Your script, virtual environment, dependency file, and both datasets remain available inside paginated-quote-collector.

Delete - I don't want to use this again

Permanent deletion can feel drastic. This choice removes only your local project folder from this computer.

  • Close the VS Code window containing paginated-quote-collector.
  • Use Windows search to open File Explorer.
  • Search your computer for paginated-quote-collector.
  • Select the matching paginated-quote-collector folder.
  • Press Shift+Delete to remove the folder permanently.
  • Confirm the permanent deletion in the Windows prompt.
  • Search your computer for paginated-quote-collector again.

You should see no matching project folder. The script, virtual environment, dependency file, and both CSV datasets are now removed.

Nice Work!

Nice Work!

You did it! Your Python quote collector now turns every current page on Quotes to Scrape into reusable CSV datasets.

You've learned how to:

  • Build a reliable page collector with Requests. Validate each HTTP response. Parse verified quote cards with Beautiful Soup.
  • Follow relative Next links until the pagination control disappears. Track page-by-page progress during the crawl.
  • Export the complete archive to quotes.csv. Rank the five most represented authors in the terminal.
  • Secret Mission: Filter the complete in-memory archive for exact tag matches. Export the focused collection to quotes_love.csv without making extra requests.

Ready to quiz yourself?