Scrape U.S. Treasury Property Listings

Build a Python scraper that extracts seized property data from Treasury.gov.

Introduction

30 Second Summary

The U.S. Treasury Department seizes real estate tied to federal law violations and auctions properties off to the public at below-market prices. The listings are published on a government website, but the data is buried in unstructured HTML that's impossible to filter or compare.

In this project, you will build a Python web scraper that pulls seized property listings from Treasury.gov, extracts structured data like addresses and square footage, and exports everything to a clean CSV file.

What You'll Build

A command-line Python script that fetches live government auction data, parses HTML with BeautifulSoup and regular expressions, and exports structured property listings to CSV.

By the end of this project, you'll have:

  • A working Python scraper (scraper.py) that fetches real property data from the U.S. Treasury website.
  • A CSV file (treasury_properties.csv) with structured columns for property type, address, square footage, bedrooms, bathrooms, and sale numbers.
  • Hands-on experience with the requests + BeautifulSoup scraping workflow used in production Python projects.
  • Secret Mission: Add interactive filtering so users can search properties by state or property type.

Are there any prerequisites?

This project assumes basic Python knowledge including variables, functions, and loops. You don't need any prior web scraping experience.

Before We Start

Every day, the U.S. Treasury Department publishes seized property listings online. These are real homes, land, and commercial buildings forfeited for federal law violations and sold at below-market prices.

In this project, you will build a Python scraper that fetches these listings, parses the HTML, and exports structured property data to a CSV file. First, you need to set up your coding environment.

In this step, get ready to:
  • Install VS Code and create the project folder.
  • Set up a Python virtual environment and install libraries.
  • Verify your setup works by importing both libraries.
Install VS Code and create the project folder

VS Code is a free code editor you will use to write and run your Python scraper.

  • Download and install VS Code from code.visualstudio.com.
  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Visual Studio Code and press Enter to open it.

You should see the VS Code Welcome tab open in a new window.

  • Open the integrated terminal by clicking Terminal in the top menu bar, then New Terminal.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create the project folder and move into it by running these commands:
mkdir treasury-scraper
cd treasury-scraper
  • Confirm the folder exists by running:
ls

You should see an empty output (the folder is new and has no files yet).

  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop, select the treasury-scraper folder, and click Open.

Why open the folder in VS Code?

Opening a folder tells VS Code to treat it as your project workspace. The integrated terminal will automatically start in that folder, so all commands you run target the right location.

Don't see the folder on your Desktop?

Make sure you ran the mkdir command without typos. Try running ls ~/Desktop in the terminal to confirm treasury-scraper appears in the list.

Help me find my project folder on the Desktop.

Create a virtual environment and install libraries

A virtual environment keeps your project's packages separate from other Python projects on your machine. This prevents version conflicts between projects.

  • Open a new terminal in VS Code by clicking Terminal then New Terminal. It should start inside treasury-scraper.
  • Create a virtual environment by running this command:
python -m venv venv

What does this command do?

python -m venv venv tells Python to create a virtual environment in a folder called venv. The first venv is the module name. The second venv is the folder it creates.

  • Activate the virtual environment. The command depends on your operating system:

macOS

  • Activate the environment by running:
source venv/bin/activate

Windows

  • Activate the environment by running:
venv\Scripts\activate

You should see (venv) appear at the beginning of your terminal prompt. This confirms the virtual environment is active.

  • Install the two libraries your scraper needs by running:
pip install requests==2.34.2 beautifulsoup4==4.15.0

What are these libraries?

requests lets Python send HTTP requests to websites, just like a browser does. beautifulsoup4 parses the HTML you receive back, making it easy to extract specific data.

Pinning exact versions (==2.34.2 and ==4.15.0) ensures your project behaves the same way regardless of when you run it.

You should see output ending with Successfully installed followed by the package names.

Seeing 'python' not found?

On macOS, you may need to use python3 instead of python for the venv creation command. Try python3 -m venv venv and then activate again.

Help me fix my Python virtual environment setup.

Verify the setup
  • Confirm both libraries are installed correctly by running this command:
python -c "import requests; from bs4 import BeautifulSoup; print('Setup complete!')"

What does this command do?

python -c runs a one-line Python script directly from the terminal. This script imports both libraries and prints a success message. If either import fails, you will see an error instead of the confirmation.

You should see Setup complete! printed in the terminal with no errors.

Seeing an ImportError?

Make sure your virtual environment is active (you should see (venv) in your prompt). If it is not, run the activation command from the previous substep again.

If the environment is active but imports still fail, re-run the pip install command and check for any error messages in its output.

Help me fix my library imports.

Your environment is ready. Next up, you will write code that fetches live property data from the U.S. Treasury website.

Set Up Your Environment

Your libraries are installed and your virtual environment is active. Now it's time to create the Python script that will fetch seized property data from the U.S. Treasury website.

Every web scraper starts with a target URL and a way to request that page. In this step, you'll create your scraper file, configure the request headers, and write a function that downloads the page.

In this step, get ready to:
  • Create scraper.py with the target URL and request headers.
  • Write the fetch_page() function to download the Treasury web page.
  • Run the script and confirm you can fetch live property data.
Create scraper.py and configure the target URL

Your scraper needs to know where to look and how to identify itself. The Treasury website expects requests that look like they come from a real browser, so you'll include a User-Agent header.

  • In the VS Code Explorer sidebar, click the New File icon at the top of the file tree.
  • Name the file scraper.py and press Enter.
  • Add the following imports and configuration to scraper.py:
import requests
from bs4 import BeautifulSoup
import csv
import re


URL = "https://www.treasury.gov/auctions/treasury/rp/brokersale.shtml"

HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
}

What does this code do?

The first four lines import the libraries your scraper needs. requests handles HTTP connections, BeautifulSoup parses HTML, csv writes spreadsheet files, and re provides regular expressions for pattern matching.

URL is the Treasury broker sales page where seized properties are listed. HEADERS contains a User-Agent string that identifies your scraper as a standard web browser. Without this header, some websites block automated requests.

  • Save scraper.py.

You won't see a change yet. The next chunk adds a function that uses this configuration to fetch the page.

Write the fetch_page() function

This function sends a request to the Treasury URL and returns the raw HTML content. It also includes error handling so the script fails clearly if the website is unreachable.

  • Below the HEADERS dictionary in scraper.py, add this function:
def fetch_page(url):
    """Fetch a web page and return its HTML content."""
    response = requests.get(url, headers=HEADERS, timeout=30)
    response.raise_for_status()
    print(f"Successfully fetched {url} (Status: {response.status_code})")
    return response.text

What does this code do?

requests.get() sends an HTTP GET request to the provided URL, passing along the browser-like headers and a 30-second timeout.

response.raise_for_status() checks the response code. If the server returns an error (like 404 or 500), Python raises an exception instead of silently continuing with bad data.

The function prints a confirmation message with the status code, then returns the full HTML text for parsing later.

  • Save scraper.py.

You won't see output yet. The next chunk adds the code that actually calls this function.

Run the script and verify the output

To test your function, you need a main block that calls fetch_page() and prints a preview of the HTML it receives.

  • At the bottom of scraper.py, add the following code:
if __name__ == "__main__":
    html = fetch_page(URL)
    print(html[:500])

What does this code do?

if __name__ == "__main__": is Python's way of saying "only run this code when the script is executed directly." It won't run if another file imports scraper.py as a module later.

html[:500] prints just the first 500 characters of the page. This gives you a quick preview without flooding the terminal.

  • Save scraper.py.
  • Run the script in your terminal by running this command:
python scraper.py

You should see a line that says Successfully fetched with status 200, followed by a block of HTML text containing property-related words like addresses and sale numbers.

Seeing a connection error or timeout?

If you see a ConnectionError or Timeout message, the Treasury website may be temporarily slow. Wait a minute and try running the script again.

If you see ModuleNotFoundError, your virtual environment may not be active. Run venv\Scripts\activate (Windows) or source venv/bin/activate (macOS) and try again.

help me debug a connection or import error when running scraper.py

✔️ Awesome, I've got everything!

Great. Double check you've saved your file before moving on.

ⓧ I'd like to double check the full code

import requests
from bs4 import BeautifulSoup
import csv
import re


URL = "https://www.treasury.gov/auctions/treasury/rp/brokersale.shtml"

HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
}


def fetch_page(url):
    """Fetch a web page and return its HTML content."""
    response = requests.get(url, headers=HEADERS, timeout=30)
    response.raise_for_status()
    print(f"Successfully fetched {url} (Status: {response.status_code})")
    return response.text


if __name__ == "__main__":
    html = fetch_page(URL)
    print(html[:500])

Your scraper can now reach the Treasury website and pull down raw HTML. Next up, you'll parse that HTML to extract individual property listings into structured data.

Fetch the Treasury Auction Page

Your scraper can now reach out to Treasury.gov and pull back raw HTML. That HTML contains property listings buried in text.

The challenge is turning that wall of unstructured text into clean, usable data. In this step, you will write two parsing functions that split the page into individual property blocks and extract structured fields from each one using regular expressions.

In this step, get ready to:
  • Write a function that splits raw HTML into individual property blocks.
  • Write a function that extracts structured fields (type, address, sqft, beds, baths, etc.) from each block.
  • Test the parser by running the script and viewing parsed property data.
Write the parse_properties() function

The Treasury broker sales page does not use semantic CSS classes that you can target with selectors. Instead, property listings appear as flat text separated by type headers like "SINGLE FAMILY HOME:" and "ESTATE HOME:".

Why regex instead of CSS selectors?

Many government websites use simple, legacy HTML without meaningful class names or structured markup. Regex-based text splitting is the appropriate technique here and teaches a transferable pattern you will reuse on other government or legacy sites.

  • In scraper.py, find the fetch_page() function.
  • Add the following parse_properties() function directly below it (before the if __name__ block):
def parse_properties(html):
    """Parse property listings from the Treasury broker sales page."""
    # Convert raw HTML into plain text, preserving line breaks
    soup = BeautifulSoup(html, "html.parser")
    body_text = soup.get_text(separator="\n")

    # Regex pattern that matches each property block by its type header
    property_pattern = r"((?:SINGLE FAMILY HOME|COMMERCIAL PROPERTY|ESTATE HOME|RESIDENTIAL LAND|CONDOMINIUM|MULTI-FAMILY)[^:]*:.*?)(?=(?:SINGLE FAMILY HOME|COMMERCIAL PROPERTY|ESTATE HOME|RESIDENTIAL LAND|CONDOMINIUM|MULTI-FAMILY)[^:]*:|$)"
    blocks = re.findall(property_pattern, body_text, re.DOTALL)

    # Extract structured data from each block
    properties = []
    for block in blocks:
        prop = extract_property_data(block)
        if prop:
            properties.append(prop)
    return properties

What does this code do?

BeautifulSoup strips all HTML tags from the page and gives you plain text. The separator="\n" argument preserves line breaks so the text structure stays intact.

The regex pattern matches any text block that starts with a property type (like SINGLE FAMILY HOME or COMMERCIAL PROPERTY) and captures everything until the next property type header appears. The re.DOTALL flag lets the dot match newlines, so multi-line descriptions are captured in a single block.

Do not save and run yet. The parse_properties() function calls extract_property_data() which does not exist yet.

Write the extract_property_data() function

Each property block contains fields like the property type, street address, square footage, bedroom count, and sale number. You will use a series of regex patterns to pull each field out of the text.

  • Add the following extract_property_data() function directly below parse_properties():
def extract_property_data(block):
    """Extract structured data from a single property text block."""
    property_data = {}

    # Extract property type and address from the first line
    type_match = re.match(r"([A-Z\s\-]+):\s*(.+)", block.strip())
    if type_match:
        property_data["type"] = type_match.group(1).strip()
        property_data["address"] = type_match.group(2).strip().split("\n")[0]
    else:
        return None

    # Extract sale number (e.g., "Sale # 26-66-164")
    sale_match = re.search(r"Sale\s*#\s*([\d\-]+)", block)
    if sale_match:
        property_data["sale_number"] = sale_match.group(1)
    else:
        property_data["sale_number"] = "N/A"

    # Extract square footage (e.g., "1,248 ± sq. ft")
    sqft_match = re.search(r"([\d,]+)\s*±?\s*sq\.?\s*ft", block, re.IGNORECASE)
    if sqft_match:
        property_data["sqft"] = sqft_match.group(1).replace(",", "")
    else:
        property_data["sqft"] = "N/A"

    # Extract bedroom count
    beds_match = re.search(r"(\d+)\s*bedroom", block, re.IGNORECASE)
    if beds_match:
        property_data["bedrooms"] = beds_match.group(1)
    else:
        property_data["bedrooms"] = "N/A"

    # Extract bathroom count
    baths_match = re.search(r"([\d.]+)\s*bath", block, re.IGNORECASE)
    if baths_match:
        property_data["bathrooms"] = baths_match.group(1)
    else:
        property_data["bathrooms"] = "N/A"

    # Extract broker contact info
    broker_match = re.search(r"BROKER SALE:\s*Contact\s*(.+?)(?:\n|$)", block)
    if broker_match:
        property_data["broker"] = broker_match.group(1).strip()
    else:
        property_data["broker"] = "N/A"

    # Extract acreage if listed
    acres_match = re.search(r"([\d.]+)\s*±?\s*acres?", block, re.IGNORECASE)
    if acres_match:
        property_data["acres"] = acres_match.group(1)
    else:
        property_data["acres"] = "N/A"

    # Extract description text after broker info
    desc_match = re.search(r"(?:BROKER SALE:.*?\n)(.+?)(?:For complete details|$)", block, re.DOTALL)
    if desc_match:
        property_data["description"] = desc_match.group(1).strip().replace("\n", " ")
    else:
        property_data["description"] = "N/A"

    return property_data

What does this code do?

Each regex follows the same pattern: search for a known label or format in the text block, capture the value if found, and fall back to "N/A" if the field is missing. This keeps the scraper robust against incomplete listings.

  • re.match checks the very start of the block for the TYPE: ADDRESS pattern.
  • re.search scans anywhere in the block for patterns like "Sale # 26-66-164" or "3 bedrooms".
  • The ± in the sqft and acres patterns is optional (±?) because Treasury listings use the plus-minus symbol for approximate measurements.
Test the parser

Now you need to update the bottom of your script to call these new functions and display the results.

  • In scraper.py, scroll down to the if __name__ == "__main__": block at the bottom of your file.
  • Find these lines:
if __name__ == "__main__":
    html = fetch_page(URL)
    print(html[:500])
  • Replace the print(html[:500]) line with code that parses the HTML and prints the first property, so the block looks like this:
if __name__ == "__main__":
    html = fetch_page(URL)
    properties = parse_properties(html)
    print(f"Found {len(properties)} properties")
    if properties:
        print(properties[0])
  • Save scraper.py.
  • Run your script by entering this command in the terminal:
python scraper.py
  • You should see output that starts with "Successfully fetched" followed by "Found X properties" and a dictionary with keys like type, address, sale_number, sqft, bedrooms, and bathrooms.

Not seeing any properties?

If the script prints "Found 0 properties", the Treasury.gov page may be temporarily unavailable (it occasionally returns 504 timeouts). Wait a few minutes and try again.

If you see a syntax error, double-check that your regex strings use raw string literals (r"...") and that all parentheses in the patterns are balanced.

help me debug why my scraper found 0 properties from the Treasury page

✔️ Awesome, I've got everything!

Great. Double check you have saved scraper.py before moving on.

ⓧ I'd like to double check the full code

import requests
from bs4 import BeautifulSoup
import csv
import re


URL = "https://www.treasury.gov/auctions/treasury/rp/brokersale.shtml"

HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
}


def fetch_page(url):
    """Fetch a web page and return its HTML content."""
    response = requests.get(url, headers=HEADERS, timeout=30)
    response.raise_for_status()
    print(f"Successfully fetched {url} (Status: {response.status_code})")
    return response.text


def parse_properties(html):
    """Parse property listings from the Treasury broker sales page."""
    soup = BeautifulSoup(html, "html.parser")
    body_text = soup.get_text(separator="\n")
    property_pattern = r"((?:SINGLE FAMILY HOME|COMMERCIAL PROPERTY|ESTATE HOME|RESIDENTIAL LAND|CONDOMINIUM|MULTI-FAMILY)[^:]*:.*?)(?=(?:SINGLE FAMILY HOME|COMMERCIAL PROPERTY|ESTATE HOME|RESIDENTIAL LAND|CONDOMINIUM|MULTI-FAMILY)[^:]*:|$)"
    blocks = re.findall(property_pattern, body_text, re.DOTALL)
    properties = []
    for block in blocks:
        prop = extract_property_data(block)
        if prop:
            properties.append(prop)
    return properties


def extract_property_data(block):
    """Extract structured data from a single property text block."""
    property_data = {}
    type_match = re.match(r"([A-Z\s\-]+):\s*(.+)", block.strip())
    if type_match:
        property_data["type"] = type_match.group(1).strip()
        property_data["address"] = type_match.group(2).strip().split("\n")[0]
    else:
        return None
    sale_match = re.search(r"Sale\s*#\s*([\d\-]+)", block)
    if sale_match:
        property_data["sale_number"] = sale_match.group(1)
    else:
        property_data["sale_number"] = "N/A"
    sqft_match = re.search(r"([\d,]+)\s*±?\s*sq\.?\s*ft", block, re.IGNORECASE)
    if sqft_match:
        property_data["sqft"] = sqft_match.group(1).replace(",", "")
    else:
        property_data["sqft"] = "N/A"
    beds_match = re.search(r"(\d+)\s*bedroom", block, re.IGNORECASE)
    if beds_match:
        property_data["bedrooms"] = beds_match.group(1)
    else:
        property_data["bedrooms"] = "N/A"
    baths_match = re.search(r"([\d.]+)\s*bath", block, re.IGNORECASE)
    if baths_match:
        property_data["bathrooms"] = baths_match.group(1)
    else:
        property_data["bathrooms"] = "N/A"
    broker_match = re.search(r"BROKER SALE:\s*Contact\s*(.+?)(?:\n|$)", block)
    if broker_match:
        property_data["broker"] = broker_match.group(1).strip()
    else:
        property_data["broker"] = "N/A"
    acres_match = re.search(r"([\d.]+)\s*±?\s*acres?", block, re.IGNORECASE)
    if acres_match:
        property_data["acres"] = acres_match.group(1)
    else:
        property_data["acres"] = "N/A"
    desc_match = re.search(r"(?:BROKER SALE:.*?\n)(.+?)(?:For complete details|$)", block, re.DOTALL)
    if desc_match:
        property_data["description"] = desc_match.group(1).strip().replace("\n", " ")
    else:
        property_data["description"] = "N/A"
    return property_data


if __name__ == "__main__":
    html = fetch_page(URL)
    properties = parse_properties(html)
    print(f"Found {len(properties)} properties")
    if properties:
        print(properties[0])

Your scraper now fetches live data and parses it into structured dictionaries. Next up, you will format the results into a table and export everything to a CSV file.

Parse Properties from HTML

Your scraper can already fetch the Treasury page and extract structured data from each property listing. Now you need to put that data somewhere useful.

In this step, you will build the export pipeline that saves every scraped property to a CSV file. You will also create a polished main() function that displays results as a formatted table in the terminal before writing them to disk.

In this step, get ready to:
  • Write a CSV export function that saves property data to a spreadsheet-ready file.
  • Write a main function that ties fetching, parsing, displaying, and saving together.
  • Run the complete scraper and verify the output CSV.
Write the CSV export function

Python's built-in csv module includes a DictWriter class that maps dictionary keys to column headers automatically. This is perfect for your list of property dictionaries.

  • In scraper.py, find the extract_property_data() function.
  • Below it (after the return property_data line at the end of that function), add the following save_to_csv() function:
def save_to_csv(properties, filename="treasury_properties.csv"):
    """Save scraped properties to a CSV file."""
    if not properties:
        print("No properties to save.")
        return
    fieldnames = ["type", "address", "sqft", "bedrooms", "bathrooms", "acres", "sale_number", "broker", "description"]
    with open(filename, "w", newline="", encoding="utf-8") as csvfile:
        writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
        writer.writeheader()
        writer.writerows(properties)
    print(f"\nSaved {len(properties)} properties to {filename}")

What does this code do?

This function takes your list of property dictionaries and writes them into a CSV file that any spreadsheet app can open.

  • The early return handles the edge case where the parser found zero properties.
  • fieldnames defines the column order in the output CSV.
  • DictWriter maps each property dictionary's keys to the matching column, so values land in the right place automatically.
  • writeheader() writes the column names as the first row, and writerows() writes all properties at once.

You won't see a change yet. The next chunk wires this function into main() so it actually gets called.

Write the main function

The main() function is the entry point that orchestrates the full scraping workflow. It fetches the page, parses properties, prints a formatted summary table, saves to CSV, and handles errors gracefully.

  • In scraper.py, find the existing if __name__ == "__main__": block at the bottom of the file:
if __name__ == "__main__":
    html = fetch_page(URL)
    properties = parse_properties(html)
    print(f"Found {len(properties)} properties")
    if properties:
        print(properties[0])
  • Delete that entire block and replace it with the following main() function and updated entry point:
def main():
    """Main function to run the Treasury property scraper."""
    print("=" * 60)
    print("U.S. Treasury Seized Property Scraper")
    print("=" * 60)
    print(f"\nTarget: {URL}")
    print("Fetching property listings...\n")
    try:
        html = fetch_page(URL)
        properties = parse_properties(html)
        if properties:
            print(f"\nFound {len(properties)} properties:\n")
            print(f"{'Type':<25} {'Address':<45} {'Sale #'}")
            print("-" * 90)
            for prop in properties:
                print(f"{prop['type']:<25} {prop['address'][:43]:<45} {prop['sale_number']}")
            save_to_csv(properties)
        else:
            print("No properties found. The page structure may have changed.")
    except requests.exceptions.HTTPError as e:
        print(f"HTTP Error: {e}")
    except requests.exceptions.ConnectionError:
        print("Connection Error: Could not reach the website.")
    except requests.exceptions.Timeout:
        print("Timeout: The request took too long.")


if __name__ == "__main__":
    main()

What does this code do?

The main() function runs the full scraping pipeline in sequence.

  • The header banner and target URL give the learner clear feedback that the script is running.
  • The formatted table uses Python f-string alignment (<25, <45) to print columns at fixed widths so the output is easy to scan.
  • Three except blocks catch the most common network failures: bad HTTP status codes, connection errors, and timeouts.
  • if __name__ == "__main__": ensures main() only runs when you execute the script directly, not when importing it as a module.
  • Save scraper.py.
Run the complete scraper

Everything is wired together now. Running the script will fetch the live Treasury page, parse every property, display a summary table, and create your CSV file.

  • Run the scraper by entering this command in your terminal:
python scraper.py
  • You should see a formatted table with columns for Type, Address, and Sale # followed by a confirmation that properties were saved to treasury_properties.csv.
  • Open treasury_properties.csv in VS Code (or any spreadsheet application) to verify the data has clean columns for type, address, sqft, bedrooms, bathrooms, acres, sale_number, broker, and description.

Not seeing any properties?

  • If you see "No properties found," the Treasury website may be temporarily down or its structure changed. Run the script again after a few minutes.
  • If you see "Connection Error" or "Timeout," check your internet connection and try again.
  • If you see a Python error like NameError, make sure save_to_csv is defined above main() in the file. Python reads top to bottom.

Help me debug my Treasury scraper

✔️ Awesome, I've got everything!

Great. Double check you have saved scraper.py and that treasury_properties.csv exists in your treasury-scraper folder.

ⓧ I'd like to double check the full code

Here is the complete scraper.py file at the end of this step:

import requests
from bs4 import BeautifulSoup
import csv
import re


URL = "https://www.treasury.gov/auctions/treasury/rp/brokersale.shtml"

HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
}


def fetch_page(url):
    """Fetch a web page and return its HTML content."""
    response = requests.get(url, headers=HEADERS, timeout=30)
    response.raise_for_status()
    print(f"Successfully fetched {url} (Status: {response.status_code})")
    return response.text


def parse_properties(html):
    """Parse property listings from the Treasury broker sales page."""
    soup = BeautifulSoup(html, "html.parser")
    body_text = soup.get_text(separator="\n")
    property_pattern = r"((?:SINGLE FAMILY HOME|COMMERCIAL PROPERTY|ESTATE HOME|RESIDENTIAL LAND|CONDOMINIUM|MULTI-FAMILY)[^:]*:.*?)(?=(?:SINGLE FAMILY HOME|COMMERCIAL PROPERTY|ESTATE HOME|RESIDENTIAL LAND|CONDOMINIUM|MULTI-FAMILY)[^:]*:|$)"
    blocks = re.findall(property_pattern, body_text, re.DOTALL)
    properties = []
    for block in blocks:
        prop = extract_property_data(block)
        if prop:
            properties.append(prop)
    return properties


def extract_property_data(block):
    """Extract structured data from a single property text block."""
    property_data = {}
    type_match = re.match(r"([A-Z\s\-]+):\s*(.+)", block.strip())
    if type_match:
        property_data["type"] = type_match.group(1).strip()
        property_data["address"] = type_match.group(2).strip().split("\n")[0]
    else:
        return None
    sale_match = re.search(r"Sale\s*#\s*([\d\-]+)", block)
    if sale_match:
        property_data["sale_number"] = sale_match.group(1)
    else:
        property_data["sale_number"] = "N/A"
    sqft_match = re.search(r"([\d,]+)\s*±?\s*sq\.?\s*ft", block, re.IGNORECASE)
    if sqft_match:
        property_data["sqft"] = sqft_match.group(1).replace(",", "")
    else:
        property_data["sqft"] = "N/A"
    beds_match = re.search(r"(\d+)\s*bedroom", block, re.IGNORECASE)
    if beds_match:
        property_data["bedrooms"] = beds_match.group(1)
    else:
        property_data["bedrooms"] = "N/A"
    baths_match = re.search(r"([\d.]+)\s*bath", block, re.IGNORECASE)
    if baths_match:
        property_data["bathrooms"] = baths_match.group(1)
    else:
        property_data["bathrooms"] = "N/A"
    broker_match = re.search(r"BROKER SALE:\s*Contact\s*(.+?)(?:\n|$)", block)
    if broker_match:
        property_data["broker"] = broker_match.group(1).strip()
    else:
        property_data["broker"] = "N/A"
    acres_match = re.search(r"([\d.]+)\s*±?\s*acres?", block, re.IGNORECASE)
    if acres_match:
        property_data["acres"] = acres_match.group(1)
    else:
        property_data["acres"] = "N/A"
    desc_match = re.search(r"(?:BROKER SALE:.*?\n)(.+?)(?:For complete details|$)", block, re.DOTALL)
    if desc_match:
        property_data["description"] = desc_match.group(1).strip().replace("\n", " ")
    else:
        property_data["description"] = "N/A"
    return property_data


def save_to_csv(properties, filename="treasury_properties.csv"):
    """Save scraped properties to a CSV file."""
    if not properties:
        print("No properties to save.")
        return
    fieldnames = ["type", "address", "sqft", "bedrooms", "bathrooms", "acres", "sale_number", "broker", "description"]
    with open(filename, "w", newline="", encoding="utf-8") as csvfile:
        writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
        writer.writeheader()
        writer.writerows(properties)
    print(f"\nSaved {len(properties)} properties to {filename}")


def main():
    """Main function to run the Treasury property scraper."""
    print("=" * 60)
    print("U.S. Treasury Seized Property Scraper")
    print("=" * 60)
    print(f"\nTarget: {URL}")
    print("Fetching property listings...\n")
    try:
        html = fetch_page(URL)
        properties = parse_properties(html)
        if properties:
            print(f"\nFound {len(properties)} properties:\n")
            print(f"{'Type':<25} {'Address':<45} {'Sale #'}")
            print("-" * 90)
            for prop in properties:
                print(f"{prop['type']:<25} {prop['address'][:43]:<45} {prop['sale_number']}")
            save_to_csv(properties)
        else:
            print("No properties found. The page structure may have changed.")
    except requests.exceptions.HTTPError as e:
        print(f"HTTP Error: {e}")
    except requests.exceptions.ConnectionError:
        print("Connection Error: Could not reach the website.")
    except requests.exceptions.Timeout:
        print("Timeout: The request took too long.")


if __name__ == "__main__":
    main()

Your scraper is fully functional. You have a script that fetches live government data, extracts structured fields, displays them in a readable format, and exports everything to a CSV file ready for analysis. Next up, you will take on the Secret Mission and add interactive filtering so users can search by state or property type.

Export to CSV and Display Results

Your scraper now has all its core pieces in place. The fetch_page(), parse_properties(), and extract_property_data() functions handle pulling data from the Treasury website and structuring it into dictionaries.

Now it's time to run the full pipeline end-to-end. You'll execute the scraper, confirm that the formatted table prints correctly in your terminal, and verify that the CSV export produces a clean, structured file you can open in any spreadsheet application.

In this step, get ready to:
  • Run the complete scraper end-to-end.
  • Verify the formatted property table in your terminal.
  • Open and inspect the exported CSV file.
Run the complete scraper

The main() function orchestrates the entire scraping pipeline: fetch the page, parse properties, display a summary table, and save everything to CSV. Running the script triggers this full sequence.

  • Run the scraper by executing this command in your terminal:
python scraper.py

What does main() do?

The main() function ties all your helper functions together in a single pipeline:

  • fetch_page(URL) grabs the raw HTML from Treasury.gov.
  • parse_properties(html) splits that HTML into individual property dictionaries.
  • The formatted print() statements display a summary table with type, address, and sale number columns.
  • save_to_csv(properties) exports everything to a file called treasury_properties.csv.

The try/except block catches network errors (HTTP errors, connection failures, timeouts) so the script fails gracefully instead of crashing with a traceback.

  • You should see a banner that says U.S. Treasury Seized Property Scraper, followed by a success message, a formatted table of properties, and a line confirming how many properties were saved.

What does save_to_csv() do?

Python's built-in csv.DictWriter takes a list of dictionaries and writes each one as a row in a CSV file. The fieldnames list defines the column headers and the order they appear.

The newline="" parameter prevents extra blank lines on Windows. The encoding="utf-8" ensures special characters in addresses render correctly.

Seeing a connection error or timeout?

  • Check that your internet connection is working. Try opening any website in your browser.
  • Treasury.gov can occasionally experience temporary outages. Wait a minute and try again.
  • If you see No properties found, the page structure may have changed since this project was written. The script still ran successfully, but could not locate property blocks in the HTML.

Why is my scraper failing to connect to Treasury.gov?

Inspect the CSV file

The real payoff of a scraper is structured, reusable data. Your terminal table is useful for a quick glance, but the CSV file is what you would share with a team, import into a spreadsheet, or feed into a data analysis tool.

  • Confirm the CSV file was created by running:
ls
  • You should see treasury_properties.csv listed alongside scraper.py and the venv folder in your treasury-scraper directory.
  • Open treasury_properties.csv by clicking on it in the VS Code **Explorer** sidebar.
  • Verify that the first row contains column headers: type, address, sqft, bedrooms, bathrooms, acres, sale_number, broker, description.
  • Check that subsequent rows contain property data with real addresses and sale numbers.

Open CSV in a spreadsheet app

For a cleaner view, double-click treasury_properties.csv in Finder (macOS) or File Explorer (Windows) to open it in your default spreadsheet application. Each column lines up neatly with headers, making it easy to sort and filter.

CSV file is empty or missing?

  • If the file is missing entirely, check your terminal output. The script should print Saved followed by a count. If it printed No properties to save., the parse step returned zero results.
  • Make sure your terminal is in the treasury-scraper folder. The CSV is saved to whatever directory you run the script from.

Why is my CSV file empty or not being created?

✔️ Awesome, I've got everything!

Great. Double check you've saved scraper.py and that your CSV file is in the treasury-scraper folder.

ⓧ I'd like to double check the full code

Here's the complete scraper.py file for comparison:

import requests
from bs4 import BeautifulSoup
import csv
import re


URL = "https://www.treasury.gov/auctions/treasury/rp/brokersale.shtml"

HEADERS = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36"
}


def fetch_page(url):
    """Fetch a web page and return its HTML content."""
    response = requests.get(url, headers=HEADERS, timeout=30)
    response.raise_for_status()
    print(f"Successfully fetched {url} (Status: {response.status_code})")
    return response.text


def parse_properties(html):
    """Parse property listings from the Treasury broker sales page."""
    soup = BeautifulSoup(html, "html.parser")
    body_text = soup.get_text(separator="\n")
    property_pattern = r"((?:SINGLE FAMILY HOME|COMMERCIAL PROPERTY|ESTATE HOME|RESIDENTIAL LAND|CONDOMINIUM|MULTI-FAMILY)[^:]*:.*?)(?=(?:SINGLE FAMILY HOME|COMMERCIAL PROPERTY|ESTATE HOME|RESIDENTIAL LAND|CONDOMINIUM|MULTI-FAMILY)[^:]*:|$)"
    blocks = re.findall(property_pattern, body_text, re.DOTALL)
    properties = []
    for block in blocks:
        prop = extract_property_data(block)
        if prop:
            properties.append(prop)
    return properties


def extract_property_data(block):
    """Extract structured data from a single property text block."""
    property_data = {}
    type_match = re.match(r"([A-Z\s\-]+):\s*(.+)", block.strip())
    if type_match:
        property_data["type"] = type_match.group(1).strip()
        property_data["address"] = type_match.group(2).strip().split("\n")[0]
    else:
        return None
    sale_match = re.search(r"Sale\s*#\s*([\d\-]+)", block)
    if sale_match:
        property_data["sale_number"] = sale_match.group(1)
    else:
        property_data["sale_number"] = "N/A"
    sqft_match = re.search(r"([\d,]+)\s*±?\s*sq\.?\s*ft", block, re.IGNORECASE)
    if sqft_match:
        property_data["sqft"] = sqft_match.group(1).replace(",", "")
    else:
        property_data["sqft"] = "N/A"
    beds_match = re.search(r"(\d+)\s*bedroom", block, re.IGNORECASE)
    if beds_match:
        property_data["bedrooms"] = beds_match.group(1)
    else:
        property_data["bedrooms"] = "N/A"
    baths_match = re.search(r"([\d.]+)\s*bath", block, re.IGNORECASE)
    if baths_match:
        property_data["bathrooms"] = baths_match.group(1)
    else:
        property_data["bathrooms"] = "N/A"
    broker_match = re.search(r"BROKER SALE:\s*Contact\s*(.+?)(?:\n|$)", block)
    if broker_match:
        property_data["broker"] = broker_match.group(1).strip()
    else:
        property_data["broker"] = "N/A"
    acres_match = re.search(r"([\d.]+)\s*±?\s*acres?", block, re.IGNORECASE)
    if acres_match:
        property_data["acres"] = acres_match.group(1)
    else:
        property_data["acres"] = "N/A"
    desc_match = re.search(r"(?:BROKER SALE:.*?\n)(.+?)(?:For complete details|$)", block, re.DOTALL)
    if desc_match:
        property_data["description"] = desc_match.group(1).strip().replace("\n", " ")
    else:
        property_data["description"] = "N/A"
    return property_data


def save_to_csv(properties, filename="treasury_properties.csv"):
    """Save scraped properties to a CSV file."""
    if not properties:
        print("No properties to save.")
        return
    fieldnames = ["type", "address", "sqft", "bedrooms", "bathrooms", "acres", "sale_number", "broker", "description"]
    with open(filename, "w", newline="", encoding="utf-8") as csvfile:
        writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
        writer.writeheader()
        writer.writerows(properties)
    print(f"\nSaved {len(properties)} properties to {filename}")


def main():
    """Main function to run the Treasury property scraper."""
    print("=" * 60)
    print("U.S. Treasury Seized Property Scraper")
    print("=" * 60)
    print(f"\nTarget: {URL}")
    print("Fetching property listings...\n")
    try:
        html = fetch_page(URL)
        properties = parse_properties(html)
        if properties:
            print(f"\nFound {len(properties)} properties:\n")
            print(f"{'Type':<25} {'Address':<45} {'Sale #'}")
            print("-" * 90)
            for prop in properties:
                print(f"{prop['type']:<25} {prop['address'][:43]:<45} {prop['sale_number']}")
            save_to_csv(properties)
        else:
            print("No properties found. The page structure may have changed.")
    except requests.exceptions.HTTPError as e:
        print(f"HTTP Error: {e}")
    except requests.exceptions.ConnectionError:
        print("Connection Error: Could not reach the website.")
    except requests.exceptions.Timeout:
        print("Timeout: The request took too long.")


if __name__ == "__main__":
    main()

You now have a fully working web scraper that pulls real government data and exports it to a reusable format. Next up, you'll take this further with a secret mission that adds interactive filtering to your scraper.

Secret mission

Add Interactive Filtering

Your scraper pulls every property on the page, but what if someone only cares about homes in Florida or commercial buildings? Add interactive filtering so users can narrow results by state or property type, with dynamic CSV filenames to match.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • The treasury-scraper project folder on your Desktop (includes scraper.py, virtual environment, and CSV output files).
  • VS Code (installed in Step 1).

Keep everything running

No action needed. Choose this if you're still actively building or want to experiment with the scraper further.

  • Your treasury-scraper folder stays on your Desktop with all files intact.
  • VS Code stays installed and ready for your next project.
  • The virtual environment and installed libraries (requests, beautifulsoup4) remain inside the project folder.

Pause - I'll come back to this later

Deactivate the virtual environment to free up your terminal session. Your files stay on disk for when you return.

  • In your VS Code terminal, deactivate the virtual environment by running:
deactivate
  • Close VS Code.
  • When you come back, reactivate the environment with venv\Scripts\activate (Windows) or source venv/bin/activate (macOS/Linux) from inside the treasury-scraper folder.

Delete - I don't want to use this again

Remove all project files and start fresh. This deletes the scraper, virtual environment, CSV output, and all installed libraries.

  • Deactivate the virtual environment first (if it is still active):
deactivate
  • Delete the entire project folder. On macOS/Linux, run:
rm -rf ~/Desktop/treasury-scraper
  • On Windows, run:
rmdir /s /q %USERPROFILE%\Desktop\treasury-scraper
  • To uninstall VS Code, open your system settings. On macOS, drag VS Code from Applications to the Trash. On Windows, go to Settings > Apps > Installed apps, find Visual Studio Code, and click Uninstall.

Nice Work!

Nice Work!

You just built a fully functional web scraper that pulls real seized property data from the U.S. Treasury Department and exports it to a clean CSV file.

You've learned how to:

  • Built a working web scraper using Python's requests and BeautifulSoup libraries to fetch and parse live government HTML.
  • Extracted structured data from unstructured web content using regular expressions to pull property types, addresses, square footage, and sale numbers.
  • Exported scraped data to a CSV file using Python's built-in csv module for analysis in any spreadsheet application.
  • Secret Mission: Added interactive filtering so users can search properties by state or type and generate dynamic CSV filenames.

Ready to quiz yourself?