Protein Structure API Scraper

Fetch live protein data from UniProt and visualize amino acid frequencies.

Introduction

30 Second Summary

Proteins are the molecular machines behind everything your body does, from fighting infections to forming memories. When researchers study diseases like Alzheimer's, one of the first things they examine is which amino acids make up the proteins involved.

In this project, you will build a Python script that pulls live Alzheimer's-related protein data from the UniProt REST API. Your script will count every amino acid across 25 proteins and produce a bar chart showing which ones dominate.

What You'll Build

Run a single command and watch your script fetch 25 real protein sequences from UniProt, then pop open a bar chart showing which amino acids dominate Alzheimer's-related proteins.

By the end of this project, you'll have:

  • A working script that fetches 25 real Alzheimer's-related protein sequences from UniProt's live API and saves structured results to a JSON file.
  • A matplotlib bar chart that reveals which amino acids dominate these disease-linked proteins, saved as a PNG you can share.
  • A complete Git-tracked project pushed to GitHub that anyone can clone and run to reproduce your analysis.
  • An optional Secret Mission to push your skills further.

Are there any prerequisites?

You'll need Python and VS Code installed on your computer, along with Git and a GitHub account.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters.

Set Up Your Project Environment

Your protein scraper will rely on two Python libraries to pull data from the web and turn it into charts. Before you can write a single line of code, you need a clean, isolated environment to install them in.

Without that isolation, packages can conflict with other projects on your machine. In this step, you'll create a project folder, set up a virtual environment, install your dependencies, and track it all with Git.

In this step, get ready to:
  • Create a protein-scraper project folder and a Python virtual environment.
  • Install requests and matplotlib and create project configuration files.
  • Initialize a Git repository and make your first commit.
Create the project folder and virtual environment

A virtual environment is an isolated Python installation for your project. Libraries you install inside it stay contained and won't clash with anything else on your machine.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Visual Studio Code and press Enter to open it.
  • Open the built-in terminal by pressing Ctrl+` (backtick key, top-left of your keyboard).
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create the project folder and move into it by running these commands:
mkdir protein-scraper
cd protein-scraper

What do these commands do?

  • cd ~/Desktop moves your terminal to the Desktop so the project folder is easy to find later.
  • mkdir protein-scraper creates a new folder called protein-scraper.
  • cd protein-scraper moves into that folder so everything you do next happens inside it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Select the protein-scraper folder on your Desktop, then click Open.

Seeing a trust dialog?

VS Code may ask if you trust the authors of the files in this folder. Click Yes, I trust the authors to continue. This is a security feature that appears when opening new folders.

  • Open a fresh terminal in VS Code by pressing Ctrl+` (backtick). This terminal is now rooted inside your protein-scraper folder.
  • Create a virtual environment by running this command:

macOS

python3 -m venv venv

Windows

python -m venv venv
  • Confirm the virtual environment was created by running:
ls

You should see a venv folder listed in the output.

  • Activate the virtual environment by running:

macOS

source venv/bin/activate

Windows

venv\Scripts\activate

You should see (venv) appear at the start of your terminal prompt. This confirms you're working inside the virtual environment.

Don't see (venv) in your prompt?

  • Make sure you ran the activate command from inside the protein-scraper folder. Run ls and confirm you see the venv folder listed.
  • On Windows, if you see an error about running scripts being disabled, run Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser and then try the activate command again.
  • Help me activate my Python virtual environment.
Install dependencies and create project files
  • Install the two libraries your project needs by running this command:
pip install requests==2.34.2 matplotlib==3.11.1

You should see download progress followed by a "Successfully installed" message listing both packages and their dependencies.

What are these libraries?

  • requests is a Python library for making HTTP requests. You'll use it to call the UniProt API and download protein data.
  • matplotlib is a plotting library. You'll use it later to create bar charts from your analysis results.

Seeing install errors?

  • Make sure (venv) is showing in your terminal prompt. If it's not, you're installing outside the virtual environment. Go back and activate it first.
  • If you see a "No matching distribution found" error, check that you typed the version numbers exactly as shown, including the double equals sign (==).
  • Help me install Python packages with pip.

Now you'll create two configuration files that keep your project organized and reproducible.

  • In the Explorer sidebar on the left, hover over the PROTEIN-SCRAPER heading.
  • Click the New File icon (a page with a plus sign).
  • Type requirements.txt and press Enter.
  • Paste the following content into the file:
requests==2.34.2
matplotlib==3.11.1

What is requirements.txt?

This file records the exact library versions your project needs. Anyone who clones your project can run pip install -r requirements.txt to get the same setup.

  • Save requirements.txt by pressing Cmd+S (macOS) or Ctrl+S (Windows).
  • Create another new file using the same New File icon.
  • Name it .gitignore and press Enter.
  • Paste this content:
__pycache__/
*.pyc
venv/
.env

What is .gitignore?

This file tells Git which files to skip. The venv/ folder is large and machine-specific. __pycache__/ is auto-generated bytecode. Neither belongs in your repository.

  • Save .gitignore by pressing Cmd+S (macOS) or Ctrl+S (Windows).

You should see both requirements.txt and .gitignore listed in the Explorer sidebar alongside the venv folder.

Don't see your files in the sidebar?

  • Make sure you saved each file after pasting the content. An unsaved file shows a dot on its tab in VS Code.
  • Check that the files appear at the same level as the venv folder, not inside it. If they're nested incorrectly, drag them to the top level in the sidebar.
  • Help me create and save files in VS Code.

✔️ Awesome, I've got everything!

Great. Double check you've saved both files before moving on.

ⓧ I'd like to double check the full code

Your requirements.txt should look exactly like this:

requests==2.34.2
matplotlib==3.11.1

Your .gitignore should look exactly like this:

__pycache__/
*.pyc
venv/
.env
Initialize Git and make the first commit

Git tracks every change you make, so you can always roll back if something breaks. Let's save this clean starting point as your first commit.

  • Initialize a new Git repository by running this command:
git init

You should see a message confirming the repository was created.

  • Stage all your files and create your first commit by running these commands:
git add .
git commit -m "Initial project setup with dependencies"

You should see a summary listing the files committed, including requirements.txt and .gitignore.

What do these Git commands do?

  • git init creates a new Git repository in your project folder.
  • git add . stages all files in the current folder for commit. The .gitignore ensures venv/ and __pycache__/ are excluded automatically.
  • git commit -m "..." saves a snapshot of your staged files with a descriptive message.

Seeing 'Author identity unknown'?

Git needs your name and email before it can create commits. Run these two commands with your own details, then retry the commit command:

git config --global user.name "Your Name"
git config --global user.email "you@example.com"

Help me configure Git for the first time.

Time to confirm everything is wired up correctly. Before you run the next command, what packages do you expect to see in the list?

  • Verify your installed packages by running:
python -m pip list

You should see requests (version 2.34.2) and matplotlib (version 3.11.1) in the output, along with their dependencies.

Your environment is locked and loaded. Next up, you'll write your first API call and pull real protein data from UniProt's servers.

Fetch Your First Protein from UniProt

Your project environment is ready with all the right tools installed. Now it's time to do something with them.

The UniProt database is the world's most comprehensive resource for protein sequence data. Its REST API lets you search for proteins by disease, organism, and other criteria, all without needing an API key. In this step, you'll send your first request to UniProt's servers and see what real protein data looks like.

In this step, get ready to:
  • Send a GET request to the UniProt REST API and explore the raw JSON response.
  • Extract and display protein names, accession IDs, and sequence previews.
  • Commit your working API script to Git.
Send your first API request to UniProt

Before you can analyze any protein data, you need to pull it from a live source. UniProt's search endpoint accepts query parameters that filter by disease keyword, organism, and review status.

  • Create a new file called protein_scraper.py inside your protein-scraper folder by right-clicking in the VS Code Explorer sidebar and selecting New File.
  • Paste the following code into protein_scraper.py:
import requests
import json


API_URL = "https://rest.uniprot.org/uniprotkb/search"


# Set query parameters to find Alzheimer's-related human proteins
params = {
    "query": "(keyword:Alzheimer) AND (organism_id:9606) AND (reviewed:true)",
    "format": "json",
    "size": 5,
}

# Send a GET request to UniProt's search endpoint
response = requests.get(API_URL, params=params)
data = response.json()

# Print the raw response to see the full data structure
print(json.dumps(data, indent=2))

What does this code do?

This script builds a search query for UniProt's REST API. The params dictionary tells the API exactly what you want:

  • keyword:Alzheimer filters for proteins annotated with the Alzheimer disease keyword.
  • organism_id:9606 restricts results to human proteins only.
  • reviewed:true limits results to manually curated Swiss-Prot entries, which have consistent field structures.
  • size controls how many results to fetch. Starting with 5 keeps the output manageable while you explore.
  • Save protein_scraper.py.
  • Run the script by entering this command in your terminal:
python protein_scraper.py

A wall of JSON fills your terminal. That's the raw data UniProt returned for 5 Alzheimer's-related proteins.

What should I see?

You'll see a large JSON object with a results array containing 5 protein entries. Each entry has fields like primaryAccession, proteinDescription, and sequence. Scroll through the output to get a feel for the structure. Those are the fields you'll extract next.

Seeing an error instead of JSON?

  • If you see ModuleNotFoundError: No module named 'requests', your virtual environment may not be activated. Run venv\Scripts\activate (Windows) or source venv/bin/activate (macOS) and try again.
  • If you see a connection error, check your internet connection. The script needs to reach UniProt's servers at rest.uniprot.org.
  • help me debug an error when running protein_scraper.py
Extract and display protein details

The raw JSON dump proved the API works, but it's too noisy to read. Each protein entry has the exact fields you need buried inside nested objects. Time to pull out just the important pieces.

  • In protein_scraper.py, find these two lines at the bottom of the file:
# Print the raw response to see the full data structure
print(json.dumps(data, indent=2))
  • Delete those two lines and replace them with this loop:
# Loop through each protein and display key details
for protein in data["results"]:
    accession = protein["primaryAccession"]
    name = protein["proteinDescription"]["recommendedName"]["fullName"]["value"]
    sequence_preview = protein["sequence"]["value"][:50]
    print(f"{accession}: {name}")
    print(f"  Sequence preview: {sequence_preview}...\n")

What does this loop do?

  • primaryAccession is UniProt's unique ID for each protein (like P02649 for Apolipoprotein E).
  • proteinDescription.recommendedName.fullName.value is the human-readable protein name, nested several layers deep in the JSON.
  • sequence["value"][:50] grabs the first 50 characters of the amino acid sequence. Full sequences can be hundreds or thousands of characters long, so a preview keeps the output readable.
  • Save protein_scraper.py.

Before you run this, what do you think the output will look like? Will you see raw JSON again, or something different?

  • Run the script again:
python protein_scraper.py

Instead of a wall of JSON, you now see 5 clean lines. Each shows a protein's accession ID, its name, and a preview of its amino acid sequence. One of them is Apolipoprotein E, the most well-known Alzheimer's-related protein.

Not seeing 5 proteins?

  • If you see a KeyError, double-check that the nested key path matches exactly: protein["proteinDescription"]["recommendedName"]["fullName"]["value"]. A missing bracket or mismatched quote breaks the chain.
  • If you still see the raw JSON dump, make sure you deleted the print(json.dumps(data, indent=2)) line and replaced it with the for loop.
  • help me fix my protein extraction loop
  • Commit your working script by running these commands:
git add .
git commit -m "Add basic UniProt API fetch"

✔️ Awesome, I've got everything!

Great. Double-check you've saved protein_scraper.py and committed it to Git.

ⓧ I'd like to double check the full code

Here is the full protein_scraper.py at this point. Compare it with yours line by line.

import requests
import json


API_URL = "https://rest.uniprot.org/uniprotkb/search"


params = {
    "query": "(keyword:Alzheimer) AND (organism_id:9606) AND (reviewed:true)",
    "format": "json",
    "size": 5,
}

response = requests.get(API_URL, params=params)
data = response.json()

for protein in data["results"]:
    accession = protein["primaryAccession"]
    name = protein["proteinDescription"]["recommendedName"]["fullName"]["value"]
    sequence_preview = protein["sequence"]["value"][:50]
    print(f"{accession}: {name}")
    print(f"  Sequence preview: {sequence_preview}...\n")

You've pulled live protein data from a real bioinformatics API and can see actual Alzheimer's-related protein names in your terminal. Next up, you'll make this script production-ready with error handling, timeouts, and the ability to save results to a file.

Build a Resilient Data Pipeline

Your script from Step 2 works, but it has three serious gaps. There is no timeout, so a slow server response hangs your script forever. There is no error handling, so one unexpected API entry crashes everything. And the data vanishes the moment you close the terminal.

In this step, you will refactor the flat script into reusable functions with built-in protection. You will also save the results to a JSON file so the data persists between runs.

In this step, get ready to:
  • Refactor the flat script into functions with timeout and error handling.
  • Save extracted protein data to a JSON file that persists between runs.
Refactor into functions with error handling

Right now, protein_scraper.py runs everything at the top level with no structure. You will replace the entire file with functions that are reusable, testable, and resilient.

  • Select all the code in protein_scraper.py and delete it. You are replacing the entire file.
  • Paste the following code to set up your imports and the fetch_proteins function:
import requests
import json


API_URL = "https://rest.uniprot.org/uniprotkb/search"


def fetch_proteins(disease_keyword, max_results=25):
    """Fetch reviewed human proteins associated with a disease keyword from UniProt."""
    params = {
        "query": f"(keyword:{disease_keyword}) AND (organism_id:9606) AND (reviewed:true)",
        "format": "json",
        "size": max_results,
    }

    response = requests.get(API_URL, params=params, timeout=30)
    response.raise_for_status()

    data = response.json()
    return data["results"]

What does this code do?

Instead of hardcoded values, fetch_proteins(disease_keyword, max_results) accepts parameters so you can fetch proteins for any disease, not just Alzheimer's.

  • timeout=30 tells requests to give up after 30 seconds instead of waiting forever if the server is slow.
  • raise_for_status() raises an HTTPError if the API returns a 4xx or 5xx error code, so your script fails loudly instead of silently processing garbage data.
  • Now add the extract_sequences function below fetch_proteins by pasting the following code:
def extract_sequences(proteins):
    """Extract protein names and amino acid sequences from API results."""
    sequences = []

    for protein in proteins:
        try:
            accession = protein["primaryAccession"]
            name = protein["proteinDescription"]["recommendedName"]["fullName"]["value"]
            sequence = protein["sequence"]["value"]
            sequences.append({
                "accession": accession,
                "name": name,
                "sequence": sequence,
                "length": len(sequence),
            })
        except (KeyError, TypeError):
            accession = protein.get("primaryAccession", "Unknown")
            print(f"  Skipping {accession}: missing expected fields")

    return sequences

What does this code do?

extract_sequences loops through each protein and pulls out four fields: accession ID, name, full amino acid sequence, and sequence length.

The try/except block catches KeyError and TypeError so that one malformed entry does not crash the whole script. It prints which protein was skipped and keeps going.

  • Save protein_scraper.py.
  • Verify both functions work by running a quick import test in the terminal:
python -c "from protein_scraper import fetch_proteins, extract_sequences; s = extract_sequences(fetch_proteins('Alzheimer', 3)); print(f'{len(s)} sequences extracted')"

You should see 3 sequences extracted printed in the terminal. This proves both functions are working together.

Seeing a SyntaxError or ImportError?

  • Make sure you deleted all the old flat code before pasting. Leftover lines like response = requests.get(API_URL, params=params) sitting outside a function will run on import and collide with your new functions.
  • Check that extract_sequences is defined at the top level (no extra indentation) and not accidentally nested inside fetch_proteins.
  • Make sure your virtual environment is still active. You should see (venv) at the start of your terminal prompt.

Help me fix an import or syntax error in my protein_scraper.py functions.

Save extracted data to JSON

Your functions fetch and extract data, but the results still vanish when the terminal closes. You need a save_results function to write the data to disk, and a main() function to tie the whole pipeline together.

  • Add the save_results and main functions below extract_sequences by pasting the following code at the bottom of the file:
def save_results(sequences, disease_keyword):
    """Save extracted sequences to a JSON file."""
    filename = f"{disease_keyword.lower()}_sequences.json"
    with open(filename, "w") as f:
        json.dump(sequences, f, indent=2)
    print(f"Results saved to {filename}")


def main():
    disease = "Alzheimer"

    print(f"Fetching {disease}-related proteins from UniProt...")
    proteins = fetch_proteins(disease, max_results=25)
    print(f"Retrieved {len(proteins)} proteins\n")

    print("Extracting sequences...")
    sequences = extract_sequences(proteins)
    print(f"Successfully extracted {len(sequences)} sequences\n")

    save_results(sequences, disease)


if __name__ == "__main__":
    main()

What does this code do?

  • save_results() writes the extracted sequences to a JSON file named after the disease keyword (e.g. alzheimer_sequences.json). The indent=2 argument makes the output human-readable.
  • main() ties the full pipeline together: fetch 25 proteins, extract their sequences, print a summary, and save to disk.
  • if __name__ == "__main__": ensures main() only runs when you execute the file directly, not when you import it.

Before you run this, what do you think the output will look like? Will you see individual protein details, or just a summary?

  • Save protein_scraper.py and run the full pipeline:
python protein_scraper.py

You should see output showing how many proteins were retrieved, a count of successfully extracted sequences, and a confirmation that alzheimer_sequences.json was saved.

  • Confirm the JSON file was created by running:
ls

You should see alzheimer_sequences.json listed alongside protein_scraper.py and your other project files.

  • Open alzheimer_sequences.json in VS Code by clicking on it in the **Explorer** sidebar.

You should see a JSON array containing up to 25 protein objects. Each object has an accession, name, sequence (a long string of amino acid letters), and length.

No JSON file created?

  • Check that if __name__ == "__main__": is at the very bottom of the file with no indentation. If it is inside another function, main() never runs.
  • If you see a connection error, check your internet connection. The script needs to reach rest.uniprot.org to fetch data.
  • If you see a NameError for fetch_proteins or extract_sequences, make sure those functions are defined above main() in the file.

Help me debug why my protein_scraper.py isn't creating the JSON file.

  • Commit your progress by running:
git commit -am "Add error handling and JSON persistence"

✔️ Awesome, I've got everything!

Great. Double check you have saved protein_scraper.py and committed your changes.

ⓧ I'd like to double check the full code

Your protein_scraper.py should contain these sections in order. Scroll up and confirm each is present:

  • import requests and import json at the top.
  • API_URL constant set to the UniProt search endpoint.
  • fetch_proteins(disease_keyword, max_results=25) with timeout=30 and raise_for_status().
  • extract_sequences(proteins) with try/except for KeyError and TypeError.
  • save_results(sequences, disease_keyword) writing to a JSON file.
  • main() calling fetch_proteins with max_results=25, then extract_sequences, then save_results.
  • if __name__ == "__main__": at the very bottom.

Your pipeline now fetches 25 proteins, handles errors gracefully, and saves structured data to disk. Next up, you will analyze those sequences to discover which amino acids dominate Alzheimer's-related proteins.

Analyze Amino Acid Frequencies

In the previous step, you built a resilient pipeline that fetches 25 Alzheimer's-related proteins from UniProt and saves them to alzheimer_sequences.json. But right now, each protein sequence is just a long string of single-letter amino acid codes.

To discover which amino acids dominate Alzheimer's proteins, you need to count every amino acid across all 25 sequences and normalize to percentages. Without normalizing, longer proteins would skew the results.

In this step, get ready to:
  • Count every amino acid occurrence across all 25 protein sequences.
  • Convert raw counts to percentage frequencies sorted alphabetically.
  • Save the full analysis results to a structured JSON file.
Count amino acids across all sequences

Each protein sequence is a string like MKVLWAALLVTFLAGC... where every character represents one amino acid. To find out which amino acids are most common in Alzheimer's proteins, you need a function that walks through every character in every sequence and tallies them up.

  • In protein_scraper.py, find the import lines at the top of the file:
import requests
import json
  • Add import matplotlib.pyplot as plt on a new line below import json, so the imports now look like this:
import requests
import json
import matplotlib.pyplot as plt

Why add matplotlib now?

You will use matplotlib in the next step to create a bar chart. Adding the import now keeps all imports at the top of the file where they belong.

  • Now scroll down to the end of the extract_sequences function.
  • Add the count_amino_acids function below it by copying this code:
def count_amino_acids(sequences):
    """Count amino acid occurrences across all sequences."""
    counts = {}
    total = 0

    for entry in sequences:
        for amino_acid in entry["sequence"]:
            counts[amino_acid] = counts.get(amino_acid, 0) + 1
            total += 1

    return counts, total

What does this code do?

  • counts is a dictionary that maps each amino acid letter (like "A", "L", "G") to how many times it appears across all sequences.
  • counts.get(amino_acid, 0) + 1 safely increments the count, defaulting to 0 for amino acids seen for the first time.
  • total tracks the grand total of all amino acids processed. You will use this to calculate percentages.
  • Next, add the calculate_frequencies function directly below count_amino_acids by copying this code:
def calculate_frequencies(counts, total):
    """Convert raw amino acid counts to sorted percentage frequencies."""
    frequencies = {}

    for aa, count in counts.items():
        frequencies[aa] = round((count / total) * 100, 2)

    return dict(sorted(frequencies.items()))

What does this code do?

  • round((count / total) * 100, 2) converts each raw count into a percentage of the total, rounded to 2 decimal places.
  • dict(sorted(frequencies.items())) sorts the results alphabetically by amino acid letter, so the output is consistent every time you run it.
  • Normalizing to percentages matters because longer proteins contribute more raw characters. Percentages let you compare fairly regardless of protein length.
  • Save protein_scraper.py.

Your two analysis functions are ready. Next, you will wire them into the pipeline so the script actually calls them.

Calculate percentage frequencies and save

The current save_results function only saves raw sequences. You need to replace it with a version that saves the full analysis: disease name, protein count, total residues, frequency data, and a summary of each protein.

  • In protein_scraper.py, find the existing save_results function:
def save_results(sequences, disease_keyword):
    """Save extracted sequences to a JSON file."""
    filename = f"{disease_keyword.lower()}_sequences.json"
    with open(filename, "w") as f:
        json.dump(sequences, f, indent=2)
    print(f"Results saved to {filename}")
  • Replace the entire save_results function with this updated version:
def save_results(sequences, frequencies, total, disease_keyword):
    """Save analysis results to a JSON file."""
    results = {
        "disease": disease_keyword,
        "protein_count": len(sequences),
        "total_residues": total,
        "frequencies": frequencies,
        "proteins": [
            {"accession": s["accession"], "name": s["name"], "length": s["length"]}
            for s in sequences
        ],
    }

    filename = f"{disease_keyword.lower()}_amino_acid_frequencies.json"
    with open(filename, "w") as f:
        json.dump(results, f, indent=2)

    print(f"Results saved to {filename}")
    return filename

What changed?

  • The function now accepts frequencies and total as parameters, so it can save the analysis results alongside the protein metadata.
  • Instead of dumping raw sequences, it builds a results dictionary with structured fields: disease name, protein count, total residues, frequencies, and a compact protein summary.
  • The output file is now named alzheimer_amino_acid_frequencies.json to reflect its richer content.

Now the main() function needs to call the two new analysis functions and print a frequency table to the terminal.

  • Find the existing main() function:
def main():
    disease = "Alzheimer"

    print(f"Fetching {disease}-related proteins from UniProt...")
    proteins = fetch_proteins(disease, max_results=25)
    print(f"Retrieved {len(proteins)} proteins\n")

    print("Extracting sequences...")
    sequences = extract_sequences(proteins)
    print(f"Successfully extracted {len(sequences)} sequences\n")

    save_results(sequences, disease)
  • Replace the entire main() function with this updated version:
def main():
    disease = "Alzheimer"

    print(f"Fetching {disease}-related proteins from UniProt...")
    proteins = fetch_proteins(disease)
    print(f"Retrieved {len(proteins)} proteins\n")

    print("Extracting sequences...")
    sequences = extract_sequences(proteins)
    print(f"Successfully extracted {len(sequences)} sequences\n")

    for entry in sequences[:5]:
        print(f"  {entry['accession']}: {entry['name']} ({entry['length']} aa)")
    if len(sequences) > 5:
        print(f"  ... and {len(sequences) - 5} more\n")

    print("Counting amino acids...")
    counts, total = count_amino_acids(sequences)
    frequencies = calculate_frequencies(counts, total)

    print(f"\nAmino Acid Frequencies ({total:,} total residues):")
    print("-" * 40)
    for aa, freq in frequencies.items():
        print(f"  {aa}: {freq}% ({counts[aa]:,})")

    save_results(sequences, frequencies, total, disease)

What does the updated main do?

  • It prints a preview of the first 5 proteins with their accession, name, and length so you can spot-check the fetched data.
  • It calls count_amino_acids(sequences) to tally every amino acid, then calculate_frequencies(counts, total) to convert tallies to percentages.
  • It prints a formatted frequency table showing each amino acid's percentage and raw count.
  • Finally, it passes the new frequencies and total data to the updated save_results().
  • Save protein_scraper.py.

Before you run the script, what do you think the output will look like? How many amino acids do you expect to see listed?

  • Run the full pipeline by running this command:
python protein_scraper.py

You should see the script fetch proteins from UniProt, print a preview of 5 proteins (including Apolipoprotein E), then print a frequency table showing all 20 amino acids with their percentage and raw count. At the bottom, you should see a confirmation that alzheimer_amino_acid_frequencies.json was saved.

Seeing a TypeError or missing output?

  • If you see a TypeError about save_results() arguments, make sure you replaced the old save_results function entirely. The new version takes four parameters instead of two.
  • If you see a NameError for count_amino_acids or calculate_frequencies, confirm both functions are defined above main() in the file. Python reads top-to-bottom, so functions must be defined before they are called.
  • If the frequency table is empty or missing amino acids, check that count_amino_acids loops through entry["sequence"] (the string), not entry (the dictionary).

Why is my protein scraper script throwing an error?

  • Open alzheimer_amino_acid_frequencies.json in VS Code by clicking on it in the Explorer sidebar.

You should see a structured JSON object with disease, protein_count, total_residues, a frequencies object listing every amino acid with its percentage, and a proteins array with each protein's accession, name, and length.

✔️ Awesome, I've got everything!

Great. Double check you have saved protein_scraper.py.

ⓧ I'd like to double check the full code

Here is the complete protein_scraper.py at this point. Scroll through and confirm each section is present in this order: imports, API_URL, fetch_proteins, extract_sequences, count_amino_acids, calculate_frequencies, save_results, main, and the if __name__ block.

import requests
import json
import matplotlib.pyplot as plt


API_URL = "https://rest.uniprot.org/uniprotkb/search"
def fetch_proteins(disease_keyword, max_results=25):
    """Fetch reviewed human proteins associated with a disease keyword from UniProt."""
    params = {
        "query": f"(keyword:{disease_keyword}) AND (organism_id:9606) AND (reviewed:true)",
        "format": "json",
        "size": max_results,
    }

    response = requests.get(API_URL, params=params, timeout=30)
    response.raise_for_status()

    data = response.json()
    return data["results"]
def extract_sequences(proteins):
    """Extract protein names and amino acid sequences from API results."""
    sequences = []

    for protein in proteins:
        try:
            accession = protein["primaryAccession"]
            name = protein["proteinDescription"]["recommendedName"]["fullName"]["value"]
            sequence = protein["sequence"]["value"]
            sequences.append({
                "accession": accession,
                "name": name,
                "sequence": sequence,
                "length": len(sequence),
            })
        except (KeyError, TypeError):
            accession = protein.get("primaryAccession", "Unknown")
            print(f"  Skipping {accession}: missing expected fields")

    return sequences
def count_amino_acids(sequences):
    """Count amino acid occurrences across all sequences."""
    counts = {}
    total = 0

    for entry in sequences:
        for amino_acid in entry["sequence"]:
            counts[amino_acid] = counts.get(amino_acid, 0) + 1
            total += 1

    return counts, total


def calculate_frequencies(counts, total):
    """Convert raw amino acid counts to sorted percentage frequencies."""
    frequencies = {}

    for aa, count in counts.items():
        frequencies[aa] = round((count / total) * 100, 2)

    return dict(sorted(frequencies.items()))
def save_results(sequences, frequencies, total, disease_keyword):
    """Save analysis results to a JSON file."""
    results = {
        "disease": disease_keyword,
        "protein_count": len(sequences),
        "total_residues": total,
        "frequencies": frequencies,
        "proteins": [
            {"accession": s["accession"], "name": s["name"], "length": s["length"]}
            for s in sequences
        ],
    }

    filename = f"{disease_keyword.lower()}_amino_acid_frequencies.json"
    with open(filename, "w") as f:
        json.dump(results, f, indent=2)

    print(f"Results saved to {filename}")
    return filename
def main():
    disease = "Alzheimer"

    print(f"Fetching {disease}-related proteins from UniProt...")
    proteins = fetch_proteins(disease)
    print(f"Retrieved {len(proteins)} proteins\n")

    print("Extracting sequences...")
    sequences = extract_sequences(proteins)
    print(f"Successfully extracted {len(sequences)} sequences\n")

    for entry in sequences[:5]:
        print(f"  {entry['accession']}: {entry['name']} ({entry['length']} aa)")
    if len(sequences) > 5:
        print(f"  ... and {len(sequences) - 5} more\n")

    print("Counting amino acids...")
    counts, total = count_amino_acids(sequences)
    frequencies = calculate_frequencies(counts, total)

    print(f"\nAmino Acid Frequencies ({total:,} total residues):")
    print("-" * 40)
    for aa, freq in frequencies.items():
        print(f"  {aa}: {freq}% ({counts[aa]:,})")

    save_results(sequences, frequencies, total, disease)


if __name__ == "__main__":
    main()
  • Commit your changes by running this command:
git commit -am "Add amino acid frequency analysis"

Your pipeline now fetches, extracts, counts, and analyzes amino acid frequencies across 25 Alzheimer's-related proteins. Next up, you will turn those numbers into a visual bar chart and push the whole project to GitHub.

Visualize and Ship to GitHub

Your amino acid analysis is complete. You have counted every amino acid across 25 Alzheimer's-related proteins and converted the results to percentage frequencies.

But scanning 20 rows of numbers in a terminal makes patterns hard to spot. A bar chart turns that data into something instantly readable. In this step, you'll use matplotlib to visualize your frequency data and push the complete project to GitHub.

In this step, get ready to:
  • Create a bar chart that visualizes amino acid frequencies.
  • Push the complete project to a GitHub repository.
Create the bar chart function

matplotlib's pyplot module gives you a simple interface for creating charts. You already imported it at the top of your file as plt in a previous step.

  • In protein_scraper.py, scroll down to the main() function.
  • Add the following create_bar_chart function directly above the def main(): line:
def create_bar_chart(frequencies, disease_keyword, protein_count, total_residues):
    """Create and save a bar chart of amino acid frequencies."""
    amino_acids = list(frequencies.keys())
    percentages = list(frequencies.values())

    plt.figure(figsize=(12, 6))
    plt.bar(amino_acids, percentages, color="steelblue", edgecolor="white")

    plt.title(
        f"Amino Acid Frequencies in {disease_keyword}-Related Human Proteins\n"
        f"({protein_count} proteins, {total_residues:,} total residues)",
        fontsize=14,
    )
    plt.xlabel("Amino Acid", fontsize=12)
    plt.ylabel("Frequency (%)", fontsize=12)
    plt.xticks(fontsize=10)
    plt.yticks(fontsize=10)
    plt.grid(axis="y", alpha=0.3)
    plt.tight_layout()

    filename = f"{disease_keyword.lower()}_amino_acid_chart.png"
    plt.savefig(filename, dpi=150)
    print(f"Chart saved to {filename}")
    plt.show()

What does this code do?

This function takes your frequency dictionary and turns it into a visual bar chart.

  • plt.figure(figsize=(12, 6)) creates a wide canvas so all 20 amino acid labels fit without overlapping.
  • plt.bar() draws one bar per amino acid, colored steelblue with white edges for clarity.
  • plt.savefig() saves the chart as a PNG file before plt.show() displays it. The save must come first because plt.show() can clear the figure on some systems.
  • plt.grid(axis="y", alpha=0.3) adds faint horizontal grid lines so you can read approximate values from the bars.

Now connect the chart to your pipeline. In main(), find this line near the bottom of the function:

    save_results(sequences, frequencies, total, disease)
  • Add these lines directly below the save_results call, so the chart generates after saving:
    print("\nGenerating chart...")
    create_bar_chart(frequencies, disease, len(sequences), total)

What does this addition do?

These two lines wire the new function into your pipeline. After saving the JSON results, the script prints a status message and calls create_bar_chart() with the frequency data, disease name, protein count, and total residue count.

  • Save protein_scraper.py.

Before you run the script, what do you think will happen differently this time? Think about what new output you expect to see beyond the frequency table.

  • Run the full pipeline by running this command:
python protein_scraper.py

After the familiar frequency table prints, a matplotlib window pops up with your bar chart. You should see steelblue bars for each of the 20 amino acids, with the tallest bars showing which amino acids are most common in Alzheimer's-related proteins.

Why does the terminal seem frozen?

The plt.show() call opens an interactive window and blocks the terminal until you close it. This is normal. Close the chart window to return to your terminal prompt.

  • Close the chart window.
  • Confirm the chart image was saved by running:
ls

You should see alzheimer_amino_acid_chart.png listed alongside your other project files.

Chart window didn't appear?

  • Make sure you saved protein_scraper.py before running the script. Unsaved changes will not execute.
  • Check that the create_bar_chart function is defined above main(), not inside it. The indentation should be at the top level (no leading spaces on the def line).
  • If you see the PNG file saved but no window appeared, your matplotlib backend may not support interactive display. The chart is still saved successfully as a file.

Why isn't my matplotlib chart showing up when I run protein_scraper.py?

✔️ Awesome, I've got everything!

Great. Double check you have saved protein_scraper.py and that alzheimer_amino_acid_chart.png appears in your project folder.

ⓧ I'd like to double check the full code

Your protein_scraper.py should contain these functions in this order. Scroll through your file and confirm each one is present:

  • Imports (requests, json, matplotlib.pyplot) and the API_URL constant at the top.
  • fetch_proteins() function.
  • extract_sequences() function.
  • count_amino_acids() function.
  • calculate_frequencies() function.
  • save_results() function.
  • create_bar_chart() function (new in this step).
  • main() function ending with create_bar_chart(frequencies, disease, len(sequences), total).
  • The if __name__ == "__main__": guard at the very bottom.
Push to GitHub

Your project is complete locally. Pushing it to GitHub gives you a shareable link and backs up your work with full version history.

  • Open GitHub in your browser and sign in.
  • In the upper-right corner, click the + icon and select New repository.
  • Enter protein-scraper as the repository name.
  • Leave all other settings as their defaults. Do not initialize with a README, .gitignore, or license since your local project already has these files.
  • Click Create repository.

GitHub shows a quick setup page with a URL for your new repository. You need this URL to connect your local project to the remote.

  • Copy the HTTPS URL from the quick setup page. It will look like https://github.com/your-username/protein-scraper.git.
  • Switch back to your terminal in VS Code. Link your local repository to GitHub by running this command:
git remote add origin [[REPO_URL="https://github.com/your-username/protein-scraper.git"]]
  • Stage all files, commit, and push to GitHub by running these commands:
git add .
git commit -m "Add visualization and complete project"
git push -u origin main

What do these commands do?

  • git remote add origin tells Git where your remote repository lives on GitHub.
  • git add . stages every file in the project for the next commit.
  • git push -u origin main uploads your local commits to GitHub. The -u flag sets origin main as the default upstream, so future pushes only need git push.

Push was rejected or authentication failed?

  • If you see an authentication error, make sure you are signed in to Git with your GitHub credentials. GitHub no longer accepts passwords for HTTPS pushes. You need a personal access token or to use Git credential manager.
  • If the push is rejected because the remote contains work you do not have locally, you may have initialized the GitHub repo with a README. Delete the repository on GitHub, recreate it without initializing, and try the push again.

Help me fix my git push to GitHub. I'm getting an authentication or rejected error.

Before you check GitHub, what files do you expect to see in the repository? Think about which files Git is tracking and which ones your .gitignore excludes.

  • Open your repository URL in your browser. You should see protein_scraper.py, requirements.txt, .gitignore, alzheimer_amino_acid_frequencies.json, and alzheimer_amino_acid_chart.png listed in the file view.

Where is the venv folder?

Your .gitignore file excludes the venv/ directory from Git. Virtual environments are machine-specific and should never be committed. Anyone who clones your repo can recreate it from requirements.txt.

Secret mission

Compare Alzheimer's vs Parkinson's Proteins

Your pipeline currently analyzes only one disease. What if you could compare the amino acid building blocks of two different neurodegenerative diseases side by side? In this secret mission, you'll extend your scraper to pull Parkinson's data and generate a grouped bar chart that reveals whether different diseases favor different amino acids.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally and uses a free public API, so there are no ongoing costs.

Resources you used:

  • protein-scraper project folder on your Desktop (includes your virtual environment, Python scripts, generated JSON files, and chart images).
  • GitHub repository containing the pushed project.

Keep everything running

No action needed. Choose this if you are still actively building or want to experiment with more disease keywords.

  • Your protein-scraper folder stays on your Desktop with all files intact.
  • Your GitHub repository remains public and shareable.
  • To reactivate your virtual environment later, open a terminal in the protein-scraper folder and run venv\Scripts\activate (Windows) or source venv/bin/activate (macOS/Linux).

Pause - I'll come back to this later

Deactivate your virtual environment to free up terminal resources. Your files stay on disk and you can pick up where you left off any time.

  • In your terminal, deactivate the virtual environment by running:
deactivate
  • Close the VS Code window. Your project folder and GitHub repository remain untouched.

Delete - I don't want to use this again

Remove all project files from your machine and delete the GitHub repository.

  • Deactivate your virtual environment by running:
deactivate
  • Close VS Code so no files are locked.
  • Delete the project folder. On macOS/Linux, open a new terminal and run:
rm -rf ~/Desktop/protein-scraper

On Windows, open a new Command Prompt and run:

rmdir /s /q %USERPROFILE%\Desktop\protein-scraper
  • Delete the GitHub repository. Open your repository page on GitHub.
  • Click the Settings tab near the top of the page.
  • Scroll to the bottom of the page and follow the prompts to delete the repository.

Nice Work!

Nice Work!

You just built a complete data pipeline that connects to a live bioinformatics API, analyzes real protein data, and produces a bar chart you can share with anyone.

From API call to amino acid analysis to GitHub, this is a real data project that proves you can pull live data, crunch it, and visualize it.

You've learned how to:

  • Build a Python data pipeline that pulls live protein data from the UniProt REST API with production-grade error handling, timeouts, and graceful recovery from malformed entries.
  • Analyze amino acid frequencies across 25 Alzheimer's-related human proteins and visualize the results with a matplotlib bar chart saved as a shareable PNG.
  • Track a full project with Git and ship it to GitHub as a complete, runnable repository anyone can clone.
  • Secret Mission: Compared amino acid distributions between Alzheimer's and Parkinson's disease proteins with a side-by-side grouped bar chart.

Ready to quiz yourself?