Build a Music Discovery Crawler
Crawl the Last.fm API to find similar artists and generate an HTML recommendations page.
Introduction
30 Second Summary
Finding new music you actually enjoy is surprisingly hard. You could scroll through algorithmically-generated playlists for hours, or you could build a tool that maps out the musical universe around your favorite artist.
In this project, you will build a Python script that crawls the Last.fm music database to discover artists similar to any musician you choose. It fetches their top albums and tracks, then generates a styled HTML page you can browse for new music recommendations.
What You'll Build
A Python crawler that queries the Last.fm API to discover similar artists and generates a dark-themed HTML page displaying their top albums, tracks, cover art, and similarity scores.
By the end of this project, you'll have:
- A working Python crawler that discovers musicians similar to any artist you choose using the Last.fm API.
- A generated HTML recommendations page with styled artist cards, album cover art, track listings, and similarity percentages.
- A multi-level crawling system that explores two layers deep to surface less obvious musical connections.
- Secret Mission: Add genre tag filtering so you can narrow recommendations to specific sounds.
Are there any prerequisites?
This project assumes basic Python knowledge (functions, loops, variables) and familiarity with running commands in the terminal. No web development experience is needed.
Before We Start
Before you can crawl the Last.fm music database, you need a few things in place. You need a way to make web requests from Python, an API key to authenticate with Last.fm, and a project file to hold your code.
In this step, you'll install your only external dependency, sign up for a free Last.fm API account, and create the starter script with its configuration constants.
In this step, get ready to:
- Install the requests library for making HTTP calls.
- Create a Last.fm API account and copy your API key.
- Create the music_crawler.py file with initial imports and configuration.
Install the requests library
Python's requests library makes it simple to send HTTP requests to web APIs. You'll use it throughout this project to talk to Last.fm's servers.
- Press Cmd+Space to open Spotlight search.
- Type Visual Studio Code and press Enter to open it.
- Open the built-in terminal by pressing Ctrl+` (backtick key, top-left of your keyboard).
- Install the requests library by running this command:
pip3 install requests==2.32.5What does this command do?
pip3 install requests==2.32.5 tells Python's package manager to download and install version 2.32.5 of the requests library. Pinning to a specific version ensures your code behaves the same way every time you run it.
- Verify the installation succeeded by running:
python3 -c "import requests; print(requests.__version__)"✔️ I see 2.32.5
The requests library is ready to go. Move on to the next substep.
ⓧ I see an error
If you see ModuleNotFoundError or an install error, try the following:
- Make sure you typed pip3 (not pip). On macOS, pip3 points to your Python 3 installation.
- If you see a permissions error, try running pip3 install --user requests==2.32.5 instead.
- Close and reopen your terminal, then run the verify command again.
Still stuck? Help me fix my requests installation error.
Create a Last.fm API account
The Last.fm API gives you free access to music data like similar artists, top albums, and track information. You need an API key to identify your application when making requests.
- Open your browser and go to https://www.last.fm/api.
- If you do not have a Last.fm account yet, click Sign Up at the top of the page and create a free account first.
- Once you are logged in, return to https://www.last.fm/api and click Get an API account.
- Fill in the application form. The name and description can be anything (e.g. Music Discovery Crawler for both fields).
- Submit the form. You will see two values: an API Key and a Shared Secret.
- Copy your API Key and paste it somewhere safe (a note, a text file). You only need the API Key for this project.
Why only the API Key?
The Shared Secret is used for write operations that modify a user's Last.fm profile. Since your crawler only reads public data (similar artists, albums, tracks), the API Key alone is sufficient for authentication.
Don't see the Get an API account link?
Make sure you are logged in to your Last.fm account first. The link only appears for authenticated users.
Help me create a Last.fm API account.
Create your project file
Now you'll create a project folder on your Desktop and set up the main Python file with its imports and configuration.
- Move to your Desktop by running this command:
cd ~/Desktop- Create your project folder and move into it by running these commands:
mkdir music-crawler
cd music-crawler- Open this folder in VS Code by running:
code .What does code . do?
The code . command opens the current folder in VS Code. The dot means "this directory." VS Code will reload with music-crawler as your workspace, and you'll see it in the Explorer sidebar on the left.
- In the Explorer sidebar on the left, click the New File... button.
- Type music_crawler.py and press Enter.
The file opens in the editor. Now you'll add the initial imports and configuration constants that the rest of the project depends on.
- Paste the following code into music_crawler.py:
import requests
import time
import webbrowser
import os
# --- Configuration ---
API_KEY = "your-api-key-here"
BASE_URL = "https://ws.audioscrobbler.com/2.0/"
SEED_ARTIST = "Radiohead"
MAX_SIMILAR = 5
MAX_ALBUMS = 5
MAX_TRACKS = 5
MAX_DEPTH = 2
REQUEST_DELAY = 0.2What does this code do?
The imports bring in the tools you need: requests for HTTP calls, time for adding delays between API requests, webbrowser for opening your final HTML page, and os for file path handling.
The configuration section sets your API_KEY placeholder (you'll swap in your real key shortly), the Last.fm BASE_URL, a starting artist to search from, and limits for how many results to fetch at each level.
- Replace your-api-key-here with the API Key you copied from Last.fm. Keep the quotes around it.
- Save music_crawler.py by pressing Cmd+S.
- Confirm the file exists by running:
lsYou should see music_crawler.py listed in the output.
Don't see music_crawler.py?
Make sure your terminal is inside the music-crawler folder. Run pwd to check your current directory. If it does not end with music-crawler, run cd ~/Desktop/music-crawler to get there.
Help me find my project file.
✔️ Awesome, I've got everything!
Great. Double check you've saved music_crawler.py and you're ready to move on.
ⓧ I'd like to double check the full code
Here is the complete music_crawler.py file at the end of this step:
import requests
import time
import webbrowser
import os
# --- Configuration ---
API_KEY = "your-api-key-here"
BASE_URL = "https://ws.audioscrobbler.com/2.0/"
SEED_ARTIST = "Radiohead"
MAX_SIMILAR = 5
MAX_ALBUMS = 5
MAX_TRACKS = 5
MAX_DEPTH = 2
REQUEST_DELAY = 0.2Your environment is ready and your starter script is configured. Next up, you'll write the function that reaches out to Last.fm and discovers artists similar to your seed artist.
Discover Similar Artists
Your environment is set up and your API key is ready to go. Now it's time to write the code that actually talks to Last.fm.
The Last.fm API has a method called artist.getSimilar that returns a list of artists who sound like any artist you give it. Each result includes a similarity score between 0 and 1, a link to the artist's page, and image URLs. In this step, you will write a function that calls this endpoint and parses the response into clean Python data.
In this step, get ready to:
- Write a function that calls the Last.fm API to find similar artists.
- Test the function with a real artist and verify you get results back.
Write the image helper function
Last.fm returns images as an array of objects, each with a different size (small, medium, large, extralarge). You need a helper that picks the best available image from that array.
- In music_crawler.py, scroll below the configuration section and add the following helper function:
def get_image_url(images):
"""Extract the largest available image URL from Last.fm image array."""
for size in ["extralarge", "large", "medium", "small"]:
for img in images:
if img.get("size") == size and img.get("#text"):
return img["#text"]
return ""What does this code do?
The function loops through image sizes from largest to smallest, looking for the first one that has an actual URL.
Last.fm stores the image URL in a key called #text (a quirk of how their XML-to-JSON translation works). If no image exists at all, the function returns an empty string so the rest of your code can handle the missing image gracefully.
You won't see a change yet. The next chunk uses this helper inside the main similarity function.
Write the function to search for similar artists
This is the core of your crawler. The function sends a request to the artist.getSimilar endpoint, checks for errors, and returns a clean list of artist dictionaries.
- Below get_image_url, add the get_similar_artists function:
def get_similar_artists(artist_name, limit=MAX_SIMILAR):
"""Fetch artists similar to the given artist from Last.fm."""
params = {
"method": "artist.getSimilar",
"artist": artist_name,
"api_key": API_KEY,
"format": "json",
"limit": limit,
}
response = requests.get(BASE_URL, params=params)
if response.status_code != 200:
print(f"Error fetching similar artists for '{artist_name}': HTTP {response.status_code}")
return []
data = response.json()
if "error" in data:
print(f"API error for '{artist_name}': {data['message']}")
return []
artists = data.get("similarartists", {}).get("artist", [])
results = []
for artist in artists:
results.append({
"name": artist.get("name", "Unknown"),
"match": float(artist.get("match", 0)),
"url": artist.get("url", ""),
"image": get_image_url(artist.get("image", [])),
})
return resultsWhat does this code do?
- params builds the query string for the REST API call, including your API key and the format=json flag so Last.fm returns JSON instead of XML.
- The first if block handles network-level failures (server down, bad URL).
- The second if block catches Last.fm-specific errors (invalid artist name, bad API key).
- The for loop extracts only the fields you need from each artist and converts the match score to a Python float.
- Save music_crawler.py.
Seeing a red underline in your editor?
If VS Code shows a warning on the requests import, make sure you installed the library in Step 1. Check that VS Code is using the correct Python interpreter by pressing Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows) and selecting Python: Select Interpreter.
help me fix a Python import error for requests in VS Code
Test the similarity search
Your function is written, but you need to confirm it actually talks to Last.fm and gets real data back. You will add temporary test code at the bottom of the file, run the script, then remove the test code afterwards.
- At the very bottom of music_crawler.py, add this temporary test block:
# --- Temporary test (remove after verifying) ---
results = get_similar_artists(SEED_ARTIST)
for artist in results:
print(f"{artist['name']} - {artist['match']:.0%} match")
print(f" URL: {artist['url']}")
print()What does this test do?
It calls your get_similar_artists function with the SEED_ARTIST value ("Radiohead" by default) and prints each result with its name, similarity percentage, and Last.fm URL.
- Save music_crawler.py.
- Run the script by entering this command in your terminal:
python3 music_crawler.py- You should see 5 artists printed with their similarity percentages and URLs. The exact artists depend on Last.fm's current data, but you will see names, percentage values, and last.fm URLs.
Not seeing any output?
- Check that you replaced your-api-key-here in the API_KEY variable with your actual Last.fm API key from Step 1.
- If you see an "API error" message mentioning an invalid key, double-check you copied the API Key (not the Shared Secret) from your Last.fm API account page.
- Make sure you are running the command from inside the music-crawler folder.
help me troubleshoot my Last.fm API connection. I am not seeing any output when I run python3 music_crawler.py
- Once you have confirmed the output looks correct, delete the temporary test block (the last 5 lines starting with # --- Temporary test) from the bottom of the file.
- Save music_crawler.py.
✔️ Awesome, I've got everything!
Great. Double check you have saved music_crawler.py with the test block removed.
ⓧ I'd like to double check the full code
Here is the complete music_crawler.py at this point:
import requests
import time
import webbrowser
import os
# --- Configuration ---
API_KEY = "your-api-key-here"
BASE_URL = "https://ws.audioscrobbler.com/2.0/"
SEED_ARTIST = "Radiohead"
MAX_SIMILAR = 5
MAX_ALBUMS = 5
MAX_TRACKS = 5
MAX_DEPTH = 2
REQUEST_DELAY = 0.2
def get_image_url(images):
"""Extract the largest available image URL from Last.fm image array."""
for size in ["extralarge", "large", "medium", "small"]:
for img in images:
if img.get("size") == size and img.get("#text"):
return img["#text"]
return ""
def get_similar_artists(artist_name, limit=MAX_SIMILAR):
"""Fetch artists similar to the given artist from Last.fm."""
params = {
"method": "artist.getSimilar",
"artist": artist_name,
"api_key": API_KEY,
"format": "json",
"limit": limit,
}
response = requests.get(BASE_URL, params=params)
if response.status_code != 200:
print(f"Error fetching similar artists for '{artist_name}': HTTP {response.status_code}")
return []
data = response.json()
if "error" in data:
print(f"API error for '{artist_name}': {data['message']}")
return []
artists = data.get("similarartists", {}).get("artist", [])
results = []
for artist in artists:
results.append({
"name": artist.get("name", "Unknown"),
"match": float(artist.get("match", 0)),
"url": artist.get("url", ""),
"image": get_image_url(artist.get("image", [])),
})
return resultsYour crawler can now find artists similar to any artist you give it. Next up, you will fetch each artist's top albums and tracks to build a richer picture of their music.
Fetch Albums and Tracks
Your crawler can find similar artists, but a name and a match score alone won't help you decide what to listen to. You need to hear what these artists actually sound like.
In this step, you'll write two more API functions to fetch each artist's top albums and top tracks. Then you'll tie everything together in a crawler function that loops through similar artists, enriches each one with album and track data, and collects the results into a single list.
In this step, get ready to:
- Write functions to fetch top albums and top tracks from the Last.fm API.
- Build a crawler function that combines similar artists with their music data.
- Add rate-limiting delays to respect the API's usage guidelines.
Write functions to get top albums and top tracks
The Last.fm API has separate endpoints for albums and tracks. Each works the same way as artist.getSimilar. You send a request with the artist name and get back a list of results in JSON.
- In music_crawler.py, find the end of your get_similar_artists function (the final return results line).
- Add the following get_top_albums function below it:
def get_top_albums(artist_name, limit=MAX_ALBUMS):
"""Fetch top albums for an artist from Last.fm."""
params = {
"method": "artist.getTopAlbums",
"artist": artist_name,
"api_key": API_KEY,
"format": "json",
"limit": limit,
}
response = requests.get(BASE_URL, params=params)
if response.status_code != 200:
return []
data = response.json()
if "error" in data:
return []
albums = data.get("topalbums", {}).get("album", [])
results = []
for album in albums:
results.append({
"name": album.get("name", "Unknown"),
"playcount": int(album.get("playcount", 0)),
"url": album.get("url", ""),
"image": get_image_url(album.get("image", [])),
})
return resultsWhat does this code do?
This function calls the artist.getTopAlbums endpoint to retrieve an artist's most popular albums.
- The limit parameter defaults to MAX_ALBUMS (5), so you get a manageable set of albums per artist.
- Each album's play count, URL, and cover art image are extracted from the JSON response.
- If the API returns an error or a non-200 status code, the function quietly returns an empty list instead of crashing.
- Now add the get_top_tracks function directly below get_top_albums:
def get_top_tracks(artist_name, limit=MAX_TRACKS):
"""Fetch top tracks for an artist from Last.fm."""
params = {
"method": "artist.getTopTracks",
"artist": artist_name,
"api_key": API_KEY,
"format": "json",
"limit": limit,
}
response = requests.get(BASE_URL, params=params)
if response.status_code != 200:
return []
data = response.json()
if "error" in data:
return []
tracks = data.get("toptracks", {}).get("track", [])
results = []
for track in tracks:
results.append({
"name": track.get("name", "Unknown"),
"playcount": int(track.get("playcount", 0)),
"url": track.get("url", ""),
})
return resultsWhat does this code do?
This function follows the same pattern as get_top_albums, but calls artist.getTopTracks instead.
- It extracts each track's name, play count, and URL.
- Tracks don't include cover art (that belongs to albums), so there's no image field here.
- Now move the get_image_url function from its current position (above get_similar_artists) to directly below get_top_tracks. Select the entire function, cut it, and paste it after get_top_tracks.
- Save music_crawler.py.
Why move get_image_url?
Grouping the helper function with the data-fetching functions that use it keeps the file organized. In Python, function definition order doesn't matter as long as everything is defined before the script's entry point calls them.
Seeing a NameError?
If you accidentally deleted get_image_url instead of moving it, you can re-type it below get_top_tracks. Make sure there are no duplicate copies of the function in the file.
Help me fix a NameError in my music crawler script.
Combine the data into a unified structure
You have three separate functions that each talk to the API. Now you need a conductor that orchestrates them. The crawl_similar_artists function loops through the similar artists, fetches albums and tracks for each one, and bundles everything into a single list of enriched dictionaries.
- Add the following crawl_similar_artists function below get_image_url:
def crawl_similar_artists(seed_artist, max_depth=MAX_DEPTH):
"""Crawl similar artists up to max_depth levels deep."""
all_artists = []
seen_names = set()
seen_names.add(seed_artist.lower())
artists_to_process = [(seed_artist, 0)]
while artists_to_process:
current_artist, current_depth = artists_to_process.pop(0)
if current_depth >= max_depth:
continue
print(f"Searching for artists similar to '{current_artist}' (depth {current_depth + 1})...")
time.sleep(REQUEST_DELAY)
similar = get_similar_artists(current_artist)
for artist in similar:
artist_lower = artist["name"].lower()
if artist_lower in seen_names:
continue
seen_names.add(artist_lower)
print(f" Found: {artist['name']} ({artist['match']:.0%} match)")
time.sleep(REQUEST_DELAY)
albums = get_top_albums(artist["name"])
time.sleep(REQUEST_DELAY)
tracks = get_top_tracks(artist["name"])
enriched_artist = {
"name": artist["name"],
"match": artist["match"],
"url": artist["url"],
"image": artist["image"],
"albums": albums,
"tracks": tracks,
"depth": current_depth + 1,
}
all_artists.append(enriched_artist)
if current_depth + 1 < max_depth:
artists_to_process.append((artist["name"], current_depth + 1))
return all_artistsWhat does this code do?
This is the main engine of your crawler. It uses a processing queue to iterate through artists level by level.
- seen_names tracks which artists have already been processed, preventing duplicates.
- time.sleep(REQUEST_DELAY) adds a 0.2 second pause between each API call to stay within Last.fm's rate limit of roughly 5 requests per second.
- Each artist gets enriched with their albums, tracks, and a depth value indicating how many hops away they are from your seed artist.
- Artists discovered at depth 1 get added to the processing queue so their similar artists are also explored (up to MAX_DEPTH).
Why add delays between API calls?
Last.fm allows roughly 5 requests per second per IP address. Without delays, your crawler could blast through dozens of requests in seconds and get temporarily blocked. The 0.2 second sleep keeps you well within the limit.
- Save music_crawler.py.
- Verify your file has no syntax errors by running this in your terminal:
python3 -c "import music_crawler; print('All good!')"You should see All good! printed in the terminal. This confirms Python can import your file without hitting any errors.
Seeing a SyntaxError or IndentationError?
Check that each function has consistent indentation (4 spaces per level). Also confirm there are no leftover blank get_image_url definitions. The function should appear only once in the file, below get_top_tracks.
Help me debug my music_crawler.py file.
✔️ Awesome, I've got everything!
Great. Make sure you've saved music_crawler.py before moving on.
ⓧ I'd like to double check the full code
Here is the full music_crawler.py file as it should look at the end of this step:
import requests
import time
import webbrowser
import os
# --- Configuration ---
API_KEY = "your-api-key-here"
BASE_URL = "https://ws.audioscrobbler.com/2.0/"
SEED_ARTIST = "Radiohead"
MAX_SIMILAR = 5
MAX_ALBUMS = 5
MAX_TRACKS = 5
MAX_DEPTH = 2
REQUEST_DELAY = 0.2
def get_similar_artists(artist_name, limit=MAX_SIMILAR):
"""Fetch artists similar to the given artist from Last.fm."""
params = {
"method": "artist.getSimilar",
"artist": artist_name,
"api_key": API_KEY,
"format": "json",
"limit": limit,
}
response = requests.get(BASE_URL, params=params)
if response.status_code != 200:
print(f"Error fetching similar artists for '{artist_name}': HTTP {response.status_code}")
return []
data = response.json()
if "error" in data:
print(f"API error for '{artist_name}': {data['message']}")
return []
artists = data.get("similarartists", {}).get("artist", [])
results = []
for artist in artists:
results.append({
"name": artist.get("name", "Unknown"),
"match": float(artist.get("match", 0)),
"url": artist.get("url", ""),
"image": get_image_url(artist.get("image", [])),
})
return results
def get_top_albums(artist_name, limit=MAX_ALBUMS):
"""Fetch top albums for an artist from Last.fm."""
params = {
"method": "artist.getTopAlbums",
"artist": artist_name,
"api_key": API_KEY,
"format": "json",
"limit": limit,
}
response = requests.get(BASE_URL, params=params)
if response.status_code != 200:
return []
data = response.json()
if "error" in data:
return []
albums = data.get("topalbums", {}).get("album", [])
results = []
for album in albums:
results.append({
"name": album.get("name", "Unknown"),
"playcount": int(album.get("playcount", 0)),
"url": album.get("url", ""),
"image": get_image_url(album.get("image", [])),
})
return results
def get_top_tracks(artist_name, limit=MAX_TRACKS):
"""Fetch top tracks for an artist from Last.fm."""
params = {
"method": "artist.getTopTracks",
"artist": artist_name,
"api_key": API_KEY,
"format": "json",
"limit": limit,
}
response = requests.get(BASE_URL, params=params)
if response.status_code != 200:
return []
data = response.json()
if "error" in data:
return []
tracks = data.get("toptracks", {}).get("track", [])
results = []
for track in tracks:
results.append({
"name": track.get("name", "Unknown"),
"playcount": int(track.get("playcount", 0)),
"url": track.get("url", ""),
})
return results
def get_image_url(images):
"""Extract the largest available image URL from Last.fm image array."""
for size in ["extralarge", "large", "medium", "small"]:
for img in images:
if img.get("size") == size and img.get("#text"):
return img["#text"]
return ""
def crawl_similar_artists(seed_artist, max_depth=MAX_DEPTH):
"""Crawl similar artists up to max_depth levels deep."""
all_artists = []
seen_names = set()
seen_names.add(seed_artist.lower())
artists_to_process = [(seed_artist, 0)]
while artists_to_process:
current_artist, current_depth = artists_to_process.pop(0)
if current_depth >= max_depth:
continue
print(f"Searching for artists similar to '{current_artist}' (depth {current_depth + 1})...")
time.sleep(REQUEST_DELAY)
similar = get_similar_artists(current_artist)
for artist in similar:
artist_lower = artist["name"].lower()
if artist_lower in seen_names:
continue
seen_names.add(artist_lower)
print(f" Found: {artist['name']} ({artist['match']:.0%} match)")
time.sleep(REQUEST_DELAY)
albums = get_top_albums(artist["name"])
time.sleep(REQUEST_DELAY)
tracks = get_top_tracks(artist["name"])
enriched_artist = {
"name": artist["name"],
"match": artist["match"],
"url": artist["url"],
"image": artist["image"],
"albums": albums,
"tracks": tracks,
"depth": current_depth + 1,
}
all_artists.append(enriched_artist)
if current_depth + 1 < max_depth:
artists_to_process.append((artist["name"], current_depth + 1))
return all_artistsYour crawler now knows how to discover similar artists, fetch their albums and tracks, and bundle everything together. Next up, you'll turn all that raw data into a polished HTML page you can browse in your browser.
Generate the HTML Recommendations Page
Your crawler can now reach out to the Last.fm API, discover similar artists, and come back with rich data about their albums and tracks. That data is sitting in memory as a list of Python dictionaries.
Raw dictionaries are useful for code, but not for browsing music. In this step, you will transform that data into a polished HTML page with a dark-themed layout, album cover art, and play counts you can open in your browser.
In this step, get ready to:
- Build a function that converts artist data into styled HTML cards.
- Write the HTML output to a file and open it automatically in your browser.
Build the HTML generation function
The generate_html function takes the list of enriched artist dictionaries and produces a complete HTML document as a single string. It includes inline CSS so the page looks polished without any external files.
- In music_crawler.py, scroll to the bottom of the file (below the crawl_similar_artists function) and add the following generate_html function:
def generate_html(seed_artist, artists):
"""Generate an HTML recommendations page."""
artist_cards = ""
for artist in sorted(artists, key=lambda a: (-a["depth"] == 1, -a["match"])):
depth_label = "Direct Match" if artist["depth"] == 1 else "Deep Discovery"
depth_class = "direct" if artist["depth"] == 1 else "deep"
albums_html = ""
for album in artist["albums"]:
album_image = album["image"] if album["image"] else "https://lastfm.freetls.fastly.net/i/u/174s/2a96cbd8b46e442fc41c2b86b821562f.png"
albums_html += f"""
<div class="album">
<img src="{album_image}" alt="{album['name']}">
<p class="album-name"><a href="{album['url']}" target="_blank">{album['name']}</a></p>
<p class="play-count">{album['playcount']:,} plays</p>
</div>"""
tracks_html = ""
for i, track in enumerate(artist["tracks"], 1):
tracks_html += f"""
<li>
<a href="{track['url']}" target="_blank">{track['name']}</a>
<span class="play-count">{track['playcount']:,} plays</span>
</li>"""
artist_image = artist["image"] if artist["image"] else "https://lastfm.freetls.fastly.net/i/u/174s/2a96cbd8b46e442fc41c2b86b821562f.png"
artist_cards += f"""
<div class="artist-card">
<div class="artist-header">
<img src="{artist_image}" alt="{artist['name']}" class="artist-image">
<div class="artist-info">
<h2><a href="{artist['url']}" target="_blank">{artist['name']}</a></h2>
<span class="match-score">{artist['match']:.0%} similar</span>
<span class="depth-badge {depth_class}">{depth_label}</span>
</div>
</div>
<div class="content-section">
<h3>Top Albums</h3>
<div class="albums-grid">{albums_html}
</div>
</div>
<div class="content-section">
<h3>Top Tracks</h3>
<ol class="tracks-list">{tracks_html}
</ol>
</div>
</div>"""
html = f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Music Discovery: Artists like {seed_artist}</title>
<style>
* {{
margin: 0;
padding: 0;
box-sizing: border-box;
}}
body {{
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
background-color: #121212;
color: #e0e0e0;
padding: 40px 20px;
line-height: 1.6;
}}
.container {{
max-width: 900px;
margin: 0 auto;
}}
h1 {{
font-size: 2rem;
margin-bottom: 8px;
color: #ffffff;
}}
.subtitle {{
color: #a0a0a0;
margin-bottom: 32px;
font-size: 1.1rem;
}}
.artist-card {{
background: #1e1e1e;
border-radius: 12px;
padding: 24px;
margin-bottom: 24px;
border: 1px solid #2a2a2a;
}}
.artist-header {{
display: flex;
align-items: center;
gap: 16px;
margin-bottom: 20px;
}}
.artist-image {{
width: 80px;
height: 80px;
border-radius: 50%;
object-fit: cover;
}}
.artist-info h2 {{
font-size: 1.4rem;
margin-bottom: 4px;
}}
.artist-info h2 a {{
color: #ffffff;
text-decoration: none;
}}
.artist-info h2 a:hover {{
text-decoration: underline;
}}
.match-score {{
background: #1db954;
color: #000;
padding: 2px 10px;
border-radius: 12px;
font-size: 0.85rem;
font-weight: 600;
}}
.depth-badge {{
padding: 2px 10px;
border-radius: 12px;
font-size: 0.8rem;
margin-left: 8px;
}}
.depth-badge.direct {{
background: #bb86fc;
color: #000;
}}
.depth-badge.deep {{
background: #03dac6;
color: #000;
}}
.content-section {{
margin-top: 16px;
}}
.content-section h3 {{
font-size: 1rem;
color: #a0a0a0;
margin-bottom: 12px;
text-transform: uppercase;
letter-spacing: 1px;
}}
.albums-grid {{
display: flex;
gap: 16px;
overflow-x: auto;
padding-bottom: 8px;
}}
.album {{
min-width: 120px;
text-align: center;
}}
.album img {{
width: 120px;
height: 120px;
border-radius: 8px;
object-fit: cover;
margin-bottom: 8px;
}}
.album-name {{
font-size: 0.85rem;
font-weight: 500;
}}
.album-name a {{
color: #e0e0e0;
text-decoration: none;
}}
.album-name a:hover {{
text-decoration: underline;
}}
.play-count {{
font-size: 0.75rem;
color: #888;
}}
.tracks-list {{
list-style: decimal;
padding-left: 20px;
}}
.tracks-list li {{
padding: 6px 0;
display: flex;
justify-content: space-between;
align-items: center;
}}
.tracks-list li a {{
color: #e0e0e0;
text-decoration: none;
}}
.tracks-list li a:hover {{
text-decoration: underline;
}}
</style>
</head>
<body>
<div class="container">
<h1>Artists Similar to {seed_artist}</h1>
<p class="subtitle">Discovered {len(artists)} artists across {max(a['depth'] for a in artists) if artists else 0} levels of similarity</p>
{artist_cards}
</div>
</body>
</html>"""
return htmlWhat does this code do?
This function builds a complete HTML page by looping over each artist in the crawled data and generating a card for them.
- sorted(artists, key=lambda a: (-a["depth"] == 1, -a["match"])) sorts direct matches first, then by similarity score.
- Each artist card includes their name, a similarity percentage badge, a depth badge ("Direct Match" or "Deep Discovery"), their top albums with cover art, and a numbered list of top tracks.
- The fallback image URL handles cases where Last.fm returns no artwork for an artist or album.
- The inline CSS uses a dark theme (#121212 background) with card-based layout, green similarity badges, and purple/teal depth indicators.
- Save music_crawler.py.
You will not see any output yet. The next chunk adds the main function that actually runs the crawler and calls this function.
Write the HTML to a file and open it
The main function ties everything together. It runs the crawler, passes the results into generate_html, writes the output to a file, and opens it in your default browser.
- Below the generate_html function, add the main function and the entry point by pasting this code:
def main():
"""Main function to run the music discovery crawler."""
print(f"Music Discovery Crawler")
print(f"=======================")
print(f"Seed artist: {SEED_ARTIST}")
print(f"Max depth: {MAX_DEPTH}")
print()
# Run the crawler to discover similar artists
artists = crawl_similar_artists(SEED_ARTIST)
if not artists:
print("No similar artists found. Check your API key and artist name.")
return
# Generate the HTML page from crawled data
print(f"\nGenerating HTML page for {len(artists)} discovered artists...")
html = generate_html(SEED_ARTIST, artists)
# Write HTML to a file
output_file = "recommendations.html"
with open(output_file, "w", encoding="utf-8") as f:
f.write(html)
print(f"Recommendations saved to {output_file}")
# Open the page in the default browser
file_path = os.path.abspath(output_file)
webbrowser.open(f"file://{file_path}")
print(f"Opened in your default browser!")
if __name__ == "__main__":
main()What does this code do?
- crawl_similar_artists(SEED_ARTIST) runs the full depth crawl you built in the previous step, returning enriched artist data.
- generate_html(SEED_ARTIST, artists) converts that data into a styled HTML string.
- The script writes the HTML to recommendations.html in the same folder as your script, then uses Python's webbrowser module to open it automatically.
- if __name__ == "__main__": ensures main() only runs when you execute the script directly, not when it is imported as a module.
- Save music_crawler.py.
- Run the script by entering this command in your terminal:
python3 music_crawler.pyYou should see terminal output listing discovered artists with their match percentages, followed by a message saying the HTML page was saved. Your default browser opens automatically with the recommendations page.
What should I see?
Your browser displays a dark-themed page titled "Artists Similar to Radiohead" with cards for each discovered artist. Each card shows the artist name, a green similarity percentage badge, a purple "Direct Match" or teal "Deep Discovery" label, a scrollable row of album covers with play counts, and a numbered list of top tracks.
Page not opening or showing errors?
- If you see "No similar artists found", double-check that your API_KEY value at the top of the file is your actual Last.fm API key, not the placeholder.
- If the browser does not open automatically, look for recommendations.html in your music-crawler folder and open it manually.
- If album images are missing, that is expected for some lesser-known artists. The page shows a grey placeholder instead.
Why isn't my music crawler producing an HTML page?
✔️ Awesome, I've got everything!
Great. Double check you have saved music_crawler.py and that your browser opened the recommendations page.
ⓧ I'd like to double check the full code
import requests
import time
import webbrowser
import os
# --- Configuration ---
API_KEY = "your-api-key-here"
BASE_URL = "https://ws.audioscrobbler.com/2.0/"
SEED_ARTIST = "Radiohead"
MAX_SIMILAR = 5
MAX_ALBUMS = 5
MAX_TRACKS = 5
MAX_DEPTH = 2
REQUEST_DELAY = 0.2
def get_similar_artists(artist_name, limit=MAX_SIMILAR):
"""Fetch artists similar to the given artist from Last.fm."""
params = {
"method": "artist.getSimilar",
"artist": artist_name,
"api_key": API_KEY,
"format": "json",
"limit": limit,
}
response = requests.get(BASE_URL, params=params)
if response.status_code != 200:
print(f"Error fetching similar artists for '{artist_name}': HTTP {response.status_code}")
return []
data = response.json()
if "error" in data:
print(f"API error for '{artist_name}': {data['message']}")
return []
artists = data.get("similarartists", {}).get("artist", [])
results = []
for artist in artists:
results.append({
"name": artist.get("name", "Unknown"),
"match": float(artist.get("match", 0)),
"url": artist.get("url", ""),
"image": get_image_url(artist.get("image", [])),
})
return results
def get_top_albums(artist_name, limit=MAX_ALBUMS):
"""Fetch top albums for an artist from Last.fm."""
params = {
"method": "artist.getTopAlbums",
"artist": artist_name,
"api_key": API_KEY,
"format": "json",
"limit": limit,
}
response = requests.get(BASE_URL, params=params)
if response.status_code != 200:
return []
data = response.json()
if "error" in data:
return []
albums = data.get("topalbums", {}).get("album", [])
results = []
for album in albums:
results.append({
"name": album.get("name", "Unknown"),
"playcount": int(album.get("playcount", 0)),
"url": album.get("url", ""),
"image": get_image_url(album.get("image", [])),
})
return results
def get_top_tracks(artist_name, limit=MAX_TRACKS):
"""Fetch top tracks for an artist from Last.fm."""
params = {
"method": "artist.getTopTracks",
"artist": artist_name,
"api_key": API_KEY,
"format": "json",
"limit": limit,
}
response = requests.get(BASE_URL, params=params)
if response.status_code != 200:
return []
data = response.json()
if "error" in data:
return []
tracks = data.get("toptracks", {}).get("track", [])
results = []
for track in tracks:
results.append({
"name": track.get("name", "Unknown"),
"playcount": int(track.get("playcount", 0)),
"url": track.get("url", ""),
})
return results
def get_image_url(images):
"""Extract the largest available image URL from Last.fm image array."""
for size in ["extralarge", "large", "medium", "small"]:
for img in images:
if img.get("size") == size and img.get("#text"):
return img["#text"]
return ""
def crawl_similar_artists(seed_artist, max_depth=MAX_DEPTH):
"""Crawl similar artists up to max_depth levels deep."""
all_artists = []
seen_names = set()
seen_names.add(seed_artist.lower())
artists_to_process = [(seed_artist, 0)]
while artists_to_process:
current_artist, current_depth = artists_to_process.pop(0)
if current_depth >= max_depth:
continue
print(f"Searching for artists similar to '{current_artist}' (depth {current_depth + 1})...")
time.sleep(REQUEST_DELAY)
similar = get_similar_artists(current_artist)
for artist in similar:
artist_lower = artist["name"].lower()
if artist_lower in seen_names:
continue
seen_names.add(artist_lower)
print(f" Found: {artist['name']} ({artist['match']:.0%} match)")
time.sleep(REQUEST_DELAY)
albums = get_top_albums(artist["name"])
time.sleep(REQUEST_DELAY)
tracks = get_top_tracks(artist["name"])
enriched_artist = {
"name": artist["name"],
"match": artist["match"],
"url": artist["url"],
"image": artist["image"],
"albums": albums,
"tracks": tracks,
"depth": current_depth + 1,
}
all_artists.append(enriched_artist)
if current_depth + 1 < max_depth:
artists_to_process.append((artist["name"], current_depth + 1))
return all_artists
def generate_html(seed_artist, artists):
"""Generate an HTML recommendations page."""
artist_cards = ""
for artist in sorted(artists, key=lambda a: (-a["depth"] == 1, -a["match"])):
depth_label = "Direct Match" if artist["depth"] == 1 else "Deep Discovery"
depth_class = "direct" if artist["depth"] == 1 else "deep"
albums_html = ""
for album in artist["albums"]:
album_image = album["image"] if album["image"] else "https://lastfm.freetls.fastly.net/i/u/174s/2a96cbd8b46e442fc41c2b86b821562f.png"
albums_html += f"""
<div class="album">
<img src="{album_image}" alt="{album['name']}">
<p class="album-name"><a href="{album['url']}" target="_blank">{album['name']}</a></p>
<p class="play-count">{album['playcount']:,} plays</p>
</div>"""
tracks_html = ""
for i, track in enumerate(artist["tracks"], 1):
tracks_html += f"""
<li>
<a href="{track['url']}" target="_blank">{track['name']}</a>
<span class="play-count">{track['playcount']:,} plays</span>
</li>"""
artist_image = artist["image"] if artist["image"] else "https://lastfm.freetls.fastly.net/i/u/174s/2a96cbd8b46e442fc41c2b86b821562f.png"
artist_cards += f"""
<div class="artist-card">
<div class="artist-header">
<img src="{artist_image}" alt="{artist['name']}" class="artist-image">
<div class="artist-info">
<h2><a href="{artist['url']}" target="_blank">{artist['name']}</a></h2>
<span class="match-score">{artist['match']:.0%} similar</span>
<span class="depth-badge {depth_class}">{depth_label}</span>
</div>
</div>
<div class="content-section">
<h3>Top Albums</h3>
<div class="albums-grid">{albums_html}
</div>
</div>
<div class="content-section">
<h3>Top Tracks</h3>
<ol class="tracks-list">{tracks_html}
</ol>
</div>
</div>"""
html = f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Music Discovery: Artists like {seed_artist}</title>
<style>
* {{
margin: 0;
padding: 0;
box-sizing: border-box;
}}
body {{
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
background-color: #121212;
color: #e0e0e0;
padding: 40px 20px;
line-height: 1.6;
}}
.container {{
max-width: 900px;
margin: 0 auto;
}}
h1 {{
font-size: 2rem;
margin-bottom: 8px;
color: #ffffff;
}}
.subtitle {{
color: #a0a0a0;
margin-bottom: 32px;
font-size: 1.1rem;
}}
.artist-card {{
background: #1e1e1e;
border-radius: 12px;
padding: 24px;
margin-bottom: 24px;
border: 1px solid #2a2a2a;
}}
.artist-header {{
display: flex;
align-items: center;
gap: 16px;
margin-bottom: 20px;
}}
.artist-image {{
width: 80px;
height: 80px;
border-radius: 50%;
object-fit: cover;
}}
.artist-info h2 {{
font-size: 1.4rem;
margin-bottom: 4px;
}}
.artist-info h2 a {{
color: #ffffff;
text-decoration: none;
}}
.artist-info h2 a:hover {{
text-decoration: underline;
}}
.match-score {{
background: #1db954;
color: #000;
padding: 2px 10px;
border-radius: 12px;
font-size: 0.85rem;
font-weight: 600;
}}
.depth-badge {{
padding: 2px 10px;
border-radius: 12px;
font-size: 0.8rem;
margin-left: 8px;
}}
.depth-badge.direct {{
background: #bb86fc;
color: #000;
}}
.depth-badge.deep {{
background: #03dac6;
color: #000;
}}
.content-section {{
margin-top: 16px;
}}
.content-section h3 {{
font-size: 1rem;
color: #a0a0a0;
margin-bottom: 12px;
text-transform: uppercase;
letter-spacing: 1px;
}}
.albums-grid {{
display: flex;
gap: 16px;
overflow-x: auto;
padding-bottom: 8px;
}}
.album {{
min-width: 120px;
text-align: center;
}}
.album img {{
width: 120px;
height: 120px;
border-radius: 8px;
object-fit: cover;
margin-bottom: 8px;
}}
.album-name {{
font-size: 0.85rem;
font-weight: 500;
}}
.album-name a {{
color: #e0e0e0;
text-decoration: none;
}}
.album-name a:hover {{
text-decoration: underline;
}}
.play-count {{
font-size: 0.75rem;
color: #888;
}}
.tracks-list {{
list-style: decimal;
padding-left: 20px;
}}
.tracks-list li {{
padding: 6px 0;
display: flex;
justify-content: space-between;
align-items: center;
}}
.tracks-list li a {{
color: #e0e0e0;
text-decoration: none;
}}
.tracks-list li a:hover {{
text-decoration: underline;
}}
</style>
</head>
<body>
<div class="container">
<h1>Artists Similar to {seed_artist}</h1>
<p class="subtitle">Discovered {len(artists)} artists across {max(a['depth'] for a in artists) if artists else 0} levels of similarity</p>
{artist_cards}
</div>
</body>
</html>"""
return html
def main():
"""Main function to run the music discovery crawler."""
print(f"Music Discovery Crawler")
print(f"=======================")
print(f"Seed artist: {SEED_ARTIST}")
print(f"Max depth: {MAX_DEPTH}")
print()
artists = crawl_similar_artists(SEED_ARTIST)
if not artists:
print("No similar artists found. Check your API key and artist name.")
return
print(f"\nGenerating HTML page for {len(artists)} discovered artists...")
html = generate_html(SEED_ARTIST, artists)
output_file = "recommendations.html"
with open(output_file, "w", encoding="utf-8") as f:
f.write(html)
print(f"Recommendations saved to {output_file}")
file_path = os.path.abspath(output_file)
webbrowser.open(f"file://{file_path}")
print(f"Opened in your default browser!")
if __name__ == "__main__":
main()Your crawler now produces a complete music discovery page you can browse visually. Next up, you will add depth crawling so the tool explores second-degree connections and surfaces even more unexpected musical finds.
Add Depth Crawling
Your crawler currently finds artists directly similar to your seed artist. These are great recommendations, but they stay close to what you already know.
The real magic of music discovery happens when you go one level deeper. By finding artists similar to your similar artists, you surface musicians you would never have found on your own. In this step, you will trace through how the crawler achieves this multi-level exploration and verify that the full depth-2 results appear in your browser.
In this step, get ready to:
- Understand how the crawler explores 2 levels of artist similarity.
- Run the full depth-2 crawl and verify the expanded HTML page.
Extend the crawler to go 2 levels deep
Your crawl_similar_artists function already contains the logic for multi-level crawling. It uses a technique called breadth-first search (BFS) to explore artist connections layer by layer.
- In music_crawler.py, scroll to the crawl_similar_artists function.
- Find the section near the bottom of the function that reads:
if current_depth + 1 < max_depth:
artists_to_process.append((artist["name"], current_depth + 1))How does depth crawling work?
The crawler uses a queue (artists_to_process) that stores pairs of (artist name, current depth). It starts with your seed artist at depth 0.
When it discovers a new similar artist at depth 1, it adds that artist back into the queue. On the next iteration, it searches for artists similar to that depth-1 artist, producing depth-2 results. The seen_names set prevents the same artist from appearing twice, even if multiple paths lead to them.
- Now find the MAX_DEPTH configuration variable near the top of the file:
MAX_DEPTH = 2What does MAX_DEPTH control?
With MAX_DEPTH = 2, the crawler explores two layers. Depth 1 finds artists directly similar to your seed. Depth 2 finds artists similar to those depth-1 artists.
Each discovered artist is labeled with its depth value (current_depth + 1), so you can tell which are direct connections and which are second-degree discoveries.
Run the full depth-2 crawl and verify the results
Your HTML generation function already displays depth badges on each artist card. Direct matches get a purple "Direct Match" badge. Second-degree discoveries get a teal "Deep Discovery" badge.
- In music_crawler.py, scroll to the generate_html function and find these lines:
for artist in sorted(artists, key=lambda a: (-a["depth"] == 1, -a["match"])):
depth_label = "Direct Match" if artist["depth"] == 1 else "Deep Discovery"
depth_class = "direct" if artist["depth"] == 1 else "deep"How are results sorted and labeled?
The sorted() call organizes artists so that depth-1 (direct) matches appear first, then depth-2 (deep) discoveries. Within each group, artists are ordered by their similarity score from highest to lowest.
Each artist card gets a depth_label (the text shown on the badge) and a depth_class (the CSS class that controls the badge colour).
- Run the full crawler by executing this in your terminal:
python3 music_crawler.pyWhat should I see in the terminal?
The crawler prints its progress as it works through both levels. You will see lines like Searching for artists similar to 'Radiohead' (depth 1)... followed by the depth-1 results.
Then it loops back through those depth-1 artists and searches for their similar artists at depth 2. The full run takes around 15-30 seconds because of the 0.2 second delay between API calls.
- Wait for the script to finish. It will automatically open recommendations.html in your default browser.
- Verify that the page shows both purple "Direct Match" badges and teal "Deep Discovery" badges on different artist cards.
- Check the subtitle at the top of the page. It should say something like "Discovered X artists across 2 levels of similarity".
Only seeing depth-1 results?
Make sure MAX_DEPTH = 2 at the top of your file. If it is set to 1, the crawler only explores one level.
If the script finishes very quickly (under 5 seconds), your API key may be invalid or the seed artist name may not exist on Last.fm. Check the terminal for error messages.
Help me debug my music crawler depth results.
✔️ Awesome, I've got everything!
Great. Double check you have saved music_crawler.py and confirmed both Direct Match and Deep Discovery badges in your browser.
ⓧ I'd like to double check the full code
Here is the complete music_crawler.py file for comparison.
import requests
import time
import webbrowser
import os
# --- Configuration ---
API_KEY = "your-api-key-here"
BASE_URL = "https://ws.audioscrobbler.com/2.0/"
SEED_ARTIST = "Radiohead"
MAX_SIMILAR = 5
MAX_ALBUMS = 5
MAX_TRACKS = 5
MAX_DEPTH = 2
REQUEST_DELAY = 0.2
def get_similar_artists(artist_name, limit=MAX_SIMILAR):
"""Fetch artists similar to the given artist from Last.fm."""
params = {
"method": "artist.getSimilar",
"artist": artist_name,
"api_key": API_KEY,
"format": "json",
"limit": limit,
}
response = requests.get(BASE_URL, params=params)
if response.status_code != 200:
print(f"Error fetching similar artists for '{artist_name}': HTTP {response.status_code}")
return []
data = response.json()
if "error" in data:
print(f"API error for '{artist_name}': {data['message']}")
return []
artists = data.get("similarartists", {}).get("artist", [])
results = []
for artist in artists:
results.append({
"name": artist.get("name", "Unknown"),
"match": float(artist.get("match", 0)),
"url": artist.get("url", ""),
"image": get_image_url(artist.get("image", [])),
})
return results
def get_top_albums(artist_name, limit=MAX_ALBUMS):
"""Fetch top albums for an artist from Last.fm."""
params = {
"method": "artist.getTopAlbums",
"artist": artist_name,
"api_key": API_KEY,
"format": "json",
"limit": limit,
}
response = requests.get(BASE_URL, params=params)
if response.status_code != 200:
return []
data = response.json()
if "error" in data:
return []
albums = data.get("topalbums", {}).get("album", [])
results = []
for album in albums:
results.append({
"name": album.get("name", "Unknown"),
"playcount": int(album.get("playcount", 0)),
"url": album.get("url", ""),
"image": get_image_url(album.get("image", [])),
})
return results
def get_top_tracks(artist_name, limit=MAX_TRACKS):
"""Fetch top tracks for an artist from Last.fm."""
params = {
"method": "artist.getTopTracks",
"artist": artist_name,
"api_key": API_KEY,
"format": "json",
"limit": limit,
}
response = requests.get(BASE_URL, params=params)
if response.status_code != 200:
return []
data = response.json()
if "error" in data:
return []
tracks = data.get("toptracks", {}).get("track", [])
results = []
for track in tracks:
results.append({
"name": track.get("name", "Unknown"),
"playcount": int(track.get("playcount", 0)),
"url": track.get("url", ""),
})
return results
def get_image_url(images):
"""Extract the largest available image URL from Last.fm image array."""
for size in ["extralarge", "large", "medium", "small"]:
for img in images:
if img.get("size") == size and img.get("#text"):
return img["#text"]
return ""
def crawl_similar_artists(seed_artist, max_depth=MAX_DEPTH):
"""Crawl similar artists up to max_depth levels deep."""
all_artists = []
seen_names = set()
seen_names.add(seed_artist.lower())
artists_to_process = [(seed_artist, 0)]
while artists_to_process:
current_artist, current_depth = artists_to_process.pop(0)
if current_depth >= max_depth:
continue
print(f"Searching for artists similar to '{current_artist}' (depth {current_depth + 1})...")
time.sleep(REQUEST_DELAY)
similar = get_similar_artists(current_artist)
for artist in similar:
artist_lower = artist["name"].lower()
if artist_lower in seen_names:
continue
seen_names.add(artist_lower)
print(f" Found: {artist['name']} ({artist['match']:.0%} match)")
time.sleep(REQUEST_DELAY)
albums = get_top_albums(artist["name"])
time.sleep(REQUEST_DELAY)
tracks = get_top_tracks(artist["name"])
enriched_artist = {
"name": artist["name"],
"match": artist["match"],
"url": artist["url"],
"image": artist["image"],
"albums": albums,
"tracks": tracks,
"depth": current_depth + 1,
}
all_artists.append(enriched_artist)
if current_depth + 1 < max_depth:
artists_to_process.append((artist["name"], current_depth + 1))
return all_artists
def generate_html(seed_artist, artists):
"""Generate an HTML recommendations page."""
artist_cards = ""
for artist in sorted(artists, key=lambda a: (-a["depth"] == 1, -a["match"])):
depth_label = "Direct Match" if artist["depth"] == 1 else "Deep Discovery"
depth_class = "direct" if artist["depth"] == 1 else "deep"
albums_html = ""
for album in artist["albums"]:
album_image = album["image"] if album["image"] else "https://lastfm.freetls.fastly.net/i/u/174s/2a96cbd8b46e442fc41c2b86b821562f.png"
albums_html += f"""
<div class="album">
<img src="{album_image}" alt="{album['name']}">
<p class="album-name"><a href="{album['url']}" target="_blank">{album['name']}</a></p>
<p class="play-count">{album['playcount']:,} plays</p>
</div>"""
tracks_html = ""
for i, track in enumerate(artist["tracks"], 1):
tracks_html += f"""
<li>
<a href="{track['url']}" target="_blank">{track['name']}</a>
<span class="play-count">{track['playcount']:,} plays</span>
</li>"""
artist_image = artist["image"] if artist["image"] else "https://lastfm.freetls.fastly.net/i/u/174s/2a96cbd8b46e442fc41c2b86b821562f.png"
artist_cards += f"""
<div class="artist-card">
<div class="artist-header">
<img src="{artist_image}" alt="{artist['name']}" class="artist-image">
<div class="artist-info">
<h2><a href="{artist['url']}" target="_blank">{artist['name']}</a></h2>
<span class="match-score">{artist['match']:.0%} similar</span>
<span class="depth-badge {depth_class}">{depth_label}</span>
</div>
</div>
<div class="content-section">
<h3>Top Albums</h3>
<div class="albums-grid">{albums_html}
</div>
</div>
<div class="content-section">
<h3>Top Tracks</h3>
<ol class="tracks-list">{tracks_html}
</ol>
</div>
</div>"""
html = f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Music Discovery: Artists like {seed_artist}</title>
<style>
* {{
margin: 0;
padding: 0;
box-sizing: border-box;
}}
body {{
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif;
background-color: #121212;
color: #e0e0e0;
padding: 40px 20px;
line-height: 1.6;
}}
.container {{
max-width: 900px;
margin: 0 auto;
}}
h1 {{
font-size: 2rem;
margin-bottom: 8px;
color: #ffffff;
}}
.subtitle {{
color: #a0a0a0;
margin-bottom: 32px;
font-size: 1.1rem;
}}
.artist-card {{
background: #1e1e1e;
border-radius: 12px;
padding: 24px;
margin-bottom: 24px;
border: 1px solid #2a2a2a;
}}
.artist-header {{
display: flex;
align-items: center;
gap: 16px;
margin-bottom: 20px;
}}
.artist-image {{
width: 80px;
height: 80px;
border-radius: 50%;
object-fit: cover;
}}
.artist-info h2 {{
font-size: 1.4rem;
margin-bottom: 4px;
}}
.artist-info h2 a {{
color: #ffffff;
text-decoration: none;
}}
.artist-info h2 a:hover {{
text-decoration: underline;
}}
.match-score {{
background: #1db954;
color: #000;
padding: 2px 10px;
border-radius: 12px;
font-size: 0.85rem;
font-weight: 600;
}}
.depth-badge {{
padding: 2px 10px;
border-radius: 12px;
font-size: 0.8rem;
margin-left: 8px;
}}
.depth-badge.direct {{
background: #bb86fc;
color: #000;
}}
.depth-badge.deep {{
background: #03dac6;
color: #000;
}}
.content-section {{
margin-top: 16px;
}}
.content-section h3 {{
font-size: 1rem;
color: #a0a0a0;
margin-bottom: 12px;
text-transform: uppercase;
letter-spacing: 1px;
}}
.albums-grid {{
display: flex;
gap: 16px;
overflow-x: auto;
padding-bottom: 8px;
}}
.album {{
min-width: 120px;
text-align: center;
}}
.album img {{
width: 120px;
height: 120px;
border-radius: 8px;
object-fit: cover;
margin-bottom: 8px;
}}
.album-name {{
font-size: 0.85rem;
font-weight: 500;
}}
.album-name a {{
color: #e0e0e0;
text-decoration: none;
}}
.album-name a:hover {{
text-decoration: underline;
}}
.play-count {{
font-size: 0.75rem;
color: #888;
}}
.tracks-list {{
list-style: decimal;
padding-left: 20px;
}}
.tracks-list li {{
padding: 6px 0;
display: flex;
justify-content: space-between;
align-items: center;
}}
.tracks-list li a {{
color: #e0e0e0;
text-decoration: none;
}}
.tracks-list li a:hover {{
text-decoration: underline;
}}
</style>
</head>
<body>
<div class="container">
<h1>Artists Similar to {seed_artist}</h1>
<p class="subtitle">Discovered {len(artists)} artists across {max(a['depth'] for a in artists) if artists else 0} levels of similarity</p>
{artist_cards}
</div>
</body>
</html>"""
return html
def main():
"""Main function to run the music discovery crawler."""
print(f"Music Discovery Crawler")
print(f"=======================")
print(f"Seed artist: {SEED_ARTIST}")
print(f"Max depth: {MAX_DEPTH}")
print()
artists = crawl_similar_artists(SEED_ARTIST)
if not artists:
print("No similar artists found. Check your API key and artist name.")
return
print(f"\nGenerating HTML page for {len(artists)} discovered artists...")
html = generate_html(SEED_ARTIST, artists)
output_file = "recommendations.html"
with open(output_file, "w", encoding="utf-8") as f:
f.write(html)
print(f"Recommendations saved to {output_file}")
file_path = os.path.abspath(output_file)
webbrowser.open(f"file://{file_path}")
print(f"Opened in your default browser!")
if __name__ == "__main__":
main()Your music discovery engine is now complete. It explores two levels of musical connections, surfacing artists you would never have stumbled on through a single similarity search. Try changing SEED_ARTIST to one of your favourites and re-running the script to see what hidden gems emerge.
Secret mission
Add Genre Tag Filtering
Your crawler finds similar artists, but what if you only want electronic recommendations? Or ambient ones? Add genre tag filtering so you can narrow your results to exactly the sound you are looking for.
Clean Up Your Resources
Clean Up Your Resources
This project runs entirely locally with a free API, so there are no ongoing costs. Decide whether to keep your files for future music discovery or remove them from your machine.
Resources you used:
- Local folder: music-crawler on your Desktop (contains music_crawler.py and recommendations.html).
Keep everything running
No action needed. Choose this if you want to keep discovering new music.
- Your music-crawler folder stays on your Desktop, ready to run anytime with a different SEED_ARTIST.
- Your Last.fm API key remains active indefinitely for personal use at no cost.
Pause - I'll come back to this later
There are no running processes or paid services to pause. Your files and API key stay ready for you whenever you return.
- No action needed. Everything is already paused by default since this is a local script.
Delete - I don't want to use this again
Remove your project folder to clean up all local files.
- Delete the project folder by running this command in your terminal:
rm -rf ~/Desktop/music-crawler- On Windows, run this instead:
rmdir /s /q %USERPROFILE%\Desktop\music-crawler- Confirm the folder is gone by running:
ls ~/DesktopYou should no longer see music-crawler in the output.
Nice Work!
Nice Work!
You just built a fully functional music discovery engine powered by Python and the Last.fm API. From a single artist name, your crawler explores the musical landscape, fetches rich data, and presents it all in a polished HTML page you can browse anytime.
Here's what you accomplished:
- Built a Python crawler that integrates with three Last.fm REST API endpoints (artist.getSimilar, artist.getTopAlbums, artist.getTopTracks) to discover similar musicians and fetch their music data.
- Implemented multi-level depth crawling with duplicate detection to surface second-degree musical connections that go beyond obvious recommendations.
- Generated a styled HTML recommendations page with album cover art, track listings, play counts, similarity percentages, and depth indicators.
- Added genre tag filtering using the artist.getTopTags endpoint to narrow recommendations by specific sounds (Secret Mission).
Ready to quiz yourself?