Google Shopping Feed Health Analyzer
Audit product feed quality with Python and visualize fix priorities.
Introduction
30 Second Summary
Every product listed on Google Shopping needs to pass strict data quality rules before shoppers can see it. If a product feed has missing identifiers, invalid prices, or poor titles, those products get disapproved and become invisible.
In this project, you will build a Python notebook that automatically audits a product feed against Google Merchant Center rules, visualizes feed health, and generates prioritized fix recommendations that account for multi-error overlap.
What You'll Build
A Google Colab notebook that generates a synthetic product feed dataset, validates it against Google Merchant Center's product data specification, and produces a visual dashboard with approval rates, error breakdowns, and prioritized recommendations.
By the end of this project, you'll have:
- A synthetic product feed with 10,000 products across multiple categories, correlated quality issues, and seller tier patterns.
- A validation engine that checks the top 5 disapproval causes: missing identifiers, price issues, title violations, missing required attributes, and image problems.
- A feed health dashboard with approval rates, error co-occurrence heatmaps, category breakdowns, and a prioritized fix list based on unique impact analysis.
- Secret Mission: a category-aware title optimization scorer that grades every product title and suggests specific improvements.
Are there any prerequisites?
You just need basic coding familiarity in any language and the ability to read Python syntax. You do not need to be a Python expert. The project walks you through every code block.
Set Up Your Notebook and Generate Feed Data
Before you can analyze product feed quality, you need two things: a place to run your code and data to analyze.
In this step, you'll create a Google Colab notebook, import your analysis libraries, and generate a synthetic product feed with 10,000 products containing realistic quality issues.
In this step, get ready to:
- Create a Colab notebook and import your analysis libraries.
- Generate a synthetic product feed with correlated data quality issues.
- Verify the dataset structure and confirm error patterns by seller tier.
Create a new Colab notebook and import libraries
Google Colab is a free, cloud-hosted notebook environment that runs Python directly in your browser. All the libraries you need for this project are already pre-installed.
- Open Google Colab by navigating to colab.research.google.com in your browser.
- Sign in with your Google account if prompted.
- Click New notebook on the welcome screen.
- Click the notebook name at the top-left (it defaults to something like Untitled0.ipynb) and rename it to feed_health_analyzer.ipynb.
You should now see a blank notebook with one empty code cell.
- Paste the following code into the first code cell to import your libraries and set the visual style:
# Cell 1: Import Libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import re
from itertools import combinations
# Set visual style
sns.set_style("whitegrid")
plt.rcParams['figure.figsize'] = (12, 6)What does this code do?
- pandas is your main data manipulation library. It provides the DataFrame structure you'll use to hold all 10,000 products.
- numpy generates random numbers for building synthetic data.
- matplotlib and seaborn produce the charts and heatmaps for your feed health dashboard later.
- re handles regular expressions for validating product titles against Google's rules.
- combinations from itertools helps identify which error types co-occur across products.
- Run the cell by clicking the play button on the left side of the cell, or by pressing Shift+Enter.
The cell should complete with no output and no errors. A green checkmark appears next to the cell when it finishes.
Seeing an import error?
All of these libraries come pre-installed in Google Colab. If you see an error, double-check that you typed the import statements exactly as shown. The most common mistake is a typo in a library name.
help me fix an import error in my Google Colab notebook
Generate a synthetic product feed dataset with correlated errors
Real product feeds submitted to Google Merchant Center contain thousands of products with data quality issues that cause disapprovals. You'll generate a realistic dataset that mimics these patterns.
The generator creates products across 10 categories with a seller tier system (gold, silver, bronze) that controls how likely each product is to have quality issues. Bronze sellers produce more errors than gold sellers, just like in real marketplace data.
- Click the + Code button below your first cell to add a new code cell.
- Paste the following code into the new cell to define and run the data generator:
# Cell 2: Generate Synthetic Product Feed Data
def generate_product_feed(n_products=10000, seed=42):
"""Generate a synthetic Google Shopping product feed with realistic correlated quality issues."""
np.random.seed(seed)
categories = [
"Electronics > Smartphones",
"Electronics > Laptops",
"Apparel > Men's Clothing",
"Apparel > Women's Clothing",
"Home & Garden > Furniture",
"Home & Garden > Kitchen",
"Sports > Fitness Equipment",
"Beauty > Skincare",
"Toys > Board Games",
"Food > Snacks"
]
brands = [
"TechCorp", "StyleMax", "HomeBase", "FitGear", "GlowUp",
"SmartLife", "UrbanWear", "KitchenPro", "PlayWorld", "NatureBite",
"EliteGadget", "ComfortZone", "ActiveEdge", "PureSkin", "FunFactory"
]
conditions = ["new", "refurbished", "used"]
availability_options = ["in_stock", "out_of_stock", "preorder"]
seller_tiers = ["gold", "silver", "bronze"]
ids = [f"PROD-{i:06d}" for i in range(n_products)]
chosen_categories = np.random.choice(categories, n_products)
chosen_brands = np.random.choice(brands, n_products)
prices = np.round(np.random.lognormal(mean=3.5, sigma=1.0, size=n_products), 2)
seller_tier = np.random.choice(seller_tiers, n_products, p=[0.3, 0.45, 0.25])
tier_multipliers = {"gold": 0.3, "silver": 1.0, "bronze": 2.5}
base_titles = []
for i in range(n_products):
brand = chosen_brands[i]
cat = chosen_categories[i].split(" > ")[1]
adjectives = ["Premium", "Classic", "Pro", "Ultra", "Essential", "Deluxe"]
adj = np.random.choice(adjectives)
title = f"{brand} {adj} {cat} - Model {np.random.randint(100, 999)}"
base_titles.append(title)
gtins = []
for i in range(n_products):
base_prob = 0.15
prob = min(base_prob * tier_multipliers[seller_tier[i]], 0.95)
roll = np.random.random()
if roll < prob:
gtins.append(None)
elif roll < prob + 0.03:
gtins.append(str(np.random.randint(100, 999)))
else:
gtins.append(str(np.random.randint(1000000000000, 9999999999999)))
df = pd.DataFrame({
"id": ids,
"title": base_titles,
"description": [
f"High-quality {chosen_categories[i].split(' > ')[1].lower()} from {chosen_brands[i]}. "
f"Perfect for everyday use." if np.random.random() > 0.06 * tier_multipliers[seller_tier[i]]
else None
for i in range(n_products)
],
"link": [f"https://example.com/products/{ids[i].lower()}" for i in range(n_products)],
"image_link": [
f"https://example.com/images/{ids[i].lower()}.jpg"
if np.random.random() > 0.05 * tier_multipliers[seller_tier[i]]
else None
for i in range(n_products)
],
"availability": np.random.choice(availability_options, n_products, p=[0.75, 0.20, 0.05]),
"price": prices,
"brand": [
chosen_brands[i] if np.random.random() > 0.08 * tier_multipliers[seller_tier[i]] else None
for i in range(n_products)
],
"gtin": gtins,
"google_product_category": chosen_categories,
"condition": np.random.choice(conditions, n_products, p=[0.85, 0.10, 0.05]),
"seller_tier": seller_tier
})
for i in range(n_products):
if np.random.random() < 0.03 * tier_multipliers[seller_tier[i]]:
df.at[i, "price"] = np.random.choice([0, -1, -5.99])
for i in range(n_products):
prob = 0.10 * tier_multipliers[seller_tier[i]]
if np.random.random() < prob:
issue_type = np.random.choice(["short", "caps", "promo"])
if issue_type == "short":
df.at[i, "title"] = "Item"
elif issue_type == "caps":
df.at[i, "title"] = df.at[i, "title"].upper()
else:
df.at[i, "title"] = "FREE SHIPPING! " + df.at[i, "title"] + " BEST DEAL!!!"
return df
# Generate the dataset
df = generate_product_feed()
print(f"Dataset shape: {df.shape}")
print(f"\nFirst 5 rows:")
df.head()What does this code do?
This function builds a realistic 10,000-product feed with intentional quality issues modeled on common Google Merchant Center disapproval patterns:
- Each product gets a unique ID, title, description, price, and other attributes matching the Merchant Center product data specification.
- A seller_tier column (gold, silver, bronze) controls error probability. Bronze sellers are 2.5x more likely to have issues than silver sellers, while gold sellers are only 0.3x as likely.
- Missing GTINs, empty descriptions, invalid prices (zero or negative), title violations (too short, ALL CAPS, promotional text), and missing images are injected at realistic rates.
- Errors are correlated. A product from a bronze seller that is missing a GTIN is also more likely to be missing a brand or have a bad title.
- Run this cell by pressing Shift+Enter.
You should see output showing Dataset shape: (10000, 12) followed by a table displaying the first 5 rows of your product feed.
Seeing a NameError for numpy or pandas?
This means your Cell 1 imports did not run successfully. Go back to Cell 1 and run it again. You must run cells in order because each cell depends on the libraries and variables from previous cells.
help me fix a NameError in my Colab notebook
Verify the dataset and explore initial quality signals
Before building a validation engine, you need to confirm the data has the expected structure and that the error correlations actually exist.
- Add a new code cell by clicking + Code.
- Paste the following code to inspect the dataset structure and seller tier quality patterns:
# Cell 3: Verify Dataset and Explore Seller Tier Patterns
print(f"Dataset shape: {df.shape}")
print(f"\nData types:")
print(df.dtypes)
print(f"\nNull counts:")
print(df.isnull().sum())
print(f"\n=== QUALITY BY SELLER TIER ===\n")
for tier in ["gold", "silver", "bronze"]:
tier_df = df[df["seller_tier"] == tier]
null_rate = tier_df.isnull().sum().sum() / (len(tier_df) * len(tier_df.columns)) * 100
print(f" {tier.UPPER():8s} | {len(tier_df):,} products | Null rate: {null_rate:.1f}%")What does this code do?
- df.shape confirms you have exactly 10,000 rows and 12 columns.
- df.isnull().sum() shows how many missing values exist in each column. You should see non-zero counts for gtin, brand, description, and image_link.
- The seller tier cross-tabulation proves that bronze sellers have a significantly higher null rate than gold sellers. This confirms the error correlations are working.
- Run the cell by pressing Shift+Enter.
You should see bronze sellers with a noticeably higher null rate than gold sellers. This pattern is the foundation of your analysis in later steps.
Not seeing any null values?
Make sure you ran Cell 2 before Cell 3. The variable df is created in Cell 2 and used in Cell 3. If you edited Cell 2 after running it, you need to re-run it to regenerate df.
help me debug missing null values in my feed dataset
✔️ Awesome, I've got everything!
Great. Make sure all three cells have run successfully and you can see the seller tier quality breakdown in the output.
ⓧ I'd like to double check the full code
Here is the complete code for all three cells combined. Compare it against what you have in your notebook:
# Cell 1: Import Libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import re
from itertools import combinations
# Set visual style
sns.set_style("whitegrid")
plt.rcParams['figure.figsize'] = (12, 6)
# Cell 2: Generate Synthetic Product Feed Data
def generate_product_feed(n_products=10000, seed=42):
"""Generate a synthetic Google Shopping product feed with realistic correlated quality issues."""
np.random.seed(seed)
categories = [
"Electronics > Smartphones",
"Electronics > Laptops",
"Apparel > Men's Clothing",
"Apparel > Women's Clothing",
"Home & Garden > Furniture",
"Home & Garden > Kitchen",
"Sports > Fitness Equipment",
"Beauty > Skincare",
"Toys > Board Games",
"Food > Snacks"
]
brands = [
"TechCorp", "StyleMax", "HomeBase", "FitGear", "GlowUp",
"SmartLife", "UrbanWear", "KitchenPro", "PlayWorld", "NatureBite",
"EliteGadget", "ComfortZone", "ActiveEdge", "PureSkin", "FunFactory"
]
conditions = ["new", "refurbished", "used"]
availability_options = ["in_stock", "out_of_stock", "preorder"]
seller_tiers = ["gold", "silver", "bronze"]
ids = [f"PROD-{i:06d}" for i in range(n_products)]
chosen_categories = np.random.choice(categories, n_products)
chosen_brands = np.random.choice(brands, n_products)
prices = np.round(np.random.lognormal(mean=3.5, sigma=1.0, size=n_products), 2)
seller_tier = np.random.choice(seller_tiers, n_products, p=[0.3, 0.45, 0.25])
tier_multipliers = {"gold": 0.3, "silver": 1.0, "bronze": 2.5}
base_titles = []
for i in range(n_products):
brand = chosen_brands[i]
cat = chosen_categories[i].split(" > ")[1]
adjectives = ["Premium", "Classic", "Pro", "Ultra", "Essential", "Deluxe"]
adj = np.random.choice(adjectives)
title = f"{brand} {adj} {cat} - Model {np.random.randint(100, 999)}"
base_titles.append(title)
gtins = []
for i in range(n_products):
base_prob = 0.15
prob = min(base_prob * tier_multipliers[seller_tier[i]], 0.95)
roll = np.random.random()
if roll < prob:
gtins.append(None)
elif roll < prob + 0.03:
gtins.append(str(np.random.randint(100, 999)))
else:
gtins.append(str(np.random.randint(1000000000000, 9999999999999)))
df = pd.DataFrame({
"id": ids,
"title": base_titles,
"description": [
f"High-quality {chosen_categories[i].split(' > ')[1].lower()} from {chosen_brands[i]}. "
f"Perfect for everyday use." if np.random.random() > 0.06 * tier_multipliers[seller_tier[i]]
else None
for i in range(n_products)
],
"link": [f"https://example.com/products/{ids[i].lower()}" for i in range(n_products)],
"image_link": [
f"https://example.com/images/{ids[i].lower()}.jpg"
if np.random.random() > 0.05 * tier_multipliers[seller_tier[i]]
else None
for i in range(n_products)
],
"availability": np.random.choice(availability_options, n_products, p=[0.75, 0.20, 0.05]),
"price": prices,
"brand": [
chosen_brands[i] if np.random.random() > 0.08 * tier_multipliers[seller_tier[i]] else None
for i in range(n_products)
],
"gtin": gtins,
"google_product_category": chosen_categories,
"condition": np.random.choice(conditions, n_products, p=[0.85, 0.10, 0.05]),
"seller_tier": seller_tier
})
for i in range(n_products):
if np.random.random() < 0.03 * tier_multipliers[seller_tier[i]]:
df.at[i, "price"] = np.random.choice([0, -1, -5.99])
for i in range(n_products):
prob = 0.10 * tier_multipliers[seller_tier[i]]
if np.random.random() < prob:
issue_type = np.random.choice(["short", "caps", "promo"])
if issue_type == "short":
df.at[i, "title"] = "Item"
elif issue_type == "caps":
df.at[i, "title"] = df.at[i, "title"].upper()
else:
df.at[i, "title"] = "FREE SHIPPING! " + df.at[i, "title"] + " BEST DEAL!!!"
return df
# Generate the dataset
df = generate_product_feed()
print(f"Dataset shape: {df.shape}")
print(f"\nFirst 5 rows:")
df.head()
# Cell 3: Verify Dataset and Explore Seller Tier Patterns
print(f"Dataset shape: {df.shape}")
print(f"\nData types:")
print(df.dtypes)
print(f"\nNull counts:")
print(df.isnull().sum())
print(f"\n=== QUALITY BY SELLER TIER ===\n")
for tier in ["gold", "silver", "bronze"]:
tier_df = df[df["seller_tier"] == tier]
null_rate = tier_df.isnull().sum().sum() / (len(tier_df) * len(tier_df.columns)) * 100
print(f" {tier.UPPER():8s} | {len(tier_df):,} products | Null rate: {null_rate:.1f}%")Your dataset is ready. Next up, you'll profile this feed data to understand exactly where the quality gaps are and how different attributes correlate with each other.
Profile the Feed Data
You now have a 10,000-product feed with realistic quality issues baked in. Before you can fix problems, you need to understand where the gaps are and how severe they are.
In this step, you'll profile the feed to measure attribute completeness, weight each gap by its impact on Google Merchant Center approval, and visualize how missing values cluster together across products.
In this step, get ready to:
- Build a completeness report with severity weighting based on Merchant Center requirements.
- Analyze data distributions and identify outlier patterns using visualizations.
Build a completeness report with severity weighting
Not all missing data is equally bad. A missing title guarantees disapproval, but a missing description only reduces ad quality. You'll assign severity weights to each attribute so the completeness score reflects actual disapproval risk.
- Create a new code cell in your notebook and add the completeness report code by pasting the following:
# Cell 4: Completeness Report with Severity Weighting
# Severity weights: required (3), strongly recommended (2), optional (1)
severity_weights = {
"id": 3, "title": 3, "link": 3, "image_link": 3,
"availability": 3, "price": 3,
"brand": 2, "gtin": 2, "google_product_category": 2,
"description": 1, "condition": 1, "seller_tier": 0
}
completeness = (df.notnull().sum() / len(df) * 100).round(2)
completeness_df = pd.DataFrame({
"attribute": completeness.index,
"completeness_pct": completeness.values,
"severity_weight": [severity_weights.get(col, 0) for col in completeness.index]
}).sort_values("completeness_pct", ascending=True).reset_index(drop=True)
# Weighted completeness score
weighted_attrs = completeness_df[completeness_df["severity_weight"] > 0]
weighted_score = (
(weighted_attrs["completeness_pct"] * weighted_attrs["severity_weight"]).sum()
/ (weighted_attrs["severity_weight"].sum() * 100) * 100
)
print("=== FEED COMPLETENESS REPORT ===\n")
print(completeness_df[completeness_df["severity_weight"] > 0].to_string(index=False))
print(f"\nWeighted Completeness Score: {weighted_score:.1f}/100")
print(f"Attributes below 100% completeness: {(completeness < 100).sum()}")What does this code do?
The severity_weights dictionary assigns a weight to each attribute based on how Google Merchant Center treats it. Required attributes like title and price get weight 3 because missing them causes automatic disapproval.
The weighted score multiplies each attribute's completeness percentage by its weight, then normalizes to a 0-100 scale. This way, a feed with perfect required attributes but gaps in optional ones scores higher than one with the reverse pattern.
Why weight by severity instead of using raw completeness?
A naive completeness check treats every column equally. A feed with 100% description but 80% price would look decent overall. In reality, those 20% missing prices mean 20% guaranteed disapprovals.
Weighting surfaces the attributes that actually control whether products appear in Shopping results.
- Run the cell by pressing Shift+Enter.
- You should see a table listing each attribute, its completeness percentage, and its severity weight. The weighted completeness score is printed at the bottom (it should be a number in the mid-90s, reflecting that most required attributes are well-populated).
Seeing a NameError for df?
Make sure you ran all previous cells first (Cell 1 through Cell 3). Colab cells must be executed in order for variables to exist in memory.
You can run all cells above by clicking Runtime then Run before.
Help me fix a NameError in my Google Colab notebook
Analyze data distributions and identify outlier patterns
Completeness tells you what's missing. Distributions tell you what's wrong with the data that IS present. You'll visualize price outliers and examine how missing values correlate across columns.
- Create a new code cell and paste the distribution analysis code:
# Cell 5: Data Distribution and Missing Value Correlation
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
# Left: Price distribution box plot
axes[0].boxplot(df["price"][df["price"] > 0], vert=True)
axes[0].set_title("Price Distribution (excluding invalid)", fontsize=12, fontweight="bold")
axes[0].set_ylabel("Price ($)")
# Right: Missing value correlation matrix
nullable_cols = ["description", "image_link", "brand", "gtin"]
null_corr = df[nullable_cols].isnull().corr()
sns.heatmap(null_corr, annot=True, fmt=".2f", cmap="coolwarm", ax=axes[1],
vmin=-1, vmax=1, linewidths=0.5)
axes[1].set_title("Missing Value Correlation Matrix", fontsize=12, fontweight="bold")
plt.tight_layout()
plt.show()
print("\n=== CATEGORY DISTRIBUTION ===\n")
print(df["google_product_category"].value_counts())
print(f"\n=== AVAILABILITY DISTRIBUTION ===\n")
print(df["availability"].value_counts())What does this code do?
- The box plot shows the price distribution after filtering out invalid (zero/negative) prices. Outliers appear as dots above the whiskers.
- The .isnull().corr() call creates a correlation matrix of missing values. A high positive correlation between two columns means they tend to be missing together on the same products.
- The category and availability counts give you a sense of how products are distributed across the feed.
- Run the cell by pressing Shift+Enter.
- You should see two charts side by side. The left chart is a box plot showing price spread with outlier dots. The right chart is a heatmap with correlation values between missing columns.
- Below the charts, you should see category counts (roughly 1,000 per category) and availability status counts.
What does the correlation heatmap reveal?
Look for positive values between columns. A positive correlation between gtin and brand means products missing a GTIN also tend to be missing a brand. This pattern emerges because bronze-tier sellers submit lower-quality data across the board.
This insight matters because fixing one error on those products still leaves them disapproved if other errors remain.
Charts not displaying?
Google Colab renders matplotlib charts inline by default. If you see text output but no charts, make sure you have the plt.show() call at the end of the plotting section. Also confirm that your Cell 1 imports ran successfully.
Help me troubleshoot matplotlib charts not showing in Colab
✔️ Awesome, I've got everything!
Great. Double check you've saved your notebook before moving on.
ⓧ I'd like to double check the full code
Here is the complete notebook code up to this point. Compare it against your cells:
# Cell 1: Import Libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import re
from itertools import combinations
# Set visual style
sns.set_style("whitegrid")
plt.rcParams['figure.figsize'] = (12, 6)
# Cell 2: Generate Synthetic Product Feed Data
def generate_product_feed(n_products=10000, seed=42):
"""Generate a synthetic Google Shopping product feed with realistic correlated quality issues."""
np.random.seed(seed)
categories = [
"Electronics > Smartphones",
"Electronics > Laptops",
"Apparel > Men's Clothing",
"Apparel > Women's Clothing",
"Home & Garden > Furniture",
"Home & Garden > Kitchen",
"Sports > Fitness Equipment",
"Beauty > Skincare",
"Toys > Board Games",
"Food > Snacks"
]
brands = [
"TechCorp", "StyleMax", "HomeBase", "FitGear", "GlowUp",
"SmartLife", "UrbanWear", "KitchenPro", "PlayWorld", "NatureBite",
"EliteGadget", "ComfortZone", "ActiveEdge", "PureSkin", "FunFactory"
]
conditions = ["new", "refurbished", "used"]
availability_options = ["in_stock", "out_of_stock", "preorder"]
seller_tiers = ["gold", "silver", "bronze"]
ids = [f"PROD-{i:06d}" for i in range(n_products)]
chosen_categories = np.random.choice(categories, n_products)
chosen_brands = np.random.choice(brands, n_products)
prices = np.round(np.random.lognormal(mean=3.5, sigma=1.0, size=n_products), 2)
seller_tier = np.random.choice(seller_tiers, n_products, p=[0.3, 0.45, 0.25])
tier_multipliers = {"gold": 0.3, "silver": 1.0, "bronze": 2.5}
base_titles = []
for i in range(n_products):
brand = chosen_brands[i]
cat = chosen_categories[i].split(" > ")[1]
adjectives = ["Premium", "Classic", "Pro", "Ultra", "Essential", "Deluxe"]
adj = np.random.choice(adjectives)
title = f"{brand} {adj} {cat} - Model {np.random.randint(100, 999)}"
base_titles.append(title)
gtins = []
for i in range(n_products):
base_prob = 0.15
prob = min(base_prob * tier_multipliers[seller_tier[i]], 0.95)
roll = np.random.random()
if roll < prob:
gtins.append(None)
elif roll < prob + 0.03:
gtins.append(str(np.random.randint(100, 999)))
else:
gtins.append(str(np.random.randint(1000000000000, 9999999999999)))
df = pd.DataFrame({
"id": ids,
"title": base_titles,
"description": [
f"High-quality {chosen_categories[i].split(' > ')[1].lower()} from {chosen_brands[i]}. "
f"Perfect for everyday use." if np.random.random() > 0.06 * tier_multipliers[seller_tier[i]]
else None
for i in range(n_products)
],
"link": [f"https://example.com/products/{ids[i].lower()}" for i in range(n_products)],
"image_link": [
f"https://example.com/images/{ids[i].lower()}.jpg"
if np.random.random() > 0.05 * tier_multipliers[seller_tier[i]]
else None
for i in range(n_products)
],
"availability": np.random.choice(availability_options, n_products, p=[0.75, 0.20, 0.05]),
"price": prices,
"brand": [
chosen_brands[i] if np.random.random() > 0.08 * tier_multipliers[seller_tier[i]] else None
for i in range(n_products)
],
"gtin": gtins,
"google_product_category": chosen_categories,
"condition": np.random.choice(conditions, n_products, p=[0.85, 0.10, 0.05]),
"seller_tier": seller_tier
})
for i in range(n_products):
if np.random.random() < 0.03 * tier_multipliers[seller_tier[i]]:
df.at[i, "price"] = np.random.choice([0, -1, -5.99])
for i in range(n_products):
prob = 0.10 * tier_multipliers[seller_tier[i]]
if np.random.random() < prob:
issue_type = np.random.choice(["short", "caps", "promo"])
if issue_type == "short":
df.at[i, "title"] = "Item"
elif issue_type == "caps":
df.at[i, "title"] = df.at[i, "title"].upper()
else:
df.at[i, "title"] = "FREE SHIPPING! " + df.at[i, "title"] + " BEST DEAL!!!"
return df
# Generate the dataset
df = generate_product_feed()
print(f"Dataset shape: {df.shape}")
print(f"\nFirst 5 rows:")
df.head()
# Cell 3: Verify Dataset and Explore Seller Tier Patterns
print(f"Dataset shape: {df.shape}")
print(f"\nData types:")
print(df.dtypes)
print(f"\nNull counts:")
print(df.isnull().sum())
print(f"\n=== QUALITY BY SELLER TIER ===\n")
for tier in ["gold", "silver", "bronze"]:
tier_df = df[df["seller_tier"] == tier]
null_rate = tier_df.isnull().sum().sum() / (len(tier_df) * len(tier_df.columns)) * 100
print(f" {tier.upper():8s} | {len(tier_df):,} products | Null rate: {null_rate:.1f}%")
# Cell 4: Completeness Report with Severity Weighting
severity_weights = {
"id": 3, "title": 3, "link": 3, "image_link": 3,
"availability": 3, "price": 3,
"brand": 2, "gtin": 2, "google_product_category": 2,
"description": 1, "condition": 1, "seller_tier": 0
}
completeness = (df.notnull().sum() / len(df) * 100).round(2)
completeness_df = pd.DataFrame({
"attribute": completeness.index,
"completeness_pct": completeness.values,
"severity_weight": [severity_weights.get(col, 0) for col in completeness.index]
}).sort_values("completeness_pct", ascending=True).reset_index(drop=True)
weighted_attrs = completeness_df[completeness_df["severity_weight"] > 0]
weighted_score = (
(weighted_attrs["completeness_pct"] * weighted_attrs["severity_weight"]).sum()
/ (weighted_attrs["severity_weight"].sum() * 100) * 100
)
print("=== FEED COMPLETENESS REPORT ===\n")
print(completeness_df[completeness_df["severity_weight"] > 0].to_string(index=False))
print(f"\nWeighted Completeness Score: {weighted_score:.1f}/100")
print(f"Attributes below 100% completeness: {(completeness < 100).sum()}")
# Cell 5: Data Distribution and Missing Value Correlation
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
axes[0].boxplot(df["price"][df["price"] > 0], vert=True)
axes[0].set_title("Price Distribution (excluding invalid)", fontsize=12, fontweight="bold")
axes[0].set_ylabel("Price ($)")
nullable_cols = ["description", "image_link", "brand", "gtin"]
null_corr = df[nullable_cols].isnull().corr()
sns.heatmap(null_corr, annot=True, fmt=".2f", cmap="coolwarm", ax=axes[1],
vmin=-1, vmax=1, linewidths=0.5)
axes[1].set_title("Missing Value Correlation Matrix", fontsize=12, fontweight="bold")
plt.tight_layout()
plt.show()
print("\n=== CATEGORY DISTRIBUTION ===\n")
print(df["google_product_category"].value_counts())
print(f"\n=== AVAILABILITY DISTRIBUTION ===\n")
print(df["availability"].value_counts())You now have a clear picture of where the gaps are and how they cluster. Next up, you'll build a validation engine that checks every product against Google's rules and flags exactly what's wrong.
Build the Feed Validation Engine
Your profiling work revealed where the gaps live in your feed data. Now you need to determine which products would actually get disapproved by Google Shopping.
In this step, you'll build a validation engine that checks every product against Google Merchant Center's product data specification. You'll then analyze how errors overlap across products to find which fixes deliver the most real impact.
In this step, get ready to:
- Define validation rules for the top 5 disapproval causes.
- Flag every product with pass/fail results and compute multi-error overlap.
- Calculate feed health metrics including unique impact per error type.
Define validation rules based on Merchant Center specs
Google's product data specification defines exactly what makes a product eligible for Shopping ads. Products that violate these rules get disapproved and become invisible to shoppers.
You'll encode the top 5 disapproval causes as Python functions that return True (valid) or False (would be disapproved).
- Create a new code cell below your existing cells and paste the following validation functions:
# Cell 6: Define Validation Functions
def validate_gtin(gtin_value):
"""Check if GTIN is valid (8, 12, 13, or 14 digits)."""
if pd.isna(gtin_value):
return False
gtin_str = str(gtin_value).strip()
if not gtin_str.isdigit():
return False
return len(gtin_str) in [8, 12, 13, 14]
def validate_price(price_value):
"""Check if price is positive and valid."""
if pd.isna(price_value):
return False
return price_value > 0
def validate_title(title_value):
"""Check title against Google Shopping best practices."""
if pd.isna(title_value) or len(str(title_value).strip()) == 0:
return False
title = str(title_value)
# Must be between 10 and 150 characters
if len(title) > 150 or len(title) < 10:
return False
# No ALL CAPS (more than 50% uppercase letters)
letters = [c for c in title if c.isalpha()]
if letters and sum(1 for c in letters if c.isupper()) / len(letters) > 0.5:
return False
# No promotional text
promo_patterns = r'\b(FREE|SALE|BEST DEAL|BUY NOW|DISCOUNT|CLEARANCE)\b'
if re.search(promo_patterns, title, re.IGNORECASE):
return False
return True
def validate_required_attrs(row):
"""Check that all required attributes are present."""
required = ["title", "link", "image_link", "availability", "price"]
for attr in required:
if pd.isna(row[attr]):
return False
if isinstance(row[attr], str) and len(row[attr].strip()) == 0:
return False
return True
def validate_image(image_link_value):
"""Check if image link is present and has valid format."""
if pd.isna(image_link_value):
return False
img = str(image_link_value).strip()
if len(img) == 0:
return False
valid_extensions = ('.jpg', '.jpeg', '.png', '.gif', '.webp')
return img.lower().endswith(valid_extensions)What do these validation functions check?
Each function targets a specific disapproval cause from the Google product data specification:
- validate_gtin() checks that GTINs are 8, 12, 13, or 14 digits. Missing or malformed identifiers are a top disapproval cause.
- validate_price() rejects zero or negative prices. Google requires prices to be positive and properly formatted.
- validate_title() enforces Google's title rules: 10-150 characters, no ALL CAPS (over 50% uppercase), and no promotional text like FREE or SALE.
- validate_required_attrs() confirms that the five mandatory attributes (title, link, image_link, availability, price) are all present and non-empty.
- validate_image() checks that the image link exists and ends with a valid image extension.
- Run the cell by pressing the play button or hitting Shift+Enter.
The cell completes silently because you've only defined functions. You haven't called them yet.
Seeing a NameError or SyntaxError?
- If you see a NameError for pd or re, make sure you ran Cell 1 (imports) earlier in this session.
- Check that the regex string in validate_title uses a raw string prefix (the r before the opening quote).
Why aren't my validation functions working?
Run validation, flag errors, and compute multi-error overlap
With the rules defined, you'll now apply them to all 10,000 products. The key insight here is that many products fail multiple checks simultaneously. A product missing its image link might also be missing its GTIN.
- Create a new code cell and paste the following code to apply all five validators and identify co-occurring error pairs:
# Cell 7: Apply Validation and Compute Multi-Error Overlap
df["has_valid_gtin"] = df["gtin"].apply(validate_gtin)
df["has_valid_price"] = df["price"].apply(validate_price)
df["has_valid_title"] = df["title"].apply(validate_title)
df["has_required_attrs"] = df.apply(validate_required_attrs, axis=1)
df["has_valid_image"] = df["image_link"].apply(validate_image)
# Overall approval: must pass ALL checks
df["approved"] = (
df["has_valid_gtin"] &
df["has_valid_price"] &
df["has_valid_title"] &
df["has_required_attrs"] &
df["has_valid_image"]
)
# Count errors per product
validation_cols = ["has_valid_gtin", "has_valid_price", "has_valid_title", "has_required_attrs", "has_valid_image"]
df["error_count"] = (~df[validation_cols]).sum(axis=1)
# Map column names to readable labels
error_col_names = {
"has_valid_gtin": "GTIN",
"has_valid_price": "Price",
"has_valid_title": "Title",
"has_required_attrs": "Required",
"has_valid_image": "Image"
}
# Find co-occurring error pairs
error_pairs = []
for _, row in df[df["error_count"] >= 2].iterrows():
errors = [error_col_names[col] for col in validation_cols if not row[col]]
for pair in combinations(errors, 2):
error_pairs.append(tuple(sorted(pair)))
pair_counts = pd.Series(error_pairs).value_counts().head(10)
print("=== MOST COMMON ERROR PAIRS ===\n")
for pair, count in pair_counts.items():
print(f" {pair[0]} + {pair[1]}: {count:,} products")What does this code do?
- .apply() runs each validation function across every row, creating a boolean column for each check.
- df["approved"] uses the & operator to combine all five checks. A product is approved only if every single check passes.
- df["error_count"] inverts the boolean columns with ~ (turning True to False) then sums them. This counts how many checks each product failed.
- The error pairs loop finds products with 2+ errors and uses combinations() from itertools to record which errors appear together.
- Run the cell.
You should see a list of the 10 most common error pairs, showing how many products share each combination of failures.
Cell running very slowly?
- The error pairs loop iterates only over products with 2+ errors (a subset of the data). If it stalls, confirm you ran Cell 2 to generate the DataFrame first.
- If you see a NameError for combinations, check that Cell 1 imported it with from itertools import combinations.
Why is my validation cell failing?
Calculate feed health metrics with error overlap analysis
Raw error counts tell you how common each problem is. But they don't tell you what to fix first. A product with 3 errors won't become approved by fixing just one of them.
The unique impact metric answers a sharper question: how many products would move from disapproved to approved if you fixed only that one error type?
- Create a new code cell and paste the following code to compute the full feed health report:
# Cell 8: Calculate Feed Health Metrics with Unique Impact
total = len(df)
approved_count = df["approved"].sum()
disapproved_count = total - approved_count
approval_rate = approved_count / total * 100
print("=" * 50)
print(" FEED HEALTH REPORT")
print("=" * 50)
print(f"\n Total Products: {total:,}")
print(f" Approved: {approved_count:,}")
print(f" Disapproved: {disapproved_count:,}")
print(f" Approval Rate: {approval_rate:.1f}%")
print(f"\n{'=' * 50}")
# Error breakdown with unique impact
error_counts = {}
unique_impacts = {}
for col in validation_cols:
error_name = error_col_names[col]
raw_count = (~df[col]).sum()
error_counts[error_name] = raw_count
# Unique impact: products that ONLY fail this check
other_cols = [c for c in validation_cols if c != col]
only_this_error = (~df[col]) & df[other_cols].all(axis=1)
unique_impacts[error_name] = only_this_error.sum()
print("\n=== ERROR BREAKDOWN (Raw Count vs Unique Impact) ===\n")
print(f" {'Error Type':<25} {'Raw Count':>10} {'Unique Impact':>15} {'Overlap %':>10}")
print(f" {'-'*25} {'-'*10} {'-'*15} {'-'*10}")
for error_name in sorted(error_counts.keys(), key=lambda x: error_counts[x], reverse=True):
raw = error_counts[error_name]
unique = unique_impacts[error_name]
overlap_pct = (1 - unique / raw) * 100 if raw > 0 else 0
print(f" {error_name:<25} {raw:>10,} {unique:>15,} {overlap_pct:>9.1f}%")
# Approval rate per category and seller tier
print("\n=== APPROVAL RATE BY SELLER TIER ===\n")
tier_approval = df.groupby("seller_tier")["approved"].mean() * 100
for tier in ["gold", "silver", "bronze"]:
print(f" {tier.UPPER():8s}: {tier_approval[tier]:.1f}%")What does the unique impact calculation do?
For each error type, the code finds products that ONLY fail that one check (all other checks pass). These are the products that would immediately become approved if you fixed just that error.
The Overlap % column shows what percentage of products with a given error also have at least one other error. A high overlap means fixing that error alone won't approve those products.
- Run the cell.
You should see a formatted FEED HEALTH REPORT showing the overall approval rate, a breakdown table comparing raw counts to unique impacts, and approval rates for each seller tier (gold, silver, bronze).
Seeing a KeyError or unexpected approval rate?
- If you get a KeyError for validation_cols or error_col_names, make sure you ran Cell 7 first. These variables are defined there.
- If your approval rate is 0% or 100%, double-check that Cell 6 (validation functions) ran without errors.
Help me debug my feed health metrics cell.
✔️ Awesome, I've got everything!
Great. Make sure all three cells (6, 7, and 8) ran without errors before continuing.
ⓧ I'd like to double check the full code
Here is the complete notebook code up to and including this step. Compare your cells against this:
# Cell 1: Import Libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import re
from itertools import combinations
# Set visual style
sns.set_style("whitegrid")
plt.rcParams['figure.figsize'] = (12, 6)
# Cell 2: Generate Synthetic Product Feed Data
def generate_product_feed(n_products=10000, seed=42):
"""Generate a synthetic Google Shopping product feed with realistic correlated quality issues."""
np.random.seed(seed)
categories = [
"Electronics > Smartphones",
"Electronics > Laptops",
"Apparel > Men's Clothing",
"Apparel > Women's Clothing",
"Home & Garden > Furniture",
"Home & Garden > Kitchen",
"Sports > Fitness Equipment",
"Beauty > Skincare",
"Toys > Board Games",
"Food > Snacks"
]
brands = [
"TechCorp", "StyleMax", "HomeBase", "FitGear", "GlowUp",
"SmartLife", "UrbanWear", "KitchenPro", "PlayWorld", "NatureBite",
"EliteGadget", "ComfortZone", "ActiveEdge", "PureSkin", "FunFactory"
]
conditions = ["new", "refurbished", "used"]
availability_options = ["in_stock", "out_of_stock", "preorder"]
seller_tiers = ["gold", "silver", "bronze"]
ids = [f"PROD-{i:06d}" for i in range(n_products)]
chosen_categories = np.random.choice(categories, n_products)
chosen_brands = np.random.choice(brands, n_products)
prices = np.round(np.random.lognormal(mean=3.5, sigma=1.0, size=n_products), 2)
seller_tier = np.random.choice(seller_tiers, n_products, p=[0.3, 0.45, 0.25])
tier_multipliers = {"gold": 0.3, "silver": 1.0, "bronze": 2.5}
base_titles = []
for i in range(n_products):
brand = chosen_brands[i]
cat = chosen_categories[i].split(" > ")[1]
adjectives = ["Premium", "Classic", "Pro", "Ultra", "Essential", "Deluxe"]
adj = np.random.choice(adjectives)
title = f"{brand} {adj} {cat} - Model {np.random.randint(100, 999)}"
base_titles.append(title)
gtins = []
for i in range(n_products):
base_prob = 0.15
prob = min(base_prob * tier_multipliers[seller_tier[i]], 0.95)
roll = np.random.random()
if roll < prob:
gtins.append(None)
elif roll < prob + 0.03:
gtins.append(str(np.random.randint(100, 999)))
else:
gtins.append(str(np.random.randint(1000000000000, 9999999999999)))
df = pd.DataFrame({
"id": ids,
"title": base_titles,
"description": [
f"High-quality {chosen_categories[i].split(' > ')[1].lower()} from {chosen_brands[i]}. "
f"Perfect for everyday use." if np.random.random() > 0.06 * tier_multipliers[seller_tier[i]]
else None
for i in range(n_products)
],
"link": [f"https://example.com/products/{ids[i].lower()}" for i in range(n_products)],
"image_link": [
f"https://example.com/images/{ids[i].lower()}.jpg"
if np.random.random() > 0.05 * tier_multipliers[seller_tier[i]]
else None
for i in range(n_products)
],
"availability": np.random.choice(availability_options, n_products, p=[0.75, 0.20, 0.05]),
"price": prices,
"brand": [
chosen_brands[i] if np.random.random() > 0.08 * tier_multipliers[seller_tier[i]] else None
for i in range(n_products)
],
"gtin": gtins,
"google_product_category": chosen_categories,
"condition": np.random.choice(conditions, n_products, p=[0.85, 0.10, 0.05]),
"seller_tier": seller_tier
})
for i in range(n_products):
if np.random.random() < 0.03 * tier_multipliers[seller_tier[i]]:
df.at[i, "price"] = np.random.choice([0, -1, -5.99])
for i in range(n_products):
prob = 0.10 * tier_multipliers[seller_tier[i]]
if np.random.random() < prob:
issue_type = np.random.choice(["short", "caps", "promo"])
if issue_type == "short":
df.at[i, "title"] = "Item"
elif issue_type == "caps":
df.at[i, "title"] = df.at[i, "title"].upper()
else:
df.at[i, "title"] = "FREE SHIPPING! " + df.at[i, "title"] + " BEST DEAL!!!"
return df
# Generate the dataset
df = generate_product_feed()
print(f"Dataset shape: {df.shape}")
print(f"\nFirst 5 rows:")
df.head()
# Cell 3: Verify Dataset and Explore Seller Tier Patterns
print(f"Dataset shape: {df.shape}")
print(f"\nData types:")
print(df.dtypes)
print(f"\nNull counts:")
print(df.isnull().sum())
print(f"\n=== QUALITY BY SELLER TIER ===\n")
for tier in ["gold", "silver", "bronze"]:
tier_df = df[df["seller_tier"] == tier]
null_rate = tier_df.isnull().sum().sum() / (len(tier_df) * len(tier_df.columns)) * 100
print(f" {tier.upper():8s} | {len(tier_df):,} products | Null rate: {null_rate:.1f}%")
# Cell 4: Completeness Report with Severity Weighting
severity_weights = {
"id": 3, "title": 3, "link": 3, "image_link": 3,
"availability": 3, "price": 3,
"brand": 2, "gtin": 2, "google_product_category": 2,
"description": 1, "condition": 1, "seller_tier": 0
}
completeness = (df.notnull().sum() / len(df) * 100).round(2)
completeness_df = pd.DataFrame({
"attribute": completeness.index,
"completeness_pct": completeness.values,
"severity_weight": [severity_weights.get(col, 0) for col in completeness.index]
}).sort_values("completeness_pct", ascending=True).reset_index(drop=True)
weighted_attrs = completeness_df[completeness_df["severity_weight"] > 0]
weighted_score = (
(weighted_attrs["completeness_pct"] * weighted_attrs["severity_weight"]).sum()
/ (weighted_attrs["severity_weight"].sum() * 100) * 100
)
print("=== FEED COMPLETENESS REPORT ===\n")
print(completeness_df[completeness_df["severity_weight"] > 0].to_string(index=False))
print(f"\nWeighted Completeness Score: {weighted_score:.1f}/100")
print(f"Attributes below 100% completeness: {(completeness < 100).sum()}")
# Cell 5: Data Distribution and Missing Value Correlation
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
axes[0].boxplot(df["price"][df["price"] > 0], vert=True)
axes[0].set_title("Price Distribution (excluding invalid)", fontsize=12, fontweight="bold")
axes[0].set_ylabel("Price ($)")
nullable_cols = ["description", "image_link", "brand", "gtin"]
null_corr = df[nullable_cols].isnull().corr()
sns.heatmap(null_corr, annot=True, fmt=".2f", cmap="coolwarm", ax=axes[1],
vmin=-1, vmax=1, linewidths=0.5)
axes[1].set_title("Missing Value Correlation Matrix", fontsize=12, fontweight="bold")
plt.tight_layout()
plt.show()
print("\n=== CATEGORY DISTRIBUTION ===\n")
print(df["google_product_category"].value_counts())
print(f"\n=== AVAILABILITY DISTRIBUTION ===\n")
print(df["availability"].value_counts())
# Cell 6: Define Validation Functions
def validate_gtin(gtin_value):
"""Check if GTIN is valid (8, 12, 13, or 14 digits)."""
if pd.isna(gtin_value):
return False
gtin_str = str(gtin_value).strip()
if not gtin_str.isdigit():
return False
return len(gtin_str) in [8, 12, 13, 14]
def validate_price(price_value):
"""Check if price is positive and valid."""
if pd.isna(price_value):
return False
return price_value > 0
def validate_title(title_value):
"""Check title against Google Shopping best practices."""
if pd.isna(title_value) or len(str(title_value).strip()) == 0:
return False
title = str(title_value)
# Must be between 10 and 150 characters
if len(title) > 150 or len(title) < 10:
return False
# No ALL CAPS (more than 50% uppercase letters)
letters = [c for c in title if c.isalpha()]
if letters and sum(1 for c in letters if c.isupper()) / len(letters) > 0.5:
return False
# No promotional text
promo_patterns = r'\b(FREE|SALE|BEST DEAL|BUY NOW|DISCOUNT|CLEARANCE)\b'
if re.search(promo_patterns, title, re.IGNORECASE):
return False
return True
def validate_required_attrs(row):
"""Check that all required attributes are present."""
required = ["title", "link", "image_link", "availability", "price"]
for attr in required:
if pd.isna(row[attr]):
return False
if isinstance(row[attr], str) and len(row[attr].strip()) == 0:
return False
return True
def validate_image(image_link_value):
"""Check if image link is present and has valid format."""
if pd.isna(image_link_value):
return False
img = str(image_link_value).strip()
if len(img) == 0:
return False
valid_extensions = ('.jpg', '.jpeg', '.png', '.gif', '.webp')
return img.lower().endswith(valid_extensions)
# Cell 7: Apply Validation and Compute Multi-Error Overlap
df["has_valid_gtin"] = df["gtin"].apply(validate_gtin)
df["has_valid_price"] = df["price"].apply(validate_price)
df["has_valid_title"] = df["title"].apply(validate_title)
df["has_required_attrs"] = df.apply(validate_required_attrs, axis=1)
df["has_valid_image"] = df["image_link"].apply(validate_image)
df["approved"] = (
df["has_valid_gtin"] &
df["has_valid_price"] &
df["has_valid_title"] &
df["has_required_attrs"] &
df["has_valid_image"]
)
validation_cols = ["has_valid_gtin", "has_valid_price", "has_valid_title", "has_required_attrs", "has_valid_image"]
df["error_count"] = (~df[validation_cols]).sum(axis=1)
error_col_names = {
"has_valid_gtin": "GTIN",
"has_valid_price": "Price",
"has_valid_title": "Title",
"has_required_attrs": "Required",
"has_valid_image": "Image"
}
error_pairs = []
for _, row in df[df["error_count"] >= 2].iterrows():
errors = [error_col_names[col] for col in validation_cols if not row[col]]
for pair in combinations(errors, 2):
error_pairs.append(tuple(sorted(pair)))
pair_counts = pd.Series(error_pairs).value_counts().head(10)
print("=== MOST COMMON ERROR PAIRS ===\n")
for pair, count in pair_counts.items():
print(f" {pair[0]} + {pair[1]}: {count:,} products")
# Cell 8: Calculate Feed Health Metrics with Unique Impact
total = len(df)
approved_count = df["approved"].sum()
disapproved_count = total - approved_count
approval_rate = approved_count / total * 100
print("=" * 50)
print(" FEED HEALTH REPORT")
print("=" * 50)
print(f"\n Total Products: {total:,}")
print(f" Approved: {approved_count:,}")
print(f" Disapproved: {disapproved_count:,}")
print(f" Approval Rate: {approval_rate:.1f}%")
print(f"\n{'=' * 50}")
error_counts = {}
unique_impacts = {}
for col in validation_cols:
error_name = error_col_names[col]
raw_count = (~df[col]).sum()
error_counts[error_name] = raw_count
other_cols = [c for c in validation_cols if c != col]
only_this_error = (~df[col]) & df[other_cols].all(axis=1)
unique_impacts[error_name] = only_this_error.sum()
print("\n=== ERROR BREAKDOWN (Raw Count vs Unique Impact) ===\n")
print(f" {'Error Type':<25} {'Raw Count':>10} {'Unique Impact':>15} {'Overlap %':>10}")
print(f" {'-'*25} {'-'*10} {'-'*15} {'-'*10}")
for error_name in sorted(error_counts.keys(), key=lambda x: error_counts[x], reverse=True):
raw = error_counts[error_name]
unique = unique_impacts[error_name]
overlap_pct = (1 - unique / raw) * 100 if raw > 0 else 0
print(f" {error_name:<25} {raw:>10,} {unique:>15,} {overlap_pct:>9.1f}%")
print("\n=== APPROVAL RATE BY SELLER TIER ===\n")
tier_approval = df.groupby("seller_tier")["approved"].mean() * 100
for tier in ["gold", "silver", "bronze"]:
print(f" {tier.UPPER():8s}: {tier_approval[tier]:.1f}%")Your validation engine now flags every product and reveals which errors genuinely matter. Next up, you'll turn these numbers into a visual dashboard that makes the insights impossible to miss.
Build the Feed Health Dashboard
Your validation engine now flags every product that would fail Google's checks. You have raw counts, unique impact numbers, and approval rates broken down by seller tier.
Numbers in a terminal are useful for debugging, but they don't communicate priorities to a team. In this step, you'll turn those metrics into a visual dashboard that makes feed health problems obvious at a glance and ranks fixes by their true impact.
In this step, get ready to:
- Build an approval rate overview with an error count distribution chart.
- Visualize error co-occurrence and category-level breakdown.
- Generate prioritized fix recommendations with a cumulative improvement curve.
Create the approval rate overview with error count distribution
A feed health dashboard needs to answer three questions immediately: how many products are approved, how are errors distributed across the feed, and which errors have the highest unique impact. One three-panel figure covers all three.
- Create a new code cell in your Colab notebook and paste the following code:
# Cell 9: Dashboard - Approval Overview and Error Count Distribution
fig, axes = plt.subplots(1, 3, figsize=(18, 5))
status_data = pd.DataFrame({
"Status": ["Approved", "Disapproved"],
"Count": [approved_count, disapproved_count]
})
colors = ["#2ecc71", "#e74c3c"]
axes[0].bar(status_data["Status"], status_data["Count"], color=colors, edgecolor="black", linewidth=0.5)
axes[0].set_title(f"Feed Approval Status\n(Rate: {approval_rate:.1f}%)", fontsize=12, fontweight="bold")
axes[0].set_ylabel("Number of Products")
axes[0].axhline(y=total * 0.9, color="orange", linestyle="--", label="90% Target")
axes[0].legend()
error_dist = df["error_count"].value_counts().sort_index()
axes[1].bar(error_dist.index, error_dist.values, color="#8e44ad", edgecolor="black", linewidth=0.5)
axes[1].set_title("Error Count Distribution\n(Errors per Product)", fontsize=12, fontweight="bold")
axes[1].set_xlabel("Number of Errors")
axes[1].set_ylabel("Number of Products")
axes[1].set_xticks(range(int(error_dist.index.max()) + 1))
impact_df = pd.DataFrame({
"Error": list(error_counts.keys()),
"Raw Count": list(error_counts.values()),
"Unique Impact": [unique_impacts[e] for e in error_counts.keys()]
}).sort_values("Unique Impact", ascending=True)
y_pos = range(len(impact_df))
axes[2].barh(y_pos, impact_df["Raw Count"].values, color="#bdc3c7", edgecolor="black", linewidth=0.5, label="Shared (overlap)")
axes[2].barh(y_pos, impact_df["Unique Impact"].values, color="#2980b9", edgecolor="black", linewidth=0.5, label="Unique Impact")
axes[2].set_yticks(y_pos)
axes[2].set_yticklabels(impact_df["Error"].values)
axes[2].set_title("Raw Count vs Unique Impact", fontsize=12, fontweight="bold")
axes[2].set_xlabel("Products")
axes[2].legend()
plt.tight_layout()
plt.show()What does this code do?
This builds a three-panel dashboard in a single figure:
- Panel 1 shows a bar chart of approved vs disapproved products, with a dashed orange line at 90% of total products (the industry target for feed health).
- Panel 2 is a histogram of how many errors each product has. Products with 0 errors are approved. Products with 2+ errors need multiple fixes before they can appear in Shopping.
- Panel 3 compares raw error counts (grey) against unique impact (blue) as a horizontal bar chart. The gap between grey and blue shows how much overlap exists for each error type.
- Run the cell.
- You should see a wide figure with three charts side by side. The left chart has green and red bars. The middle chart shows a distribution skewed toward 0 errors. The right chart shows horizontal bars where the blue (unique impact) portion is smaller than the grey (raw count) portion for most error types.
Charts not rendering?
- Make sure you ran all previous cells in order. This cell depends on approved_count, error_counts, and unique_impacts from Cell 8.
- If you see a NameError, click **Runtime** in the Colab menu bar, then **Run all** to re-execute every cell from the top.
My dashboard cell is throwing a NameError for variables like approved_count or error_counts. How do I fix this in Google Colab?
Visualize error overlap and category breakdown
The overview dashboard tells you how bad things are. The next two visualizations tell you why: which errors travel together and which product categories are hurting the most.
- Create a new code cell and paste the error co-occurrence heatmap code:
# Cell 10: Error Co-occurrence Heatmap
cooccurrence = pd.DataFrame(index=error_col_names.values(), columns=error_col_names.values(), dtype=float)
for col_a, name_a in error_col_names.items():
for col_b, name_b in error_col_names.items():
if col_a == col_b:
cooccurrence.loc[name_a, name_b] = 100.0
else:
has_error_a = ~df[col_a]
has_error_b = ~df[col_b]
if has_error_a.sum() > 0:
cooccurrence.loc[name_a, name_b] = (has_error_a & has_error_b).sum() / has_error_a.sum() * 100
else:
cooccurrence.loc[name_a, name_b] = 0.0
fig, ax = plt.subplots(figsize=(8, 6))
sns.heatmap(cooccurrence.astype(float), annot=True, fmt=".1f", cmap="YlOrRd",
ax=ax, linewidths=0.5, vmin=0, vmax=100)
ax.set_title("Error Co-occurrence Rate (%)\n(Of products with Row error, % that also have Column error)", fontsize=11, fontweight="bold")
plt.tight_layout()
plt.show()How to read the co-occurrence heatmap
Each cell answers: "Of all products that have the row error, what percentage also have the column error?" The diagonal is always 100% (a product with GTIN errors always has GTIN errors).
High off-diagonal values reveal error clusters. If 40% of products with missing images also have missing required attributes, those two issues share a root cause (likely bronze-tier sellers submitting incomplete listings).
- Run the cell. You should see a 5x5 heatmap with yellow-to-red coloring. The diagonal cells show 100.0 and off-diagonal cells show varying co-occurrence percentages.
- Create another new code cell and paste the category breakdown code:
# Cell 11: Category Breakdown
category_approval = df.groupby("google_product_category")["approved"].mean() * 100
category_approval = category_approval.sort_values(ascending=True)
fig, ax = plt.subplots(figsize=(12, 6))
category_approval_df = category_approval.reset_index()
category_approval_df.columns = ["Category", "Approval Rate"]
category_approval_df["Category_Short"] = category_approval_df["Category"].apply(lambda x: x.split(" > ")[1])
bars = ax.barh(
category_approval_df["Category_Short"],
category_approval_df["Approval Rate"],
color=["#e74c3c" if x < 60 else "#f39c12" if x < 80 else "#2ecc71" for x in category_approval_df["Approval Rate"]],
edgecolor="black",
linewidth=0.5
)
ax.axvline(x=90, color="black", linestyle="--", linewidth=1, label="90% Target")
ax.set_xlabel("Approval Rate (%)")
ax.set_title("Feed Approval Rate by Product Category", fontsize=13, fontweight="bold")
ax.set_xlim(0, 100)
ax.legend()
plt.tight_layout()
plt.show()What do the colors mean?
Bars are colored by severity threshold. Red means the category's approval rate is below 60%. Orange means it's between 60% and 80%. Green means 80% or above. The dashed vertical line marks the 90% industry target.
This tells a feed manager exactly which product categories need attention first, without reading a single number.
- Run the cell. You should see a horizontal bar chart with 10 product categories listed vertically, colored red, orange, or green depending on their approval rate.
Heatmap or category chart looks wrong?
- If the heatmap shows all zeros, confirm that Cell 7 ran successfully and that error_col_names contains the expected dictionary mapping validation columns to display names.
- If the category chart has no bars, check that df["google_product_category"] is not empty by running df["google_product_category"].nunique() in a scratch cell.
My heatmap is showing all zeros or my category chart has no bars. How do I debug what went wrong with my data?
Generate prioritized fix recommendations with unique impact
The naive approach to improving feed health is to fix the error with the highest raw count first. But that ignores overlap. If most products with error A also have error B, fixing error A alone won't move those products to "approved" status.
Unique impact accounts for this. It measures how many products would actually become approved if you fixed only that one error type. This is the metric that drives real prioritization.
- Create a new code cell and paste the prioritized recommendations code:
# Cell 12: Prioritized Fix Recommendations with Cumulative Improvement
print("=" * 70)
print(" PRIORITIZED FIX RECOMMENDATIONS (by Unique Impact)")
print("=" * 70)
print(f"\n Current Approval Rate: {approval_rate:.1f}%\n")
sorted_errors = sorted(unique_impacts.items(), key=lambda x: x[1], reverse=True)
recommendations = []
cumulative_improvement = approval_rate
improvement_curve = [approval_rate]
for priority, (error_name, unique_count) in enumerate(sorted_errors, 1):
raw = error_counts[error_name]
true_improvement = unique_count / total * 100
cumulative_improvement += true_improvement
improvement_curve.append(cumulative_improvement)
recommendations.append({
"Priority": priority,
"Error Type": error_name,
"Raw Count": raw,
"Unique Impact": unique_count,
"True Improvement": f"+{true_improvement:.1f}pp",
"Cumulative Rate": f"{cumulative_improvement:.1f}%"
})
rec_df = pd.DataFrame(recommendations)
print(rec_df.to_string(index=False))
print(f"\n{'=' * 70}")
print(" 'Unique Impact' = products that become approved by fixing ONLY this error.")
print(" This accounts for multi-error overlap, unlike naive raw-count prioritization.")
print(f"{'=' * 70}")
fig, ax = plt.subplots(figsize=(10, 5))
x_labels = ["Current"] + [r["Error Type"] for r in recommendations]
ax.plot(range(len(improvement_curve)), improvement_curve, marker='o', linewidth=2, color="#2980b9")
ax.axhline(y=90, color="orange", linestyle="--", label="90% Target")
ax.set_xticks(range(len(x_labels)))
ax.set_xticklabels(x_labels, rotation=30, ha="right")
ax.set_ylabel("Approval Rate (%)")
ax.set_title("Cumulative Approval Rate Improvement\n(Fixing Errors in Priority Order)", fontsize=12, fontweight="bold")
ax.legend()
ax.set_ylim(0, 100)
plt.tight_layout()
plt.show()What does this code do?
- It sorts errors by unique impact (descending), not raw count. The error that would convert the most products from disapproved to approved gets priority 1.
- For each error, it calculates the true improvement in percentage points (pp) and the cumulative approval rate if all higher-priority errors were already fixed.
- The line chart at the bottom plots how the feed's approval rate climbs as you fix errors in priority order. Each point on the curve represents one more error type resolved.
- Run the cell.
- You should see a printed table with columns Priority, Error Type, Raw Count, Unique Impact, True Improvement, and Cumulative Rate. Below it, a line chart shows the approval rate climbing from left to right as each error is fixed.
Why unique impact changes the priority order
Notice that the error with the highest raw count may not be priority 1 in the recommendations table. Products with that error often have other errors too, so fixing it alone doesn't move them to approved status.
This is the core insight of multi-error overlap analysis. A feed manager who fixes errors by raw count wastes effort on products that remain disapproved anyway.
Cumulative rate doesn't reach 100%?
- This is expected. The cumulative improvement curve only accounts for products that have exactly one remaining error at each step. Products with errors that were not covered by your five validation checks remain disapproved.
- If the curve is flat (no improvement at all), confirm that unique_impacts contains non-zero values by running print(unique_impacts) in a scratch cell.
My cumulative improvement curve is completely flat. Help me debug why unique_impacts might contain all zeros.
✔️ Awesome, I've got everything!
Great. Make sure all four cells (9 through 12) have been saved and run successfully in your notebook.
ⓧ I'd like to double check the full code
Here is the complete notebook code through Cell 12. Compare it against your cells to spot any differences.
# Cell 1: Import Libraries
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
import re
from itertools import combinations
# Set visual style
sns.set_style("whitegrid")
plt.rcParams['figure.figsize'] = (12, 6)
# Cell 2: Generate Synthetic Product Feed Data
def generate_product_feed(n_products=10000, seed=42):
"""Generate a synthetic Google Shopping product feed with realistic correlated quality issues."""
np.random.seed(seed)
categories = [
"Electronics > Smartphones",
"Electronics > Laptops",
"Apparel > Men's Clothing",
"Apparel > Women's Clothing",
"Home & Garden > Furniture",
"Home & Garden > Kitchen",
"Sports > Fitness Equipment",
"Beauty > Skincare",
"Toys > Board Games",
"Food > Snacks"
]
brands = [
"TechCorp", "StyleMax", "HomeBase", "FitGear", "GlowUp",
"SmartLife", "UrbanWear", "KitchenPro", "PlayWorld", "NatureBite",
"EliteGadget", "ComfortZone", "ActiveEdge", "PureSkin", "FunFactory"
]
conditions = ["new", "refurbished", "used"]
availability_options = ["in_stock", "out_of_stock", "preorder"]
seller_tiers = ["gold", "silver", "bronze"]
ids = [f"PROD-{i:06d}" for i in range(n_products)]
chosen_categories = np.random.choice(categories, n_products)
chosen_brands = np.random.choice(brands, n_products)
prices = np.round(np.random.lognormal(mean=3.5, sigma=1.0, size=n_products), 2)
seller_tier = np.random.choice(seller_tiers, n_products, p=[0.3, 0.45, 0.25])
tier_multipliers = {"gold": 0.3, "silver": 1.0, "bronze": 2.5}
base_titles = []
for i in range(n_products):
brand = chosen_brands[i]
cat = chosen_categories[i].split(" > ")[1]
adjectives = ["Premium", "Classic", "Pro", "Ultra", "Essential", "Deluxe"]
adj = np.random.choice(adjectives)
title = f"{brand} {adj} {cat} - Model {np.random.randint(100, 999)}"
base_titles.append(title)
gtins = []
for i in range(n_products):
base_prob = 0.15
prob = min(base_prob * tier_multipliers[seller_tier[i]], 0.95)
roll = np.random.random()
if roll < prob:
gtins.append(None)
elif roll < prob + 0.03:
gtins.append(str(np.random.randint(100, 999)))
else:
gtins.append(str(np.random.randint(1000000000000, 9999999999999)))
df = pd.DataFrame({
"id": ids,
"title": base_titles,
"description": [
f"High-quality {chosen_categories[i].split(' > ')[1].lower()} from {chosen_brands[i]}. "
f"Perfect for everyday use." if np.random.random() > 0.06 * tier_multipliers[seller_tier[i]]
else None
for i in range(n_products)
],
"link": [f"https://example.com/products/{ids[i].lower()}" for i in range(n_products)],
"image_link": [
f"https://example.com/images/{ids[i].lower()}.jpg"
if np.random.random() > 0.05 * tier_multipliers[seller_tier[i]]
else None
for i in range(n_products)
],
"availability": np.random.choice(availability_options, n_products, p=[0.75, 0.20, 0.05]),
"price": prices,
"brand": [
chosen_brands[i] if np.random.random() > 0.08 * tier_multipliers[seller_tier[i]] else None
for i in range(n_products)
],
"gtin": gtins,
"google_product_category": chosen_categories,
"condition": np.random.choice(conditions, n_products, p=[0.85, 0.10, 0.05]),
"seller_tier": seller_tier
})
for i in range(n_products):
if np.random.random() < 0.03 * tier_multipliers[seller_tier[i]]:
df.at[i, "price"] = np.random.choice([0, -1, -5.99])
for i in range(n_products):
prob = 0.10 * tier_multipliers[seller_tier[i]]
if np.random.random() < prob:
issue_type = np.random.choice(["short", "caps", "promo"])
if issue_type == "short":
df.at[i, "title"] = "Item"
elif issue_type == "caps":
df.at[i, "title"] = df.at[i, "title"].upper()
else:
df.at[i, "title"] = "FREE SHIPPING! " + df.at[i, "title"] + " BEST DEAL!!!"
return df
# Generate the dataset
df = generate_product_feed()
print(f"Dataset shape: {df.shape}")
print(f"\nFirst 5 rows:")
df.head()
# Cell 3: Verify Dataset and Explore Seller Tier Patterns
print(f"Dataset shape: {df.shape}")
print(f"\nData types:")
print(df.dtypes)
print(f"\nNull counts:")
print(df.isnull().sum())
print(f"\n=== QUALITY BY SELLER TIER ===\n")
for tier in ["gold", "silver", "bronze"]:
tier_df = df[df["seller_tier"] == tier]
null_rate = tier_df.isnull().sum().sum() / (len(tier_df) * len(tier_df.columns)) * 100
print(f" {tier.upper():8s} | {len(tier_df):,} products | Null rate: {null_rate:.1f}%")
# Cell 4: Completeness Report with Severity Weighting
severity_weights = {
"id": 3, "title": 3, "link": 3, "image_link": 3,
"availability": 3, "price": 3,
"brand": 2, "gtin": 2, "google_product_category": 2,
"description": 1, "condition": 1, "seller_tier": 0
}
completeness = (df.notnull().sum() / len(df) * 100).round(2)
completeness_df = pd.DataFrame({
"attribute": completeness.index,
"completeness_pct": completeness.values,
"severity_weight": [severity_weights.get(col, 0) for col in completeness.index]
}).sort_values("completeness_pct", ascending=True).reset_index(drop=True)
weighted_attrs = completeness_df[completeness_df["severity_weight"] > 0]
weighted_score = (
(weighted_attrs["completeness_pct"] * weighted_attrs["severity_weight"]).sum()
/ (weighted_attrs["severity_weight"].sum() * 100) * 100
)
print("=== FEED COMPLETENESS REPORT ===\n")
print(completeness_df[completeness_df["severity_weight"] > 0].to_string(index=False))
print(f"\nWeighted Completeness Score: {weighted_score:.1f}/100")
print(f"Attributes below 100% completeness: {(completeness < 100).sum()}")
# Cell 5: Data Distribution and Missing Value Correlation
fig, axes = plt.subplots(1, 2, figsize=(14, 5))
axes[0].boxplot(df["price"][df["price"] > 0], vert=True)
axes[0].set_title("Price Distribution (excluding invalid)", fontsize=12, fontweight="bold")
axes[0].set_ylabel("Price ($)")
nullable_cols = ["description", "image_link", "brand", "gtin"]
null_corr = df[nullable_cols].isnull().corr()
sns.heatmap(null_corr, annot=True, fmt=".2f", cmap="coolwarm", ax=axes[1],
vmin=-1, vmax=1, linewidths=0.5)
axes[1].set_title("Missing Value Correlation Matrix", fontsize=12, fontweight="bold")
plt.tight_layout()
plt.show()
print("\n=== CATEGORY DISTRIBUTION ===\n")
print(df["google_product_category"].value_counts())
print(f"\n=== AVAILABILITY DISTRIBUTION ===\n")
print(df["availability"].value_counts())
# Cell 6: Define Validation Functions
def validate_gtin(gtin_value):
"""Check if GTIN is valid (8, 12, 13, or 14 digits)."""
if pd.isna(gtin_value):
return False
gtin_str = str(gtin_value).strip()
if not gtin_str.isdigit():
return False
return len(gtin_str) in [8, 12, 13, 14]
def validate_price(price_value):
"""Check if price is positive and valid."""
if pd.isna(price_value):
return False
return price_value > 0
def validate_title(title_value):
"""Check title against Google Shopping best practices."""
if pd.isna(title_value) or len(str(title_value).strip()) == 0:
return False
title = str(title_value)
if len(title) > 150 or len(title) < 10:
return False
letters = [c for c in title if c.isalpha()]
if letters and sum(1 for c in letters if c.isupper()) / len(letters) > 0.5:
return False
promo_patterns = r'\b(FREE|SALE|BEST DEAL|BUY NOW|DISCOUNT|CLEARANCE)\b'
if re.search(promo_patterns, title, re.IGNORECASE):
return False
return True
def validate_required_attrs(row):
"""Check that all required attributes are present."""
required = ["title", "link", "image_link", "availability", "price"]
for attr in required:
if pd.isna(row[attr]):
return False
if isinstance(row[attr], str) and len(row[attr].strip()) == 0:
return False
return True
def validate_image(image_link_value):
"""Check if image link is present and has valid format."""
if pd.isna(image_link_value):
return False
img = str(image_link_value).strip()
if len(img) == 0:
return False
valid_extensions = ('.jpg', '.jpeg', '.png', '.gif', '.webp')
return img.lower().endswith(valid_extensions)
# Cell 7: Apply Validation and Compute Multi-Error Overlap
df["has_valid_gtin"] = df["gtin"].apply(validate_gtin)
df["has_valid_price"] = df["price"].apply(validate_price)
df["has_valid_title"] = df["title"].apply(validate_title)
df["has_required_attrs"] = df.apply(validate_required_attrs, axis=1)
df["has_valid_image"] = df["image_link"].apply(validate_image)
df["approved"] = (
df["has_valid_gtin"] &
df["has_valid_price"] &
df["has_valid_title"] &
df["has_required_attrs"] &
df["has_valid_image"]
)
validation_cols = ["has_valid_gtin", "has_valid_price", "has_valid_title", "has_required_attrs", "has_valid_image"]
df["error_count"] = (~df[validation_cols]).sum(axis=1)
error_col_names = {
"has_valid_gtin": "GTIN",
"has_valid_price": "Price",
"has_valid_title": "Title",
"has_required_attrs": "Required",
"has_valid_image": "Image"
}
error_pairs = []
for _, row in df[df["error_count"] >= 2].iterrows():
errors = [error_col_names[col] for col in validation_cols if not row[col]]
for pair in combinations(errors, 2):
error_pairs.append(tuple(sorted(pair)))
pair_counts = pd.Series(error_pairs).value_counts().head(10)
print("=== MOST COMMON ERROR PAIRS ===\n")
for pair, count in pair_counts.items():
print(f" {pair[0]} + {pair[1]}: {count:,} products")
# Cell 8: Calculate Feed Health Metrics with Unique Impact
total = len(df)
approved_count = df["approved"].sum()
disapproved_count = total - approved_count
approval_rate = approved_count / total * 100
print("=" * 50)
print(" FEED HEALTH REPORT")
print("=" * 50)
print(f"\n Total Products: {total:,}")
print(f" Approved: {approved_count:,}")
print(f" Disapproved: {disapproved_count:,}")
print(f" Approval Rate: {approval_rate:.1f}%")
print(f"\n{'=' * 50}")
error_counts = {}
unique_impacts = {}
for col in validation_cols:
error_name = error_col_names[col]
raw_count = (~df[col]).sum()
error_counts[error_name] = raw_count
other_cols = [c for c in validation_cols if c != col]
only_this_error = (~df[col]) & df[other_cols].all(axis=1)
unique_impacts[error_name] = only_this_error.sum()
print("\n=== ERROR BREAKDOWN (Raw Count vs Unique Impact) ===\n")
print(f" {'Error Type':<25} {'Raw Count':>10} {'Unique Impact':>15} {'Overlap %':>10}")
print(f" {'-'*25} {'-'*10} {'-'*15} {'-'*10}")
for error_name in sorted(error_counts.keys(), key=lambda x: error_counts[x], reverse=True):
raw = error_counts[error_name]
unique = unique_impacts[error_name]
overlap_pct = (1 - unique / raw) * 100 if raw > 0 else 0
print(f" {error_name:<25} {raw:>10,} {unique:>15,} {overlap_pct:>9.1f}%")
print("\n=== APPROVAL RATE BY SELLER TIER ===\n")
tier_approval = df.groupby("seller_tier")["approved"].mean() * 100
for tier in ["gold", "silver", "bronze"]:
print(f" {tier.UPPER():8s}: {tier_approval[tier]:.1f}%")
# Cell 9: Dashboard - Approval Overview and Error Count Distribution
fig, axes = plt.subplots(1, 3, figsize=(18, 5))
status_data = pd.DataFrame({
"Status": ["Approved", "Disapproved"],
"Count": [approved_count, disapproved_count]
})
colors = ["#2ecc71", "#e74c3c"]
axes[0].bar(status_data["Status"], status_data["Count"], color=colors, edgecolor="black", linewidth=0.5)
axes[0].set_title(f"Feed Approval Status\n(Rate: {approval_rate:.1f}%)", fontsize=12, fontweight="bold")
axes[0].set_ylabel("Number of Products")
axes[0].axhline(y=total * 0.9, color="orange", linestyle="--", label="90% Target")
axes[0].legend()
error_dist = df["error_count"].value_counts().sort_index()
axes[1].bar(error_dist.index, error_dist.values, color="#8e44ad", edgecolor="black", linewidth=0.5)
axes[1].set_title("Error Count Distribution\n(Errors per Product)", fontsize=12, fontweight="bold")
axes[1].set_xlabel("Number of Errors")
axes[1].set_ylabel("Number of Products")
axes[1].set_xticks(range(int(error_dist.index.max()) + 1))
impact_df = pd.DataFrame({
"Error": list(error_counts.keys()),
"Raw Count": list(error_counts.values()),
"Unique Impact": [unique_impacts[e] for e in error_counts.keys()]
}).sort_values("Unique Impact", ascending=True)
y_pos = range(len(impact_df))
axes[2].barh(y_pos, impact_df["Raw Count"].values, color="#bdc3c7", edgecolor="black", linewidth=0.5, label="Shared (overlap)")
axes[2].barh(y_pos, impact_df["Unique Impact"].values, color="#2980b9", edgecolor="black", linewidth=0.5, label="Unique Impact")
axes[2].set_yticks(y_pos)
axes[2].set_yticklabels(impact_df["Error"].values)
axes[2].set_title("Raw Count vs Unique Impact", fontsize=12, fontweight="bold")
axes[2].set_xlabel("Products")
axes[2].legend()
plt.tight_layout()
plt.show()
# Cell 10: Error Co-occurrence Heatmap
cooccurrence = pd.DataFrame(index=error_col_names.values(), columns=error_col_names.values(), dtype=float)
for col_a, name_a in error_col_names.items():
for col_b, name_b in error_col_names.items():
if col_a == col_b:
cooccurrence.loc[name_a, name_b] = 100.0
else:
has_error_a = ~df[col_a]
has_error_b = ~df[col_b]
if has_error_a.sum() > 0:
cooccurrence.loc[name_a, name_b] = (has_error_a & has_error_b).sum() / has_error_a.sum() * 100
else:
cooccurrence.loc[name_a, name_b] = 0.0
fig, ax = plt.subplots(figsize=(8, 6))
sns.heatmap(cooccurrence.astype(float), annot=True, fmt=".1f", cmap="YlOrRd",
ax=ax, linewidths=0.5, vmin=0, vmax=100)
ax.set_title("Error Co-occurrence Rate (%)\n(Of products with Row error, % that also have Column error)", fontsize=11, fontweight="bold")
plt.tight_layout()
plt.show()
# Cell 11: Category Breakdown
category_approval = df.groupby("google_product_category")["approved"].mean() * 100
category_approval = category_approval.sort_values(ascending=True)
fig, ax = plt.subplots(figsize=(12, 6))
category_approval_df = category_approval.reset_index()
category_approval_df.columns = ["Category", "Approval Rate"]
category_approval_df["Category_Short"] = category_approval_df["Category"].apply(lambda x: x.split(" > ")[1])
bars = ax.barh(
category_approval_df["Category_Short"],
category_approval_df["Approval Rate"],
color=["#e74c3c" if x < 60 else "#f39c12" if x < 80 else "#2ecc71" for x in category_approval_df["Approval Rate"]],
edgecolor="black",
linewidth=0.5
)
ax.axvline(x=90, color="black", linestyle="--", linewidth=1, label="90% Target")
ax.set_xlabel("Approval Rate (%)")
ax.set_title("Feed Approval Rate by Product Category", fontsize=13, fontweight="bold")
ax.set_xlim(0, 100)
ax.legend()
plt.tight_layout()
plt.show()
# Cell 12: Prioritized Fix Recommendations with Cumulative Improvement
print("=" * 70)
print(" PRIORITIZED FIX RECOMMENDATIONS (by Unique Impact)")
print("=" * 70)
print(f"\n Current Approval Rate: {approval_rate:.1f}%\n")
sorted_errors = sorted(unique_impacts.items(), key=lambda x: x[1], reverse=True)
recommendations = []
cumulative_improvement = approval_rate
improvement_curve = [approval_rate]
for priority, (error_name, unique_count) in enumerate(sorted_errors, 1):
raw = error_counts[error_name]
true_improvement = unique_count / total * 100
cumulative_improvement += true_improvement
improvement_curve.append(cumulative_improvement)
recommendations.append({
"Priority": priority,
"Error Type": error_name,
"Raw Count": raw,
"Unique Impact": unique_count,
"True Improvement": f"+{true_improvement:.1f}pp",
"Cumulative Rate": f"{cumulative_improvement:.1f}%"
})
rec_df = pd.DataFrame(recommendations)
print(rec_df.to_string(index=False))
print(f"\n{'=' * 70}")
print(" 'Unique Impact' = products that become approved by fixing ONLY this error.")
print(" This accounts for multi-error overlap, unlike naive raw-count prioritization.")
print(f"{'=' * 70}")
fig, ax = plt.subplots(figsize=(10, 5))
x_labels = ["Current"] + [r["Error Type"] for r in recommendations]
ax.plot(range(len(improvement_curve)), improvement_curve, marker='o', linewidth=2, color="#2980b9")
ax.axhline(y=90, color="orange", linestyle="--", label="90% Target")
ax.set_xticks(range(len(x_labels)))
ax.set_xticklabels(x_labels, rotation=30, ha="right")
ax.set_ylabel("Approval Rate (%)")
ax.set_title("Cumulative Approval Rate Improvement\n(Fixing Errors in Priority Order)", fontsize=12, fontweight="bold")
ax.legend()
ax.set_ylim(0, 100)
plt.tight_layout()
plt.show()Your dashboard now visualizes the full picture: approval rates, error distributions, co-occurrence patterns, category-level health, and a data-driven fix priority list. This is the kind of deliverable a feed data analyst hands to a product team to drive action.
Secret mission
Add a Title Optimization Scorer
Your validation engine catches broken titles, but it only flags pass or fail. What if you could score every title from 0-100 and generate specific improvement suggestions? Build a category-aware title scorer that grades titles against Google Shopping best practices and shows exactly how to fix the worst ones.
Clean Up Your Resources
Clean Up Your Resources
This project runs entirely in Google Colab, so there are no ongoing costs. Your notebook is saved in Google Drive and the runtime disconnects automatically after inactivity.
Decide whether to keep your notebook available, disconnect the runtime to free up resources, or delete everything entirely.
Resources you used:
- Google Colab notebook (stored in your Google Drive).
- Colab runtime (GPU/CPU compute session).
Keep everything running
No action needed. Choose this if you want to revisit your feed health analyzer or extend it further.
- Your notebook remains in Google Drive under the Colab Notebooks folder.
- The runtime will disconnect automatically after about 90 minutes of inactivity. You can reconnect anytime by reopening the notebook.
Pause - I'll come back to this later
Disconnect the runtime to free up compute resources. Your notebook code stays saved in Google Drive.
- In your Colab notebook, click Runtime in the top menu bar.
- Click Disconnect and delete runtime.
- This frees up the compute session. Your code and outputs remain saved in the notebook file.
Delete - I don't want to use this again
Remove the notebook entirely from your Google Drive.
- In your Colab notebook, click Runtime in the top menu bar, then click Disconnect and delete runtime.
- Open Google Drive at drive.google.com.
- Navigate to the Colab Notebooks folder.
- Right-click on your feed health analyzer notebook and select Move to trash.
- Empty your trash to permanently delete the file.
Nice Work!
Nice Work!
Nice work! You just built a complete feed health analyzer that validates 10,000 products against Google Merchant Center rules and surfaces the fixes that actually matter.
You've learned how to:
- Build a feed validation engine that checks the top 5 causes of Google Shopping disapprovals: missing identifiers, price issues, title violations, missing required attributes, and image problems.
- Analyze multi-error overlap to calculate unique impact for each error type, proving that naive raw-count prioritization is suboptimal.
- Produce a feed health dashboard with approval rate charts, error co-occurrence heatmaps, category breakdowns, and a cumulative improvement curve showing prioritized fix recommendations.
- Secret Mission: Built a category-aware title optimization scorer that grades every product title from 0-100 and generates specific improvement suggestions.
Ready to quiz yourself?