Build an AI Job Posting Analyzer

Extract structured data from job postings using Google Gemini and Python.

Introduction

30 Second Summary

Job postings are everywhere online, but they're messy blocks of unstructured text. Extracting useful patterns from hundreds of listings by hand would take hours.

In this project, you will build a Python pipeline that uses Google Gemini to automatically extract structured data from raw job postings. The pipeline turns messy text into clean, analyzable datasets complete with visualizations.

What You'll Build

A Python automation pipeline that reads raw job posting text from a CSV file, sends each posting to Google Gemini with a Pydantic-defined schema, and collects validated JSON responses into a pandas DataFrame for analysis and visualization.

By the end of this project, you'll have:

  • A working extraction pipeline that sends unstructured text to Google Gemini and receives perfectly structured responses validated against your schema.
  • A Pydantic data model that enforces the exact shape of AI output, covering skills, experience level, salary range, and work type.
  • Automated matplotlib visualizations revealing job market insights like top skills, salary ranges, and work type distribution.
  • Secret Mission: Add a second AI-powered analysis pass that scores each job's "AI-readiness" and generates a ranked recommendation list.

Are there any prerequisites?

Basic Python familiarity (variables, functions, loops) is helpful. All libraries and API setup are covered step by step in this project.

Set Up Your Project

Your packages are installed and your API key is ready to go. Now you need two things: data to feed the AI, and a schema that tells Gemini exactly what structure to return.

In this step, you'll create a project folder, add a CSV file containing sample job postings, and define a Pydantic model that acts as the extraction contract between your code and the AI.

In this step, get ready to:
  • Create a project folder with your two source files.
  • Add sample job posting data to a CSV file.
  • Define the Pydantic extraction schema that Gemini will follow.
Create your project folder

You need a dedicated folder to hold all the files for this pipeline. You'll create it on your Desktop so it's easy to find.

  • Open your terminal. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir job-posting-analyzer
cd job-posting-analyzer
  • Create the two files you'll need by running:
touch job_postings.csv models.py
  • Confirm the files exist by running:
ls
  • You should see job_postings.csv and models.py in the output.
  • Open the job-posting-analyzer folder in your code editor. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code (or Cursor) and press Enter to open it.
  • Click File in the top menu bar.
  • Click Open Folder.
  • Navigate to your Desktop, select the job-posting-analyzer folder, and click Open.

Don't see the folder on your Desktop?

Make sure you ran cd ~/Desktop before the mkdir command. If you skipped that step, the folder was created in your home directory instead.

Help me find my project folder

Add the sample job posting data

Your pipeline needs raw, unstructured job posting text to extract data from. This CSV file contains 8 realistic postings across different roles, experience levels, and work arrangements.

  • Open job_postings.csv in your editor's file explorer (the Explorer sidebar on the left).
  • Paste the following content into the file:
id,description
1,"Senior Data Analyst needed at TechFlow Inc. We're looking for someone with 5+ years of experience in SQL, Python, and Tableau. You'll be analyzing customer behavior data and building dashboards for leadership. Salary range $95,000-$120,000. This is a hybrid role based in Austin, TX with 2 days in office per week."
2,"Entry-level Machine Learning Engineer at NeuralPath AI. Join our growing team to build and deploy ML models for natural language processing. Requirements: Python, PyTorch, scikit-learn, and basic understanding of transformer architectures. Bachelor's degree in CS or related field. Remote-first company. Salary: $75,000-$95,000."
3,"Mid-level Software Engineer at CloudScale Systems. We need someone experienced in Go, Kubernetes, and AWS to help build our distributed data platform. 3-5 years of experience required. Strong understanding of microservices architecture. Competitive salary $130,000-$160,000. Fully remote, US-based."
4,"Lead Data Scientist at HealthMetrics Corp. You'll lead a team of 4 data scientists building predictive models for patient outcomes. Must have 8+ years experience with Python, R, TensorFlow, and strong statistics background. Experience with healthcare data (HIPAA compliance) required. Onsite in Boston. $165,000-$195,000."
5,"Junior Data Analyst at GreenLeaf Analytics. Perfect for recent graduates! You'll support the analytics team with data cleaning, basic statistical analysis, and report generation. Skills needed: Excel, SQL, basic Python. We'll train you on our tools. $55,000-$70,000. Hybrid role, 3 days in NYC office."
6,"Senior ML Ops Engineer at DataForge. Build and maintain our ML pipeline infrastructure. Deep experience with Docker, Kubernetes, MLflow, and CI/CD pipelines required. Must be comfortable with both Python and Terraform. 6+ years experience. Fully remote. $150,000-$180,000."
7,"Data Engineer at StreamData Co. Design and build real-time data pipelines using Apache Kafka, Spark, and Airflow. Experience with data warehousing (Snowflake or BigQuery) essential. 4+ years experience. Python and SQL required. Hybrid role in Seattle. $125,000-$155,000."
8,"AI Product Manager at InnovateTech. Lead the development of our AI-powered recommendation engine. Requires understanding of ML concepts, experience with product roadmaps, and strong communication skills. 5+ years in product management, 2+ years working with AI/ML teams. Remote-first. $140,000-$175,000."

What does this data look like?

Each row has two columns: an id and a description. The descriptions are messy, unstructured paragraphs. They mention salaries in different formats, list skills inconsistently, and describe work arrangements in varied ways.

This is exactly the kind of real-world text that's painful to parse with regex but trivial for an LLM to understand.

  • Save job_postings.csv.
  • Verify the file loads correctly by running this command in your terminal:
python -c "import pandas as pd; df = pd.read_csv('job_postings.csv'); print(f'{len(df)} postings loaded')"
  • You should see 8 postings loaded printed in the terminal.

Seeing a FileNotFoundError?

Make sure your terminal is inside the job-posting-analyzer folder. Run pwd to check your current directory. If you're in the wrong place, run cd ~/Desktop/job-posting-analyzer to get back.

Help me fix my FileNotFoundError

✔️ Awesome, I've got everything!

Great. Make sure you've saved job_postings.csv before moving on.

ⓧ I'd like to double check the full code

Here's the complete job_postings.csv file for comparison:

id,description
1,"Senior Data Analyst needed at TechFlow Inc. We're looking for someone with 5+ years of experience in SQL, Python, and Tableau. You'll be analyzing customer behavior data and building dashboards for leadership. Salary range $95,000-$120,000. This is a hybrid role based in Austin, TX with 2 days in office per week."
2,"Entry-level Machine Learning Engineer at NeuralPath AI. Join our growing team to build and deploy ML models for natural language processing. Requirements: Python, PyTorch, scikit-learn, and basic understanding of transformer architectures. Bachelor's degree in CS or related field. Remote-first company. Salary: $75,000-$95,000."
3,"Mid-level Software Engineer at CloudScale Systems. We need someone experienced in Go, Kubernetes, and AWS to help build our distributed data platform. 3-5 years of experience required. Strong understanding of microservices architecture. Competitive salary $130,000-$160,000. Fully remote, US-based."
4,"Lead Data Scientist at HealthMetrics Corp. You'll lead a team of 4 data scientists building predictive models for patient outcomes. Must have 8+ years experience with Python, R, TensorFlow, and strong statistics background. Experience with healthcare data (HIPAA compliance) required. Onsite in Boston. $165,000-$195,000."
5,"Junior Data Analyst at GreenLeaf Analytics. Perfect for recent graduates! You'll support the analytics team with data cleaning, basic statistical analysis, and report generation. Skills needed: Excel, SQL, basic Python. We'll train you on our tools. $55,000-$70,000. Hybrid role, 3 days in NYC office."
6,"Senior ML Ops Engineer at DataForge. Build and maintain our ML pipeline infrastructure. Deep experience with Docker, Kubernetes, MLflow, and CI/CD pipelines required. Must be comfortable with both Python and Terraform. 6+ years experience. Fully remote. $150,000-$180,000."
7,"Data Engineer at StreamData Co. Design and build real-time data pipelines using Apache Kafka, Spark, and Airflow. Experience with data warehousing (Snowflake or BigQuery) essential. 4+ years experience. Python and SQL required. Hybrid role in Seattle. $125,000-$155,000."
8,"AI Product Manager at InnovateTech. Lead the development of our AI-powered recommendation engine. Requires understanding of ML concepts, experience with product roadmaps, and strong communication skills. 5+ years in product management, 2+ years working with AI/ML teams. Remote-first. $140,000-$175,000."
Define the Pydantic extraction schema

Pydantic models define the exact shape of the data you want Gemini to return. When you pass a Pydantic schema to Gemini's structured output feature, the API guarantees valid JSON conforming to your schema. No regex. No JSON repair.

Why Pydantic instead of a plain dictionary?

Pydantic generates a JSON Schema from your Python class. This schema includes field types, allowed values, and descriptions that guide the model's output.

The descriptions you put on each field act as mini-prompts. They tell Gemini what belongs in each field without cluttering your main prompt.

  • Open models.py in your editor.
  • Add the imports and the JobPosting model by pasting this code:
from pydantic import BaseModel, Field
from typing import List, Optional, Literal


class JobPosting(BaseModel):
    """Schema for structured job posting extraction."""

    job_title: str = Field(description="The job title or role name.")
    company_name: str = Field(description="The name of the hiring company.")
    required_skills: List[str] = Field(
        description="List of required technical skills and tools mentioned."
    )
    experience_level: Literal["entry", "mid", "senior", "lead"] = Field(
        description="The seniority level: entry, mid, senior, or lead."
    )
    salary_min: Optional[int] = Field(
        description="Minimum salary in USD, or null if not mentioned."
    )
    salary_max: Optional[int] = Field(
        description="Maximum salary in USD, or null if not mentioned."
    )
    work_type: Literal["remote", "hybrid", "onsite"] = Field(
        description="The work arrangement: remote, hybrid, or onsite."
    )

What does this model do?

Each field defines one piece of structured data you want extracted from the raw posting text:

  • Literal types constrain the AI to only return values from your allowed list (e.g. "remote", "hybrid", or "onsite").
  • Optional fields let the model return null when salary info is missing from a posting.
  • The description on each Field becomes part of the JSON Schema, guiding Gemini on what to extract.
  • Now add the AIReadinessScore model below the JobPosting class. Paste this code at the bottom of models.py:


class AIReadinessScore(BaseModel):
    """Schema for AI readiness scoring of a job posting."""

    score: int = Field(
        description="AI readiness score from 1 (not AI related) to 10 (deeply AI focused)."
    )
    reasoning: str = Field(
        description="Brief explanation of why this score was given."
    )
    ai_skills_mentioned: List[str] = Field(
        description="List of AI/ML specific skills or concepts mentioned in the posting."
    )

What is this second model for?

The AIReadinessScore model defines a second extraction schema you'll use later to score how AI-focused each job is. You're defining it now so all your schemas live in one file.

  • Save models.py.
  • Verify your schema generates valid JSON Schema by running this in your terminal:
python -c "from models import JobPosting; import json; print(json.dumps(JobPosting.model_json_schema(), indent=2))"
  • You should see a JSON object printed with properties, required, and title keys. This is the exact schema that Gemini will use to structure its output.

Seeing a ModuleNotFoundError?

If you see an error about pydantic not being found, make sure you installed it in Step 1. Run pip install pydantic to install it.

If you see an error about models not being found, make sure your terminal is inside the job-posting-analyzer folder where models.py lives.

Help me fix my ModuleNotFoundError

✔️ Awesome, I've got everything!

Great. Double check you've saved models.py before moving on.

ⓧ I'd like to double check the full code

Here's the complete models.py file for comparison:

from pydantic import BaseModel, Field
from typing import List, Optional, Literal


class JobPosting(BaseModel):
    """Schema for structured job posting extraction."""

    job_title: str = Field(description="The job title or role name.")
    company_name: str = Field(description="The name of the hiring company.")
    required_skills: List[str] = Field(
        description="List of required technical skills and tools mentioned."
    )
    experience_level: Literal["entry", "mid", "senior", "lead"] = Field(
        description="The seniority level: entry, mid, senior, or lead."
    )
    salary_min: Optional[int] = Field(
        description="Minimum salary in USD, or null if not mentioned."
    )
    salary_max: Optional[int] = Field(
        description="Maximum salary in USD, or null if not mentioned."
    )
    work_type: Literal["remote", "hybrid", "onsite"] = Field(
        description="The work arrangement: remote, hybrid, or onsite."
    )


class AIReadinessScore(BaseModel):
    """Schema for AI readiness scoring of a job posting."""

    score: int = Field(
        description="AI readiness score from 1 (not AI related) to 10 (deeply AI focused)."
    )
    reasoning: str = Field(
        description="Brief explanation of why this score was given."
    )
    ai_skills_mentioned: List[str] = Field(
        description="List of AI/ML specific skills or concepts mentioned in the posting."
    )

Your data and schema are ready. Next up, you'll build the extraction pipeline that sends each job posting through Gemini and collects perfectly structured results.

Build the Extraction Pipeline

You have your sample data and your Pydantic schema ready. Now it is time to build the automation that connects them through Google Gemini.

In this step, you will create the core Python script that reads each job posting from your CSV, sends it to the Gemini API with your schema as a contract, and collects perfectly structured JSON responses. The result is a repeatable pipeline that turns messy text into clean data.

In this step, get ready to:
  • Write the extraction function that calls Gemini with your Pydantic schema.
  • Build a processing loop with error handling and rate limiting.
  • Run the full pipeline and see structured results printed to the console.
Create the pipeline file and load function

The pipeline needs to read your CSV, connect to Gemini, and extract structured data from each posting. You will build this piece by piece, starting with the file setup and a helper to load the data.

  • Create a new file called pipeline.py in your project folder (the same folder where models.py and job_postings.csv live).
  • Add the imports and the CSV loading function by pasting this code:
import os
import time
import pandas as pd
import matplotlib.pyplot as plt
from google import genai
from models import JobPosting, AIReadinessScore


def load_job_postings(csv_path: str) -> pd.DataFrame:
    """Load job postings from CSV file."""
    return pd.read_csv(csv_path)

What does this code do?

The imports bring in everything the pipeline needs: os to read your environment variable, time for rate limiting pauses, pandas for data handling, and the genai client to talk to Gemini.

The load_job_postings() function is a thin wrapper around pd.read_csv(). Wrapping it in a function keeps the main logic clean and makes it easy to swap data sources later.

  • Save pipeline.py.

You will not see any output yet. The next chunk adds the core extraction logic.

Write the Gemini extraction function

This is where the magic happens. You will write a function that sends a single job posting to Gemini along with your JobPosting schema, and gets back validated structured data.

  • In pipeline.py, add the following function below load_job_postings():
def extract_job_data(client: genai.Client, description: str) -> JobPosting:
    """Extract structured data from a single job posting using Gemini."""
    prompt = f"""Extract the structured job posting information from the following text.
If salary information is not explicitly mentioned, set salary_min and salary_max to null.

Job Posting:
{description}"""

    response = client.models.generate_content(
        model="gemini-3.5-flash",
        contents=prompt,
        config={
            "response_format": {
                "text": {
                    "mime_type": "application/json",
                    "schema": JobPosting.model_json_schema(),
                }
            }
        },
    )
    return JobPosting.model_validate_json(response.text)

What does this code do?

The extract_job_data() function does three things in sequence:

  • It builds a prompt telling Gemini what to extract, injecting the raw job description text.
  • It calls client.models.generate_content() with the response_format config. This tells Gemini to return JSON that conforms exactly to your JobPosting schema.
  • It validates the response using JobPosting.model_validate_json(response.text). If Gemini returns malformed data, Pydantic raises an error instead of letting bad data slip through.

Why structured output instead of raw prompting?

Without structured output, you would need to parse free-text JSON, handle missing fields, and write retry logic for malformed responses. Gemini's structured output guarantees valid JSON conforming to your schema on every call.

This is the production pattern data teams use. No regex parsing, no JSON repair loops.

  • Save pipeline.py.

You will not see output yet. The next chunk builds the loop that feeds all 8 postings through this function.

Add the processing loop with error handling

A single extraction call is useful, but you need to process all 8 postings reliably. This means handling API errors gracefully and respecting the free tier rate limit of 10 requests per minute.

  • In pipeline.py, add the process_all_postings() function below extract_job_data():
def process_all_postings(client: genai.Client, df: pd.DataFrame) -> list[dict]:
    """Process all job postings through the extraction pipeline."""
    results = []
    total = len(df)

    for index, row in df.iterrows():
        print(f"Processing posting {index + 1}/{total}...")
        try:
            job_data = extract_job_data(client, row["description"])
            results.append(job_data.model_dump())
        except Exception as e:
            print(f"  Error processing posting {row['id']}: {e}")
            continue
        time.sleep(4)

    return results

What does this code do?

  • The loop iterates through every row in the DataFrame and passes the description column to extract_job_data().
  • The try/except block catches any API or validation errors. If one posting fails, the loop prints the error and continues to the next one instead of crashing.
  • Each successful result is converted to a plain dictionary with .model_dump() and appended to the results list.
  • The time.sleep(4) pauses 4 seconds between calls. With 8 postings, that is about 32 seconds total. This keeps you safely under the free tier rate limit.
  • Save pipeline.py.

You still cannot run the pipeline because there is no entry point yet. The next chunk adds the main() function that ties everything together.

  • Add the main() function and entry point at the bottom of pipeline.py:
def main():
    """Main pipeline execution."""
    api_key = os.environ.get("GEMINI_API_KEY")
    if not api_key:
        print("Error: GEMINI_API_KEY environment variable not set.")
        return

    client = genai.Client(api_key=api_key)

    print("Loading job postings...")
    postings_df = load_job_postings("job_postings.csv")
    print(f"Found {len(postings_df)} postings.\n")

    print("Starting AI extraction pipeline...")
    print("-" * 40)
    results = process_all_postings(client, postings_df)
    print("-" * 40)
    print(f"\nSuccessfully extracted {len(results)}/{len(postings_df)} postings.\n")

    results_df = pd.DataFrame(results)
    print("Extracted Data:")
    print(results_df[["job_title", "company_name", "experience_level", "work_type"]].to_string())
    print()


if __name__ == "__main__":
    main()

What does this code do?

  • The main() function first checks that your GEMINI_API_KEY environment variable is set. If it is missing, the script exits with a clear error message.
  • It creates a genai.Client using your API key, loads the CSV, and feeds all postings through the extraction pipeline.
  • After processing, it builds a pandas DataFrame from the results and prints a summary table showing the job title, company, experience level, and work type for each posting.
  • The if __name__ == "__main__" block ensures main() only runs when you execute the file directly.
  • Save pipeline.py.
  • Run the pipeline by executing this command in your terminal:
python pipeline.py

The script takes about 30-40 seconds to finish because of the rate limiting pauses between calls. You should see progress messages for each posting, followed by a table of extracted data.

What should I see?

Your terminal prints something like this:

Loading job postings...
Found 8 postings.

Starting AI extraction pipeline...
----------------------------------------
Processing posting 1/8...
Processing posting 2/8...
...
----------------------------------------

Successfully extracted 8/8 postings.

Extracted Data:
                    job_title      company_name experience_level work_type
0        Senior Data Analyst      TechFlow Inc.           senior    hybrid
1  Machine Learning Engineer     NeuralPath AI            entry    remote
...

The exact values in the table may vary slightly because Gemini interprets each posting. The important thing is that all 8 postings are successfully extracted.

Seeing an error about GEMINI_API_KEY?

If you see the message Error: GEMINI_API_KEY environment variable not set., your environment variable is not loaded. Make sure you restarted your terminal or editor after setting it in the previous step.

If you see a rate limit or quota error, wait 60 seconds and try again. The free tier resets quickly.

Help me debug my pipeline.py errors.

✔️ Awesome, I've got everything!

Great. Double check you have saved pipeline.py and that your output shows all 8 postings successfully extracted.

ⓧ I'd like to double check the full code

Here is the complete pipeline.py file at this point:

import os
import time
import pandas as pd
import matplotlib.pyplot as plt
from google import genai
from models import JobPosting, AIReadinessScore


def load_job_postings(csv_path: str) -> pd.DataFrame:
    """Load job postings from CSV file."""
    return pd.read_csv(csv_path)


def extract_job_data(client: genai.Client, description: str) -> JobPosting:
    """Extract structured data from a single job posting using Gemini."""
    prompt = f"""Extract the structured job posting information from the following text.
If salary information is not explicitly mentioned, set salary_min and salary_max to null.

Job Posting:
{description}"""

    response = client.models.generate_content(
        model="gemini-3.5-flash",
        contents=prompt,
        config={
            "response_format": {
                "text": {
                    "mime_type": "application/json",
                    "schema": JobPosting.model_json_schema(),
                }
            }
        },
    )
    return JobPosting.model_validate_json(response.text)


def process_all_postings(client: genai.Client, df: pd.DataFrame) -> list[dict]:
    """Process all job postings through the extraction pipeline."""
    results = []
    total = len(df)

    for index, row in df.iterrows():
        print(f"Processing posting {index + 1}/{total}...")
        try:
            job_data = extract_job_data(client, row["description"])
            results.append(job_data.model_dump())
        except Exception as e:
            print(f"  Error processing posting {row['id']}: {e}")
            continue
        time.sleep(4)

    return results


def main():
    """Main pipeline execution."""
    api_key = os.environ.get("GEMINI_API_KEY")
    if not api_key:
        print("Error: GEMINI_API_KEY environment variable not set.")
        return

    client = genai.Client(api_key=api_key)

    print("Loading job postings...")
    postings_df = load_job_postings("job_postings.csv")
    print(f"Found {len(postings_df)} postings.\n")

    print("Starting AI extraction pipeline...")
    print("-" * 40)
    results = process_all_postings(client, postings_df)
    print("-" * 40)
    print(f"\nSuccessfully extracted {len(results)}/{len(postings_df)} postings.\n")

    results_df = pd.DataFrame(results)
    print("Extracted Data:")
    print(results_df[["job_title", "company_name", "experience_level", "work_type"]].to_string())
    print()


if __name__ == "__main__":
    main()

Your extraction pipeline is working. You now have structured, validated job posting data sitting in a DataFrame. Next up, you will turn that data into visualizations that reveal job market insights at a glance.

Analyze and Visualize the Results

Your pipeline can extract structured data from raw job postings. That's powerful, but a list of dictionaries sitting in memory doesn't reveal patterns.

In this step, you'll use pandas and matplotlib to transform your extracted data into visual insights. You'll see which skills are most in-demand, how salaries vary by experience level, and how work types are distributed.

In this step, get ready to:
  • Build a visualization function that counts skill frequency across all postings.
  • Create three charts: a skills bar chart, a salary-by-level chart, and a work type pie chart.
  • Run the full pipeline end-to-end and generate PNG files.
Add the create_visualizations function

Each of your 8 extracted job postings contains a required_skills list. To find which skills appear most often, you need to "explode" those lists into individual rows so pandas can count them. The same DataFrame also holds salary and work type data ready for charting.

  • In pipeline.py, find the process_all_postings function. Add the following new function directly below it (above def main():):
def create_visualizations(results_df: pd.DataFrame) -> None:
    """Create and save matplotlib visualizations."""
    # 1. Top skills bar chart
    all_skills = results_df["required_skills"].explode()
    skill_counts = all_skills.value_counts().head(10)

    fig, ax = plt.subplots(figsize=(10, 6))
    skill_counts.plot(kind="barh", ax=ax, color="steelblue")
    ax.set_xlabel("Number of Postings")
    ax.set_title("Top 10 Most Requested Skills")
    ax.invert_yaxis()
    plt.tight_layout()
    plt.savefig("top_skills.png", dpi=150)
    plt.close()
    print("Saved: top_skills.png")

    # 2. Salary by experience level
    salary_data = results_df.dropna(subset=["salary_min", "salary_max"])
    if not salary_data.empty:
        salary_avg = salary_data.groupby("experience_level")[
            ["salary_min", "salary_max"]
        ].mean()

        fig, ax = plt.subplots(figsize=(8, 5))
        salary_avg.plot(kind="bar", ax=ax, color=["#2196F3", "#4CAF50"])
        ax.set_ylabel("Salary (USD)")
        ax.set_title("Average Salary Range by Experience Level")
        ax.set_xticklabels(ax.get_xticklabels(), rotation=45)
        ax.legend(["Min Salary", "Max Salary"])
        plt.tight_layout()
        plt.savefig("salary_by_level.png", dpi=150)
        plt.close()
        print("Saved: salary_by_level.png")

    # 3. Work type distribution
    work_counts = results_df["work_type"].value_counts()

    fig, ax = plt.subplots(figsize=(7, 7))
    work_counts.plot(
        kind="pie", ax=ax, autopct="%1.1f%%", colors=["#66BB6A", "#42A5F5", "#FFA726"]
    )
    ax.set_ylabel("")
    ax.set_title("Work Type Distribution")
    plt.tight_layout()
    plt.savefig("work_type.png", dpi=150)
    plt.close()
    print("Saved: work_type.png")

What does this code do?

This function creates three distinct visualizations from your extracted data:

  • Top skills chart: explode() takes each list of skills and turns it into separate rows. value_counts().head(10) then counts how many postings mention each skill and keeps the top 10.
  • Salary chart: Groups postings by experience level and calculates the average min/max salary for each group. The dropna() call filters out any postings where salary data wasn't available.
  • Work type chart: Counts how many postings fall into each work arrangement (remote, hybrid, onsite) and displays the distribution as a pie chart with percentages.
  • Save pipeline.py.

You won't see a change yet. The next substep connects this function to your main pipeline.

Seeing a red underline on the function?

If your editor shows a warning on create_visualizations, make sure you placed it ABOVE the def main(): line. Python reads files top-to-bottom, so functions must be defined before they're called.

help me fix a function placement error in pipeline.py

Update main() to generate charts

Your main() function currently prints the extracted DataFrame and stops. You need to call your new visualization function so the charts are generated when the pipeline runs.

  • In pipeline.py, scroll down to the main() function.
  • Find this line near the bottom:
    print()
  • Add the following lines directly below that print() line, still inside main():
    print("Generating visualizations...")
    create_visualizations(results_df)
    print("\nPipeline complete!")

What does this addition do?

These three lines pass your results DataFrame into the visualization function. When the pipeline finishes extracting data, it now automatically generates all three charts and prints a completion message.

  • Save pipeline.py.
Run the complete pipeline

Everything is wired up. Running the pipeline now will extract data from all 8 postings AND generate your visualizations as PNG files.

  • Run the full pipeline by executing this command in your terminal:
python pipeline.py

What should I see?

The pipeline will process all 8 postings (taking about 32 seconds due to the rate-limiting sleep between calls). You should see progress messages for each posting, then a summary table, followed by three "Saved:" confirmations for each chart file.

The final line should read Pipeline complete!.

  • Confirm the PNG files were created by running:
ls *.png
  • You should see top_skills.png, salary_by_level.png, and work_type.png listed in the output.
  • Open any of the PNG files to view your chart. You'll see which skills dominate the job market and how salaries shift by experience level.

Pipeline failing with an API error?

If you see an error mentioning rate limits, the 4-second sleep between calls may not be enough for your current usage. Wait a minute and try again.

If you see an error about the environment variable, make sure you restarted your terminal after setting GEMINI_API_KEY. The variable only loads when a new terminal session starts.

help me debug my pipeline.py API errors

No PNG files appearing?

Check that your terminal is in the same folder as pipeline.py. The PNG files are saved to the current working directory.

Also check for indentation errors. The three new lines you added must be indented at the same level as the other code inside main() (4 spaces).

help me figure out why my charts aren't saving

✔️ Awesome, I've got everything!

Great. Double check you've saved pipeline.py and that all three PNG files exist in your project folder.

ⓧ I'd like to double check the full code

Here is the full pipeline.py as it should look at this point:

import os
import time
import pandas as pd
import matplotlib.pyplot as plt
from google import genai
from models import JobPosting, AIReadinessScore


def load_job_postings(csv_path: str) -> pd.DataFrame:
    """Load job postings from CSV file."""
    return pd.read_csv(csv_path)


def extract_job_data(client: genai.Client, description: str) -> JobPosting:
    """Extract structured data from a single job posting using Gemini."""
    prompt = f"""Extract the structured job posting information from the following text.
If salary information is not explicitly mentioned, set salary_min and salary_max to null.

Job Posting:
{description}"""

    response = client.models.generate_content(
        model="gemini-3.5-flash",
        contents=prompt,
        config={
            "response_format": {
                "text": {
                    "mime_type": "application/json",
                    "schema": JobPosting.model_json_schema(),
                }
            }
        },
    )
    return JobPosting.model_validate_json(response.text)


def process_all_postings(client: genai.Client, df: pd.DataFrame) -> list[dict]:
    """Process all job postings through the extraction pipeline."""
    results = []
    total = len(df)

    for index, row in df.iterrows():
        print(f"Processing posting {index + 1}/{total}...")
        try:
            job_data = extract_job_data(client, row["description"])
            results.append(job_data.model_dump())
        except Exception as e:
            print(f"  Error processing posting {row['id']}: {e}")
            continue
        time.sleep(4)

    return results


def create_visualizations(results_df: pd.DataFrame) -> None:
    """Create and save matplotlib visualizations."""
    # 1. Top skills bar chart
    all_skills = results_df["required_skills"].explode()
    skill_counts = all_skills.value_counts().head(10)

    fig, ax = plt.subplots(figsize=(10, 6))
    skill_counts.plot(kind="barh", ax=ax, color="steelblue")
    ax.set_xlabel("Number of Postings")
    ax.set_title("Top 10 Most Requested Skills")
    ax.invert_yaxis()
    plt.tight_layout()
    plt.savefig("top_skills.png", dpi=150)
    plt.close()
    print("Saved: top_skills.png")

    # 2. Salary by experience level
    salary_data = results_df.dropna(subset=["salary_min", "salary_max"])
    if not salary_data.empty:
        salary_avg = salary_data.groupby("experience_level")[
            ["salary_min", "salary_max"]
        ].mean()

        fig, ax = plt.subplots(figsize=(8, 5))
        salary_avg.plot(kind="bar", ax=ax, color=["#2196F3", "#4CAF50"])
        ax.set_ylabel("Salary (USD)")
        ax.set_title("Average Salary Range by Experience Level")
        ax.set_xticklabels(ax.get_xticklabels(), rotation=45)
        ax.legend(["Min Salary", "Max Salary"])
        plt.tight_layout()
        plt.savefig("salary_by_level.png", dpi=150)
        plt.close()
        print("Saved: salary_by_level.png")

    # 3. Work type distribution
    work_counts = results_df["work_type"].value_counts()

    fig, ax = plt.subplots(figsize=(7, 7))
    work_counts.plot(
        kind="pie", ax=ax, autopct="%1.1f%%", colors=["#66BB6A", "#42A5F5", "#FFA726"]
    )
    ax.set_ylabel("")
    ax.set_title("Work Type Distribution")
    plt.tight_layout()
    plt.savefig("work_type.png", dpi=150)
    plt.close()
    print("Saved: work_type.png")


def main():
    """Main pipeline execution."""
    api_key = os.environ.get("GEMINI_API_KEY")
    if not api_key:
        print("Error: GEMINI_API_KEY environment variable not set.")
        return

    client = genai.Client(api_key=api_key)

    print("Loading job postings...")
    postings_df = load_job_postings("job_postings.csv")
    print(f"Found {len(postings_df)} postings.\n")

    print("Starting AI extraction pipeline...")
    print("-" * 40)
    results = process_all_postings(client, postings_df)
    print("-" * 40)
    print(f"\nSuccessfully extracted {len(results)}/{len(postings_df)} postings.\n")

    results_df = pd.DataFrame(results)
    print("Extracted Data:")
    print(results_df[["job_title", "company_name", "experience_level", "work_type"]].to_string())
    print()

    print("Generating visualizations...")
    create_visualizations(results_df)
    print("\nPipeline complete!")


if __name__ == "__main__":
    main()

You now have a complete AI extraction and visualization pipeline. Your charts reveal real patterns: which skills employers want most, how much each experience level earns, and whether the market favors remote or hybrid work. Next up, you'll extend this pipeline with a Secret Mission that scores each job's AI-readiness.

Secret mission

AI Readiness Scorer

Your pipeline already extracts structured data from job postings. But what if you could add a second AI-powered pass that scores how AI-focused each role is and generates a ranked recommendation list? Chain a second Gemini call for each posting and turn your pipeline into a multi-pass analysis engine.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs locally and uses the Gemini free tier, so there are no ongoing costs.

Resources you used:

  • Project files (pipeline.py, models.py, job_postings.csv, and generated PNG charts).
  • Python packages (google-genai, pandas, matplotlib, pydantic).
  • GEMINI_API_KEY environment variable.
  • Gemini API key in Google AI Studio.

Keep everything running

No action needed. Choose this if you're still actively building or plan to extend the pipeline with more job postings or additional analysis passes.

  • Your project files stay on disk, ready to rerun anytime.
  • Your GEMINI_API_KEY environment variable remains set for future Gemini projects.
  • Your installed Python packages remain available for other scripts.

Pause - I'll come back to this later

Since this project runs locally with no cloud resources, there is nothing to pause. Your files and packages stay on your machine at no cost until you return.

  • Your project files, packages, and environment variable all persist with zero ongoing cost.
  • When you come back, run python pipeline.py from your project folder to pick up where you left off.

Delete - I don't want to use this again

Remove all project files, uninstall packages, and revoke your API key.

  • Delete all project files by running this command from the folder containing your project:

macOS

rm pipeline.py models.py job_postings.csv top_skills.png salary_by_level.png work_type.png ai_readiness.png

Windows

del pipeline.py, models.py, job_postings.csv, top_skills.png, salary_by_level.png, work_type.png, ai_readiness.png
  • Uninstall the Python packages by running:
pip uninstall google-genai pandas matplotlib pydantic -y
  • Remove the GEMINI_API_KEY environment variable. Open Windows Environment Variables settings, find GEMINI_API_KEY under User variables, select it, and click Delete.
  • Delete your API key from Google AI Studio. Go to aistudio.google.com, click Get API key in the left sidebar, find your key in the list, click the trash icon next to it, and confirm deletion.

Nice Work!

Nice Work!

You just built a complete AI-powered data extraction pipeline that turns unstructured job postings into clean, structured data and actionable visualizations.

You've learned how to:

  • Define Pydantic schemas that enforce structured output from an LLM, eliminating the need for regex parsing or JSON repair.
  • Build an automated extraction pipeline using Google Gemini's structured output API with rate limiting and error handling.
  • Analyze extracted data with pandas and create publication-quality visualizations with matplotlib.
  • Secret Mission: Chained a second LLM pass to score each job's AI readiness and generate a ranked recommendation list.

Ready to quiz yourself?