Build a Workforce Upskilling Agent
Build a 3-agent AI crew that analyzes skill gaps and generates upskilling plans.
Introduction
30 Second Summary
Every team has skill gaps. The challenge is figuring out exactly which skills are falling behind and what each person should learn next.
In this project, you will build "SkillGap," a multi-agent AI system that autonomously analyzes a team's skills against job market trends and generates personalized upskilling recommendations.
What You'll Build
A crew of 3 AI agents powered by Groq's free Llama 3.3 70B model that work together to identify workforce skill gaps and produce a professional upskilling report.
By the end of this project, you'll have:
- A working CrewAI project with 3 specialized agents (Market Researcher, Gap Analyst, Learning Advisor) orchestrated in sequence.
- A custom TF-IDF skill matching tool that agents use autonomously to quantify how close each team member's skills are to emerging market requirements.
- A personalized upskilling report generated from a team CSV input using agentic reasoning.
- Secret Mission: Add a skills heatmap visualization showing gap severity across the entire team.
Are there any prerequisites?
You should have basic Python experience and VS Code installed. You'll also need a free Groq account (no credit card required). Step 1 walks you through installing everything else.
Before We Start
Before We Start...
Install Python 3.13, uv, and CrewAI
Before you can build your multi-agent system, you need the right tools installed. CrewAI is the framework that orchestrates your AI agents, and it requires Python 3.13 specifically (not 3.14).
In this step, you will install Python, the uv package manager, and the CrewAI CLI. Then you will scaffold your project directory so everything is ready for building agents.
In this step, get ready to:
- Install Python 3.13 and verify it is on your PATH.
- Install the uv package manager and CrewAI CLI.
- Scaffold the SkillGap project and add dependencies.
Install Python 3.13
CrewAI requires Python >=3.10 and <3.14. The latest compatible version is 3.13.14.
- Open your terminal. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
- Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.
- Check whether you already have the right version by running this command:
macOS
python3 --versionWindows
python --version✔️ I see Python 3.13.x
You are good to go. Skip ahead to the next substep.
ⓧ I see an older version
You have Python installed, but it is the wrong version for CrewAI. You need to install Python 3.13.14 alongside your existing version.
- Open the Python 3.13.14 release page:
- https://www.python.org/downloads/release/python-31314/
macOS
- Scroll down to the files table and click Download macOS installer to download the .pkg file.
- Double-click the downloaded file and follow the installer prompts.
- Close and reopen your terminal, then verify by running:
python3 --versionYou should see Python 3.13.14 in the output.
Windows
- Scroll down to the files table and click the Windows installer (64-bit) download link.
- Run the downloaded .exe file.
- On the first installer screen, check Add Python to PATH before clicking Install Now.
- Close and reopen your terminal, then verify by running:
python --versionYou should see Python 3.13.14 in the output.
Still seeing the old version?
On macOS, the system Python may shadow your new install. Try running python3.13 --version instead. If that works, use python3.13 for all commands in this project.
On Windows, make sure you checked Add Python to PATH during installation. If you missed it, re-run the installer and select Modify to enable the PATH option.
Help me fix my Python version for CrewAI
ⓧ Command not found
Python is not installed on your machine yet. You need to install Python 3.13.14.
- Open the Python 3.13.14 release page:
- https://www.python.org/downloads/release/python-31314/
macOS
- Scroll down to the files table and click Download macOS installer to download the .pkg file.
- Double-click the downloaded file and follow the installer prompts.
- Close and reopen your terminal, then verify by running:
python3 --versionYou should see Python 3.13.14 in the output.
Windows
- Scroll down to the files table and click the Windows installer (64-bit) download link.
- Run the downloaded .exe file.
- On the first installer screen, check Add Python to PATH before clicking Install Now.
- Close and reopen your terminal, then verify by running:
python --versionYou should see Python 3.13.14 in the output.
Still not working?
Make sure you fully closed and reopened your terminal after installing. A fresh terminal session picks up the new PATH entries.
Help me install Python 3.13 on my machine
Why Python 3.13, not 3.14?
CrewAI explicitly requires Python >=3.10 and <3.14. If you use Python 3.14, dependency installation will fail with version-incompatibility errors.
Install uv and the CrewAI CLI
CrewAI uses uv as its package manager. You will install uv first, then use it to install the CrewAI command-line tool.
- Install uv by running the installer for your operating system:
macOS
curl -LsSf https://astral.sh/uv/install.sh | shWindows
powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"What does this command do?
This downloads a small installer script from Astral (the company behind uv) and runs it. The script places the uv binary on your PATH so you can use it from any terminal session.
- If the installer shows a PATH warning, refresh your shell by running:
uv tool update-shell- Close and reopen your terminal so the PATH changes take effect.
- Install the CrewAI CLI by running:
uv tool install crewai- Verify the installation by running:
uv tool listYou should see crewai listed in the output.
Don't see crewai in the list?
Make sure you reopened your terminal after the uv installation. If uv itself is not found, the PATH was not updated. Run uv tool update-shell again and restart your terminal.
Help me get the CrewAI CLI working with uv
Create and scaffold the SkillGap project
CrewAI includes a scaffolding command that generates the project structure for you. This creates config files, a crew module, and a main entry point.
- Move to your Desktop by running this command:
cd ~/Desktop- Scaffold your project by running:
crewai create crew skillgapWhat does this command do?
This generates a skillgap folder on your Desktop with the standard CrewAI project layout. Inside you will find src/skillgap/config/agents.yaml, src/skillgap/config/tasks.yaml, src/skillgap/crew.py, and src/skillgap/main.py.
- Move into the new project folder by running:
cd skillgap- Add the project dependencies by running:
uv add scikit-learn python-dotenv- Add Groq support via LiteLLM by running:
uv add 'crewai[litellm]'Why crewai[litellm]?
Groq is not a native CrewAI provider. The litellm extra adds a translation layer that lets CrewAI talk to Groq's API using the model string format groq/llama-3.3-70b-versatile.
- Create a .env file in the skillgap project folder to store your Groq API key. Create the file and add this content:
GROQ_API_KEY=your-api-key-hereWhere do I find my Groq API key?
Log in to your Groq account at console.groq.com. Navigate to API Keys and create a new key. Copy the key value and replace your-api-key-here in your .env file.
- Verify the full project setup by running:
crewai installThis installs all dependencies into a virtual environment. You should see output indicating packages are being installed, finishing without errors.
Seeing errors during crewai install?
If you see a Python version error, double-check that python3 --version shows 3.13.x (not 3.14). CrewAI will refuse to install on Python 3.14.
If you see network errors, check your internet connection and try running the command again.
Help me fix crewai install errors
- Open VS Code in the project directory. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
- Type VS Code and press Enter to open it.
- In VS Code, click File in the top menu bar.
- Click Open Folder.
- Select the skillgap folder on your Desktop, then click Open.
In the VS Code Explorer sidebar, you should see the full project structure including src/skillgap/, .env, and pyproject.toml.
✔️ Awesome, I've got everything!
Great. Double-check you have saved your .env file with your actual Groq API key replacing the placeholder.
ⓧ I'd like to double check the full code
Your .env file should look like this (with your real key replacing the placeholder):
GROQ_API_KEY=your-api-key-hereYour development environment is locked and loaded. Next up, you will define your three AI agents and configure how they work together as a crew.
Define Your Agent Crew
Your environment is ready and your project is scaffolded. Now it is time to define the AI agents that will do the actual work.
In CrewAI, each agent has a distinct role, goal, and backstory that shapes how it reasons. You will define three specialists that pass their outputs to each other in sequence, forming an autonomous analysis pipeline.
In this step, get ready to:
- Configure three specialized AI agents in a YAML file.
- Define the sequential task pipeline that chains agent outputs together.
- Wire agents, tasks, and tools together in the crew orchestration file.
Configure the three agents
Each agent in your crew needs a role, a goal, and a backstory. These are not cosmetic labels. CrewAI uses these fields to shape the LLM's reasoning at runtime.
- In VS Code, open src/skillgap/config/agents.yaml.
- Select all the existing scaffold content and replace it with the following:
market_researcher:
role: >
Senior Job Market Research Analyst for {industry}
goal: >
Identify the top emerging skills and technologies that will be critical
in the {industry} industry over the next 6-12 months
backstory: >
You are a seasoned labor market analyst who tracks hiring trends,
technology adoption patterns, and workforce shifts. You excel at
identifying which skills are gaining momentum before they become
mainstream requirements.
verbose: true
llm: groq/llama-3.3-70b-versatile
gap_analyst:
role: >
Workforce Skill Gap Analyst
goal: >
Quantify the gap between each team member's current skills and the
emerging market requirements using data-driven analysis
backstory: >
You are a data-driven workforce analyst who uses quantitative methods
to assess skill readiness. You believe in measuring gaps precisely
rather than making subjective judgments.
verbose: true
llm: groq/llama-3.3-70b-versatile
learning_advisor:
role: >
Personalized Learning Path Designer
goal: >
Create actionable, personalized upskilling recommendations for each
team member based on their specific gaps and existing strengths
backstory: >
You are an L&D specialist who designs efficient learning paths.
You prioritize recommendations that build on existing strengths
and focus on the highest-impact skills first.
verbose: true
llm: groq/llama-3.3-70b-versatileWhat does this configuration do?
Each agent definition has four key fields:
- role tells the LLM what persona to adopt. The {industry} placeholder gets filled at runtime with your input.
- goal defines what the agent is optimizing for. CrewAI uses this to judge when the agent has completed its task.
- backstory provides context that guides the agent's reasoning style and priorities.
- llm specifies Groq's free Llama 3.3 70B model using the groq/llama-3.3-70b-versatile provider string.
- Save agents.yaml.
- Confirm VS Code shows no red squiggly underlines in the file. If you see any, check your indentation matches the code above exactly.
Seeing red underlines in the YAML?
- YAML is whitespace-sensitive. Make sure each field is indented with exactly 2 spaces, not tabs.
- The > character after each field name means "folded block scalar" in YAML. It must appear on the same line as the key, with the value text on the next line indented further.
- Check that there are no trailing spaces after the > characters.
Why is my agents.yaml showing syntax errors even after fixing indentation?
✔️ Awesome, I've got everything!
Great. Make sure you have saved the file before moving on.
ⓧ I'd like to double check the full code
market_researcher:
role: >
Senior Job Market Research Analyst for {industry}
goal: >
Identify the top emerging skills and technologies that will be critical
in the {industry} industry over the next 6-12 months
backstory: >
You are a seasoned labor market analyst who tracks hiring trends,
technology adoption patterns, and workforce shifts. You excel at
identifying which skills are gaining momentum before they become
mainstream requirements.
verbose: true
llm: groq/llama-3.3-70b-versatile
gap_analyst:
role: >
Workforce Skill Gap Analyst
goal: >
Quantify the gap between each team member's current skills and the
emerging market requirements using data-driven analysis
backstory: >
You are a data-driven workforce analyst who uses quantitative methods
to assess skill readiness. You believe in measuring gaps precisely
rather than making subjective judgments.
verbose: true
llm: groq/llama-3.3-70b-versatile
learning_advisor:
role: >
Personalized Learning Path Designer
goal: >
Create actionable, personalized upskilling recommendations for each
team member based on their specific gaps and existing strengths
backstory: >
You are an L&D specialist who designs efficient learning paths.
You prioritize recommendations that build on existing strengths
and focus on the highest-impact skills first.
verbose: true
llm: groq/llama-3.3-70b-versatileConfigure the sequential tasks
Agents need instructions. In CrewAI, a task defines what an agent should accomplish, what output format to produce, and which prior task outputs it receives as context.
- Open src/skillgap/config/tasks.yaml.
- Select all the existing scaffold content and replace it with the following:
market_research_task:
description: >
Research the current job market for the {industry} industry.
Identify the top 5-7 emerging skills that are growing in demand.
For each skill, explain why it is becoming important and what
level of proficiency employers typically expect.
Focus on skills that are actionable to learn within 3-6 months.
expected_output: >
A structured list of 5-7 emerging skills with:
- Skill name
- Why it is emerging (market driver)
- Expected proficiency level
- Key topics or subtopics within the skill
agent: market_researcher
gap_analysis_task:
description: >
Using the team skills data from {team_csv_path}, analyze each team
member's current skills against the emerging skills identified by
the market researcher. Use the skill_matcher tool to compute
quantitative similarity scores. For each person, identify their
top 3 skill gaps (lowest similarity scores) and their closest
existing strengths.
expected_output: >
A structured analysis for each team member containing:
- Name and current role
- Gap scores for each emerging skill (0-100 scale)
- Top 3 priority gaps
- Existing strengths that are closest to emerging skills
agent: gap_analyst
context:
- market_research_task
recommendation_task:
description: >
Based on the gap analysis results, create a personalized upskilling
plan for each team member. Each plan should include specific learning
resources, estimated time commitment, and a priority order. Consider
each person's existing strengths as stepping stones to new skills.
Format the output as a professional report suitable for an L&D team.
expected_output: >
A professional markdown report with:
- Executive summary of team-wide skill gaps
- Individual upskilling plans for each team member
- Each plan includes: priority skills, recommended resources,
estimated timeline, and how existing skills provide a foundation
- Team-level recommendations for group training opportunities
agent: learning_advisor
context:
- gap_analysis_task
output_file: output/upskilling_report.mdHow do tasks chain together?
The context field is what makes this a pipeline instead of three isolated tasks. When gap_analysis_task lists market_research_task in its context, CrewAI automatically passes the first task's output into the second task's prompt.
The output_file field on the final task tells CrewAI to save the Learning Advisor's output as a markdown file. This is the deliverable that proves the system works.
- Save tasks.yaml.
- Confirm VS Code shows no red underlines. Pay attention to the context field indentation. The - market_research_task list item must be indented under context:.
Seeing indentation errors in tasks.yaml?
- The context field uses a YAML list (indicated by the - prefix). The dash must be indented 4 spaces from the left margin.
- Make sure output_file is at the same indentation level as description, expected_output, and agent.
Help me fix YAML indentation errors in my tasks.yaml file.
✔️ Awesome, I've got everything!
Great. Make sure you have saved the file before moving on.
ⓧ I'd like to double check the full code
market_research_task:
description: >
Research the current job market for the {industry} industry.
Identify the top 5-7 emerging skills that are growing in demand.
For each skill, explain why it is becoming important and what
level of proficiency employers typically expect.
Focus on skills that are actionable to learn within 3-6 months.
expected_output: >
A structured list of 5-7 emerging skills with:
- Skill name
- Why it is emerging (market driver)
- Expected proficiency level
- Key topics or subtopics within the skill
agent: market_researcher
gap_analysis_task:
description: >
Using the team skills data from {team_csv_path}, analyze each team
member's current skills against the emerging skills identified by
the market researcher. Use the skill_matcher tool to compute
quantitative similarity scores. For each person, identify their
top 3 skill gaps (lowest similarity scores) and their closest
existing strengths.
expected_output: >
A structured analysis for each team member containing:
- Name and current role
- Gap scores for each emerging skill (0-100 scale)
- Top 3 priority gaps
- Existing strengths that are closest to emerging skills
agent: gap_analyst
context:
- market_research_task
recommendation_task:
description: >
Based on the gap analysis results, create a personalized upskilling
plan for each team member. Each plan should include specific learning
resources, estimated time commitment, and a priority order. Consider
each person's existing strengths as stepping stones to new skills.
Format the output as a professional report suitable for an L&D team.
expected_output: >
A professional markdown report with:
- Executive summary of team-wide skill gaps
- Individual upskilling plans for each team member
- Each plan includes: priority skills, recommended resources,
estimated timeline, and how existing skills provide a foundation
- Team-level recommendations for group training opportunities
agent: learning_advisor
context:
- gap_analysis_task
output_file: output/upskilling_report.mdWire agents and tasks in crew.py
The YAML files define what your agents are and what they do. The crew.py file is where you wire everything together into a runnable crew using CrewAI's decorator pattern.
- Open src/skillgap/crew.py.
- Select all the existing scaffold content and replace it with the following:
from crewai import Agent, Crew, Process, Task
from crewai.project import CrewBase, agent, crew, task
from skillgap.tools.skill_matcher import SkillMatcherTool
@CrewBase
class SkillgapCrew:
"""SkillGap crew for workforce upskilling analysis."""
agents_config = "config/agents.yaml"
tasks_config = "config/tasks.yaml"
@agent
def market_researcher(self) -> Agent:
return Agent(
config=self.agents_config["market_researcher"],
verbose=True,
)
@agent
def gap_analyst(self) -> Agent:
return Agent(
config=self.agents_config["gap_analyst"],
verbose=True,
tools=[SkillMatcherTool()],
)
@agent
def learning_advisor(self) -> Agent:
return Agent(
config=self.agents_config["learning_advisor"],
verbose=True,
)
@task
def market_research_task(self) -> Task:
return Task(
config=self.tasks_config["market_research_task"],
)
@task
def gap_analysis_task(self) -> Task:
return Task(
config=self.tasks_config["gap_analysis_task"],
)
@task
def recommendation_task(self) -> Task:
return Task(
config=self.tasks_config["recommendation_task"],
output_file="output/upskilling_report.md",
)
@crew
def crew(self) -> Crew:
"""Creates the SkillGap crew."""
return Crew(
agents=self.agents,
tasks=self.tasks,
process=Process.sequential,
verbose=True,
)What does this code do?
- @CrewBase is a class decorator that gives your crew auto-discovery of agents and tasks defined in YAML config files.
- Each @agent method maps to a key in agents.yaml. The method name must match the YAML key exactly.
- The gap_analyst agent gets the SkillMatcherTool passed in its tools list. This lets the agent autonomously call the tool during reasoning.
- Process.sequential means agents run one after another. The Market Researcher goes first, then the Gap Analyst, then the Learning Advisor.
Why is SkillMatcherTool imported before it exists?
You will build the SkillMatcherTool in the next step. VS Code may highlight the import line with an error. This is expected and will resolve once you create the tool file.
- Save crew.py.
- Verify the project dependencies are intact by running this command in the terminal:
crewai install- You should see output ending in a success message with no errors. This confirms your pyproject.toml and lock file are consistent.
Seeing errors when running crewai install?
- Make sure you are running the command from the skillgap project root directory (the folder containing pyproject.toml).
- If you see a dependency resolution error, try running uv lock first, then crewai install again.
- The import error for SkillMatcherTool in VS Code is expected at this point. It does not affect crewai install because that command only syncs dependencies from the lock file.
Help me debug a crewai install error in my SkillGap project.
✔️ Awesome, I've got everything!
Great. Double check you have saved all three files before moving on.
ⓧ I'd like to double check the full code
from crewai import Agent, Crew, Process, Task
from crewai.project import CrewBase, agent, crew, task
from skillgap.tools.skill_matcher import SkillMatcherTool
@CrewBase
class SkillgapCrew:
"""SkillGap crew for workforce upskilling analysis."""
agents_config = "config/agents.yaml"
tasks_config = "config/tasks.yaml"
@agent
def market_researcher(self) -> Agent:
return Agent(
config=self.agents_config["market_researcher"],
verbose=True,
)
@agent
def gap_analyst(self) -> Agent:
return Agent(
config=self.agents_config["gap_analyst"],
verbose=True,
tools=[SkillMatcherTool()],
)
@agent
def learning_advisor(self) -> Agent:
return Agent(
config=self.agents_config["learning_advisor"],
verbose=True,
)
@task
def market_research_task(self) -> Task:
return Task(
config=self.tasks_config["market_research_task"],
)
@task
def gap_analysis_task(self) -> Task:
return Task(
config=self.tasks_config["gap_analysis_task"],
)
@task
def recommendation_task(self) -> Task:
return Task(
config=self.tasks_config["recommendation_task"],
output_file="output/upskilling_report.md",
)
@crew
def crew(self) -> Crew:
"""Creates the SkillGap crew."""
return Crew(
agents=self.agents,
tasks=self.tasks,
process=Process.sequential,
verbose=True,
)Your agent crew is fully defined. Next up, you will build the TF-IDF skill matching tool that gives the Gap Analyst its analytical power.
Build the TF-IDF Skill Matching Tool
Your agents and tasks are configured, but the Gap Analyst agent still needs a way to measure skill gaps with actual numbers. Without a quantitative tool, the agent would rely on subjective LLM guesses instead of data-driven scores.
In this step, you will build a custom CrewAI tool that uses TF-IDF vectorization and cosine similarity to compute how close each team member's skills are to emerging market requirements. The Gap Analyst agent will call this tool autonomously during its task.
In this step, get ready to:
- Create a sample team dataset for analysis.
- Build a custom TF-IDF skill matching tool.
- Test the tool in isolation to confirm it produces similarity scores.
Create a sample team dataset
The agent crew needs real data to analyze. You will create a CSV file containing five team members with different skill profiles, experience levels, and roles.
- In VS Code's Explorer sidebar, right-click on the src/skillgap folder.
- Click New Folder and name it data.
- Right-click on the new data folder, click New File, and name it team_skills.csv.
- Paste the following CSV content into team_skills.csv:
name,current_skills,years_experience,current_role
Alice Chen,"python, sql, pandas, data visualization, statistics, machine learning basics",4,Data Analyst
Bob Martinez,"javascript, react, html, css, node.js, rest apis, git",6,Frontend Developer
Carol Johnson,"project management, agile, stakeholder communication, jira, confluence, risk assessment",8,Project Manager
David Kim,"python, tensorflow, deep learning, computer vision, numpy, docker",3,ML Engineer
Eva Patel,"java, spring boot, microservices, aws, kubernetes, ci/cd, sql",5,Backend DeveloperWhat does this data represent?
Each row represents one team member with their comma-separated skill list, years of experience, and current role.
The TF-IDF tool will compare these skill strings against emerging market skills to produce a similarity score. The variety of profiles (data, frontend, management, ML, backend) ensures the tool produces meaningfully different gap scores for each person.
- Save team_skills.csv.
- Confirm the file appears in your Explorer sidebar at src/skillgap/data/team_skills.csv.
Don't see the data folder?
Make sure you created the data folder inside src/skillgap/, not at the project root. The tool's file-loading logic expects CSV files to live at src/skillgap/data/.
help me find where to put team_skills.csv in my CrewAI project
✔️ Awesome, I've got everything!
Great. Make sure you have saved team_skills.csv before moving on.
ⓧ I'd like to double check the full code
name,current_skills,years_experience,current_role
Alice Chen,"python, sql, pandas, data visualization, statistics, machine learning basics",4,Data Analyst
Bob Martinez,"javascript, react, html, css, node.js, rest apis, git",6,Frontend Developer
Carol Johnson,"project management, agile, stakeholder communication, jira, confluence, risk assessment",8,Project Manager
David Kim,"python, tensorflow, deep learning, computer vision, numpy, docker",3,ML Engineer
Eva Patel,"java, spring boot, microservices, aws, kubernetes, ci/cd, sql",5,Backend DeveloperBuild the custom CrewAI tool
CrewAI agents can use custom tools to perform specific computations. You will build a tool that uses scikit-learn's TF-IDF vectorizer to convert skill text into numerical vectors, then compute cosine similarity to score how close each person's skills are to emerging market requirements.
- In VS Code's Explorer sidebar, expand the src/skillgap/tools/ folder.
- Open the file skill_matcher.py (if it does not exist yet, right-click the tools folder, click New File, and name it skill_matcher.py).
- Delete any existing content, then paste in the imports and input schema:
import csv
from pathlib import Path
from typing import Type
from crewai.tools import BaseTool
from pydantic import BaseModel, Field
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
class SkillMatcherInput(BaseModel):
"""Input schema for the SkillMatcher tool."""
emerging_skills: str = Field(
..., description="Comma-separated list of emerging skills to match against"
)
team_csv_path: str = Field(
..., description="Path to the team skills CSV file"
)What does this code do?
The imports bring in BaseTool from CrewAI (the base class every custom tool must extend), TfidfVectorizer for converting text into numerical vectors, and cosine_similarity for measuring how similar two vectors are.
The SkillMatcherInput class tells the agent exactly what inputs the tool expects. CrewAI uses this schema to validate the agent's tool call before running it.
- Below the SkillMatcherInput class, paste the main tool class:
class SkillMatcherTool(BaseTool):
name: str = "skill_matcher"
description: str = (
"Computes TF-IDF cosine similarity scores between each team member's "
"current skills and a set of target emerging skills. Returns a "
"quantitative gap analysis showing how close each person is to each "
"emerging skill on a 0-100 scale."
)
args_schema: Type[BaseModel] = SkillMatcherInput
def _run(self, emerging_skills: str, team_csv_path: str) -> str:
# Load team data from CSV
team_data = self._load_team_data(team_csv_path)
target_skills = [s.strip() for s in emerging_skills.split(",")]
# Build document list: team skills + target skills
all_documents = []
member_names = []
for member in team_data:
all_documents.append(member["current_skills"])
member_names.append(member["name"])
for skill in target_skills:
all_documents.append(skill)
# Compute TF-IDF vectors for all documents
vectorizer = TfidfVectorizer()
tfidf_matrix = vectorizer.fit_transform(all_documents)
# Split into member vectors and skill vectors
num_members = len(member_names)
member_vectors = tfidf_matrix[:num_members]
skill_vectors = tfidf_matrix[num_members:]
# Compute cosine similarity between members and skills
similarity_matrix = cosine_similarity(member_vectors, skill_vectors)
# Format results for the agent
results = []
for i, name in enumerate(member_names):
member_result = f"\n## {name} (Role: {team_data[i]['current_role']})\n"
member_result += f"Experience: {team_data[i]['years_experience']} years\n"
member_result += "Skill Gap Scores (0=no match, 100=strong match):\n"
scores = []
for j, skill in enumerate(target_skills):
score = round(similarity_matrix[i][j] * 100, 1)
member_result += f" - {skill}: {score}/100\n"
scores.append((skill, score))
scores.sort(key=lambda x: x[1])
member_result += f"Top gaps: {', '.join([s[0] for s in scores[:3]])}\n"
member_result += f"Closest strengths: {', '.join([s[0] for s in scores[-2:]])}\n"
results.append(member_result)
return "\n".join(results)
def _load_team_data(self, csv_path: str) -> list:
# Resolve path relative to the data directory
data = []
file_path = Path(csv_path)
if not file_path.is_absolute():
file_path = Path(__file__).parent.parent / "data" / file_path.name
with open(file_path, newline="", encoding="utf-8") as f:
reader = csv.DictReader(f)
for row in reader:
data.append(row)
return dataWhat does this code do?
- The _run method is what the agent calls. It loads the CSV, splits the emerging skills string into a list, then builds a combined document list containing both team member skill strings and target skill strings.
- TfidfVectorizer converts all text into numerical vectors where each dimension represents a word's importance relative to the whole corpus.
- cosine_similarity compares each team member's vector against each target skill vector, producing a score from 0 (no overlap) to 1 (identical). The code multiplies by 100 to produce a 0-100 scale.
- The _load_team_data helper resolves relative paths against the data/ directory, so the agent can simply pass team_skills.csv as the path.
- Save skill_matcher.py.
Seeing a red underline on the imports?
If VS Code shows import errors for crewai.tools or sklearn, your editor may not be using the project's virtual environment. Press Cmd+Shift+P (macOS) or Ctrl+Shift+P (Windows), type Python: Select Interpreter, and choose the interpreter inside .venv.
help me fix import errors for crewai or sklearn in VS Code
✔️ Awesome, I've got everything!
Great. Double check you have saved skill_matcher.py with both the input schema and the tool class.
ⓧ I'd like to double check the full code
import csv
from pathlib import Path
from typing import Type
from crewai.tools import BaseTool
from pydantic import BaseModel, Field
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity
class SkillMatcherInput(BaseModel):
"""Input schema for the SkillMatcher tool."""
emerging_skills: str = Field(
..., description="Comma-separated list of emerging skills to match against"
)
team_csv_path: str = Field(
..., description="Path to the team skills CSV file"
)
class SkillMatcherTool(BaseTool):
name: str = "skill_matcher"
description: str = (
"Computes TF-IDF cosine similarity scores between each team member's "
"current skills and a set of target emerging skills. Returns a "
"quantitative gap analysis showing how close each person is to each "
"emerging skill on a 0-100 scale."
)
args_schema: Type[BaseModel] = SkillMatcherInput
def _run(self, emerging_skills: str, team_csv_path: str) -> str:
team_data = self._load_team_data(team_csv_path)
target_skills = [s.strip() for s in emerging_skills.split(",")]
all_documents = []
member_names = []
for member in team_data:
all_documents.append(member["current_skills"])
member_names.append(member["name"])
for skill in target_skills:
all_documents.append(skill)
vectorizer = TfidfVectorizer()
tfidf_matrix = vectorizer.fit_transform(all_documents)
num_members = len(member_names)
member_vectors = tfidf_matrix[:num_members]
skill_vectors = tfidf_matrix[num_members:]
similarity_matrix = cosine_similarity(member_vectors, skill_vectors)
results = []
for i, name in enumerate(member_names):
member_result = f"\n## {name} (Role: {team_data[i]['current_role']})\n"
member_result += f"Experience: {team_data[i]['years_experience']} years\n"
member_result += "Skill Gap Scores (0=no match, 100=strong match):\n"
scores = []
for j, skill in enumerate(target_skills):
score = round(similarity_matrix[i][j] * 100, 1)
member_result += f" - {skill}: {score}/100\n"
scores.append((skill, score))
scores.sort(key=lambda x: x[1])
member_result += f"Top gaps: {', '.join([s[0] for s in scores[:3]])}\n"
member_result += f"Closest strengths: {', '.join([s[0] for s in scores[-2:]])}\n"
results.append(member_result)
return "\n".join(results)
def _load_team_data(self, csv_path: str) -> list:
data = []
file_path = Path(csv_path)
if not file_path.is_absolute():
file_path = Path(__file__).parent.parent / "data" / file_path.name
with open(file_path, newline="", encoding="utf-8") as f:
reader = csv.DictReader(f)
for row in reader:
data.append(row)
return dataTest the tool in isolation
Before handing the tool to an agent, you want to confirm it produces meaningful similarity scores on its own. Running the tool directly lets you verify the TF-IDF logic works without waiting for an LLM call.
- Test the tool by running this command in your terminal:
uv run python -c "from skillgap.tools.skill_matcher import SkillMatcherTool; tool = SkillMatcherTool(); print(tool._run('machine learning, kubernetes, data engineering', 'team_skills.csv'))"What does this command do?
This imports your SkillMatcherTool, creates an instance, and calls its _run method with three sample emerging skills. The uv run prefix ensures Python uses your project's virtual environment with all installed dependencies.
- You should see output for each of the five team members, with similarity scores on a 0-100 scale for each emerging skill.
- Verify that David Kim scores highest on machine learning (he has deep learning and TensorFlow in his profile) and that Carol Johnson scores lowest (she has no technical skills listed).
Seeing a ModuleNotFoundError?
If you see an error about sklearn not being found, make sure you are running the command from the skillgap project root (the folder containing pyproject.toml). The uv run command only uses your project's virtual environment when run from the project directory.
If you see an error about skillgap not being found as a module, run crewai install first to ensure your project package is installed in development mode.
help me fix a ModuleNotFoundError when testing my CrewAI tool
Your custom tool is working. Next up, you will wire everything together with the main entry point and run the full agent crew to generate a complete upskilling report.
Run the Agent Crew and Generate the Report
In the previous step, you built the TF-IDF skill matching tool that quantifies how close each team member is to emerging market skills. Now it is time to tie everything together and watch your agents collaborate autonomously.
You will create a main entry point that loads your environment variables and kicks off the full CrewAI crew. Each agent will run its task in sequence, passing its output to the next agent as context.
In this step, get ready to:
- Create the main entry point that launches the crew.
- Run the full agent crew with crewai run.
- Review the generated upskilling report.
Create the main entry point
The crew needs an entry point that loads your Groq API key from the .env file, defines the inputs (industry and team CSV path), and starts the sequential workflow.
- Create a new file at src/skillgap/main.py and paste in the following code:
import os
from pathlib import Path
from dotenv import load_dotenv
from skillgap.crew import SkillgapCrew
def run():
"""Run the SkillGap crew."""
# Load the GROQ_API_KEY from .env
load_dotenv()
# Create the output directory if it doesn't exist
output_dir = Path("output")
output_dir.mkdir(exist_ok=True)
# Define inputs that fill the {placeholders} in your YAML configs
inputs = {
"industry": "Technology / Software Engineering",
"team_csv_path": "team_skills.csv",
}
# Kick off the crew - agents run sequentially
result = SkillgapCrew().crew().kickoff(inputs=inputs)
print("\n\n=== SKILLGAP ANALYSIS COMPLETE ===\n")
print(result.raw)
print("\nReport saved to output/upskilling_report.md")
if __name__ == "__main__":
run()What does this code do?
- load_dotenv() reads the .env file so CrewAI can access your GROQ_API_KEY environment variable when making LLM calls.
- The inputs dictionary fills the {industry} and {team_csv_path} placeholders you defined in agents.yaml and tasks.yaml.
- SkillgapCrew().crew().kickoff(inputs=inputs) builds the crew from your config, attaches the agents and tasks, then starts the sequential pipeline.
- result.raw contains the final text output from the last agent (the Learning Advisor).
- Save src/skillgap/main.py.
Seeing a ModuleNotFoundError?
- Make sure the file is at src/skillgap/main.py, not in the project root.
- Confirm you ran crewai install earlier to set up the project's virtual environment.
- help me debug a ModuleNotFoundError when running my CrewAI project
✔️ Awesome, I've got everything!
Great. Double check you have saved main.py before moving on.
ⓧ I'd like to double check the full code
import os
from pathlib import Path
from dotenv import load_dotenv
from skillgap.crew import SkillgapCrew
def run():
"""Run the SkillGap crew."""
load_dotenv()
output_dir = Path("output")
output_dir.mkdir(exist_ok=True)
inputs = {
"industry": "Technology / Software Engineering",
"team_csv_path": "team_skills.csv",
}
result = SkillgapCrew().crew().kickoff(inputs=inputs)
print("\n\n=== SKILLGAP ANALYSIS COMPLETE ===\n")
print(result.raw)
print("\nReport saved to output/upskilling_report.md")
if __name__ == "__main__":
run()Run the full crew
This is the moment everything comes together. When you run the crew, each agent processes its task in order, and you can watch the entire reasoning loop unfold in your terminal.
- Launch the agent crew by running this command:
crewai runWhat should I see?
Because verbose is set to true on all agents, you will see each agent's thought process, tool calls, and outputs streaming in your terminal.
- The Market Researcher identifies emerging skills for the Technology / Software Engineering industry.
- The Gap Analyst calls the skill_matcher tool to compute TF-IDF similarity scores for each team member.
- The Learning Advisor produces a personalized upskilling report and saves it to output/upskilling_report.md.
The full run typically takes 1-3 minutes depending on Groq response times.
- Wait for the run to finish. You should see === SKILLGAP ANALYSIS COMPLETE === printed in the terminal along with the raw output from the Learning Advisor.
Seeing a rate limit or API error?
- Groq's free tier allows 30 requests per minute. If you hit a rate limit, wait 60 seconds and run crewai run again.
- If you see an authentication error, double check that your .env file contains a valid GROQ_API_KEY and that you have saved the file.
- If the Gap Analyst fails to use the tool, confirm that crew.py passes tools=[SkillMatcherTool()] to the gap_analyst agent.
- help me debug an error when running crewai run for my SkillGap project
Review the generated output
The final product is a markdown report that an L&D team could use to make upskilling decisions.
- Open output/upskilling_report.md in VS Code.
- Confirm the report contains an executive summary of team-wide skill gaps.
- Check that each of the 5 team members (Alice Chen, Bob Martinez, Carol Johnson, David Kim, Eva Patel) has a personalized upskilling plan with priority skills, recommended resources, and estimated timelines.
What makes this report valuable?
The report is not just generic advice. Because the Gap Analyst used quantitative TF-IDF similarity scores, each person's recommendations are ranked by the size of their actual skill gap.
The Learning Advisor also uses each person's existing strengths as stepping stones, so the plans are realistic and build on what the team member already knows.
Secret mission
Add a Skills Heatmap Visualization
Your upskilling report is informative, but numbers in a markdown file don't tell the story at a glance. What if you could generate a color-coded heatmap that shows every team member's skill gaps in a single image? Red zones scream 'urgent gap,' green zones say 'we're covered.' That's the kind of artifact an L&D director pins on a wall.
Clean Up Your Resources
Clean Up Your Resources
This project runs entirely locally, so there are no ongoing costs. The Groq free tier requires no credit card and has no charges to worry about.
Decide whether to keep your project files on your machine, or delete them entirely.
Resources you used:
- The skillgap project directory on your Desktop (all source code, config files, and generated output).
Keep everything running
No action needed. Choose this if you want to keep the project as a portfolio piece or plan to extend it further.
- Your skillgap directory stays on your Desktop with all source code and generated reports.
- Your Groq account remains active on the free tier with no charges.
- The uv package manager and CrewAI CLI remain installed on your system.
Pause - I'll come back to this later
Since this project is entirely local with no running processes or cloud services, there is nothing to pause. Your files stay as they are until you return.
- No background processes are running. The agent crew only executes when you explicitly run crewai run.
- Your Groq API key stays in your local .env file, ready for next time.
Delete - I don't want to use this again
Remove the project directory to start fresh. This deletes all source code, configuration, and generated output.
- Delete the project folder by running the command for your operating system.
macOS
- Remove the project directory by running:
rm -rf ~/Desktop/skillgapWindows
- Remove the project directory by running:
rmdir /s /q %USERPROFILE%\Desktop\skillgap- Uninstall the CrewAI CLI by running:
uv tool uninstall crewai- Your Groq account is free and has no charges. You can leave it active or delete it from your Groq account settings if you prefer.
Nice Work!
Nice Work!
You just built a fully autonomous workforce upskilling system powered by three AI agents. From market research to gap analysis to personalized learning recommendations, your CrewAI crew handles the entire pipeline without manual intervention.
You've learned how to:
- Built a multi-agent AI system with 3 specialized agents (Market Researcher, Gap Analyst, Learning Advisor) orchestrated sequentially by CrewAI.
- Implemented a custom TF-IDF skill matching tool using scikit-learn that agents invoke autonomously to quantify skill gaps with cosine similarity scores.
- Generated a personalized upskilling report from a team CSV input using Groq's free Llama 3.3 70B model as the LLM backbone for agentic reasoning.
- Secret Mission: Added a skills heatmap visualization showing gap severity across the entire team with color-coded scores.
Ready to quiz yourself?