Build a GitHub Issue Recommender

Build a local ML pipeline that matches developers to open-source issues.

Introduction

30 Second Summary

Contributing to open source is one of the best ways to grow as a developer. The hard part is finding an issue that actually matches your skills across thousands of repositories.

In this project, you will build a local machine learning pipeline that scrapes GitHub issues, cleans them with a local AI model, and recommends the best-matching issues for any developer based on their skills.

What You'll Build

A Python CLI tool that implements a full machine learning pipeline for matching developers with open-source issues based on their skills and experience.

By the end of this project, you'll have:

  • A GitHub issue scraper that pulls real open-source issues via the REST API.
  • A local LLM cleaner that transforms raw issue text into structured data using Ollama and Llama 3.2.
  • An embedding pipeline and vector database that stores issues as searchable vectors and matches them to a developer's profile.
  • Secret Mission: Add an LLM-powered reranker that scores matches and explains why each issue suits a developer.

Do I need any prior experience?

This project assumes basic Python knowledge. You should be comfortable with variables, functions, loops, and dictionaries.

Before We Start

Before you can build a recommendation engine, you need the right tools in place. This project uses a local ML stack, meaning everything runs on your machine. No cloud services, no API keys, no ongoing costs.

In this step, you will install Ollama (a local LLM runtime), set up a Python virtual environment with the required packages, and create the project files that make up each stage of the pipeline.

In this step, get ready to:
  • Install Ollama and pull the Llama 3.2 3B model.
  • Create a Python virtual environment and install project dependencies.
  • Create the five project files that make up the ML pipeline.
Install Ollama and pull the Llama 3.2 3B model

Ollama is a runtime that lets you run large language models locally. Your project uses it to run Llama 3.2 3B, a small but capable model that fits comfortably in your GPU's 6GB of VRAM.

  • Check if Ollama is already installed by running this command in your terminal:
ollama -v

✔️ I see a version number

Ollama is already installed. Skip ahead to the model pull step below.

ⓧ Command not found

Ollama is not installed yet. Follow the instructions below for your operating system.

Windows

  • Install Ollama by running this command in PowerShell:
irm https://ollama.com/install.ps1 | iex
What does this command do?

This downloads and runs the official Ollama installer for Windows. It does not require Administrator rights and installs Ollama in your user home directory.

  • After installation completes, close your current terminal and open a new PowerShell window so the updated PATH takes effect.

macOS

  • Install Ollama by running this command in Terminal:
curl -fsSL https://ollama.com/install.sh | sh
What does this command do?

This downloads and runs the official Ollama installer for macOS. It sets up the Ollama CLI and runtime on your machine.

  • After installation completes, close your current terminal and open a new one so the updated PATH takes effect.
  • Verify the installation by running:
ollama -v

You should see output that includes a version number.

Still seeing command not found?

Make sure you opened a brand new terminal window after the installation finished. The old terminal does not pick up the updated PATH.

Help me troubleshoot why Ollama is not recognized as a command after I installed it.

Now pull the Llama 3.2 3B model. This is approximately a 2GB download.

  • Pull and test the Llama 3.2 model by running:
ollama run llama3.2

What does this command do?

This downloads the Llama 3.2 3B model (the default size for the llama3.2 tag) and starts an interactive chat session. You can type a question to verify the model is working.

  • Type a short question like What is Python? and press Enter to confirm the model responds.
  • Exit the chat session by typing /bye and pressing Enter.

Model download stuck or failing?

The model is approximately 2GB. On slower connections this can take several minutes. If the download fails partway through, run ollama run llama3.2 again and it will resume from where it left off.

Help me troubleshoot my Ollama model download.

Finally, verify the Ollama API server is running.

  • Open your browser and navigate to http://localhost:11434.

You should see a plain text page that says "Ollama is running". This confirms the local API server is active and ready for your Python code to connect to.

Page not loading?

Ollama runs a background server automatically after installation. If the page does not load, try running ollama serve in a new terminal window, then refresh the browser page.

Help me get the Ollama API server running.

Create a Python virtual environment and install dependencies

A virtual environment keeps this project's packages isolated from other Python projects on your machine. This way, the specific versions you install here do not conflict with anything else.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder, move into it, and set up a virtual environment by running these commands:

macOS

mkdir issue-recommender
cd issue-recommender
python3 -m venv venv
  • Activate the virtual environment by running:
source venv/bin/activate

Windows

mkdir issue-recommender
cd issue-recommender
python -m venv venv
  • Activate the virtual environment by running:
venv\Scripts\activate

You should see (venv) appear at the start of your terminal prompt. This confirms the virtual environment is active.

Not seeing (venv) in your prompt?

On Windows, you may need to enable script execution first. Run Set-ExecutionPolicy -Scope CurrentUser RemoteSigned in PowerShell, then try the activate command again.

Help me activate my Python virtual environment.

  • Install the four project dependencies by running:
pip install ollama==0.6.2 chromadb==1.5.9 sentence-transformers==5.6.0 requests

What are these packages?

  • ollama is the official Python client for communicating with the Ollama API.
  • chromadb is a vector database for storing and querying embeddings.
  • sentence-transformers loads the embedding model that converts text into numerical vectors.
  • requests makes HTTP requests to the GitHub API.

The installation may take a few minutes as sentence-transformers downloads several supporting libraries including PyTorch.

  • Verify each package installed correctly by running:
python -c "import ollama; import chromadb; import sentence_transformers; import requests; print('All packages installed.')"

You should see All packages installed. printed in your terminal.

Import errors?

Make sure the (venv) prefix is visible in your terminal prompt. If you accidentally installed packages outside the virtual environment, activate it and run the pip install command again.

Help me fix my package installation.

Create the project file structure

Each file in this project corresponds to one component of the ML pipeline. Creating them now gives you a clear picture of the full architecture before you start coding.

  • Create the five project files by running:

macOS

touch scraper.py janitor.py vectorizer.py matcher.py main.py

Windows

New-Item scraper.py, janitor.py, vectorizer.py, matcher.py, main.py
  • Confirm the files exist by running:
ls

You should see scraper.py, janitor.py, vectorizer.py, matcher.py, and main.py in the output alongside the venv folder.

What does each file do?

  • scraper.py pulls raw issue data from the GitHub API.
  • janitor.py cleans raw issue text into structured data using the local LLM.
  • vectorizer.py converts cleaned text into vector embeddings and stores them in a database.
  • matcher.py matches a developer's profile against stored issue embeddings.
  • main.py wires all the components together into a single runnable pipeline.
  • Open the project folder in VS Code. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Navigate to Desktop, select the issue-recommender folder, and click Open.

You should see all five .py files listed in the Explorer sidebar on the left.

Your ML toolkit is ready. Next up, you will build the scraper that pulls real GitHub issues from the API.

Build the Scraper

Your ML toolkit is set up and ready to go. Now it's time to build the first component of your pipeline: data ingestion.

GitHub hosts millions of open-source issues tagged as beginner-friendly. Your scraper will pull these issues through GitHub's REST API, giving your pipeline real data to clean, embed, and search through.

In this step, get ready to:
  • Learn how the GitHub Search API works with query parameters, pagination, and rate limits.
  • Write a fetch_issues() function that pulls "good first issue" issues from GitHub.
  • Test the scraper and verify you get real issue data.
Understand the GitHub search API

Before writing any code, it helps to understand the data source your scraper will talk to. GitHub provides a public search API that lets you find issues across all public repositories using filters.

How does the GitHub Search API work?

The endpoint GET https://api.github.com/search/issues accepts a q parameter with search qualifiers. You can filter by label, state, and programming language.

For example, label:"good first issue" state:open language:python finds all open Python issues tagged for beginners.

The API returns results in pages. You control page size with per_page and move between pages with page. For unauthenticated requests, the search endpoint is rate-limited to 10 requests per minute. Your scraper will need to handle this gracefully.

Write the scraper function

The scraper loops through each language, sends paginated requests to the GitHub search endpoint, and extracts the fields your pipeline needs from each issue.

  • In VS Code, open scraper.py from the Explorer sidebar and paste the following code:
import requests
import time


def fetch_issues(languages=None, max_issues=30):
    """Fetch 'good first issue' labeled issues from GitHub."""
    # Default to Python and JavaScript if no languages specified
    if languages is None:
        languages = ["python", "javascript"]

    all_issues = []
    per_page = 30
    # Split the total issue count evenly across languages
    issues_per_language = max_issues // len(languages)

    for language in languages:
        # Build the GitHub search query with label, state, and language filters
        query = f'label:"good first issue" state:open language:{language}'
        page = 1
        collected = 0

        while collected < issues_per_language:
            # Set up the API request with pagination parameters
            url = "https://api.github.com/search/issues"
            params = {
                "q": query,
                "sort": "created",
                "order": "desc",
                "per_page": per_page,
                "page": page,
            }
            headers = {"Accept": "application/vnd.github+json"}

            response = requests.get(url, params=params, headers=headers)

            # Handle rate limiting with a 60-second backoff
            if response.status_code == 403:
                print("Rate limited. Waiting 60 seconds...")
                time.sleep(60)
                continue

            if response.status_code != 200:
                print(f"Error {response.status_code}: {response.text}")
                break

            data = response.json()
            items = data.get("items", [])

            if not items:
                break

            # Extract the fields we need from each issue
            for item in items:
                issue = {
                    "title": item["title"],
                    "body": (item.get("body") or "")[:2000],
                    "labels": [label["name"] for label in item.get("labels", [])],
                    "repo": item["repository_url"].split("/")[-2]
                    + "/"
                    + item["repository_url"].split("/")[-1],
                    "url": item["html_url"],
                    "language": language,
                }
                all_issues.append(issue)
                collected += 1

                if collected >= issues_per_language:
                    break

            # Move to the next page and pause to respect rate limits
            page += 1
            time.sleep(2)

    print(f"Scraped {len(all_issues)} issues.")
    return all_issues

What does this code do?

  • fetch_issues() takes a list of languages and a maximum issue count, then splits the target evenly across languages.
  • For each language, it builds a search query using GitHub's label, state, and language qualifiers.
  • A while loop sends paginated requests to the search endpoint, incrementing page each time until enough issues are collected.
  • If the API returns a 403 status code (rate limit hit), the scraper waits 60 seconds and retries. A 2-second pause between pages keeps requests under the rate limit.
  • Each issue is trimmed to the six fields your pipeline needs: title, body (capped at 2000 characters), labels, repo name, URL, and language.
  • Save scraper.py.

You won't see any output yet. The next substep adds a test block that runs the function.

Seeing red underlines in VS Code?

  • Check that every line inside the function is indented with 4 spaces. Python is sensitive to indentation.
  • Make sure the f-string on the query line uses single quotes on the outside and double quotes around good first issue.
  • Confirm your virtual environment is still active. VS Code should show the venv name in the bottom status bar.

help me fix a syntax error in my scraper.py file

Test the scraper

Time to see if your scraper can pull real data from GitHub. You'll add a test block at the bottom of the file that runs the function and prints a few sample issues.

  • In scraper.py, scroll to the bottom of the file and add this test block after the fetch_issues function:
if __name__ == "__main__":
    # Fetch 6 issues (3 per language) and print the first 3
    issues = fetch_issues(max_issues=6)
    for issue in issues[:3]:
        print(f"\n--- {issue['title']} ---")
        print(f"Repo: {issue['repo']}")
        print(f"Labels: {issue['labels']}")
        print(f"Body preview: {issue['body'][:200]}...")

What does this test block do?

The if __name__ == "__main__": guard means this code only runs when you execute scraper.py directly. It won't run when other files import the fetch_issues function later.

It fetches 6 issues (3 per language) and prints the title, repo, labels, and a body preview for the first 3. This is enough to confirm the scraper is working without hitting the rate limit.

  • Save scraper.py.
  • Run the scraper by entering this command in your terminal:
python scraper.py

You should see output showing 3 issues with their titles, repo names, labels, and body previews. The exact issues will vary since the scraper pulls live data from GitHub.

Not seeing any output?

  • If you see ModuleNotFoundError: No module named 'requests', your virtual environment may not be active. Activate it and try again.
  • If you see Rate limited. Waiting 60 seconds..., the GitHub API has throttled your requests. Wait for the 60-second backoff to complete. The scraper handles this automatically.
  • If you see a connection error, check that you have an active internet connection.

help me debug why my GitHub issue scraper isn't returning any results

✔️ Awesome, I've got everything!

Great. Double check you've saved scraper.py before moving on.

ⓧ I'd like to double check the full code

import requests
import time


def fetch_issues(languages=None, max_issues=30):
    """Fetch 'good first issue' labeled issues from GitHub."""
    if languages is None:
        languages = ["python", "javascript"]

    all_issues = []
    per_page = 30
    issues_per_language = max_issues // len(languages)

    for language in languages:
        query = f'label:"good first issue" state:open language:{language}'
        page = 1
        collected = 0

        while collected < issues_per_language:
            url = "https://api.github.com/search/issues"
            params = {
                "q": query,
                "sort": "created",
                "order": "desc",
                "per_page": per_page,
                "page": page,
            }
            headers = {"Accept": "application/vnd.github+json"}

            response = requests.get(url, params=params, headers=headers)

            if response.status_code == 403:
                print("Rate limited. Waiting 60 seconds...")
                time.sleep(60)
                continue

            if response.status_code != 200:
                print(f"Error {response.status_code}: {response.text}")
                break

            data = response.json()
            items = data.get("items", [])

            if not items:
                break

            for item in items:
                issue = {
                    "title": item["title"],
                    "body": (item.get("body") or "")[:2000],
                    "labels": [label["name"] for label in item.get("labels", [])],
                    "repo": item["repository_url"].split("/")[-2]
                    + "/"
                    + item["repository_url"].split("/")[-1],
                    "url": item["html_url"],
                    "language": language,
                }
                all_issues.append(issue)
                collected += 1

                if collected >= issues_per_language:
                    break

            page += 1
            time.sleep(2)

    print(f"Scraped {len(all_issues)} issues.")
    return all_issues


if __name__ == "__main__":
    issues = fetch_issues(max_issues=6)
    for issue in issues[:3]:
        print(f"\n--- {issue['title']} ---")
        print(f"Repo: {issue['repo']}")
        print(f"Labels: {issue['labels']}")
        print(f"Body preview: {issue['body'][:200]}...")

Your scraper is pulling real data from GitHub. Next up, you'll build a janitor that uses a local LLM to clean this raw text into structured data.

Build the Janitor

Your scraper is pulling real GitHub issues from the API. The next challenge is that this raw data is messy.

GitHub issues are full of markdown formatting, code blocks, stack traces, and inconsistent language. In this step, you will build a "janitor" that uses a local LLM running on Ollama to clean each issue into structured JSON.

In this step, get ready to:
  • Define a JSON schema for structured LLM output.
  • Write functions to clean individual and batch issues using Ollama.
  • Test the janitor on a sample issue to verify structured output.
Understand LLM-based data cleaning

Raw GitHub issues contain markdown headers, inline code, stack traces, emoji, and wildly inconsistent formatting. This makes them unreliable for generating embeddings later in the pipeline.

A small local LLM can read this messy text and extract the core meaning. This is far more reliable than trying to write regex patterns for every possible formatting style.

Why use an LLM instead of regex?

Regex can only match patterns it has been explicitly programmed to recognize. An LLM understands natural language context, so it can extract the core problem from an issue regardless of how the author formatted it.

This pattern of using LLMs for data cleaning is common in production ML pipelines. You define the output shape you want, and the LLM does the hard work of understanding messy input.

Ollama supports a feature called structured output. You pass a JSON schema to the format parameter, and the LLM is guaranteed to respond with valid JSON matching that exact shape.

What is structured output?

Instead of hoping the LLM returns well-formatted JSON, structured output enforces it at the model level. You define the fields you want (like problem, skills, and difficulty), and Ollama constrains the LLM's token generation to only produce valid JSON matching your schema.

Define the schema and system prompt

The janitor needs two things before it can clean anything: a JSON schema describing the output shape, and a system prompt telling the LLM what to extract.

  • Open janitor.py in VS Code and add the imports, schema, and system prompt by pasting this code:
import json
from ollama import chat

# Define the JSON schema that the LLM must follow
CLEAN_SCHEMA = {
    "type": "object",
    "properties": {
        "problem": {"type": "string"},
        "skills": {
            "type": "array",
            "items": {"type": "string"},
        },
        "difficulty": {"type": "string"},
    },
    "required": ["problem", "skills", "difficulty"],
}

# Instructions for the LLM on how to clean issue data
SYSTEM_PROMPT = (
    "You are a data cleaning assistant. Given a raw GitHub issue, "
    "extract a concise summary of the core problem, the technical "
    "skills needed to solve it, and the difficulty level "
    "(beginner, intermediate, or advanced). Be concise and precise."
)

What does this code do?

  • from ollama import chat imports the chat() function from the official Ollama Python package. This is how you talk to the local LLM.
  • CLEAN_SCHEMA is a JSON schema dict with three required fields: problem (string), skills (array of strings), and difficulty (string). Passing this to the format parameter forces the LLM to only produce JSON matching this shape.
  • SYSTEM_PROMPT tells the LLM its role and what to extract from each issue. This is the instruction that guides the cleaning.
  • Save janitor.py.
  • You won't see a change yet. The next chunk uses these constants to call the LLM.
Write the cleaning functions

With the schema and prompt defined, you need two functions. One cleans a single issue by calling the LLM, and the other runs that cleaner across a batch of issues.

  • In janitor.py, add the clean_issue() function below SYSTEM_PROMPT by pasting this code:
def clean_issue(raw_issue):
    """Clean a raw GitHub issue into structured data using a local LLM."""
    # Format the raw issue data into a prompt for the LLM
    user_content = (
        f"Issue title: {raw_issue['title']}\n\n"
        f"Repository: {raw_issue['repo']}\n\n"
        f"Labels: {', '.join(raw_issue['labels'])}\n\n"
        f"Body:\n{raw_issue['body']}"
    )

    # Send the issue to the local LLM with structured output enforced
    response = chat(
        model="llama3.2",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_content},
        ],
        format=CLEAN_SCHEMA,
        stream=False,
    )

    # Parse the JSON response and attach original metadata
    cleaned = json.loads(response.message.content)
    cleaned["title"] = raw_issue["title"]
    cleaned["repo"] = raw_issue["repo"]
    cleaned["url"] = raw_issue["url"]
    cleaned["language"] = raw_issue["language"]

    return cleaned

What does this code do?

  • user_content assembles the raw issue's title, repo, labels, and body into a single string the LLM can read.
  • chat() sends two messages to the llama3.2 model: a system message with your instructions, and a user message with the issue data. The format=CLEAN_SCHEMA parameter guarantees the response is valid JSON matching your schema. Setting stream=False returns the full response at once.
  • json.loads(response.message.content) parses the LLM's JSON string into a Python dictionary. The function then attaches the original metadata (title, repo, URL, language) so you can trace each cleaned issue back to its source.
  • Now add the clean_all_issues() batch function below clean_issue():
def clean_all_issues(raw_issues):
    """Clean a list of raw issues."""
    cleaned = []
    for i, issue in enumerate(raw_issues):
        # Show progress as each issue is cleaned
        print(f"Cleaning issue {i + 1}/{len(raw_issues)}: {issue['title'][:60]}...")
        try:
            result = clean_issue(issue)
            cleaned.append(result)
        except Exception as e:
            # Skip issues that fail (network errors, LLM timeouts, etc.)
            print(f"  Skipped (error: {e})")
    print(f"Cleaned {len(cleaned)}/{len(raw_issues)} issues.")
    return cleaned

What does this code do?

  • clean_all_issues() loops through every raw issue and calls clean_issue() on each one. It prints progress so you can see how far along the batch is.
  • The try/except block catches any errors (network issues, LLM timeouts, malformed responses) and skips the problematic issue instead of crashing the whole pipeline.
  • Save janitor.py.
  • You won't see a change yet. The next substep adds a test block so you can run the janitor.
Test the janitor on a sample issue

Before running the janitor on live GitHub data, test it with a hardcoded sample issue. This confirms the LLM connection works and the structured output is shaped correctly.

  • Add this test block at the bottom of janitor.py, below clean_all_issues():
if __name__ == "__main__":
    # Create a sample issue to test the cleaning pipeline
    sample = {
        "title": "Add dark mode support to settings page",
        "body": "The settings page does not respect the user's dark mode preference. "
        "We need to update the CSS variables and add a toggle component. "
        "See the design mockup in Figma link attached.",
        "labels": ["good first issue", "UI", "enhancement"],
        "repo": "example/webapp",
        "url": "https://github.com/example/webapp/issues/42",
        "language": "javascript",
    }
    result = clean_issue(sample)
    print(json.dumps(result, indent=2))

What does this code do?

  • The if __name__ == "__main__" block only runs when you execute janitor.py directly. It creates a fake issue about dark mode, cleans it through the LLM, and pretty-prints the structured JSON result.
  • json.dumps(result, indent=2) formats the dictionary as indented JSON so you can easily read the LLM's output.
  • Save janitor.py.
  • Run the janitor test by running this command in your terminal:
python janitor.py

The LLM takes a few seconds to process. You should see a JSON object printed with problem, skills, difficulty, title, repo, url, and language fields.

What should I see?

The exact wording from the LLM will vary each time, but the structure is always the same. You should see something like:

{
  "problem": "Settings page does not support dark mode preferences",
  "skills": ["CSS", "JavaScript", "UI components"],
  "difficulty": "beginner",
  "title": "Add dark mode support to settings page",
  "repo": "example/webapp",
  "url": "https://github.com/example/webapp/issues/42",
  "language": "javascript"
}

The problem, skills, and difficulty fields are extracted by the LLM. The title, repo, url, and language fields are attached from the original issue metadata.

Not seeing any output?

  • If you see ConnectionError: Failed to connect to Ollama, Ollama is not running. Open a separate terminal and run ollama serve to start the server.
  • If the command hangs for more than 60 seconds, the model may still be loading into GPU memory. Wait for it to finish. The first run is always slower.
  • Make sure your virtual environment is activated. You should see the environment name in your terminal prompt.

Still stuck? Help me debug why janitor.py is not producing output.

✔️ Awesome, I've got everything!

Great. Double check you have saved janitor.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete janitor.py file for comparison:

import json
from ollama import chat


CLEAN_SCHEMA = {
    "type": "object",
    "properties": {
        "problem": {"type": "string"},
        "skills": {
            "type": "array",
            "items": {"type": "string"},
        },
        "difficulty": {"type": "string"},
    },
    "required": ["problem", "skills", "difficulty"],
}

SYSTEM_PROMPT = (
    "You are a data cleaning assistant. Given a raw GitHub issue, "
    "extract a concise summary of the core problem, the technical "
    "skills needed to solve it, and the difficulty level "
    "(beginner, intermediate, or advanced). Be concise and precise."
)


def clean_issue(raw_issue):
    """Clean a raw GitHub issue into structured data using a local LLM."""
    user_content = (
        f"Issue title: {raw_issue['title']}\n\n"
        f"Repository: {raw_issue['repo']}\n\n"
        f"Labels: {', '.join(raw_issue['labels'])}\n\n"
        f"Body:\n{raw_issue['body']}"
    )

    response = chat(
        model="llama3.2",
        messages=[
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": user_content},
        ],
        format=CLEAN_SCHEMA,
        stream=False,
    )

    cleaned = json.loads(response.message.content)
    cleaned["title"] = raw_issue["title"]
    cleaned["repo"] = raw_issue["repo"]
    cleaned["url"] = raw_issue["url"]
    cleaned["language"] = raw_issue["language"]

    return cleaned


def clean_all_issues(raw_issues):
    """Clean a list of raw issues."""
    cleaned = []
    for i, issue in enumerate(raw_issues):
        print(f"Cleaning issue {i + 1}/{len(raw_issues)}: {issue['title'][:60]}...")
        try:
            result = clean_issue(issue)
            cleaned.append(result)
        except Exception as e:
            print(f"  Skipped (error: {e})")
    print(f"Cleaned {len(cleaned)}/{len(raw_issues)} issues.")
    return cleaned


if __name__ == "__main__":
    sample = {
        "title": "Add dark mode support to settings page",
        "body": "The settings page does not respect the user's dark mode preference. "
        "We need to update the CSS variables and add a toggle component. "
        "See the design mockup in Figma link attached.",
        "labels": ["good first issue", "UI", "enhancement"],
        "repo": "example/webapp",
        "url": "https://github.com/example/webapp/issues/42",
        "language": "javascript",
    }
    result = clean_issue(sample)
    print(json.dumps(result, indent=2))

Your janitor is working. It takes messy, unstructured issue text and produces clean, consistent JSON. Next up, you will turn that clean text into numerical vectors and store them in a database for fast similarity search.

Build the Vectorizer and Database

Your janitor is now turning messy GitHub issues into clean, structured data. The next question is: how do you search that data for the best matches?

Traditional keyword search would miss issues that use different words for the same concept. In this step, you will convert cleaned issues into numerical vectors using a dedicated embedding model, then store those vectors in a ChromaDB database so you can search by meaning instead of exact words.

In this step, get ready to:
  • Learn how text embeddings and vector databases power semantic search.
  • Build the vectorizer and storage pipeline in vectorizer.py.
  • Test the embedding model by generating a sample vector.
Understand text embeddings and vector databases

Before writing code, it helps to understand the two core ideas behind semantic search: text embeddings and vector databases.

What is a text embedding?

An embedding is a fixed-length array of numbers (a vector) that captures the meaning of a piece of text. Two sentences about the same topic produce vectors that point in similar directions, even if they use completely different words.

The closeness between two vectors is measured using cosine similarity, which compares the angle between them. A smaller angle means higher similarity.

A vector database like ChromaDB stores these embeddings alongside metadata and enables fast nearest-neighbor search. Instead of scanning every issue one by one, it finds the closest vectors in milliseconds.

Why use a dedicated embedding model instead of the LLM?

You already have Llama 3.2 running locally, so you might wonder why you need a separate model for embeddings. Embedding models like BGE-small are specifically trained to map text into a vector space where semantic similarity is meaningful.

BGE-small is also much smaller and faster than the LLM. It produces 384-dimensional vectors and processes text in a fraction of the time that a full language model would take.

Write the vectorizer and storage pipeline

The vectorizer.py file has three jobs: load the embedding model, connect to the ChromaDB database, and convert cleaned issues into stored vectors.

  • In VS Code, open vectorizer.py and add the imports, configuration constants, and two utility functions by copying the following code:
from sentence_transformers import SentenceTransformer
import chromadb

# Configuration constants for the embedding model and database
EMBEDDING_MODEL_NAME = "BAAI/bge-small-en-v1.5"
CHROMA_DB_PATH = "./chroma_db"
COLLECTION_NAME = "github_issues"


def get_embedding_model():
    """Load the BGE embedding model."""
    print(f"Loading embedding model: {EMBEDDING_MODEL_NAME}...")
    model = SentenceTransformer(EMBEDDING_MODEL_NAME)
    print("Embedding model loaded.")
    return model


def get_chroma_collection():
    """Get or create the ChromaDB collection."""
    client = chromadb.PersistentClient(path=CHROMA_DB_PATH)
    collection = client.get_or_create_collection(
        name=COLLECTION_NAME,
        metadata={"hnsw:space": "cosine"},
    )
    return collection

What does this code do?

  • EMBEDDING_MODEL_NAME points to the BAAI/bge-small-en-v1.5 model on Hugging Face. This model produces 384-dimensional vectors optimized for semantic similarity.
  • CHROMA_DB_PATH sets ./chroma_db as the folder where ChromaDB persists data to disk. This means you can re-run the matcher later without re-embedding everything.
  • get_embedding_model() loads the BGE model using sentence-transformers. The first time this runs, it downloads the model files (about 130 MB).
  • get_chroma_collection() creates a persistent ChromaDB client and a collection configured with cosine distance. The get_or_create_collection() call returns an existing collection if one already exists.

You won't see a change yet. The next chunk adds the function that actually uses these utilities to embed and store issues.

  • Below get_chroma_collection(), add the build_index() function that ties everything together by copying this code:
def build_index(cleaned_issues, model=None):
    """Embed cleaned issues and store them in ChromaDB."""
    if model is None:
        model = get_embedding_model()

    collection = get_chroma_collection()

    # Prepare text, IDs, and metadata for each issue
    texts = []
    ids = []
    metadatas = []

    for i, issue in enumerate(cleaned_issues):
        # Combine problem description and skills into one searchable string
        text = f"{issue['problem']} Skills: {', '.join(issue['skills'])}"
        texts.append(text)
        ids.append(f"issue_{i}")
        metadatas.append(
            {
                "title": issue["title"],
                "repo": issue["repo"],
                "url": issue["url"],
                "difficulty": issue["difficulty"],
                "skills": ", ".join(issue["skills"]),
                "language": issue["language"],
            }
        )

    # Generate embeddings with normalized vectors for cosine similarity
    print(f"Generating embeddings for {len(texts)} issues...")
    embeddings = model.encode(texts, normalize_embeddings=True).tolist()

    # Store embeddings, documents, and metadata in ChromaDB
    collection.upsert(
        ids=ids,
        embeddings=embeddings,
        documents=texts,
        metadatas=metadatas,
    )

    print(f"Stored {collection.count()} issues in ChromaDB.")
    return collection, model

What does this code do?

  • The loop combines each issue's problem description and skills into a single text string. This gives the embedding model both the context and the technical requirements in one shot.
  • model.encode(texts, normalize_embeddings=True) converts all the text strings into 384-dimensional vectors at once. The normalize_embeddings=True flag ensures cosine similarity works correctly.
  • collection.upsert() stores the embeddings, the original text (as documents), and metadata like the repo name and difficulty. Using upsert instead of add means you can re-run the pipeline without getting duplicate ID errors.
  • The embeddings are passed directly to ChromaDB using the embeddings parameter. This gives you explicit control over which model generates them, instead of relying on ChromaDB's built-in default.

You won't see a change yet. The next chunk adds a test block so you can verify the embedding model works.

Test the vectorizer by running it standalone

Before wiring the vectorizer into the full pipeline, you should confirm the embedding model loads and produces vectors of the correct shape.

  • At the bottom of vectorizer.py, add the test block by copying this code:
if __name__ == "__main__":
    # Quick test: load the model and generate one embedding
    model = get_embedding_model()
    test_embedding = model.encode(["test query"], normalize_embeddings=True)
    print(f"Embedding shape: {test_embedding.shape}")
    print(f"First 5 values: {test_embedding[0][:5]}")

What does this code do?

This test block loads the BGE-small model and encodes a single string ("test query") into a vector. It then prints the shape of the resulting array and the first five values so you can confirm the model is working.

  • Save vectorizer.py.
  • Run the vectorizer test by running this command:
python vectorizer.py

What should I see?

The first time you run this, the script downloads the BGE-small model (about 130 MB). This may take a minute depending on your internet connection.

After the download finishes, you should see output like Embedding shape: (1, 384) followed by an array of five float values. The shape (1, 384) means one text input was converted into a 384-dimensional vector.

Not seeing the embedding shape output?

  • Make sure you saved vectorizer.py before running the command.
  • Check that your virtual environment is activated. You should see the environment name in parentheses at the start of your terminal prompt.
  • If you see an import error for sentence_transformers or chromadb, re-run the install command from Step 1 to make sure the packages are installed.

Why isn't my vectorizer.py producing the expected embedding shape output?

✔️ Awesome, I've got everything!

Great. Double check you've saved vectorizer.py and move on to the next step.

ⓧ I'd like to double check the full code

Here is the complete vectorizer.py file. Compare it against yours and make sure everything matches.

from sentence_transformers import SentenceTransformer
import chromadb


EMBEDDING_MODEL_NAME = "BAAI/bge-small-en-v1.5"
CHROMA_DB_PATH = "./chroma_db"
COLLECTION_NAME = "github_issues"


def get_embedding_model():
    """Load the BGE embedding model."""
    print(f"Loading embedding model: {EMBEDDING_MODEL_NAME}...")
    model = SentenceTransformer(EMBEDDING_MODEL_NAME)
    print("Embedding model loaded.")
    return model


def get_chroma_collection():
    """Get or create the ChromaDB collection."""
    client = chromadb.PersistentClient(path=CHROMA_DB_PATH)
    collection = client.get_or_create_collection(
        name=COLLECTION_NAME,
        metadata={"hnsw:space": "cosine"},
    )
    return collection


def build_index(cleaned_issues, model=None):
    """Embed cleaned issues and store them in ChromaDB."""
    if model is None:
        model = get_embedding_model()

    collection = get_chroma_collection()

    texts = []
    ids = []
    metadatas = []

    for i, issue in enumerate(cleaned_issues):
        text = f"{issue['problem']} Skills: {', '.join(issue['skills'])}"
        texts.append(text)
        ids.append(f"issue_{i}")
        metadatas.append(
            {
                "title": issue["title"],
                "repo": issue["repo"],
                "url": issue["url"],
                "difficulty": issue["difficulty"],
                "skills": ", ".join(issue["skills"]),
                "language": issue["language"],
            }
        )

    print(f"Generating embeddings for {len(texts)} issues...")
    embeddings = model.encode(texts, normalize_embeddings=True).tolist()

    collection.upsert(
        ids=ids,
        embeddings=embeddings,
        documents=texts,
        metadatas=metadatas,
    )

    print(f"Stored {collection.count()} issues in ChromaDB.")
    return collection, model


if __name__ == "__main__":
    model = get_embedding_model()
    test_embedding = model.encode(["test query"], normalize_embeddings=True)
    print(f"Embedding shape: {test_embedding.shape}")
    print(f"First 5 values: {test_embedding[0][:5]}")

Your vectorizer can now turn any cleaned issue into a searchable vector and store it in a persistent database. Next up, you will build the matcher that takes a real GitHub developer's profile and finds the issues that best fit their skills.

Build the Matcher and Wire the Pipeline

In the last step, you built the vectorizer that embeds cleaned issues and stores them in ChromaDB. Now it's time to close the loop by connecting a developer's skills to those stored issues.

You'll build a profile matcher that turns a GitHub username into a natural-language skill description. Then you'll embed that profile with the same BGE model and search ChromaDB for the most similar issues.

In this step, get ready to:
  • Build a profile matcher that fetches a developer's GitHub repos and constructs a skill profile.
  • Write a vector similarity search function that queries ChromaDB for matching issues.
  • Wire the full pipeline together in main.py and run it end-to-end.
Understand profile-based semantic matching

The GitHub REST API lets you fetch a user's public repositories, including what programming language each repo uses and what topics are tagged. By aggregating these languages and topics, you can build a natural-language "developer profile" string.

This profile string gets embedded using the same BGE-small model that embedded the issues. Because both the issues and the profile live in the same vector space, you can use cosine similarity to find issues that are semantically close to the developer's skills.

Why embed the profile with the same model?

The embedding model maps text into a shared vector space where proximity equals semantic similarity. If you used a different model for the profile than for the issues, those vectors would live in different spaces with incompatible dimensions. Using the same model ensures the comparisons are meaningful.

Write the matcher in matcher.py
  • Switch to matcher.py in VS Code.
  • Add the imports and build_profile() function by copying this code:
import requests
from vectorizer import get_embedding_model, get_chroma_collection


def build_profile(username):
    """Build a developer profile from their GitHub repos."""
    url = f"https://api.github.com/users/{username}/repos"
    params = {"per_page": 100, "sort": "updated"}
    headers = {"Accept": "application/vnd.github+json"}

    response = requests.get(url, params=params, headers=headers)

    if response.status_code != 200:
        print(f"Error fetching repos for {username}: {response.status_code}")
        return None

    repos = response.json()

    if not repos:
        print(f"No public repos found for {username}.")
        return None

    # Count how often each language appears across repos
    languages = {}
    topics = set()

    for repo in repos:
        lang = repo.get("language")
        if lang:
            languages[lang] = languages.get(lang, 0) + 1

        # Collect all repository topics
        for topic in repo.get("topics", []):
            topics.add(topic)

    # Sort languages by frequency, keep the top 5
    sorted_langs = sorted(languages.items(), key=lambda x: x[1], reverse=True)
    top_languages = [lang for lang, count in sorted_langs[:5]]

    # Build a natural-language profile string
    profile_text = f"Developer experienced in {', '.join(top_languages)}."
    if topics:
        profile_text += f" Interests: {', '.join(list(topics)[:10])}."

    print(f"Profile for {username}: {profile_text}")
    return profile_text

What does this code do?

  • build_profile(username) calls the GitHub REST API endpoint GET /users/{username}/repos to fetch up to 100 public repositories.
  • Each repo object includes a language field (the primary language) and a topics array (user-defined tags). The function counts how often each language appears across all repos.
  • The five most-used languages and up to ten topics get assembled into a plain-English string like "Developer experienced in Python, JavaScript. Interests: machine-learning, web-dev."
  • This natural-language string is what the embedding model converts into a vector for similarity search.

You won't see a change yet. The next chunk adds the search function that uses this profile.

  • Add the find_matches() function below build_profile() in matcher.py:

def find_matches(profile_text, model=None, n_results=10):
    """Find matching issues using vector similarity search."""
    if model is None:
        model = get_embedding_model()

    collection = get_chroma_collection()

    # Embed the profile text with the same BGE model
    profile_embedding = model.encode(
        [profile_text], normalize_embeddings=True
    ).tolist()

    # Query ChromaDB for the closest issue vectors
    results = collection.query(
        query_embeddings=profile_embedding,
        n_results=n_results,
    )

    # Unpack the results into a clean list of dicts
    matches = []
    for i in range(len(results["ids"][0])):
        match = {
            "title": results["metadatas"][0][i]["title"],
            "repo": results["metadatas"][0][i]["repo"],
            "url": results["metadatas"][0][i]["url"],
            "difficulty": results["metadatas"][0][i]["difficulty"],
            "skills": results["metadatas"][0][i]["skills"],
            "distance": results["distances"][0][i],
            "summary": results["documents"][0][i],
        }
        matches.append(match)

    return matches

What does this code do?

  • find_matches() loads the BGE-small model and connects to the ChromaDB collection you built in the previous step.
  • It encodes the profile text into a vector using the same normalize_embeddings=True setting the vectorizer used.
  • collection.query() performs a nearest-neighbor search, returning the closest issue vectors along with their metadata and distances.
  • The function unpacks the nested results structure into a flat list of dicts with title, repo, URL, difficulty, skills, distance, and summary.

You won't see a change yet. The next chunk adds a test block to verify both functions.

  • Add the test block at the bottom of matcher.py:

if __name__ == "__main__":
    # Test with GitHub's demo user
    profile = build_profile("octocat")
    if profile:
        model = get_embedding_model()
        matches = find_matches(profile, model=model, n_results=3)
        for m in matches:
            print(f"\n{m['title']}")
            print(f"  Repo: {m['repo']}")
            print(f"  Distance: {m['distance']:.4f}")

What does this test do?

  • The __main__ block calls build_profile("octocat") to build a profile from GitHub's demo user account.
  • It passes that profile text to find_matches() to search ChromaDB for the 3 closest issues.
  • Each result prints its title, repo, and distance score so you can verify the matching pipeline works.
  • Save matcher.py.
  • Run the matcher standalone by running this command:
python matcher.py

You should see a profile string for octocat followed by 3 matched issues with their titles, repos, and distance values.

Not seeing any matched issues?

  • Make sure the ChromaDB database was populated in the previous step. If you deleted the ./chroma_db folder, re-run vectorizer.py first to rebuild the index.
  • If you see "No public repos found", the username has no public repositories. Try a different GitHub username that has public repos.
  • If you see a connection error, make sure your internet connection is active. The GitHub API call requires network access.

Help me debug why my matcher isn't returning results from ChromaDB.

✔️ Awesome, I've got everything!

Great. Make sure you've saved matcher.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete matcher.py file:

import requests
from vectorizer import get_embedding_model, get_chroma_collection


def build_profile(username):
    """Build a developer profile from their GitHub repos."""
    url = f"https://api.github.com/users/{username}/repos"
    params = {"per_page": 100, "sort": "updated"}
    headers = {"Accept": "application/vnd.github+json"}

    response = requests.get(url, params=params, headers=headers)

    if response.status_code != 200:
        print(f"Error fetching repos for {username}: {response.status_code}")
        return None

    repos = response.json()

    if not repos:
        print(f"No public repos found for {username}.")
        return None

    languages = {}
    topics = set()

    for repo in repos:
        lang = repo.get("language")
        if lang:
            languages[lang] = languages.get(lang, 0) + 1

        for topic in repo.get("topics", []):
            topics.add(topic)

    sorted_langs = sorted(languages.items(), key=lambda x: x[1], reverse=True)
    top_languages = [lang for lang, count in sorted_langs[:5]]

    profile_text = f"Developer experienced in {', '.join(top_languages)}."
    if topics:
        profile_text += f" Interests: {', '.join(list(topics)[:10])}."

    print(f"Profile for {username}: {profile_text}")
    return profile_text


def find_matches(profile_text, model=None, n_results=10):
    """Find matching issues using vector similarity search."""
    if model is None:
        model = get_embedding_model()

    collection = get_chroma_collection()

    profile_embedding = model.encode(
        [profile_text], normalize_embeddings=True
    ).tolist()

    results = collection.query(
        query_embeddings=profile_embedding,
        n_results=n_results,
    )

    matches = []
    for i in range(len(results["ids"][0])):
        match = {
            "title": results["metadatas"][0][i]["title"],
            "repo": results["metadatas"][0][i]["repo"],
            "url": results["metadatas"][0][i]["url"],
            "difficulty": results["metadatas"][0][i]["difficulty"],
            "skills": results["metadatas"][0][i]["skills"],
            "distance": results["distances"][0][i],
            "summary": results["documents"][0][i],
        }
        matches.append(match)

    return matches


if __name__ == "__main__":
    profile = build_profile("octocat")
    if profile:
        model = get_embedding_model()
        matches = find_matches(profile, model=model, n_results=3)
        for m in matches:
            print(f"\n{m['title']}")
            print(f"  Repo: {m['repo']}")
            print(f"  Distance: {m['distance']:.4f}")
Wire everything together in main.py

Each pipeline component now works on its own. The final step is to connect them into a single script that runs the full flow from scraping to recommendations.

  • Switch to main.py in VS Code.
  • Add the full pipeline orchestration by copying this code:
from scraper import fetch_issues
from janitor import clean_all_issues
from vectorizer import build_index, get_embedding_model
from matcher import build_profile, find_matches


def run_pipeline():
    """Run the full ML pipeline."""
    print("=" * 60)
    print("GitHub Issue Recommender - ML Pipeline")
    print("=" * 60)

    # Step 1: Scrape open issues from GitHub
    print("\n[Step 1] Scraping GitHub issues...")
    raw_issues = fetch_issues(
        languages=["python", "javascript"],
        max_issues=20,
    )

    # Step 2: Clean each issue with the local LLM
    print("\n[Step 2] Cleaning issues with local LLM...")
    cleaned_issues = clean_all_issues(raw_issues)

    if not cleaned_issues:
        print("No issues were cleaned. Exiting.")
        return

    # Step 3: Embed cleaned issues and store in ChromaDB
    print("\n[Step 3] Embedding and storing in ChromaDB...")
    collection, model = build_index(cleaned_issues)

    # Step 4: Match a developer's profile to stored issues
    print("\n[Step 4] Finding matches for a developer...")
    username = input("\nEnter a GitHub username: ").strip()

    profile = build_profile(username)
    if not profile:
        return

    matches = find_matches(profile, model=model, n_results=5)

    # Display the top recommendations
    print("\n" + "=" * 60)
    print(f"Top {len(matches)} Recommended Issues for @{username}")
    print("=" * 60)

    for i, match in enumerate(matches, 1):
        similarity = 1 - match["distance"]
        print(f"\n{i}. {match['title']}")
        print(f"   Repo: {match['repo']}")
        print(f"   Difficulty: {match['difficulty']}")
        print(f"   Skills: {match['skills']}")
        print(f"   Similarity: {similarity:.1%}")
        print(f"   Link: {match['url']}")


if __name__ == "__main__":
    run_pipeline()

What does this pipeline do?

  • The pipeline runs in four sequential steps, each calling a component you built in a previous step.
  • Step 1 scrapes 20 open "good first issue" issues from GitHub (10 Python, 10 JavaScript) using your scraper.
  • Step 2 sends each raw issue through the Llama 3.2 janitor for structured cleaning.
  • Step 3 embeds the cleaned issues with BGE-small and stores them in ChromaDB.
  • Step 4 prompts you for a GitHub username, builds their profile, and queries ChromaDB for the 5 closest matches.
  • The similarity score is calculated as 1 - distance because ChromaDB returns cosine distance where lower values mean more similar.
  • Save main.py.
  • Run the full pipeline by running this command:
python main.py

The pipeline takes a few minutes to scrape and clean 20 issues. When prompted, enter a GitHub username (try your own or any public account).

You should see 5 recommended issues printed with their title, repo, difficulty, skills, similarity percentage, and URL.

Pipeline stuck on cleaning?

  • The LLM cleaning step processes each issue one at a time. With 20 issues, this can take a few minutes depending on your GPU. Wait for it to finish.
  • If you see a connection error during scraping, GitHub's search API limits unauthenticated requests to 10 per minute. The scraper handles this automatically with a 60-second wait.
  • Make sure Ollama is still running. If it stopped, restart it by running ollama run llama3.2 in a separate terminal, then exit with /bye and try again.

Help me debug why my pipeline is failing or not returning recommendations.

✔️ Awesome, I've got everything!

Great. Make sure you've saved main.py before moving on.

ⓧ I'd like to double check the full code

Here is the complete main.py file:

from scraper import fetch_issues
from janitor import clean_all_issues
from vectorizer import build_index, get_embedding_model
from matcher import build_profile, find_matches


def run_pipeline():
    """Run the full ML pipeline."""
    print("=" * 60)
    print("GitHub Issue Recommender - ML Pipeline")
    print("=" * 60)

    print("\n[Step 1] Scraping GitHub issues...")
    raw_issues = fetch_issues(
        languages=["python", "javascript"],
        max_issues=20,
    )

    print("\n[Step 2] Cleaning issues with local LLM...")
    cleaned_issues = clean_all_issues(raw_issues)

    if not cleaned_issues:
        print("No issues were cleaned. Exiting.")
        return

    print("\n[Step 3] Embedding and storing in ChromaDB...")
    collection, model = build_index(cleaned_issues)

    print("\n[Step 4] Finding matches for a developer...")
    username = input("\nEnter a GitHub username: ").strip()

    profile = build_profile(username)
    if not profile:
        return

    matches = find_matches(profile, model=model, n_results=5)

    print("\n" + "=" * 60)
    print(f"Top {len(matches)} Recommended Issues for @{username}")
    print("=" * 60)

    for i, match in enumerate(matches, 1):
        similarity = 1 - match["distance"]
        print(f"\n{i}. {match['title']}")
        print(f"   Repo: {match['repo']}")
        print(f"   Difficulty: {match['difficulty']}")
        print(f"   Skills: {match['skills']}")
        print(f"   Similarity: {similarity:.1%}")
        print(f"   Link: {match['url']}")


if __name__ == "__main__":
    run_pipeline()

Secret mission

Add an LLM-Powered Reranker

Your recommender finds matching issues using vector similarity, but similarity scores are just numbers. What if the LLM could reason about each match and explain why it suits a developer? Add a reranker that scores and explains every recommendation.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely locally, so there are no ongoing cloud costs. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Project folder on your Desktop containing scraper.py, janitor.py, vectorizer.py, matcher.py, main.py, the Python virtual environment, and the chroma_db database.
  • Ollama installed locally with the Llama 3.2 3B model (~2GB).
  • Hugging Face model cache for BGE-small-en-v1.5 (downloaded automatically by sentence-transformers).

Keep everything running

No action needed. Choose this if you plan to keep experimenting with the pipeline or try different GitHub usernames.

  • Ollama continues running in the background and starts automatically when Windows boots.
  • Your project files, virtual environment, and ChromaDB data stay on disk.
  • The Llama 3.2 model (~2GB) and the BGE-small embedding model cache remain stored locally.

Pause - I'll come back to this later

Shut down running processes to free up memory and GPU resources while keeping all your files on disk.

  • Close VS Code.
  • Right-click the Ollama icon in your system tray and select Quit Ollama. This stops the background server and frees your GPU memory.
  • Your project files, virtual environment, ChromaDB data, and downloaded models all remain on disk for next time.

Delete - I don't want to use this again

Remove all project files, models, and Ollama to free up disk space.

  • Close VS Code.
  • Right-click the Ollama icon in your system tray and select Quit Ollama to stop the background server.
  • Remove the Llama 3.2 model from Ollama by running this command in PowerShell:
ollama rm llama3.2
  • Uninstall Ollama. Press the Windows key and search for Add or remove programs.
  • Find Ollama in the list and click Uninstall.
  • Delete your project folder from your Desktop by running this command in PowerShell:
cd ~\Desktop
Remove-Item -Recurse -Force .\[[PROJECT_FOLDER="your project folder name"]]
  • Remove the Ollama configuration and any remaining model data by running:
Remove-Item -Recurse -Force "$env:USERPROFILE\.ollama"
  • Remove the Hugging Face model cache for BGE-small by running:
Remove-Item -Recurse -Force "$env:USERPROFILE\.cache\huggingface\hub\models--BAAI--bge-small-en-v1.5"

Nice Work!

Nice Work!

Nice work! You just built a complete ML pipeline that scrapes GitHub issues, cleans them with a local LLM, embeds them into vectors, and recommends open-source issues tailored to a developer's skills.

You've learned how to:

  • Ingested live data from the GitHub REST API using a scraper that handles pagination and rate limiting.
  • Cleaned raw issue text into structured JSON using a local LLM (Llama 3.2 via Ollama) with enforced schema output.
  • Built a full vector search pipeline: embedded text with BGE-small via sentence-transformers, stored vectors in ChromaDB, and matched developer profiles to issues using cosine similarity.
  • Secret Mission: Added an LLM-powered reranker that scores each recommendation and explains why the issue suits the developer.

Ready to quiz yourself?