Build an AI Support Ticket Router

Train and evaluate a local ML model that routes support tickets.

Introduction

30 Second Summary

Support messages rarely use the exact words a routing rule expects. That leaves valid requests without a useful destination.

In this project, you will build a local terminal app that routes support tickets into three queues: account, billing, or technical. The project moves from a failed keyword baseline through an evaluated text classifier to a documented GitHub demo.

What You'll Build

Your terminal demo shows the brittle router returning unknown beside Expected: billing, followed by an evaluated model ready to predict queues for your own support messages.

By the end of this project, you'll have:

  • A visible keyword failure that shows an ambiguous billing ticket falling into unknown.
  • An evaluated text-classification pipeline with held-out accuracy, precision, recall, and F1 output.
  • An interactive portfolio demo that routes new messages from the terminal and is documented in a public GitHub repository.
  • Secret Mission: Add a confidence gate that sends uncertain predictions to a human reviewer.

Are there any prerequisites?

You should be comfortable with small Python projects plus basic functions, lists, loops, and terminal commands.

You also need a Windows laptop with Python, Visual Studio Code, Git, and a GitHub account ready to use.

Before We Start

Before the hands-on work begins, this step gives you a clear picture of the local AI support-ticket router you are building. The finished project becomes portfolio evidence of how you evaluate a trained text classifier against keyword rules across account, billing, and technical queues.

Set Up the Repository and ML Environment

Your router needs a shared home where someone else can inspect the code. A public GitHub repository gives the project that professional starting point.

Python package conflicts can make a machine-learning project difficult to reproduce. You will use Visual Studio Code to isolate scikit-learn 1.9.1 inside a project-specific virtual environment.

In this step, get ready to:
  • Create a public GitHub repository with a starter README.
  • Clone the repository into Visual Studio Code.
  • Prepare an isolated Python environment with scikit-learn 1.9.1.
Create and clone the GitHub repository

The repository becomes the public home for your finished router. Initializing it with a README also gives the cloned project a file from the start.

  • Sign in to GitHub with your existing account.
  • Select New repository from the top-right menu.
  • Enter ai-support-ticket-router in Repository name.
  • Choose Public.
  • Select Add a README file.
  • Click Create repository.

That is the project's shared home in place. You should see the repository page with its starter README.

Why Start With a README?

A README explains what a repository contains. Starting with one also lets GitHub create the initial commit that you clone to your computer.

  • Click Code above the repository files.
  • Select HTTPS.
  • Copy the repository URL.
  • Press the Windows key to open the Windows search bar.
  • Type Command Prompt into the search bar.
  • Press Enter to open Command Prompt.
  • Move to your Desktop by running this command:
cd /d "%USERPROFILE%\Desktop"

What Does This Command Do?

The cd /d command changes the current drive and folder. %USERPROFILE% resolves to your Windows user folder.

  • Clone the repository by replacing YOUR-USERNAME with your GitHub username in this command before you run it:
git clone https://github.com/YOUR-USERNAME/ai-support-ticket-router

What Does Git Clone Do?

Git uses git clone to copy the repository history and files onto your computer. The copy stays connected to the GitHub repository for future pushes.

Command Prompt should confirm that repository data was received. You should also see a new ai-support-ticket-router folder on your Desktop.

  • Enter the cloned folder and open it in Visual Studio Code by running these commands:
cd ai-support-ticket-router
code .

What Do These Commands Do?

The first command moves Command Prompt into ai-support-ticket-router. The second command opens that current folder in Visual Studio Code.

  • Wait for Visual Studio Code to show ai-support-ticket-router in the Explorer sidebar.
  • Press Ctrl+` to open the integrated terminal.
  • Select the Launch Profile dropdown in the terminal panel.
  • Select Command Prompt.

The Command Prompt terminal opens inside the cloned ai-support-ticket-router folder. Your local work now has a direct path back to GitHub.

Code Command Not Working?

Use Windows search to open Visual Studio Code if Command Prompt does not recognize code .. Select File followed by Open Folder.

Choose the ai-support-ticket-router folder from your Desktop. Help me open my cloned repository in Visual Studio Code.

Prepare the Python environment

scikit-learn 1.9.1 requires Python 3.11 or newer. Checking the interpreter first prevents a confusing package installation failure.

  • Check the Python version from the Command Prompt terminal by running:
py --version

What Does This Command Check?

The Python launcher prints the version available to your Windows terminal. Use the result to choose the matching tab below.

✔️ I see version 3.11 or higher

Your Python interpreter meets the project's requirement. You can continue to the virtual environment setup below.

ⓧ I see an older version

The installed interpreter is too old for scikit-learn 1.9.1. Install a current 64-bit Python 3 release before creating the environment.

  • Visit the official Python downloads page.
  • Download the current 64-bit Python 3 release for Windows.
  • Run the downloaded installer.
  • Keep the installer's recommended options.
  • Complete the installation.
  • Return to Visual Studio Code.
  • Open a new Command Prompt terminal from the Launch Profile dropdown.
  • Confirm the upgraded version by running:
py --version

What Should I See?

The new terminal should report Python 3.11 or higher. A fresh terminal picks up the newly installed interpreter.

ⓧ Command not found

Windows cannot find a Python installation through the launcher. Install a current 64-bit Python 3 release before continuing.

  • Visit the official Python downloads page.
  • Download the current 64-bit Python 3 release for Windows.
  • Run the downloaded installer.
  • Keep the installer's recommended options.
  • Complete the installation.
  • Return to Visual Studio Code.
  • Open a new Command Prompt terminal from the Launch Profile dropdown.
  • Confirm the installation by running:
py --version

What Should I See?

The new terminal should print Python 3.11 or higher. That result confirms the Python launcher is available.

A virtual environment gives this repository its own package installation area. Packages inside .venv stay separate from your other Python projects.

  • Create the project-specific virtual environment by running:
py -m venv .venv

What Does This Command Do?

Python creates a .venv folder containing an isolated interpreter and package directory. This makes the project's dependencies reproducible.

  • Confirm the Explorer sidebar now lists the .venv folder.

Virtual Environment Missing?

Confirm the terminal path ends with ai-support-ticket-router. Reopen the folder in Visual Studio Code if the terminal started somewhere else.

Help me create the .venv folder in my Windows project.

  • Activate the virtual environment by running:
.venv\Scripts\activate

What Does Activation Change?

Activation makes this terminal use the interpreter and package installer inside .venv. New packages now stay attached to this repository.

You should see the environment name at the start of the Command Prompt line. That prompt confirms the isolated environment is active.

Environment Did Not Activate?

Make sure the terminal profile is Command Prompt. The activation command in this project uses the Windows Command Prompt path.

Help me activate my .venv environment in Command Prompt.

Finish the reproducible environment

The dependency file records the exact package version required by the router. The ignore file keeps generated environment files out of future Git commits.

  • Select the Explorer view in the Visual Studio Code Activity Bar.
  • Select New File... beside the ai-support-ticket-router folder.
  • Enter requirements.txt as the file name.
  • Press Enter to create the file.
  • Add the dependency pin by pasting this content:
scikit-learn==1.9.1

What Does This File Control?

The double equals sign pins scikit-learn to version 1.9.1. Anyone installing from this file receives the same package version.

  • Press Ctrl+S to save requirements.txt.
  • Confirm the Explorer sidebar lists requirements.txt.

Requirements File Looks Different?

Check that the file contains one line. Confirm the version uses two equals signs.

Help me check my requirements.txt dependency pin.

  • Select New File... beside the ai-support-ticket-router folder.
  • Enter .gitignore as the file name.
  • Press Enter to create the file.
  • Add the ignored folders by pasting this content:
.venv/
__pycache__/

What Does This File Exclude?

The .venv/ entry excludes the local environment from Git. The __pycache__/ entry excludes Python's generated cache folders.

  • Press Ctrl+S to save .gitignore.
  • Confirm the Explorer sidebar lists .gitignore.

Gitignore File Missing?

Confirm the file name starts with a period. Check that Visual Studio Code did not add a .txt extension.

Help me create the .gitignore file with the correct name.

✔️ Awesome, I've got everything!

Great. Double-check that both files are saved before installing the dependency.

ⓧ I'd like to double check the full code

Compare your two environment files with these exact versions.

scikit-learn==1.9.1

What Should Requirements Contain?

The file contains one exact dependency pin for scikit-learn 1.9.1.

.venv/
__pycache__/

What Should Git Ignore?

The file excludes the local virtual environment and generated Python cache folders.

Your active environment can now install the exact dependency recorded in requirements.txt. The installation output provides the first check that the environment is using the file correctly.

  • Install the project dependency by running:
pip install -r requirements.txt

What Does This Command Do?

pip reads requirements.txt inside the active environment. It installs the pinned scikit-learn package with the libraries that package requires.

You should see package download and installation progress in the terminal. The command returns to the environment prompt when installation finishes.

Installation Failed?

Check that the terminal still shows the active environment name. Confirm requirements.txt is saved inside ai-support-ticket-router.

Help me troubleshoot the scikit-learn installation.

Before you inspect the installed package, which version do you expect the active environment to report?

  • Verify the installed scikit-learn package by running:
python -m pip show scikit-learn

What Does This Verification Show?

The command asks the active Python interpreter for scikit-learn's installed package details. This confirms the package came from the current virtual environment.

You should see package details containing Version: 1.9.1. Your repository now has a confirmed interpreter and a reproducible machine-learning dependency.

Version Is Missing or Different?

Confirm the environment name still appears in the terminal prompt. Rerun the requirements installation after saving the exact dependency pin.

Help me verify scikit-learn 1.9.1 in my virtual environment.

Your public repository is cloned and your isolated ML environment is ready. Next, you will build a keyword router that works just well enough to expose its own weakness.

Build a Router That Fails

Your repository now has an isolated environment with its machine-learning dependency installed. You are ready to build the first working version of the support router.

This step builds a rule-based baseline that matches literal words to three support queues. You will test it with a billing message whose phrasing stretches those rules.

In this step, get ready to:
  • Define keyword rules for account, billing, and technical tickets.
  • Test an ambiguous billing message against the baseline.
Build the keyword baseline

Each queue gets a short list of words associated with its support category. The routing function scans a ticket for those words before choosing a queue.

  • In the Visual Studio Code file sidebar, use the new-file control to create router.py inside ai-support-ticket-router.
  • Paste the following baseline into router.py:
KEYWORD_RULES = {
    "account": ["password", "login", "account"],
    "billing": ["charged", "refund", "invoice"],
    "technical": ["crash", "error", "slow"],
}

AMBIGUOUS_TICKET = "I paid for Pro, but the dashboard still shows the free plan."


def keyword_route(ticket):
    ticket_lower = ticket.lower()
    for category, keywords in KEYWORD_RULES.items():
        if any(keyword in ticket_lower for keyword in keywords):
            return category
    return "unknown"

What does this code do?

  • The KEYWORD_RULES dictionary maps each queue to the words that trigger it.
  • The AMBIGUOUS_TICKET variable holds the billing message you will test.
  • The keyword_route() function converts the ticket to lowercase so capitalization cannot block a match.
  • The any(...) check returns the first category with a matching keyword.
  • Save router.py by pressing Ctrl+S.
  • Confirm router.py remains visible inside the ai-support-ticket-router folder in the file sidebar.

Seeing syntax warnings?

  • Check that every queue line ends with a comma.
  • Match the indentation inside keyword_route() to the code above.

help me find the syntax problem in my keyword baseline.

Expose the ambiguous ticket

The routing logic needs a visible comparison between its chosen route and the ticket's intended destination. A small display function makes that comparison repeatable whenever you run the file.

  • Move your cursor two blank lines below return "unknown" in router.py.
  • Add the following code at the end of the file:
def show_baseline_obstacle():
    print("Keyword baseline")
    print(f"Ticket: {AMBIGUOUS_TICKET}")
    print(f"Route: {keyword_route(AMBIGUOUS_TICKET)}")
    print("Expected: billing")


if __name__ == "__main__":
    show_baseline_obstacle()

How does the comparison work?

  • The show_baseline_obstacle() function prints the ticket beside its selected route.
  • The expected label provides a direct comparison with the intended billing queue.
  • The main guard calls the display function when Python runs router.py directly.
  • Save router.py by pressing Ctrl+S.

Before you run it, ask yourself which route the literal keyword rules will choose.

  • Run the keyword router in the activated VS Code terminal by entering this command:
python router.py

What does this command do?

The command asks Python to execute router.py. The main guard starts the baseline comparison.

You will see Route: unknown beside Expected: billing.

That mismatch is the intended result. Your working router has exposed its own limitation.

Why did the router miss it?

The billing rule only checks charged, refund, or invoice.

The ticket expresses a billing problem with words such as paid and plan. Literal matching misses the underlying intent.

Missing the baseline mismatch?

  • Confirm you saved router.py after adding show_baseline_obstacle().
  • Check that the terminal path ends with ai-support-ticket-router.
  • Check that show_baseline_obstacle() is indented beneath if __name__ == "__main__":.

help me debug why my router is missing the expected baseline output.

✔️ Awesome, I've got everything!

Great. Your keyword baseline is producing the designed mismatch.

ⓧ I'd like to double check the full code

Compare your complete router.py with this reference. Your existing requirements.txt and .gitignore remain unchanged.

KEYWORD_RULES = {
    "account": ["password", "login", "account"],
    "billing": ["charged", "refund", "invoice"],
    "technical": ["crash", "error", "slow"],
}

AMBIGUOUS_TICKET = "I paid for Pro, but the dashboard still shows the free plan."


def keyword_route(ticket):
    ticket_lower = ticket.lower()
    for category, keywords in KEYWORD_RULES.items():
        if any(keyword in ticket_lower for keyword in keywords):
            return category
    return "unknown"


def show_baseline_obstacle():
    print("Keyword baseline")
    print(f"Ticket: {AMBIGUOUS_TICKET}")
    print(f"Route: {keyword_route(AMBIGUOUS_TICKET)}")
    print("Expected: billing")


if __name__ == "__main__":
    show_baseline_obstacle()

How do the pieces connect?

The rules provide the lookup terms. keyword_route() chooses a queue.

The show_baseline_obstacle() function makes the mismatch visible every time you run the file.

You now have a working baseline plus clear evidence of its blind spot. Next, you will build a model that learns routing patterns from labeled examples.

Train the Machine Learning Pipeline

Your keyword router works on familiar words. The ambiguous paid-plan ticket showed that fixed rules can miss the meaning of a message.

A machine learning classifier learns patterns from labeled examples. You will use scikit-learn to build a reusable TF-IDF pipeline. You will measure its mistakes on unseen tickets.

In this step, get ready to:
  • Create 45 balanced synthetic tickets across three support queues.
  • Train a text-classification pipeline with a reproducible data split.
  • Evaluate held-out tickets before routing the ambiguous billing message again.
Create the labeled training data

A classifier needs examples paired with correct answers. Each tuple in data.py will contain one support message plus its target queue.

  • Select the ai-support-ticket-router folder in the VS Code Explorer sidebar.
  • Click the New File icon beside the folder name.
  • Enter data.py as the file name.
  • Press Enter to create the file.
  • Add the account examples by pasting this code into data.py:
TRAINING_DATA = [
    ("I forgot my password and cannot sign in", "account"),
    ("The password reset link never arrived in my email", "account"),
    ("My account is locked after too many login attempts", "account"),
    ("I need to change the email on my account", "account"),
    ("The two factor authentication code is not working", "account"),
    ("I cannot create a new user profile", "account"),
    ("Please delete my account", "account"),
    ("My username is not recognized", "account"),
    ("I want to update my phone number", "account"),
    ("I was signed out and cannot log back in", "account"),
    ("My profile name is incorrect", "account"),
    ("I cannot verify my email address", "account"),
    ("Someone else may have accessed my account", "account"),
    ("I need to recover an old account", "account"),
    ("The login page says my credentials are invalid", "account"),
]

How is the training data structured?

  • Each tuple keeps one support message beside its correct label.
  • The account examples cover sign-in problems.
  • The examples also cover profile changes plus account security issues.
  • Fifteen examples give this category the same weight as the other categories.
  • Save data.py by pressing Ctrl+S.
  • Press Ctrl+F to open the search field.
  • Enter "account" in the search field.

You should see 15 matches. This confirms that the first category contains 15 labeled examples.

Account count looks wrong?

Check that every tuple ends with "account". Remove any duplicated lines.

If the count still looks wrong, help me check the account examples in my training data.

Balanced training data gives every queue the same number of examples. The next group teaches the model how payment language maps to billing.

  • Find the closing square bracket at the bottom of data.py.
  • Insert the billing examples immediately above that bracket by pasting this code:
    ("I was charged twice for the same month", "billing"),
    ("Please send a copy of my invoice", "billing"),
    ("I need a refund for my last payment", "billing"),
    ("My credit card payment was declined", "billing"),
    ("I want to cancel my paid subscription", "billing"),
    ("The price on my bill is incorrect", "billing"),
    ("My renewal happened earlier than expected", "billing"),
    ("I paid for premium but my plan did not upgrade", "billing"),
    ("How can I change my billing address", "billing"),
    ("There is an unknown charge on my statement", "billing"),
    ("My discount was not applied at checkout", "billing"),
    ("I need to update my payment method", "billing"),
    ("Why did my monthly fee increase", "billing"),
    ("The trial ended and I was billed", "billing"),
    ("My receipt has the wrong company name", "billing"),

What do these examples teach?

The billing examples cover charges plus invoices. They also cover refunds plus subscriptions.

Several examples describe plan upgrades. Those phrases help the model recognize the intent of the ambiguous ticket.

  • Save data.py.
  • Replace the search text with "billing".

You should see 15 matches. The account queue now has the same number of examples as the billing queue.

Technical tickets complete the three-way classification problem. These examples describe observable software failures.

  • Find the closing square bracket at the bottom of data.py again.
  • Insert the technical examples immediately above that bracket by pasting this code:
    ("The app crashes whenever I upload a file", "technical"),
    ("The dashboard shows a blank white page", "technical"),
    ("I see an error when I save my work", "technical"),
    ("The website is extremely slow today", "technical"),
    ("Notifications are not appearing", "technical"),
    ("The export button does nothing", "technical"),
    ("My files are stuck while syncing", "technical"),
    ("The mobile app freezes on startup", "technical"),
    ("The search feature returns no results", "technical"),
    ("The page keeps reloading in a loop", "technical"),
    ("Images fail to load in the browser", "technical"),
    ("The API request returns a server error", "technical"),
    ("Video playback has no sound", "technical"),
    ("The desktop application will not open", "technical"),
    ("The report download stops halfway", "technical"),

Why vary the failure descriptions?

Technical problems can appear as crashes or blank pages. They can also appear as slow performance or failed downloads.

The varied wording gives TF-IDF more patterns to compare across tickets.

  • Save data.py.
  • Replace the search text with "technical".

You should see 15 matches. Your dataset now contains 45 balanced tickets.

The model needs messages plus labels as separate sequences. Two list comprehensions unpack the tuples while preserving their order.

  • Move below the closing square bracket in data.py.
  • Create the separate model inputs by pasting this code:
TEXTS = [text for text, _ in TRAINING_DATA]
LABELS = [label for _, label in TRAINING_DATA]

What do the derived lists contain?

  • TEXTS stores the 45 support messages used as model input.
  • LABELS stores the correct queue for every message.
  • The underscore ignores the tuple value that each comprehension does not need.
  • Save data.py.
  • Confirm that TEXTS appears above LABELS at the bottom of the file.

Training data shows a syntax error?

Confirm that every ticket has matching parentheses. Check that the list ends with one closing square bracket.

If the syntax still looks wrong, help me find the error in my training data.

Build the classification pipeline

A classification pipeline keeps text conversion beside the classifier that uses it. Future tickets pass through the same transformations as the training messages.

Why use a pipeline?

Manual preprocessing can drift between training plus prediction. A pipeline packages the vectorizer with the classifier.

That single object can train or score or predict without separate transformation steps.

  • Switch back to router.py.
  • Move to the first line of the file.
  • Add the model imports above KEYWORD_RULES by pasting this code:
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics import classification_report
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import ComplementNB
from sklearn.pipeline import Pipeline

from data import LABELS, TEXTS

What do these imports provide?

  • TfidfVectorizer converts ticket language into numeric features.
  • ComplementNB supplies the classifier for the three queue labels.
  • Pipeline connects the feature extraction stage to the classifier.
  • LABELS plus TEXTS bring the training data into the router.
  • Save router.py.
  • Check that Python can load the new imports by running:
python router.py

What does this check prove?

You should see Route: unknown beside Expected: billing. Reaching that output proves Python loaded the new data file plus the model imports.

Imports fail when the router runs?

Confirm that the terminal prompt still includes .venv. Check that data.py sits beside router.py.

If an import still fails, help me debug the imports in my ticket router.

A train-test split holds back part of the dataset before training. Those unseen tickets provide a separate test of what the model learned.

  • Find the end of show_baseline_obstacle() in router.py.
  • Add the model-building function below it by pasting this code:
def build_model():
    X_train, X_test, y_train, y_test = train_test_split(
        TEXTS,
        LABELS,
        test_size=0.25,
        random_state=42,
        stratify=LABELS,
    )

    model = Pipeline(
        [
            ("tfidf", TfidfVectorizer(ngram_range=(1, 2))),
            ("classifier", ComplementNB()),
        ]
    )
    model.fit(X_train, y_train)
    return model, X_test, y_test

How does the model learn?

  • The split reserves 25 percent of the tickets for testing.
  • The fixed random state makes the split reproducible.
  • Stratification preserves the category proportions in both groups.
  • TF-IDF extracts single words plus two-word sequences.
  • Complement Naive Bayes learns which features are associated with each queue.
  • The function returns the fitted model plus the held-out data.
  • Save router.py.
  • Check that Python can parse build_model() by running:
python router.py

What should you see?

You should reach the complete keyword baseline output without a syntax error. Python has successfully parsed build_model().

Model function stops the script?

Check the indentation inside build_model(). Confirm that the pipeline brackets close before model.fit.

If the function still fails to parse, help me fix the model-building function.

Evaluate held-out tickets

A classification report shows precision plus recall plus F1 score for every queue. Accuracy gives one overall summary of the held-out predictions.

  • Find the end of build_model() in router.py.
  • Add the evaluation function below it by pasting this code:
def show_evaluation(model, X_test, y_test):
    predictions = model.predict(X_test)
    accuracy = model.score(X_test, y_test)
    print("\nModel evaluation")
    print(f"Accuracy: {accuracy:.2f}")
    print(classification_report(y_test, predictions, zero_division=0))

What does the evaluation measure?

  • Predictions stores the model's chosen queue for every held-out ticket.
  • Accuracy measures the share of held-out tickets assigned to the correct queue.
  • The classification report breaks performance down by queue.
  • The small synthetic dataset makes these metrics evidence of the workflow.
  • Save router.py.
  • Check that Python can parse show_evaluation() by running:
python router.py

What does this run confirm?

You should reach the keyword baseline output without an error. Python has successfully parsed the evaluation function.

The final flow keeps the failed baseline visible before showing the trained result. One command will now demonstrate both approaches.

  • Find the final block that directly calls show_baseline_obstacle().
  • Delete that final block.
  • Add the complete program flow at the bottom of router.py by pasting this code:
def main():
    show_baseline_obstacle()
    model, X_test, y_test = build_model()
    show_evaluation(model, X_test, y_test)
    ml_route = model.predict([AMBIGUOUS_TICKET])[0]
    print(f"ML route: {ml_route}")


if __name__ == "__main__":
    main()

How does the complete flow work?

  • Main runs the keyword baseline first.
  • The model trains on the training portion of the 45 tickets.
  • The evaluation uses only the held-out tickets.
  • The fitted model routes the original ambiguous ticket at the end.
  • Save router.py.

Before you run the complete pipeline, do you think the learned text patterns will recover the billing intent that the keyword rules missed?

  • Train plus evaluate the completed router by running:
python router.py

What should you see?

The keyword section prints Route: unknown beside Expected: billing.

The model section prints an accuracy value. A classification report follows for the three queues.

The final line prints ML route: billing. The learned model now resolves the ambiguous paid-plan ticket.

Model output missing or incomplete?

Confirm that main() calls build_model() before show_evaluation(). Check that data.py contains 15 examples for every label.

If the final route is missing, help me debug the evaluation flow.

That closes the gap exposed by your baseline. Your trained model now routes the ambiguous paid-plan message to billing.

✔️ Awesome, I've got everything!

Your training data plus model code are complete. Double check that both files are saved before moving on.

ⓧ I'd like to double check the full code

Compare your complete data.py with this reference:

TRAINING_DATA = [
    ("I forgot my password and cannot sign in", "account"),
    ("The password reset link never arrived in my email", "account"),
    ("My account is locked after too many login attempts", "account"),
    ("I need to change the email on my account", "account"),
    ("The two factor authentication code is not working", "account"),
    ("I cannot create a new user profile", "account"),
    ("Please delete my account", "account"),
    ("My username is not recognized", "account"),
    ("I want to update my phone number", "account"),
    ("I was signed out and cannot log back in", "account"),
    ("My profile name is incorrect", "account"),
    ("I cannot verify my email address", "account"),
    ("Someone else may have accessed my account", "account"),
    ("I need to recover an old account", "account"),
    ("The login page says my credentials are invalid", "account"),
    ("I was charged twice for the same month", "billing"),
    ("Please send a copy of my invoice", "billing"),
    ("I need a refund for my last payment", "billing"),
    ("My credit card payment was declined", "billing"),
    ("I want to cancel my paid subscription", "billing"),
    ("The price on my bill is incorrect", "billing"),
    ("My renewal happened earlier than expected", "billing"),
    ("I paid for premium but my plan did not upgrade", "billing"),
    ("How can I change my billing address", "billing"),
    ("There is an unknown charge on my statement", "billing"),
    ("My discount was not applied at checkout", "billing"),
    ("I need to update my payment method", "billing"),
    ("Why did my monthly fee increase", "billing"),
    ("The trial ended and I was billed", "billing"),
    ("My receipt has the wrong company name", "billing"),
    ("The app crashes whenever I upload a file", "technical"),
    ("The dashboard shows a blank white page", "technical"),
    ("I see an error when I save my work", "technical"),
    ("The website is extremely slow today", "technical"),
    ("Notifications are not appearing", "technical"),
    ("The export button does nothing", "technical"),
    ("My files are stuck while syncing", "technical"),
    ("The mobile app freezes on startup", "technical"),
    ("The search feature returns no results", "technical"),
    ("The page keeps reloading in a loop", "technical"),
    ("Images fail to load in the browser", "technical"),
    ("The API request returns a server error", "technical"),
    ("Video playback has no sound", "technical"),
    ("The desktop application will not open", "technical"),
    ("The report download stops halfway", "technical"),
]

TEXTS = [text for text, _ in TRAINING_DATA]
LABELS = [label for _, label in TRAINING_DATA]

This reference contains the 45 balanced tickets plus the two derived lists.

Compare your complete router.py with this reference:

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics import classification_report
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import ComplementNB
from sklearn.pipeline import Pipeline

from data import LABELS, TEXTS

KEYWORD_RULES = {
    "account": ["password", "login", "account"],
    "billing": ["charged", "refund", "invoice"],
    "technical": ["crash", "error", "slow"],
}

AMBIGUOUS_TICKET = "I paid for Pro, but the dashboard still shows the free plan."


def keyword_route(ticket):
    ticket_lower = ticket.lower()
    for category, keywords in KEYWORD_RULES.items():
        if any(keyword in ticket_lower for keyword in keywords):
            return category
    return "unknown"


def show_baseline_obstacle():
    print("Keyword baseline")
    print(f"Ticket: {AMBIGUOUS_TICKET}")
    print(f"Route: {keyword_route(AMBIGUOUS_TICKET)}")
    print("Expected: billing")


def build_model():
    X_train, X_test, y_train, y_test = train_test_split(
        TEXTS,
        LABELS,
        test_size=0.25,
        random_state=42,
        stratify=LABELS,
    )

    model = Pipeline(
        [
            ("tfidf", TfidfVectorizer(ngram_range=(1, 2))),
            ("classifier", ComplementNB()),
        ]
    )
    model.fit(X_train, y_train)
    return model, X_test, y_test


def show_evaluation(model, X_test, y_test):
    predictions = model.predict(X_test)
    accuracy = model.score(X_test, y_test)
    print("\nModel evaluation")
    print(f"Accuracy: {accuracy:.2f}")
    print(classification_report(y_test, predictions, zero_division=0))


def main():
    show_baseline_obstacle()
    model, X_test, y_test = build_model()
    show_evaluation(model, X_test, y_test)
    ml_route = model.predict([AMBIGUOUS_TICKET])[0]
    print(f"ML route: {ml_route}")


if __name__ == "__main__":
    main()

This reference preserves the keyword baseline. It adds model training plus held-out evaluation.

Confirm that your existing requirements.txt still matches this reference:

scikit-learn==1.9.1

This pin keeps the project on scikit-learn version 1.9.1.

Confirm that your existing .gitignore still matches this reference:

.venv/
__pycache__/

These entries keep the virtual environment plus Python cache files out of Git.

Your router now learns from labeled tickets. Next up, you will turn it into an interactive demo that another person can test.

Publish the Portfolio Demo

Your classifier now routes the ambiguous billing ticket that defeated the keyword baseline. The next goal is turning that working model into a demo someone else can run.

An isolated training script asks an interviewer to infer too much. A public GitHub repository gives them a clear path to inspect your work.

In this step, get ready to:
  • Add an interactive prompt for account, billing, or technical tickets.
  • Document the project approach with clear usage instructions.
  • Publish the finished demo to the public repository.
Add the interactive demo

A portfolio demo should let another person try the classifier with their own message. The input loop keeps accepting tickets until they enter quit.

  • Select router.py in the VS Code Explorer sidebar.
  • Place the new function below show_evaluation() by pasting this code:
def interactive_demo(model):
    print("\nInteractive router")
    print("Type a support ticket, or type quit to stop.")
    while True:
        ticket = input("Ticket: ").strip()
        if ticket.lower() == "quit":
            print("Demo finished.")
            break
        if not ticket:
            print("Enter a message or type quit.")
            continue
        route = model.predict([ticket])[0]
        print(f"Predicted queue: {route}")

What does this code do?

  • The loop keeps the demo available for repeated predictions.
  • The empty-message check prevents the model from receiving blank input.
  • The model.predict() call turns each ticket into a predicted queue.
  • The quit check gives the learner a clean way to stop the program.
  • Inside main(), find the line that prints the machine-learning route:
    print(f"ML route: {ml_route}")

Why find this line?

This line marks the end of the existing evaluation flow. The interactive demo belongs directly after it.

  • Add the function call directly below that line so the end of main() looks like this:
    print(f"ML route: {ml_route}")
    interactive_demo(model)

How does the demo start?

The new call passes the fitted model into interactive_demo(). The prompt starts after the evaluation output finishes.

  • Save router.py.

Before you run the script, do you expect it to stop after the machine-learning route or wait for a support ticket?

  • Start the updated router by running this command in the activated terminal:
python router.py

What does this command check?

Python runs the complete routing workflow. The final prompt proves that the new function receives the trained model.

You should see the existing evaluation output followed by Interactive router and a Ticket: prompt.

  • Enter quit at the Ticket: prompt.

You should see Demo finished. before the terminal returns to the command prompt.

The ticket prompt does not appear?

Check that interactive_demo(model) is indented inside main(). Confirm that the function definition appears above main().

If the terminal reports a syntax problem, compare the quotation marks with the function shown above.

help me debug why interactive_demo does not start in router.py

✔️ Awesome, I've got everything!

Your router now reaches the interactive ticket prompt. Double check that router.py is saved before continuing.

ⓧ I'd like to double check the full code

Compare your saved router.py with this complete version.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics import classification_report
from sklearn.model_selection import train_test_split
from sklearn.naive_bayes import ComplementNB
from sklearn.pipeline import Pipeline

from data import LABELS, TEXTS

KEYWORD_RULES = {
    "account": ["password", "login", "account"],
    "billing": ["charged", "refund", "invoice"],
    "technical": ["crash", "error", "slow"],
}

AMBIGUOUS_TICKET = "I paid for Pro, but the dashboard still shows the free plan."


def keyword_route(ticket):
    ticket_lower = ticket.lower()
    for category, keywords in KEYWORD_RULES.items():
        if any(keyword in ticket_lower for keyword in keywords):
            return category
    return "unknown"


def show_baseline_obstacle():
    print("Keyword baseline")
    print(f"Ticket: {AMBIGUOUS_TICKET}")
    print(f"Route: {keyword_route(AMBIGUOUS_TICKET)}")
    print("Expected: billing")


def build_model():
    X_train, X_test, y_train, y_test = train_test_split(
        TEXTS,
        LABELS,
        test_size=0.25,
        random_state=42,
        stratify=LABELS,
    )

    model = Pipeline(
        [
            ("tfidf", TfidfVectorizer(ngram_range=(1, 2))),
            ("classifier", ComplementNB()),
        ]
    )
    model.fit(X_train, y_train)
    return model, X_test, y_test


def show_evaluation(model, X_test, y_test):
    predictions = model.predict(X_test)
    accuracy = model.score(X_test, y_test)
    print("\nModel evaluation")
    print(f"Accuracy: {accuracy:.2f}")
    print(classification_report(y_test, predictions, zero_division=0))


def interactive_demo(model):
    print("\nInteractive router")
    print("Type a support ticket, or type quit to stop.")
    while True:
        ticket = input("Ticket: ").strip()
        if ticket.lower() == "quit":
            print("Demo finished.")
            break
        if not ticket:
            print("Enter a message or type quit.")
            continue
        route = model.predict([ticket])[0]
        print(f"Predicted queue: {route}")


def main():
    show_baseline_obstacle()
    model, X_test, y_test = build_model()
    show_evaluation(model, X_test, y_test)
    ml_route = model.predict([AMBIGUOUS_TICKET])[0]
    print(f"ML route: {ml_route}")
    interactive_demo(model)


if __name__ == "__main__":
    main()

What should match?

This reference keeps the baseline, model training, evaluation, and interactive demo in one runnable file. Your version should match it exactly.

Document the project

A README turns the code into a project another person can understand. It also records the limits of the synthetic dataset so the evaluation stays credible.

  • Select README.md in the VS Code Explorer sidebar.
  • Select all existing content in README.md.
  • Replace the selected content by pasting this first section:
# AI Support Ticket Router

A local text-classification project that routes support messages to `account`, `billing`, or `technical` queues.

## Why this project exists

A keyword router can miss messages that express the right intent without using an expected keyword. This project demonstrates that failure, then replaces the rules with a trained and evaluated machine-learning pipeline.

## Approach

- 45 balanced synthetic support tickets
- Stratified train-test split
- TF-IDF text features
- Complement Naive Bayes classifier
- Accuracy and classification-report output
- Interactive terminal predictions

What does this section explain?

  • The opening names the three queues produced by the router.
  • The problem statement connects the keyword failure to the trained classifier.
  • The approach list makes the implementation choices easy to scan.
  • Save README.md.
  • Confirm the editor shows the Why this project exists heading.

The README now explains the problem behind the project. The next section gives another person everything needed to run it responsibly.

  • Place the remaining documentation below the final Interactive terminal predictions list item by pasting this section:
## Setup on Windows

```text
py -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt
```

## Run

```text
python router.py
```

Type a support message at the prompt. Type `quit` to exit.

## Limitations

The dataset is small and synthetic. The reported metrics demonstrate the workflow, not production readiness. A real service would need representative labeled data, broader evaluation, monitoring, privacy controls, and a human-review path.

Why include these details?

  • The setup section makes the environment reproducible on Windows.
  • The run section tells another person how to start the demo.
  • The limitations section separates an educational result from a production claim.
  • Save README.md.
  • Confirm the editor shows the Setup on Windows, Run, and Limitations headings.

✔️ Awesome, I've got everything!

Your README now explains how the router works. It also gives another person a reproducible path to run it.

ⓧ I'd like to double check the full code

Compare your saved README.md with this complete version.

# AI Support Ticket Router

A local text-classification project that routes support messages to `account`, `billing`, or `technical` queues.

## Why this project exists

A keyword router can miss messages that express the right intent without using an expected keyword. This project demonstrates that failure, then replaces the rules with a trained and evaluated machine-learning pipeline.

## Approach

- 45 balanced synthetic support tickets
- Stratified train-test split
- TF-IDF text features
- Complement Naive Bayes classifier
- Accuracy and classification-report output
- Interactive terminal predictions

## Setup on Windows

```text
py -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt
```

## Run

```text
python router.py
```

Type a support message at the prompt. Type `quit` to exit.

## Limitations

The dataset is small and synthetic. The reported metrics demonstrate the workflow, not production readiness. A real service would need representative labeled data, broader evaluation, monitoring, privacy controls, and a human-review path.

What should match?

This reference contains the project summary, approach, setup instructions, run instructions, and limitations. Your saved README should match it exactly.

Test and publish the demo

The final local test proves that the interactive prompt handles two different queues. Publishing then turns the local project into evidence another person can inspect.

Before you run the final test, do you expect the same model to classify both an account message and a technical message?

  • Start the finished demo by running this command in the activated terminal:
python router.py

What does this final run prove?

The script trains the model before opening the interactive prompt. Each message then uses that fitted model for its prediction.

  • Enter I forgot my password and cannot sign in at the first Ticket: prompt.

You should see Predicted queue: account.

  • Enter The app crashes whenever I upload a file at the next Ticket: prompt.

You should see Predicted queue: technical.

  • Enter quit at the final Ticket: prompt.

You should see Demo finished. before the program exits. Your router now accepts messages from another person and returns a visible queue.

A test message reaches the wrong queue?

Confirm that data.py still contains all 45 labeled tickets. Check that interactive_demo() calls the same fitted model created by build_model().

help me trace an unexpected queue prediction in the interactive router

Publishing creates a permanent version of your work in Git. The commit records the finished state before the push sends it to GitHub.

Before you publish, do you expect the updated README and Python files to appear on the main branch?

  • Stage, commit, and publish the finished project by running these commands:
git add .
git commit -m "Build AI support ticket router"
git push

What do these commands do?

  • The first command stages every changed project file.
  • The second command records the staged files with the portfolio-ready message.
  • The third command publishes the commit to the configured upstream branch.

You should see Git record the commit before completing the push. The command prompt should return without a failure.

The commit or push did not complete?

If Git asks for your identity, follow the terminal guidance to configure the required details. If authentication opens in your browser, complete the prompted GitHub sign-in before returning to VS Code.

help me troubleshoot the failed Git commit or push

  • Return to GitHub in your browser.
  • Open the public ai-support-ticket-router repository.
  • Confirm main is the selected branch.
  • Confirm router.py, data.py, requirements.txt, .gitignore, and README.md appear in the file list.
  • Confirm the README renders above the project files.

Strong finish. Your public repository now shows a runnable classifier with clear documentation and honest limitations.

Secret mission

Add a Human-Review Safety Route

Your router currently assigns every ticket to a support queue, even when the message contains no useful clues. Add a confidence gate that sends uncertain predictions to human review and displays the model's confidence as a percentage.

Clean Up Your Resources

Clean Up Your Resources

Your classifier runs locally on your Windows computer. Keeping its public GitHub repository costs $0 on GitHub Free.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • The local virtual environment at .venv, including the installed Python packages.
  • The local Git repository in ai-support-ticket-router, including your project files and commit history.
  • The public GitHub repository named ai-support-ticket-router, including the published documentation and code.

Keep everything running

Keep the current setup unchanged if you want to demonstrate the router or continue improving its predictions.

  • Leave the ai-support-ticket-router folder open in Visual Studio Code.
  • Keep the .venv folder so the project retains its installed dependencies.
  • Leave the public GitHub repository available as your published project.

The router runs only while its terminal program is active. No model process continues after the program exits.

Pause - I'll come back to this later

Pausing releases the active virtual environment from your terminal. Your local files and public repository remain ready for later.

  • Switch back to the integrated terminal in Visual Studio Code.
  • Deactivate the project environment by running this command:
deactivate

What does this command do?

This command exits the active .venv environment. The installed packages remain inside the project folder.

The virtual environment marker should disappear from your terminal prompt.

  • Leave the local ai-support-ticket-router folder on your computer.
  • Leave the public GitHub repository unchanged.

Your program remains stopped until you run it again from the project folder.

Delete - I don't want to use this again

Deleting clears the local project plus its public copy. A full reset can feel drastic because both copies disappear when these actions finish.

Deletion Needs Care

Removing the local folder also removes its virtual environment and Git history.

GitHub warns that repository deletion cannot be undone. Some deleted repositories may be restorable within 90 days.

  • Switch back to the integrated terminal in Visual Studio Code.
  • Deactivate the environment and remove the local repository by running these commands:
deactivate
cd ..
rmdir /s /q ai-support-ticket-router

What do these commands do?

  • The first command exits the active virtual environment.
  • The second command moves the terminal into the folder that contains your repository.
  • The final command removes ai-support-ticket-router, including .venv and the local Git history.
  • Confirm the local repository is gone by running this command:
dir

What does this command do?

This command lists the folders beside the deleted repository. It gives you a visible check that the local copy is gone.

You should no longer see ai-support-ticket-router in the folder listing.

  • Close the Visual Studio Code window that previously showed the deleted folder.
  • Return to the public repository page on GitHub from earlier.
  • Select Settings.
  • Scroll to Danger Zone.
  • Select Delete this repository.
  • Complete GitHub's acknowledgement prompts.
  • Enter ai-support-ticket-router for the repository-name verification.
  • Select Delete this repository to remove the public copy.

The project should no longer appear in your GitHub repositories list. This confirms that the published copy is gone.

Nice Work!

Nice Work!

You did it. You built a support-ticket classifier that learns from labeled messages.

You've learned how to:

  • Build a keyword baseline that produces a visible routing result. You diagnosed its failure on the ambiguous billing ticket.
  • Create 45 balanced synthetic tickets for three support queues. You trained a TF-IDF plus Complement Naive Bayes classification pipeline with a reproducible split. You generated a classification report from held-out predictions.
  • Build an interactive terminal demo for new support messages. You documented its setup plus limitations. You published the finished project in a public GitHub repository.
  • Secret Mission: Added confidence-aware routing through route_with_confidence. Tickets below 60% now go to human_review. The displayed confidence remains a product-routing signal.

Ready to quiz yourself?