Build an AI Support Ticket Router

Train and serve an AI classifier that routes support tickets with human review.

Introduction

30 Second Summary

When support requests pile up, customers can describe the same problem in dozens of different ways. Rigid routing rules can send a customer to the wrong team.

In this project, you will build a local AI service that routes support tickets into billing, technical, account, or shipping queues. Its interactive API sends uncertain tickets to human review.

What You'll Build

Picture entering an unclear support request in your browser and seeing the service pause automatic routing for a person to review.

By the end of this project, you'll have:

  • A runnable keyword baseline that visibly misses a paraphrased shipping ticket. Your multiclass classifier then prints a holdout accuracy score on examples it did not train on.
  • A browser-based API demo where each response shows a predicted queue plus confidence. Vague requests return human_review for a person to assess.
  • A public GitHub repository with reproducible setup instructions. Its README explains the baseline failure, evaluation approach, review policy, and dataset limitations.
  • Secret Mission: Red-team the router with eight difficult tickets. Turn at least two failures into a GitHub issue with two targeted improvements.

Are there any prerequisites?

You should be comfortable reading basic Python. Experience completing at least one personal coding project will help you focus on the engineering decisions.

Before We Start

An employer-facing AI project needs a clear user and a safe decision boundary. This checkpoint commits you to building a support ticket router for support teams, with uncertain predictions escalated to a person.

Set Up the Local Project

Trustworthy model behavior starts with a reproducible workspace. A shared Python installation can collect unrelated packages from other projects.

You'll prepare a Visual Studio Code workspace with Git history. A project-specific virtual environment pins FastAPI and scikit-learn to the versions used throughout this project.

In this step, get ready to:
  • Confirm a supported Python runtime on Windows.
  • Prepare a Git-initialized workspace in Visual Studio Code.
  • Build an isolated environment with pinned dependencies.
Check the runtime and prepare the workspace

The Windows Python launcher lists the runtimes installed on your computer. The selected runtime must be Python 3.11 or newer because that is the minimum supported by the pinned scikit-learn package.

  • Check the installed Python runtimes in PowerShell by running this command:
py list

What does this command show?

The launcher displays the Python runtimes available on Windows. The selected runtime determines which installation later commands use.

✔️ I see a supported version

Your selected runtime is Python 3.11 or newer. That runtime supports both pinned project packages.

ⓧ I see an older version

The selected runtime is below the project's minimum requirement. Install Python 3.11 or newer before creating the environment.

  • Visit the official Python downloads page.
  • Download a Windows installer for Python 3.11 or newer.
  • Run the downloaded installer.
  • Allow the Windows permission request if one appears.
  • Return to PowerShell after the installation finishes.
  • Check the available runtimes again by running this command:
py list

What does this command confirm?

The refreshed list confirms that the newer Python runtime is available to the Windows launcher.

Still seeing the older runtime?

Close PowerShell after the installer finishes. Start a fresh PowerShell session so it can detect the new installation.

Help me make the newer Python runtime available to the Windows launcher. If the older runtime remains selected, share the runtime list with the NextWork community.

ⓧ Command not found

The Python launcher is unavailable in this PowerShell session. Install a supported Python runtime from the official source.

  • Visit the official Python downloads page.
  • Download a Windows installer for Python 3.11 or newer.
  • Run the downloaded installer.
  • Start a fresh PowerShell session after the installation finishes.
  • Check that the launcher is available by running this command:
py list

What does this command confirm?

A runtime list confirms that PowerShell can access the Windows Python launcher.

Launcher still unavailable?

Restart Windows if a fresh PowerShell session still cannot find the launcher. This refreshes the system paths changed by the installer.

Help me diagnose why the Python launcher is unavailable in PowerShell. If the launcher remains unavailable, share the installation details with the NextWork community.

The project folder belongs on your Desktop so it stays easy to find. PowerShell uses its current location when it creates a new folder.

  • Move to your Desktop by running this command:
cd ~/Desktop

What does this command do?

PowerShell changes its current location to your Desktop. The next command creates the project folder there.

The workspace commands create the folder before initializing its Git history. The final command launches Visual Studio Code with that folder open.

  • Create the project workspace and open it by running these commands:
mkdir ai-support-ticket-router
cd ai-support-ticket-router
git init -b main
code .

What do these commands do?

  • The first command creates the ai-support-ticket-router folder on your Desktop.
  • The second command moves PowerShell into the new folder.
  • The third command initializes a local Git repository with main as its first branch.
  • The final command opens the current folder in Visual Studio Code.
  • Approve the trust prompt if Visual Studio Code asks about the folder you created.
  • Confirm the Explorer view shows ai-support-ticket-router as the open folder.

PowerShell reports that Git initialized the repository. Visual Studio Code now uses the local ai-support-ticket-router folder as its workspace.

Workspace command stopped?

The folder and Git repository still exist if the final command fails. Open the existing folder through Visual Studio Code instead.

  • Press the Windows key to open Windows search.
  • Type Visual Studio Code into the search field.
  • Press Enter to open the app.
  • Select File from the top menu.
  • Select Open Folder.
  • Choose the ai-support-ticket-router folder from your Desktop.

Help me open the local project folder or initialize Git on the main branch. If the workspace still does not open, share the PowerShell output with the NextWork community.

Isolate the project packages

A project-specific environment keeps these dependencies separate from packages used elsewhere. Its .venv folder contains the isolated Python environment.

  • Switch back to the PowerShell session inside ai-support-ticket-router.
  • Create the isolated environment by running this command:
py -m venv .venv

What does this command do?

Python creates an isolated environment inside the project's .venv folder. Packages installed after activation stay scoped to this project.

  • Return to the Explorer view in Visual Studio Code.
  • Confirm the .venv folder appears inside ai-support-ticket-router.

You'll see the new .venv folder in the workspace. That folder is the visible boundary around this project's Python packages.

Environment folder missing?

Confirm PowerShell is inside the ai-support-ticket-router folder. A different current location creates .venv somewhere else.

Help me locate or recreate the virtual environment for this project.

Activation makes the environment's Python installation the active one for this PowerShell session. The prompt changes to show that isolation is active.

  • Activate the existing virtual environment by running this command:
.venv\Scripts\Activate.ps1

What does activation change?

Activation points Python package commands at the environment inside .venv. The environment name appears at the beginning of your PowerShell prompt.

✔️ The environment name appears

That's the isolation in place. Future package installations now target this project environment.

ⓧ PowerShell blocked the script

PowerShell's execution policy can block local activation scripts. A current-user policy change allows locally created scripts to run.

PowerShell may ask you to confirm the policy change. Approve the update for your Windows user account.

  • Apply the documented policy change before retrying activation by running these commands:
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser
.venv\Scripts\Activate.ps1

What do these commands change?

The first command updates the execution policy for your Windows user account. The second command activates the existing project environment.

The environment name should now appear at the beginning of your PowerShell prompt.

Activation still blocked?

Confirm PowerShell is inside the ai-support-ticket-router folder. The activation path starts from that folder.

Help me activate this project's virtual environment in PowerShell. If the script remains blocked, share the policy output with the NextWork community.

Add and verify the setup files

Pinned dependencies make the setup reproducible on another computer. The requirements file records the exact package versions used by this project.

  • Switch back to the Explorer view in Visual Studio Code.
  • Use the new-file control to create requirements.txt inside ai-support-ticket-router.
  • Add the pinned dependency list by pasting this code:
fastapi[standard]==0.162.0
scikit-learn==1.9.1

What does this file control?

  • The fastapi[standard] entry installs FastAPI 0.162.0 with its command-line tools and server dependencies.
  • The scikit-learn entry installs version 1.9.1 for the classification pipeline.
  • The exact version pins keep future installations aligned with this project.
  • Press Ctrl+S to save requirements.txt.
  • Confirm the Explorer view lists requirements.txt inside ai-support-ticket-router.

Requirements file looks different?

Confirm the file is named exactly requirements.txt. Each pinned dependency must appear on its own line.

Help me check the pinned dependency file in my project.

The environment folder contains generated files that belong only on your computer. A Git ignore file keeps that folder out of future commits.

  • Use the new-file control in the Explorer view to create .gitignore inside ai-support-ticket-router.
  • Add the environment and cache rules by pasting this code:
.venv/
__pycache__/
*.pyc

What do these rules protect?

  • The .venv/ rule keeps the local virtual environment out of Git.
  • The __pycache__/ rule excludes generated Python cache folders.
  • The *.pyc rule excludes compiled Python files.
  • Press Ctrl+S to save .gitignore.
  • Confirm the Explorer view lists .gitignore beside requirements.txt.

Ignore file missing?

Confirm the filename begins with a full stop. Confirm Visual Studio Code saved it directly inside ai-support-ticket-router.

Help me verify the location and contents of my Git ignore file.

✔️ Awesome, I've got everything!

Both setup files are saved inside ai-support-ticket-router. The local Git repository uses main as its branch.

ⓧ I'd like to double check the full code

Compare each file with these exact contents. Keep both files directly inside ai-support-ticket-router.

fastapi[standard]==0.162.0
scikit-learn==1.9.1

What should match in requirements.txt?

The file contains exactly two pinned package entries. Each entry appears on its own line.

.venv/
__pycache__/
*.pyc

What should match in .gitignore?

The file contains exactly three ignore rules. They cover the local environment folder and generated Python cache files.

The Git repository is initialized in the local ai-support-ticket-router folder with branch main.

The first dependency installation can take a few minutes while package files download. A quiet pause during this command is normal.

  • Switch back to the activated PowerShell session.
  • Install the pinned dependencies by running this command:
py -m pip install -r requirements.txt

What does this command do?

The package installer reads requirements.txt before installing both pinned dependencies into the active environment. Their supporting packages are installed at the same time.

You'll see package download and installation progress in PowerShell. The command returns to the prompt when installation finishes.

Dependency installation failed?

  • Confirm the environment name appears at the beginning of the PowerShell prompt.
  • Confirm requirements.txt is saved inside ai-support-ticket-router.
  • Compare the two package lines with the full-code tab above.
  • Retry the installation command after correcting the environment or file.

Help me diagnose a failed installation from requirements.txt. If the installation still fails, share the package output with the NextWork community.

Before you run the final check, which two pinned package versions do you expect to find in the installed-package list?

  • Print the installed package versions by running this command in the activated environment:
py -m pip freeze

What does this command confirm?

The command prints the packages installed in the active environment with their versions. This confirms whether the pinned dependencies landed inside .venv.

You'll see fastapi==0.162.0 in the output. You'll also see scikit-learn==1.9.1.

That's the reproducible foundation locked in. Your isolated project environment now contains the required packages.

A pinned package is missing?

  • Confirm the environment name appears at the beginning of the PowerShell prompt.
  • Open requirements.txt in Visual Studio Code.
  • Compare its two package lines with the full-code tab above.
  • Rerun the installation command after correcting the file.

Help me find why a pinned dependency is missing from the installed-package list.

Your isolated workspace is ready for repeatable development. Next, you'll build a keyword baseline and watch it fail on an unfamiliar phrase.

Test a Keyword Baseline

Your local project is ready to run Python in an isolated environment. The next question is whether a small routing rule can make a useful decision.

Customer wording varies even when the intent stays the same. This step creates a keyword baseline that handles familiar language before you test it against a paraphrase.

In this step, get ready to:
  • Define four destination vocabularies for support tickets.
  • Route tickets by scoring their keyword matches.
  • Compare the routes for familiar wording and a paraphrase.
Define the queue keywords

A keyword router needs a fixed vocabulary for every destination. The first version uses four queues that mirror common support teams.

  • Create baseline.py inside ai-support-ticket-router using the new-file control in the file sidebar of Visual Studio Code.
  • Add the four queue vocabularies by pasting this code into baseline.py:
KEYWORDS = {
    "billing": ["charge", "charged", "invoice", "payment", "refund"],
    "technical": ["error", "crash", "broken", "login", "password"],
    "account": ["account", "profile", "email", "subscription"],
    "shipping": ["delivery", "shipment", "tracking", "package"],
}

What does this keyword map do?

  • The KEYWORDS dictionary connects each destination label to its vocabulary.
  • Each label represents the queue that receives a matching ticket.
  • Each list gives the baseline words it can recognize for that queue.
  • Save baseline.py.
  • Confirm that baseline.py appears directly under ai-support-ticket-router in the file sidebar.

Good progress. Your first project script now contains the vocabulary that powers the baseline.

Can't see baseline.py?

  • Check the file sidebar for baseline.py under ai-support-ticket-router.
  • Save the file again if its editor tab shows an unsaved-change indicator.
  • Help me find or create baseline.py in my existing VS Code project.
Score each ticket

Matching text is easier after every ticket uses lowercase. Each queue receives a score based on the keywords found in that normalized text.

  • Place the routing function below the closing brace of KEYWORDS by pasting this code:
def route_with_keywords(text: str) -> str:
    normalized_text = text.lower()
    scores = {
        label: sum(keyword in normalized_text for keyword in keywords)
        for label, keywords in KEYWORDS.items()
    }
    best_label = max(scores, key=scores.get)
    return best_label if scores[best_label] > 0 else "unknown"

How does the scoring work?

  • The normalized_text value stores a lowercase copy of the ticket.
  • The scores dictionary counts matching keywords for each label.
  • The best_label value stores the label with the highest score.
  • The final line returns unknown when every score is zero.
  • Save baseline.py.
  • Check that the routing code loads in the activated PowerShell terminal from the setup step by running this command:
py baseline.py

What does this check prove?

PowerShell returns to the prompt without a traceback. That confirms Python can load the keyword map and routing function.

Seeing a Python traceback?

  • Compare the indentation inside route_with_keywords() with the code above.
  • Check that every opening brace has a matching closing brace.
  • Help me fix the traceback in my keyword routing function.
Test familiar and unfamiliar wording

Executable examples turn the routing rules into visible evidence. One ticket uses familiar billing language while the other describes a delivery problem differently.

  • Place the example block below route_with_keywords() by pasting this code:
if __name__ == "__main__":
    examples = [
        "I need a refund for a duplicate charge",
        "The parcel still has not arrived",
    ]

    for example in examples:
        print(f"{example} -> {route_with_keywords(example)}")

What does this test block do?

  • The if __name__ == "__main__": block runs when you execute baseline.py directly.
  • The examples list stores the two tickets being tested.
  • The loop prints each ticket beside the route returned by route_with_keywords().
  • Save baseline.py.

Before you run the baseline, do you expect both tickets to reach a queue?

  • Run both built-in examples in the activated terminal by using this command:
py baseline.py

What does this result show?

The duplicate-charge line ends with billing. Its wording matches the billing vocabulary.

The paraphrased shipping line ends with unknown. Its wording expresses shipping intent without using a listed shipping keyword.

That miss is intentional evidence that exact word rules cannot handle every paraphrase.

Routes don't match the expected result?

  • Compare both ticket strings with the example block above.
  • Check that the loop passes each example into route_with_keywords().
  • Help me debug unexpected routes from baseline.py.

You have captured the baseline's weakness in a reproducible script. The failed paraphrase now gives you evidence for improving the router.

✔️ Awesome, I've got everything!

Great. Double-check that you saved baseline.py.

ⓧ I'd like to double check the full code

KEYWORDS = {
    "billing": ["charge", "charged", "invoice", "payment", "refund"],
    "technical": ["error", "crash", "broken", "login", "password"],
    "account": ["account", "profile", "email", "subscription"],
    "shipping": ["delivery", "shipment", "tracking", "package"],
}


def route_with_keywords(text: str) -> str:
    normalized_text = text.lower()
    scores = {
        label: sum(keyword in normalized_text for keyword in keywords)
        for label, keywords in KEYWORDS.items()
    }
    best_label = max(scores, key=scores.get)
    return best_label if scores[best_label] > 0 else "unknown"


if __name__ == "__main__":
    examples = [
        "I need a refund for a duplicate charge",
        "The parcel still has not arrived",
    ]

    for example in examples:
        print(f"{example} -> {route_with_keywords(example)}")

Your baseline now provides a clear before-state for the project. Next, you will replace exact keyword matching with a classifier that learns from varied ticket language.

Train and Evaluate the Classifier

Your keyword baseline exposed a real weakness when familiar intent appeared in unfamiliar wording. That failure gives this step a clear target: learn language patterns from labeled examples.

A scikit-learn pipeline will combine TF-IDF features with logistic regression. Holdout evaluation will test the trained classifier on examples it did not use for fitting.

In this step, get ready to:
  • Build a balanced dataset for four support queues.
  • Train a text-classification pipeline on a stratified training split.
  • Inspect holdout accuracy plus prediction confidence.
Build a balanced training dataset

A supervised classifier learns from examples paired with their correct labels. Your dataset will contain eight tickets for each queue so every class has equal representation.

  • Click the new-file control in the VS Code Explorer sidebar.
  • Enter model.py as the file name.
  • Add the model imports plus the first two billing examples by pasting this code:
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline

TRAINING_EXAMPLES = [
    ("I was charged twice for the same order", "billing"),
    ("My refund has not arrived", "billing"),
]

What does this code set up?

  • The imports provide text conversion, classification, evaluation, data splitting, and pipeline tools.
  • The TRAINING_EXAMPLES list pairs each ticket with the queue the model should learn.
  • The first two pairs establish the dataset structure before you fill out all four classes.
  • Save model.py.
  • Confirm the file contains five imports plus two labeled billing examples.

Seeing import warnings?

Confirm VS Code is using the Python interpreter from the active .venv. Import warnings can appear when the editor uses a different interpreter.

Help me check the Python interpreter for this project.

The remaining examples give each queue enough varied wording for a stratified split. Balanced labels also stop the largest class from dominating this small dataset.

  • Locate the closing bracket of TRAINING_EXAMPLES in model.py.
  • Insert the remaining labeled examples immediately above that bracket by pasting this code:
    ("The invoice amount is incorrect", "billing"),
    ("My payment was declined", "billing"),
    ("Cancel my plan and refund the payment", "billing"),
    ("There is an unknown charge on my card", "billing"),
    ("I need to update my billing address", "billing"),
    ("Please send a receipt for my purchase", "billing"),
    ("The app crashes during startup", "technical"),
    ("The page shows a server error", "technical"),
    ("I cannot log in after the update", "technical"),
    ("The password reset link is not working", "technical"),
    ("The screen freezes during upload", "technical"),
    ("The submit button does nothing", "technical"),
    ("The website loads a blank page", "technical"),
    ("My two factor code is rejected", "technical"),
    ("I want to change my email address", "account"),
    ("Please delete my account", "account"),
    ("Update the name on my profile", "account"),
    ("I want to upgrade my subscription plan", "account"),
    ("I cannot access my profile settings", "account"),
    ("Please merge my duplicate accounts", "account"),
    ("Change my notification preferences", "account"),
    ("I want to reactivate my closed account", "account"),
    ("Where is my package", "shipping"),
    ("My delivery is late", "shipping"),
    ("The tracking number is not updating", "shipping"),
    ("The parcel arrived damaged", "shipping"),
    ("I need to change the delivery address", "shipping"),
    ("The shipment says delivered but it is missing", "shipping"),
    ("When will my order ship", "shipping"),
    ("The package was returned to the sender", "shipping"),

How is the dataset balanced?

  • Billing examples cover charges, refunds, invoices, payments, addresses, and receipts.
  • Technical examples cover crashes, errors, login failures, broken controls, and rejected codes.
  • Account examples cover profiles, subscriptions, preferences, duplicate accounts, and reactivation.
  • Shipping examples cover packages, deliveries, tracking, damage, addresses, and missing shipments.
  • Save model.py.
  • Confirm TRAINING_EXAMPLES now contains eight examples for each queue.

Does the dataset look incomplete?

Check that every tuple sits inside the TRAINING_EXAMPLES brackets. Confirm each tuple ends with a comma.

Help me find a syntax problem in my training examples.

Train the pipeline on a stratified split

The holdout split reserves one quarter of the examples for evaluation. Stratification preserves the class balance across the training data and the holdout data.

  • Locate the closing bracket of TRAINING_EXAMPLES in model.py.
  • Add the data split plus the fitted pipeline below that bracket by pasting this code:
TEXTS = [text for text, _ in TRAINING_EXAMPLES]
LABELS = [label for _, label in TRAINING_EXAMPLES]

X_TRAIN, X_TEST, Y_TRAIN, Y_TEST = train_test_split(
    TEXTS,
    LABELS,
    test_size=0.25,
    random_state=42,
    stratify=LABELS,
)

MODEL = make_pipeline(
    TfidfVectorizer(ngram_range=(1, 2)),
    LogisticRegression(max_iter=1000),
)
MODEL.fit(X_TRAIN, Y_TRAIN)

How does training work?

  • TEXTS extracts the ticket wording from every labeled pair.
  • LABELS extracts the matching support queues.
  • train_test_split creates reproducible training and holdout sets with random_state=42.
  • TfidfVectorizer learns useful single-word plus two-word features before LogisticRegression learns the class boundaries.
  • MODEL.fit fits the complete pipeline using only X_TRAIN plus Y_TRAIN.
  • Save model.py.
  • Confirm no red syntax-error underlines appear in the new split or pipeline code.

Seeing a syntax error in the split?

Check the closing parentheses for train_test_split plus make_pipeline. Confirm the MODEL.fit line sits outside both constructor calls.

Help me debug the data split or pipeline syntax.

Accuracy measures the fraction of holdout tickets assigned to the correct queue. Evaluating against X_TEST keeps those examples separate from model fitting.

  • Place the evaluation function below MODEL.fit(X_TRAIN, Y_TRAIN).
  • Add the first runnable check by pasting this code:
def evaluate_model() -> float:
    predictions = MODEL.predict(X_TEST)
    return float(accuracy_score(Y_TEST, predictions))


if __name__ == "__main__":
    print(f"Holdout accuracy: {evaluate_model():.2f}")

What does the evaluation check?

  • MODEL.predict assigns a queue to every held-out ticket.
  • accuracy_score compares those predictions with the correct holdout labels.
  • evaluate_model returns the result as a floating-point value.
  • The main block prints the holdout accuracy with two decimal places.
  • Save model.py.

Before you run the file, what result would convince you that the learned pipeline has captured useful patterns?

  • Run the holdout evaluation in the activated VS Code terminal with this command:
py model.py

What does this run prove?

This command trains the pipeline before calling evaluate_model. A printed accuracy value proves the complete training and holdout-evaluation path can run.

You should see Holdout accuracy: followed by a decimal value. That output is your first measured evidence that the learned pipeline can classify held-out tickets.

Did the evaluation fail?

Confirm the terminal still shows the active .venv. Check the first reported line in the traceback for a misspelled identifier or unmatched bracket.

Help me debug the classifier evaluation.

Inspect labels and confidence

A predicted label shows the model's preferred queue. The highest class probability provides a confidence value that later supports a human-review decision.

  • Locate evaluate_model in model.py.
  • Add the prediction function below it by pasting this code:
def predict_ticket(text: str) -> dict[str, str | float]:
    model_prediction = str(MODEL.predict([text])[0])
    probabilities = MODEL.predict_proba([text])[0]
    confidence = float(max(probabilities))

    return {
        "model_prediction": model_prediction,
        "confidence": round(confidence, 3),
    }

How does prediction confidence work?

  • MODEL.predict selects the classifier's preferred queue.
  • MODEL.predict_proba returns one probability estimate for each queue.
  • confidence stores the highest probability from that set.
  • The returned dictionary keeps the chosen label visible beside its rounded confidence.
  • Save model.py.
  • Confirm no red syntax-error underlines appear inside predict_ticket.

Is the prediction function underlined?

Check that the dictionary keys use matching quotes. Confirm the closing brace aligns with the return statement.

Help me debug the prediction function.

  • Scroll to the main block at the bottom of model.py.
  • Find this current block:
if __name__ == "__main__":
    print(f"Holdout accuracy: {evaluate_model():.2f}")

What is this reference showing?

This is the current runnable entry point. It only prints the holdout accuracy, so the next edit adds two prediction checks.

  • Replace the current main block with this expanded version:
if __name__ == "__main__":
    print(f"Holdout accuracy: {evaluate_model():.2f}")

    examples = [
        "The parcel still has not arrived",
        "Please help me with this problem",
    ]

    for example in examples:
        print(f"{example} -> {predict_ticket(example)}")

Why test these two tickets?

  • The parcel example repeats the paraphrase that defeated the keyword baseline.
  • The help request contains little information about the correct destination.
  • Printing both results exposes the classifier's label plus confidence for contrasting inputs.
  • Save model.py.

Use the tabs below to confirm that your completed file matches the target before the final run.

✔️ Awesome, I've got everything!

Your classifier now includes the balanced data, stratified split, fitted pipeline, holdout evaluation, and confidence-aware prediction function.

ⓧ I'd like to double check the full code

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline

TRAINING_EXAMPLES = [
    ("I was charged twice for the same order", "billing"),
    ("My refund has not arrived", "billing"),
    ("The invoice amount is incorrect", "billing"),
    ("My payment was declined", "billing"),
    ("Cancel my plan and refund the payment", "billing"),
    ("There is an unknown charge on my card", "billing"),
    ("I need to update my billing address", "billing"),
    ("Please send a receipt for my purchase", "billing"),
    ("The app crashes during startup", "technical"),
    ("The page shows a server error", "technical"),
    ("I cannot log in after the update", "technical"),
    ("The password reset link is not working", "technical"),
    ("The screen freezes during upload", "technical"),
    ("The submit button does nothing", "technical"),
    ("The website loads a blank page", "technical"),
    ("My two factor code is rejected", "technical"),
    ("I want to change my email address", "account"),
    ("Please delete my account", "account"),
    ("Update the name on my profile", "account"),
    ("I want to upgrade my subscription plan", "account"),
    ("I cannot access my profile settings", "account"),
    ("Please merge my duplicate accounts", "account"),
    ("Change my notification preferences", "account"),
    ("I want to reactivate my closed account", "account"),
    ("Where is my package", "shipping"),
    ("My delivery is late", "shipping"),
    ("The tracking number is not updating", "shipping"),
    ("The parcel arrived damaged", "shipping"),
    ("I need to change the delivery address", "shipping"),
    ("The shipment says delivered but it is missing", "shipping"),
    ("When will my order ship", "shipping"),
    ("The package was returned to the sender", "shipping"),
]

TEXTS = [text for text, _ in TRAINING_EXAMPLES]
LABELS = [label for _, label in TRAINING_EXAMPLES]

X_TRAIN, X_TEST, Y_TRAIN, Y_TEST = train_test_split(
    TEXTS,
    LABELS,
    test_size=0.25,
    random_state=42,
    stratify=LABELS,
)

MODEL = make_pipeline(
    TfidfVectorizer(ngram_range=(1, 2)),
    LogisticRegression(max_iter=1000),
)
MODEL.fit(X_TRAIN, Y_TRAIN)


def evaluate_model() -> float:
    predictions = MODEL.predict(X_TEST)
    return float(accuracy_score(Y_TEST, predictions))


def predict_ticket(text: str) -> dict[str, str | float]:
    model_prediction = str(MODEL.predict([text])[0])
    probabilities = MODEL.predict_proba([text])[0]
    confidence = float(max(probabilities))

    return {
        "model_prediction": model_prediction,
        "confidence": round(confidence, 3),
    }


if __name__ == "__main__":
    print(f"Holdout accuracy: {evaluate_model():.2f}")

    examples = [
        "The parcel still has not arrived",
        "Please help me with this problem",
    ]

    for example in examples:
        print(f"{example} -> {predict_ticket(example)}")

How should I use this reference?

Compare this complete file with your saved model.py. Pay close attention to ordering, indentation, brackets, and identifier spelling.

Before you run the completed classifier, which ticket do you expect to receive the stronger confidence?

  • Run the completed classifier in the activated VS Code terminal with this command:
py model.py

What does the final run verify?

This run evaluates the holdout set before predicting both sample tickets. Each prediction exposes the model's selected label plus its highest class probability.

You should see a Holdout accuracy: value followed by two prediction dictionaries. Each dictionary should contain model_prediction plus confidence.

That is the weak baseline replaced with an evaluated classifier. The paraphrased shipping ticket now receives a learned prediction instead of unknown.

Missing predictions or confidence?

Confirm the expanded main block calls predict_ticket(example). Check that predict_ticket returns both dictionary entries.

Help me debug the completed classifier output.

Your model now learns from examples, reports holdout performance, and exposes prediction confidence. Next, you will place this classifier behind a documented local API.

Serve Predictions with FastAPI

Your trained classifier can interpret ticket language plus report prediction confidence. Its predictions still live behind a Python function inside your project.

In this step, you will use FastAPI to expose the classifier through a local REST API. The service gives other applications a documented way to request predictions.

Why an API instead of a notebook?

A notebook demonstrates model behavior during analysis. An API provides a stable interface that other software can call.

FastAPI also generates interactive documentation from your Python operations. You can test the service without building a separate user interface.

In this step, get ready to:
  • Create a FastAPI application that reports service details.
  • Expose the trained classifier through a prediction operation.
  • Submit a ticket through the interactive API documentation.
Create the service shell

The application object stores your service metadata. It also connects Python functions to web operations.

  • Select New File... in the VS Code Explorer sidebar.
  • Enter main.py as the file name.
  • Press Enter to create the file.

You will see the blank main.py file beside model.py in the Explorer sidebar.

  • Add the application setup to main.py by pasting this code:
from fastapi import FastAPI

from model import evaluate_model, predict_ticket

app = FastAPI(
    title="AI Support Ticket Router",
    description="Routes support tickets while reporting prediction confidence.",
)

What does this setup do?

  • The FastAPI import provides the application class used by the service.
  • The import from model.py makes evaluate_model() available to the service.
  • The same import makes predict_ticket() available to the service.
  • The app object supplies the title shown in the generated documentation.
  • Save main.py.
  • Confirm main.py remains visible beside model.py in the Explorer sidebar.

Seeing an import warning?

  • Confirm VS Code is using Python from the existing .venv environment.
  • Return to the PowerShell terminal where .venv remains activated.

Help me diagnose an unresolved FastAPI import in main.py.

The root operation gives callers a quick description of the running service. It also reports the holdout accuracy from your existing model evaluation.

  • Add the root operation below the app setup in main.py by pasting this code:
@app.get("/")
def read_root() -> dict[str, str | float]:
    return {
        "service": "AI Support Ticket Router",
        "holdout_accuracy": round(evaluate_model(), 2),
        "docs": "/docs",
    }

What does the root operation return?

  • The @app.get("/") decorator connects a GET request at / to read_root().
  • The service field identifies the running application.
  • The holdout_accuracy field reports the result from evaluate_model().
  • The docs field points callers toward the interactive documentation.
  • Save main.py.

Before you run the service, do you expect the documentation to use the title from main.py?

  • Switch back to the PowerShell terminal from earlier.
  • Start the local development server by running this command:
fastapi dev

What does this command do?

The command starts your FastAPI application as a local development server. Keep the terminal running while you test the service.

  • Open http://127.0.0.1:8000/docs in your browser.

You will see interactive documentation titled AI Support Ticket Router. The page will list the root GET operation.

Good progress. Your classifier now has a running service with generated documentation.

Documentation page not loading?

  • Confirm the PowerShell terminal still shows the local server process running.
  • Confirm the browser address is http://127.0.0.1:8000/docs.
  • Confirm main.py is saved inside the ai-support-ticket-router folder.

Help me find why my FastAPI documentation page does not load.

Add the prediction operation

A required query parameter carries the ticket text into the prediction operation. FastAPI creates the matching documentation field from the typed text parameter.

  • Add the prediction operation below read_root() in main.py by pasting this code:
@app.post("/predict")
def predict(text: str) -> dict[str, str | float]:
    return predict_ticket(text)

How does prediction reach the model?

  • The @app.post("/predict") decorator creates the prediction operation at /predict.
  • The text: str annotation makes ticket text required.
  • The predict() function passes the ticket to predict_ticket().
  • The prediction dictionary becomes the API response.
  • Save main.py.
  • Return to the interactive documentation from earlier.
  • Refresh the browser page.

You will see a POST operation for /predict beneath the root operation. This confirms that the running service loaded your prediction function.

Prediction operation missing?

  • Confirm the server terminal has finished loading the saved application.
  • Confirm the predict() function sits below read_root() in main.py.
  • Refresh the interactive documentation again.

Help me find why the prediction operation is missing.

✔️ Awesome, I've got everything!

Your saved main.py now contains the application setup plus both operations.

ⓧ I'd like to double check the full code

from fastapi import FastAPI

from model import evaluate_model, predict_ticket

app = FastAPI(
    title="AI Support Ticket Router",
    description="Routes support tickets while reporting prediction confidence.",
)


@app.get("/")
def read_root() -> dict[str, str | float]:
    return {
        "service": "AI Support Ticket Router",
        "holdout_accuracy": round(evaluate_model(), 2),
        "docs": "/docs",
    }


@app.post("/predict")
def predict(text: str) -> dict[str, str | float]:
    return predict_ticket(text)
Submit a ticket through the documentation

The interactive documentation can send a real request to your running service. This tests the same interface that another application would call.

  • Select the POST operation for /predict to expand it.
  • Click Try it out.
  • Enter My package tracking has not updated in the text field.

Before you send the request, which queue do you expect the model to choose?

  • Click Execute.

You will see a JSON response containing model_prediction plus confidence.

That is the interface working. Your classifier now answers a live request through documented software.

Response not appearing?

  • Confirm the server remains active in the PowerShell terminal.
  • Confirm the text field contains the complete ticket.
  • Confirm you expanded the POST operation for /predict.

Help me troubleshoot the missing prediction response.

Your classifier now has a callable interface with interactive documentation. Next, you will add a human-review fallback for uncertain predictions.

Add Human Review and Publish

Your FastAPI service can now return a model label with confidence. Every request still receives a prediction even when the model has little evidence for its choice.

Unclear tickets need an explicit path to a person. A confidence threshold gives the service a visible human-review decision.

You will also turn your local Git project into a documented public GitHub repository. The finished repository shows the reasoning behind your model as well as its limitations.

In this step, get ready to:
  • Reproduce the API's unsafe forced-routing behavior.
  • Escalate low-confidence predictions to human review.
  • Document the service and publish it to GitHub.
Test the forced routing behavior

Before you test the current service, do you think a vague ticket receives a cautious decision or a normal queue prediction?

  • Switch back to the browser tab containing http://127.0.0.1:8000/docs.
  • Refresh the interactive documentation.
  • Expand the POST /predict operation.
  • Click Try it out.
  • Enter Please help me with this problem in the text query parameter.
  • Click Execute.

You will see a response containing model_prediction plus confidence. The response has no route or needs_review decision.

This Shortfall Is Intentional

The model always has a highest-scoring class. The current API presents that class without deciding whether the score is strong enough for automation.

This vague ticket exposes the missing safety policy. The next change makes uncertainty visible to the application using the model.

Add the human-review fallback

A review threshold turns confidence into an application decision. This project uses 0.45 as a teaching policy for its tiny dataset.

What Does the Threshold Mean?

Predictions below the threshold go to human_review. The response still preserves the model's original guess for debugging and evaluation.

The threshold is a project policy choice. Production teams would test thresholds against representative data before using them for real routing decisions.

Your Visual Studio Code editor from earlier already has the model code that produces probabilities. You will add the policy directly beside the labels that it governs.

  • Switch back to model.py in your editor.
  • Find the existing TEXTS line.
  • Replace the current TEXTS and LABELS lines with this configuration:
TEXTS = [text for text, _ in TRAINING_EXAMPLES]
LABELS = [label for _, label in TRAINING_EXAMPLES]
REVIEW_THRESHOLD = 0.45

What Does This Configuration Do?

  • The TEXTS list continues to hold the ticket text used by the model.
  • The LABELS list continues to hold the matching queues.
  • The REVIEW_THRESHOLD constant keeps the routing policy in one named location.
  • Find the existing predict_ticket function in model.py.
  • Replace the complete function with this version:
def predict_ticket(text: str) -> dict[str, str | float | bool]:
    model_prediction = str(MODEL.predict([text])[0])
    probabilities = MODEL.predict_proba([text])[0]
    confidence = float(max(probabilities))
    needs_review = confidence < REVIEW_THRESHOLD
    route = "human_review" if needs_review else model_prediction

    return {
        "model_prediction": model_prediction,
        "confidence": round(confidence, 3),
        "route": route,
        "needs_review": needs_review,
    }

How Does Human Review Work?

  • The function keeps model_prediction so reviewers can inspect the classifier's original choice.
  • The needs_review flag compares confidence with REVIEW_THRESHOLD.
  • The final route becomes human_review when the confidence is below the policy threshold.
  • Save model.py.

Before you repeat the request, do you think the response will hide the model's guess or show it beside the review decision?

  • Return to the interactive documentation in your browser.
  • Refresh the page after the development server reloads.
  • Expand the POST /predict operation.
  • Click Try it out.
  • Enter Please help me with this problem in the text query parameter.
  • Click Execute.

You will see route equal to human_review. You will also see the original model_prediction with its confidence value.

That is the safety gap closed. Your router now asks for human judgment when its evidence is weak.

Still Seeing Only Two Response Fields?

Confirm that you saved model.py. Refresh the documentation after the development server reloads.

Check that predict_ticket returns all four dictionary fields. Make sure REVIEW_THRESHOLD is defined above the function.

Help me debug why my FastAPI prediction response still omits the human-review fields.

✔️ Awesome, I've got everything!

  • Keep model.py saved with the working human-review response.

ⓧ I'd like to double check the full code

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score
from sklearn.model_selection import train_test_split
from sklearn.pipeline import make_pipeline

TRAINING_EXAMPLES = [
    ("I was charged twice for the same order", "billing"),
    ("My refund has not arrived", "billing"),
    ("The invoice amount is incorrect", "billing"),
    ("My payment was declined", "billing"),
    ("Cancel my plan and refund the payment", "billing"),
    ("There is an unknown charge on my card", "billing"),
    ("I need to update my billing address", "billing"),
    ("Please send a receipt for my purchase", "billing"),
    ("The app crashes during startup", "technical"),
    ("The page shows a server error", "technical"),
    ("I cannot log in after the update", "technical"),
    ("The password reset link is not working", "technical"),
    ("The screen freezes during upload", "technical"),
    ("The submit button does nothing", "technical"),
    ("The website loads a blank page", "technical"),
    ("My two factor code is rejected", "technical"),
    ("I want to change my email address", "account"),
    ("Please delete my account", "account"),
    ("Update the name on my profile", "account"),
    ("I want to upgrade my subscription plan", "account"),
    ("I cannot access my profile settings", "account"),
    ("Please merge my duplicate accounts", "account"),
    ("Change my notification preferences", "account"),
    ("I want to reactivate my closed account", "account"),
    ("Where is my package", "shipping"),
    ("My delivery is late", "shipping"),
    ("The tracking number is not updating", "shipping"),
    ("The parcel arrived damaged", "shipping"),
    ("I need to change the delivery address", "shipping"),
    ("The shipment says delivered but it is missing", "shipping"),
    ("When will my order ship", "shipping"),
    ("The package was returned to the sender", "shipping"),
]

TEXTS = [text for text, _ in TRAINING_EXAMPLES]
LABELS = [label for _, label in TRAINING_EXAMPLES]
REVIEW_THRESHOLD = 0.45

X_TRAIN, X_TEST, Y_TRAIN, Y_TEST = train_test_split(
    TEXTS,
    LABELS,
    test_size=0.25,
    random_state=42,
    stratify=LABELS,
)

MODEL = make_pipeline(
    TfidfVectorizer(ngram_range=(1, 2)),
    LogisticRegression(max_iter=1000),
)
MODEL.fit(X_TRAIN, Y_TRAIN)


def evaluate_model() -> float:
    predictions = MODEL.predict(X_TEST)
    return float(accuracy_score(Y_TEST, predictions))


def predict_ticket(text: str) -> dict[str, str | float | bool]:
    model_prediction = str(MODEL.predict([text])[0])
    probabilities = MODEL.predict_proba([text])[0]
    confidence = float(max(probabilities))
    needs_review = confidence < REVIEW_THRESHOLD
    route = "human_review" if needs_review else model_prediction

    return {
        "model_prediction": model_prediction,
        "confidence": round(confidence, 3),
        "route": route,
        "needs_review": needs_review,
    }


if __name__ == "__main__":
    print(f"Holdout accuracy: {evaluate_model():.2f}")

    examples = [
        "The parcel still has not arrived",
        "Please help me with this problem",
    ]

    for example in examples:
        print(f"{example} -> {predict_ticket(example)}")

The model now returns two extra value types. The API description should also state that uncertain tickets are escalated.

  • Switch back to main.py.
  • Replace the complete file with this updated API definition:
from fastapi import FastAPI

from model import evaluate_model, predict_ticket

app = FastAPI(
    title="AI Support Ticket Router",
    description="Routes support tickets while escalating uncertain predictions.",
)


@app.get("/")
def read_root() -> dict[str, str | float]:
    return {
        "service": "AI Support Ticket Router",
        "holdout_accuracy": round(evaluate_model(), 2),
        "docs": "/docs",
    }


@app.post("/predict")
def predict(text: str) -> dict[str, str | float | bool]:
    return predict_ticket(text)

What Changed in the API?

  • The application description now states that uncertain predictions are escalated.
  • The predict return annotation now allows Boolean values for needs_review.
  • The operation continues to delegate routing decisions to predict_ticket.
  • Save main.py.
  • Refresh the interactive documentation in your browser.

You will see the service description state that uncertain predictions are escalated. The prediction operation remains available at /predict.

Documentation Still Shows the Old Description?

Confirm that main.py contains the updated description. Save the file before refreshing the browser.

Help me find why my FastAPI documentation still shows the previous application description.

✔️ Awesome, I've got everything!

  • Keep main.py saved with the updated description and return annotation.

ⓧ I'd like to double check the full code

from fastapi import FastAPI

from model import evaluate_model, predict_ticket

app = FastAPI(
    title="AI Support Ticket Router",
    description="Routes support tickets while escalating uncertain predictions.",
)


@app.get("/")
def read_root() -> dict[str, str | float]:
    return {
        "service": "AI Support Ticket Router",
        "holdout_accuracy": round(evaluate_model(), 2),
        "docs": "/docs",
    }


@app.post("/predict")
def predict(text: str) -> dict[str, str | float | bool]:
    return predict_ticket(text)
Document and publish the project

A public repository needs enough context for another developer to reproduce the result. Its Markdown README also explains why the baseline failed and why the fallback exists.

  • Select the ai-support-ticket-router folder in the VS Code Explorer sidebar.
  • Use the new-file control in the Explorer sidebar.
  • Enter README.md as the file name.
  • Add the project overview by pasting this first section:
# AI Support Ticket Router

A small AI engineering project that routes support tickets into billing, technical, account, or shipping queues. It reports prediction confidence and sends uncertain tickets to human review.

## Why this project exists

A keyword baseline works for familiar words but fails on paraphrases such as "The parcel still has not arrived." The learned pipeline replaces exact keyword matching with TF-IDF text features and logistic regression.

## Engineering flow

1. Reproduce the keyword baseline failure.
2. Split labeled examples into training and holdout sets.
3. Fit preprocessing and classification together in one pipeline.
4. Report holdout accuracy.
5. Serve predictions through FastAPI.
6. Route low-confidence predictions to `human_review`.

What Does This Section Show?

The opening explains the service in plain language. The engineering flow shows the path from a weak baseline to a confidence-aware API.

This framing makes the decisions behind the project visible. Readers can see why each technical layer exists.

  • Save README.md.
  • Press Cmd+Shift+V on macOS or Ctrl+Shift+V on Windows to open the Markdown preview.

You will see the project title followed by its purpose and a six-stage engineering flow.

Markdown Preview Looks Empty?

Confirm that README.md is the active editor before using the preview shortcut. Save the file if the preview does not contain the latest text.

Help me troubleshoot why my README content does not appear in the VS Code Markdown preview.

  • Return to the README.md editor.
  • Place the cursor after the final engineering-flow item.
  • Add the setup instructions and limitations by pasting this section:
## Run locally on Windows

```powershell
py -m venv .venv
.venv\Scripts\Activate.ps1
py -m pip install -r requirements.txt
py baseline.py
py model.py
fastapi dev
```

Open `http://127.0.0.1:8000/docs` and use the prediction operation.

## Response fields

- `model_prediction`: The classifier's best label.
- `confidence`: The highest class probability reported by the model.
- `route`: The automated queue or `human_review`.
- `needs_review`: Whether the confidence falls below the project threshold.

## Limitations

- The included dataset is tiny and illustrative, not representative of production traffic.
- The confidence threshold is a project policy choice, not a calibrated production guarantee.
- A production system would require more labeled data, class-specific metrics, monitoring, security controls, and repeated evaluation on new traffic.

Why Include Limitations?

The setup section makes the project reproducible on another Windows computer. The response-field section documents the API contract.

The limitations prevent a small demonstration dataset from being mistaken for production evidence. Responsible documentation strengthens the engineering story.

  • Save README.md.
  • Return to the Markdown preview.

You will see sections for local setup and response fields. You will also see three explicit project limitations.

Missing the Final README Sections?

Check that the second block appears after item six in the engineering flow. Confirm that every triple-backtick fence has a matching closing fence.

Help me compare my README with the required project documentation.

✔️ Awesome, I've got everything!

  • Keep README.md saved with the setup instructions and limitations.

ⓧ I'd like to double check the full code

# AI Support Ticket Router

A small AI engineering project that routes support tickets into billing, technical, account, or shipping queues. It reports prediction confidence and sends uncertain tickets to human review.

## Why this project exists

A keyword baseline works for familiar words but fails on paraphrases such as "The parcel still has not arrived." The learned pipeline replaces exact keyword matching with TF-IDF text features and logistic regression.

## Engineering flow

1. Reproduce the keyword baseline failure.
2. Split labeled examples into training and holdout sets.
3. Fit preprocessing and classification together in one pipeline.
4. Report holdout accuracy.
5. Serve predictions through FastAPI.
6. Route low-confidence predictions to `human_review`.

## Run locally on Windows

```powershell
py -m venv .venv
.venv\Scripts\Activate.ps1
py -m pip install -r requirements.txt
py baseline.py
py model.py
fastapi dev
```

Open `http://127.0.0.1:8000/docs` and use the prediction operation.

## Response fields

- `model_prediction`: The classifier's best label.
- `confidence`: The highest class probability reported by the model.
- `route`: The automated queue or `human_review`.
- `needs_review`: Whether the confidence falls below the project threshold.

## Limitations

- The included dataset is tiny and illustrative, not representative of production traffic.
- The confidence threshold is a project policy choice, not a calibrated production guarantee.
- A production system would require more labeled data, class-specific metrics, monitoring, security controls, and repeated evaluation on new traffic.

The local development server currently occupies your PowerShell terminal. Stop it before creating the commit so the same terminal is ready for Git commands.

  • Switch back to the terminal running the development server.
  • Press Ctrl+C to stop the server.
  • Stage the project files and create the first commit by running:
git add .
git commit -m "First commit"

What Does This Commit Capture?

The first command stages the project files. The .gitignore rules keep the local environment and generated Python cache files out of the commit.

The second command records the staged state with the message First commit.

You will see a commit summary listing the files Git recorded.

Commit Did Not Complete?

Git may ask you to configure an author name or email before creating your first commit. Follow the terminal guidance before running the commit command again.

Help me diagnose why my first Git commit did not complete.

The remote repository must be empty so your existing history can become its starting point. You will create it without generated files that could conflict with your local project.

  • Go to GitHub in your browser.
  • Sign in to your existing account if prompted.
  • Select New repository.
  • Enter ai-support-ticket-router as the repository name.
  • Select the public visibility option.
  • Leave the README option unselected.
  • Leave the license option unselected.
  • Leave the .gitignore option unselected.
  • Select Create repository.
  • Copy the remote repository URL from the setup page.
  • Replace REMOTE-URL in this command with the copied URL.
  • Connect the local repository to GitHub by running:
git remote add origin REMOTE-URL

What Does the Remote Do?

The command names your GitHub repository origin. Future push commands can use that short name instead of the complete URL.

  • Confirm the remote connection by running:
git remote -v

What Should the Remote Show?

You will see the GitHub repository URL listed for fetch and push operations. Both entries should point to ai-support-ticket-router.

A browser or credential-manager sign-in prompt during the first push is normal. Complete that authentication flow before retrying the command if Git pauses.

Before you push, do you expect GitHub to receive the commit and render the README from the repository root?

  • Publish the local main branch by running:
git push -u origin main

What Does the Push Do?

The command uploads the local main branch to origin. The -u flag connects the local branch to its remote counterpart for later pushes.

  • Complete the GitHub sign-in flow if Git requests authentication.
  • Run the push command again after authentication completes.

Before you check the repository, do you think the project overview or the raw file list will be the first thing GitHub emphasizes?

  • Return to the public ai-support-ticket-router repository in your browser.
  • Refresh the repository page.
  • Confirm that the rendered README is visible below the project files.

You will see the project purpose and engineering flow on the public repository page. You will also see the setup instructions and limitations.

Push or README Missing?

If the push stops for authentication, complete the browser or credential-manager sign-in. Run the push command again afterward.

If the repository page has no rendered README, confirm that the committed file is named exactly README.md.

Help me troubleshoot why my GitHub push or rendered README is missing.

You have turned a local classifier into a documented AI service with an explicit safety fallback. The public repository now shows the model behavior as well as the engineering judgment behind it.

Secret mission

Red-Team the Router

Push the router beyond clean examples with eight difficult tickets. Document the observed routes and confidence values in a GitHub issue. Turn at least two failures into targeted engineering improvements.

Clean Up Your Resources

Clean Up Your Resources

Your local FastAPI service uses no paid cloud resources. Choose whether to keep, pause, or delete the project knowing GitHub Free is $0 USD per month.

Resources you used:

  • The local FastAPI development server that provides the interactive documentation.
  • The Git repository inside the local ai-support-ticket-router folder.
  • The disposable .venv environment inside ai-support-ticket-router.
  • The public GitHub repository named ai-support-ticket-router, including the Red-team evaluation findings issue.

Keep everything running

No action needed. Choose this if you want to demonstrate the router or continue improving its evaluation.

  • Keep the local ai-support-ticket-router folder unchanged.
  • Leave the public GitHub repository available for future improvements.
  • Press Ctrl+C in the server terminal when you finish a demonstration.

Pause - I'll come back to this later

This option shuts down the local server while preserving everything needed to continue later.

  • Press Ctrl+C in the terminal running the FastAPI development server.
  • Keep the local ai-support-ticket-router folder unchanged.
  • Leave the public GitHub repository available.

The terminal returns to its prompt after the server stops. Your local files remain available for the next session.

Delete - I don't want to use this again

This option removes all project resources for a fresh start. Deleting the repository can feel drastic because GitHub asks for confirmation before removal.

  • Press Ctrl+C in the terminal running the FastAPI development server.
  • Remove the local ai-support-ticket-router folder from the location where you created it by using Windows file management.

You should no longer see the ai-support-ticket-router folder in its previous location. This also removes the local Git repository plus the .venv environment.

  • Open the public ai-support-ticket-router repository on GitHub.
  • Select Settings.
  • Select General.
  • Scroll to Danger Zone.
  • Choose Delete this repository.
  • Complete the confirmation steps.

After confirmation, you should no longer be able to open the repository URL. The Red-team evaluation findings issue is removed with the repository.

Nice Work!

Nice Work!

You did it! Your published AI support ticket router now classifies requests through a local API. Uncertain requests go to human review.

You learned how to:

  • Reproduce a realistic paraphrase failure with a keyword baseline. Replace that baseline with a TF-IDF multiclass classifier evaluated on held-out tickets.
  • Serve predictions through FastAPI with interactive documentation. Send low-confidence tickets to human_review. Preserve the classifier's original guess for transparency.
  • Publish a reproducible GitHub repository with setup instructions. Document the tiny dataset limitation. Explain the confidence threshold as a policy choice.
  • Secret Mission: Test eight difficult tickets through the API. Turn unacceptable failures into targeted improvements through the Red-team evaluation findings issue.

Ready to quiz yourself?