Build a Support Ticket Classifier

Build a Python tool that classifies support tickets with machine learning.

Introduction

30 Second Summary

Support teams often receive the same problem in dozens of different phrasings. A rigid keyword list can leave unfamiliar messages without a useful destination.

In this project, you will build a support ticket triage tool with Python that begins with handwritten routing rules. You will then replace those brittle rules with a text classifier trained through machine learning in Google Colab.

What You'll Build

You will enter a fresh support message into your notebook to see its predicted queue printed instantly.

By the end of this project, you'll have:

  • A working rule-based prototype that routes obvious tickets among three support queues.
  • A trained classifier that uses TF-IDF features from scikit-learn to recognize a receipt request the keyword rules miss.
  • An interactive notebook that reports an accuracy percentage before routing a support message you type.
  • Secret Mission: Add confidence-based routing that sends uncertain tickets to human review.

Are there any prerequisites?

Bring a Google account plus a current supported browser. Everything runs in Google Colab, so you do not need local setup.

Before We Start

Before you begin, commit to building a classifier you can explain. Comparing a handwritten keyword router with a trained text classifier helps you understand how billing, technical, and feature_request tickets can be routed more reliably.

Set Up the Colab Workspace

The keyword router and trained classifier must run in the same predictable environment. A pinned library version removes setup surprises before you compare their results.

Google Colab gives you a browser-based notebook with output beside each cell. This setup pins scikit-learn to 1.9.1 before a version check proves the temporary runtime is ready.

In this step, get ready to:
  • Create a new Google Colab notebook.
  • Install the pinned scikit-learn package.
  • Verify the package version in the temporary runtime.
Create a new Colab notebook

Your notebook stores the cells that make this project reproducible. The notebook persists in Google Drive while its code runs inside a temporary virtual machine.

  • Open the Google Colab new notebook link in Safari or another supported browser.
  • Sign in with your Google account if prompted.
  • Locate the empty code cell in the main workspace to confirm the new notebook is open.

You should see a blank notebook with an empty code cell. You're in the right place for every setup check that follows.

Install the pinned package

Packages installed in a Colab runtime belong to that temporary environment. Keeping the installation in the first cell makes the notebook repeatable when Colab assigns a fresh runtime.

  • Select the first empty code cell.
  • Install scikit-learn 1.9.1 into the notebook runtime by running this code:
%pip install scikit-learn==1.9.1

What does this command do?

  • The %pip command installs a package inside the current notebook kernel.
  • The scikit-learn==1.9.1 pin requests the exact package release used throughout this project.
  • Wait for the installation cell to finish.

The cell should finish without an installation error.

Installation cell failed?

  • Confirm your browser can load another web page.
  • Rerun the installation cell once.

Help me troubleshoot the scikit-learn installation in Google Colab.

Verify the scikit-learn version

The sklearn.show_versions() function reports library information from the current runtime. Finding 1.9.1 proves the notebook is using the pinned package.

  • Use the notebook toolbar's add-code control to insert a second cell beneath the installation cell.
  • Predict the scikit-learn version that the runtime will report.
  • Test your prediction by running this code:
import sklearn
sklearn.show_versions()

What does this check do?

  • The import sklearn line loads the installed library into the notebook.
  • The sklearn.show_versions() call prints version information for scikit-learn and its environment.

✔️ I see the pinned version

Locate 1.9.1 in the printed scikit-learn version information. That's your environment locked down for the classifier work ahead.

ⓧ I see a different version

A different version means the runtime has not applied the exact package pin.

  • Return to the installation cell.
  • Run the installation cell again.
  • Return to the verification cell.
  • Run the verification cell again.

You should now locate 1.9.1 in the printed version information.

ⓧ The import fails

An import failure means the package installation did not complete in the current runtime.

  • Return to the installation cell.
  • Run the installation cell again.
  • Return to the verification cell.
  • Run the verification cell again.

The check should now print the scikit-learn version information with 1.9.1 included.

Your Colab workspace now uses the verified package version. Next, you'll build the keyword router that sends obvious tickets to three support queues.

Build the Rule-Based Prototype

Your Google Colab runtime already has scikit-learn 1.9.1 verified. That setup keeps environment surprises out of this prototype.

A handwritten baseline gives you an immediate routing result. In this step, you will use Python dictionaries, conditions, loops, strings, and functions to classify obvious support tickets.

In this step, get ready to:
  • Map each support queue to a list of keywords.
  • Build a function that searches a ticket for matching keywords.
  • Run three tickets through the prototype to confirm every queue works.
Create the routing rules

A dictionary stores labels beside related values. Here, each queue label points to words that commonly appear in that type of support ticket.

  • In the Google Colab notebook from earlier, add a code cell below the version verification cell.
  • Create the queue rules by pasting this code into the new cell:
KEYWORD_RULES = {
    "billing": ["charge", "refund", "invoice", "payment", "card"],
    "technical": ["login", "log in", "crash", "password", "blank", "error"],
    "feature_request": ["add", "feature", "export", "integration", "mobile app"],
}

How Do the Rules Work?

  • The KEYWORD_RULES dictionary uses queue names as keys.
  • Each key points to a list of related words.
  • The feature_request label uses an underscore so it works as one consistent queue name.
  • Run the cell that defines KEYWORD_RULES.
  • Add another code cell below the definition cell.
  • Display the stored rules by running this expression:
KEYWORD_RULES

What Does This Check Prove?

Google Colab displays the dictionary currently stored in KEYWORD_RULES. You should see the three queue labels beside their keyword lists.

Your notebook now holds the vocabulary for billing, technical, and feature_request tickets.

Don't See the Three Rules?

  • Run the cell containing the complete KEYWORD_RULES definition before running the verification cell.
  • Check that every opening brace has a matching closing brace.
  • Compare the commas between the three dictionary entries.
  • Ask for help with the dictionary syntax.
Write the matching function

The rules need a function that can inspect one ticket at a time. The function normalizes the text before scanning each queue for a match.

  • Add a code cell below the rules check.
  • Build the keyword matcher by pasting this function into the new cell:
def keyword_triage(ticket):
    text = ticket.lower()

    for label, keywords in KEYWORD_RULES.items():
        if any(keyword in text for keyword in keywords):
            return label

    return "unknown"

What Does This Function Do?

  • The lower() method creates a lowercase copy of the ticket text.
  • The for loop visits each queue in KEYWORD_RULES.
  • The any() check succeeds when at least one keyword appears in the ticket.
  • The function returns unknown when every queue finishes without a match.
  • Run the cell that defines keyword_triage(ticket).
  • Add a code cell below the function definition.
  • Confirm the function exists by running this expression:
keyword_triage

What Does This Check Prove?

Google Colab displays a function reference containing the name keyword_triage. This confirms the definition cell ran successfully.

The matcher is ready to inspect ticket text. The next function gives it three clear examples to route.

Don't See a Function Reference?

  • Run the complete keyword_triage(ticket) definition before evaluating its name.
  • Check that the lines inside the function use consistent indentation.
  • Check that KEYWORD_RULES matches the dictionary name from the earlier cell.
  • Ask for help with the function definition.
Run the three-queue demo

A demo function keeps several test tickets together. Its loop sends each message through the same matcher before printing the destination.

  • Add a code cell below the function check.
  • Create the demo by pasting this function into the new cell:
def show_keyword_demo():
    demo_tickets = [
        "Please refund my payment",
        "The app crashes when I upload a file",
        "Please add dark mode",
    ]

    for ticket in demo_tickets:
        print(f"{ticket} -> {keyword_triage(ticket)}")

How Does the Demo Work?

  • The demo_tickets list holds one obvious example for each queue.
  • The loop passes every ticket to keyword_triage(ticket).
  • The formatted string prints each original ticket beside its predicted queue.
  • Run the cell that defines show_keyword_demo().
  • Add a code cell below the demo definition.

Before you run the demo, which queue do you expect each ticket to reach?

  • Run the full routing demo with this call:
show_keyword_demo()

What Should You See?

  • The first line ends with Please refund my payment -> billing.
  • The second line ends with The app crashes when I upload a file -> technical.
  • The third line ends with Please add dark mode -> feature_request.

You have your first working support router. The prototype now sends three obvious tickets to the correct queues.

Seeing the Wrong Queue?

  • Rerun the KEYWORD_RULES cell so the current dictionary contains all three queues.
  • Rerun the keyword_triage(ticket) cell so the function uses the current rules.
  • Compare each demo ticket with the keyword list for its expected queue.
  • Ask for help with the routing results.

✔️ Awesome, I've got everything!

Your notebook contains the rules, the matcher, and the three-ticket demo. Keep each definition cell above the cell that calls show_keyword_demo().

ⓧ I'd like to double check the full code

  • Compare your three definition cells with this cumulative code:
KEYWORD_RULES = {
    "billing": ["charge", "refund", "invoice", "payment", "card"],
    "technical": ["login", "log in", "crash", "password", "blank", "error"],
    "feature_request": ["add", "feature", "export", "integration", "mobile app"],
}


def keyword_triage(ticket):
    text = ticket.lower()

    for label, keywords in KEYWORD_RULES.items():
        if any(keyword in text for keyword in keywords):
            return label

    return "unknown"


def show_keyword_demo():
    demo_tickets = [
        "Please refund my payment",
        "The app crashes when I upload a file",
        "Please add dark mode",
    ]

    for ticket in demo_tickets:
        print(f"{ticket} -> {keyword_triage(ticket)}")

Your baseline routes obvious requests successfully. Next, you will give it a ticket whose wording exposes the limits of handwritten keywords.

Create Labeled Training Examples

Your keyword prototype routes familiar tickets. The next test asks whether its limited vocabulary covers a normal customer phrase.

You will use your Google Colab notebook to test one receipt request. After you see the result, you will create balanced labeled data for three support queues.

In this step, get ready to:
  • Test the keyword rules with an unfamiliar receipt request.
  • Create fifteen support ticket examples across three queues.
  • Pair every ticket with a label and verify the class balance.
Test an unfamiliar receipt request

The current rules compare each ticket with a fixed keyword list. A new customer phrase gives you a direct test of how far that vocabulary reaches.

Before you run this cell, which queue do you think the rule-based prototype will choose?

  • Use the notebook toolbar to add a code cell below the cell containing show_keyword_demo().
  • Create the obstacle ticket and test it by entering the code below:
OBSTACLE_TICKET = "I need a receipt"

keyword_triage(OBSTACLE_TICKET)

What does this code do?

  • The OBSTACLE_TICKET variable stores the new customer message.
  • The call to keyword_triage() checks that message against the existing keyword rules.
  • The final expression tells the notebook to display the function's return value.

You will see unknown. This is the intended shortfall.

The billing list contains words such as invoice and payment. It does not contain receipt.

No result from the obstacle cell?

Run the earlier cell that defines KEYWORD_RULES and keyword_triage(). The notebook must load those definitions before it can test the new ticket.

Check that OBSTACLE_TICKET uses the same capital letters in both lines.

Help me debug the obstacle cell in my ticket triage notebook.

Add balanced ticket examples

Examples give the future classifier more ways to recognize each queue. The billing group includes a receipt message that expands beyond the original keyword vocabulary.

  • Use the notebook toolbar to add a code cell between the KEYWORD_RULES cell and the OBSTACLE_TICKET cell.
  • Create the ticket examples by entering the code below:
TRAINING_TICKETS = [
    "I was charged twice for my subscription",
    "Please refund my last payment",
    "My invoice has the wrong amount",
    "The payment page rejected my card",
    "Where can I download a receipt",
    "I cannot log in to my account",
    "The app crashes when I upload a file",
    "The reset password link does not work",
    "The dashboard shows a blank screen",
    "Notifications stopped working yesterday",
    "Please add dark mode",
    "I would like an export to PDF option",
    "Can you support team accounts",
    "Please add calendar integration",
    "I want a mobile app",
]

TRAINING_TICKETS

How are the examples organized?

  • The first five messages describe billing issues.
  • The next five messages describe technical issues.
  • The final five messages describe feature requests.
  • The receipt example gives the billing queue a phrase that the keyword rules missed.
  • Run the cell.

You will see the complete list of fifteen training tickets below the cell.

No ticket list below the cell?

Check that the final line contains only TRAINING_TICKETS. That expression displays the list after the assignment runs.

Check every ticket for matching quotation marks. One missing mark prevents the entire cell from running.

Help me find a syntax problem in my TRAINING_TICKETS list.

Pair labels and verify the balance

Each label occupies the same position as its matching ticket. The first label describes the first ticket.

Before you run the count check, do you expect any queue to have more examples than the others?

  • Use the notebook toolbar to add a code cell below the TRAINING_TICKETS cell.
  • Pair the tickets with their labels and count each class by entering the code below:
TRAINING_LABELS = [
    "billing",
    "billing",
    "billing",
    "billing",
    "billing",
    "technical",
    "technical",
    "technical",
    "technical",
    "technical",
    "feature_request",
    "feature_request",
    "feature_request",
    "feature_request",
    "feature_request",
]

for label in ["billing", "technical", "feature_request"]:
    print(f"{label}: {TRAINING_LABELS.count(label)}")

How does the count check work?

  • The order of TRAINING_LABELS matches the order of TRAINING_TICKETS.
  • The loop visits each queue name once.
  • The count() method measures how many times that label appears.
  • Run the cell.

Each queue prints a count of 5.

Good work. Your classifier now has five labeled examples for every destination.

Seeing an uneven count?

Compare the label order with the ticket groups. The list needs five billing entries before five technical entries.

Confirm that the list ends with five feature_request entries.

Help me balance the labels in my training data.

✔️ Awesome, I've got everything!

Your notebook now contains the designed obstacle and a balanced set of labeled training examples.

ⓧ I'd like to double check the full code

  • Compare your notebook cells with the complete code below.
KEYWORD_RULES = {
    "billing": ["charge", "refund", "invoice", "payment", "card"],
    "technical": ["login", "log in", "crash", "password", "blank", "error"],
    "feature_request": ["add", "feature", "export", "integration", "mobile app"],
}

TRAINING_TICKETS = [
    "I was charged twice for my subscription",
    "Please refund my last payment",
    "My invoice has the wrong amount",
    "The payment page rejected my card",
    "Where can I download a receipt",
    "I cannot log in to my account",
    "The app crashes when I upload a file",
    "The reset password link does not work",
    "The dashboard shows a blank screen",
    "Notifications stopped working yesterday",
    "Please add dark mode",
    "I would like an export to PDF option",
    "Can you support team accounts",
    "Please add calendar integration",
    "I want a mobile app",
]

TRAINING_LABELS = [
    "billing",
    "billing",
    "billing",
    "billing",
    "billing",
    "technical",
    "technical",
    "technical",
    "technical",
    "technical",
    "feature_request",
    "feature_request",
    "feature_request",
    "feature_request",
    "feature_request",
]

OBSTACLE_TICKET = "I need a receipt"


def keyword_triage(ticket):
    text = ticket.lower()

    for label, keywords in KEYWORD_RULES.items():
        if any(keyword in text for keyword in keywords):
            return label

    return "unknown"


def show_keyword_demo():
    demo_tickets = [
        "Please refund my payment",
        "The app crashes when I upload a file",
        "Please add dark mode",
    ]

    for ticket in demo_tickets:
        print(f"{ticket} -> {keyword_triage(ticket)}")

Your rules have exposed their vocabulary gap. Next, you will train a text classifier that learns from these labeled examples.

Train the Text Classifier

Your Google Colab notebook now holds a balanced labeled data set. The rule-based prototype still returns unknown for the receipt ticket.

Now you will use scikit-learn to train a model for text classification. The model learns word-to-label patterns from your examples.

In this step, get ready to:
  • Import the text-processing tools.
  • Fit a classifier pipeline on the labeled tickets.
  • Predict the queue for the missed receipt ticket.
Import the classifier tools

A classifier needs numeric features before it can compare language. These imports provide the feature converter plus the classifier.

  • Add the classifier imports at the top of your notebook by pasting this code into a new code cell:
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline

What do these imports provide?

  • `TfidfVectorizer` converts raw ticket text into numeric features.
  • `MultinomialNB` provides a lightweight classifier suited to text features.
  • `Pipeline` keeps feature conversion connected to classification.
  • Run the imports cell.

You'll see the cell finish without an exception. The three tools are now available to later cells.

Imports not completing?

  • Run the package installation cell from Step 1 again.
  • Confirm that the installed scikit-learn version is still 1.9.1.
  • Run the imports cell again.

Help me diagnose the failed scikit-learn imports in my Colab notebook.

Fit the classifier pipeline

A Pipeline runs the same TF-IDF conversion before Multinomial Naive Bayes learns from the results. The fitted model carries that sequence into every prediction.

  • Add a new code cell below show_keyword_demo() by using the notebook toolbar.
  • Define build_model() by pasting this code into the new cell:
def build_model():
    model = Pipeline(
        [
            ("tfidf", TfidfVectorizer(lowercase=True)),
            ("classifier", MultinomialNB()),
        ]
    )
    model.fit(TRAINING_TICKETS, TRAINING_LABELS)
    return model

What does this function do?

  • `Pipeline` stores the feature converter under the name `tfidf`.
  • `TfidfVectorizer(lowercase=True)` normalizes letter case before creating numeric features.
  • `MultinomialNB()` learns which features are associated with each queue.
  • `model.fit()` learns from `TRAINING_TICKETS` plus `TRAINING_LABELS`.
  • `return model` makes the fitted pipeline available for predictions.
  • Run the build_model() cell.

You'll see the cell finish without an exception. The model-building function is now ready to run.

Seeing an exception?

  • Run the imports cell before running build_model().
  • Compare TRAINING_TICKETS against the earlier cell to confirm the name matches.
  • Compare TRAINING_LABELS against the earlier cell to confirm the name matches.

Help me fix the exception in my build_model function.

Predict the missed receipt ticket

The fitted pipeline needs a small prediction function. This function passes one ticket to the model before returning its predicted queue.

  • Add a new code cell below build_model() by using the notebook toolbar.
  • Define triage_ticket() by pasting this code into the new cell:
def triage_ticket(model, ticket):
    return model.predict([ticket])[0]

What does this function do?

  • `[ticket]` wraps one message in the collection format expected by the model.
  • `model.predict()` returns a collection of predicted labels.
  • `[0]` selects the queue for the single ticket.
  • Run the triage_ticket() cell.

You'll see the cell finish without an exception. The notebook can now request one prediction at a time.

Before you run the final check, which queue do you expect the trained model to choose for the receipt ticket?

  • Train the model plus test the designed obstacle by running this code in a new cell:
model = build_model()
print(f"{OBSTACLE_TICKET} -> {triage_ticket(model, OBSTACLE_TICKET)}")

What does this check do?

  • `build_model()` fits the pipeline on all 15 labeled examples.
  • `triage_ticket()` applies the fitted pipeline to `OBSTACLE_TICKET`.
  • `print()` places the ticket beside its predicted queue.

You'll see I need a receipt -> billing. The trained model now handles the synonym that the handwritten rules missed.

Not seeing billing?

  • Run the training data cells before running the final check.
  • Confirm that OBSTACLE_TICKET contains I need a receipt.
  • Confirm that the receipt example in TRAINING_TICKETS lines up with a billing entry in TRAINING_LABELS.

Help me find why the trained model does not classify the receipt ticket as billing.

✔️ Awesome, I've got everything!

Great. Double check that every notebook cell has finished running.

ⓧ I'd like to double check the full code

Compare your notebook code against this cumulative version.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline

KEYWORD_RULES = {
    "billing": ["charge", "refund", "invoice", "payment", "card"],
    "technical": ["login", "log in", "crash", "password", "blank", "error"],
    "feature_request": ["add", "feature", "export", "integration", "mobile app"],
}

TRAINING_TICKETS = [
    "I was charged twice for my subscription",
    "Please refund my last payment",
    "My invoice has the wrong amount",
    "The payment page rejected my card",
    "Where can I download a receipt",
    "I cannot log in to my account",
    "The app crashes when I upload a file",
    "The reset password link does not work",
    "The dashboard shows a blank screen",
    "Notifications stopped working yesterday",
    "Please add dark mode",
    "I would like an export to PDF option",
    "Can you support team accounts",
    "Please add calendar integration",
    "I want a mobile app",
]

TRAINING_LABELS = [
    "billing",
    "billing",
    "billing",
    "billing",
    "billing",
    "technical",
    "technical",
    "technical",
    "technical",
    "technical",
    "feature_request",
    "feature_request",
    "feature_request",
    "feature_request",
    "feature_request",
]

OBSTACLE_TICKET = "I need a receipt"


def keyword_triage(ticket):
    text = ticket.lower()

    for label, keywords in KEYWORD_RULES.items():
        if any(keyword in text for keyword in keywords):
            return label

    return "unknown"


def show_keyword_demo():
    demo_tickets = [
        "Please refund my payment",
        "The app crashes when I upload a file",
        "Please add dark mode",
    ]

    for ticket in demo_tickets:
        print(f"{ticket} -> {keyword_triage(ticket)}")


def build_model():
    model = Pipeline(
        [
            ("tfidf", TfidfVectorizer(lowercase=True)),
            ("classifier", MultinomialNB()),
        ]
    )
    model.fit(TRAINING_TICKETS, TRAINING_LABELS)
    return model


def triage_ticket(model, ticket):
    return model.predict([ticket])[0]

You closed the exact gap exposed by the handwritten rules. Next up, you will test several labeled tickets before trying a message of your own.

Evaluate and Use the Triage Tool

Your trained classification pipeline now routes the receipt ticket to billing. That result proves the model can handle the synonym that defeated your keyword rules.

One successful prediction cannot show how consistently the model works. In this step, you will evaluate several labeled tickets before trying a new message yourself.

In this step, get ready to:
  • Compare model predictions with six expected queue labels.
  • Calculate an accuracy percentage for the demonstration set.
  • Enter a new support ticket through an interactive prompt.
Evaluate the classifier on labeled tickets

An evaluation set contains messages with known labels. Comparing those labels with the model's predictions gives you evidence about its behavior beyond one example.

  • Return to the first imports cell in Google Colab.
  • Place this import below the existing TfidfVectorizer import:
from sklearn.metrics import accuracy_score

What does this import provide?

The accuracy_score function measures the fraction of predictions that match their expected labels. It turns the six comparisons into one demonstration score.

  • Add a new code cell below the TRAINING_LABELS cell.
  • Create the evaluation messages with their expected labels by copying this code into the new cell:
TEST_TICKETS = [
    "Please refund the extra charge",
    "My password reset page is blank",
    "Can you add a mobile app",
    "I need a copy of my invoice",
    "The dashboard crashes after login",
    "Please add an export to PDF",
]

EXPECTED_LABELS = [
    "billing",
    "technical",
    "feature_request",
    "billing",
    "technical",
    "feature_request",
]

How does the evaluation data work?

  • TEST_TICKETS holds six messages that cover billing, technical problems, and feature requests.
  • EXPECTED_LABELS stores the correct queue for each message at the matching list position.
  • The matching order lets the evaluation compare each prediction with the correct answer.

Before you test the new data, think about whether the model will produce one queue label for every ticket.

  • Run the evaluation-data cell.
  • Add a temporary code cell below it.
  • Generate predictions for all six messages by running this expression:
model.predict(TEST_TICKETS)

What should you see?

You should see an array containing six queue labels. This confirms that the trained scikit-learn model can process a list of tickets in one prediction call.

Prediction cell not working?

Run the earlier cell that assigns the trained model to model. A restarted Colab runtime forgets variables until their cells run again.

Confirm that the evaluation-data cell defines TEST_TICKETS with the same capitalization used in the prediction expression.

help me diagnose why model.predict(TEST_TICKETS) is failing in Google Colab

  • Add a new code cell below triage_ticket().
  • Build the reusable evaluation function by copying this code into the new cell:
def evaluate_model(model):
    predictions = model.predict(TEST_TICKETS)

    for ticket, expected, predicted in zip(
        TEST_TICKETS, EXPECTED_LABELS, predictions, strict=True
    ):
        print(f"Ticket: {ticket}")
        print(f"Expected: {expected} | Predicted: {predicted}\n")

    accuracy = accuracy_score(EXPECTED_LABELS, predictions)
    print(f"Accuracy on demo test set: {accuracy:.0%}")

What does this function do?

  • model.predict(TEST_TICKETS) generates one predicted label for every test message.
  • zip(..., strict=True) keeps each ticket beside its expected label and predicted label. Strict matching also catches lists with different lengths.
  • accuracy_score compares the complete expected and predicted label lists.
  • :.0% formats the resulting fraction as a percentage with no decimal places.

Before you run the evaluation, predict whether every expected label will match its prediction.

  • Run the cell that defines evaluate_model().
  • Add a temporary code cell below the function.
  • Evaluate the trained model by running this call:
evaluate_model(model)

What should you see?

You should see one expected-versus-predicted comparison for each of the six tickets. The final line shows the accuracy percentage for this demonstration set.

This tiny curated set is a learning check. A production evaluation needs more varied tickets that reflect the messages customers actually submit.

Seeing missing or mismatched results?

Confirm that TEST_TICKETS and EXPECTED_LABELS each contain six entries. Their positions must stay aligned.

Check that the function uses the existing trained variable named model.

help me troubleshoot the output from evaluate_model(model)

Accept a new support ticket

A reusable triage prototype needs a simple way to receive a new message. The interactive function uses Python's input() function before passing the message to your existing classifier.

  • Add a new code cell below evaluate_model().
  • Create the interactive prompt by copying this function into the new cell:
def run_triage_demo(model):
    ticket = input("Enter a support ticket: ").strip()

    if not ticket:
        print("Please enter a ticket.")
        return

    prediction = triage_ticket(model, ticket)
    print(f"Predicted queue: {prediction}")

How does the prompt work?

  • input() pauses the cell while it waits for a support message.
  • strip() removes whitespace from the beginning and end of the message.
  • if not ticket stops an empty message from reaching the classifier.
  • triage_ticket(model, ticket) returns the model's predicted queue for the entered message.

Before you start the prompt, think about which queue your own message should reach.

  • Run the cell that defines run_triage_demo().
  • Add a temporary code cell below the function.
  • Start the interactive triage prompt by running this call:
run_triage_demo(model)

What happens next?

The cell pauses at Enter a support ticket: while it waits for your message. The completed result begins with Predicted queue: followed by one of the model's three classes.

  • Type a new support message into the input field.
  • Submit the message by pressing Enter.

You should see Predicted queue: followed by billing, technical, or feature_request. Your classifier can now route a message you provide yourself.

Prompt still waiting or returning an error?

Click inside the input field before typing your message. Pressing Enter submits the text to the running cell.

Run the earlier cells that define triage_ticket() and model if the runtime was restarted.

help me troubleshoot run_triage_demo(model) in my Colab notebook

Run the complete triage workflow

The final main() function connects the rule-based baseline, designed obstacle, trained classifier, evaluation, and interactive prompt. One run now demonstrates the full progression of your project.

This final cell pauses when it reaches the support-ticket prompt. Keep a new ticket in mind before you run it.

  • Add one final code cell below run_triage_demo().
  • Connect the complete workflow by copying this code into the final cell:
def main():
    print("Rule-based prototype")
    show_keyword_demo()

    print("\nDesigned obstacle")
    print(f"{OBSTACLE_TICKET} -> {keyword_triage(OBSTACLE_TICKET)}")

    print("\nTraining label counts")
    for label in ["billing", "technical", "feature_request"]:
        print(f"{label}: {TRAINING_LABELS.count(label)}")

    model = build_model()

    print("\nTrained model fixes the obstacle")
    print(f"{OBSTACLE_TICKET} -> {triage_ticket(model, OBSTACLE_TICKET)}")

    print("\nEvaluation")
    evaluate_model(model)

    print("\nTry your own ticket")
    run_triage_demo(model)


if __name__ == "__main__":
    main()

How does the complete workflow run?

  • main() runs each completed part of the project in its learning order.
  • build_model() creates a freshly trained model before evaluation begins.
  • evaluate_model(model) prints the labeled comparisons and demonstration accuracy.
  • run_triage_demo(model) finishes the workflow with your own support message.

Before you run the final cell, predict where the receipt ticket will route after the keyword version returns unknown.

  • Run the final cell.
  • Type a new support ticket when the input field appears.
  • Submit the ticket by pressing Enter.

You should see three obvious keyword results before the receipt ticket returns unknown. The trained model then routes the same receipt ticket to billing.

The remaining output shows six evaluation comparisons plus an accuracy percentage. It finishes with Predicted queue: followed by the class for your ticket.

Complete workflow stops early?

Run the notebook cells from top to bottom if the output mentions an undefined name. The complete workflow depends on every earlier constant and function.

Check that main() calls evaluate_model(model) before run_triage_demo(model).

help me find why the complete support ticket triage workflow stops before the interactive prediction

✔️ Awesome, I've got everything!

Your complete notebook now evaluates the classifier before accepting a support ticket from you. Make sure every code cell is saved in your Colab notebook.

ⓧ I'd like to double check the full code

Compare your combined notebook code with this complete support_ticket_triage.py reference.

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics import accuracy_score
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline

KEYWORD_RULES = {
    "billing": ["charge", "refund", "invoice", "payment", "card"],
    "technical": ["login", "log in", "crash", "password", "blank", "error"],
    "feature_request": ["add", "feature", "export", "integration", "mobile app"],
}

TRAINING_TICKETS = [
    "I was charged twice for my subscription",
    "Please refund my last payment",
    "My invoice has the wrong amount",
    "The payment page rejected my card",
    "Where can I download a receipt",
    "I cannot log in to my account",
    "The app crashes when I upload a file",
    "The reset password link does not work",
    "The dashboard shows a blank screen",
    "Notifications stopped working yesterday",
    "Please add dark mode",
    "I would like an export to PDF option",
    "Can you support team accounts",
    "Please add calendar integration",
    "I want a mobile app",
]

TRAINING_LABELS = [
    "billing",
    "billing",
    "billing",
    "billing",
    "billing",
    "technical",
    "technical",
    "technical",
    "technical",
    "technical",
    "feature_request",
    "feature_request",
    "feature_request",
    "feature_request",
    "feature_request",
]

TEST_TICKETS = [
    "Please refund the extra charge",
    "My password reset page is blank",
    "Can you add a mobile app",
    "I need a copy of my invoice",
    "The dashboard crashes after login",
    "Please add an export to PDF",
]

EXPECTED_LABELS = [
    "billing",
    "technical",
    "feature_request",
    "billing",
    "technical",
    "feature_request",
]

OBSTACLE_TICKET = "I need a receipt"


def keyword_triage(ticket):
    text = ticket.lower()

    for label, keywords in KEYWORD_RULES.items():
        if any(keyword in text for keyword in keywords):
            return label

    return "unknown"


def show_keyword_demo():
    demo_tickets = [
        "Please refund my payment",
        "The app crashes when I upload a file",
        "Please add dark mode",
    ]

    for ticket in demo_tickets:
        print(f"{ticket} -> {keyword_triage(ticket)}")


def build_model():
    model = Pipeline(
        [
            ("tfidf", TfidfVectorizer(lowercase=True)),
            ("classifier", MultinomialNB()),
        ]
    )
    model.fit(TRAINING_TICKETS, TRAINING_LABELS)
    return model


def triage_ticket(model, ticket):
    return model.predict([ticket])[0]


def evaluate_model(model):
    predictions = model.predict(TEST_TICKETS)

    for ticket, expected, predicted in zip(
        TEST_TICKETS, EXPECTED_LABELS, predictions, strict=True
    ):
        print(f"Ticket: {ticket}")
        print(f"Expected: {expected} | Predicted: {predicted}\n")

    accuracy = accuracy_score(EXPECTED_LABELS, predictions)
    print(f"Accuracy on demo test set: {accuracy:.0%}")


def run_triage_demo(model):
    ticket = input("Enter a support ticket: ").strip()

    if not ticket:
        print("Please enter a ticket.")
        return

    prediction = triage_ticket(model, ticket)
    print(f"Predicted queue: {prediction}")


def main():
    print("Rule-based prototype")
    show_keyword_demo()

    print("\nDesigned obstacle")
    print(f"{OBSTACLE_TICKET} -> {keyword_triage(OBSTACLE_TICKET)}")

    print("\nTraining label counts")
    for label in ["billing", "technical", "feature_request"]:
        print(f"{label}: {TRAINING_LABELS.count(label)}")

    model = build_model()

    print("\nTrained model fixes the obstacle")
    print(f"{OBSTACLE_TICKET} -> {triage_ticket(model, OBSTACLE_TICKET)}")

    print("\nEvaluation")
    evaluate_model(model)

    print("\nTry your own ticket")
    run_triage_demo(model)


if __name__ == "__main__":
    main()

Secret mission

Add Confidence-Based Human Review

Your classifier always chooses a support queue. Some tickets are too uncertain for automatic routing. In this mission, you will expose the prediction confidence and send low-confidence tickets to human review.

Clean Up Your Resources

Clean Up Your Resources

This project has no ongoing costs. Decide whether to keep your resources available, pause your work, or delete everything.

Resources you used:

  • A Google Colab notebook stored in Google Drive.
  • A temporary Google Colab runtime with scikit-learn version 1.9.1 installed.

Keep everything running

No action is needed. Choose this if you want to test more tickets or continue improving the classifier.

  • Leave the Google Colab notebook in Google Drive.
  • Keep the scikit-learn setup cell at the top of the notebook.
  • Rerun the setup cell whenever Google Colab assigns a fresh runtime.

Pause - I'll come back to this later

Pausing leaves your notebook in Google Drive. The temporary runtime can be deleted after it sits idle.

  • Close the Google Colab browser tab.
  • Reopen the notebook from Google Drive when you are ready to continue.
  • Rerun the scikit-learn setup cell after Google Colab assigns a new runtime.

Delete - I don't want to use this again

Deleting these resources removes your saved project work. Google Colab asks for confirmation before resetting the assigned virtual machine.

  • In Google Drive, locate the Colab notebook you created for this project.
  • Delete the notebook so it no longer appears among your saved Drive files.
  • Switch back to the Google Colab tab from this project.
  • Open Runtime in the top menu.
  • Select Disconnect and delete runtime.
  • Confirm the reset when Google Colab prompts you.
  • Check the connection area to confirm the previous runtime is no longer active.

Nice Work!

Nice Work!

Mission complete! Your Python support ticket triage tool now compares handwritten rules with a trained classifier.

You learned how to:

  • Built a rule-based baseline with strings, dictionaries, conditions, loops, and functions. Then triggered the receipt-ticket failure that exposed the limits of handwritten keywords.
  • Trained a text-classification pipeline in scikit-learn with TF-IDF features plus Multinomial Naive Bayes. The model routed the missed receipt ticket to billing.
  • Evaluated labeled test tickets by comparing expected queues with predictions. Then used run_triage_demo(model) to classify a ticket entered in the notebook.
  • Secret Mission: Added confidence-based routing with triage_with_confidence(model, ticket). Scores below 0.55 now go to human_review.

Ready to quiz yourself?