Bygg en Azure ML-triagetjeneste

Bygg og drift en observerbar ML-tjeneste for supporttriage på Azure.

Introduction

30 Second Summary

En supportsak kan se enkel ut helt til feil team får den. Feil ruting koster tid for både kunden og service desk-teamet.

I dette prosjektet skal du bygge en triagetjeneste på Azure som ruter supportsaker til billing, identity eller technical. Du kobler lekkasjefri evaluering til observerbar serverless inferens.

What You'll Build

Du sender en supportsak som JSON til et beskyttet Azure-endepunkt som svarer med en maskinlesbar triagebeslutning.

By the end of this project, you'll have:

  • En bevisst naiv baseline som gjør datalekkasje synlig med leaked_incidents=12 i terminalen.
  • En kvalitetssikret modellpipeline som lager et kuratert Parquet-datasett med DuckDB. Den dokumenterer group_overlap=0 i holdouten. Den lagrer TF-IDF sammen med logistisk regresjon som ett modellartefakt.
  • Et nøkkelbeskyttet endepunkt på Azure Functions Flex Consumption som returnerer team, confidence og model_version. Du kan finne den eksterne kjøringen i Azure Application Insights.
  • Secret Mission: En abstention-regel som setter needs_human_review for prediksjoner med konfidens under 0.60.

Are there any prerequisites?

Du trenger en Microsoft- eller GitHub-identitet med en aktiv Azure-subscription. Guiden tar deg gjennom prosjekt- og verktøyoppsettet på Linux.

Before We Start

This is your moment to commit to the Azure-based support-ticket triage service you are about to build. Your goal connects leakage-free evaluation with observable inference in an AI or ML engineering role.

Set Up Linux and Azure

Your triage service must use the same Python version locally and in Azure. A consistent runtime prevents deployment failures caused by incompatible packages.

This step prepares uv, the Azure command-line tools, and the complete project structure in Visual Studio Code. You will also confirm that your active Azure subscription is ready for a later Azure Functions deployment.

In this step, get ready to:
  • Create a project folder with uv 0.13.0 and a Python 3.13 virtual environment.
  • Install Azure CLI 2.91.0 and Azure Functions Core Tools v4.
  • Create the starter files and install azure-functions 2.3.0, joblib 1.6.0, scikit-learn 1.9.1, and DuckDB 1.5.6.
Prepare the project runtime

A virtual environment isolates the project's packages from your system installation. uv creates that environment with the exact Python version required by the Azure runtime and project dependencies.

  • Press the Super key to open your Linux app search.
  • Type Terminal into the search field.
  • Press Enter to open the terminal.
  • Move to your Desktop by running this command:
cd ~/Desktop

What does this command do?

The command moves your terminal into the Desktop directory. Your project folder will appear somewhere easy to find.

  • Create the ai-triage-service folder by running these commands:
mkdir ai-triage-service
cd ai-triage-service

What do these commands do?

The first command creates the project folder. The second command makes that folder your current terminal location.

The uv installer and later API checks both depend on curl. Check whether it is already available before you install uv.

  • Check your curl installation by running:
curl --version

What does this command show?

The output identifies your installed curl build. It also lists the protocols and features available on your machine.

✔️ I see curl version details

curl is available for the uv installer and the external request you will send later.

ⓧ Command not found

Your Ubuntu or Debian installation needs curl plus the packages used to configure Microsoft's package repositories.

  • Install the required APT packages by running:
sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates curl gnupg lsb-release

What do these commands install?

APT refreshes its package index before installing curl and the certificate tooling. The same tooling supports the Azure CLI installation later in this step.

  • Approve the package installation if APT asks for confirmation.
  • Confirm curl is available by running:
curl --version

What confirms success?

You should now see curl version information followed by its supported protocols and features.

uv manages the pinned Python installation without replacing your existing system Python. Start by checking whether the required uv release is already present.

  • Check your uv version by running:
uv --version

What does this command show?

The output reports the uv release currently available through your terminal. This project uses 0.13.0.

✔️ I see the required uv version

Great. uv 0.13.0 is ready to manage the project's Python runtime.

ⓧ I see an older uv version

  • Install uv 0.13.0 by running the official versioned installer:
curl -LsSf https://astral.sh/uv/0.13.0/install.sh | sh

What does this installer do?

The script downloads the pinned uv release and installs its command-line binary for your user account. Pinning the installer keeps this setup reproducible.

  • Follow the installer's final message to refresh your shell environment.
  • Confirm the updated version by running:
uv --version

What confirms the update?

The output should identify uv 0.13.0.

ⓧ Command not found

  • Install uv 0.13.0 by running the official versioned installer:
curl -LsSf https://astral.sh/uv/0.13.0/install.sh | sh

What does this installer do?

The script installs the pinned uv binary for your user account. Its final output tells you how to make the command available in your current shell.

  • Follow the installer's final message to refresh your shell environment.
  • Confirm the installation by running:
uv --version

What confirms the installation?

The output should identify uv 0.13.0.

  • Install Python 3.13 through uv by running:
uv python install 3.13

What does this command do?

uv downloads a managed Python 3.13 runtime. Your system Python remains untouched.

  • Create the project's .venv environment by running:
uv venv --python 3.13

What does this command create?

uv creates an isolated .venv directory backed by its managed Python 3.13 runtime.

  • Activate the virtual environment by running:
source .venv/bin/activate

What does activation change?

Activation makes the environment's Python executable and installed packages the defaults for this terminal session.

  • Check the environment's Python version by running:
.venv/bin/python --version

What does this check prove?

This command reads the interpreter inside .venv. It confirms which Python runtime the project actually uses.

✔️ I see the required Python version

The virtual environment is using Python 3.13. Your local runtime now matches the version planned for Azure.

ⓧ I see an older Python version

  • Remove the .venv directory with your Linux file manager.
  • Recreate the environment with Python 3.13 by running:
uv venv --python 3.13

Why recreate the environment?

A virtual environment keeps the interpreter selected when it was created. Recreating it applies the required Python version cleanly.

  • Activate the recreated environment by running:
source .venv/bin/activate

What does activation confirm?

The terminal now resolves Python and installed packages from the recreated .venv directory.

  • Check the corrected version by running:
.venv/bin/python --version

What should you see?

The output should identify Python 3.13.

ⓧ The environment is missing

  • Create the missing environment by running:
uv venv --python 3.13

What does this command restore?

The command creates .venv with the uv-managed Python 3.13 interpreter.

  • Activate the new environment by running:
source .venv/bin/activate

What changes after activation?

The active terminal session now uses the project environment for Python commands and packages.

  • Confirm the new interpreter by running:
.venv/bin/python --version

What should you see?

The output should identify Python 3.13.

Install Azure tools and sign in

The Azure CLI creates cloud resources from your terminal. Core Tools publishes the function project to those resources later.

Why use command-line setup?

A command-line workflow makes the cloud setup repeatable and reviewable. It also avoids hidden portal settings that are difficult to reproduce.

  • Check your Azure CLI version by running:
az version

What does this command show?

The output lists the Azure CLI release and its supporting components. This project expects Azure CLI 2.91.0.

✔️ I see the required Azure CLI version

Azure CLI 2.91.0 is ready for authentication and resource creation.

ⓧ I see an older Azure CLI version

The APT repository can install the pinned Azure CLI release on supported Ubuntu and Debian versions.

  • Configure Microsoft's Azure CLI repository and install version 2.91.0 by running:
sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates curl gnupg lsb-release
sudo mkdir -p /etc/apt/keyrings
curl -sLS https://packages.microsoft.com/keys/microsoft.asc |
  gpg --dearmor | sudo tee /etc/apt/keyrings/microsoft.gpg > /dev/null
AZ_DIST=$(lsb_release -cs)
echo "Types: deb
URIs: https://packages.microsoft.com/repos/azure-cli/
Suites: ${AZ_DIST}
Components: main
Architectures: $(dpkg --print-architecture)
Signed-by: /etc/apt/keyrings/microsoft.gpg" | sudo tee /etc/apt/sources.list.d/azure-cli.sources
sudo apt-get update
AZ_VER=2.91.0
sudo apt-get install azure-cli=${AZ_VER}-1~${AZ_DIST}

What do these commands do?

The commands add Microsoft's signing key and the Azure CLI repository for your Linux distribution. APT then installs the pinned 2.91.0 package.

  • Approve the package installation if APT asks for confirmation.
  • Confirm the updated release by running:
az version

What confirms the update?

The Azure CLI version field should show 2.91.0.

ⓧ Command not found

The Azure CLI repository must be added before APT can install the command.

  • Configure Microsoft's Azure CLI repository and install version 2.91.0 by running:
sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates curl gnupg lsb-release
sudo mkdir -p /etc/apt/keyrings
curl -sLS https://packages.microsoft.com/keys/microsoft.asc |
  gpg --dearmor | sudo tee /etc/apt/keyrings/microsoft.gpg > /dev/null
AZ_DIST=$(lsb_release -cs)
echo "Types: deb
URIs: https://packages.microsoft.com/repos/azure-cli/
Suites: ${AZ_DIST}
Components: main
Architectures: $(dpkg --print-architecture)
Signed-by: /etc/apt/keyrings/microsoft.gpg" | sudo tee /etc/apt/sources.list.d/azure-cli.sources
sudo apt-get update
AZ_VER=2.91.0
sudo apt-get install azure-cli=${AZ_VER}-1~${AZ_DIST}

What do these commands do?

The commands register Microsoft's signed Azure CLI package source for your distribution. APT then installs the pinned release.

  • Approve the package installation if APT asks for confirmation.
  • Confirm the installation by running:
az version

What confirms the installation?

The Azure CLI version field should show 2.91.0.

  • Check your Core Tools version by running:
func --version

What does this command show?

The output reports the installed Core Tools release. A version beginning with 4 belongs to the required GA major version.

✔️ I see the required major version

Core Tools 4.x is ready to publish the Python function project later.

ⓧ I see an older major version

Ubuntu and Debian can install Core Tools v4 from Microsoft's Linux package repository.

  • Configure the Microsoft package repository and install Core Tools v4 by running:
source /etc/os-release
curl -sSL -O https://packages.microsoft.com/config/$ID/$VERSION_ID/packages-microsoft-prod.deb
sudo apt install ./packages-microsoft-prod.deb
sudo apt-get update
sudo apt-get install azure-functions-core-tools-4

What do these commands do?

The commands select the repository configuration for your Linux distribution. APT then installs the GA Core Tools package for Functions runtime 4.x.

  • Approve each package installation if APT asks for confirmation.
  • Confirm the installed major version by running:
func --version

What confirms the update?

The output should begin with 4.

ⓧ Command not found

Ubuntu and Debian need Microsoft's Linux package repository before Core Tools can be installed.

  • Configure the Microsoft package repository and install Core Tools v4 by running:
source /etc/os-release
curl -sSL -O https://packages.microsoft.com/config/$ID/$VERSION_ID/packages-microsoft-prod.deb
sudo apt install ./packages-microsoft-prod.deb
sudo apt-get update
sudo apt-get install azure-functions-core-tools-4

What do these commands do?

The commands register Microsoft's repository for your detected Linux distribution. APT then installs Core Tools v4.

  • Approve each package installation if APT asks for confirmation.
  • Confirm the installation by running:
func --version

What confirms the installation?

The output should begin with 4.

Using Another Linux Distribution

The APT path above targets Ubuntu and Debian. Follow Microsoft's Core Tools Linux instructions for the repository path supported by your distribution.

Azure authentication opens a browser-based sign-in flow. Your password stays in the browser and is not entered into the terminal.

  • Use the account link in Microsoft's Flex Consumption guide to create an Azure account with an active subscription if you do not already have one.
  • Start the Azure sign-in flow by running:
az login

What happens during sign-in?

Azure CLI opens a browser so you can authenticate with your Microsoft or GitHub identity. Return to the terminal after the authentication flow completes.

  • Complete the browser sign-in with the identity that owns your active subscription.
  • Confirm the active subscription by running:
az account show

What does this output prove?

The returned account object identifies the subscription Azure CLI uses for later resource commands. Confirm that the subscription is active before continuing.

Azure sign-in not completing?

Check that the browser uses the same identity that owns your Azure subscription. A personal identity and an organisation identity can expose different subscriptions.

If your Linux distribution cannot open the browser flow, use the alternative sign-in guidance shown by Azure CLI.

Help me troubleshoot Azure CLI authentication on Linux.

Create the starter files and verify setup

The project separates development-only data tooling from the smaller dependency set deployed to Azure. Its starter files also define the complete training pipeline and HTTP function that later steps activate.

  • Open the current ai-triage-service folder in VS Code by running:
code .

What does this command open?

VS Code opens the current terminal directory as a workspace. The ai-triage-service folder becomes the project root shown in the Explorer sidebar.

  • Select the trust option if VS Code shows the Workspace Trust dialog for the folder you created.
  • Create requirements.txt in the ai-triage-service project root.
  • Create requirements-dev.txt in the ai-triage-service project root.
  • Create host.json in the ai-triage-service project root.
  • Create .funcignore in the ai-triage-service project root.

How are these files divided?

The requirements files separate cloud packages from development-only DuckDB tooling. The Functions configuration files define the host version and exclude local-only content from deployment.

  • Create the data folder inside ai-triage-service.
  • Create tickets.csv inside the new data folder.
  • Create train.py in the ai-triage-service project root.
  • Create function_app.py in the ai-triage-service project root.
  • Copy the matching contents from the full-code reference below into each file.

✔️ I copied the starter files

Save every file in VS Code. Your Explorer sidebar should show six files in the project root plus tickets.csv inside the data folder.

ⓧ I'd like to double check the full code

azure-functions==2.3.0
joblib==1.6.0
scikit-learn==1.9.1

What does this file control?

This file contains the packages needed by the deployed function. DuckDB stays out of the cloud dependency set because training happens locally.

-r requirements.txt
duckdb==1.5.6

What does this file add?

The development requirements include every deployment package through requirements.txt. They add DuckDB for local data curation.

{
  "version": "2.0"
}

What does this file configure?

The host configuration targets the schema used by Azure Functions runtime version 2.0 and later.

.venv
.git
__pycache__
data
requirements-dev.txt
train.py
artifacts/curated_tickets.parquet

What stays out of deployment?

The ignore file excludes the local environment, raw training data, development dependencies, training script, and curated dataset. The trained model remains available for the function package.

incident_id,text,team
B001,I was charged twice for the same monthly plan,billing
B001,The subscription payment appears two times on my card,billing
B001,Duplicate charge for one subscription renewal,billing
B002,My invoice total is higher than the agreed price,billing
B002,The amount on this invoice does not match our contract,billing
B002,Please explain the unexpected amount on my latest invoice,billing
B003,The refund was approved but has not reached my account,billing
B003,I am still waiting for the promised refund,billing
B003,Approved refund is missing from the payment account,billing
B004,I cancelled but the next renewal was still billed,billing
B004,Charges continued after I ended the subscription,billing
B004,The cancelled plan renewed and created a new charge,billing
I001,I cannot sign in after resetting my password,identity
I001,The new password is accepted but login still fails,identity
I001,Password reset completed and I am still unable to log in,identity
I002,My authentication codes stopped working after changing phones,identity
I002,MFA codes are rejected on my new device,identity
I002,I replaced my phone and cannot complete multifactor login,identity
I003,My account is locked after several login attempts,identity
I003,Too many sign in attempts locked my user,identity
I003,Please unlock the account blocked during login,identity
I004,The invited colleague cannot access the workspace,identity
I004,A new user accepted the invite but has no workspace access,identity
I004,Workspace permission is missing for an invited employee,identity
T001,The desktop application crashes when I open reports,technical
T001,Opening the reports screen closes the app,technical
T001,Report loading causes the client application to crash,technical
T002,API requests time out during normal traffic,technical
T002,The integration receives repeated gateway timeouts,technical
T002,Our API calls fail because the service stops responding,technical
T003,Customer records are no longer synchronizing,technical
T003,The data sync stopped updating records this morning,technical
T003,Changes in customer records do not reach the integration,technical
T004,The export button creates an empty file,technical
T004,CSV export completes but the downloaded file has no rows,technical
T004,Data export returns an empty document,technical

What does this dataset contain?

The dataset contains 12 incident groups across billing, identity, and technical routing targets. Each incident has three related text variants.

import argparse
import csv
from datetime import datetime, timezone
from pathlib import Path

import duckdb
import joblib
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report
from sklearn.model_selection import StratifiedGroupKFold
from sklearn.pipeline import Pipeline

ROOT = Path(__file__).parent
RAW_DATA = ROOT / "data" / "tickets.csv"
ARTIFACTS = ROOT / "artifacts"
CURATED_DATA = ARTIFACTS / "curated_tickets.parquet"
MODEL_PATH = ARTIFACTS / "model.joblib"
MODEL_VERSION = "1.0.0"
ALLOWED_TEAMS = {"billing", "identity", "technical"}


def build_pipeline() -> Pipeline:
    return Pipeline(
        [
            ("tfidf", TfidfVectorizer(ngram_range=(1, 2), min_df=1)),
            ("classifier", LogisticRegression(l1_ratio=0.0, max_iter=500)),
        ]
    )


def load_raw_rows() -> list[dict[str, str]]:
    with RAW_DATA.open(encoding="utf-8", newline="") as source:
        return list(csv.DictReader(source))


def run_naive() -> None:
    rows = load_raw_rows()
    train_rows = [row for index, row in enumerate(rows) if index % 3 != 2]
    test_rows = [row for index, row in enumerate(rows) if index % 3 == 2]

    train_groups = {row["incident_id"] for row in train_rows}
    test_groups = {row["incident_id"] for row in test_rows}
    leaked_groups = sorted(train_groups & test_groups)

    model = build_pipeline()
    model.fit(
        [row["text"] for row in train_rows],
        [row["team"] for row in train_rows],
    )
    accuracy = model.score(
        [row["text"] for row in test_rows],
        [row["team"] for row in test_rows],
    )

    print(f"naive_accuracy={accuracy:.3f}")
    print(f"leaked_incidents={len(leaked_groups)}")
    print(f"leaked_ids={','.join(leaked_groups)}")
    print("OBSTACLE: row-level split leaks incident history.")


def curate_rows() -> list[tuple[str, str, str]]:
    ARTIFACTS.mkdir(exist_ok=True)
    CURATED_DATA.unlink(missing_ok=True)

    conflicts = duckdb.sql(
        f"""
        SELECT incident_id
        FROM '{RAW_DATA.as_posix()}'
        GROUP BY incident_id
        HAVING count(DISTINCT team) > 1
        """
    ).fetchall()
    if conflicts:
        raise ValueError(f"Conflicting labels for incidents: {conflicts}")

    duckdb.sql(
        f"""
        COPY (
            SELECT DISTINCT incident_id, text, team
            FROM '{RAW_DATA.as_posix()}'
            WHERE incident_id IS NOT NULL
              AND text IS NOT NULL
              AND team IN ('billing', 'identity', 'technical')
        ) TO '{CURATED_DATA.as_posix()}'
        """
    )

    rows = duckdb.sql(
        f"SELECT incident_id, text, team FROM '{CURATED_DATA.as_posix()}'"
    ).fetchall()
    if not rows:
        raise ValueError("The quality gate produced no training rows.")
    if {row[2] for row in rows} != ALLOWED_TEAMS:
        raise ValueError("The curated data does not contain every allowed team.")
    return rows


def run_clean() -> None:
    rows = curate_rows()
    groups = [row[0] for row in rows]
    texts = [row[1] for row in rows]
    labels = [row[2] for row in rows]
    dummy_features = [[0] for _ in rows]

    splitter = StratifiedGroupKFold(
        n_splits=3,
        shuffle=True,
        random_state=42,
    )
    train_indices, test_indices = next(
        splitter.split(dummy_features, labels, groups)
    )

    train_texts = [texts[index] for index in train_indices]
    test_texts = [texts[index] for index in test_indices]
    train_labels = [labels[index] for index in train_indices]
    test_labels = [labels[index] for index in test_indices]
    train_groups = {groups[index] for index in train_indices}
    test_groups = {groups[index] for index in test_indices}
    overlap = train_groups & test_groups

    if overlap:
        raise ValueError(f"Group leakage detected: {sorted(overlap)}")

    model = build_pipeline()
    model.fit(train_texts, train_labels)
    predictions = model.predict(test_texts)
    accuracy = model.score(test_texts, test_labels)

    print(f"group_overlap={len(overlap)}")
    print(f"clean_accuracy={accuracy:.3f}")
    print(
        classification_report(
            test_labels,
            predictions,
            labels=sorted(ALLOWED_TEAMS),
            zero_division=0.0,
        )
    )

    metadata = {
        "model_version": MODEL_VERSION,
        "trained_at_utc": datetime.now(timezone.utc).isoformat(),
        "holdout_accuracy": accuracy,
        "training_incidents": sorted(train_groups),
        "holdout_incidents": sorted(test_groups),
    }
    joblib.dump({"pipeline": model, "metadata": metadata}, MODEL_PATH)
    print(f"saved_model={MODEL_PATH.relative_to(ROOT)}")


def main() -> None:
    parser = argparse.ArgumentParser()
    parser.add_argument("--mode", choices=("naive", "clean"), required=True)
    args = parser.parse_args()

    if args.mode == "naive":
        run_naive()
    else:
        run_clean()


if __name__ == "__main__":
    main()

What does the training script contain?

The script contains the naive leakage demonstration and the clean group-aware training path. It also defines the curated Parquet output and versioned model artifact used later.

import json
import logging
from pathlib import Path

import azure.functions as func
import joblib

MODEL_PATH = Path(__file__).parent / "artifacts" / "model.joblib"
MODEL_BUNDLE = joblib.load(MODEL_PATH)
PIPELINE = MODEL_BUNDLE["pipeline"]
METADATA = MODEL_BUNDLE["metadata"]

app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION)


def predict_ticket(text: str) -> dict[str, object]:
    probabilities = PIPELINE.predict_proba([text])[0]
    best_index = max(range(len(probabilities)), key=probabilities.__getitem__)
    return {
        "team": str(PIPELINE.classes_[best_index]),
        "confidence": round(float(probabilities[best_index]), 4),
        "model_version": METADATA["model_version"],
    }


@app.function_name(name="TriageTicket")
@app.route(route="triage", methods=[func.HttpMethod.POST])
def triage(req: func.HttpRequest) -> func.HttpResponse:
    try:
        payload = req.get_json()
    except ValueError:
        return func.HttpResponse(
            body=json.dumps({"error": "Request body must be valid JSON."}),
            status_code=400,
        )

    text = payload.get("text") if isinstance(payload, dict) else None
    if not isinstance(text, str) or not text.strip():
        return func.HttpResponse(
            body=json.dumps({"error": "A non-empty text field is required."}),
            status_code=400,
        )

    result = predict_ticket(text.strip())
    logging.info(
        "triage_prediction team=%s confidence=%.4f model_version=%s",
        result["team"],
        result["confidence"],
        result["model_version"],
    )
    return func.HttpResponse(body=json.dumps(result), status_code=200)


if __name__ == "__main__":
    print(
        json.dumps(
            predict_ticket("MFA codes stopped working after I changed phones"),
            indent=2,
        )
    )

What does the function module contain?

The module loads the locally generated model bundle and defines the protected HTTP route. It returns the predicted team, confidence, and model version without logging raw support text.

Only load the model artifact generated by this project. joblib.load relies on Python pickle behavior and must never receive an untrusted model file.

  • Save every project file in VS Code.
  • Switch back to the terminal where .venv is active.
  • Install the development dependencies by running:
uv pip install -r requirements-dev.txt

What does this command install?

uv reads both requirements files and installs the pinned runtime packages into .venv. DuckDB is included for local development while the Azure deployment keeps the smaller runtime dependency set.

The install can take a little while because scikit-learn includes compiled numerical components. Package download and installation output means the command is still working.

Dependency installation failed?

Confirm that your terminal prompt still shows the active .venv environment. Also confirm that .venv/bin/python --version reports Python 3.13.

Check that the package pins in both requirements files match the full-code reference exactly.

Help me diagnose my uv dependency installation.

Before the final check, which versions do you expect the terminal to report for uv, Python, Azure CLI, and Core Tools?

  • Verify every required command from the ai-triage-service project root by running:
uv --version
.venv/bin/python --version
az version
func --version
curl --version
az account show

What does the final check prove?

These checks cover the package manager, project interpreter, Azure tooling, HTTP client, and authenticated subscription. Together they prove the local and cloud setup is ready for the model workflow.

You should see uv 0.13.0, Python 3.13, Azure CLI 2.91.0, and a Core Tools version beginning with 4. curl should print its version details, while Azure CLI should return your active subscription.

Setup complete. Your project root now contains the full starter structure, and every required local tool is responding from the same environment.

Your development environment and Azure access are ready. Next, you will run the deliberately naive classifier and expose incident leakage in its holdout split.

Expose the Leaky Baseline

Your Linux toolchain is ready. The project files are in place.

A high holdout score is unreliable when text variants from one incident appear in both training data and test data. This form of data leakage is the flaw you will make visible in the terminal.

In this step, get ready to:
  • Inspect how repeated text variants map to shared incident IDs.
  • Run the naive row-level holdout.
  • Identify the incident overlap behind the baseline score.
Inspect the repeated incident groups

The data/tickets.csv file contains 36 support ticket rows. Each incident appears as three paraphrased descriptions with one routing label.

  • Expand the data folder in the VS Code Explorer sidebar.
  • Select tickets.csv.
  • Scan the incident_id column from B001 through T004.
  • Compare the three rows for B001.

You'll see each incident ID repeated three times. The wording changes across those rows.

The incident ID stays fixed. The routing label also stays fixed.

Why Incident IDs Matter

The incident_id value identifies one underlying support history. Its three text values are alternate descriptions of that history.

Treating those descriptions as unrelated rows can place knowledge from one incident on both sides of an evaluation split. The model then receives clues about its test data during training.

Test the independent-row assumption

The naive baseline assumes each row is independent. Its row-position split tests that assumption with a first holdout score.

Before you run it, do you think a row-level split can keep related incident histories apart?

  • Return to the VS Code terminal panel from earlier.
  • Execute the naive mode from the project root by running this command:
python train.py --mode naive

What Did This Run Prove?

  • The run_naive() function loads all 36 ticket rows.
  • The split keeps the first two variants from each incident in the training set.
  • The split places the third variant from each incident in the test set.
  • The scikit-learn pipeline combines TF-IDF features with logistic regression.
  • The overlap check compares the incident IDs on both sides of the holdout.

You'll see naive_accuracy= followed by the baseline score. You'll also see leaked_incidents=12.

The leaked_ids= line lists B001,B002,B003,B004,I001,I002,I003,I004,T001,T002,T003,T004. The final line reads OBSTACLE: row-level split leaks incident history..

This shortfall is intentional. You caught the model's credibility problem before treating its score as evidence.

Baseline Command Failed?

  • Confirm the terminal is in the project root where train.py exists.
  • Confirm the virtual environment from the previous step is still active.
  • If Python reports a missing package, reinstall the development requirements from the previous step.

Help me diagnose the naive training command.

The baseline now exposes all 12 leaked incident histories. Next, you'll replace this row-level assumption with a quality gate and group-aware evaluation.

Build the Quality-Gated Pipeline

The naive run exposed the weakness in the first evaluation. All 12 incident groups appeared in both the training data and the holdout data.

A high holdout score has little value when related text from one incident appears on both sides of the split. The clean path must remove that data leakage before evaluation and model storage.

DuckDB will enforce the row-level quality rules. Scikit-learn will keep incident groups intact while it fits the classifier.

In this step, get ready to:
  • Audit the quality gate that produces the curated Parquet dataset.
  • Trace the group-aware holdout plus the TF-IDF and logistic regression pipeline.
  • Run the clean training path to verify zero group overlap and save the model bundle.
Inspect the quality gate

A quality gate defines which records can reach training. This one detects incidents assigned conflicting labels before it creates the curated dataset.

The gate then keeps distinct rows with a valid incident ID. It only accepts the three supported routing teams.

  • Select train.py from the VS Code Explorer sidebar.
  • Find the curate_rows() function.
  • Confirm the first query searches for incident IDs assigned to multiple team labels.
  • Check the filtering query for null incident IDs and null ticket text.
  • Confirm the filtering query only accepts billing, identity, or technical as the team.
  • Confirm the COPY query writes distinct valid rows to artifacts/curated_tickets.parquet.
  • Check the final guards for an empty dataset or a missing allowed team.

Why gate the data first?

The quality gate gives training a defined input contract. Conflicting labels would make the expected answer ambiguous.

Null fields or unknown teams would also weaken evaluation. The curated dataset gives every later run the same validated source.

Trace the group-aware training path

A group-aware split treats every incident_id as one indivisible unit. Text variants from one incident therefore stay within a single partition.

The training pipeline also packages text transformation with classification. The saved object can apply the same transformations during inference.

  • Scroll to build_pipeline() in train.py.
  • Confirm the vectorizer includes single words plus two-word sequences.
  • Confirm the classifier uses a zero L1 ratio with a limit of 500 iterations.
  • Scroll to run_clean().
  • Trace each incident_id into the groups list.
  • Confirm the splitter uses three folds with shuffling plus the fixed seed of 42.
  • Follow the returned indices into the training groups and holdout groups.
  • Confirm the overlap guard runs before the model is fitted.

Why use grouped stratification?

Stratification seeks a useful balance of routing labels across each partition. Grouping keeps all variants of one incident together.

The holdout now measures performance on unseen incidents. That makes the evaluation evidence defensible.

Prove the clean split

Clean mode runs the quality gate before creating the group-aware holdout. It then trains the combined pipeline from the curated rows.

Before you run it, consider whether any incident IDs will remain in both partitions.

  • Return to the VS Code terminal from the previous step.
  • Execute the clean training path by running this command:
python train.py --mode clean

What does clean mode do?

  • The curate_rows() gate checks label conflicts before producing the curated dataset.
  • The StratifiedGroupKFold splitter assigns complete incident groups to training or holdout.
  • The fitted Pipeline produces the predictions used in the classification report.
  • The joblib.dump call stores the fitted pipeline with its evaluation metadata.

You'll see group_overlap=0 before the classification report. The run ends with saved_model=artifacts/model.joblib.

That's the evaluation gap closed. Your holdout now contains incidents the model did not see during training.

  • Expand the artifacts folder in the VS Code Explorer sidebar.

You'll see curated_tickets.parquet and model.joblib inside the folder.

What did the run save?

The artifacts/curated_tickets.parquet dataset contains the 36 distinct valid ticket rows across billing, identity, and technical.

The artifacts/model.joblib bundle contains the fitted pipeline plus its metadata.

Its metadata keys are model_version, trained_at_utc, holdout_accuracy, training_incidents, and holdout_incidents. The stored model version is 1.0.0.

Missing an artifact or seeing overlap?

  • Confirm the terminal is in the project folder that contains train.py.
  • Check that the activated virtual environment still has the dependencies from requirements-dev.txt.
  • Verify the group-aware split still receives the incident groups if the overlap count is greater than zero.

Ask for help with the exact terminal output: Help me debug the clean training pipeline.

Your leakage-free model bundle is ready. Next, you'll turn it into a validated inference function.

Create the Azure Function API

Your leakage-free scikit-learn pipeline is now stored in artifacts/model.joblib. A model file alone cannot receive support tickets or return predictions.

This step turns that artifact into an Azure Functions module using the Python v2 programming model. The protected HTTP route validates each request. Its JSON response exposes a stable prediction contract.

In this step, get ready to:
  • Load the trusted model bundle once when the function module starts.
  • Convert class probabilities into a stable triage response.
  • Expose a protected POST route with input validation.
Load the model and build the response contract

The serialized bundle contains the fitted pipeline plus its training metadata. Loading it when the module starts keeps prediction requests independent from the training program.

  • Select function_app.py in the VS Code Explorer sidebar.
  • Press Ctrl+A to select the existing contents.
  • Replace the selected contents with the model-loading setup by pasting this code:
import json
import logging
from pathlib import Path

import azure.functions as func
import joblib

MODEL_PATH = Path(__file__).parent / "artifacts" / "model.joblib"
MODEL_BUNDLE = joblib.load(MODEL_PATH)
PIPELINE = MODEL_BUNDLE["pipeline"]
METADATA = MODEL_BUNDLE["metadata"]

app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION)

What does this setup do?

  • MODEL_PATH points to the model artifact beside the function source.
  • Joblib reads the trusted bundle that you created during training.
  • PIPELINE holds the fitted text classifier used for inference.
  • METADATA provides the saved model version for every response.
  • FunctionApp applies FUNCTION authorization to the hosted app.
  • Only load the artifact you generated locally because Joblib persistence relies on pickle.
  • Save function_app.py.
  • Expand the artifacts folder in the VS Code Explorer sidebar.

You should see model.joblib inside artifacts. This confirms that MODEL_PATH targets the artifact produced by your clean training run.

The prediction helper converts the classifier's probability array into the three fields expected by every caller. A direct entry point lets you validate that contract without starting the Functions host.

  • Place your cursor below the app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION) line.
  • Add the prediction helper plus its direct validation entry point by pasting this code:


def predict_ticket(text: str) -> dict[str, object]:
    probabilities = PIPELINE.predict_proba([text])[0]
    best_index = max(range(len(probabilities)), key=probabilities.__getitem__)
    return {
        "team": str(PIPELINE.classes_[best_index]),
        "confidence": round(float(probabilities[best_index]), 4),
        "model_version": METADATA["model_version"],
    }


if __name__ == "__main__":
    print(
        json.dumps(
            predict_ticket("MFA codes stopped working after I changed phones"),
            indent=2,
        )
    )

How does prediction work?

  • predict_proba() returns one probability for each class in PIPELINE.classes_.
  • best_index identifies the class with the highest probability.
  • predict_ticket() returns the predicted team as a string.
  • confidence rounds the winning probability to four decimal places.
  • model_version comes from the metadata stored with the fitted model.
  • The final block calls the helper directly with the MFA example.
  • Save function_app.py.

Before you run the module, which support team do you expect the MFA example to select?

  • Validate the prediction helper from the activated terminal by running:
python function_app.py

What should I see?

  • You should see one formatted JSON object.
  • team contains the predicted support route.
  • confidence contains a decimal probability.
  • model_version contains 1.0.0.

Local validation failing?

  • Confirm that your terminal prompt still shows the activated virtual environment.
  • Check that artifacts/model.joblib exists inside the project folder.
  • Compare the indentation in predict_ticket() with the code block above.

Help me debug the local prediction helper.

Add the protected POST route

The HTTP trigger gives the model a stable inference boundary. Function-level authorization requires a key after deployment.

The handler also rejects malformed input before prediction. Its log records operational details without storing the submitted customer text.

  • Find if __name__ == "__main__": near the bottom of function_app.py.
  • Place your cursor on the blank line directly above that block.
  • Insert the protected triage handler by pasting this code:
@app.function_name(name="TriageTicket")
@app.route(route="triage", methods=[func.HttpMethod.POST])
def triage(req: func.HttpRequest) -> func.HttpResponse:
    try:
        payload = req.get_json()
    except ValueError:
        return func.HttpResponse(
            body=json.dumps({"error": "Request body must be valid JSON."}),
            status_code=400,
        )

    text = payload.get("text") if isinstance(payload, dict) else None
    if not isinstance(text, str) or not text.strip():
        return func.HttpResponse(
            body=json.dumps({"error": "A non-empty text field is required."}),
            status_code=400,
        )

    result = predict_ticket(text.strip())
    logging.info(
        "triage_prediction team=%s confidence=%.4f model_version=%s",
        result["team"],
        result["confidence"],
        result["model_version"],
    )
    return func.HttpResponse(body=json.dumps(result), status_code=200)

What does the route do?

  • function_name registers the function as TriageTicket.
  • route exposes the handler at triage.
  • methods limits the handler to POST requests.
  • req.get_json() parses the request body before inference.
  • The validation branches return status code 400 for invalid JSON or empty text.
  • logging.info() records the predicted team.
  • The same log entry records confidence plus model version without raw customer text.
  • HttpResponse serializes the successful result with status code 200.
  • Save function_app.py.

✔️ Awesome, I've got everything!

Your saved module now contains the model loader, prediction helper, protected route, input validation, safe logging, and direct local check.

ⓧ I'd like to double check the full code

import json
import logging
from pathlib import Path

import azure.functions as func
import joblib

MODEL_PATH = Path(__file__).parent / "artifacts" / "model.joblib"
MODEL_BUNDLE = joblib.load(MODEL_PATH)
PIPELINE = MODEL_BUNDLE["pipeline"]
METADATA = MODEL_BUNDLE["metadata"]

app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION)


def predict_ticket(text: str) -> dict[str, object]:
    probabilities = PIPELINE.predict_proba([text])[0]
    best_index = max(range(len(probabilities)), key=probabilities.__getitem__)
    return {
        "team": str(PIPELINE.classes_[best_index]),
        "confidence": round(float(probabilities[best_index]), 4),
        "model_version": METADATA["model_version"],
    }


@app.function_name(name="TriageTicket")
@app.route(route="triage", methods=[func.HttpMethod.POST])
def triage(req: func.HttpRequest) -> func.HttpResponse:
    try:
        payload = req.get_json()
    except ValueError:
        return func.HttpResponse(
            body=json.dumps({"error": "Request body must be valid JSON."}),
            status_code=400,
        )

    text = payload.get("text") if isinstance(payload, dict) else None
    if not isinstance(text, str) or not text.strip():
        return func.HttpResponse(
            body=json.dumps({"error": "A non-empty text field is required."}),
            status_code=400,
        )

    result = predict_ticket(text.strip())
    logging.info(
        "triage_prediction team=%s confidence=%.4f model_version=%s",
        result["team"],
        result["confidence"],
        result["model_version"],
    )
    return func.HttpResponse(body=json.dumps(result), status_code=200)


if __name__ == "__main__":
    print(
        json.dumps(
            predict_ticket("MFA codes stopped working after I changed phones"),
            indent=2,
        )
    )

This reference shows the exact cumulative contents of function_app.py for this step.

Before you run the complete module, do you expect the new HTTP handler to change the helper's local response contract?

  • Verify the completed module from the activated terminal by running:
python function_app.py

What should I see?

That is the serving contract working. You should see one JSON object containing team, confidence, and model_version for the MFA example.

Response contract missing?

  • Compare function_app.py with the complete reference above.
  • Confirm that the route decorators sit above triage().
  • Confirm that the direct validation block remains at the bottom of the file.

Help me fix the completed Azure Function module.

Your protected Function module now returns the trained model's response contract. Next, you'll publish it to Azure so Application Insights can trace an external request.

Deploy and Observe the Service

Den lokale funksjonen leverer nå en stabil responskontrakt. Nå skal du bevise den samme kontrakten i et ekte serverless-miljø med Azure Functions Flex Consumption.

En ekstern forespørsel må bruke en funksjonsnøkkel. Application Insights må samtidig vise at kjøringen lyktes.

In this step, get ready to:
  • Velg en Flex Consumption-region som støtter Python 3.13.
  • Opprett Azure-ressursene og publiser funksjonen.
  • Send en beskyttet forespørsel og bekreft telemetrien.
Velg region og opprett Azure-ressurser

Flex Consumption er tilgjengelig i utvalgte regioner. Python-stakkene kan også variere mellom regionene.

Dette oppretter fakturerbare ressurser

Denne hurtigstarten koster vanligvis noen få USD-cent eller mindre. Azure Storage er priset separat fra den kostnadsfrie Functions-kvoten.

Alle skyressursene samles i ai-triage-rg. Senere kan du fjerne hele utrullingen ved å slette denne ressursgruppen.

  • List de støttede Flex Consumption-regionene fra terminalen i prosjektroten ved å kjøre:
az functionapp list-flexconsumption-locations --query "sort_by(@, &name)[].{Region:name}" -o table

Hva viser kommandoen?

Azure CLI spør den aktive subscriptionen om regioner som støtter Flex Consumption. Tabellen gir deg gyldige kandidater for ressursene.

Du ser en tabell med tilgjengelige regionnavn. Velg én kandidat fra listen.

  • Kontroller Python-stakkene i kandidatregionen ved å erstatte <REGION> i kommandoen under:
az functionapp list-flexconsumption-runtimes --location <REGION> --runtime python --query '[].{version:version}' -o tsv

Hvorfor sjekke regionen?

Kommandoen viser Python-versjonene som kan velges i den aktuelle regionen. Funksjonen trenger Python 3.13 fordi den lokale pakken azure-functions==2.3.0 krever denne versjonen eller nyere.

Utdata må inneholde 3.13 før du oppretter Function App-ressursen.

  • Velg en annen region fra tabellen dersom kandidaten mangler Python 3.13.
  • Registrer den støttede regionen her: <REGION>.
  • Opprett ressursgruppen ved å erstatte <RESOURCE_GROUP> med ai-triage-rg i mønsteret under:
az group create --name <RESOURCE_GROUP> --location <REGION>

Hva oppretter kommandoen?

Ressursgruppen blir én administrativ grense for Storage-kontoen og Function App-ressursen. Den vil også inneholde de tilknyttede overvåkingsressursene.

Terminalen returnerer informasjon som bekrefter navnet og den valgte regionen.

En Azure Storage-konto trenger et globalt unikt navn med 3 til 24 små bokstaver eller tall. Denne kontoen lagrer data som Functions-plattformen trenger.

  • Velg et globalt unikt lagringskontonavn.
  • Registrer lagringskontonavnet her: <STORAGE_NAME>.
  • Opprett Storage-kontoen ved å erstatte plassholderne i mønsteret under:
az storage account create --name <STORAGE_NAME> --location <REGION> --resource-group <RESOURCE_GROUP> --sku Standard_LRS --allow-blob-public-access false

Hva konfigurerer kommandoen?

Kommandoen oppretter en Storage-konto med Standard_LRS-redundans. Den blokkerer offentlig blob-tilgang.

Terminalen returnerer ressursinformasjon når kontoen er klar.

  • Velg et unikt navn for Function App-ressursen.
  • Registrer Function App-navnet her: <APP_NAME>.

Opprettelsen kan bruke noen minutter mens Azure klargjør plattformressursene. Vent til terminalen returnerer ressursinformasjonen.

  • Opprett Function App-ressursen ved å bruke 3.13 som <LANGUAGE_VERSION> i mønsteret under:
az functionapp create --resource-group <RESOURCE_GROUP> --name <APP_NAME> --storage-account <STORAGE_NAME> --flexconsumption-location <REGION> --runtime python --runtime-version <LANGUAGE_VERSION>

Hva blir opprettet?

Kommandoen oppretter en Linux-basert Function App på Flex Consumption. Den bruker Python 3.13 som kjøretid.

Azure oppretter også en tilknyttet Application Insights-ressurs. Det gir funksjonen et mål for forespørsler og logger.

Godt jobbet. Skygrunnlaget er nå klart med ressursgruppe, Storage, Function App og overvåking.

Utrullingen bruker den samme function_app.py som allerede fungerte lokalt. Ingen kodeendringer er nødvendige i denne delen.

✔️ Awesome, I've got everything!

Flott. Bekreft at function_app.py fortsatt er lagret før du publiserer.

ⓧ I'd like to double check the full code

import json
import logging
from pathlib import Path

import azure.functions as func
import joblib

MODEL_PATH = Path(__file__).parent / "artifacts" / "model.joblib"
MODEL_BUNDLE = joblib.load(MODEL_PATH)
PIPELINE = MODEL_BUNDLE["pipeline"]
METADATA = MODEL_BUNDLE["metadata"]

app = func.FunctionApp(http_auth_level=func.AuthLevel.FUNCTION)


def predict_ticket(text: str) -> dict[str, object]:
    probabilities = PIPELINE.predict_proba([text])[0]
    best_index = max(range(len(probabilities)), key=probabilities.__getitem__)
    return {
        "team": str(PIPELINE.classes_[best_index]),
        "confidence": round(float(probabilities[best_index]), 4),
        "model_version": METADATA["model_version"],
    }


@app.function_name(name="TriageTicket")
@app.route(route="triage", methods=[func.HttpMethod.POST])
def triage(req: func.HttpRequest) -> func.HttpResponse:
    try:
        payload = req.get_json()
    except ValueError:
        return func.HttpResponse(
            body=json.dumps({"error": "Request body must be valid JSON."}),
            status_code=400,
        )

    text = payload.get("text") if isinstance(payload, dict) else None
    if not isinstance(text, str) or not text.strip():
        return func.HttpResponse(
            body=json.dumps({"error": "A non-empty text field is required."}),
            status_code=400,
        )

    result = predict_ticket(text.strip())
    logging.info(
        "triage_prediction team=%s confidence=%.4f model_version=%s",
        result["team"],
        result["confidence"],
        result["model_version"],
    )
    return func.HttpResponse(body=json.dumps(result), status_code=200)


if __name__ == "__main__":
    print(
        json.dumps(
            predict_ticket("MFA codes stopped working after I changed phones"),
            indent=2,
        )
    )
Publiser og test det beskyttede endepunktet

Publiseringen sender funksjonskoden og den lokale modellartefakten til Function App-ressursen. .funcignore holder utviklingsdataene utenfor pakken.

Publiseringen kan stå stille i opptil 60 sekunder mens remote build installerer avhengighetene. Denne ventetiden betyr at Azure bygger Python-miljøet.

  • Bytt tilbake til terminalen i prosjektroten.
  • Publiser prosjektet ved å erstatte <APP_NAME> i kommandoen under:
func azure functionapp publish <APP_NAME>

Hva skjer under publiseringen?

Core Tools pakker prosjektet og sender det til den eksisterende Function App-ressursen. Azure bygger Python-avhengighetene før funksjonen blir tilgjengelig.

Filen artifacts/model.joblib følger med fordi den ikke er ekskludert i .funcignore.

Terminalen bekrefter at publiseringen er fullført og viser den oppdagede funksjonen.

Stopper publiseringen under bygging?

Kontroller at requirements.txt bare inneholder pakkene som trengs i Azure. DuckDB skal bare ligge i requirements-dev.txt.

Kontroller at artifacts/model.joblib finnes i prosjektroten før du publiserer på nytt.

Bruk den dokumenterte lokale byggemetoden dersom remote build overskrider grensen for avhengighetsinstallasjon.

Hjelp meg å feilsøke en mislykket Azure Functions-publisering.

Beskytt funksjonsnøkkelen

Den neste kommandoen viser en Invoke URL med en funksjonsnøkkel. Behandle hele URL-en som en legitimasjon.

Unngå å lime inn URL-en i valideringer eller skjermbilder. Nøkkelen gir tilgang til det beskyttede endepunktet.

  • Hent Invoke URL-en ved å erstatte <APP_NAME> i kommandoen under:
func azure functionapp list-functions <APP_NAME> --show-keys

Hva viser kommandoen?

Core Tools viser URL-en til TriageTicket med en funksjonsnøkkel. Nøkkelen kreves fordi appen bruker FUNCTION-autorisasjon.

Utdata viser en Invoke URL for ruten triage.

Før du sender forespørselen, tror du Azure-responsen vil ha de samme feltene som den lokale responsen?

  • Erstatt <your-function-app-base-url> med Function App-ens base-URL i kommandoen under.
  • Erstatt <your-function-key> med nøkkelen fra Invoke URL-en.
  • Send supportsaken ved å kjøre den oppdaterte kommandoen:
curl --request POST "<your-function-app-base-url>/api/triage?code=<your-function-key>" --header "Content-Type: application/json" --data '{"text":"MFA codes stopped working after I changed phones"}'

Hva sender forespørselen?

  • Forespørselen bruker POST mot den publiserte /api/triage-ruten.
  • Overskriften identifiserer innholdet som application/json.
  • JSON-dataene inneholder det ikke-tomme text-feltet som funksjonen validerer.

Du ser et JSON-objekt med team, confidence og model_version.

Der satt den. Den lokalt trente modellen svarer nå gjennom et nøkkelbeskyttet Azure-endepunkt.

Får du ikke en prediksjon?

Kontroller at URL-en peker til /api/triage. Kontroller også at funksjonsnøkkelen er kopiert uten ekstra mellomrom.

Kjør publiseringskommandoen på nytt dersom Function App-ressursen mangler TriageTicket.

Kontroller at JSON-feltet heter text og inneholder en streng.

Hjelp meg å feilsøke den eksterne triage-forespørselen.

Bekreft kjøringen i Application Insights

Den eksterne responsen beviser at endepunktet fungerer. Telemetrien beviser at Azure registrerte kjøringen med resultatkode og varighet.

  • Gå til Azure-portalen i nettleseren.
  • Søk etter Function App-ressursen <APP_NAME>.
  • Åpne den tilknyttede Application Insights-ressursen.
  • Velg Logs for å åpne spørringseditoren.

Telemetrien kan bruke opptil fem minutter på å nå Logs. Denne forsinkelsen skyldes at data sendes i grupper.

Før du kjører spørringen, tror du den nyeste forespørselen vil være markert som vellykket?

  • Hent requesttelemetrien ved å kjøre denne KQL-spørringen i Logs-editoren:
requests

Hva inneholder requests?

Tabellen inneholder én requestoppføring per funksjonsinvokasjon. Hver oppføring kan vise resultatkode og varighet.

Feltet success viser om Application Insights registrerte kjøringen som vellykket.

  • Finn den nyeste raden som samsvarer med tidspunktet for curl-forespørselen.
  • Kontroller feltene resultCode, duration og success i raden.

Du ser 200 som resultatkode. Du ser også en varighet og true som success-verdi.

Ser du ingen requestoppføring?

Vent til det har gått fem minutter fra forespørselen. Kjør deretter spørringen på nytt.

Kontroller at Logs-tidsintervallet inkluderer tidspunktet for testen. Send den samme curl-forespørselen på nytt dersom tidsintervallet er riktig.

Hjelp meg å finne Azure Functions-telemetrien i Application Insights.

Planlegg kostnad og opprydding

Flex Consumption kan skalere til null når funksjonen ikke mottar forespørsler. Storage og lagret telemetri kan fortsatt koste penger.

En kort hurtigstart koster vanligvis noen få USD-cent eller mindre. Den faktiske kostnaden avhenger av subscriptionen og regionen.

Sletting av ai-triage-rg fjerner Storage-kontoen, Function App-ressursen og de tilknyttede overvåkingsressursene.

Tjenesten er nå publisert, beskyttet og observerbar. Du har bevist hele veien fra lokal modellartefakt til ekstern Azure-inferens.

Secret mission

Legg til en menneskelig kontrollsløyfe

Endepunktet ruter nå alle saker direkte til teamet modellen velger. I dette oppdraget legger du til en konfidensgrense som sender usikre prediksjoner til menneskelig vurdering. Du tester beslutningslogikken lokalt før du publiserer den til den samme Function App-en.

Clean Up Your Resources

Clean Up Your Resources

Ressursene i Azure kan fortsatt gi kostnader. Velg om du vil beholde dem, la appen skalere til null eller slette dem helt.

Cost warning

Azure Functions Flex Consumption har en månedlig gratis on-demand-kvote for kvalifiserte betalte consumption-subscriptions.

Storage-kontoen er ikke inkludert i denne kvoten. Vanlige priser for lagring og nettverk gjelder fortsatt.

Pause reduserer aktiv bruk. Delete avslutter kostnadene ved å fjerne hele ressursgruppen.

Resources you used:

  • Azure-ressursgruppen ai-triage-rg.
  • Storage-kontoen <unique-storage-name> med Standard_LRS.
  • Flex Consumption Function App <APP_NAME>.
  • Den tilknyttede Application Insights-ressursen <linked-insights-name>.
  • Det kuraterte Parquet-datasettet i artifacts/curated_tickets.parquet.
  • Modellpakken i artifacts/model.joblib.

Keep everything running

Ingen handling er nødvendig. Velg dette hvis du vil fortsette å teste triagetjenesten.

  • Behold endepunktet i <APP_NAME> for flere inferenstester.
  • Behold artifacts/curated_tickets.parquet for senere datakontroller.
  • Behold artifacts/model.joblib for lokal inferens.
  • Overvåk de faktiske kostnadene under Cost Management i Azure.

Pause - I'll come back to this later

En pause betyr at du lar endepunktet stå urørt. Function App har ingen always-ready-instanser og kan skalere til null.

  • Unngå å sende flere HTTP-forespørsler til <APP_NAME>.
  • Behold ai-triage-rg slik at endepunktet kan brukes igjen senere.
  • Behold de lokale artefaktene slik at modellen kan testes uten ny trening.
  • Kontroller Cost Management for å se om lagring eller overvåkingsdata fortsetter å påløpe kostnader.

Delete - I don't want to use this again

Dette alternativet fjerner alle Azure-ressursene og de genererte lokale artefaktene. Det er et stort grep, men kildekoden kan beholdes slik at tjenesten kan bygges opp igjen.

  • Slett ressursgruppen ai-triage-rg ved å kjøre denne kommandoen:
az group delete --name ai-triage-rg

Hva gjør denne kommandoen?

Kommandoen sletter ai-triage-rg.

Storage-kontoen, Function App og den tilknyttede overvåkingsressursen fjernes sammen med ressursgruppen.

  • Bekreft slettingen når Azure CLI ber om det.

Når kommandoen er ferdig, er endepunktet og de tilknyttede skyressursene ikke lenger tilgjengelige.

Ble ressursgruppen ikke slettet?

Kontroller at Azure CLI fortsatt bruker subscriptionen som inneholder ai-triage-rg.

Få hjelp med slettingen.

  • Bytt tilbake til terminalen du brukte til å publisere fra prosjektmappen.
  • Slett de to genererte artefaktene ved å kjøre denne kommandoen:
rm -f artifacts/curated_tickets.parquet artifacts/model.joblib

Hva fjernes lokalt?

Kommandoen fjerner det kuraterte datasettet og den serialiserte modellpakken. Kildekodefilene blir liggende i prosjektmappen.

  • Bekreft i prosjektets filoversikt at artifacts/curated_tickets.parquet ikke lenger vises.
  • Bekreft i prosjektets filoversikt at artifacts/model.joblib ikke lenger vises.

Nice Work!

Nice Work!

Sterkt levert! Du har bygget en produksjonsnær triagetjeneste på Azure Functions som klassifiserer supportsaker med en reproduserbar maskinlæringspipeline.

Du har lært å:

  • Avsløre datalekkasje i en bevisst naiv holdout. Erstatte radbasert splitting med StratifiedGroupKFold for å holde incident-grupper adskilt.
  • Bygge en DuckDB-kvalitetsport som skriver gyldige saker til et kuratert Parquet-datasett. Trene en samlet TF-IDF- og logistisk regresjonspipeline som lagres med metadata i artifacts/model.joblib.
  • Publisere et nøkkelbeskyttet HTTP-endepunkt på Azure Functions Flex Consumption. Bekrefte ekstern inferens gjennom telemetri i Application Insights.
  • Secret Mission: Lagt til en menneskelig kontrollsløyfe som bruker REVIEW_THRESHOLD med verdien 0.60. Usikre prediksjoner sendes nå til human_review uten å endre modellartefakten eller triage-ruten.

Nå kan du teste kunnskapen din.