Deploy an ML Topic Router on EKS

Train, deploy, and monitor a text classifier on Amazon EKS.

Introduction

30 Second Summary

A promising result on one computer says little about whether other people can use it reliably. Once a prediction service goes live, every release needs evidence that the model still works.

In this project, you will train a four-topic document router that proves its quality against a simple baseline. The finished model becomes a public API with an observable release path.

What You'll Build

A spacecraft sentence sent to your public API comes back labeled sci.space by the model.

By the end of this project, you'll have:

  • A model comparison that exposes the majority-class baseline on held-out posts. The production classifier combines TF-IDF features with LinearSVC. Its macro F1 plus confusion matrix proves the improvement.
  • A public prediction API on Amazon EKS that labels new text through /predict. Its /health endpoint confirms the service is ready.
  • An automated release path where a push to main moves a tested commit-tagged container image from Amazon ECR into Amazon EKS. GitHub Actions handles the release. Amazon CloudWatch captures structured prediction logs plus container metrics.
  • Secret Mission: A model quality release gate that rejects an over-strict 0.99 macro F1 threshold. You restore 0.75 to let the measured model deploy.

Will this project create AWS charges?

The Amazon EKS control plane costs $0.10 per cluster-hour under standard support.

Other AWS resources add usage-based charges until you delete the project resources.

Before We Start

This is your moment to commit to building a publicly callable four-topic ML classification API. You will prove its held-out model quality while making its deployment health, structured prediction logs, and Kubernetes metrics observable.

Set Up the Repositories and Cloud Tooling

Your classifier eventually moves from local files to a public Kubernetes service. Reproducibility starts with a stable source that every later build can use.

PyCharm becomes your local workspace. GitHub becomes the shared home for your project files.

Docker provides the pinned model runtime. AWS CloudShell provides an authenticated terminal for the cloud work.

In this step, get ready to:
  • Create the local project with its pinned dependency file.
  • Publish the setup files in a public GitHub repository.
  • Confirm the complete local and cloud toolchain.
Create the local project files

The first two files define the Python packages used by future container builds. They also keep generated artifacts out of version control.

  • Press Cmd+Space on macOS or the Windows key on Windows.
  • Search for PyCharm.
  • Press Enter to start PyCharm.

You will see the PyCharm welcome screen.

  • Select New Project.
  • Choose Pure Python as the project type.

The project dialog now shows options for the project location.

  • Set the Location to a folder named eks-ml-topic-router on your Desktop.
  • Click Create.

You will see the eks-ml-topic-router folder in PyCharm's project panel.

  • Right-click the eks-ml-topic-router folder in the project panel.
  • Select New.

You will see the available file types.

  • Select File.
  • Enter requirements.txt as the filename.

An empty requirements.txt editor tab is ready for the dependency pins.

  • Add the pinned project dependencies by copying this file content:
scikit-learn==1.9.1
joblib==1.6.0
fastapi==0.141.1
pydantic==2.13.5
uvicorn==0.53.0

What Do These Pins Control?

  • scikit-learn provides the text features plus the classifier used later.
  • joblib saves the fitted model inside the image.
  • FastAPI provides the prediction service.
  • Pydantic validates the incoming request body.
  • Uvicorn runs the API inside the container.
  • Save requirements.txt.
  • Confirm that PyCharm lists requirements.txt under eks-ml-topic-router.

Dependency File Missing?

  • Check that the file sits directly inside eks-ml-topic-router.
  • Check that the filename ends with .txt only once.
  • Still stuck? Help me check my requirements file in PyCharm.

✔️ Awesome, I've got everything!

Your pinned dependency file is saved in the local project.

ⓧ I'd like to double check the full code

scikit-learn==1.9.1
joblib==1.6.0
fastapi==0.141.1
pydantic==2.13.5
uvicorn==0.53.0

What Should Match?

Compare every package name plus every version pin. The order should match the reference.

  • Right-click the eks-ml-topic-router folder in the project panel.
  • Select New.

The file menu opens again.

  • Select File.
  • Enter .dockerignore as the filename.

The leading dot makes this a Docker configuration file.

  • Define the files excluded from future image builds by copying this content:
.git
.github
.idea
.venv
__pycache__
*.pyc
model.joblib
metrics.json

What Does This File Exclude?

The file prevents local editor data plus version-control metadata from entering the build context.

It also excludes generated Python caches plus the trained model artifacts. Future image builds create those artifacts from source.

  • Save .dockerignore.
  • Confirm that PyCharm lists .dockerignore under eks-ml-topic-router.

Docker Ignore File Missing?

  • Check that the filename begins with a dot.
  • Remove any extra .txt extension added by your operating system.
  • Still stuck? Help me fix my Docker ignore file.

✔️ Awesome, I've got everything!

Your Docker ignore rules are saved beside requirements.txt.

ⓧ I'd like to double check the full code

.git
.github
.idea
.venv
__pycache__
*.pyc
model.joblib
metrics.json

What Should Match?

Compare all eight ignore patterns. Check the leading dots plus the wildcard on *.pyc.

The container image fixes the model environment at Python 3.11. Your host Python version does not control that environment.

  • Press Cmd+Space on macOS or the Windows key on Windows.
  • Search for Docker Desktop.
  • Press Enter to start Docker Desktop.

The first start can take a minute while the container engine becomes ready.

  • Return to PyCharm.
  • Open the terminal panel below the editor.
  • Verify the local Docker command by running:
docker --version

What Does This Check?

This command asks the local Docker client to report its installed version. A version result confirms that your terminal can reach the command.

✔️ I see a Docker version

Your local container tool is ready for the baseline build in the next step.

ⓧ Command not found

The Docker command is unavailable from your terminal. Install Docker Desktop for your operating system before continuing.

  • Follow the official macOS installation guide if you use a Mac.
  • Follow the official Windows installation guide if you use Windows.
  • Start Docker Desktop through your operating-system search.
  • Return to the PyCharm terminal.
  • Verify the installation by running:
docker --version

What Confirms the Installation?

A version result confirms that Docker Desktop installed its command-line client successfully.

Docker Still Unavailable?

Restart PyCharm after the installation so its terminal receives the updated command path.

Still stuck? Help me make Docker available in my PyCharm terminal.

Publish the starter repository

The remote repository becomes the source used by later builds. Its first commit contains only the pinned dependencies plus the Docker ignore rules.

The deployment workflow stays out of this commit. A later step adds it after the AWS deployment role exists.

✔️ I'm signed in

The new repository form is ready for your project details.

ⓧ I need a GitHub account

  • Open the GitHub sign-up page.
  • Complete the account creation prompts.
  • Return to the GitHub new repository page.
  • Select your account in the Owner field.
  • Enter eks-ml-topic-router in the Repository name field.
  • Choose Public as the repository visibility.
  • Leave the optional initialization choices empty.
  • Click Create repository.

Why Start With an Empty Repository?

An empty remote accepts the existing local files without creating competing starter content. The first upload establishes main as the repository's default branch.

  • Use the existing-file upload option on the repository's quick setup page.
  • Choose requirements.txt from the local eks-ml-topic-router folder.
  • Choose .dockerignore from the same folder.

The upload page now lists both setup files.

  • Enter Add project setup files in the Commit message field.
  • Commit the upload to the main branch.

You will see requirements.txt plus .dockerignore in the public repository. The branch selector shows main.

Repository Files Missing?

  • Confirm that the repository name is eks-ml-topic-router.
  • Confirm that its visibility is public.
  • Still stuck? Help me verify my starter GitHub repository.
Verify the cloud command-line tools

The cloud work needs an authenticated terminal with AWS CLI plus the Kubernetes clients. CloudShell supplies that environment inside your signed-in AWS session.

Why Use CloudShell?

CloudShell keeps AWS authentication inside the browser session. This avoids storing long-lived AWS credentials on your computer.

It also includes the AWS CLI plus Git and kubectl.

  • Return to the signed-in AWS Management Console.
  • Set the console region to us-east-1.
  • Start AWS CloudShell from the console toolbar.

You will see an authenticated shell prompt in us-east-1.

Before you check the toolchain, do you expect all four commands to print a version?

  • Check the cloud toolchain by running:
aws --version && git --version && kubectl version --client && eksctl version

What Does This Check?

  • The aws result confirms that the AWS command-line client is available.
  • The git result confirms that version control is available.
  • The kubectl result confirms that the Kubernetes client is available.
  • The eksctl result confirms whether the EKS cluster tool meets the required version.

You will see version text for every available command. Choose the tab that matches the eksctl result.

✔️ I see version 0.227.0 or higher

Your EKS command-line tool meets the project requirement.

ⓧ I see an older version

The installed release needs an update before it creates the cluster.

  • Keep ARCH=amd64 on an x86 system.
  • Replace the first line with ARCH=arm64 on an ARM system.
  • Install the current release by running this official block:
ARCH=amd64
PLATFORM=$(uname -s)_$ARCH
curl -sLO "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_$PLATFORM.tar.gz"
curl -sL "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_checksums.txt" | grep $PLATFORM | sha256sum --check
tar -xzf eksctl_$PLATFORM.tar.gz -C /tmp && rm eksctl_$PLATFORM.tar.gz
sudo install -m 0755 /tmp/eksctl /usr/local/bin && rm /tmp/eksctl

What Does This Installation Do?

The block downloads the release archive that matches the CloudShell operating system plus processor architecture. It verifies the downloaded archive before extracting it.

The final command installs the executable in the shared command path.

A successful installation returns you to the CloudShell prompt.

Installation Failed?

Check that the first line matches the processor architecture of your CloudShell environment.

Still stuck? Help me troubleshoot the eksctl installation.

ⓧ Command not found

CloudShell needs the EKS command-line tool installed before this project can create a cluster.

  • Keep ARCH=amd64 on an x86 system.
  • Replace the first line with ARCH=arm64 on an ARM system.
  • Install the current release by running this official block:
ARCH=amd64
PLATFORM=$(uname -s)_$ARCH
curl -sLO "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_$PLATFORM.tar.gz"
curl -sL "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_checksums.txt" | grep $PLATFORM | sha256sum --check
tar -xzf eksctl_$PLATFORM.tar.gz -C /tmp && rm eksctl_$PLATFORM.tar.gz
sudo install -m 0755 /tmp/eksctl /usr/local/bin && rm /tmp/eksctl

What Does This Installation Do?

The block downloads the release archive that matches the CloudShell operating system plus processor architecture. It verifies the downloaded archive before extracting it.

The final command installs the executable in the shared command path.

A successful installation returns you to the CloudShell prompt.

Installation Failed?

Check that the first line matches the processor architecture of your CloudShell environment.

Still stuck? Help me troubleshoot the eksctl installation.

Before you run the final check, do you expect every tool to return a version now?

  • Verify the complete CloudShell toolchain by running:
aws --version && git --version && kubectl version --client && eksctl version

What Proves the Toolchain Is Ready?

The command stops if any tool is unavailable. Reaching the final result proves that all four commands ran successfully.

You will see four version results. The eksctl version is 0.227.0 or higher.

One Version Still Missing?

  • Restart the CloudShell session if an installed command is still unavailable.
  • Confirm that the console region remains us-east-1.
  • Still stuck? Help me diagnose my CloudShell tool versions.

Your foundation is ready. The public repository contains the setup files while the local Docker runtime can run future builds.

CloudShell now has every command required for the AWS work. Next, you will run the majority-class baseline and see why it falls short.

Prove the Naive Baseline Falls Short

Your PyCharm project is ready, so your next goal is to establish the score that a useful classifier must beat. Docker supplies the pinned Python 3.11 environment for a reproducible baseline.

A classifier has no value until it beats a simple model that ignores every word. This step deliberately exposes that limitation with a majority-class baseline measured using macro F1.

In this step, get ready to:
  • Define the container stages for the baseline build.
  • Create a classifier that always predicts the most frequent category.
  • Observe the baseline's poor macro F1 result.
Create the baseline image stages

A multi-stage Dockerfile gives each build phase a named target. The base stage installs the pinned dependencies used by the baseline stage.

  • Switch back to the existing eks-ml-topic-router project in PyCharm.
  • Create Dockerfile inside the eks-ml-topic-router folder using PyCharm's new-file control.
  • Add the base stage by pasting this code into Dockerfile:
FROM python:3.11-slim AS base

WORKDIR /app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

What Does the Base Stage Do?

  • The base stage starts from the pinned Python 3.11 image.
  • The WORKDIR instruction makes /app the location for the files used during the build.
  • The final two instructions copy requirements.txt into the image. They install the project's pinned packages.
  • Save Dockerfile.
  • Confirm Dockerfile appears beside requirements.txt in the project sidebar.

You should see both files listed inside eks-ml-topic-router.

Dockerfile Missing From the Project?

  • Check that the filename is exactly Dockerfile with no file extension.
  • Check that the file sits inside eks-ml-topic-router beside requirements.txt.
  • Still stuck? Help me locate or create my Dockerfile in PyCharm.

The next stage inherits the installed environment from base. Its build runs baseline.py so the score becomes part of the Docker build output.

  • Append the baseline stage below the base stage in Dockerfile by pasting this code:
FROM base AS baseline
COPY baseline.py ./
RUN python baseline.py

What Does the Baseline Stage Do?

  • The baseline stage reuses the packages installed in base.
  • The COPY instruction places baseline.py inside the image.
  • The RUN instruction executes the script during the image build.
  • Save Dockerfile.
  • Confirm the final line in Dockerfile is RUN python baseline.py.

Your Dockerfile now has a reusable dependency stage plus a named baseline stage.

Baseline Stage in the Wrong Place?

  • Place FROM base AS baseline after the dependency installation line.
  • Remove any indentation before the three baseline-stage instructions.
  • Still stuck? Help me fix the stages in my Dockerfile.

✔️ Awesome, I've got everything!

Great. Your saved Dockerfile now defines both required stages.

ⓧ I'd like to double check the full code

  • Compare your complete Dockerfile with this reference:
FROM python:3.11-slim AS base

WORKDIR /app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

FROM base AS baseline
COPY baseline.py ./
RUN python baseline.py

What Should Match?

The stage names must be base and baseline. The final instruction must run baseline.py during the build.

Build the majority-class baseline

The 20 Newsgroups dataset provides predefined train and test subsets. Removing headers, footers, and quoted text reduces clues that come from message metadata.

  • Create baseline.py inside the eks-ml-topic-router folder using PyCharm's new-file control.
  • Define the imports plus the four categories by pasting this code into baseline.py:
from sklearn.datasets import fetch_20newsgroups
from sklearn.dummy import DummyClassifier
from sklearn.metrics import f1_score

CATEGORIES = [
    "comp.graphics",
    "comp.os.ms-windows.misc",
    "sci.med",
    "sci.space",
]
REMOVE_METADATA = ("headers", "footers", "quotes")

What Does This Setup Define?

  • The imports provide the dataset loader, the baseline estimator, and the F1 calculation.
  • The CATEGORIES list limits the router to four document topics.
  • The REMOVE_METADATA tuple identifies the metadata-like content removed by the loader.
  • Save baseline.py.
  • Confirm baseline.py appears beside Dockerfile in the project sidebar.

You should now see the new Python file inside eks-ml-topic-router.

Baseline File Not Listed?

  • Check that the filename is exactly baseline.py.
  • Check that the file sits beside Dockerfile.
  • Still stuck? Help me create baseline.py in the correct folder.

Separate dataset subsets keep evaluation examples outside the fitting process. Both loads use the same categories plus the same metadata-removal settings.

  • Append the training and test dataset loaders below REMOVE_METADATA by pasting this code:
train_data = fetch_20newsgroups(
    subset="train",
    categories=CATEGORIES,
    remove=REMOVE_METADATA,
)
test_data = fetch_20newsgroups(
    subset="test",
    categories=CATEGORIES,
    remove=REMOVE_METADATA,
)

Why Load Two Subsets?

  • The train_data object supplies examples used to fit the baseline.
  • The test_data object supplies separate examples used to measure its predictions.
  • The shared category list keeps the training labels aligned with the test labels.
  • Save baseline.py.
  • Confirm the file now contains one loader for train plus one loader for test.

You should see two separate fetch_20newsgroups calls below the constants.

Dataset Blocks Look Misaligned?

  • Check that each loader ends with its own closing parenthesis.
  • Check that both loaders use CATEGORIES plus REMOVE_METADATA.
  • Still stuck? Help me fix the dataset-loading blocks.

The DummyClassifier uses strategy="most_frequent" to predict one category for every document. The f1_score calculation gives each category equal weight through average="macro".

  • Append the fitting, prediction, scoring, and output logic below the dataset loaders by pasting this code:
baseline = DummyClassifier(strategy="most_frequent")
baseline.fit(train_data.data, train_data.target)
predictions = baseline.predict(test_data.data)
score = f1_score(test_data.target, predictions, average="macro")

print(f"baseline_macro_f1={score:.4f}")
print("The baseline predicts only the most frequent class and ignores every input word.")

How Is the Baseline Measured?

  • The baseline learns which training category appears most often.
  • The predictions array contains that same category for every test document.
  • The score value averages the per-category F1 results equally.
  • The final lines print the measured score plus the reason for the shortfall.
  • Save baseline.py.
  • Confirm the final line mentions that the baseline ignores every input word.

Your completed script now loads the data, fits the naive classifier, and prints its evaluation result.

Seeing Python Syntax Highlighting Errors?

  • Check that every opening parenthesis has a matching closing parenthesis.
  • Check that the category strings keep their surrounding quotation marks.
  • Still stuck? Help me compare my baseline script with the required structure.

✔️ Awesome, I've got everything!

Great. Your saved baseline.py now contains the complete naive evaluation path.

ⓧ I'd like to double check the full code

  • Compare your complete baseline.py with this reference:
from sklearn.datasets import fetch_20newsgroups
from sklearn.dummy import DummyClassifier
from sklearn.metrics import f1_score

CATEGORIES = [
    "comp.graphics",
    "comp.os.ms-windows.misc",
    "sci.med",
    "sci.space",
]
REMOVE_METADATA = ("headers", "footers", "quotes")

train_data = fetch_20newsgroups(
    subset="train",
    categories=CATEGORIES,
    remove=REMOVE_METADATA,
)
test_data = fetch_20newsgroups(
    subset="test",
    categories=CATEGORIES,
    remove=REMOVE_METADATA,
)

baseline = DummyClassifier(strategy="most_frequent")
baseline.fit(train_data.data, train_data.target)
predictions = baseline.predict(test_data.data)
score = f1_score(test_data.target, predictions, average="macro")

print(f"baseline_macro_f1={score:.4f}")
print("The baseline predicts only the most frequent class and ignores every input word.")

What Should Match?

The full file should contain separate train_data plus test_data objects. It should end with the score plus the baseline explanation.

Run the designed obstacle

The baseline image provides the first measurable checkpoint for the project. This first build downloads the runtime, packages, and dataset, so pauses in the terminal output are expected.

Before you build, do you think a classifier that predicts one category can achieve balanced performance across all four topics?

  • Build only the baseline stage by running this command from the terminal in PyCharm:
docker build --target baseline -t eks-ml-topic-router:baseline .

What Does This Build Do?

  • The --target baseline option stops after the named baseline stage.
  • The -t eks-ml-topic-router:baseline option gives the resulting image a local name plus tag.
  • The final build instruction runs baseline.py inside the pinned container environment.

You'll see baseline_macro_f1= followed by the measured score. You'll also see that the baseline predicts only the most frequent class and ignores every input word.

You have now exposed the intended shortfall. The build is reproducible, but the one-category prediction strategy fails to route three of the four classes.

Baseline Build Did Not Finish?

  • Confirm Docker Desktop is still running before retrying the build.
  • Confirm Dockerfile plus baseline.py sit beside requirements.txt.
  • Check your internet connection if the dataset download stops.
  • Still stuck? Help me troubleshoot my baseline Docker build.
  • Record the printed score here: your baseline macro F1.

Why Does Macro F1 Expose the Problem?

Raw accuracy counts the total number of correct predictions. A frequent category can contribute many correct guesses even when the remaining categories receive none.

Macro F1 calculates an F1 result for each category before taking an unweighted mean. Missing three categories therefore pulls the final score down sharply.

Your reproducible baseline now gives the real model a clear target to beat. Next, you'll train the text-aware classifier and serve its predictions through an API.

Train, Evaluate, and Serve the Classifier

Your majority-class baseline proved that a classifier can produce predictions while ignoring every input word. A useful topic router must learn patterns from the document text.

You will now train a measured text model inside a Docker image. You will expose that model through a tested FastAPI service.

In this step, get ready to:
  • Train a TF-IDF text-classification pipeline.
  • Evaluate the model against the held-out test subset.
  • Send a document to the local prediction API.
Train and evaluate the production model

A scikit-learn Pipeline keeps text preprocessing with the classifier that consumes those features. The vectorizer and classifier are fitted only on the training subset.

Why this model fits text routing

TF-IDF turns documents into sparse numerical features based on informative words. Word bigrams also capture short phrases.

LinearSVC works with sparse multiclass input. This creates a compact classifier for the four learned topics.

  • In the eks-ml-topic-router project sidebar in PyCharm, create train.py.
  • Add the imports and shared model settings by pasting this first section:
import json

from joblib import dump
from sklearn.datasets import fetch_20newsgroups
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics import classification_report, confusion_matrix, f1_score
from sklearn.pipeline import Pipeline
from sklearn.svm import LinearSVC

CATEGORIES = [
    "comp.graphics",
    "comp.os.ms-windows.misc",
    "sci.med",
    "sci.space",
]
REMOVE_METADATA = ("headers", "footers", "quotes")
MODEL_PATH = "model.joblib"
METRICS_PATH = "metrics.json"

What does this section do?

  • The imports provide dataset loading, feature extraction, classification, evaluation, JSON writing, and model persistence.
  • The CATEGORIES list keeps the production model focused on the same four topics as the baseline.
  • The artifact paths give the trained model and evaluation evidence stable filenames.
  • Save train.py.
  • Confirm that the file shows the four category names plus the MODEL_PATH and METRICS_PATH constants.

Imports or constants look incomplete?

  • Check that every import appears above CATEGORIES.
  • Check that each category remains inside double quotes.
  • Still stuck? Help me compare the imports and constants in train.py.
  • Load the separate training and test subsets by adding this code below METRICS_PATH = "metrics.json":
train_data = fetch_20newsgroups(
    subset="train",
    categories=CATEGORIES,
    remove=REMOVE_METADATA,
)
test_data = fetch_20newsgroups(
    subset="test",
    categories=CATEGORIES,
    remove=REMOVE_METADATA,
)

How does this protect the evaluation?

  • The first call loads the predefined training subset.
  • The second call loads the predefined test subset.
  • Both calls remove headers, footers, and quoted text that could provide misleading clues.
  • Save train.py.
  • Confirm that train_data and test_data use different subset values.

Dataset calls look identical?

  • Set the first subset value to train.
  • Set the second subset value to test.
  • Need another pair of eyes? Help me verify the dataset split in train.py.
  • Define the feature extraction and classification pipeline by adding this code below the test_data assignment:
model = Pipeline(
    [
        (
            "tfidf",
            TfidfVectorizer(
                strip_accents="unicode",
                lowercase=True,
                stop_words="english",
                ngram_range=(1, 2),
                min_df=2,
                max_df=0.95,
                sublinear_tf=True,
            ),
        ),
        ("classifier", LinearSVC(random_state=42, max_iter=2000)),
    ]
)

What does this pipeline learn?

  • The vectorizer normalizes text before creating TF-IDF features.
  • The ngram_range value includes individual words plus two-word phrases.
  • The classifier learns boundaries between the four topic categories from the sparse features.
  • Save train.py.
  • Confirm that model contains a tfidf step followed by a classifier step.

Pipeline parentheses do not line up?

  • Check that the two named steps remain inside the list passed to Pipeline.
  • Check that TfidfVectorizer closes before the classifier tuple starts.
  • Still seeing a syntax problem? Help me inspect the Pipeline structure.
  • Fit the pipeline and calculate the held-out evaluation values by adding this code below the Pipeline definition:
model.fit(train_data.data, train_data.target)
predictions = model.predict(test_data.data)
labels = list(range(len(train_data.target_names)))
macro_f1 = float(f1_score(test_data.target, predictions, average="macro"))
report = classification_report(
    test_data.target,
    predictions,
    labels=labels,
    target_names=train_data.target_names,
    output_dict=True,
    zero_division=0.0,
)
matrix = confusion_matrix(test_data.target, predictions, labels=labels)

How is model quality measured?

  • The pipeline fits against train_data only.
  • The model predicts labels for the separate test documents.
  • Macro F1 gives every class equal weight before averaging their scores.
  • The report and confusion matrix expose performance beyond one summary number.
  • Save train.py.
  • Confirm that model.fit receives training data while model.predict receives test data.

Training and test data mixed up?

  • Keep train_data.data inside model.fit.
  • Keep test_data.data inside model.predict.
  • Want help checking for leakage? Help me review the fit and evaluation lines.
  • Assemble the evaluation artifact and persist both output files by adding this code below the confusion matrix calculation:
metrics = {
    "macro_f1": macro_f1,
    "classification_report": report,
    "confusion_matrix": matrix.tolist(),
    "target_names": list(train_data.target_names),
}

with open(METRICS_PATH, "w", encoding="utf-8") as metrics_file:
    json.dump(metrics, metrics_file, indent=2, default=float)

dump(
    {"model": model, "target_names": list(train_data.target_names)},
    MODEL_PATH,
)

What gets saved?

  • The metrics dictionary contains the score, class report, confusion matrix, and label names.
  • The JSON file preserves readable evaluation evidence.
  • The joblib artifact keeps the fitted pipeline with the label names needed by the API.
  • Save train.py.
  • Confirm that metrics.json and model.joblib are both assigned through their path constants.

Artifact paths do not match?

  • Check that the JSON writer uses METRICS_PATH.
  • Check that dump uses MODEL_PATH.
  • Need help tracing the saved values? Help me inspect the artifact-writing section.
  • Print the evaluation summary during the image build by adding this final section to train.py:
print(f"model_macro_f1={macro_f1:.4f}")
print(classification_report(
    test_data.target,
    predictions,
    labels=labels,
    target_names=train_data.target_names,
    zero_division=0.0,
))
print("confusion_matrix=")
print(matrix)

Why print the evaluation?

  • The macro F1 line makes the headline score easy to find in build output.
  • The classification report shows precision, recall, and F1 for each category.
  • The matrix shows which topic pairs the classifier confuses.
  • Save the completed train.py file.
  • Confirm that the final lines print model_macro_f1 plus confusion_matrix.

Evaluation output section looks broken?

  • Keep the multiline classification_report call inside the outer print call.
  • Check that the final line prints matrix.
  • Still unsure? Help me check the final train.py output section.

✔️ Awesome, I've got everything!

Your training script now covers loading, fitting, evaluation, persistence, and build output. Make sure train.py is saved.

ⓧ I'd like to double check the full code

  • Compare your complete train.py file with this reference:
import json

from joblib import dump
from sklearn.datasets import fetch_20newsgroups
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics import classification_report, confusion_matrix, f1_score
from sklearn.pipeline import Pipeline
from sklearn.svm import LinearSVC

CATEGORIES = [
    "comp.graphics",
    "comp.os.ms-windows.misc",
    "sci.med",
    "sci.space",
]
REMOVE_METADATA = ("headers", "footers", "quotes")
MODEL_PATH = "model.joblib"
METRICS_PATH = "metrics.json"

train_data = fetch_20newsgroups(
    subset="train",
    categories=CATEGORIES,
    remove=REMOVE_METADATA,
)
test_data = fetch_20newsgroups(
    subset="test",
    categories=CATEGORIES,
    remove=REMOVE_METADATA,
)

model = Pipeline(
    [
        (
            "tfidf",
            TfidfVectorizer(
                strip_accents="unicode",
                lowercase=True,
                stop_words="english",
                ngram_range=(1, 2),
                min_df=2,
                max_df=0.95,
                sublinear_tf=True,
            ),
        ),
        ("classifier", LinearSVC(random_state=42, max_iter=2000)),
    ]
)

model.fit(train_data.data, train_data.target)
predictions = model.predict(test_data.data)
labels = list(range(len(train_data.target_names)))
macro_f1 = float(f1_score(test_data.target, predictions, average="macro"))
report = classification_report(
    test_data.target,
    predictions,
    labels=labels,
    target_names=train_data.target_names,
    output_dict=True,
    zero_division=0.0,
)
matrix = confusion_matrix(test_data.target, predictions, labels=labels)

metrics = {
    "macro_f1": macro_f1,
    "classification_report": report,
    "confusion_matrix": matrix.tolist(),
    "target_names": list(train_data.target_names),
}

with open(METRICS_PATH, "w", encoding="utf-8") as metrics_file:
    json.dump(metrics, metrics_file, indent=2, default=float)

dump(
    {"model": model, "target_names": list(train_data.target_names)},
    MODEL_PATH,
)

print(f"model_macro_f1={macro_f1:.4f}")
print(classification_report(
    test_data.target,
    predictions,
    labels=labels,
    target_names=train_data.target_names,
    zero_division=0.0,
))
print("confusion_matrix=")
print(matrix)
Package and test the API image

The trained artifact needs a stable inference interface before it can become a deployed service. The API loads the fitted pipeline once when the container starts.

  • In the eks-ml-topic-router project sidebar, create app.py.
  • Add the imports, logging configuration, model loading, and API object with this section:
import json
import logging
import time

from fastapi import FastAPI
from joblib import load
from pydantic import BaseModel

logging.basicConfig(level=logging.INFO, format="%(message)s")
logger = logging.getLogger("topic-router")

artifact = load("model.joblib")
model = artifact["model"]
target_names = artifact["target_names"]

app = FastAPI(title="ML Topic Router")

How does the API start?

  • Logging uses a message-only format so each prediction event reaches standard output as JSON.
  • The artifact supplies the fitted model plus its ordered category names.
  • The app object becomes the web service entry point.
  • Save app.py.
  • Confirm that artifact loads model.joblib before the API object is created.

Model loading section looks incomplete?

  • Check that the artifact key is spelled model.
  • Check that the label key is spelled target_names.
  • Need help comparing the setup? Help me inspect the app.py startup section.
  • Define the request body and prediction function by adding this code below the app assignment:
class PredictionRequest(BaseModel):
    text: str


def classify_text(text: str) -> dict[str, str]:
    started = time.perf_counter()
    label_index = int(model.predict([text])[0])
    category = target_names[label_index]
    latency_ms = round((time.perf_counter() - started) * 1000, 2)
    logger.info(
        json.dumps(
            {
                "event": "prediction",
                "category": category,
                "latency_ms": latency_ms,
            }
        )
    )
    return {"category": category}

What happens during a prediction?

  • The request model requires a text value in the JSON body.
  • The fitted model converts the text into a numeric class index.
  • The target-name list converts that index into a learned category.
  • The structured log records the category plus prediction latency.
  • Save app.py.
  • Confirm that classify_text returns a dictionary containing category.

Prediction function has a nesting issue?

  • Keep the event fields inside the dictionary passed to json.dumps.
  • Keep the final return statement outside logger.info.
  • Still stuck? Help me inspect classify_text.
  • Expose the health and prediction routes by adding this final section to app.py:
@app.get("/health")
def health() -> dict[str, str]:
    return {"status": "ok"}


@app.post("/predict")
def predict(request: PredictionRequest) -> dict[str, str]:
    return classify_text(request.text)

What do the routes expose?

  • The /health route provides a lightweight service check.
  • The /predict route validates the request body before classification.
  • Both routes return JSON-compatible dictionaries.
  • Save the completed app.py file.
  • Confirm that the file ends with the predict route.

Route decorators not aligned?

  • Place each decorator directly above its function.
  • Keep both function bodies indented by four spaces.
  • Need help with the route definitions? Help me check the FastAPI routes.

✔️ Awesome, I've got everything!

Your API now loads the trained artifact, serves health checks, classifies text, and emits structured prediction logs.

ⓧ I'd like to double check the full code

  • Compare your complete app.py file with this reference:
import json
import logging
import time

from fastapi import FastAPI
from joblib import load
from pydantic import BaseModel

logging.basicConfig(level=logging.INFO, format="%(message)s")
logger = logging.getLogger("topic-router")

artifact = load("model.joblib")
model = artifact["model"]
target_names = artifact["target_names"]

app = FastAPI(title="ML Topic Router")


class PredictionRequest(BaseModel):
    text: str


def classify_text(text: str) -> dict[str, str]:
    started = time.perf_counter()
    label_index = int(model.predict([text])[0])
    category = target_names[label_index]
    latency_ms = round((time.perf_counter() - started) * 1000, 2)
    logger.info(
        json.dumps(
            {
                "event": "prediction",
                "category": category,
                "latency_ms": latency_ms,
            }
        )
    )
    return {"category": category}


@app.get("/health")
def health() -> dict[str, str]:
    return {"status": "ok"}


@app.post("/predict")
def predict(request: PredictionRequest) -> dict[str, str]:
    return classify_text(request.text)
  • In the eks-ml-topic-router project sidebar, create the tests directory.
  • Inside tests, create test_app.py.
  • Add the API prediction test by pasting this code:
import unittest

from app import classify_text, target_names


class TopicRouterTest(unittest.TestCase):
    def test_prediction_uses_a_known_label(self) -> None:
        result = classify_text("A spacecraft entered orbit around the moon")
        self.assertIn(result["category"], target_names)


if __name__ == "__main__":
    unittest.main()

What does this test prove?

The test sends a spacecraft sentence through the real fitted model. It passes only when the returned category belongs to the artifact's known labels.

  • Save tests/test_app.py.
  • Confirm that test_prediction_uses_a_known_label checks the returned category against target_names.

Test file not detected in the project?

  • Check that test_app.py is inside the tests directory.
  • Check that the test method name starts with test_.
  • Need help checking the test layout? Help me inspect tests/test_app.py.

✔️ Awesome, I've got everything!

Your test file is saved inside tests and exercises the real prediction function.

ⓧ I'd like to double check the full code

  • Compare tests/test_app.py with this reference:
import unittest

from app import classify_text, target_names


class TopicRouterTest(unittest.TestCase):
    def test_prediction_uses_a_known_label(self) -> None:
        result = classify_text("A spacecraft entered orbit around the moon")
        self.assertIn(result["category"], target_names)


if __name__ == "__main__":
    unittest.main()

The existing Dockerfile already provides the pinned Python base and baseline stage. The runtime stage trains the production artifact before copying the API and running its test.

  • Return to the existing Dockerfile in PyCharm.
  • Add this runtime stage below RUN python baseline.py:
FROM base AS runtime
COPY train.py ./
RUN python train.py
COPY app.py ./
COPY tests ./tests
RUN python -m unittest discover -s tests
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

What does the runtime stage do?

  • The training command creates model.joblib and metrics.json inside the image.
  • The test command checks the fitted artifact through the API function.
  • The final command starts the service on port 8000.
  • Save the updated Dockerfile.
  • Confirm that the runtime stage ends with the Uvicorn command.

Runtime stage in the wrong place?

  • Keep the existing baseline stage unchanged.
  • Place FROM base AS runtime after the baseline stage.
  • Need help checking the stage order? Help me compare the Dockerfile stages.

✔️ Awesome, I've got everything!

Your multi-stage Dockerfile now supports both the original baseline build and the final API image.

ⓧ I'd like to double check the full code

  • Compare your complete Dockerfile with this reference:
FROM python:3.11-slim AS base

WORKDIR /app
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt

FROM base AS baseline
COPY baseline.py ./
RUN python baseline.py

FROM base AS runtime
COPY train.py ./
RUN python train.py
COPY app.py ./
COPY tests ./tests
RUN python -m unittest discover -s tests
EXPOSE 8000
CMD ["uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

This build downloads the dataset inside the image before fitting the pipeline. Give it time to finish while the model trains and the unit test runs.

  • Switch back to the local terminal you used for the baseline build.
  • Build the final image by running this command:
docker build -t eks-ml-topic-router:local .

What happens during this build?

  • Docker installs the pinned Python dependencies in the reusable base stage.
  • The runtime stage trains the model against the training subset.
  • The same stage prints held-out evaluation evidence before saving both artifacts.
  • The unit test checks that inference returns a known category.

You should see model_macro_f1, the per-class report, and confusion_matrix in the build output. The unit test should complete successfully before Docker tags eks-ml-topic-router:local.

That is the production artifact built and tested inside its pinned runtime.

Image build did not complete?

  • Confirm that Docker Desktop is still running.
  • Check that train.py, app.py, and the tests directory appear beside the Dockerfile.
  • If the build stops in a Python section, compare the named file with its full-code reference above.
  • Still blocked? Help me troubleshoot the final Docker build.
Run the local prediction path

The image now contains the trained artifact and the service that uses it. Running the container exposes the same prediction path that the cloud deployment will use.

  • Start the local API container by running this command:
docker run --rm -p 8000:8000 eks-ml-topic-router:local

What does this command do?

Docker starts the runtime image and maps the container's port 8000 to the same port on your computer. The terminal remains attached so you can see the API logs.

You should see the server start listening on port 8000. Keep this terminal running for the request.

Container stops during startup?

  • Confirm that the completed image tag is eks-ml-topic-router:local.
  • Check the startup output for the file named immediately before the failure.
  • Need help reading the container logs? Help me troubleshoot the local API startup.
  • Create a second local terminal in PyCharm while the container keeps running.

Before you send the request, which learned topic do you expect the spacecraft sentence to match?

  • Send the spacecraft document to the prediction endpoint by running:
curl -X POST http://localhost:8000/predict \
  -H "Content-Type: application/json" \
  -d '{"text":"The spacecraft entered orbit around the moon"}'

What does this request test?

  • The request sends JSON to the local /predict endpoint.
  • The API validates the text field before calling the fitted model.
  • The response returns the learned category selected by the classifier.

You should see a JSON response containing category in the second terminal. The first terminal should print a JSON prediction event containing event, category, and latency_ms.

Your topic router now turns raw text into a learned category while producing an operational event for every prediction.

Request cannot reach the API?

  • Confirm that the first terminal still shows the running server.
  • Confirm that the container was started with port 8000 mapped to your computer.
  • Compare the request body with the required text field.
  • Still not receiving a category? Help me troubleshoot the local prediction request.

Your measured classifier is now trained, tested, and serving predictions from a container. Next, you will create the AWS runtime and a short-lived deployment identity.

Provision EKS and Secure the Deployment Identity

Your classifier now trains inside Docker and answers prediction requests locally. A local container cannot prove that the service can be released safely into a cloud environment.

In this step, Amazon EKS Auto Mode provides the runtime for your API. Amazon ECR stores the versioned container images. GitHub Actions authenticates through OpenID Connect so the workflow can use short-lived AWS credentials.

In this step, get ready to:
  • Create the Kubernetes deployment files and AWS identity policies.
  • Provision the EKS cluster and ECR repository.
  • Authorize the GitHub deployment role for the ml-api namespace.
Create the cloud configuration files

The cloud resources need declarative files that describe the workload and its permissions. These files keep the deployment reviewable inside the same repository as the model.

  • Switch back to the eks-ml-topic-router project in PyCharm.
  • Create these paths from the PyCharm Project sidebar: k8s/namespace.yaml, k8s/app.yaml, iam/create-open-id-connect-provider.json, iam/trust-policy.json, iam/permissions-policy.json, and .github/workflows/deploy.yml.
  • Copy the exact content for each file from the full-code tab below.

✔️ I created the cloud files

Confirm that the k8s, iam, and .github/workflows directories appear in the PyCharm Project sidebar.

ⓧ I'd like to double check the full code

Compare each project file with the complete reference below.

.git
.github
.idea
.venv
__pycache__
*.pyc
model.joblib
metrics.json

The Docker ignore rules keep repository metadata and generated model artifacts out of the build context.

apiVersion: v1
kind: Namespace
metadata:
  name: ml-api

This manifest gives the API its own Kubernetes namespace. The later access policy is limited to this boundary.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: topic-router
  namespace: ml-api
spec:
  replicas: 1
  selector:
    matchLabels:
      app: topic-router
  template:
    metadata:
      labels:
        app: topic-router
    spec:
      containers:
        - name: topic-router
          image: IMAGE_URI_PLACEHOLDER
          ports:
            - containerPort: 8000
          resources:
            requests:
              cpu: 250m
              memory: 512Mi
            limits:
              cpu: "1"
              memory: 1Gi
          readinessProbe:
            httpGet:
              path: /health
              port: 8000
            initialDelaySeconds: 5
            periodSeconds: 10
---
apiVersion: v1
kind: Service
metadata:
  name: topic-router
  namespace: ml-api
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
spec:
  selector:
    app: topic-router
  ports:
    - port: 80
      targetPort: 8000
      protocol: TCP
  type: LoadBalancer
  loadBalancerClass: eks.amazonaws.com/nlb

The Deployment runs one API replica with resource boundaries and a readiness probe. The Service requests an internet-facing Network Load Balancer from EKS Auto Mode.

{
  "Url": "https://token.actions.githubusercontent.com",
  "ClientIDList": [
    "sts.amazonaws.com"
  ]
}

This input registers GitHub's token issuer with AWS. The client ID limits the intended audience to AWS Security Token Service.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::your-aws-account-id:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com",
          "token.actions.githubusercontent.com:sub": "repo:your-github-owner@your-owner-id/eks-ml-topic-router@your-repository-id:ref:refs/heads/main"
        }
      }
    }
  ]
}

The trust policy accepts tokens only from the identified repository. It also restricts access to pushes from the main branch.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "GetAuthorizationToken",
      "Effect": "Allow",
      "Action": "ecr:GetAuthorizationToken",
      "Resource": "*"
    },
    {
      "Sid": "PushPullTopicRouterImage",
      "Effect": "Allow",
      "Action": [
        "ecr:BatchGetImage",
        "ecr:BatchCheckLayerAvailability",
        "ecr:CompleteLayerUpload",
        "ecr:GetDownloadUrlForLayer",
        "ecr:InitiateLayerUpload",
        "ecr:PutImage",
        "ecr:UploadLayerPart"
      ],
      "Resource": "arn:aws:ecr:us-east-1:your-aws-account-id:repository/eks-ml-topic-router"
    },
    {
      "Sid": "DescribeDeploymentCluster",
      "Effect": "Allow",
      "Action": "eks:DescribeCluster",
      "Resource": "arn:aws:eks:us-east-1:your-aws-account-id:cluster/eks-ml-topic-router"
    }
  ]
}

The inline policy permits authentication to ECR. It scopes image operations to the topic router repository and cluster discovery to the topic router cluster.

name: Deploy ML API to Amazon EKS

on:
  push:
    branches:
      - main

permissions:
  id-token: write
  contents: read

env:
  AWS_REGION: us-east-1
  ECR_REPOSITORY: eks-ml-topic-router
  EKS_CLUSTER: eks-ml-topic-router
  K8S_NAMESPACE: ml-api
  AWS_ROLE_ARN: arn:aws:iam::your-aws-account-id:role/GitHubActionsEKSRole

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Check out the repository
        uses: actions/checkout@v7.0.1

      - name: Configure short-lived AWS credentials
        uses: aws-actions/configure-aws-credentials@v6.3.0
        with:
          role-to-assume: ${{ env.AWS_ROLE_ARN }}
          role-session-name: topic-router-deploy
          aws-region: ${{ env.AWS_REGION }}

      - name: Log in to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2.1.7

      - name: Build, test, and push the image
        env:
          REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          IMAGE_TAG: ${{ github.sha }}
        run: |
          docker build -t $REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
          docker push $REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG

      - name: Install kubectl
        uses: Azure/setup-kubectl@v5.1.0
        with:
          version: v1.36.0

      - name: Deploy the commit image
        env:
          REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          IMAGE_TAG: ${{ github.sha }}
        run: |
          aws eks update-kubeconfig --region $AWS_REGION --name $EKS_CLUSTER
          sed "s|IMAGE_URI_PLACEHOLDER|$REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG|g" k8s/app.yaml > /tmp/app.yaml
          kubectl apply -f /tmp/app.yaml
          kubectl rollout status deployment/topic-router --namespace $K8S_NAMESPACE

The workflow requests a short-lived identity before building the image. It tags the image with the commit SHA and waits for the Kubernetes rollout to complete.

scikit-learn==1.9.1
joblib==1.6.0
fastapi==0.141.1
pydantic==2.13.5
uvicorn==0.53.0

The pinned Python dependencies remain unchanged from the tested local image.

  • Save every new file in PyCharm.
  • Confirm all six new files appear under their expected directories in the Project sidebar.

Why Separate These Policies?

The AWS IAM policy controls which AWS APIs the workflow can call. The EKS access entry separately controls what the role can change inside the cluster.

This separation lets the workflow publish images without granting unrestricted Kubernetes administration.

The trust policy must identify your repository through immutable numeric IDs. This prevents a renamed or recreated repository from inheriting the deployment role.

  • Use GitHub's public repository API to request GET /repos/{owner}/{repo} for your eks-ml-topic-router repository.
  • Record your GitHub owner as your GitHub owner.
  • Record the nested owner.id value as your GitHub owner ID.
  • Record the top-level id value as your GitHub repository ID.
  • Switch back to AWS CloudShell.
  • Read your current AWS account ID by running this command:
aws sts get-caller-identity --query Account --output text
  • Record the printed value as your AWS account ID.
  • Return to the project in PyCharm.
  • Replace your-github-owner in iam/trust-policy.json with your GitHub owner.
  • Replace your-owner-id with your GitHub owner ID.
  • Replace your-repository-id with your GitHub repository ID.
  • Replace every your-aws-account-id value with your AWS account ID.
  • Keep the trust subject in the immutable form repo:OWNER@OWNER-ID/REPO@REPO-ID:ref:refs/heads/BRANCH.
  • Press Cmd+Shift+F on macOS or Ctrl+Shift+F on Windows.
  • Search the project for your-.
  • Confirm the search returns no remaining placeholders.
  • Save the personalized files.
  • Commit the k8s directory with the three iam files using the Git flow from Step 1.
  • Push that commit to main.
  • Keep .github/workflows/deploy.yml uncommitted until the AWS role exists.

Still Finding Placeholders?

  • Check the spelling of each replacement against the value returned by its source.
  • Confirm that your search includes hidden directories such as .github.
  • Still stuck? Help me check my EKS deployment placeholders.
Provision the EKS runtime

The manifests describe the desired runtime. The next commands create the AWS infrastructure that can enforce that desired state.

  • Switch back to AWS CloudShell.
  • Clone the public repository into your CloudShell home directory by running these commands:
git clone https://github.com/[[GITHUB_OWNER="your GitHub owner"]]/eks-ml-topic-router.git
cd eks-ml-topic-router

What Do These Commands Do?

The first command copies the public repository into a new eks-ml-topic-router directory. The second command moves your CloudShell session into that directory.

  • Confirm the clone contains the k8s and iam directories in the CloudShell file tree.

Repository Clone Failed?

  • Confirm that the repository is public.
  • Check that your GitHub owner matches the owner shown in the repository URL.
  • Still stuck? Help me clone my public repository into AWS CloudShell.

Costs Start With the Cluster

Creating the cluster starts usage-based AWS charges. The EKS control plane costs $0.10 per cluster-hour under standard support.

Compute and networking add separate charges. The cleanup section removes these resources when your demo is complete.

Cluster creation takes several minutes because AWS builds the control plane and networking resources. A quiet stretch in CloudShell is expected while provisioning continues.

  • Create the EKS Auto Mode cluster in us-east-1 by running this command:
eksctl create cluster --name eks-ml-topic-router --region us-east-1 --enable-auto-mode --version 1.36

The command completes after the cluster becomes available. Your CloudShell session is then configured to communicate with eks-ml-topic-router.

Cluster Creation Stopped?

  • Confirm that CloudShell is using us-east-1.
  • Check that your AWS account can create EKS and networking resources.
  • Still stuck? Help me diagnose my EKS Auto Mode cluster creation.

The cluster is live. Your model now has a managed Kubernetes destination waiting for its container image.

  • Create the private ECR repository by running this command:
aws ecr create-repository --repository-name eks-ml-topic-router --region us-east-1

The response identifies the new eks-ml-topic-router repository in us-east-1.

  • Create the application namespace from the repository manifest by running this command:
kubectl apply -f k8s/namespace.yaml

Kubectl reports that the ml-api namespace was created. This gives the deployment role a narrow authorization boundary.

Repository or Namespace Creation Failed?

  • Confirm that the ECR repository name is eks-ml-topic-router.
  • Confirm that CloudShell is inside the cloned eks-ml-topic-router directory before applying the manifest.
  • Still stuck? Help me create the ECR repository and ml-api namespace.
Authorize the GitHub deployment role

Long-lived AWS access keys would leave a reusable credential inside GitHub. The OIDC trust path lets each workflow run request a temporary AWS identity instead.

  • Use the IAM identity provider inventory in the AWS console to determine whether a provider for https://token.actions.githubusercontent.com already exists.
  • Create the provider only when it is absent by running this command from the CloudShell repository directory:
aws iam create-open-id-connect-provider --cli-input-json file://iam/create-open-id-connect-provider.json

The provider connects GitHub's token issuer to AWS. An existing provider can be reused by every compatible workflow in the account.

  • Skip the provider creation command when the account already contains the GitHub provider.
  • Confirm that iam/trust-policy.json points to the existing provider ARN.

Provider Creation Did Not Complete?

  • Check the IAM provider inventory again before attempting another creation.
  • Confirm that iam/create-open-id-connect-provider.json contains the GitHub provider URL and AWS audience from the reference file.
  • Still stuck? Help me verify my GitHub OIDC provider configuration.
  • Create the deployment role and attach its inline policy by running these commands:
aws iam create-role --role-name GitHubActionsEKSRole --assume-role-policy-document file://iam/trust-policy.json
aws iam put-role-policy --role-name GitHubActionsEKSRole --policy-name EcrPushAndEksDeploy --policy-document file://iam/permissions-policy.json

What Does This Role Allow?

  • The trust policy decides which GitHub repository can assume GitHubActionsEKSRole.
  • The inline EcrPushAndEksDeploy policy permits image operations against the project repository.
  • The policy also permits the workflow to describe the named EKS cluster.
  • Replace your-aws-account-id in the next commands with your AWS account ID before running them.
  • Create the EKS access entry and scope its edit policy to ml-api by running these commands:
aws eks create-access-entry --cluster-name eks-ml-topic-router --principal-arn arn:aws:iam::your-aws-account-id:role/GitHubActionsEKSRole
aws eks associate-access-policy \
  --cluster-name eks-ml-topic-router \
  --principal-arn arn:aws:iam::your-aws-account-id:role/GitHubActionsEKSRole \
  --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSEditPolicy \
  --access-scope type=namespace,namespaces=ml-api

How Is Access Scoped?

The access entry connects the AWS role to the EKS cluster. The associated AmazonEKSEditPolicy grants editing only inside ml-api.

The workflow can deploy the topic router without receiving cluster-wide administrator access.

  • Return to the project in PyCharm.
  • Commit the personalized .github/workflows/deploy.yml file.
  • Push the workflow commit to main.

Role or Access Entry Failed?

  • Confirm that every account ID in the IAM files matches the account reported by CloudShell.
  • Confirm that the ml-api namespace exists before associating the namespace-scoped policy.
  • Still stuck? Help me troubleshoot my GitHub deployment role.

Before you verify the role, do you expect it to appear as a cluster principal with access limited to one namespace?

  • Replace your-aws-account-id in the second command with your AWS account ID.
  • Verify the access entry and associated policy by running these commands:
aws eks list-access-entries --cluster-name eks-ml-topic-router
aws eks list-associated-access-policies --cluster-name eks-ml-topic-router --principal-arn arn:aws:iam::your-aws-account-id:role/GitHubActionsEKSRole

What Should You See?

The first response includes the ARN for GitHubActionsEKSRole as an access entry. The second response includes AmazonEKSEditPolicy with namespace scope for ml-api.

That result proves the AWS role exists and the Kubernetes authorization boundary is in place.

That is the cloud security foundation complete. Your cluster can now accept releases from GitHub without storing long-lived AWS keys, so the next step can automate the public deployment and monitoring path.

Deploy Automatically and Monitor in CloudWatch

Your Amazon EKS cluster now has a namespace-scoped OIDC identity for releases. The next goal is a public API that deploys from main.

Manual image pushes make each release harder to reproduce. Terminal-only logs hide whether the deployed service stays healthy.

GitHub Actions turns each main push into a CI/CD release through Amazon ECR. Amazon CloudWatch then captures workload evidence outside the terminal.

In this step, get ready to:
  • Trigger a commit-tagged release from main.
  • Call the public prediction service through its Network Load Balancer.
  • Confirm end-to-end observability in Amazon CloudWatch.
Release through GitHub Actions

The deployment workflow uses short-lived credentials from the role you configured earlier. It tags each image with the commit SHA before updating the Kubernetes Deployment.

  • Switch back to the eks-ml-topic-router project in PyCharm.
  • Return to .github/workflows/deploy.yml from earlier.
  • Confirm that AWS_ROLE_ARN contains your numeric AWS account ID.
  • Save .github/workflows/deploy.yml.

✔️ Awesome, I've got everything!

Your workflow is ready to authenticate through OIDC. Double check that your account-specific role ARN is still saved.

ⓧ I'd like to double check the full code

name: Deploy ML API to Amazon EKS

on:
  push:
    branches:
      - main

permissions:
  id-token: write
  contents: read

env:
  AWS_REGION: us-east-1
  ECR_REPOSITORY: eks-ml-topic-router
  EKS_CLUSTER: eks-ml-topic-router
  K8S_NAMESPACE: ml-api
  AWS_ROLE_ARN: arn:aws:iam::your-aws-account-id:role/GitHubActionsEKSRole

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Check out the repository
        uses: actions/checkout@v7.0.1

      - name: Configure short-lived AWS credentials
        uses: aws-actions/configure-aws-credentials@v6.3.0
        with:
          role-to-assume: ${{ env.AWS_ROLE_ARN }}
          role-session-name: topic-router-deploy
          aws-region: ${{ env.AWS_REGION }}

      - name: Log in to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2.1.7

      - name: Build, test, and push the image
        env:
          REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          IMAGE_TAG: ${{ github.sha }}
        run: |
          docker build -t $REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
          docker push $REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG

      - name: Install kubectl
        uses: Azure/setup-kubectl@v5.1.0
        with:
          version: v1.36.0

      - name: Deploy the commit image
        env:
          REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          IMAGE_TAG: ${{ github.sha }}
        run: |
          aws eks update-kubeconfig --region $AWS_REGION --name $EKS_CLUSTER
          sed "s|IMAGE_URI_PLACEHOLDER|$REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG|g" k8s/app.yaml > /tmp/app.yaml
          kubectl apply -f /tmp/app.yaml
          kubectl rollout status deployment/topic-router --namespace $K8S_NAMESPACE

Compare every line with your workflow. Keep your numeric AWS account ID in place of your-aws-account-id before you commit the file.

Before you push, do you expect the workflow to reuse a floating image tag or tag this release with the commit SHA?

  • Use the Git controls from earlier to commit .github/workflows/deploy.yml.
  • Use the same controls to push the commit to main.

This cloud build can take several minutes while it trains the model inside the image. The running job means your push successfully triggered the release path.

  • Return to the public GitHub repository from earlier.
  • Open the newest GitHub Actions run.
  • Follow the deployment job until it succeeds.

You'll see the workflow authenticate without stored AWS keys. The image tag matches the commit SHA from this push.

How does this release stay reproducible?

  • The OIDC step exchanges the workflow identity for short-lived AWS credentials.
  • The Docker build runs model training plus the unit test inside one image.
  • The commit SHA gives every image an immutable release tag.
  • The final step replaces IMAGE_URI_PLACEHOLDER before applying the Kubernetes resources.

Workflow run not succeeding?

  • Check that AWS_ROLE_ARN contains the numeric account ID you used when creating the role.
  • Confirm the workflow ran from the main branch covered by the OIDC trust policy.
  • Open the first failed workflow step to identify whether authentication or deployment needs attention.
  • Still stuck? Help me diagnose my GitHub Actions deployment failure.
Call the public prediction service

The Network Load Balancer created for the topic-router Service gives your API a public hostname. That hostname routes port 80 traffic to container port 8000.

Before you check, do you think the public endpoint is ready immediately or still being provisioned?

  • Check the public Service from AWS CloudShell by running this command:
kubectl get service topic-router --namespace ml-api

What does this command show?

The command reads the topic-router Service from the ml-api namespace. Its EXTERNAL-IP column holds the public Network Load Balancer hostname once provisioning finishes.

The load balancer may need a few minutes to finish provisioning. A temporary value in EXTERNAL-IP means AWS is still building the public network path.

  • Repeat the Service check until EXTERNAL-IP shows a Network Load Balancer hostname.
  • Record the hostname here: your Network Load Balancer hostname.

Before you send the requests, do you expect both routes to return valid JSON through the public load balancer?

  • Test the health route plus prediction route by running these commands in AWS CloudShell:
curl http://[[HOSTNAME="your Network Load Balancer hostname"]]/health

curl -X POST http://[[HOSTNAME="your Network Load Balancer hostname"]]/predict \
  -H "Content-Type: application/json" \
  -d '{"text":"The spacecraft entered orbit around the moon"}'

What do these requests prove?

  • The /health request proves the load balancer can reach a ready FastAPI container.
  • The /predict request proves the deployed model accepts JSON text.
  • The prediction response proves the container can load model.joblib inside Amazon EKS.

You'll see {"status":"ok"} from the health route. The prediction route returns a JSON object containing a learned category.

Public request not returning JSON?

  • Check that you copied the entire hostname from the EXTERNAL-IP column.
  • Confirm the GitHub Actions rollout completed before testing the routes.
  • Repeat the Service check if the load balancer hostname is still being provisioned.
  • Still stuck? Help me troubleshoot my public topic-router endpoint.
Send telemetry to CloudWatch

CloudWatch telemetry adds usage-based charges. This project limits application-log retention to 7 days so the data does not remain indefinitely.

OTel Container Insights collects container logs plus infrastructure metrics. EKS Pod Identity gives the CloudWatch agent permission without placing AWS credentials inside the cluster.

  • Create the CloudWatch observability role by running these commands in AWS CloudShell:
aws iam create-role \
  --role-name EKS-CloudWatch-Observability-Role \
  --assume-role-policy-document '{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": {"Service": "pods.eks.amazonaws.com"},
    "Action": ["sts:AssumeRole", "sts:TagSession"]
  }]
}'
aws iam attach-role-policy \
  --role-name EKS-CloudWatch-Observability-Role \
  --policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy

What does this role allow?

The trust policy allows EKS pods to assume EKS-CloudWatch-Observability-Role through Pod Identity. The managed policy gives the CloudWatch agent permission to publish telemetry.

The first command prints the new role details. The policy attachment completes without an error when the permissions are in place.

Role creation not completing?

  • Confirm your AWS identity can create IAM roles.
  • Check that the JSON trust policy remains enclosed by matching single quotes.
  • Reuse EKS-CloudWatch-Observability-Role if the role already exists from an earlier attempt.
  • Still stuck? Help me create the CloudWatch observability role safely.
  • Replace your-aws-account-id below with the numeric AWS account ID you used earlier.
  • Create the Pod Identity association plus observability add-on by running these commands:
aws eks create-pod-identity-association \
  --cluster-name eks-ml-topic-router \
  --namespace amazon-cloudwatch \
  --service-account cloudwatch-agent \
  --role-arn arn:aws:iam::your-aws-account-id:role/EKS-CloudWatch-Observability-Role
aws eks create-addon \
  --cluster-name eks-ml-topic-router \
  --addon-name amazon-cloudwatch-observability \
  --configuration-values '{"otelContainerInsights":{"enabled":true}}'

How does telemetry reach CloudWatch?

The Pod Identity association connects the cloudwatch-agent service account to the IAM role. The add-on starts OTel Container Insights on the cluster.

The otelContainerInsights setting enables the open-source telemetry path used for the container metrics.

The add-on may need a few minutes to become active. That wait covers the agent rollout across the EKS compute environment.

  • Confirm amazon-cloudwatch-observability reaches ACTIVE for the eks-ml-topic-router cluster in the EKS console.
  • Generate a fresh JSON prediction event by running the public request again:
curl -X POST http://[[HOSTNAME="your Network Load Balancer hostname"]]/predict \
  -H "Content-Type: application/json" \
  -d '{"text":"The spacecraft entered orbit around the moon"}'

Why send another prediction?

The FastAPI container writes a structured JSON prediction event to standard output. Container Insights collects that event into the application log group.

You'll receive another JSON response containing category. That response confirms the event-producing code ran.

Telemetry normally takes 3 to 5 minutes to arrive. The quiet period is CloudWatch collecting logs plus infrastructure metrics from the new add-on.

  • Wait 3 to 5 minutes for the prediction event to reach CloudWatch.

Before you run the final log check, do you expect StoredBytes to remain at zero or reflect the prediction event?

  • Set the application-log retention period plus inspect the Container Insights log groups by running:
aws logs put-retention-policy \
  --log-group-name "/aws/containerinsights/eks-ml-topic-router/application" \
  --retention-in-days 7
aws logs describe-log-groups \
  --log-group-name-prefix "/aws/containerinsights/eks-ml-topic-router" \
  --query "logGroups[].{Name:logGroupName,StoredBytes:storedBytes}" \
  --output table

What does this check confirm?

The first command caps the application log group at 7 days. The second command lists each matching group with its stored byte count.

A non-zero StoredBytes value proves CloudWatch has ingested container output from the cluster.

You'll see /aws/containerinsights/eks-ml-topic-router/application in the table with a non-zero StoredBytes value.

Before you run the metric query, do you expect Container Insights to expose infrastructure activity from the running API container?

  • Return to Amazon CloudWatch in the AWS Management Console.
  • Open Query Studio.
  • Select PromQL as the query language.
  • Search for container_cpu_usage_seconds_total.
  • Run the metric query.

You'll see metric results for container CPU usage. This confirms infrastructure telemetry is flowing alongside the JSON application logs.

StoredBytes still zero?

  • Confirm the observability add-on reached ACTIVE before checking the log group.
  • Repeat the public prediction request to create another application event.
  • Allow another 3 to 5 minutes for log delivery.
  • Still stuck? Help me trace missing EKS telemetry in CloudWatch.

That's the full delivery path working. Each main push can now produce a versioned release with public predictions plus managed operational evidence.

Secret mission

Add a Model Quality Release Gate

Add a held-out macro F1 gate before image publication. You will prove that the workflow enforces the release policy across two threshold settings.

Clean Up Your Resources

The Amazon EKS standard-support control plane costs $0.10 per cluster-hour. Delete the cluster promptly when you finish using it.

Amazon EKS Auto Mode compute adds usage charges.

The Network Load Balancer adds usage charges.

Amazon ECR storage adds usage charges. Amazon CloudWatch telemetry can add more.

Nice Work!

Nice Work!

You completed a full MLOps release path. Your four-topic machine learning router now moves from held-out evaluation to a public prediction API on Amazon EKS Auto Mode.

You've learned how to:

  • Measure a majority-class baseline with macro F1. The score exposes its failure across three routing classes.
  • Build a leak-safe text-classification pipeline with TF-IDF features. Train LinearSVC on sparse word unigrams plus bigrams. Write held-out evaluation results to metrics.json. Persist the fitted pipeline in model.joblib.
  • Serve predictions from a tested FastAPI container. Deploy commit-tagged images from Amazon ECR to Amazon EKS Auto Mode through GitHub Actions. Authenticate each workflow through short-lived OIDC credentials. Inspect structured prediction events in Amazon CloudWatch. Query container metrics with PromQL.
  • Complete the Secret Mission by adding a held-out macro F1 release gate before image publication. Prove the policy with a deliberate 0.99 failure. Restore 0.75 to complete the rollout.

Ready to quiz yourself?