KubeQuest: Neon Incident Arena

Deploy a resilient Kubernetes app on Amazon EKS and safely roll back releases.

Introduction

30 Second Summary

A website can look healthy until the one machine serving it disappears. The real test is whether it stays available when something fails.

In this project, you will build a neon web arena on Amazon EKS Auto Mode. The public screen makes Kubernetes resilience visible each time a Pod fails or a new release rolls out.

What You'll Build

You will refresh a neon public page to watch serving Pod names change during recovery before its release label advances from v1 to v2 during a live rollout.

By the end of this project, you'll have:

  • A public KubeQuest URL that displays the current release beside the hostname of the Pod serving each request.
  • A visible outage test where deleting an unmanaged Pod leaves the public page unavailable.
  • A self-healing recovery demo where a three-replica Deployment replaces a deleted Pod while the public endpoint keeps responding.
  • Secret Mission: Add a readiness probe that blocks a broken v3 release before using a rollback to restore healthy v2.

Are there any prerequisites?

You need a Linux computer with curl, tar, sha256sum, and sudo already available.

You also need an Amazon Web Services account with credits plus enough IAM access to create the cloud resources in this project.

Before We Start

Before any hands-on work begins, this is your moment to define the purpose of your public Kubernetes resilience demo. That commitment gives the failures and recoveries ahead a clear reason to matter.

Prepare Your Linux Workstation

KubeQuest needs one reliable path from your Linux workstation to Docker, Amazon Elastic Container Registry, and Amazon Elastic Kubernetes Service.

This step establishes that authenticated command-line path. You will prove every tool works before creating any paid cloud resources.

In this step, get ready to:
  • Install AWS CLI v2 plus Docker Engine.
  • Install kubectl 1.37.0 plus eksctl 0.230.0.
  • Authenticate to AWS with the required infrastructure permissions.
Install the cloud and container tools

AWS CLI connects your terminal to your AWS account. Docker Engine builds plus runs the container image you will publish in the next step.

  • Check whether AWS CLI v2 is installed by running:
aws --version

What does this check prove?

The command reports whether AWS CLI is available. Its output also identifies the installed major version.

✔️ I see AWS CLI version 2

Good. AWS CLI v2 is ready to authenticate with your AWS account.

ⓧ I see AWS CLI version 1

The installed AWS CLI is older than the release required by this project. Install AWS CLI v2 into the shared system locations.

  • Install AWS CLI v2 by running:
curl -fsSL https://awscli.amazonaws.com/v2/install.sh | sudo bash -s -- --system

What does this installer do?

The official installer selects the AWS CLI build for your Linux machine. The system option makes the aws command available to all users.

  • Confirm that AWS CLI v2 now takes precedence by running:
aws --version

What should you see?

The output should identify an AWS CLI 2.x release. This confirms that your shell resolves the newer installation.

ⓧ Command not found

AWS CLI is missing from your workstation. The official installer supports current 64-bit Linux systems.

  • Install AWS CLI v2 for all users by running:
curl -fsSL https://awscli.amazonaws.com/v2/install.sh | sudo bash -s -- --system

What does this installer do?

The script downloads the AWS CLI build for your Linux machine. It places the command in shared system locations.

  • Confirm the installation by running:
aws --version

What should you see?

The output should identify an AWS CLI 2.x release. Your terminal can now call AWS services.

AWS CLI still unavailable?

Restart your terminal so the shell reloads its command path. Repeat the version check in the new terminal.

If the command still fails, help me diagnose the AWS CLI installation.

The Docker convenience installer is intended for development workstations. A privileged installer deserves a careful review before it changes your system.

  • Check whether Docker can start a test container by running:
sudo docker run hello-world

What does this test prove?

Docker downloads a small test image when needed. It starts a container that prints a confirmation before exiting.

✔️ The container prints a confirmation

That confirms Docker Engine is running. Your workstation can now download images plus start containers.

ⓧ Command not found

Docker Engine needs to be installed. You will preview the official development installer before running it with elevated permissions.

  • Download the Docker development installer by running:
curl -fsSL https://get.docker.com -o get-docker.sh

What did you download?

This saves Docker's convenience installer as get-docker.sh in your current directory. Your system has not been changed yet.

  • Preview the installer's planned changes by running:
sudo sh ./get-docker.sh --dry-run

Why preview the script?

The dry run lists the packages plus repository changes without applying them. This gives you a chance to inspect the privileged operation first.

  • Install Docker Engine by running:
sudo sh get-docker.sh

What does the installer add?

The script detects your Linux distribution. It configures the matching package source before installing Docker Engine.

RPM-based distributions require a service command after installation. Debian-based installations normally start Docker automatically.

  • Start Docker on an RPM-based distribution by running:
sudo systemctl enable --now docker

What does this service command do?

The command starts Docker immediately. It also enables Docker when your machine boots.

  • Verify Docker Engine by running:
sudo docker run hello-world

What should you see?

The test container should print a successful confirmation before exiting. That result proves the Docker daemon accepts container requests.

ⓧ The Docker daemon is unavailable

The Docker command exists. Its background service needs to be running before it can start containers.

  • Start Docker immediately by running:
sudo systemctl enable --now docker

What does this service command do?

The command starts the Docker daemon. It also enables the service for future restarts.

  • Retry the test container by running:
sudo docker run hello-world

What should you see?

The test image should print a successful confirmation. Docker can now start containers on your workstation.

Docker test still failing?

Confirm that your distribution uses systemd before relying on the service command. Confirm that your account can run privileged commands with sudo.

If the daemon remains unavailable, help me troubleshoot Docker Engine.

Install the Kubernetes clients

The future cluster uses Kubernetes 1.36. A kubectl 1.37.0 client stays within the supported one-minor-version compatibility range.

eksctl creates the cluster from configuration later. This project pins eksctl 0.230.0 so the cluster workflow stays consistent.

  • Detect your Linux machine's CPU architecture by running:
uname -m

How do you read the result?

An x86_64 result uses the amd64 downloads. An aarch64 or arm64 result uses the arm64 downloads.

  • Check the current kubectl plus eksctl versions by running:
kubectl version --client
eksctl version

What do these checks show?

The first command reports the local kubectl client version without contacting a cluster. The second command reports the installed eksctl release.

✔️ I see kubectl 1.37.0 and eksctl 0.230.0

Both clients match the project pins. Your workstation is ready for the cluster configuration you will create later.

ⓧ I see an older version

Older clients can behave differently from the pinned workflow. Replace them with the builds for your detected architecture.

x86_64 or amd64

  • Download the amd64 kubectl binary plus its checksum by running:
curl -LO https://dl.k8s.io/release/v1.37.0/bin/linux/amd64/kubectl
curl -LO https://dl.k8s.io/release/v1.37.0/bin/linux/amd64/kubectl.sha256
echo "$(cat kubectl.sha256)  kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
What does this install do?

These commands download kubectl 1.37.0 for amd64. The checksum verifies the binary before installation.

A successful integrity check prints kubectl: OK.

  • Download plus verify eksctl 0.230.0 for amd64 by running:
curl -LO https://github.com/eksctl-io/eksctl/releases/download/v0.230.0/eksctl_Linux_amd64.tar.gz
echo "a2060956f117c3065abafda5c1f681679b9c3716675d70ce4ffff46033b02c35  eksctl_Linux_amd64.tar.gz" | sha256sum --check
tar -xzf eksctl_Linux_amd64.tar.gz -C /tmp
sudo install -m 0755 /tmp/eksctl /usr/local/bin
What does this install do?

These commands download the pinned Linux archive. The published checksum verifies it before eksctl is installed in /usr/local/bin.

  • Confirm both pinned versions by running:
kubectl version --client
eksctl version
What should you see?

kubectl should report 1.37.0. eksctl should report 0.230.0.

aarch64 or arm64

  • Download the arm64 kubectl binary plus its checksum by running:
curl -LO https://dl.k8s.io/release/v1.37.0/bin/linux/arm64/kubectl
curl -LO https://dl.k8s.io/release/v1.37.0/bin/linux/arm64/kubectl.sha256
echo "$(cat kubectl.sha256)  kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
What does this install do?

These commands download kubectl 1.37.0 for arm64. The checksum verifies the binary before installation.

A successful integrity check prints kubectl: OK.

  • Download plus verify eksctl 0.230.0 for arm64 by running:
curl -LO https://github.com/eksctl-io/eksctl/releases/download/v0.230.0/eksctl_Linux_arm64.tar.gz
echo "21afe8a1e38f0e8153a1f27ff7af6b90e309a0411a1438139463dac2f866674d  eksctl_Linux_arm64.tar.gz" | sha256sum --check
tar -xzf eksctl_Linux_arm64.tar.gz -C /tmp
sudo install -m 0755 /tmp/eksctl /usr/local/bin
What does this install do?

These commands download the pinned Linux archive. The published checksum verifies it before eksctl is installed in /usr/local/bin.

  • Confirm both pinned versions by running:
kubectl version --client
eksctl version
What should you see?

kubectl should report 1.37.0. eksctl should report 0.230.0.

ⓧ One or both commands are missing

Both pinned clients need to be available before the cluster work begins. Use the installer for your detected architecture.

x86_64 or amd64

  • Download the amd64 kubectl binary plus its checksum by running:
curl -LO https://dl.k8s.io/release/v1.37.0/bin/linux/amd64/kubectl
curl -LO https://dl.k8s.io/release/v1.37.0/bin/linux/amd64/kubectl.sha256
echo "$(cat kubectl.sha256)  kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
What does this install do?

These commands download kubectl 1.37.0 for amd64. The checksum verifies the binary before installation.

A successful integrity check prints kubectl: OK.

  • Download plus verify eksctl 0.230.0 for amd64 by running:
curl -LO https://github.com/eksctl-io/eksctl/releases/download/v0.230.0/eksctl_Linux_amd64.tar.gz
echo "a2060956f117c3065abafda5c1f681679b9c3716675d70ce4ffff46033b02c35  eksctl_Linux_amd64.tar.gz" | sha256sum --check
tar -xzf eksctl_Linux_amd64.tar.gz -C /tmp
sudo install -m 0755 /tmp/eksctl /usr/local/bin
What does this install do?

These commands download the pinned Linux archive. The published checksum verifies it before eksctl is installed in /usr/local/bin.

  • Confirm both pinned versions by running:
kubectl version --client
eksctl version
What should you see?

kubectl should report 1.37.0. eksctl should report 0.230.0.

aarch64 or arm64

  • Download the arm64 kubectl binary plus its checksum by running:
curl -LO https://dl.k8s.io/release/v1.37.0/bin/linux/arm64/kubectl
curl -LO https://dl.k8s.io/release/v1.37.0/bin/linux/arm64/kubectl.sha256
echo "$(cat kubectl.sha256)  kubectl" | sha256sum --check
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
What does this install do?

These commands download kubectl 1.37.0 for arm64. The checksum verifies the binary before installation.

A successful integrity check prints kubectl: OK.

  • Download plus verify eksctl 0.230.0 for arm64 by running:
curl -LO https://github.com/eksctl-io/eksctl/releases/download/v0.230.0/eksctl_Linux_arm64.tar.gz
echo "21afe8a1e38f0e8153a1f27ff7af6b90e309a0411a1438139463dac2f866674d  eksctl_Linux_arm64.tar.gz" | sha256sum --check
tar -xzf eksctl_Linux_arm64.tar.gz -C /tmp
sudo install -m 0755 /tmp/eksctl /usr/local/bin
What does this install do?

These commands download the pinned Linux archive. The published checksum verifies it before eksctl is installed in /usr/local/bin.

  • Confirm both pinned versions by running:
kubectl version --client
eksctl version
What should you see?

kubectl should report 1.37.0. eksctl should report 0.230.0.

Seeing the wrong client version?

An older executable may appear earlier in your command path. Restart your terminal before checking both versions again.

If the version remains wrong, help me locate the conflicting installation.

Authenticate with AWS

The tools now exist locally. Your final checkpoint proves they can use an IAM identity with enough authority to build the KubeQuest environment.

An existing authenticated AWS profile can supply credentials. The main path uses the browser-based AWS sign-in flow.

  • Authenticate through AWS sign-in by running:
aws login

What happens during login?

AWS CLI attempts to open your default browser. Complete the sign-in flow with the identity approved for this project.

If you already use an authenticated AWS profile, keep that profile active. The identity check below reveals which credentials the CLI resolves.

  • Set the project region by running:
aws configure set region us-west-2

Why set the region now?

The command stores us-west-2 as the default AWS region. Later ECR plus EKS commands use this setting.

  • Reveal the AWS identity used by your terminal by running:
aws sts get-caller-identity

What does the response prove?

The response identifies the caller through its Account, UserId, and Arn fields. This proves your terminal can make an authenticated AWS request.

Authentication confirms who you are. Permission validation confirms that this identity can create every infrastructure category used later.

  • Confirm that the returned identity can manage Amazon EKS resources.
  • Confirm that the returned identity can manage Amazon EC2 networking resources.
  • Confirm that the returned identity can create plus pass the required IAM roles.
  • Confirm that the returned identity can create Amazon ECR repositories.
  • Confirm that the returned identity can manage Elastic Load Balancing resources.
  • Request the missing permissions from your AWS administrator before continuing.

AWS access not working?

A failed identity request usually means the login session is missing or expired. Repeat the browser sign-in flow before checking the identity again.

A permissions failure requires a policy change from whoever manages your AWS account. Help me identify the missing AWS permission.

Before you run the final check, which command do you think is most likely to expose an unfinished part of your setup?

  • Run the complete workstation verification with these commands:
aws --version
sudo docker run hello-world
kubectl version --client
eksctl version
aws sts get-caller-identity

What should the full check prove?

  • AWS CLI reports a 2.x release.
  • Docker prints its successful test-container confirmation.
  • kubectl reports client version 1.37.0.
  • eksctl reports version 0.230.0.
  • AWS returns the Account, UserId, and Arn for your authenticated identity.

That is the workstation foundation locked in. Every local tool can now complete its part of the path to AWS.

Your Linux workstation is ready for KubeQuest. Next up, you will build the neon application locally before publishing the tested image to Amazon ECR.

Build and Publish KubeQuest v1

Your workstation now has one authenticated path to AWS. The next goal is to turn that setup into a visible application you can test before any cluster exists.

You will package KubeQuest as a container image with Docker. You will run the exact image locally before publishing the immutable v1 artifact to Amazon Elastic Container Registry.

In this step, get ready to:
  • Create the KubeQuest application and package it as a Docker image.
  • Run KubeQuest locally and confirm the release details appear.
  • Publish the tested v1 image to Amazon ECR.
Create and build the v1 image

The application files need to share one directory because Docker uses that directory as its build context. You will keep the project on your Desktop so it is easy to find in VS Code.

  • Move to your Desktop by running this command:
cd ~/Desktop

What does this command do?

This command moves your terminal into the Desktop directory. The next command creates the project in this known location.

  • Create the kubequest-arena folder and move into it by running these commands:
mkdir kubequest-arena
cd kubequest-arena

What do these commands do?

The first command creates the directory that holds the application. The second command makes that directory your current terminal location.

  • Use your Linux application launcher to start VS Code.
  • Open the ~/Desktop/kubequest-arena folder in VS Code.
  • Create app.py inside the kubequest-arena folder.
  • Paste this first section into app.py.
from html import escape
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from socket import gethostname

PORT = 8080
RELEASE = "v1"
ACCENT = "#22d3ee"
READINESS_OK = True


class ArenaHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        path = self.path.split("?", 1)[0]

        if path == "/ready":
            status = 200 if READINESS_OK else 503
            message = "ready" if READINESS_OK else "not ready"
            self.send_page(message, status, "text/plain; charset=utf-8")
            return

        if path not in ("/", "/index.html"):
            self.send_page("Not found", 404, "text/plain; charset=utf-8")
            return

        pod_name = escape(gethostname())

What does this section do?

  • The constants define the server port, visible release, accent colour, and readiness state.
  • The do_GET() method separates readiness requests from page requests.
  • The gethostname() result identifies the container serving each request.
  • Continue directly below pod_name = escape(gethostname()) with this section:
        body = f"""<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>KubeQuest Neon Incident Arena</title>
  <style>
    :root {{ --accent: {ACCENT}; }}
    * {{ box-sizing: border-box; }}
    body {{
      min-height: 100vh;
      margin: 0;
      display: grid;
      place-items: center;
      color: #f8fafc;
      background: radial-gradient(circle at top, #312e81, #020617 58%);
      font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
    }}
    main {{
      width: min(760px, 92vw);
      padding: 2.5rem;
      border: 1px solid var(--accent);
      border-radius: 24px;
      background: rgba(15, 23, 42, 0.88);
      box-shadow: 0 0 55px color-mix(in srgb, var(--accent) 45%, transparent);
    }}

How does the arena get its look?

This section begins the HTML response and defines the page layout. The ACCENT value feeds a CSS variable so each release can carry a distinct colour.

  • Continue directly below the closing brace for main with this section:
    .status {{ color: var(--accent); letter-spacing: 0.18em; text-transform: uppercase; }}
    h1 {{ font-size: clamp(2rem, 7vw, 4.5rem); margin: 0.4rem 0 1.5rem; }}
    .panel {{ padding: 1rem; margin: 1rem 0; border-radius: 14px; background: #0f172a; }}
    code {{ color: var(--accent); overflow-wrap: anywhere; }}
    button {{
      width: 100%; padding: 1rem; margin-top: 1rem; border: 0; border-radius: 12px;
      color: #020617; background: var(--accent); font: inherit; font-weight: 800; cursor: pointer;
    }}
  </style>
</head>
<body>
  <main>
    <div class="status">System online</div>
    <h1>KubeQuest</h1>
    <p>Neon Incident Arena release <code>{RELEASE}</code></p>
    <div class="panel">Request served by Pod <code>{pod_name}</code></div>
    <p>Refresh repeatedly while replicas change or a rollout is running.</p>
    <button onclick="window.location.reload()">Refresh Arena</button>
  </main>
</body>
</html>"""
        self.send_page(body, 200, "text/html; charset=utf-8")

What appears in the browser?

The page displays the current release and the hostname returned by the container. The refresh button makes those workload details easy to check during later recovery tests.

  • Finish app.py by adding this final section directly below the page response:
    def send_page(self, body, status, content_type):
        payload = body.encode("utf-8")
        self.send_response(status)
        self.send_header("Content-Type", content_type)
        self.send_header("Content-Length", str(len(payload)))
        self.send_header("Cache-Control", "no-store")
        self.end_headers()
        self.wfile.write(payload)


if __name__ == "__main__":
    print(f"KubeQuest {RELEASE} listening on port {PORT}", flush=True)
    ThreadingHTTPServer(("0.0.0.0", PORT), ArenaHandler).serve_forever()

How does the server respond?

  • The send_page() helper turns each response into bytes and sends the matching HTTP headers.
  • The final two lines start a threaded server on port 8080 when the container runs.
  • Save app.py.
  • Create Dockerfile inside the same kubequest-arena folder.
  • Paste this container definition into Dockerfile.
FROM python:3.14-slim@sha256:cad9a2c871761c413caa6fdd6441c783451e740a48aaeba60ae62a8b53525ef6

WORKDIR /app
COPY app.py /app/app.py
USER 10001:10001
EXPOSE 8080
CMD ["python", "/app/app.py"]

What does the Dockerfile do?

  • The pinned Python 3.14-slim digest gives every build the same base image.
  • The image copies app.py into /app and runs it as a non-root user.
  • Port 8080 becomes the application port exposed by the image.
  • Save Dockerfile.

✔️ Awesome, I've got everything!

Both project files are ready. Double check that you saved app.py and Dockerfile before building the image.

ⓧ I'd like to double check the full code

Compare your complete app.py with this version:

from html import escape
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from socket import gethostname

PORT = 8080
RELEASE = "v1"
ACCENT = "#22d3ee"
READINESS_OK = True


class ArenaHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        path = self.path.split("?", 1)[0]

        if path == "/ready":
            status = 200 if READINESS_OK else 503
            message = "ready" if READINESS_OK else "not ready"
            self.send_page(message, status, "text/plain; charset=utf-8")
            return

        if path not in ("/", "/index.html"):
            self.send_page("Not found", 404, "text/plain; charset=utf-8")
            return

        pod_name = escape(gethostname())
        body = f"""<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>KubeQuest Neon Incident Arena</title>
  <style>
    :root {{ --accent: {ACCENT}; }}
    * {{ box-sizing: border-box; }}
    body {{
      min-height: 100vh;
      margin: 0;
      display: grid;
      place-items: center;
      color: #f8fafc;
      background: radial-gradient(circle at top, #312e81, #020617 58%);
      font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
    }}
    main {{
      width: min(760px, 92vw);
      padding: 2.5rem;
      border: 1px solid var(--accent);
      border-radius: 24px;
      background: rgba(15, 23, 42, 0.88);
      box-shadow: 0 0 55px color-mix(in srgb, var(--accent) 45%, transparent);
    }}
    .status {{ color: var(--accent); letter-spacing: 0.18em; text-transform: uppercase; }}
    h1 {{ font-size: clamp(2rem, 7vw, 4.5rem); margin: 0.4rem 0 1.5rem; }}
    .panel {{ padding: 1rem; margin: 1rem 0; border-radius: 14px; background: #0f172a; }}
    code {{ color: var(--accent); overflow-wrap: anywhere; }}
    button {{
      width: 100%; padding: 1rem; margin-top: 1rem; border: 0; border-radius: 12px;
      color: #020617; background: var(--accent); font: inherit; font-weight: 800; cursor: pointer;
    }}
  </style>
</head>
<body>
  <main>
    <div class="status">System online</div>
    <h1>KubeQuest</h1>
    <p>Neon Incident Arena release <code>{RELEASE}</code></p>
    <div class="panel">Request served by Pod <code>{pod_name}</code></div>
    <p>Refresh repeatedly while replicas change or a rollout is running.</p>
    <button onclick="window.location.reload()">Refresh Arena</button>
  </main>
</body>
</html>"""
        self.send_page(body, 200, "text/html; charset=utf-8")

    def send_page(self, body, status, content_type):
        payload = body.encode("utf-8")
        self.send_response(status)
        self.send_header("Content-Type", content_type)
        self.send_header("Content-Length", str(len(payload)))
        self.send_header("Cache-Control", "no-store")
        self.end_headers()
        self.wfile.write(payload)


if __name__ == "__main__":
    print(f"KubeQuest {RELEASE} listening on port {PORT}", flush=True)
    ThreadingHTTPServer(("0.0.0.0", PORT), ArenaHandler).serve_forever()

Compare your complete Dockerfile with this version:

FROM python:3.14-slim@sha256:cad9a2c871761c413caa6fdd6441c783451e740a48aaeba60ae62a8b53525ef6

WORKDIR /app
COPY app.py /app/app.py
USER 10001:10001
EXPOSE 8080
CMD ["python", "/app/app.py"]

The first build may take a few minutes while Docker downloads the pinned Python layers. A quiet download does not mean the build has stalled.

  • Build the local kubequest-arena:v1 image by running this command:
sudo docker build -t kubequest-arena:v1 .

What does this build create?

Docker reads Dockerfile from the current directory and copies app.py into the image. The -t option gives the finished image the local name kubequest-arena:v1.

The build returns to the prompt without an error. The completed output identifies the image with the kubequest-arena:v1 tag.

Build not completing?

Check that the terminal is inside ~/Desktop/kubequest-arena. Docker needs to find both Dockerfile and app.py in that directory.

Compare the first line of Dockerfile with the pinned image digest above if the base image cannot be resolved.

Help me debug the KubeQuest Docker build.

Test KubeQuest locally

A local run proves that the packaged image starts correctly before it becomes a cloud workload. This keeps the test focused on the exact artifact you plan to publish.

Before you start the container, what release and hostname details do you expect the page to display?

  • Start KubeQuest on localhost by running this command:
sudo docker run --rm -p 127.0.0.1:8080:8080 kubequest-arena:v1

What does this command do?

The port mapping sends local traffic on 127.0.0.1:8080 to port 8080 inside the container. The --rm option removes the stopped container while keeping the image.

This process occupies the terminal until you stop it. Leave it running while you check the page.

  • Open http://127.0.0.1:8080 in your web browser.
  • Click Refresh Arena on the KubeQuest page.

You will see the neon KubeQuest page showing release v1 and the container hostname. The page remains available after you refresh it.

Page not loading?

Keep the container command running while you open the page. If the command stopped, compare app.py with the full-code tab for an indentation or quote mismatch.

Check that your browser URL uses port 8080 on 127.0.0.1.

Help me debug why the KubeQuest container is not loading locally.

  • Return to the terminal running the container.
  • Stop the local container by pressing Ctrl+C.

You have proved that the packaged v1 application starts and serves its release details. That tested image is now ready for the registry.

Publish v1 to Amazon ECR

Amazon ECR gives the tested image an account-specific address that Amazon EKS can pull later. The repository preserves the same v1 tag you just ran locally.

This action creates billable AWS storage. Your project usage is covered by your AWS credits, but the cleanup section still matters because credits do not stop resources automatically.

  • Set the AWS region and derive the registry values by running these commands:
AWS_REGION=us-west-2
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
REPO_URI="$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/kubequest-arena"
export AWS_REGION AWS_ACCOUNT_ID REPO_URI

What do these variables hold?

  • The AWS_REGION variable keeps every registry command in us-west-2.
  • The AWS_ACCOUNT_ID variable captures the account returned by your authenticated identity.
  • The REPO_URI variable combines the account, region, and repository name into the image destination.
  • Create the private kubequest-arena repository by running this command:
aws ecr create-repository --repository-name kubequest-arena --region "$AWS_REGION"

What does this command create?

Amazon ECR creates the repository in your configured account and region. The response includes a repository URI ending in /kubequest-arena.

  • Authenticate Docker to your private registry by running this command:
aws ecr get-login-password --region "$AWS_REGION" | sudo docker login --username AWS --password-stdin "$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com"

How does registry authentication work?

The AWS CLI retrieves a temporary registry password and sends it directly to Docker. The password stays out of the command text while Docker gains permission to push the image.

  • Tag the tested local image with its Amazon ECR address by running this command:
sudo docker tag kubequest-arena:v1 "$REPO_URI:v1"

Why add another image tag?

The new tag points the existing local image at your private repository URI. Docker can now match the local artifact to its remote destination without rebuilding it.

Before you push, what evidence do you expect Docker to print when the image reaches Amazon ECR?

  • Push the tested v1 image to Amazon ECR by running this command:
sudo docker push "$REPO_URI:v1"

What does the push prove?

Docker uploads the image layers associated with v1 to your private repository. The final digest identifies the immutable image content that Amazon EKS can pull.

You will see the layers complete followed by an image digest. That digest confirms the tested kubequest-arena:v1 image reached Amazon ECR.

Push not completing?

Repeat the registry authentication command if Docker reports an authentication problem. Confirm that AWS_REGION still contains us-west-2.

Confirm that the repository creation response named kubequest-arena and that REPO_URI ends with the same name.

Help me debug an Amazon ECR image push failure.

Your local application and remote image now match. Next up, you will use that published artifact to launch the neon arena on Amazon EKS.

Launch the Arena on Amazon EKS

Your tested v1 image is now waiting in Amazon Elastic Container Registry (Amazon ECR). The next goal is to run that exact image on a public Amazon Elastic Kubernetes Service (Amazon EKS) cluster.

This first cloud release uses one unmanaged Pod. It creates a visible baseline with no desired-state controller capable of restoring the workload.

In this step, get ready to:
  • Create a Kubernetes 1.36 cluster with Amazon EKS Auto Mode.
  • Launch the KubeQuest v1 image as one unmanaged Pod.
  • Expose the arena through a public Network Load Balancer.
Create the Auto Mode cluster

A cluster configuration file gives eksctl a repeatable description of the control plane. Amazon EKS Auto Mode also manages workload compute based on what you deploy.

Why Auto Mode instead of managed node groups?

Managed node groups give you direct control over the node lifecycle. Auto Mode keeps this project focused on Kubernetes workloads.

AWS provisions compute in response to your Pods. You can spend your time observing deployments and releases.

  • Switch back to Visual Studio Code from the previous step.
  • Use the Explorer sidebar to create cluster-config.yaml beside app.py and Dockerfile.
  • Paste this cluster configuration into cluster-config.yaml:
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: kubequest-arena
  region: us-west-2
  version: "1.36"

autoModeConfig:
  enabled: true

What does this configuration do?

  • The metadata names the cluster kubequest-arena in us-west-2.
  • The version field pins the Kubernetes control plane to 1.36.
  • The Auto Mode setting enables AWS-managed compute and networking capabilities.
  • Save cluster-config.yaml.

The Explorer sidebar now lists cluster-config.yaml beside your application files.

Configuration file missing?

Check that cluster-config.yaml sits in the same folder as app.py. A file nested inside another folder will not match the command you run next.

Ask for help checking the file location and YAML indentation.

This Creates Paid Resources

Paid AWS usage begins when you create the cluster. Pause here if you need to confirm your available credits.

The EKS control plane costs $0.10 per cluster per hour during standard support. Compute and public networking add separate charges.

  • Create the cluster from cluster-config.yaml by running this command:
eksctl create cluster -f cluster-config.yaml

What does this command do?

The command asks AWS to create the EKS control plane described in your file. It also configures kubectl to communicate with the new cluster.

Cluster creation typically takes 15-20 minutes. Steady log output means AWS is still provisioning the resources.

The command finishes after the cluster becomes ready. Your terminal output confirms that the cluster is active.

Cluster creation failed?

An authorization failure means your AWS identity lacks permission for one of the required EKS, EC2 networking, IAM, or load-balancing operations. Confirm the permissions from the workstation setup before retrying.

A configuration error usually points to spacing in cluster-config.yaml or an unavailable Kubernetes version in the selected region.

Ask for help diagnosing the eksctl output.

Before you check the cluster, what evidence would prove that your terminal can reach its control plane?

  • Confirm cluster access and inspect the built-in node pools by running:
kubectl cluster-info
kubectl get nodepools

What do these checks prove?

  • The cluster information confirms that your current Kubernetes context reaches the new control plane.
  • The node-pool list confirms that Auto Mode created the built-in general-purpose and system pools.

You should see control-plane details followed by the built-in node pools. That confirms your cluster is ready to accept workloads.

Launch the unmanaged Pod

A Pod is the smallest deployable workload in Kubernetes. This manifest asks Auto Mode for compute that can run the KubeQuest image from your private registry.

  • Return to the Explorer sidebar in VS Code.
  • Create a folder named k8s beside app.py.
  • Create pod.yaml inside the k8s folder.
  • Paste this Pod manifest into k8s/pod.yaml:
apiVersion: v1
kind: Pod
metadata:
  name: neon-arena
  labels:
    app: neon-arena
spec:
  nodeSelector:
    eks.amazonaws.com/compute-type: auto
  containers:
    - name: arena
      image: IMAGE_URI_PLACEHOLDER:v1
      ports:
        - name: http
          containerPort: 8080
      resources:
        requests:
          cpu: "100m"
          memory: "128Mi"
        limits:
          cpu: "500m"
          memory: "256Mi"

What does this Pod manifest do?

  • The metadata creates one Pod named neon-arena with the label app=neon-arena.
  • The node selector requires compute managed by EKS Auto Mode.
  • The container listens on port 8080 with defined CPU and memory boundaries.
  • The image placeholder keeps the local manifest reusable across AWS accounts.
  • Save k8s/pod.yaml.

The Explorer sidebar now shows pod.yaml inside the k8s folder.

Pod manifest not saving?

Confirm that the file path is exactly k8s/pod.yaml. Check the indentation below containers if VS Code highlights invalid YAML.

Ask for help checking the manifest structure.

  • Replace the image placeholder in the command stream by running:
sed "s|IMAGE_URI_PLACEHOLDER|$REPO_URI|" k8s/pod.yaml | kubectl apply -f -

How does the streamed replacement work?

The first stage replaces IMAGE_URI_PLACEHOLDER with the repository URI stored in REPO_URI. The transformed manifest flows directly to the Kubernetes API.

Your saved k8s/pod.yaml keeps the placeholder. The live Pod receives the account-specific v1 image address.

Before you watch the Pod, what state do you expect while Auto Mode finds compute for it?

  • Watch the Pod progress by running:
kubectl get pods -w

What does the watch show?

The watch keeps printing changes to Pod state. The first Pod can remain pending for several minutes while Auto Mode provisions compute.

A running state proves that EKS pulled your private v1 image from Amazon ECR. The container is now serving KubeQuest inside the cluster.

  • Stop the watch after neon-arena reaches its running state by pressing Ctrl+C.

Pod stuck pending?

Give Auto Mode several minutes to provision its first compute capacity. Persistent scheduling messages can point to an AWS quota or permission problem.

An image-pull problem usually means REPO_URI was empty or the v1 image is missing from the repository.

Ask for help interpreting the Pod state.

The project also keeps the three-replica release definition beside the Pod manifest. This gives the later recovery workflow an exact desired-state file.

  • Create deployment.yaml inside the k8s folder.
  • Begin k8s/deployment.yaml by pasting this first section:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: neon-arena
spec:
  replicas: 3
  progressDeadlineSeconds: 60
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  selector:
    matchLabels:
      app: neon-arena
  template:
    metadata:
      labels:
        app: neon-arena
    spec:
      nodeSelector:
        eks.amazonaws.com/compute-type: auto
      containers:
        - name: arena
          image: IMAGE_URI_PLACEHOLDER:v2
          ports:
            - name: http
              containerPort: 8080

What does this first section define?

A Deployment records a desired replica count and rollout strategy. Its selector matches Pods carrying the app=neon-arena label.

The Pod template targets Auto Mode compute. It also reserves the v2 image tag for the later release workflow.

  • Complete the same container definition by adding this section directly below the port configuration:
          resources:
            requests:
              cpu: "100m"
              memory: "128Mi"
            limits:
              cpu: "500m"
              memory: "256Mi"

What does this final section define?

The resource requests reserve enough CPU and memory for scheduling. The limits cap how much one arena container can consume.

  • Save k8s/deployment.yaml.

The Explorer sidebar now lists a complete three-replica Deployment manifest inside k8s.

Deployment manifest incomplete?

Make sure the resources section stays under the arena container. Its indentation must align with the image and ports fields.

Ask for help combining the two YAML sections.

Expose the arena publicly

A Kubernetes Service gives traffic a stable destination even when its selected Pods change. A public Network Load Balancer carries internet requests to that Service on port 80.

  • Create service.yaml inside the k8s folder.
  • Paste this Service manifest into k8s/service.yaml:
apiVersion: v1
kind: Service
metadata:
  name: neon-arena
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
spec:
  type: LoadBalancer
  loadBalancerClass: eks.amazonaws.com/nlb
  selector:
    app: neon-arena
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 8080

What does this Service manifest do?

  • The selector sends requests to Pods carrying the app=neon-arena label.
  • The load-balancer class asks EKS Auto Mode to provision a Network Load Balancer.
  • The annotation makes the load balancer internet-facing.
  • Port 80 forwards requests to container port 8080.
  • Save k8s/service.yaml.

The k8s folder now contains the Pod, Service, and Deployment manifests.

✔️ Awesome, I've got everything!

Great. Confirm that cluster-config.yaml and all three files inside k8s are saved.

ⓧ I'd like to double check the full code

Compare your cluster configuration with this complete file:

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: kubequest-arena
  region: us-west-2
  version: "1.36"

autoModeConfig:
  enabled: true

Cluster Configuration Reference

This reference pins the cluster name, region, Kubernetes version, and Auto Mode setting.

Compare your unmanaged Pod manifest with this complete file:

apiVersion: v1
kind: Pod
metadata:
  name: neon-arena
  labels:
    app: neon-arena
spec:
  nodeSelector:
    eks.amazonaws.com/compute-type: auto
  containers:
    - name: arena
      image: IMAGE_URI_PLACEHOLDER:v1
      ports:
        - name: http
          containerPort: 8080
      resources:
        requests:
          cpu: "100m"
          memory: "128Mi"
        limits:
          cpu: "500m"
          memory: "256Mi"

Pod Manifest Reference

This reference defines one unmanaged v1 Pod on Auto Mode compute.

Compare your staged Deployment manifest with this complete file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: neon-arena
spec:
  replicas: 3
  progressDeadlineSeconds: 60
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  selector:
    matchLabels:
      app: neon-arena
  template:
    metadata:
      labels:
        app: neon-arena
    spec:
      nodeSelector:
        eks.amazonaws.com/compute-type: auto
      containers:
        - name: arena
          image: IMAGE_URI_PLACEHOLDER:v2
          ports:
            - name: http
              containerPort: 8080
          resources:
            requests:
              cpu: "100m"
              memory: "128Mi"
            limits:
              cpu: "500m"
              memory: "256Mi"

Deployment Manifest Reference

This reference defines three replicas with a rolling-update strategy and the v2 image placeholder.

Compare your public Service manifest with this complete file:

apiVersion: v1
kind: Service
metadata:
  name: neon-arena
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-scheme: internet-facing
spec:
  type: LoadBalancer
  loadBalancerClass: eks.amazonaws.com/nlb
  selector:
    app: neon-arena
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 8080

Service Manifest Reference

This reference selects the arena Pod and exposes it through an internet-facing Network Load Balancer.

  • Create the public Service from k8s/service.yaml by running this command:
kubectl apply -f k8s/service.yaml

What does this command do?

The command submits the saved Service definition to the Kubernetes API. EKS Auto Mode begins provisioning the matching internet-facing load balancer.

Before you run the final check, what new address do you expect the Service to gain?

  • Watch the Service until AWS assigns its external hostname by running:
kubectl get service neon-arena -w

What does the Service watch show?

Load-balancer provisioning can take several minutes. The watch updates when AWS publishes the public hostname to the Service status.

That hostname proves the Network Load Balancer exists. It also gives your browser a route to the arena Pod.

  • Stop the watch after the public hostname appears by pressing Ctrl+C.
  • Record the public hostname here: your Network Load Balancer hostname.
  • Return to the browser from your local container test.
  • Enter http:// followed by your Network Load Balancer hostname in the address bar.

You should see the neon KubeQuest page showing release v1. The serving Pod hostname appears as neon-arena.

Public page not loading?

Wait until the Service has a public hostname before opening the URL. A hostname that is still being provisioned cannot serve the page.

Confirm that the neon-arena Pod remains in its running state. Check that the Service selector matches app=neon-arena.

Ask for help tracing traffic from the load balancer to the Pod.

Your arena is now public through Amazon EKS. Next, you will test what this single-Pod design can survive.

Trigger the Failure

KubeQuest v1 is live on your Amazon EKS cluster. The public page proves that your image can run in the cloud.

The current application runs as one unmanaged Pod. No controller holds a desired state replica target for Kubernetes to maintain.

In this step, get ready to:
  • Capture the working public hostname.
  • Delete the unmanaged Pod.
  • Confirm the public endpoint loses its application backend.
Record the working endpoint

A controlled incident needs a healthy baseline. The current hostname gives you one address to test before the deletion and after it.

  • Return to the browser tab from the previous step.
  • Confirm that KubeQuest shows release v1 served by Pod neon-arena.
  • Record the hostname here: your public Network Load Balancer hostname.
  • Confirm the cluster has one application Pod by running:
kubectl get pods

What Does This Command Prove?

  • The kubectl client asks your current cluster for its Pods.
  • The neon-arena row establishes the healthy starting point for this incident.

You should see one row for neon-arena. This confirms that the public endpoint has an application backend before the incident.

Missing the Starting Pod?

  • Return to the launch step if neon-arena is missing.
  • Restore the v1 Pod before continuing.

Help me restore the missing unmanaged Pod before I run the incident test.

Delete the unmanaged Pod

Deleting this Pod is intentionally destructive. Your local project files remain untouched.

The cluster remains active. The public Service also remains active.

Before you run this, make a quick prediction about whether Kubernetes will create a replacement.

  • Delete the unmanaged Pod by running:
kubectl delete pod neon-arena

What Did the Deletion Change?

  • The command removes the named Pod object from the cluster.
  • It leaves k8s/pod.yaml unchanged on your workstation.
  • It leaves the public Service active.
  • Watch the cluster for Pod changes by running:
kubectl get pods -w

How Does Watch Mode Help?

  • The -w flag keeps the request open for new Pod events.
  • Any replacement Pod would appear as a new row.
  • Watch for about one minute.
  • Stop the watch by pressing Ctrl+C.

No replacement Pod appears during the watch. This is the intended shortfall in the unmanaged workload.

Did a Replacement Appear?

A replacement means your cluster state includes a controller or another process that recreated the Pod. That state falls outside this incident's starting point.

Help me identify what recreated the neon-arena Pod.

Confirm the public outage

The public hostname still belongs to the Service. Its selector needs a matching Pod before it can route KubeQuest traffic.

  • Confirm the final Pod state by running:
kubectl get pods

What Does This Final Check Prove?

  • This fresh list confirms the cluster state after watch mode ends.
  • No neon-arena row means the Service has no matching application backend.

You should see no row for neon-arena. Kubernetes has not recreated the unmanaged Pod.

Before the final check, predict whether the recorded hostname can serve KubeQuest without a Pod behind it.

  • Switch back to the browser tab for your public Network Load Balancer hostname.
  • Refresh the page.

The browser fails to load KubeQuest. The public hostname remains available as an address with no application backend.

That failed refresh is the result you were looking for. Your demo now exposes the availability gap in an unmanaged Pod.

Why Did the Hostname Survive?

The Kubernetes Service still exists. The Network Load Balancer also keeps its public hostname.

The Service selector can route traffic only to matching ready Pods. With no matching Pod, the load balancer has nowhere to send KubeQuest traffic.

Does KubeQuest Still Load?

  • Confirm that the address bar uses your public Network Load Balancer hostname.
  • Refresh the page after the Pod list shows no neon-arena row.

Help me diagnose why KubeQuest still loads after its only Pod was deleted.

You have reproduced a real availability failure without changing any local files. Next, you will introduce a three-replica Deployment that restores KubeQuest and replaces deleted Pods.

Recover, Self-Heal, and Roll Forward

Your unmanaged Pod is gone. The public arena is down because no controller owns a desired replica count.

A Deployment closes that gap by reconciling three replicas. The same desired-state loop replaces Pods gradually during a rolling update.

In this step, get ready to:
  • Restore KubeQuest v1 with three Deployment-managed replicas.
  • Delete one managed Pod to prove that Kubernetes replaces it.
  • Publish KubeQuest v2 to Amazon ECR before completing a rolling release.

Why use a Deployment?

The unmanaged Pod demonstrated that a single workload has no replacement mechanism. A Deployment continuously compares the running Pods with the replica count in its manifest.

This approach keeps the lesson focused on recovery plus releases. Direct control over individual Pods cannot provide that workflow.

Restore KubeQuest with three replicas

The existing Service still routes to Pods carrying the app=neon-arena label. A three-replica Deployment restores those matching backends.

  • In the Explorer sidebar in Visual Studio Code, select k8s/deployment.yaml.
  • Change the image tag in k8s/deployment.yaml from v2 to v1.
  • Save k8s/deployment.yaml.
  • Render the current repository URI into the manifest stream by running:
sed "s|IMAGE_URI_PLACEHOLDER|$REPO_URI|" k8s/deployment.yaml | kubectl apply -f -

What does this command do?

  • The sed substitution replaces IMAGE_URI_PLACEHOLDER only in the streamed manifest.
  • The saved file keeps its reusable placeholder.
  • The kubectl apply -f - portion sends the rendered manifest from standard input to the cluster.

Before you run the next check, decide whether you expect all three replicas to become available.

  • Wait for the Deployment before listing its Pods by running:
kubectl rollout status deployment/neon-arena
kubectl get pods -l app=neon-arena

What should this prove?

  • The rollout status command waits for the latest Deployment revision to finish.
  • The label selector limits the Pod list to KubeQuest replicas.
  • Three available Pods prove that the Deployment restored its declared replica count.

You should see the rollout finish. The Pod list should contain three running entries with generated names beginning with neon-arena-.

  • Return to the recorded public KubeQuest endpoint from the previous step.
  • Refresh the page.

Your arena is serving release v1 again. The displayed hostname now belongs to a Deployment-managed Pod.

Deployment not becoming available?

Confirm that k8s/deployment.yaml temporarily uses the v1 tag. Confirm that REPO_URI still contains the ECR repository URI from earlier.

Check that the Deployment labels remain app: neon-arena. The existing Service depends on that exact label.

Help me diagnose the unavailable KubeQuest Deployment.

Prove the Deployment self-heals

The Deployment now owns a desired state of three replicas. Deleting one managed Pod tests whether its controller restores that count.

  • List the Deployment-managed Pods by running:
kubectl get pods -l app=neon-arena

Why use the label selector?

The app=neon-arena selector returns only the Pods managed for this application. It keeps unrelated cluster workloads out of the test.

  • Record one Pod name here: one neon-arena Pod name.

Before you delete it, predict whether a differently named Pod will appear.

  • Delete the selected Pod before watching the replacement by running:
kubectl delete pod [[POD_NAME="one neon-arena Pod name"]]
kubectl get pods -w

What happens during self-healing?

  • The first command removes the selected Pod by name.
  • The watch command streams later Pod state changes.
  • The Deployment controller detects a replica deficit before creating a replacement Pod.

You should see the deleted name disappear. A new neon-arena- name should progress to a running state while the replica count returns to three.

  • Stop the watch by pressing Ctrl+C.
  • Refresh the public KubeQuest endpoint several times.

The page should remain available on release v1. Its Pod hostname may change as requests reach different replicas.

No replacement Pod appearing?

Confirm that the deleted name begins with neon-arena-. The original unmanaged Pod used the fixed name neon-arena.

Confirm that the Deployment still declares three replicas. A lower desired count changes what the controller restores.

Help me determine why the Deployment did not replace my deleted Pod.

Ship KubeQuest v2 with a rolling update

A new container image gives the Deployment a new Pod template to reconcile. Docker packages the redesigned release before Kubernetes replaces the replicas gradually.

  • In app.py, replace the two release styling lines with this block:
RELEASE = "v2"
ACCENT = "#a78bfa"

What changes in v2?

  • The RELEASE value changes the version displayed on the KubeQuest page.
  • The ACCENT value switches the interface to the v2 purple accent.
  • The readiness behavior remains healthy through READINESS_OK = True.
  • Save app.py.
  • Build the local v2 image by running:
sudo docker build -t kubequest-arena:v2 .

What does the build create?

Docker reads the existing Dockerfile from the current project folder. The -t option names the resulting image kubequest-arena:v2.

You should see the build finish successfully. The local image now contains the v2 application source.

  • Publish the v2 image to the existing ECR repository by running:
sudo docker tag kubequest-arena:v2 "$REPO_URI:v2"
sudo docker push "$REPO_URI:v2"

How does the image reach ECR?

  • The tag command adds the full private registry name to the local v2 image.
  • The push command uploads that tagged image to the existing repository.
  • The completed push prints an image digest that identifies the published artifact.

Image build or push failing?

Confirm that you saved app.py before building. Confirm that REPO_URI still points to the kubequest-arena repository.

Repeat the Amazon ECR authentication flow from Step 2 if the registry rejects the push.

Help me debug the KubeQuest v2 Docker build or ECR push.

  • In k8s/deployment.yaml, change the temporary image tag back to v2 so the line matches this reference:
          image: IMAGE_URI_PLACEHOLDER:v2

Why does this trigger a rollout?

The image belongs to the Deployment Pod template. Changing that template makes Kubernetes create a new revision.

The rolling strategy permits one extra Pod through maxSurge: 1. The maxUnavailable: 0 setting preserves the current available replica count during replacement.

  • Save k8s/deployment.yaml.

✔️ Awesome, I've got everything!

Great. Confirm that both updated files are saved before deploying v2.

ⓧ I'd like to double check the full code

Compare your finished app.py with this complete v2 file.

from html import escape
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
from socket import gethostname

PORT = 8080
RELEASE = "v2"
ACCENT = "#a78bfa"
READINESS_OK = True


class ArenaHandler(BaseHTTPRequestHandler):
    def do_GET(self):
        path = self.path.split("?", 1)[0]

        if path == "/ready":
            status = 200 if READINESS_OK else 503
            message = "ready" if READINESS_OK else "not ready"
            self.send_page(message, status, "text/plain; charset=utf-8")
            return

        if path not in ("/", "/index.html"):
            self.send_page("Not found", 404, "text/plain; charset=utf-8")
            return

        pod_name = escape(gethostname())
        body = f"""<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <title>KubeQuest Neon Incident Arena</title>
  <style>
    :root {{ --accent: {ACCENT}; }}
    * {{ box-sizing: border-box; }}
    body {{
      min-height: 100vh;
      margin: 0;
      display: grid;
      place-items: center;
      color: #f8fafc;
      background: radial-gradient(circle at top, #312e81, #020617 58%);
      font-family: ui-monospace, SFMono-Regular, Menlo, monospace;
    }}
    main {{
      width: min(760px, 92vw);
      padding: 2.5rem;
      border: 1px solid var(--accent);
      border-radius: 24px;
      background: rgba(15, 23, 42, 0.88);
      box-shadow: 0 0 55px color-mix(in srgb, var(--accent) 45%, transparent);
    }}
    .status {{ color: var(--accent); letter-spacing: 0.18em; text-transform: uppercase; }}
    h1 {{ font-size: clamp(2rem, 7vw, 4.5rem); margin: 0.4rem 0 1.5rem; }}
    .panel {{ padding: 1rem; margin: 1rem 0; border-radius: 14px; background: #0f172a; }}
    code {{ color: var(--accent); overflow-wrap: anywhere; }}
    button {{
      width: 100%; padding: 1rem; margin-top: 1rem; border: 0; border-radius: 12px;
      color: #020617; background: var(--accent); font: inherit; font-weight: 800; cursor: pointer;
    }}
  </style>
</head>
<body>
  <main>
    <div class="status">System online</div>
    <h1>KubeQuest</h1>
    <p>Neon Incident Arena release <code>{RELEASE}</code></p>
    <div class="panel">Request served by Pod <code>{pod_name}</code></div>
    <p>Refresh repeatedly while replicas change or a rollout is running.</p>
    <button onclick="window.location.reload()">Refresh Arena</button>
  </main>
</body>
</html>"""
        self.send_page(body, 200, "text/html; charset=utf-8")

    def send_page(self, body, status, content_type):
        payload = body.encode("utf-8")
        self.send_response(status)
        self.send_header("Content-Type", content_type)
        self.send_header("Content-Length", str(len(payload)))
        self.send_header("Cache-Control", "no-store")
        self.end_headers()
        self.wfile.write(payload)


if __name__ == "__main__":
    print(f"KubeQuest {RELEASE} listening on port {PORT}", flush=True)
    ThreadingHTTPServer(("0.0.0.0", PORT), ArenaHandler).serve_forever()

What should you compare?

Confirm that RELEASE is v2. Confirm that ACCENT is #a78bfa.

Keep READINESS_OK set to True for this healthy release.

Compare your finished k8s/deployment.yaml with this complete manifest.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: neon-arena
spec:
  replicas: 3
  progressDeadlineSeconds: 60
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  selector:
    matchLabels:
      app: neon-arena
  template:
    metadata:
      labels:
        app: neon-arena
    spec:
      nodeSelector:
        eks.amazonaws.com/compute-type: auto
      containers:
        - name: arena
          image: IMAGE_URI_PLACEHOLDER:v2
          ports:
            - name: http
              containerPort: 8080
          resources:
            requests:
              cpu: "100m"
              memory: "128Mi"
            limits:
              cpu: "500m"
              memory: "256Mi"

What should you compare?

Confirm that the manifest declares three replicas. Confirm that its selector matches the Pod template label.

Keep the reusable IMAGE_URI_PLACEHOLDER:v2 value in the saved file. The deployment command renders the real ECR URI in the stream.

  • Apply the v2 Deployment revision by running:
sed "s|IMAGE_URI_PLACEHOLDER|$REPO_URI|" k8s/deployment.yaml | kubectl apply -f -

What changes in the cluster?

The applied Pod template now references the published v2 image. Kubernetes creates a new ReplicaSet for this revision.

The rolling strategy adds new replicas gradually. Available old replicas remain until replacements are ready.

Before you check, predict whether the rollout reaches v2 without taking the public page offline.

  • Monitor the v2 release by running:
kubectl rollout status deployment/neon-arena

What does rollout status confirm?

The command watches the latest Deployment revision until it finishes. Completion means the Deployment has reached its updated desired state.

  • Refresh the public KubeQuest endpoint.
  • Press Refresh Arena several times.

You should see release v2 with the purple accent. The page should report hostnames created by the three-replica Deployment.

Rollout not reaching v2?

Confirm that the ECR push completed with the v2 tag. Confirm that the saved Deployment manifest also uses IMAGE_URI_PLACEHOLDER:v2.

Check that the rendered repository URI belongs to the same AWS account and region as the cluster.

Help me diagnose the stalled KubeQuest v2 rollout.

That completes the recovery loop. KubeQuest now restores deleted replicas while rolling forward to a new release without losing its public endpoint.

Secret mission

Block and Roll Back a Broken Release

Ship a deliberately unhealthy v3 release behind a readiness gate. You will watch the rollout stall while healthy v2 Pods keep serving traffic. Then you will inspect the failure and roll back safely.

Clean Up Your Resources

Clean Up Your Resources

Your KubeQuest resources are still active in AWS, so charges continue until you pause or delete them. Decide whether to keep the live arena, pause its public workload, or remove the project resources entirely.

Cost warning

The Amazon EKS standard-support control plane costs $0.10 per cluster-hour.

Amazon EKS Auto Mode compute adds EC2 charges. Its management fee is separate.

The Network Load Balancer can add charges. Amazon ECR storage can add charges.

Public IPv4 addresses can add charges. Data transfer can add charges. AWS credits do not stop these resources automatically.

Resources you used:

  • The kubequest-arena Amazon EKS cluster running Kubernetes 1.36.
  • Amazon EKS Auto Mode compute plus the cluster's VPC and IAM roles.
  • The kubequest-arena ECR repository containing ${REPO_URI}:v1, ${REPO_URI}:v2, and ${REPO_URI}:v3.
  • The internet-facing Network Load Balancer created for the neon-arena Service.
  • The Kubernetes Service, Deployment, ReplicaSets, and Pods in the cluster.
  • The local app.py, Dockerfile, cluster-config.yaml, and k8s artifacts.

Keep everything running

No action needed. Choose this if you want to keep demonstrating the public KubeQuest arena or continue experimenting with releases.

  • Keep the kubequest-arena cluster active to preserve the Kubernetes environment.
  • Keep the neon-arena Service active to preserve the public endpoint.
  • Retain the kubequest-arena ECR repository to preserve all three image versions.
  • Preserve the local project files for future release experiments.

Keep monitoring costs

The control plane, load balancer, registry storage, and supporting infrastructure continue generating charges while they remain active.

Pause - I'll come back to this later

Pausing removes the public workload while preserving the cluster, ECR images, and local files. The EKS control plane continues generating charges.

  • Delete the public neon-arena Service by running this command:
kubectl delete service neon-arena

What the Service deletion removes

Deleting the Service starts cleanup of its internet-facing Network Load Balancer. This removes the public KubeQuest endpoint.

  • Check the AWS Management Console load balancer list in us-west-2 until the load balancer created for neon-arena disappears.
  • Delete the neon-arena Deployment by running this command:
kubectl delete deployment neon-arena

What the Deployment deletion removes

Deleting the Deployment removes its managed ReplicaSets. Kubernetes also removes their Pods.

  • Confirm that no KubeQuest workload Pods remain by running:
kubectl get pods

What should I see?

You should see no Pods belonging to the neon-arena workload. The cluster and ECR repository remain available for your return.

The retained EKS control plane still incurs charges. The retained ECR images can also incur storage charges.

Delete - I don't want to use this again

This option permanently removes the cloud resources and local artifacts listed above. Your public KubeQuest endpoint stops working after the Service is deleted.

  • Delete the public neon-arena Service by running this command:
kubectl delete service neon-arena

Why the Service goes first

Deleting the Service lets Amazon EKS Auto Mode remove the associated Network Load Balancer before the cluster disappears. This avoids leaving the load balancer behind.

  • Check the AWS Management Console load balancer list in us-west-2 until the load balancer created for neon-arena disappears.

Cluster deletion has several parts. A busy terminal is expected while AWS removes the cluster infrastructure.

  • Delete the kubequest-arena cluster by running:
eksctl delete cluster -f cluster-config.yaml --wait

What the cluster command removes

The command removes the eksctl-managed cluster infrastructure. The --wait flag keeps the command active so deletion errors are reported.

A successful deletion returns you to the shell prompt without a cluster deletion error.

  • Delete the kubequest-arena ECR repository by running:
aws ecr delete-repository --repository-name kubequest-arena --force --region us-west-2

What the ECR command removes

The --force flag deletes the repository even though it contains images. This removes the stored v1, v2, and v3 releases.

  • Return to the local directory that contains cluster-config.yaml.
  • Remove the known project artifacts by running:
rm -rf app.py Dockerfile cluster-config.yaml k8s

What the local command removes

This removes the Python application, container definition, cluster configuration, and Kubernetes manifests from the current directory.

  • Confirm that the local artifacts are gone by running:
ls

What should I see?

You should no longer see app.py, Dockerfile, cluster-config.yaml, or the k8s directory in the output.

Nice Work!

Nice Work!

You did it! KubeQuest now runs as a public resilience demo on Amazon Elastic Kubernetes Service (Amazon EKS).

You've learned how to:

  • Build a custom Python web application as a pinned container image. Publish versioned images to Amazon Elastic Container Registry (Amazon ECR). Expose KubeQuest publicly through an internet-facing Network Load Balancer.
  • Reproduce the outage caused by deleting an unmanaged Pod. Restore the arena with a three-replica Deployment that replaces deleted Pods automatically.
  • Release v2 with a rolling update while the public endpoint stays available. Verify the new release across Deployment-managed Pod hostnames.
  • Complete the Secret Mission. Add an HTTP readiness probe that blocks broken v3 Pods from Service traffic. Roll the Deployment back to healthy v2.

Ready to quiz yourself?