Deploy a Production App on AWS EKS

Introduction

30 Second Summary

Every time you stream a show, order food, or send a message, your request lands on infrastructure that scales, heals, and secures itself without anyone clicking buttons. That infrastructure runs on Kubernetes.

In this project, you will build a production-grade Kubernetes cluster on AWS EKS that runs a multi-tier web application with automated DNS, TLS certificates, autoscaling, and full monitoring.

What You'll Build

A complete production Kubernetes environment on AWS EKS featuring automated infrastructure, a custom Helm-deployed application, and a Prometheus + Grafana observability stack.

By the end of this project, you'll have:

• A 2-node EKS cluster with persistent storage, load balancing, RBAC, and IAM integration running a multi-tier application (nginx frontend + httpbin API + PostgreSQL database).
• Automated DNS and TLS management using External-DNS and cert-manager with a real domain and Let's Encrypt certificates.
• A full Prometheus + Grafana monitoring stack with pre-built Kubernetes dashboards and a Horizontal Pod Autoscaler responding to live traffic.
• Secret Mission: Load-test your application and watch autoscaling respond in real-time through Grafana dashboards.

Are there any prerequisites?

You need an AWS account with billing enabled. You should also have a basic understanding of Kubernetes concepts like pods, deployments, services, and namespaces.

This project assumes you already have the AWS CLI, kubectl, eksctl, and Helm installed on a Linux environment. The estimated cost is $3-5 if you tear down resources promptly after completion.

Create the EKS Cluster

Every production Kubernetes deployment starts with a cluster. The cluster is the foundation that hosts your application, networking, storage, and monitoring components.

In this step, you'll use eksctl to provision a 2-node EKS cluster on AWS. You'll also enable an OIDC provider, which is how AWS securely grants individual pods the exact IAM permissions they need.

In this step, get ready to:

• Create an eksctl cluster configuration file.
• Provision the EKS cluster with 2 t3.medium nodes.
• Associate the IAM OIDC provider for service account integration.

Create the cluster configuration file

Instead of passing dozens of flags to a single command, eksctl lets you define your entire cluster in a YAML configuration file. This makes your infrastructure reproducible and version-controllable.

• Create the cluster configuration file by running this command:

cat <<'EOF' > ~/cluster.yaml
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: prod-cluster
  region: us-east-1
  version: "1.29"

iam:
  withOIDC: true

managedNodeGroups:
  - name: managed-ng
    instanceType: t3.medium
    minSize: 2
    maxSize: 4
    desiredCapacity: 2
    volumeSize: 20
    iam:
      attachPolicyARNs:
        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy
EOF

What does this configuration do?

• metadata sets the cluster name to prod-cluster in the us-east-1 region running Kubernetes 1.29.
• withOIDC: true enables the OIDC identity provider. This is what allows Kubernetes service accounts to assume IAM roles later.
• managedNodeGroups defines a node group of 2 t3.medium instances (4 GiB RAM each). The maxSize: 4 allows the cluster to scale up if needed.
• The three attached IAM policies give nodes permission to join the cluster, pull container images, and manage pod networking.

Why t3.medium instead of t3.small?

The monitoring stack (Prometheus + Grafana) alone needs around 2GB of RAM. Each t3.medium instance provides 4 GiB, giving enough headroom for your application and monitoring on 2 nodes. Using t3.small (2 GiB) would cause out-of-memory issues once the full stack is running.

• Confirm the file was created by running:

cat ~/cluster.yaml

You should see the full YAML configuration printed to your terminal.

✔️ Awesome, I've got everything!

Great. Your cluster configuration is ready to go.

ⓧ I'd like to double check the full code

apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig

metadata:
  name: prod-cluster
  region: us-east-1
  version: "1.29"

iam:
  withOIDC: true

managedNodeGroups:
  - name: managed-ng
    instanceType: t3.medium
    minSize: 2
    maxSize: 4
    desiredCapacity: 2
    volumeSize: 20
    iam:
      attachPolicyARNs:
        - arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy
        - arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
        - arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy

Provision the EKS cluster

With the configuration file ready, eksctl can now provision the entire cluster. This includes creating a VPC, subnets, security groups, the EKS control plane, and your worker nodes.

• Create your EKS cluster by running this command:

eksctl create cluster -f ~/cluster.yaml

This takes 15-20 minutes

Cluster provisioning involves creating many AWS resources behind the scenes. You'll see progress messages in your terminal as each component is created.

While you wait, you can move ahead to Step 2 and set up your Route53 hosted zone. DNS propagation also takes time, so starting that process now is efficient.

Once the command completes, eksctl automatically updates your kubectl context to point at the new cluster.

• Verify your cluster is running and your nodes are ready by running:

kubectl get nodes

What should I see?

You should see 2 nodes listed, both with a STATUS of Ready. The node names will be internal AWS hostnames like ip-192-168-x-x.ec2.internal.

Cluster creation failed or nodes not Ready?

• If eksctl reports an error about insufficient permissions, verify your AWS CLI credentials have AdministratorAccess or equivalent permissions for EKS, EC2, IAM, CloudFormation, and VPC.
• If nodes show NotReady status, wait 1-2 minutes. Nodes need time to register with the control plane after provisioning.
• If you see a region error, check that your AWS CLI default region matches us-east-1 or that the region in cluster.yaml is correct.

Why is my EKS cluster creation failing?

Associate the IAM OIDC provider

Throughout this project, you'll install controllers (like the Load Balancer Controller and EBS CSI Driver) that need specific AWS permissions. IRSA (IAM Roles for Service Accounts) lets each controller assume only the permissions it needs, rather than granting broad permissions to every pod on the node.

Your cluster config already enabled OIDC during creation. Running the association command explicitly confirms it's active and ready for IRSA.

• Associate the IAM OIDC provider with your cluster by running:

eksctl utils associate-iam-oidc-provider --region us-east-1 --cluster prod-cluster --approve

What does this command do?

This command creates a trust relationship between your EKS cluster and IAM. Once established, Kubernetes service accounts can assume IAM roles. Each add-on you install in later steps will get its own IAM role with only the permissions it needs.

You should see a message confirming the OIDC provider is associated, or a note that it already exists (since your cluster config enabled it).

Seeing an access denied error?

• Confirm your AWS CLI user has the iam:CreateOpenIDConnectProvider permission.
• Double check the cluster name and region match exactly: prod-cluster in us-east-1.

Help me fix my OIDC provider association error.

Your EKS cluster is live with 2 healthy nodes and IRSA ready to go. Next up, you'll configure Route53 and point your domain's nameservers to AWS so that your application can be reached over the internet.

Configure Route53 and Update Porkbun Nameservers

Your cluster is running, but it's not reachable from the internet yet. For users to access your application (and for TLS certificates to work later), you need a domain name that points to your cluster's load balancer.

In this step, you'll create a Route53 hosted zone that manages DNS records for your domain. Then you'll update your Porkbun domain registrar to delegate DNS authority to Route53.

In this step, get ready to:

• Create a Route53 public hosted zone for your domain.
• Retrieve the four NS records from the hosted zone.
• Update your Porkbun domain nameservers to point to Route53.

Create a Route53 public hosted zone

A hosted zone is a container for DNS records. When you create one in Route53, AWS automatically generates nameserver (NS) records that tell the internet where to look up your domain.

• Create the hosted zone for your domain by running this command:

aws route53 create-hosted-zone \
  --caller-reference $(uuidgen) \
  --name [[DOMAIN="your-domain.com"]]

What does this command do?

• --caller-reference $(uuidgen) generates a unique ID to prevent duplicate hosted zones if you accidentally run the command twice.
• --name specifies your domain name (e.g., example.com). Use your root domain without www.

The command outputs a JSON response containing your hosted zone details. Look for the HostedZone.Id field in the output.

• Copy the hosted zone ID from the output. It looks like /hostedzone/Z1234567890ABC. You only need the part after /hostedzone/ (e.g., Z1234567890ABC). Save this value for the next substep.

What is a hosted zone?

A Route53 hosted zone is like a phone book for your domain. It stores DNS records that map human-readable names (like app.example.com) to IP addresses or load balancer endpoints.

Later in this project, External-DNS will automatically create records in this hosted zone whenever you deploy new Ingress resources.

Seeing an access denied error?

• Make sure your AWS CLI credentials have Route53 permissions. Your IAM user or role needs route53:CreateHostedZone permission.
• Verify your AWS CLI is configured by running aws sts get-caller-identity. You should see your account ID and user ARN.

help me troubleshoot Route53 access denied errors with my AWS CLI

Retrieve the NS records from the hosted zone

Route53 automatically created four NS (nameserver) records when the hosted zone was provisioned. You need these values to configure Porkbun.

• Query the NS records from your hosted zone by running this command:

aws route53 list-resource-record-sets \
  --hosted-zone-id [[HOSTED_ZONE_ID="your-hosted-zone-id"]] \
  --query "ResourceRecordSets[?Type == 'NS'].ResourceRecords[].Value" \
  --output text

What does this command do?

• --hosted-zone-id targets the specific hosted zone you just created.
• --query uses JMESPath to filter only the NS record values from all the records in the zone.
• --output text formats the output as plain text, making it easy to copy.

• You should see four nameserver addresses in the output, each ending with a trailing dot. They look something like:

ns-123.awsdns-45.com.
ns-678.awsdns-90.net.
ns-1234.awsdns-12.co.uk.
ns-567.awsdns-34.org.

• Copy all four nameserver values. You will enter these in Porkbun in the next substep.

Remove the trailing dots

When entering nameservers in Porkbun, remove the trailing dot from each value. For example, ns-123.awsdns-45.com. becomes ns-123.awsdns-45.com.

Getting an empty result or error?

• Double-check the hosted zone ID. It should start with Z and contain only uppercase letters and numbers.
• Make sure you're using the zone ID (e.g., Z1234567890ABC), not the full path (/hostedzone/Z1234567890ABC).

help me retrieve NS records from my Route53 hosted zone

Update your Porkbun domain nameservers

Right now, Porkbun handles DNS for your domain using its own nameservers. You need to replace them with the Route53 nameservers so that AWS controls your domain's DNS records.

• Open your browser and go to porkbun.com.
• Sign in to your Porkbun account.
• Click Domain Management in the top navigation.
• Find your domain in the list and click the Details dropdown (or the domain name itself) to expand its settings.
• Scroll down to the Nameservers section.
• Delete the existing Porkbun nameservers.
• Enter the four Route53 nameservers you copied earlier, one per field. Remember to remove the trailing dots.
• Click Submit to save the new nameservers.

Why change nameservers?

By pointing your domain's nameservers to Route53, you're delegating DNS authority to AWS. This means Route53 becomes the source of truth for all DNS records under your domain.

Later, External-DNS and cert-manager will automatically create and validate records in Route53. They can only do this if Route53 is authoritative for your domain.

After saving, verify that DNS propagation has started by running this command:

• Check whether Route53 nameservers are being returned for your domain by running:

dig NS [[DOMAIN="your-domain.com"]]

• Look for your Route53 nameservers (awsdns addresses) in the ANSWER section of the output.

What if I still see the old nameservers?

DNS propagation can take anywhere from 15 minutes to 48 hours. For this project, it typically resolves within 15-30 minutes. You can continue to the next step while waiting.

The critical point where propagation must be complete is Step 9, when cert-manager uses DNS-01 validation to issue TLS certificates. If dig NS still shows old nameservers by then, wait a bit longer before creating your ClusterIssuer.

dig command not found?

• Install the dnsutils package. On Ubuntu/Debian, run sudo apt install dnsutils.
• Alternatively, use nslookup -type=NS your-domain.com as a substitute.

help me verify DNS propagation for my domain

Hosted zone cost reminder

Route53 charges $0.50 per hosted zone per month (prorated). Remember to delete this hosted zone during teardown in Step 11 to stop charges.

Your domain is now configured to use Route53 for DNS. While propagation continues in the background, you'll install the AWS Load Balancer Controller that provisions the actual load balancer your domain will point to.

Install AWS Load Balancer Controller

Your cluster is running and your domain's DNS is pointed at Route53. The next challenge is getting internet traffic into the cluster.

Kubernetes uses Ingress resources to define how external HTTP traffic reaches your pods. The AWS Load Balancer Controller watches for these Ingress resources and automatically provisions an ALB (Application Load Balancer) to route traffic into your cluster.

In this step, get ready to:

• Create an IAM policy and IRSA service account for the Load Balancer Controller.
• Install the AWS Load Balancer Controller via Helm.
• Verify the controller deployment is running in your cluster.

Create the IAM policy and IRSA service account

The Load Balancer Controller needs AWS permissions to create and manage ALBs on your behalf. Instead of granting these permissions to every pod on the node, you will use IRSA (IAM Roles for Service Accounts) to scope permissions to only the controller's service account.

Why IRSA instead of node IAM roles?

With node-level IAM roles, every pod on the node inherits the same broad permissions. IRSA provides least-privilege access by binding an IAM role to a specific Kubernetes service account.

Only pods using that service account receive the credentials. This is the AWS-recommended pattern for EKS add-ons.

• Download the IAM policy document for the AWS Load Balancer Controller by running this command:

curl -O https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.14.1/docs/install/iam_policy.json

• Create the IAM policy from the downloaded file by running:

aws iam create-policy \
  --policy-name AWSLoadBalancerControllerIAMPolicy \
  --policy-document file://iam_policy.json

What do these commands do?

The first command downloads a JSON file containing all the IAM permissions the Load Balancer Controller needs. These include permissions for elasticloadbalancing, ec2:Describe*, and related actions.

The second command creates an IAM policy named AWSLoadBalancerControllerIAMPolicy in your AWS account using that JSON document.

You should see JSON output containing the policy ARN. It looks like arn:aws:iam::123456789012:policy/AWSLoadBalancerControllerIAMPolicy.

Seeing EntityAlreadyExists?

This means the policy already exists in your AWS account from a previous attempt. You can safely continue to the next command. The existing policy has the same permissions.

help me troubleshoot creating the IAM policy for the AWS Load Balancer Controller

Now create the IRSA service account that binds this policy to a Kubernetes service account in your cluster.

• Create the IRSA service account by running this command:

eksctl create iamserviceaccount \
  --cluster=prod-cluster \
  --namespace=kube-system \
  --name=aws-load-balancer-controller \
  --role-name AmazonEKS_LB_Controller_Role \
  --attach-policy-arn=arn:aws:iam::[[AWS_ACCOUNT_ID="your 12-digit AWS account ID"]]:policy/AWSLoadBalancerControllerIAMPolicy \
  --approve

What does this command do?

• Creates an IAM role named AmazonEKS_LB_Controller_Role with a trust policy scoped to your cluster's OIDC provider.
• Attaches the AWSLoadBalancerControllerIAMPolicy to that role.
• Creates a Kubernetes service account named aws-load-balancer-controller in the kube-system namespace, annotated with the role ARN.

Find your AWS account ID

Run aws sts get-caller-identity --query Account --output text to print your 12-digit AWS account ID.

This command takes 1-2 minutes while CloudFormation creates the IAM resources. You should see output ending with a confirmation that the service account was created.

Command taking too long or erroring out?

Make sure you replaced the account ID placeholder with your actual 12-digit account ID. You can also check that your cluster's OIDC provider exists by running aws iam list-open-id-connect-providers and looking for an entry that matches your cluster.

help me fix eksctl create iamserviceaccount for the Load Balancer Controller

Install the AWS Load Balancer Controller via Helm

With the IAM permissions in place, you can now install the controller itself. Helm makes this a single command that deploys all the required Kubernetes resources (Deployment, Service, webhooks) into your cluster.

• Add the EKS Helm chart repository by running:

helm repo add eks https://aws.github.io/eks-charts
helm repo update eks

• Install the AWS Load Balancer Controller by running:

helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
  -n kube-system \
  --set clusterName=prod-cluster \
  --set serviceAccount.create=false \
  --set serviceAccount.name=aws-load-balancer-controller

What does this Helm install do?

• helm repo add eks registers the official AWS EKS charts repository so Helm can find the controller chart.
• --set serviceAccount.create=false tells the chart to use the IRSA service account you already created, instead of making a new one.
• --set clusterName=prod-cluster tells the controller which EKS cluster it manages.

You should see output confirming the Helm release was deployed successfully.

Seeing a ServiceAccount error?

If you see an error about the service account not existing, make sure the previous eksctl create iamserviceaccount command completed successfully. Verify the service account exists by running kubectl get sa aws-load-balancer-controller -n kube-system.

help me troubleshoot Helm install of the AWS Load Balancer Controller

Verify the controller is running

• Check that the controller deployment is running by running:

kubectl get deployment -n kube-system aws-load-balancer-controller

You should see output showing the deployment with 2/2 replicas in the READY column.

What should I see?

The output looks like this:

NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
aws-load-balancer-controller   2/2     2            2           60s

The 2/2 confirms both controller replicas are healthy and ready to process Ingress resources.

Seeing 0/2 or pods in CrashLoopBackOff?

Check the controller pod logs by running kubectl logs -n kube-system -l app.kubernetes.io/name=aws-load-balancer-controller. Common causes include the service account missing the IAM role annotation or the OIDC provider not matching your cluster.

help me fix the AWS Load Balancer Controller pods that are not starting

Your cluster now has a controller watching for Ingress resources. Whenever you create an Ingress, the controller automatically provisions an ALB and routes traffic into your pods. Next up, you will install the EBS CSI Driver so your cluster can provision persistent storage for databases.

Install EBS CSI Driver

Your cluster can now route internet traffic through an ALB thanks to the AWS Load Balancer Controller. The next challenge is persistent storage.

Kubernetes pods are ephemeral. When a pod restarts, any data stored inside it disappears. For a database like PostgreSQL, that means losing all your data on every restart.

The EBS CSI Driver solves this by letting Kubernetes automatically provision and attach EBS volumes to your pods. In this step, you will install the driver and configure a StorageClass so your database can survive restarts.

In this step, get ready to:

• Create an IAM role for the EBS CSI Driver with the correct policy attached.
• Install the EBS CSI Driver as an EKS managed add-on.
• Create a gp3 StorageClass and verify it works with a test PVC.

Create the IAM role for the EBS CSI Driver

Just like the Load Balancer Controller, the EBS CSI Driver needs AWS permissions to create and manage EBS volumes on your behalf. You will use IRSA again to grant least-privilege access.

• Create the IAM role with the managed EBS CSI policy attached by running this command:

eksctl create iamserviceaccount \
  --name ebs-csi-controller-sa \
  --namespace kube-system \
  --cluster prod-cluster \
  --role-name AmazonEKS_EBS_CSI_DriverRole \
  --role-only \
  --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicyV2 \
  --approve

What does this command do?

This creates an IAM role called AmazonEKS_EBS_CSI_DriverRole with a trust relationship that only allows the EBS CSI controller service account in kube-system to assume it.

• --role-only creates the IAM role without creating a Kubernetes service account. The EKS managed add-on will handle the service account itself.
• --attach-policy-arn attaches the AWS-managed AmazonEBSCSIDriverPolicyV2 policy, which grants permissions to create, attach, detach, and delete EBS volumes.

• Wait for the command to complete. This typically takes 1-2 minutes as eksctl creates a CloudFormation stack.
• You should see output ending with a line that confirms the role was created successfully.

Seeing a permission error?

• Make sure your AWS CLI credentials have IAM permissions to create roles and CloudFormation stacks.
• If you see a conflict error about the role already existing, you may have run this command before. You can proceed to the next substep.

Help me troubleshoot this eksctl IAM role creation error.

Install the EBS CSI Driver as an EKS managed add-on

AWS provides the EBS CSI Driver as an EKS managed add-on. This means AWS handles updates and security patches automatically, rather than you managing a Helm chart yourself.

• First, retrieve your AWS account ID (you will need it for the role ARN) by running:

ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
echo $ACCOUNT_ID

• You should see your 12-digit AWS account ID printed.
• Install the EBS CSI Driver add-on by running:

aws eks create-addon \
  --cluster-name prod-cluster \
  --addon-name aws-ebs-csi-driver \
  --service-account-role-arn arn:aws:iam::${ACCOUNT_ID}:role/AmazonEKS_EBS_CSI_DriverRole

What does this command do?

• --addon-name aws-ebs-csi-driver tells EKS to install the managed EBS CSI Driver component.
• --service-account-role-arn links the add-on to the IAM role you just created, so the driver can provision EBS volumes using those permissions.

• Wait about 1 minute for the add-on to install, then verify it is active by running:

aws eks describe-addon \
  --cluster-name prod-cluster \
  --addon-name aws-ebs-csi-driver \
  --query 'addon.status' \
  --output text

• You should see ACTIVE in the output. If you see CREATING, wait another minute and run the command again.

Add-on stuck in CREATING or showing DEGRADED?

• If the status stays at CREATING for more than 5 minutes, check that the IAM role exists with aws iam get-role --role-name AmazonEKS_EBS_CSI_DriverRole.
• If you see DEGRADED, the IAM role trust relationship may be incorrect. Delete the add-on with aws eks delete-addon --cluster-name prod-cluster --addon-name aws-ebs-csi-driver and re-run the create command.

Help me troubleshoot my EBS CSI Driver add-on installation.

Create a gp3 StorageClass and verify with a test PVC

The driver is running, but Kubernetes needs a StorageClass to know what type of EBS volume to create when a pod requests storage. You will create a gp3 StorageClass and set it as the cluster default.

• Create a file called gp3-storageclass.yaml by running:

cat <<EOF > gp3-storageclass.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: gp3
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: ebs.csi.aws.com
parameters:
  type: gp3
volumeBindingMode: WaitForFirstConsumer
reclaimPolicy: Delete
EOF

What does this StorageClass configure?

• provisioner: ebs.csi.aws.com tells Kubernetes to use the EBS CSI Driver for volume provisioning.
• type: gp3 requests General Purpose SSD volumes with baseline 3000 IOPS and 125 MB/s throughput.
• WaitForFirstConsumer delays volume creation until a pod actually needs it. This ensures the volume is created in the same availability zone as the pod.
• is-default-class: "true" makes this the default StorageClass. Any PVC that does not specify a StorageClass will use gp3 automatically.

• Apply the StorageClass to your cluster by running:

kubectl apply -f gp3-storageclass.yaml

• Verify the StorageClass was created and is marked as default by running:

kubectl get storageclass

• You should see gp3 (default) in the output with the provisioner listed as ebs.csi.aws.com.

Now verify the full flow works end-to-end by creating a test PVC and a pod that uses it.

• Create the test resources by running:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: test-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: gp3
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: test-pod
spec:
  containers:
    - name: test
      image: nginx:alpine
      volumeMounts:
        - mountPath: /data
          name: test-volume
  volumes:
    - name: test-volume
      persistentVolumeClaim:
        claimName: test-pvc
EOF

What does this test do?

The PVC requests 1Gi of storage from the gp3 StorageClass. The pod mounts that PVC at /data. Because the volumeBindingMode is WaitForFirstConsumer, the EBS volume only gets created once the pod is scheduled to a node.

• Wait about 30 seconds for the pod to start, then check if the PVC is bound by running:

kubectl get pvc test-pvc

• The STATUS column should show Bound. This confirms the EBS CSI Driver successfully provisioned a gp3 volume and attached it to your pod.

• Clean up the test resources by running:

kubectl delete pod test-pod
kubectl delete pvc test-pvc

• Both resources should be deleted. The EBS volume is also automatically removed because the reclaimPolicy is Delete.

PVC stuck in Pending?

• Check if the test pod is running with kubectl get pod test-pod. The PVC will not bind until the pod is scheduled.
• If the pod shows Pending as well, run kubectl describe pod test-pod and look at the Events section for scheduling issues.
• Verify the EBS CSI Driver pods are running with kubectl get pods -n kube-system -l app.kubernetes.io/name=aws-ebs-csi-driver.

Help me debug why my test PVC is not binding.

Your cluster now has persistent storage ready to go. Next up, you will configure RBAC to organize your workloads into namespaces and control who can do what inside each one.

Configure RBAC

Your cluster now has persistent storage and load balancing ready to go. The next step is controlling who can do what inside the cluster.

In production, you never want every pod or user to have full admin access. RBAC (Role-Based Access Control) lets you define fine-grained permissions. You'll create isolated namespaces, a restricted developer role, and a dedicated service account for your application workloads.

In this step, get ready to:

• Create namespaces for the application and monitoring.
• Create a developer Role and RoleBinding with limited permissions.
• Create a ServiceAccount for the application workloads.

Create namespaces for application and monitoring

Namespaces are logical partitions inside a Kubernetes cluster. They keep resources isolated so that your application pods, monitoring tools, and other workloads don't collide with each other.

• Create a file called rbac.yaml in your working directory by running:

touch rbac.yaml

• Open rbac.yaml in your text editor and add the following namespace definitions:

# Namespace for application workloads
apiVersion: v1
kind: Namespace
metadata:
  name: app
---
# Namespace for monitoring tools (Prometheus, Grafana)
apiVersion: v1
kind: Namespace
metadata:
  name: monitoring

What does this code do?

You're creating two namespaces that separate concerns in your cluster.

• app holds your frontend, API, and database workloads.
• monitoring holds Prometheus and Grafana, which you'll install later.

The --- separator tells Kubernetes that these are separate resource definitions within the same file.

• Save rbac.yaml.
• Apply the namespaces to your cluster by running:

kubectl apply -f rbac.yaml

• Verify the namespaces exist by running:

kubectl get namespaces

You should see app and monitoring in the list alongside the default Kubernetes namespaces.

Don't see the new namespaces?

• Check the output of kubectl apply for errors. A typo in the YAML (wrong indentation or missing colon) will cause a parse failure.
• Make sure you saved the file before running kubectl apply.

Help me debug why my namespaces aren't being created

Create a developer Role and RoleBinding

A Role defines what actions are allowed. A RoleBinding connects that Role to a user or group. Together, they enforce least-privilege access within a specific namespace.

• In rbac.yaml, add the following Role and RoleBinding below the monitoring namespace (after the last --- separator):

---
# Role granting read-only access to common resources in the app namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: developer
  namespace: app
rules:
  - apiGroups: [""]
    resources: ["pods", "services", "configmaps"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch"]
---
# Bind the developer Role to the 'developers' group
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: developer-binding
  namespace: app
subjects:
  - kind: Group
    name: developers
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: developer
  apiGroup: rbac.authorization.k8s.io

What does this code do?

The Role named developer grants read-only access (get, list, watch) to pods, services, configmaps, and deployments in the app namespace. It cannot create, update, or delete anything.

The RoleBinding attaches this Role to a group called developers. Any IAM user mapped to this group would inherit these limited permissions.

Why two separate apiGroups?

Kubernetes organizes resources into API groups. Core resources like pods, services, and configmaps live in the empty string group "". Deployments live in the apps group. You need a separate rule for each group.

• Save rbac.yaml and apply the updated file:

kubectl apply -f rbac.yaml

• Verify the Role and RoleBinding were created by running:

kubectl get role,rolebinding -n app

You should see developer listed under ROLE and developer-binding listed under ROLEBINDING.

Seeing a validation error?

• YAML is indentation-sensitive. Make sure rules: is indented with 2 spaces under spec, and each - apiGroups: item is indented consistently.
• Check that each --- separator is on its own line with no trailing spaces.

Help me fix my RBAC YAML validation error

Create a ServiceAccount for application workloads

A ServiceAccount gives your application pods an identity inside the cluster. When you deploy your app later, the pods will run under this account instead of the default one.

• In rbac.yaml, add the following ServiceAccount definition at the bottom of the file:

---
# ServiceAccount for application pods
apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-service-account
  namespace: app

What does this code do?

This creates a named ServiceAccount called app-service-account in the app namespace. Your Helm chart will reference this account when deploying pods, giving them a distinct identity separate from the default ServiceAccount.

• Save rbac.yaml and apply the final version:

kubectl apply -f rbac.yaml

• Verify the ServiceAccount was created by running:

kubectl get serviceaccount -n app

You should see app-service-account in the list alongside the default ServiceAccount that Kubernetes creates automatically.

Only seeing the default ServiceAccount?

• Make sure the namespace: app line is present in the ServiceAccount metadata. Without it, the resource goes into the default namespace.
• Confirm you saved the file before running kubectl apply.

Help me troubleshoot my ServiceAccount creation

✔️ Awesome, I've got everything!

Great. Double check you've saved rbac.yaml and all resources are showing up in your cluster.

ⓧ I'd like to double check the full code

Here's the complete rbac.yaml file for comparison:

apiVersion: v1
kind: Namespace
metadata:
  name: app
---
apiVersion: v1
kind: Namespace
metadata:
  name: monitoring
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: developer
  namespace: app
rules:
  - apiGroups: [""]
    resources: ["pods", "services", "configmaps"]
    verbs: ["get", "list", "watch"]
  - apiGroups: ["apps"]
    resources: ["deployments"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: developer-binding
  namespace: app
subjects:
  - kind: Group
    name: developers
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: developer
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: app-service-account
  namespace: app

Your cluster now has proper access controls in place. Next up, you'll deploy a multi-tier application into the app namespace using a custom Helm chart.

Deploy the Multi-Tier Application with Helm

Your cluster now has networking, storage, and access controls in place. The next challenge is deploying an actual application that uses all of these capabilities together.

In this step, you will build a custom Helm chart that packages a multi-tier application: an nginx frontend, an httpbin API, and a PostgreSQL database with persistent storage. Helm lets you template all of your Kubernetes manifests into a single deployable unit, so you can configure images, replicas, and resource limits from one values.yaml file.

In this step, get ready to:

• Create a custom Helm chart with templates for three application tiers.
• Configure an ALB Ingress resource for internet-facing traffic routing.
• Install the chart and verify all pods are running with an ALB provisioned.

Create the Helm chart and application templates

A Helm chart is a directory containing templated Kubernetes manifests and a values.yaml that centralizes configuration. Instead of writing separate YAML for each environment, you define templates once and swap values at deploy time.

• Create the chart directory structure by running these commands:

mkdir -p ~/myapp/templates
cd ~/myapp

• Confirm the directories exist by running:

ls

You should see a templates directory listed in the output.

Why build the chart from scratch?

The helm create command generates boilerplate, but it includes many files you do not need. Building the chart manually teaches you exactly how Helm connects templates to values.

• Create the chart metadata file by pasting the following into a new file called Chart.yaml in the ~/myapp directory:

apiVersion: v2
name: myapp
description: A multi-tier application Helm chart
type: application
version: 0.1.0
appVersion: "1.0"

What does Chart.yaml do?

This file identifies your chart to Helm. The apiVersion: v2 tells Helm this is a Helm 3+ chart. The version field tracks chart revisions, while appVersion tracks the version of the application inside the chart.

• Create the values file by pasting the following into values.yaml in the ~/myapp directory:

# Domain configuration - replace with your actual domain
domain: [[YOUR_DOMAIN="example.com"]]
host: [[YOUR_HOST="app.example.com"]]

# Frontend tier - nginx web server
frontend:
  image: nginx
  tag: "1.25"
  replicas: 2
  resources:
    requests:
      cpu: 100m
      memory: 128Mi
    limits:
      cpu: 500m
      memory: 256Mi

# API tier - httpbin for request inspection
api:
  image: kennethreitz/httpbin
  tag: latest
  replicas: 1
  resources:
    requests:
      cpu: 100m
      memory: 128Mi
    limits:
      cpu: 500m
      memory: 256Mi

# Database tier - PostgreSQL with persistent storage
postgresql:
  image: postgres
  tag: "15"
  storage: 5Gi
  resources:
    requests:
      cpu: 100m
      memory: 256Mi
    limits:
      cpu: 500m
      memory: 512Mi

What does values.yaml control?

Every template in your chart references values from this file using {{ .Values.* }} syntax. To change the frontend image version or add more API replicas, you only edit this one file.

Resource requests tell the Kubernetes scheduler how much CPU and memory a pod needs. Limits cap the maximum a pod can consume.

• Save both Chart.yaml and values.yaml.

Seeing YAML parsing errors later?

YAML is whitespace-sensitive. Make sure you use spaces (not tabs) for indentation. Each nested level uses exactly 2 spaces.

Help me debug a YAML parsing error in my Helm values file.

Now create the template files for each application tier. Start with the frontend.

• Create frontend-deployment.yaml inside the ~/myapp/templates directory with this content:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  namespace: app
spec:
  replicas: {{ .Values.frontend.replicas }}
  selector:
    matchLabels:
      app: frontend
  template:
    metadata:
      labels:
        app: frontend
    spec:
      serviceAccountName: app-service-account
      containers:
        - name: frontend
          image: {{ .Values.frontend.image }}:{{ .Values.frontend.tag }}
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: {{ .Values.frontend.resources.requests.cpu }}
              memory: {{ .Values.frontend.resources.requests.memory }}
            limits:
              cpu: {{ .Values.frontend.resources.limits.cpu }}
              memory: {{ .Values.frontend.resources.limits.memory }}

What does this template do?

This creates a Deployment with 2 replicas of nginx. The {{ .Values.frontend.* }} placeholders pull values from your values.yaml at install time.

The serviceAccountName: app-service-account binds the pod to the ServiceAccount you created in the RBAC step.

• Create frontend-service.yaml in the same templates directory:

apiVersion: v1
kind: Service
metadata:
  name: frontend
  namespace: app
spec:
  selector:
    app: frontend
  ports:
    - port: 80
      targetPort: 80
  type: ClusterIP

Why ClusterIP instead of LoadBalancer?

Each Service uses ClusterIP because traffic will enter through a single ALB managed by the Ingress resource. The Ingress handles external routing, so individual services only need to be reachable inside the cluster.

• Save both frontend files.

Next, create the API tier templates.

• Create api-deployment.yaml in ~/myapp/templates:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
  namespace: app
spec:
  replicas: {{ .Values.api.replicas }}
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      serviceAccountName: app-service-account
      containers:
        - name: api
          image: {{ .Values.api.image }}:{{ .Values.api.tag }}
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: {{ .Values.api.resources.requests.cpu }}
              memory: {{ .Values.api.resources.requests.memory }}
            limits:
              cpu: {{ .Values.api.resources.limits.cpu }}
              memory: {{ .Values.api.resources.limits.memory }}

What is httpbin?

httpbin is a simple HTTP request/response service. It echoes back details about incoming requests, making it perfect for testing API routing and verifying your Ingress paths work correctly.

• Create api-service.yaml in ~/myapp/templates:

apiVersion: v1
kind: Service
metadata:
  name: api
  namespace: app
spec:
  selector:
    app: api
  ports:
    - port: 80
      targetPort: 80
  type: ClusterIP

• Save both API files.

Finally, create the database tier. PostgreSQL uses a StatefulSet instead of a Deployment because it needs stable network identity and persistent storage that survives pod restarts.

• Create postgresql-statefulset.yaml in ~/myapp/templates:

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgresql
  namespace: app
spec:
  serviceName: postgresql
  replicas: 1
  selector:
    matchLabels:
      app: postgresql
  template:
    metadata:
      labels:
        app: postgresql
    spec:
      containers:
        - name: postgresql
          image: {{ .Values.postgresql.image }}:{{ .Values.postgresql.tag }}
          ports:
            - containerPort: 5432
          env:
            - name: POSTGRES_PASSWORD
              value: changeme
          resources:
            requests:
              cpu: {{ .Values.postgresql.resources.requests.cpu }}
              memory: {{ .Values.postgresql.resources.requests.memory }}
            limits:
              cpu: {{ .Values.postgresql.resources.limits.cpu }}
              memory: {{ .Values.postgresql.resources.limits.memory }}
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        storageClassName: gp3
        resources:
          requests:
            storage: {{ .Values.postgresql.storage }}

What does volumeClaimTemplates do?

The volumeClaimTemplates section automatically provisions a 5Gi EBS volume (using the gp3 StorageClass you created earlier) for each PostgreSQL replica. This volume persists even if the pod is deleted and rescheduled.

• Create postgresql-service.yaml in ~/myapp/templates:

apiVersion: v1
kind: Service
metadata:
  name: postgresql
  namespace: app
spec:
  selector:
    app: postgresql
  ports:
    - port: 5432
      targetPort: 5432
  type: ClusterIP

• Save both PostgreSQL files.

• Verify all template files are in place by running:

ls ~/myapp/templates

You should see frontend-deployment.yaml, frontend-service.yaml, api-deployment.yaml, api-service.yaml, postgresql-statefulset.yaml, and postgresql-service.yaml in the output.

Missing a file?

Double check you created each file inside ~/myapp/templates/ and not in the parent ~/myapp/ directory. Helm only processes files inside the templates folder.

Help me figure out why my Helm template files are not being found.

Configure the ALB Ingress

An Ingress resource tells Kubernetes how to route external HTTP traffic to your services. The AWS Load Balancer Controller watches for Ingress resources and provisions an ALB automatically.

• Create ingress.yaml in ~/myapp/templates:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-ingress
  namespace: app
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  rules:
    - host: {{ .Values.host }}
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: frontend
                port:
                  number: 80
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: api
                port:
                  number: 80

What do the ALB annotations mean?

The kubernetes.io/ingress.class: alb annotation tells the AWS Load Balancer Controller to handle this Ingress. The scheme: internet-facing makes the ALB publicly accessible. The target-type: ip routes traffic directly to pod IPs for better performance.

The path rules route / to the frontend and /api to the httpbin service. This gives you host-based and path-based routing from a single ALB.

• Save ingress.yaml.
• Verify the complete chart structure by running:

ls ~/myapp/templates

You should now see 7 files including ingress.yaml alongside the 6 application template files.

✔️ Awesome, I've got everything!

Double check you have saved all files before moving on to the install.

ⓧ I'd like to double check the full code

Here is the complete chart structure for comparison.

apiVersion: v2
name: myapp
description: A multi-tier application Helm chart
type: application
version: 0.1.0
appVersion: "1.0"

domain: example.com
host: app.example.com

frontend:
  image: nginx
  tag: "1.25"
  replicas: 2
  resources:
    requests:
      cpu: 100m
      memory: 128Mi
    limits:
      cpu: 500m
      memory: 256Mi

api:
  image: kennethreitz/httpbin
  tag: latest
  replicas: 1
  resources:
    requests:
      cpu: 100m
      memory: 128Mi
    limits:
      cpu: 500m
      memory: 256Mi

postgresql:
  image: postgres
  tag: "15"
  storage: 5Gi
  resources:
    requests:
      cpu: 100m
      memory: 256Mi
    limits:
      cpu: 500m
      memory: 512Mi

apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  namespace: app
spec:
  replicas: {{ .Values.frontend.replicas }}
  selector:
    matchLabels:
      app: frontend
  template:
    metadata:
      labels:
        app: frontend
    spec:
      serviceAccountName: app-service-account
      containers:
        - name: frontend
          image: {{ .Values.frontend.image }}:{{ .Values.frontend.tag }}
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: {{ .Values.frontend.resources.requests.cpu }}
              memory: {{ .Values.frontend.resources.requests.memory }}
            limits:
              cpu: {{ .Values.frontend.resources.limits.cpu }}
              memory: {{ .Values.frontend.resources.limits.memory }}

apiVersion: v1
kind: Service
metadata:
  name: frontend
  namespace: app
spec:
  selector:
    app: frontend
  ports:
    - port: 80
      targetPort: 80
  type: ClusterIP

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
  namespace: app
spec:
  replicas: {{ .Values.api.replicas }}
  selector:
    matchLabels:
      app: api
  template:
    metadata:
      labels:
        app: api
    spec:
      serviceAccountName: app-service-account
      containers:
        - name: api
          image: {{ .Values.api.image }}:{{ .Values.api.tag }}
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: {{ .Values.api.resources.requests.cpu }}
              memory: {{ .Values.api.resources.requests.memory }}
            limits:
              cpu: {{ .Values.api.resources.limits.cpu }}
              memory: {{ .Values.api.resources.limits.memory }}

apiVersion: v1
kind: Service
metadata:
  name: api
  namespace: app
spec:
  selector:
    app: api
  ports:
    - port: 80
      targetPort: 80
  type: ClusterIP

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgresql
  namespace: app
spec:
  serviceName: postgresql
  replicas: 1
  selector:
    matchLabels:
      app: postgresql
  template:
    metadata:
      labels:
        app: postgresql
    spec:
      containers:
        - name: postgresql
          image: {{ .Values.postgresql.image }}:{{ .Values.postgresql.tag }}
          ports:
            - containerPort: 5432
          env:
            - name: POSTGRES_PASSWORD
              value: changeme
          resources:
            requests:
              cpu: {{ .Values.postgresql.resources.requests.cpu }}
              memory: {{ .Values.postgresql.resources.requests.memory }}
            limits:
              cpu: {{ .Values.postgresql.resources.limits.cpu }}
              memory: {{ .Values.postgresql.resources.limits.memory }}
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        storageClassName: gp3
        resources:
          requests:
            storage: {{ .Values.postgresql.storage }}

apiVersion: v1
kind: Service
metadata:
  name: postgresql
  namespace: app
spec:
  selector:
    app: postgresql
  ports:
    - port: 5432
      targetPort: 5432
  type: ClusterIP

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-ingress
  namespace: app
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
spec:
  rules:
    - host: {{ .Values.host }}
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: frontend
                port:
                  number: 80
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: api
                port:
                  number: 80

Install the chart and verify the deployment

With all templates in place, you can now install the chart as a Helm release. Helm will render every template, substitute values, and apply the resulting manifests to your cluster.

• Install the Helm chart into the app namespace by running:

helm install myapp ~/myapp -n app

What does this command do?

This creates a Helm release named myapp from the chart at ~/myapp. The -n app flag targets the app namespace. Helm renders every template file, substitutes the values, and applies all resources to the cluster.

You should see output confirming the release was deployed successfully.

• Verify all pods are running by running:

kubectl get pods -n app

You should see 2 frontend pods, 1 api pod, and 1 postgresql pod, all with status Running. The PostgreSQL pod may take a minute to become ready while its EBS volume is provisioned.

• Check that the ALB was provisioned by running:

kubectl get ingress -n app

You should see myapp-ingress with an ADDRESS column showing an ALB DNS name (something like k8s-app-myappin-xxxxxxxxxx.us-east-1.elb.amazonaws.com). The ALB may take 2-3 minutes to provision. If the ADDRESS column is empty, wait and re-run the command.

Pods stuck in Pending or CrashLoopBackOff?

Run kubectl describe pod <pod-name> -n app to see the reason. Common causes include insufficient node resources (the nodes are full) or the ServiceAccount app-service-account not existing in the app namespace.

If the Ingress ADDRESS stays empty, check that the AWS Load Balancer Controller is running with kubectl get deployment -n kube-system aws-load-balancer-controller. The controller must be healthy to provision ALBs.

Help me debug why my pods are not starting in my EKS cluster.

Your multi-tier application is live on EKS with an ALB routing traffic to it. Next up, you will add health checks and rolling update strategies so your deployments can update without downtime.

Configure Health Checks and Rolling Updates

Your multi-tier application is deployed and reachable through the ALB. But right now, Kubernetes has no way to know if your containers are actually healthy or just running without responding.

In this step, you will add health checks so Kubernetes can detect and replace unhealthy pods automatically. You will also configure a rolling update strategy that ensures zero downtime during deployments.

In this step, get ready to:

• Add liveness and readiness probes to the frontend and API deployments.
• Configure a rolling update strategy with maxSurge and maxUnavailable.
• Perform a rolling update and verify zero-downtime deployment.

Add liveness and readiness probes to the frontend and API deployments

Kubernetes uses two types of health checks to monitor containers. A liveness probe tells Kubernetes whether the container is still alive. If it fails, Kubernetes kills and restarts the container.

A readiness probe tells Kubernetes whether the container is ready to receive traffic. If it fails, the pod is removed from the Service endpoints until it recovers.

• Open myapp/templates/frontend-deployment.yaml in your editor.
• Find the ports section inside the container spec:

          ports:
            - containerPort: 80

• Add the following liveness and readiness probes directly below the ports block:

          livenessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 10
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 5
            periodSeconds: 5

What do these probe settings mean?

Both probes send an HTTP GET request to / on port 80. If nginx responds with a 200 status, the probe passes.

The liveness probe waits 10 seconds before its first check (initialDelaySeconds) and checks every 10 seconds. The readiness probe starts checking after 5 seconds and checks every 5 seconds. The readiness probe has shorter intervals because you want traffic routing to respond quickly.

• Now open myapp/templates/api-deployment.yaml in your editor.
• Find the same ports section inside the container spec:

          ports:
            - containerPort: 80

• Add the following probes directly below the ports block. Notice the path is /get instead of / because httpbin serves responses at that endpoint:

          livenessProbe:
            httpGet:
              path: /get
              port: 80
            initialDelaySeconds: 10
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /get
              port: 80
            initialDelaySeconds: 5
            periodSeconds: 5

Why /get for the API probe?

Health check endpoints should be lightweight and always respond when the application is healthy. The httpbin /get endpoint returns a simple JSON response, making it a reliable indicator of whether the container is serving requests.

• Save both frontend-deployment.yaml and api-deployment.yaml.

Configure the rolling update strategy

By default, Kubernetes uses a rolling update strategy, but it allows some pods to be unavailable during the process. For production, you want zero-downtime deployments where new pods are fully ready before old ones shut down.

• Open myapp/templates/frontend-deployment.yaml again.
• Find the selector block:

  selector:
    matchLabels:
      app: frontend

• Add the following strategy block directly below the selector block and above the template block:

  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0

What does this strategy do?

• maxSurge: 1 allows Kubernetes to create 1 extra pod above the desired replica count during an update.
• maxUnavailable: 0 means Kubernetes will never take an existing pod offline until a new one passes its readiness probe.

Together, these settings guarantee zero downtime. The new pod must be serving traffic before the old pod is terminated.

• Open myapp/templates/api-deployment.yaml and find the same selector block:

  selector:
    matchLabels:
      app: api

• Add the same strategy block directly below the selector block:

  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0

• Save both files.
• Deploy the probe and strategy changes by running this Helm upgrade:

helm upgrade myapp ./myapp -n app

• Verify the probes are configured by describing the frontend deployment:

kubectl describe deployment frontend -n app

In the output, look for the Liveness and Readiness fields under the container section. They should show the HTTP GET paths you configured.

Pods stuck in CrashLoopBackOff?

If pods are restarting, the liveness probe might be failing. Check that the path matches a real endpoint in your container. For nginx it must be / and for httpbin it must be /get. Also verify your YAML indentation is correct. Probes must be at the same level as ports inside the container spec.

Why are my pods crashing after adding health probes?

✔️ Awesome, I've got everything!

Great. Double check you have saved both deployment files before continuing.

ⓧ I'd like to double check the full code

Here is the complete frontend-deployment.yaml at this point:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: frontend
  namespace: app
spec:
  replicas: {{ .Values.frontend.replicas }}
  selector:
    matchLabels:
      app: frontend
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: frontend
    spec:
      serviceAccountName: app-service-account
      containers:
        - name: frontend
          image: {{ .Values.frontend.image }}:{{ .Values.frontend.tag }}
          ports:
            - containerPort: 80
          livenessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 10
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /
              port: 80
            initialDelaySeconds: 5
            periodSeconds: 5
          resources:
            requests:
              cpu: {{ .Values.frontend.resources.requests.cpu }}
              memory: {{ .Values.frontend.resources.requests.memory }}
            limits:
              cpu: {{ .Values.frontend.resources.limits.cpu }}
              memory: {{ .Values.frontend.resources.limits.memory }}

And here is the complete api-deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
  namespace: app
spec:
  replicas: {{ .Values.api.replicas }}
  selector:
    matchLabels:
      app: api
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  template:
    metadata:
      labels:
        app: api
    spec:
      serviceAccountName: app-service-account
      containers:
        - name: api
          image: {{ .Values.api.image }}:{{ .Values.api.tag }}
          ports:
            - containerPort: 80
          livenessProbe:
            httpGet:
              path: /get
              port: 80
            initialDelaySeconds: 10
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: /get
              port: 80
            initialDelaySeconds: 5
            periodSeconds: 5
          resources:
            requests:
              cpu: {{ .Values.api.resources.requests.cpu }}
              memory: {{ .Values.api.resources.requests.memory }}
            limits:
              cpu: {{ .Values.api.resources.limits.cpu }}
              memory: {{ .Values.api.resources.limits.memory }}

Perform a rolling update and verify zero downtime

Now that your deployments have probes and a zero-downtime strategy, you can test a rolling update. You will bump the nginx version from 1.25 to 1.26 and watch Kubernetes swap pods without dropping traffic.

• Open myapp/values.yaml.
• Find this line under the frontend section:

  tag: "1.25"

• Change "1.25" to "1.26" so the line looks like this:

  tag: "1.26"

• Save values.yaml.
• Trigger the rolling update by upgrading the Helm release:

helm upgrade myapp ./myapp -n app

• Immediately watch the rollout progress by running:

kubectl rollout status deployment/frontend -n app

You should see output like Waiting for deployment "frontend" rollout to finish: 1 old replicas are pending termination... followed by deployment "frontend" successfully rolled out.

What just happened during the rolling update?

Because you set maxUnavailable: 0, Kubernetes first spun up a new pod with nginx:1.26. It waited for the readiness probe to pass (confirming the new pod is serving traffic). Only then did it terminate one of the old nginx:1.25 pods.

This repeated until all replicas were running the new version. At no point did the number of healthy, traffic-serving pods drop below the desired count.

• Confirm the new image is running by checking the pods:

kubectl get pods -n app -l app=frontend -o jsonpath='{.items[*].spec.containers[0].image}'

The output should show nginx:1.26 for all frontend pods.

Rollout stuck or timing out?

If the rollout hangs, the new pods might be failing their readiness probes. Run kubectl get pods -n app to see if any pods are in a non-Ready state. Then run kubectl describe pod <pod-name> -n app to check the Events section for probe failure messages.

My Kubernetes rolling update is stuck and not completing. How do I debug it?

Your deployments now self-heal and update without downtime. Next up, you will add a Horizontal Pod Autoscaler so your frontend automatically scales under load.

Set Up Horizontal Pod Autoscaler

Your application has health checks and rolling updates, but it still runs a fixed number of pods regardless of traffic. If a spike in visitors overwhelms your two frontend pods, users get slow responses or timeouts.

A Horizontal Pod Autoscaler solves this by watching CPU usage and automatically adding pods when demand rises. In this step, you will install the Metrics Server (which feeds usage data to the HPA) and create an autoscaling rule for the frontend.

In this step, get ready to:

• Install the Kubernetes Metrics Server so the cluster can measure resource usage.
• Create an HPA resource that scales the frontend between 2 and 10 pods based on CPU.
• Verify the HPA is reading live metrics from your nodes and pods.

Install the Kubernetes Metrics Server

The HPA needs real-time CPU and memory data to make scaling decisions. The Kubernetes Metrics Server collects this data from every node and exposes it through the Kubernetes Metrics API.

• Deploy the Metrics Server to your cluster by running this command:

kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml

What does this command do?

This downloads the official Metrics Server manifest from GitHub and applies it to your cluster. It creates a Deployment, Service, ServiceAccount, and RBAC resources in the kube-system namespace.

Once running, the Metrics Server scrapes resource usage from every kubelet every 15 seconds and makes it available via kubectl top and the HPA controller.

• Verify the Metrics Server deployment is running by running:

kubectl get deployment metrics-server -n kube-system

You should see metrics-server with 1/1 in the READY column.

• Wait about 60 seconds for the Metrics Server to collect its first round of data, then test it by running:

kubectl top nodes

You should see your two nodes listed with CPU and memory percentages populated.

Seeing 'error: Metrics API not available'?

The Metrics Server needs about 60 seconds after starting before it begins serving data. Wait a minute and try kubectl top nodes again.

If it still fails, check that the metrics-server pod is healthy:

kubectl get pods -n kube-system -l k8s-app=metrics-server

The pod should show Running with 1/1 ready. Help me troubleshoot why kubectl top nodes is failing on my EKS cluster.

Create the HPA resource

Now that your cluster can measure CPU usage, you can create a Horizontal Pod Autoscaler that watches the frontend Deployment and adjusts its replica count automatically.

• Create the HPA manifest file by running:

cat <<EOF > hpa.yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: frontend-hpa
  namespace: app
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: frontend
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 50
EOF

What does this manifest do?

scaleTargetRef tells the HPA which Deployment to scale. Here it targets the frontend Deployment in the app namespace.

minReplicas: 2 keeps at least 2 pods running at all times, matching your current replica count. maxReplicas: 10 caps scaling at 10 pods to protect against runaway resource consumption.

The averageUtilization: 50 target means the HPA adds pods when average CPU usage across all frontend pods exceeds 50% of their requested CPU (100m per pod).

• Apply the HPA manifest by running:

kubectl apply -f hpa.yaml

You should see horizontalpodautoscaler.autoscaling/frontend-hpa created in the output.

Seeing a 'not found' error?

Make sure you are in the same directory where you created hpa.yaml. Run ls hpa.yaml to confirm the file exists in your current location.

Also confirm the app namespace exists by running kubectl get ns app. Help me fix my HPA creation error on EKS.

Verify the HPA is reading metrics

• Check the HPA status by running:

kubectl get hpa -n app

You should see frontend-hpa listed with a value in the TARGETS column (for example, 2%/50%) and MINPODS/MAXPODS showing 2 and 10.

What if TARGETS shows '<unknown>/50%'?

This means the HPA has not received its first metrics reading yet. Wait 30-60 seconds and run kubectl get hpa -n app again. Once the Metrics Server feeds data to the HPA controller, the percentage will appear.

• For a more detailed view of the HPA's scaling decisions, run:

kubectl describe hpa frontend-hpa -n app

Look for the Conditions section. You should see AbleToScale: True and ScalingActive: True, confirming the HPA is operational.

ScalingActive showing False?

This usually means the Metrics Server is not yet reporting data for the target pods. Confirm the frontend pods have CPU resource requests defined by running:

kubectl get pods -n app -l app=frontend -o jsonpath='{.items[0].spec.containers[0].resources.requests.cpu}'

You should see 100m returned. If this is empty, the HPA cannot calculate a percentage and will stay inactive. Why is my HPA showing ScalingActive False?

Your cluster now automatically scales the frontend between 2 and 10 pods based on real CPU demand. Next up, you will wire up automated DNS and TLS certificates so your application is reachable over HTTPS with a real domain name.

Install External-DNS and cert-manager

Your application is live behind an ALB, but visitors still need to remember the load balancer's auto-generated hostname. DNS records must be created manually, and traffic flows over plain HTTP without encryption.

In this step, you will install two controllers that automate these problems away. External-DNS watches your Ingress resources and automatically creates Route53 records. cert-manager requests free TLS certificates from Let's Encrypt using DNS-01 validation.

In this step, get ready to:

• Set up External-DNS with IAM permissions and install it via Helm.
• Install cert-manager and configure an IAM role for DNS-01 validation.
• Create a ClusterIssuer, update your Ingress with TLS, and verify the certificate is issued.

Set up External-DNS with IAM and Helm

External-DNS needs permission to create and update records in your Route53 hosted zone. You will create a dedicated IAM policy, bind it to a Kubernetes service account via IRSA, then install External-DNS using Helm.

• Create a file called external-dns-policy.json with the IAM policy by running:

cat <<'EOF' > external-dns-policy.json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "route53:ChangeResourceRecordSets"
      ],
      "Resource": [
        "arn:aws:route53:::hostedzone/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "route53:ListHostedZones",
        "route53:ListResourceRecordSets",
        "route53:ListTagsForResource"
      ],
      "Resource": ["*"]
    }
  ]
}
EOF

What does this policy allow?

The first statement lets External-DNS create, update, and delete DNS records in any hosted zone in your account. The second statement lets it list your zones and their records so it knows which ones to manage.

• Create the IAM policy in AWS by running:

aws iam create-policy \
  --policy-name ExternalDNSPolicy \
  --policy-document file://external-dns-policy.json

• Note the policy ARN from the output. It follows the pattern arn:aws:iam::<account-id>:policy/ExternalDNSPolicy.
• Create the IRSA service account for External-DNS by running:

eksctl create iamserviceaccount \
  --cluster prod-cluster \
  --name external-dns \
  --namespace kube-system \
  --attach-policy-arn arn:aws:iam::[[AWS_ACCOUNT_ID="your AWS account ID"]]:policy/ExternalDNSPolicy \
  --approve

What does this command do?

This creates an IAM role with the ExternalDNSPolicy attached, creates a Kubernetes service account called external-dns in kube-system, and annotates that service account with the role ARN. Pods using this service account automatically receive AWS credentials via IRSA.

• Add the External-DNS Helm repository by running:

helm repo add external-dns https://kubernetes-sigs.github.io/external-dns/
helm repo update

• Install External-DNS with your domain filter by running:

helm install external-dns external-dns/external-dns \
  --namespace kube-system \
  --set provider.name=aws \
  --set policy=sync \
  --set domainFilters[0]=[[DOMAIN="your domain e.g. example.com"]] \
  --set serviceAccount.create=false \
  --set serviceAccount.name=external-dns

What do these Helm values do?

• provider.name=aws tells External-DNS to manage Route53 records.
• domainFilters restricts which hosted zones External-DNS can touch.
• serviceAccount.create=false tells Helm to use the IRSA service account you already created instead of making a new one.

• Verify External-DNS is running by checking the pod:

kubectl get pods -n kube-system -l app.kubernetes.io/name=external-dns

You should see one pod in a Running state with 1/1 containers ready.

Pod stuck in CrashLoopBackOff?

• Check the pod logs with kubectl logs -n kube-system -l app.kubernetes.io/name=external-dns. Look for "AccessDenied" errors which mean the IRSA annotation is missing or the policy ARN is incorrect.
• Verify the service account has the annotation by running kubectl get sa external-dns -n kube-system -o yaml and looking for the eks.amazonaws.com/role-arn annotation.

help me debug External-DNS pod errors

Install cert-manager and configure IAM for DNS-01

cert-manager automates TLS certificate lifecycle. It will use the DNS-01 challenge type to prove you own your domain by creating a temporary TXT record in Route53.

• Install cert-manager via Helm with CRDs enabled by running:

helm install cert-manager oci://quay.io/jetstack/charts/cert-manager \
  --namespace cert-manager \
  --create-namespace \
  --set crds.enabled=true

What does this command do?

This installs cert-manager v1.20.2 into a new cert-manager namespace. The crds.enabled=true flag installs the Custom Resource Definitions (Certificate, ClusterIssuer, etc.) that cert-manager needs to function.

• Verify cert-manager pods are running:

kubectl get pods -n cert-manager

You should see three pods running: cert-manager, cert-manager-cainjector, and cert-manager-webhook.

cert-manager pods not starting?

If the webhook pod is stuck, wait 30 seconds and check again. The webhook needs the CRDs to be fully registered before it becomes healthy. If pods remain in Pending state, check node resources with kubectl describe pod -n cert-manager.

help me troubleshoot cert-manager installation

Now cert-manager needs its own IAM role to interact with Route53 for DNS-01 challenges. You will use the same ExternalDNSPolicy since both tools need identical Route53 permissions.

• Create an IRSA service account for cert-manager by running:

eksctl create iamserviceaccount \
  --cluster prod-cluster \
  --name cert-manager \
  --namespace cert-manager \
  --role-name cert-manager-route53 \
  --attach-policy-arn arn:aws:iam::[[AWS_ACCOUNT_ID="your AWS account ID"]]:policy/ExternalDNSPolicy \
  --approve \
  --override-existing-serviceaccounts

Why override existing service accounts?

The Helm install already created a cert-manager service account. The --override-existing-serviceaccounts flag adds the IAM role annotation to the existing service account without deleting and recreating it.

• Restart the cert-manager controller so it picks up the new IRSA credentials:

kubectl rollout restart deployment cert-manager -n cert-manager

• Verify the restart completes:

kubectl rollout status deployment cert-manager -n cert-manager

You should see deployment "cert-manager" successfully rolled out.

Configure the ClusterIssuer and verify your certificate

With cert-manager running and authenticated to Route53, you can now create a ClusterIssuer that tells cert-manager how to obtain certificates from Let's Encrypt using DNS-01 validation.

• Create a file called cluster-issuer.yaml with the ClusterIssuer configuration:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    # Let's Encrypt production ACME server
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [[LETSENCRYPT_EMAIL="your email address for Let's Encrypt notifications"]]
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
      - dns01:
          route53:
            region: us-east-1
            hostedZoneID: [[HOSTED_ZONE_ID="your Route53 hosted zone ID e.g. Z1234567890ABC"]]

What does this ClusterIssuer do?

• server points to the Let's Encrypt production CA. Certificates issued here are trusted by all browsers.
• email receives expiration warnings if cert-manager fails to auto-renew.
• dns01.route53 tells cert-manager to prove domain ownership by creating a TXT record in the specified hosted zone.
• hostedZoneID scopes the solver to your specific zone so cert-manager does not attempt to write records in other zones.

Why DNS-01 instead of HTTP-01?

DNS-01 works before your domain resolves to the cluster. There is no chicken-and-egg problem with ALB provisioning. It also supports wildcard certificates if you need them later.

• Apply the ClusterIssuer by running:

kubectl apply -f cluster-issuer.yaml

• Verify the ClusterIssuer is ready:

kubectl get clusterissuer letsencrypt-prod

The READY column should show True. If it shows False, wait a few seconds and run the command again.

Now you need to update your Ingress to request a TLS certificate and tell External-DNS which hostname to create a record for.

• Open myapp/templates/ingress.yaml and find the annotations section:

  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip

• Add the cert-manager and External-DNS annotations, and add the TLS spec block so the full file looks like this:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myapp-ingress
  namespace: app
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    # Tell cert-manager which issuer to use
    cert-manager.io/cluster-issuer: letsencrypt-prod
    # Tell External-DNS to create an A record for this host
    external-dns.alpha.kubernetes.io/hostname: app.example.com
spec:
  tls:
    - hosts:
        - app.example.com
      secretName: myapp-tls
  rules:
    - host: {{ .Values.host }}
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: frontend
                port:
                  number: 80
          - path: /api
            pathType: Prefix
            backend:
              service:
                name: api
                port:
                  number: 80

What changed in the Ingress?

• cert-manager.io/cluster-issuer: letsencrypt-prod triggers cert-manager to automatically request a certificate for hosts listed in the tls block.
• external-dns.alpha.kubernetes.io/hostname tells External-DNS to create an A record pointing app.example.com to the ALB.
• The tls block specifies which hosts get a certificate and names the Kubernetes Secret (myapp-tls) where the certificate will be stored.

• Upgrade the Helm release to apply the Ingress changes:

helm upgrade myapp ./myapp -n app

• Check the certificate status by running:

kubectl get certificate -n app

You should see a certificate called myapp-tls with READY=True. This can take 1-3 minutes while cert-manager creates the DNS-01 challenge TXT record and waits for Let's Encrypt to validate it.

Check progress while waiting

Run kubectl describe certificate myapp-tls -n app to see detailed status. You can also watch the cert-manager logs with kubectl logs -n cert-manager -l app.kubernetes.io/name=cert-manager -f to observe the DNS-01 challenge in progress.

Certificate stuck in pending state?

• Run kubectl describe certificate myapp-tls -n app and look at the Events section. Common issues include the ClusterIssuer not being ready or DNS propagation delays.
• Verify your Route53 nameservers are authoritative by running dig NS your-domain.com. If the nameservers do not point to Route53, the DNS-01 challenge will fail because Let's Encrypt cannot find the TXT record.
• Check that the cert-manager service account has the IAM annotation by running kubectl get sa cert-manager -n cert-manager -o yaml.

help me debug my cert-manager certificate not becoming ready

• Verify External-DNS created the Route53 A record by running:

kubectl logs -n kube-system -l app.kubernetes.io/name=external-dns | grep app.example.com

You should see log entries indicating External-DNS created or updated an A record for your domain.

Your cluster now has fully automated DNS and TLS. External-DNS creates Route53 records whenever an Ingress appears, and cert-manager issues and renews certificates without manual intervention. Next up, you will add a Prometheus and Grafana monitoring stack to observe everything in real time.

Install Prometheus and Grafana Monitoring Stack

Your cluster is secured with TLS, scales automatically, and manages its own DNS records. The missing piece is observability. Without monitoring, you have no visibility into CPU pressure, memory saturation, or pod health across your nodes.

In this step, you will install the kube-prometheus-stack, a community-maintained Helm chart that bundles Prometheus (metrics collection), Grafana (dashboards), and Alertmanager into a single deployment. Once installed, you will have pre-built Kubernetes dashboards showing node and pod metrics out of the box.

In this step, get ready to:

• Add the prometheus-community Helm repository.
• Install kube-prometheus-stack in the monitoring namespace.
• Access Grafana and verify Prometheus is scraping cluster metrics.

Add the prometheus-community Helm repo

The kube-prometheus-stack chart lives in the prometheus-community Helm repository. You need to register this repo with Helm before you can install from it.

• Add the prometheus-community Helm repository and update your local chart index by running these commands:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update

What do these commands do?

The first command registers the prometheus-community chart repository with your local Helm client. The second command refreshes the local index so Helm knows about the latest chart versions available.

• You should see "prometheus-community" has been added to your repositories followed by Update Complete..

Seeing a connection error?

Make sure you have internet connectivity from your machine. If you see a timeout, check that your network allows outbound HTTPS to prometheus-community.github.io.

If you see the repo already exists, that is fine. Run helm repo update to refresh it.

Help me troubleshoot adding the prometheus-community Helm repo.

Install kube-prometheus-stack with persistence disabled

The kube-prometheus-stack chart deploys Prometheus, Grafana, and Alertmanager together. For this learning project, you will install it without persistent storage to save on EBS costs since the cluster will be torn down after completion.

• Install the kube-prometheus-stack chart into the monitoring namespace by running this command:

helm install kube-prometheus-stack prometheus-community/kube-prometheus-stack \
  --namespace monitoring

What does this command do?

This installs the full monitoring stack as a Helm release named kube-prometheus-stack in the monitoring namespace (which you created in Step 5).

The chart's defaults already have persistence disabled for Prometheus and Grafana. This means metrics data lives only in memory and will be lost if pods restart. That is acceptable for a short-lived learning project.

Why skip persistent storage here?

Enabling persistence would provision additional EBS volumes for Prometheus time-series data and Grafana database files. Since you will tear this cluster down within hours, the extra cost and complexity are not worth it. The pre-built dashboards are loaded from ConfigMaps anyway, so no Grafana configuration is lost.

• Wait 1-2 minutes for all pods to start, then verify the monitoring pods are running:

kubectl get pods -n monitoring

• You should see multiple pods with names starting with kube-prometheus-stack, prometheus-kube-prometheus-stack-prometheus, and alertmanager. All pods should show Running status.

Pods stuck in Pending or CrashLoopBackOff?

If pods are Pending, your nodes may not have enough resources. Run kubectl describe pod <pod-name> -n monitoring and check the Events section for scheduling failures.

If pods are in CrashLoopBackOff, check logs with kubectl logs <pod-name> -n monitoring. The most common cause is insufficient memory on the t3.medium nodes when the full stack is running.

Help me troubleshoot kube-prometheus-stack pods that are not running.

Access Grafana and verify cluster metrics

Grafana provides the visualization layer for your monitoring stack. You will access it through a port-forward and confirm that Prometheus is actively scraping metrics from your cluster nodes and pods.

• Start a port-forward to the Grafana service by running this command:

kubectl port-forward svc/kube-prometheus-stack-grafana 3000:80 -n monitoring

What does this command do?

This forwards port 3000 on your local machine to port 80 on the Grafana service inside the cluster. As long as this command is running, you can access Grafana in your browser.

• Open your browser and navigate to http://localhost:3000.
• Log in with username admin and password prom-operator.

Why prom-operator as the password?

The kube-prometheus-stack chart sets prom-operator as the default Grafana admin password. In a real production environment, you would override this with a strong password using Helm values or a Kubernetes Secret.

• In the Grafana sidebar, click the hamburger menu icon (three horizontal lines) on the top left.
• Click Dashboards.
• Open the Kubernetes / Compute Resources / Cluster dashboard.

You should see live graphs showing CPU and memory usage across your cluster nodes. This confirms Prometheus is actively scraping metrics from your EKS nodes and pods.

• Navigate back to Dashboards and open Kubernetes / Compute Resources / Namespace (Pods).
• Select the app namespace from the dropdown at the top.

You should see your frontend, api, and postgresql pods with CPU and memory utilization graphs. This is the same data the HPA uses to make scaling decisions.

Dashboards showing No Data?

If dashboards show "No data", Prometheus may still be performing its first scrape. Wait 1-2 minutes and refresh the page.

If the issue persists, check that Prometheus is running with kubectl get pods -n monitoring | grep prometheus. The prometheus pod must be in Running state with all containers ready (e.g. 2/2).

Help me troubleshoot Grafana dashboards showing no data from Prometheus.

Your cluster now has full observability. Prometheus collects metrics from every node, pod, and Kubernetes component. Grafana visualizes them with pre-built dashboards. Next up, you will tear down all resources cleanly to stop AWS charges.

Secret mission

Load-Test and Watch Autoscaling in Real-Time

Your HPA is configured, Grafana is running, and your application is live. But how does it actually behave under pressure? In this secret mission, you'll flood your frontend with traffic and watch Kubernetes scale pods up automatically while monitoring the resource spike in Grafana dashboards.

Clean Up Your Resources

Clean Up Your Resources

This project uses several AWS resources that incur ongoing costs while running. The total cost is approximately $0.22/hr (around $5/day), so you should decide now whether to keep, pause, or delete everything.

Cost warning

Your EKS cluster charges $0.10/hr for the control plane alone, even if you scale nodes to zero. The two t3.medium instances add another ~$0.083/hr, plus EBS volumes and the ALB. If you leave everything running overnight, expect charges of $5 or more.

The recommended option for a learning project is to delete everything immediately after you finish.

Resources you used:

• EKS cluster (control plane): $0.10/hr.
• 2x t3.medium EC2 instances (worker nodes): ~$0.083/hr total.
• EBS volumes (node storage and PVCs): ~$0.01/hr.
• Route53 hosted zone: $0.50/month (prorated).
• Application Load Balancer: ~$0.02/hr.
• IAM roles and policies (no ongoing cost).
• Helm releases (Prometheus, Grafana, cert-manager, External-DNS, AWS LB Controller, application chart).

Keep everything running

No action needed. Choose this if you are still actively experimenting with your cluster and want to keep testing.

• Your cluster and all resources will remain running.
• Expect ongoing charges of approximately $0.22/hr (~$5/day).
• Monitor your AWS billing dashboard to avoid surprise costs.

Pause - I'll come back to this later

Pausing is not practical with EKS. The cluster control plane charges $0.10/hr even with zero worker nodes. There is no way to truly pause an EKS cluster without deleting it.

• You could scale your node group to zero nodes to save on EC2 costs, but the control plane fee continues.
• The ALB and EBS volumes also incur charges while they exist.
• For a learning project, deleting and recreating is more cost-effective than pausing.

Delete - I don't want to use this again

Remove all project resources to stop all charges immediately. This is the recommended option for a learning project.

• Follow the complete teardown instructions in Step 11 of this project.
• Step 11 walks you through deleting Helm releases, the Route53 hosted zone, reverting Porkbun nameservers, and running eksctl delete cluster.
• After deletion, verify no lingering resources remain (load balancers, EBS volumes, IAM roles) in the AWS console.
• Once everything is deleted, all charges stop immediately.

That's a Wrap!

That's a Wrap!

You just built a production-grade Kubernetes platform on AWS from scratch. This is the kind of infrastructure that powers real applications in the wild.

You've learned how to:

• Provisioned a multi-node EKS cluster with IRSA, EBS CSI Driver for persistent storage, RBAC policies, and an AWS Load Balancer Controller for ALB-based ingress.
• Deployed a multi-tier application using a custom Helm chart with health checks, rolling updates, and a Horizontal Pod Autoscaler for dynamic scaling.
• Automated DNS with External-DNS, provisioned TLS certificates with cert-manager and Let's Encrypt, and deployed a full Prometheus + Grafana observability stack.
• Secret Mission: Load-tested the application and watched HPA scale pods in real-time through Grafana dashboards.

Ready to quiz yourself?