CI/CD Pipeline with Terraform and EKS

Build a production CI/CD pipeline that deploys containers to Kubernetes on AWS.

Introduction

30 Second Summary

Every time you use a popular app, there's a pipeline behind the scenes that automatically tests, packages, and deploys code changes in minutes. This is how professional teams ship software dozens of times a day without breaking things.

In this project, you will build a complete CI/CD pipeline that automatically deploys a containerized application to a Kubernetes cluster on AWS every time you push code to GitHub.

What You'll Build

A production CI/CD pipeline using Terraform for infrastructure, Docker for containerization, Amazon EKS for Kubernetes orchestration, and GitHub Actions for automated deployments with OIDC authentication.

By the end of this project, you'll have:

  • A Terraform configuration that provisions a VPC, EKS cluster, ECR repository, and GitHub OIDC identity provider.
  • A Dockerized Node.js application with a health check endpoint, built using a multi-stage Dockerfile.
  • A GitHub Actions workflow that builds, pushes to ECR, and deploys to EKS using OIDC authentication with no stored secrets.
  • CloudWatch Container Insights monitoring your cluster with real-time metrics and dashboards.
  • Secret Mission: Add Prometheus and Grafana to your cluster for advanced monitoring with custom dashboards.

Are there any prerequisites?

This is an advanced project. You'll need an AWS account with billing enabled, a GitHub account, and Terraform installed on your machine.

Install Required Tools

Before you can provision infrastructure or deploy containers, you need three tools installed on your local machine: the AWS CLI for managing AWS services, Docker Desktop for building and running containers, and kubectl for communicating with your Kubernetes cluster.

Each of these tools plays a specific role in the CI/CD pipeline you are building. The AWS CLI authenticates your terminal to AWS. Docker packages your application into images. kubectl sends deployment commands to your EKS cluster.

In this step, get ready to:
  • Install and configure the AWS CLI v2 with your credentials.
  • Install Docker Desktop and verify it is running.
  • Install kubectl for Kubernetes cluster management.
Install and configure the AWS CLI v2

The AWS CLI lets you manage AWS services directly from your terminal. You will use it throughout this project to configure EKS, push images to ECR, and enable monitoring add-ons.

  • Open your terminal and check if the AWS CLI is already installed by running:
aws --version

✔️ I see aws-cli/2.x.x

The AWS CLI v2 is already installed. Skip ahead to the configuration step below.

ⓧ I see aws-cli/1.x.x

You have AWS CLI version 1 installed. You need to upgrade to version 2.

Uninstall version 1 first, then install version 2 using the instructions in the next tab.

ⓧ Command not found

The AWS CLI is not installed yet. Install it using the instructions for your operating system.

macOS

  • Download the macOS installer by visiting this URL in your browser:
  • https://awscli.amazonaws.com/AWSCLIV2.pkg
  • Double-click the downloaded AWSCLIV2.pkg file and follow the on-screen instructions.
  • When prompted, click Continue and then Install.
  • Enter your password if asked for administrator access.

Windows

  • Download the Windows MSI installer by visiting this URL in your browser:
  • https://awscli.amazonaws.com/AWSCLIV2.msi
  • Double-click the downloaded AWSCLIV2.msi file and follow the installation wizard.
  • Accept the license agreement and click Install.
  • Click Finish when the installation completes.

After installing, close and reopen your terminal, then verify by running:

aws --version

You should see output starting with aws-cli/2. followed by a version number.

Now configure the AWS CLI with your credentials. This tells the CLI which AWS account to use when running commands.

  • Run the configuration command:
aws configure

What will aws configure ask?

The command prompts you for four values. Enter them one at a time when prompted:

  • AWS Access Key ID and AWS Secret Access Key from your IAM user credentials.
  • Default region name: enter us-east-1.
  • Default output format: enter json.
  • Verify your credentials are working by running:
aws sts get-caller-identity

You should see a JSON response with your account ID, user ID, and ARN. This confirms the CLI can authenticate to your AWS account.

Seeing an error about credentials?

  • Double check that you entered your Access Key ID and Secret Access Key correctly. They are case-sensitive.
  • Make sure the IAM user has programmatic access enabled in the AWS console.
  • If you see a region error, re-run aws configure and enter us-east-1 as the default region.

help me fix my AWS CLI credentials

Install Docker Desktop

Docker packages your Node.js application into a portable container image. Docker Desktop provides the Docker engine, CLI, and a graphical interface for managing containers on your local machine.

  • Check if Docker is already installed by running:
docker --version

✔️ I see Docker version

Docker is already installed. Make sure Docker Desktop is running by looking for the Docker whale icon in your system tray (Windows) or menu bar (macOS).

  • If the icon is not visible, open Docker Desktop using your system search (Cmd+Space on macOS or the Windows key on Windows), type Docker Desktop, and press Enter.

ⓧ Command not found

Docker is not installed yet. Download and install Docker Desktop for your operating system.

macOS

  • Visit https://docs.docker.com/desktop/setup/install/mac-install/ and click the download button for your chip (Apple Silicon or Intel).
  • Double-click the downloaded Docker.dmg file.
  • Drag the Docker icon to the Applications folder.
  • Open Docker.app from your Applications folder.
  • Accept the Docker Subscription Service Agreement when prompted.
  • Enter your password if asked for privileged access.

Wait for the Docker engine to start. The whale icon in your menu bar will stop animating when Docker is ready.

Windows

  • Visit https://docs.docker.com/desktop/install/windows-install/ and click Docker Desktop for Windows - x86_64.
  • Run the downloaded installer.
  • Ensure the Use WSL 2 instead of Hyper-V option is checked during installation.
  • Click Ok to proceed with the installation.
  • Restart your computer if prompted.
  • After restarting, Docker Desktop launches automatically. Accept the service agreement.
What is WSL 2?

WSL 2 (Windows Subsystem for Linux 2) provides a lightweight Linux kernel that Docker uses to run containers on Windows. Docker Desktop requires WSL 2.1.5 or later as a backend.

After installation, verify Docker is running:

docker --version

You should see a version number confirming Docker is installed and accessible from your terminal.

Docker Desktop licensing

Docker Desktop is free for personal use, education, and small businesses (fewer than 250 employees and under $10M revenue). Since you are using it for personal learning, it qualifies for free use.

Docker command not found after install?

  • Close and reopen your terminal. Docker Desktop adds itself to your PATH during installation, but existing terminal sessions may not pick up the change.
  • On Windows, make sure Docker Desktop is running (look for the whale icon in your system tray).
  • On macOS, ensure you dragged Docker to the Applications folder and launched it at least once.

help me get Docker Desktop running

Install kubectl

kubectl is the command-line tool for sending commands to a Kubernetes cluster. You will use it to deploy your application to EKS and check the status of your pods.

  • Check if kubectl is already installed by running:
kubectl version --client

✔️ I see a version number

kubectl is already installed and ready to use. You can move on to the next step.

ⓧ Command not found

kubectl is not installed yet. Install it using the instructions for your operating system.

macOS

  • Install kubectl using Homebrew by running:
brew install kubectl
  • Verify the installation:
kubectl version --client

You should see a client version number in the output.

Windows

You can install kubectl using Chocolatey, Scoop, or winget. Pick whichever package manager you have available.

  • Using Chocolatey:
choco install kubernetes-cli
  • Using winget:
winget install -e --id Kubernetes.kubectl
  • After installing, close and reopen your terminal, then verify:
kubectl version --client

You should see a client version number in the output.

Docker Desktop includes kubectl

Docker Desktop for Windows adds its own version of kubectl to your PATH. If you installed Docker Desktop first, you may already have kubectl available. The version check above will confirm this.

kubectl command still not found?

  • Make sure you closed and reopened your terminal after installation.
  • On Windows, ensure the kubectl binary folder is in your PATH environment variable.
  • On macOS, if Homebrew is not installed, visit https://brew.sh to install it first, then re-run the install command.

help me install kubectl

All three tools are installed and ready. Next up, you will create the Node.js application and Dockerfile that your pipeline will build and deploy.

Create the Application and Dockerfile

Before you can deploy anything to Kubernetes, you need an application to deploy. Every CI/CD pipeline starts with a working, containerized application that has been tested locally.

In this step, you'll create a Node.js Express application with a health check endpoint, containerize it with a multi-stage Docker build, and verify everything works locally.

In this step, get ready to:
  • Create a Node.js Express application with a health check endpoint.
  • Write a multi-stage Dockerfile to containerize the app.
  • Build and test the container locally with Docker.
Create the Node.js Express application

Your application needs two things: a main route that returns useful information, and a health check endpoint that Kubernetes will use to determine whether your app is running correctly.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir eks-cicd-pipeline
cd eks-cicd-pipeline
  • Create the app subdirectory by running:
mkdir app
  • Confirm the folders exist by running:
ls

You should see app in the output.

  • Open the eks-cicd-pipeline folder in your code editor.
  • Create a new file called server.js inside the app folder.
  • Paste the following code into app/server.js:
const express = require('express');
const app = express();
const port = process.env.PORT || 3000;

// Main route - returns app info as JSON
app.get('/', (req, res) => {
  res.json({
    message: 'Hello from EKS!',
    version: process.env.APP_VERSION || '1.0.0',
    timestamp: new Date().toISOString()
  });
});

// Health check endpoint - Kubernetes uses this to verify the app is alive
app.get('/health', (req, res) => {
  res.status(200).json({ status: 'healthy' });
});

// Start the server
app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});

What does this code do?

This Express app defines two routes. The / route returns a JSON response with a greeting, the app version, and a timestamp.

The /health route returns a 200 status with {status: 'healthy'}. Kubernetes will call this endpoint repeatedly to check whether your pod is still alive and ready to receive traffic.

  • Save app/server.js.
  • Create a new file called package.json inside the app folder.
  • Paste the following code into app/package.json:
{
  "name": "eks-cicd-app",
  "version": "1.0.0",
  "description": "Sample app for EKS CI/CD pipeline",
  "main": "server.js",
  "scripts": {
    "start": "node server.js"
  },
  "dependencies": {
    "express": "^4.21.0"
  }
}

What does this file do?

The package.json file tells Node.js what your project is and what packages it depends on. The express dependency is the web framework that powers your server routes.

Docker will use this file during the build process to install dependencies inside the container.

  • Save app/package.json.

✔️ Awesome, I've got everything!

Great. Make sure both app/server.js and app/package.json are saved.

ⓧ I'd like to double check the full code

Here is app/server.js:

const express = require('express');
const app = express();
const port = process.env.PORT || 3000;

app.get('/', (req, res) => {
  res.json({
    message: 'Hello from EKS!',
    version: process.env.APP_VERSION || '1.0.0',
    timestamp: new Date().toISOString()
  });
});

app.get('/health', (req, res) => {
  res.status(200).json({ status: 'healthy' });
});

app.listen(port, () => {
  console.log(`Server running on port ${port}`);
});

Here is app/package.json:

{
  "name": "eks-cicd-app",
  "version": "1.0.0",
  "description": "Sample app for EKS CI/CD pipeline",
  "main": "server.js",
  "scripts": {
    "start": "node server.js"
  },
  "dependencies": {
    "express": "^4.21.0"
  }
}
Write a multi-stage Dockerfile

A multi-stage build keeps your final container small and secure. The first stage installs dependencies, and the second stage copies only the production files into a clean image.

  • Create a new file called Dockerfile in the root of your eks-cicd-pipeline folder (not inside app).
  • Paste the following code into Dockerfile:
# Stage 1: Install production dependencies
FROM node:20-alpine AS builder
WORKDIR /app
COPY app/package*.json ./
RUN npm ci --only=production

# Stage 2: Create the final lightweight image
FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/node_modules ./node_modules
COPY app/server.js ./
EXPOSE 3000
USER node
CMD ["node", "server.js"]

What does this Dockerfile do?

The first stage (builder) installs packages with npm ci --only=production. This creates a node_modules folder containing only the dependencies your app needs to run.

The second stage starts from a fresh node:20-alpine image. It copies the node_modules from the builder and your application code. The USER node line runs the process as a non-root user for security.

  • Save Dockerfile.

File not recognized as a Dockerfile?

Make sure the file is named exactly Dockerfile with a capital D and no file extension. It should sit at the same level as your app folder, not inside it.

help me fix my Dockerfile location

✔️ Awesome, I've got everything!

Great. Make sure your Dockerfile is saved.

ⓧ I'd like to double check the full code

FROM node:20-alpine AS builder
WORKDIR /app
COPY app/package*.json ./
RUN npm ci --only=production

FROM node:20-alpine
WORKDIR /app
COPY --from=builder /app/node_modules ./node_modules
COPY app/server.js ./
EXPOSE 3000
USER node
CMD ["node", "server.js"]
Build and test the container locally

Before pushing anything to the cloud, you should confirm the container builds and runs correctly on your local machine.

  • In your terminal, make sure you are inside the eks-cicd-pipeline folder.
  • Build the Docker image by running this command:
docker build -t cicd-demo-app .

What does this command do?

The -t cicd-demo-app flag names your image cicd-demo-app. The . at the end tells Docker to use the current directory as the build context, which is where it looks for your Dockerfile and application files.

You should see Docker pull the node:20-alpine image, install dependencies, and finish with a success message.

Build failing?

Make sure Docker Desktop is running. Check that your Dockerfile is in the root of eks-cicd-pipeline (not inside app). Verify that app/package.json exists with the correct content.

help me debug my Docker build failure

  • Run the container in detached mode by running this command:
docker run -d -p 3000:3000 --name cicd-test cicd-demo-app

What does this command do?

The -d flag runs the container in the background so you can keep using your terminal. The -p 3000:3000 flag maps port 3000 on your machine to port 3000 inside the container. The --name cicd-test flag gives the container a name you can reference later.

  • Test the main route by running:
curl http://localhost:3000

You should see a JSON response with "message": "Hello from EKS!", a version, and a timestamp.

  • Test the health check endpoint by running:
curl http://localhost:3000/health

You should see {"status":"healthy"} in the response.

Getting "connection refused"?

Run docker ps to check if your container is running. If it exited immediately, run docker logs cicd-test to see the error output. A common cause is a typo in server.js that prevents the app from starting.

help me debug my container not responding

  • Stop and remove the test container by running:
docker stop cicd-test
docker rm cicd-test

This cleans up the test container so port 3000 is free for later use.

Your application is containerized and tested. Next up, you'll provision the AWS infrastructure that will host this container in production.

Provision Infrastructure with Terraform

Your application and Dockerfile are ready. Now you need somewhere to run them. Instead of clicking through the AWS Console to create each resource manually, you will define your entire infrastructure in code.

In this step, you will write a Terraform configuration that provisions a VPC, an EKS cluster, an ECR repository, and OIDC authentication for GitHub Actions. Terraform will create all of these resources with a single command.

In this step, get ready to:
  • Write Terraform configuration for VPC, EKS, ECR, and IAM resources.
  • Initialize and apply the Terraform configuration to provision infrastructure.
  • Connect kubectl to the new EKS cluster.
Write the Terraform configuration

Terraform uses .tf files to describe the desired state of your infrastructure. You will create a single file that defines everything your CI/CD pipeline needs.

  • Create an infra directory inside your project folder by running this command:
mkdir infra
  • Create main.tf inside the infra directory by running:
touch infra/main.tf
  • Open infra/main.tf in your editor and add the Terraform provider configuration, variables, and data source:
terraform {
  required_version = ">= 1.5.7"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 6.28"
    }
  }
}

provider "aws" {
  region = var.aws_region
}

variable "aws_region" {
  description = "AWS region"
  default     = "us-east-1"
}

variable "cluster_name" {
  description = "EKS cluster name"
  default     = "cicd-demo-cluster"
}

variable "github_repo" {
  description = "GitHub repository in format owner/repo"
  type        = string
}

data "aws_availability_zones" "available" {
  state = "available"
}

What does this code do?

The terraform block pins the Terraform version to 1.5.7+ and the AWS provider to 6.28+. This ensures everyone on your team uses compatible versions.

The three variables let you customize the deployment. github_repo has no default because it must be your specific repository. The data source fetches available AZs in your region so the VPC spans multiple zones automatically.

  • Below the data source, add the VPC and EKS module configurations:
module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "${var.cluster_name}-vpc"
  cidr = "10.0.0.0/16"

  azs             = slice(data.aws_availability_zones.available.names, 0, 2)
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24"]

  enable_nat_gateway   = true
  single_nat_gateway   = true
  enable_dns_hostnames = true

  public_subnet_tags = {
    "kubernetes.io/role/elb" = 1
  }

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = 1
  }

  tags = {
    Environment = "dev"
    Terraform   = "true"
  }
}

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 21.0"

  cluster_name    = var.cluster_name
  cluster_version = "1.35"

  endpoint_public_access = true
  enable_cluster_creator_admin_permissions = true

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnets

  eks_managed_node_groups = {
    default = {
      ami_type       = "AL2023_x86_64_STANDARD"
      instance_types = ["t3.medium"]
      min_size       = 1
      max_size       = 2
      desired_size   = 1
    }
  }

  tags = {
    Environment = "dev"
    Terraform   = "true"
  }
}

What does this code do?

The VPC module creates a network with public and private subnets across two availability zones. The subnet tags tell Kubernetes where to place load balancers. A single NAT gateway keeps costs low while giving private subnets outbound internet access.

The EKS module provisions a Kubernetes 1.35 cluster with one t3.medium worker node. enable_cluster_creator_admin_permissions grants your AWS user admin access to the cluster so you can manage it with kubectl after provisioning.

Why use community Terraform modules?

The terraform-aws-modules/eks/aws module is the industry-standard Terraform module for EKS. It handles the complexity of VPC CNI, node groups, IAM roles, and add-ons. Writing raw aws_eks_cluster resources would triple your Terraform code without teaching additional concepts.

  • Below the EKS module, add the ECR repository, GitHub OIDC provider, IAM role, and outputs:
resource "aws_ecr_repository" "app" {
  name                 = "cicd-demo-app"
  image_tag_mutability = "MUTABLE"
  force_delete         = true

  image_scanning_configuration {
    scan_on_push = true
  }
}

resource "aws_iam_openid_connect_provider" "github" {
  url            = "https://token.actions.githubusercontent.com"
  client_id_list = ["sts.amazonaws.com"]
}

data "aws_iam_policy_document" "github_actions_assume" {
  statement {
    actions = ["sts:AssumeRoleWithWebIdentity"]
    effect  = "Allow"

    principals {
      type        = "Federated"
      identifiers = [aws_iam_openid_connect_provider.github.arn]
    }

    condition {
      test     = "StringEquals"
      variable = "token.actions.githubusercontent.com:aud"
      values   = ["sts.amazonaws.com"]
    }

    condition {
      test     = "StringLike"
      variable = "token.actions.githubusercontent.com:sub"
      values   = ["repo:${var.github_repo}:*"]
    }
  }
}

resource "aws_iam_role" "github_actions" {
  name               = "github-actions-cicd-role"
  assume_role_policy = data.aws_iam_policy_document.github_actions_assume.json
}

resource "aws_iam_role_policy_attachment" "ecr_push" {
  role       = aws_iam_role.github_actions.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPowerUser"
}

data "aws_iam_policy_document" "eks_deploy" {
  statement {
    actions = [
      "eks:DescribeCluster",
      "eks:ListClusters"
    ]
    resources = ["*"]
  }
}

resource "aws_iam_policy" "eks_deploy" {
  name   = "github-actions-eks-deploy"
  policy = data.aws_iam_policy_document.eks_deploy.json
}

resource "aws_iam_role_policy_attachment" "eks_deploy" {
  role       = aws_iam_role.github_actions.name
  policy_arn = aws_iam_policy.eks_deploy.arn
}

output "ecr_repository_url" {
  value = aws_ecr_repository.app.repository_url
}

output "cluster_name" {
  value = module.eks.cluster_name
}

output "github_actions_role_arn" {
  value = aws_iam_role.github_actions.arn
}

output "aws_region" {
  value = var.aws_region
}

What does this code do?

The ECR repository stores your Docker images. force_delete = true means Terraform can destroy the repository even if it contains images, which simplifies cleanup later.

The OIDC provider registers GitHub as a trusted identity with AWS. The IAM role uses a condition that scopes trust to your specific repository, so only your repo's workflows can assume the role. Two policy attachments grant the role permission to push images to ECR and describe the EKS cluster.

The outputs expose key values you will need in later steps for configuring the GitHub Actions workflow.

What is OIDC authentication?

OIDC (OpenID Connect) lets GitHub Actions prove its identity to AWS without storing long-lived access keys. When your workflow runs, GitHub mints a short-lived token. AWS verifies that token against the registered OIDC provider and grants temporary credentials scoped to your IAM role.

This eliminates the security risk of static AWS credentials stored in GitHub Secrets.

  • Save infra/main.tf.

✔️ Awesome, I've got everything!

Great. Double check you have saved infra/main.tf with all three sections of code.

ⓧ I'd like to double check the full code

Here is the complete infra/main.tf file. Compare yours against it:

terraform {
  required_version = ">= 1.5.7"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 6.28"
    }
  }
}

provider "aws" {
  region = var.aws_region
}

variable "aws_region" {
  description = "AWS region"
  default     = "us-east-1"
}

variable "cluster_name" {
  description = "EKS cluster name"
  default     = "cicd-demo-cluster"
}

variable "github_repo" {
  description = "GitHub repository in format owner/repo"
  type        = string
}

data "aws_availability_zones" "available" {
  state = "available"
}

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "~> 5.0"

  name = "${var.cluster_name}-vpc"
  cidr = "10.0.0.0/16"

  azs             = slice(data.aws_availability_zones.available.names, 0, 2)
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
  public_subnets  = ["10.0.101.0/24", "10.0.102.0/24"]

  enable_nat_gateway   = true
  single_nat_gateway   = true
  enable_dns_hostnames = true

  public_subnet_tags = {
    "kubernetes.io/role/elb" = 1
  }

  private_subnet_tags = {
    "kubernetes.io/role/internal-elb" = 1
  }

  tags = {
    Environment = "dev"
    Terraform   = "true"
  }
}

module "eks" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 21.0"

  cluster_name    = var.cluster_name
  cluster_version = "1.35"

  endpoint_public_access = true
  enable_cluster_creator_admin_permissions = true

  vpc_id     = module.vpc.vpc_id
  subnet_ids = module.vpc.private_subnets

  eks_managed_node_groups = {
    default = {
      ami_type       = "AL2023_x86_64_STANDARD"
      instance_types = ["t3.medium"]
      min_size       = 1
      max_size       = 2
      desired_size   = 1
    }
  }

  tags = {
    Environment = "dev"
    Terraform   = "true"
  }
}

resource "aws_ecr_repository" "app" {
  name                 = "cicd-demo-app"
  image_tag_mutability = "MUTABLE"
  force_delete         = true

  image_scanning_configuration {
    scan_on_push = true
  }
}

resource "aws_iam_openid_connect_provider" "github" {
  url            = "https://token.actions.githubusercontent.com"
  client_id_list = ["sts.amazonaws.com"]
}

data "aws_iam_policy_document" "github_actions_assume" {
  statement {
    actions = ["sts:AssumeRoleWithWebIdentity"]
    effect  = "Allow"

    principals {
      type        = "Federated"
      identifiers = [aws_iam_openid_connect_provider.github.arn]
    }

    condition {
      test     = "StringEquals"
      variable = "token.actions.githubusercontent.com:aud"
      values   = ["sts.amazonaws.com"]
    }

    condition {
      test     = "StringLike"
      variable = "token.actions.githubusercontent.com:sub"
      values   = ["repo:${var.github_repo}:*"]
    }
  }
}

resource "aws_iam_role" "github_actions" {
  name               = "github-actions-cicd-role"
  assume_role_policy = data.aws_iam_policy_document.github_actions_assume.json
}

resource "aws_iam_role_policy_attachment" "ecr_push" {
  role       = aws_iam_role.github_actions.name
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryPowerUser"
}

data "aws_iam_policy_document" "eks_deploy" {
  statement {
    actions = [
      "eks:DescribeCluster",
      "eks:ListClusters"
    ]
    resources = ["*"]
  }
}

resource "aws_iam_policy" "eks_deploy" {
  name   = "github-actions-eks-deploy"
  policy = data.aws_iam_policy_document.eks_deploy.json
}

resource "aws_iam_role_policy_attachment" "eks_deploy" {
  role       = aws_iam_role.github_actions.name
  policy_arn = aws_iam_policy.eks_deploy.arn
}

output "ecr_repository_url" {
  value = aws_ecr_repository.app.repository_url
}

output "cluster_name" {
  value = module.eks.cluster_name
}

output "github_actions_role_arn" {
  value = aws_iam_role.github_actions.arn
}

output "aws_region" {
  value = var.aws_region
}
Initialize and apply Terraform

With your configuration written, Terraform needs to download the required providers and modules before it can create any resources.

  • Move into the infra directory and initialize Terraform by running:
cd infra
terraform init

You should see a message that says Terraform has been successfully initialized! along with confirmation that providers and modules were downloaded.

Seeing provider or module errors?

  • Check that your required_providers block has source = "hashicorp/aws" and version = ">= 6.28" exactly as shown.
  • Make sure the module source values are spelled correctly: terraform-aws-modules/vpc/aws and terraform-aws-modules/eks/aws.
  • Verify you have internet connectivity. Terraform downloads modules from the public registry.

help me troubleshoot my terraform init errors

Now you will provision all the resources. Terraform will show you a plan of what it intends to create, then ask for confirmation.

  • Apply the Terraform configuration by running:
terraform apply -var="github_repo=[[GITHUB_REPO="your-github-username/your-repo-name"]]"

What value should I use for github_repo?

Use the format owner/repo for the GitHub repository you will push your code to in Step 5. For example, if your GitHub username is janedev and you plan to name your repo eks-cicd-pipeline, enter janedev/eks-cicd-pipeline.

You do not need to have created the repository yet. You just need to know what you will name it.

  • Terraform will display a plan showing all resources it will create. Review the plan, then type yes and press Enter to confirm.

This takes approximately 15 minutes

The EKS cluster is the slowest resource to provision. Terraform will show progress as it creates each resource. Do not close your terminal while this runs.

This is normal. EKS creates a managed Kubernetes control plane, networking, and worker nodes behind the scenes.

When the apply completes, Terraform prints your outputs. You should see ecr_repository_url, cluster_name, github_actions_role_arn, and aws_region displayed with their values.

Save those output values

Copy the github_actions_role_arn and ecr_repository_url values somewhere safe. You will need them when configuring the GitHub Actions workflow in Step 5.

Terraform apply failed?

  • If you see permission errors, confirm your AWS CLI credentials have admin access by running aws sts get-caller-identity.
  • If you see quota errors, check that your account has not exceeded the EKS cluster limit (default is 100 per region).
  • If you see VPC or subnet errors, verify the us-east-1 region has at least two availability zones (it does by default).

help me debug my terraform apply failure

Configure kubectl to connect to the cluster

Your EKS cluster is running, but kubectl on your machine does not know about it yet. You need to update your local kubeconfig so kubectl can communicate with the cluster.

  • Configure kubectl to connect to your new cluster by running:
aws eks update-kubeconfig --name cicd-demo-cluster --region us-east-1

You should see a message confirming the context was updated in your kubeconfig file.

  • Verify the connection by checking the cluster nodes:
kubectl get nodes

You should see one node with a Ready status. This confirms your local machine can communicate with the EKS cluster.

kubectl command not connecting?

  • Make sure your AWS CLI is configured with the same credentials that ran terraform apply. The EKS module grants admin access to the cluster creator.
  • If you see a timeout, check that your internet connection allows outbound traffic on port 443. The EKS API endpoint uses HTTPS.
  • Run aws sts get-caller-identity to confirm you are authenticated as the expected IAM user.

help me connect kubectl to my EKS cluster

Your infrastructure is fully provisioned and kubectl is connected. Next up, you will create the Kubernetes manifests that tell EKS how to run your containerized application.

Create Kubernetes Manifests

Your EKS cluster is running and kubectl is connected to it. Now you need to tell Kubernetes how to run your application.

Kubernetes uses manifest files to define the desired state of your workloads. You describe what you want (two replicas, health checks, resource limits) and Kubernetes continuously works to make it happen.

In this step, get ready to:
  • Write a Deployment manifest with resource limits and health checks.
  • Write a Service manifest to expose the application.
  • Apply the manifests to verify cluster connectivity.
Write the Deployment manifest

A Deployment tells Kubernetes how many copies of your container to run and how to keep them healthy. It includes resource limits so a single pod cannot consume all of a node's CPU or memory.

  • Create a k8s directory inside your project folder by running this command:
mkdir k8s
  • Create a file called deployment.yaml inside the k8s folder and paste the following code:
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cicd-demo-app
  labels:
    app: cicd-demo-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: cicd-demo-app
  template:
    metadata:
      labels:
        app: cicd-demo-app
    spec:
      containers:
        - name: cicd-demo-app
          image: PLACEHOLDER_IMAGE
          ports:
            - containerPort: 3000
          resources:
            requests:
              memory: "64Mi"
              cpu: "50m"
            limits:
              memory: "128Mi"
              cpu: "100m"
          readinessProbe:
            httpGet:
              path: /health
              port: 3000
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /health
              port: 3000
            initialDelaySeconds: 10
            periodSeconds: 30

What does this manifest do?

This Deployment creates 2 replicas (pods) of your application. The selector and labels (app: cicd-demo-app) tie the Deployment to its pods.

The resources block sets guardrails. Each pod requests 50m CPU and 64Mi memory, but cannot exceed 100m CPU and 128Mi memory. This prevents a single pod from starving the node.

The readinessProbe checks /health to decide when a pod is ready to receive traffic. The livenessProbe checks the same endpoint to restart a pod if it becomes unresponsive.

The image is set to PLACEHOLDER_IMAGE for now. Your CI/CD pipeline will replace this with the real ECR image URL on every deployment.

  • Save k8s/deployment.yaml.

YAML indentation errors?

YAML is whitespace-sensitive. Make sure you use spaces (not tabs) and that nested keys are indented by exactly 2 spaces relative to their parent.

Check that containers: is indented under spec: (the inner one inside template:), not the outer spec:.

Help me fix a YAML indentation error in my Kubernetes Deployment manifest.

Write the Service manifest

A Deployment creates pods, but those pods are not accessible from outside the cluster by default. A Service provides a stable network endpoint that routes traffic to your pods.

  • Create a file called service.yaml inside the k8s folder and paste the following code:
apiVersion: v1
kind: Service
metadata:
  name: cicd-demo-app
  labels:
    app: cicd-demo-app
spec:
  type: LoadBalancer
  selector:
    app: cicd-demo-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 3000

What does this Service do?

The type: LoadBalancer tells AWS to provision an Elastic Load Balancer that routes external traffic into your cluster. The Service listens on port 80 and forwards requests to port 3000 on your pods.

The selector field (app: cicd-demo-app) matches the labels on your Deployment's pods. This is how the Service knows which pods to send traffic to.

  • Save k8s/service.yaml.
  • Confirm both files exist by running:
ls k8s/

You should see deployment.yaml and service.yaml listed in the output.

Only see one file?

Make sure you saved both files inside the k8s directory, not in the project root. Check that the filenames end in .yaml (not .yml or .txt).

Help me find my missing Kubernetes manifest file.

✔️ Awesome, I've got everything!

Great. Make sure both files are saved before continuing.

ⓧ I'd like to double check the full code

Here is the complete k8s/deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: cicd-demo-app
  labels:
    app: cicd-demo-app
spec:
  replicas: 2
  selector:
    matchLabels:
      app: cicd-demo-app
  template:
    metadata:
      labels:
        app: cicd-demo-app
    spec:
      containers:
        - name: cicd-demo-app
          image: PLACEHOLDER_IMAGE
          ports:
            - containerPort: 3000
          resources:
            requests:
              memory: "64Mi"
              cpu: "50m"
            limits:
              memory: "128Mi"
              cpu: "100m"
          readinessProbe:
            httpGet:
              path: /health
              port: 3000
            initialDelaySeconds: 5
            periodSeconds: 10
          livenessProbe:
            httpGet:
              path: /health
              port: 3000
            initialDelaySeconds: 10
            periodSeconds: 30

And here is the complete k8s/service.yaml:

apiVersion: v1
kind: Service
metadata:
  name: cicd-demo-app
  labels:
    app: cicd-demo-app
spec:
  type: LoadBalancer
  selector:
    app: cicd-demo-app
  ports:
    - protocol: TCP
      port: 80
      targetPort: 3000
Apply the manifests to verify cluster connectivity

Before wiring up the CI/CD pipeline, it is worth confirming that your cluster accepts these manifests. Applying them manually proves that kubectl can communicate with your EKS cluster.

  • Apply both manifests to your cluster by running these commands:
kubectl apply -f k8s/deployment.yaml
kubectl apply -f k8s/service.yaml

What does kubectl apply do?

The kubectl apply command sends your manifest to the Kubernetes API server. The server validates it, stores the desired state, and begins working to make the actual state match.

  • Verify the Deployment and Service were created by running:
kubectl get deployments
kubectl get services

You should see cicd-demo-app listed under both Deployments and Services. The Deployment will show 0/2 ready pods because the image PLACEHOLDER_IMAGE does not exist yet. That is expected.

Why are my pods not running?

The Deployment references PLACEHOLDER_IMAGE as the container image. Kubernetes cannot pull this because it does not exist in any registry. Your CI/CD pipeline will replace this placeholder with the real ECR image URL on every push.

The important thing is that kubectl apply succeeded. That confirms your cluster is accepting commands from your machine.

Seeing a connection refused error?

This means kubectl cannot reach your EKS cluster. Run kubectl config current-context to check that your context points to cicd-demo-cluster.

If the context is wrong or missing, re-run the kubeconfig update command from the previous step.

Help me fix a kubectl connection refused error when applying manifests to EKS.

Your cluster is connected and your manifests are valid. Next up, you will build the GitHub Actions workflow that replaces that placeholder image with the real thing on every push to main.

Build the GitHub Actions CI/CD Pipeline

Your infrastructure is live and your application deploys successfully when you apply manifests manually. The next goal is to eliminate that manual step entirely.

In this step, you will create a GitHub Actions workflow that automatically builds your Docker image, pushes it to ECR, and deploys it to your EKS cluster every time you push code to main. The workflow uses OIDC authentication so you never store long-lived AWS credentials in GitHub.

In this step, get ready to:
  • Create the GitHub Actions workflow file that builds, pushes, and deploys your app.
  • Push your project to GitHub with the OIDC secret configured.
  • Verify the pipeline runs end-to-end successfully.
Create the GitHub Actions workflow file

GitHub Actions workflows live in a special directory at the root of your repository. The workflow file defines the trigger, permissions, and sequence of steps that run on every push.

  • Create the workflow directory structure by running these commands:
mkdir -p .github/workflows
  • Create a new file called deploy.yml inside .github/workflows/ and paste the following workflow configuration:
name: Build and Deploy to EKS

on:
  push:
    branches: [main]

permissions:
  id-token: write
  contents: read

env:
  AWS_REGION: us-east-1
  ECR_REPOSITORY: cicd-demo-app
  EKS_CLUSTER_NAME: cicd-demo-cluster

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2

      - name: Build, tag, and push image to Amazon ECR
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          IMAGE_TAG: ${{ github.sha }}
        run: |
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:latest .
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest

      - name: Update kubeconfig
        run: |
          aws eks update-kubeconfig --name $EKS_CLUSTER_NAME --region $AWS_REGION

      - name: Deploy to EKS
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          IMAGE_TAG: ${{ github.sha }}
        run: |
          sed -i "s|PLACEHOLDER_IMAGE|$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG|g" k8s/deployment.yaml
          kubectl apply -f k8s/deployment.yaml
          kubectl apply -f k8s/service.yaml
          kubectl rollout status deployment/cicd-demo-app --timeout=120s

What does this workflow do?

This workflow triggers on every push to the main branch and runs through six steps in sequence:

  • permissions: id-token: write allows the runner to request an OIDC token from GitHub, which AWS uses to verify the caller's identity without stored secrets.
  • aws-actions/configure-aws-credentials@v4 exchanges the OIDC token for temporary AWS credentials via the IAM role you created with Terraform.
  • aws-actions/amazon-ecr-login@v2 authenticates Docker to your ECR registry so you can push images.
  • The build step tags each image with the Git commit SHA for traceability, plus a latest tag for convenience.
  • sed -i replaces PLACEHOLDER_IMAGE in your deployment manifest with the actual ECR image URI before applying it to the cluster.
  • kubectl rollout status waits up to 120 seconds for the deployment to complete. If pods fail to start, the workflow fails.

Why OIDC instead of static credentials?

Storing AWS access keys in GitHub secrets is a security anti-pattern. If those keys leak, an attacker has long-lived access to your AWS account.

OIDC solves this by having GitHub prove the runner's identity to AWS using a short-lived token. AWS verifies the token came from your specific repository and branch, then issues temporary credentials that expire after the job finishes. There are no permanent secrets to leak.

  • Save .github/workflows/deploy.yml.

YAML syntax errors?

YAML is indentation-sensitive. Make sure you use spaces (not tabs) and that each nested level is indented by exactly 2 spaces. help me fix YAML indentation in my GitHub Actions workflow

Check that the run: | lines (multi-line shell commands) have the pipe character immediately after the colon with no trailing spaces.

Push your project to GitHub and configure the secret

Your workflow references secrets.AWS_ROLE_ARN to assume the IAM role via OIDC. You need to create a GitHub repository, set that secret, and grant the GitHub Actions role access to your EKS cluster.

  • First, get the ARN of your GitHub Actions IAM role by running:
terraform -chdir=infra output github_actions_role_arn
  • Copy the ARN value from the output. It looks like arn:aws:iam::123456789012:role/github-actions-cicd-role.
  • Initialize a Git repository and make your first commit by running:
git init
git add .
git commit -m "Initial commit: app, infra, and CI/CD pipeline"
  • Create a GitHub repository and push your code by running:
gh repo create cicd-demo-eks --public --source=. --push

What does this command do?

gh repo create creates a new repository on GitHub directly from your terminal. The --source=. flag tells it to use the current directory as the source. The --push flag pushes your local commits to the new remote automatically.

Not logged in to GitHub CLI?

If you see an authentication error, run gh auth login first and follow the prompts to authenticate with your GitHub account. help me authenticate with GitHub CLI

  • Set the AWS_ROLE_ARN secret on your new repository by running:
gh secret set AWS_ROLE_ARN --body "your-role-arn-here"

What value do I paste?

Replace your-role-arn-here with the ARN you copied from the Terraform output earlier. It should look like arn:aws:iam::123456789012:role/github-actions-cicd-role.

  • Grant the GitHub Actions role access to your EKS cluster by creating an access entry:
aws eks create-access-entry --cluster-name cicd-demo-cluster --principal-arn your-role-arn-here --type STANDARD
  • Associate the cluster admin policy so the role can deploy workloads:
aws eks associate-access-policy --cluster-name cicd-demo-cluster --principal-arn your-role-arn-here --policy-arn arn:aws:eks::aws:cluster-access-policy/AmazonEKSClusterAdminPolicy --access-scope type=cluster

Why do I need an access entry?

EKS uses access entries to map IAM roles to Kubernetes permissions. Without this mapping, the GitHub Actions runner can authenticate to AWS but cannot communicate with the Kubernetes API. The AmazonEKSClusterAdminPolicy grants full cluster access, which the deploy job needs to apply manifests and check rollout status.

Access entry already exists?

If you get a conflict error saying the principal ARN already has an access entry, you can skip this step. The entry was already created. help me troubleshoot EKS access entries

Trigger the pipeline and verify it succeeds

Your initial push already triggered the workflow because it landed on the main branch. However, that run happened before you configured the secret and access entry.

  • Trigger a fresh pipeline run by pushing an empty commit:
git commit --allow-empty -m "Trigger CI/CD pipeline"
git push
  • Open your repository in a browser and click the **Actions** tab.
  • Click the most recent workflow run (titled "Trigger CI/CD pipeline").
  • Watch the job progress through each step: checkout, configure credentials, ECR login, build and push, update kubeconfig, and deploy.

What should I see?

Each step shows a green checkmark when it completes successfully. The full pipeline typically takes 2-4 minutes. The final "Deploy to EKS" step should show deployment/cicd-demo-app successfully rolled out.

  • Verify the deployment is running on your cluster by running:
kubectl get pods -l app=cicd-demo-app
  • You should see 2 pods with status Running.

Pipeline failing at the deploy step?

The most common causes are:

  • The AWS_ROLE_ARN secret value has a typo or extra whitespace. Run gh secret list to confirm the secret exists, then re-set it if needed.
  • The EKS access entry was not created for the correct role ARN. Verify with aws eks list-access-entries --cluster-name cicd-demo-cluster.
  • The OIDC trust policy in Terraform does not match your GitHub repository name. Check that the github_repo variable in infra/main.tf matches your actual owner/repo format exactly.

help me debug my GitHub Actions CI/CD pipeline failing at the deploy step

✔️ Awesome, I've got everything!

Great. Double check you have saved .github/workflows/deploy.yml and that your pipeline completed with all green checkmarks.

ⓧ I'd like to double check the full code

Here is the complete workflow file:

name: Build and Deploy to EKS

on:
  push:
    branches: [main]

permissions:
  id-token: write
  contents: read

env:
  AWS_REGION: us-east-1
  ECR_REPOSITORY: cicd-demo-app
  EKS_CLUSTER_NAME: cicd-demo-cluster

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2

      - name: Build, tag, and push image to Amazon ECR
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          IMAGE_TAG: ${{ github.sha }}
        run: |
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
          docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:latest .
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
          docker push $ECR_REGISTRY/$ECR_REPOSITORY:latest

      - name: Update kubeconfig
        run: |
          aws eks update-kubeconfig --name $EKS_CLUSTER_NAME --region $AWS_REGION

      - name: Deploy to EKS
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          IMAGE_TAG: ${{ github.sha }}
        run: |
          sed -i "s|PLACEHOLDER_IMAGE|$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG|g" k8s/deployment.yaml
          kubectl apply -f k8s/deployment.yaml
          kubectl apply -f k8s/service.yaml
          kubectl rollout status deployment/cicd-demo-app --timeout=120s

Your CI/CD pipeline is fully operational. Every push to main now triggers an automated build, push, and deploy cycle with zero manual intervention. Next up, you will add monitoring to your cluster so you can see what is happening inside your pods in real time.

Enable CloudWatch Container Insights

Your CI/CD pipeline deploys code automatically, but how do you know your application is healthy after each deployment?

In this step, you will enable CloudWatch Container Insights on your EKS cluster. This gives you real-time visibility into CPU, memory, network, and pod-level metrics without installing any additional agents manually.

In this step, get ready to:
  • Enable the CloudWatch Observability EKS add-on using the AWS CLI.
  • Navigate the CloudWatch Container Insights dashboard to view cluster, node, and pod metrics.
  • Trigger a deployment and observe the metrics update in real time.
Enable the CloudWatch Observability EKS add-on

The CloudWatch Observability EKS add-on installs a CloudWatch agent and Fluent Bit on your cluster. It collects infrastructure metrics and container logs automatically, then sends them to CloudWatch where you can view curated dashboards.

Before you can install the add-on, you need to set up EKS Pod Identity so the CloudWatch agent has permission to send metrics to AWS.

  • Install the EKS Pod Identity Agent add-on on your cluster by running this command:
aws eks create-addon --cluster-name cicd-demo-cluster --addon-name eks-pod-identity-agent

What is the Pod Identity Agent?

The Pod Identity Agent runs on each node and provides AWS credentials to pods. It works like EC2 instance profiles but for Kubernetes pods, giving each pod only the permissions it needs.

  • Verify the Pod Identity Agent pods are running by running:
kubectl get pods -n kube-system | grep 'eks-pod-identity-agent'

You should see one or more pods with a status of Running.

Pods not showing as Running?

Wait 30-60 seconds and run the command again. The agent pods take a moment to start up. If they remain in a pending state, confirm your node group has the AmazonEKSWorkerNodePolicy attached to its IAM role.

help me troubleshoot the EKS Pod Identity Agent not starting on my cluster

Now you need to create an IAM role that the CloudWatch agent will use. This role needs a trust policy that allows EKS Pod Identity to assume it.

  • Create the trust policy file by running:
cat > cloudwatch-trust-policy.json << 'EOF'
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowEksAuthToAssumeRoleForPodIdentity",
            "Effect": "Allow",
            "Principal": {
                "Service": "pods.eks.amazonaws.com"
            },
            "Action": [
                "sts:AssumeRole",
                "sts:TagSession"
            ]
        }
    ]
}
EOF

What does this trust policy do?

This policy allows the pods.eks.amazonaws.com service to assume this role. The sts:AssumeRole action lets Pod Identity get credentials, and sts:TagSession allows session tagging for traceability.

  • Create the IAM role by running:
aws iam create-role \
  --role-name cloudwatch-observability-role \
  --assume-role-policy-document file://cloudwatch-trust-policy.json
  • Attach the CloudWatchAgentServerPolicy to the role by running:
aws iam attach-role-policy \
  --role-name cloudwatch-observability-role \
  --policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy

What is CloudWatchAgentServerPolicy?

This is an AWS-managed IAM policy that grants the CloudWatch agent permission to send metrics, logs, and traces to CloudWatch. Using a managed policy means AWS keeps the permissions up to date as new features are added.

Now you can install the CloudWatch Observability add-on and link it to the IAM role you just created.

  • Get your AWS account ID by running:
aws sts get-caller-identity --query Account --output text
  • Install the CloudWatch Observability add-on by running the following command, replacing the account ID placeholder with your actual account ID from the previous command:
aws eks create-addon \
  --addon-name amazon-cloudwatch-observability \
  --cluster-name cicd-demo-cluster \
  --pod-identity-associations 'serviceAccount=cloudwatch-agent,roleArn=arn:aws:iam::[[AWS_ACCOUNT_ID="your 12-digit AWS account ID"]]:role/cloudwatch-observability-role'

What does this command do?

The --pod-identity-associations flag tells EKS to map the cloudwatch-agent service account to your IAM role. This way, the CloudWatch agent pods automatically receive AWS credentials without storing any secrets in the cluster.

  • Verify the add-on installed successfully by running:
aws eks describe-addon \
  --cluster-name cicd-demo-cluster \
  --addon-name amazon-cloudwatch-observability \
  --query 'addon.status' \
  --output text

You should see ACTIVE as the output. If you see CREATING, wait a minute and try again.

Seeing DEGRADED or CREATE_FAILED?

Check that the IAM role ARN is correct and that the CloudWatchAgentServerPolicy is attached. Run aws iam list-attached-role-policies --role-name cloudwatch-observability-role to verify.

Also confirm the Pod Identity Agent is running. The CloudWatch add-on depends on it to get credentials.

help me fix a DEGRADED or CREATE_FAILED status on the CloudWatch Observability EKS add-on

Navigate the CloudWatch Container Insights dashboard

Container Insights takes a few minutes to start collecting data after the add-on is installed. Once it does, you get curated dashboards showing cluster, node, and pod-level metrics with no additional configuration.

  • Open the AWS Console in your browser.
  • Search for CloudWatch in the top search bar and open the CloudWatch service.
  • In the left navigation pane, click Insights, then click Container Insights.
  • Use the dropdown near the top to select cicd-demo-cluster as your cluster.

You should see a dashboard with metrics for your cluster. The dropdown at the top lets you switch between different views.

  • Use the top dropdown to switch between EKS Clusters, EKS Nodes, and EKS Pods views.

What metrics are available?

At the cluster level, you see aggregate CPU and memory utilization, network traffic, and the number of running pods. At the node level, you get per-node resource usage. At the pod level, you can drill down into individual containers.

These metrics update automatically. You do not need to configure any scrapers or exporters.

Metrics take a few minutes to appear

If you see an empty dashboard, wait 3-5 minutes for the CloudWatch agent to collect its first batch of metrics. Refresh the page after waiting.

Trigger a deployment and observe metrics updating

The real power of Container Insights shows up during deployments. When new pods roll out, you can watch the metrics change in real time. Let's trigger a fresh deployment through your CI/CD pipeline.

  • Open app/server.js in your code editor.
  • Find this line:
    message: 'Hello from EKS!',
  • Change the message to something different, such as:
    message: 'Hello from EKS with monitoring!',
  • Save the file, then commit and push the change by running:
git add .
git commit -m "Update message to test monitoring"
git push origin main

This triggers your GitHub Actions pipeline. The pipeline builds a new image, pushes it to ECR, and deploys to EKS.

  • Switch back to the CloudWatch Container Insights dashboard in your browser.
  • Select the EKS Pods view from the dropdown.
  • Watch for new pods appearing as the deployment rolls out. You will see old pods terminating and new pods starting.

The CPU and memory graphs update to reflect the new pods spinning up. You can see the brief spike in resource usage during the rolling deployment.

Why is this useful?

In production, Container Insights helps you catch issues immediately. If a deployment causes a memory leak or CPU spike, you see it on the dashboard in real time. You can also set CloudWatch alarms to alert you automatically when metrics cross a threshold.

Not seeing new pods in the dashboard?

First, check that your GitHub Actions pipeline completed successfully. Go to the Actions tab in your GitHub repository and verify the latest workflow run is green.

If the pipeline passed but the dashboard hasn't updated, wait another minute. Container Insights collects metrics in intervals, so there is a short delay.

help me troubleshoot why my Container Insights dashboard is not showing updated metrics after a deployment

You now have full observability into your Kubernetes cluster. Every deployment, scaling event, and resource change is captured and visualized automatically.

Your production CI/CD pipeline is complete. You have infrastructure as code, automated deployments, and monitoring all working together. Next up is the Secret Mission, where you can add even more advanced monitoring with Prometheus and Grafana.

Secret mission

Add Prometheus and Grafana

CloudWatch Container Insights gives you AWS-native monitoring, but professional teams often want deeper observability with custom dashboards. Deploy the kube-prometheus-stack to your cluster for advanced monitoring with Prometheus metrics collection and Grafana visualizations.

Clean Up Your Resources

Clean Up Your Resources

This project uses paid AWS resources that will continue to charge you if left running. Your EKS cluster costs $0.10 per hour for the control plane alone, plus EC2 instance costs for worker nodes.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Cost warning

Your EKS cluster charges approximately $0.10/hour for the control plane plus additional costs for the t3.medium worker node. If left running for 24 hours, that is roughly $2.40 for the control plane alone.

Strongly consider deleting all resources immediately after completing this project unless you have an active use for the cluster.

Resources you used:

  • EKS cluster (cicd-demo-cluster) with managed node group.
  • VPC, subnets, NAT gateway, and networking resources.
  • ECR repository (cicd-demo-app).
  • IAM OIDC provider, GitHub Actions IAM role, and EKS deploy policy.
  • CloudWatch Observability add-on and its IAM role.
  • Local project files, Docker images, and GitHub repository.

Keep everything running

No action needed. Choose this if you are still actively building on this cluster or plan to return within the next few hours.

  • Your EKS cluster will continue to incur charges at $0.10/hour for the control plane plus EC2 costs for the worker node.
  • Your ECR repository stores images but incurs minimal storage cost.
  • Docker Desktop and your local project files remain on your machine at no cost.

Pause - I'll come back to this later

EKS does not support pausing. You cannot stop the control plane or worker nodes without destroying them. If you want to reduce costs while keeping your infrastructure code, delete the AWS resources and re-provision them later with Terraform.

  • Keep your local project files and Terraform configuration. These allow you to re-create everything with a single terraform apply later.
  • Run terraform destroy to tear down all AWS resources (see the Delete tab for full instructions).
  • Your GitHub repository and workflow file will remain intact and ready for the next deploy.

Delete - I don't want to use this again

Remove all project resources from AWS, your local machine, and GitHub. This is the recommended path unless you have an active use for the cluster.

Destroy all AWS resources with Terraform
  • Open your terminal and navigate to your infra directory.
  • Run the following command to destroy all Terraform-managed resources:
cd infra
terraform destroy
  • When prompted, type yes and press Enter to confirm.
  • Wait for the destroy to complete. This takes approximately 10-15 minutes as EKS resources are removed.
Delete the CloudWatch IAM role

The IAM role you created for CloudWatch Container Insights was not managed by Terraform. Delete it from the AWS Console.

  • Open the AWS Console and navigate to IAM > Roles.
  • Search for the CloudWatch agent role you created in Step 6.
  • Select the role and click Delete.
  • Type the role name to confirm deletion.
Verify in the AWS Console
  • In the AWS Console, navigate to EKS and confirm no clusters remain.
  • Navigate to ECR and confirm the cicd-demo-app repository is gone.
  • Navigate to VPC and confirm the cicd-demo-cluster-vpc is removed.
  • Navigate to IAM > Identity providers and confirm the GitHub OIDC provider is removed.
Remove local Docker images
  • Remove all Docker images and build cache from this project by running:
docker system prune -a --volumes -f
Delete local project files

macOS

  • Remove your project folder by running:
rm -rf ~/Desktop/cicd-demo-pipeline

Windows

  • Remove your project folder by running:
Remove-Item -Recurse -Force ~\Desktop\cicd-demo-pipeline
Delete the GitHub repository
  • Go to your repository on GitHub.
  • Click Settings in the top navigation bar.
  • Scroll down to the Danger Zone section.
  • Click Delete this repository and follow the confirmation prompts.

That's a Wrap!

That's a Wrap!

Nice work! You just built a production-grade CI/CD pipeline from scratch.

You've learned how to:

  • Provisioned an EKS cluster, VPC, ECR repository, and OIDC authentication entirely with Terraform.
  • Containerized a Node.js application with Docker and a multi-stage build.
  • Automated the full build-push-deploy cycle with GitHub Actions using OIDC (no stored secrets).
  • Monitored your cluster with CloudWatch Container Insights.
  • Added Prometheus and Grafana for advanced observability with custom dashboards.

Ready to quiz yourself?