Jenkins CI/CD Pipeline with AWS ECR

Build a Jenkins pipeline that builds and pushes Docker images to AWS ECR.

Introduction

30 Second Summary

Every time a developer pushes code, someone (or something) needs to build it, test it, and ship it to production. In the real world, that "something" is a CI/CD pipeline.

In this project, you will build a complete CI/CD pipeline using Jenkins, Docker, and AWS. You'll containerize a Python Flask app, write automated tests, and configure Jenkins to build and push Docker images to a private container registry.

What You'll Build

A locally-hosted Jenkins pipeline that automatically lints, tests, builds, and pushes a Dockerized Flask application to AWS ECR.

By the end of this project, you'll have:

  • A Jenkins server running inside Docker with Docker-in-Docker capability for building container images.
  • An AWS ECR repository provisioned entirely through Terraform (infrastructure as code).
  • A multi-stage Jenkinsfile that lints, tests, builds, and pushes versioned Docker images on demand.
  • Secret Mission: Add a security scanning stage that checks Docker images for vulnerabilities before they reach your registry.

Are there any prerequisites?

This project assumes you already have an AWS account with the AWS CLI configured, a GitHub account, Docker Desktop installed, and VS Code installed. You do not need prior Jenkins experience.

Set Up Jenkins with Docker-in-Docker

To automate building and pushing Docker images, you need a CI/CD server that can orchestrate the entire workflow. Jenkins is one of the most widely used automation servers in enterprise DevOps.

In this step, you will run Jenkins locally inside Docker using a Docker-in-Docker (dind) setup. This lets Jenkins build Docker images from inside its own container. You will also install Terraform, the infrastructure-as-code tool you will use later to provision cloud resources.

In this step, get ready to:
  • Install the Terraform CLI.
  • Create Docker-in-Docker infrastructure for Jenkins.
  • Run Jenkins and complete initial setup.
Install Terraform CLI

Terraform lets you define cloud infrastructure in code files instead of clicking through a web console. You will use it in a later step to create your AWS ECR repository.

  • Check whether Terraform is already installed by running this command:
terraform --version

✔️ I see a version number

Terraform is ready. Move on to the next substep.

ⓧ I see a version lower than 1.0

You need Terraform 1.0 or higher for this project. Upgrade by running these commands:

brew tap hashicorp/tap
brew install hashicorp/tap/terraform

After the install finishes, verify the new version:

terraform --version

You should now see a version of 1.0 or higher.

ⓧ Command not found

  • Install Terraform using Homebrew (macOS) by running these commands:
brew tap hashicorp/tap
brew install hashicorp/tap/terraform

What do these commands do?

The first command adds HashiCorp's official Homebrew tap (a repository of installable packages). The second command installs the Terraform CLI from that tap.

On Windows, install Terraform using Chocolatey:

choco install terraform
  • After installation completes, verify it worked by running:
terraform --version

You should see a version number like 1.15.5 or similar.

Create the Docker network and Docker-in-Docker container

Jenkins needs to build Docker images, but it runs inside a container itself. The solution is Docker-in-Docker (dind), a separate container that provides a Docker daemon for Jenkins to use.

  • Create a dedicated Docker network for Jenkins and the dind container to communicate by running:
docker network create jenkins
  • Start the Docker-in-Docker container by running this command:
docker run --name jenkins-docker --rm --detach --privileged --network jenkins --network-alias docker --env DOCKER_TLS_CERTDIR=/certs --volume jenkins-docker-certs:/certs/client --volume jenkins-data:/var/jenkins_home --publish 2376:2376 docker:dind --storage-driver overlay2

What does this command do?

This starts a privileged Docker container that runs its own Docker daemon. The --network jenkins flag connects it to the network you just created. The --network-alias docker flag lets Jenkins find this container by the hostname docker.

TLS certificates are shared via the jenkins-docker-certs volume so that Jenkins can securely connect to the Docker daemon.

You should see Docker pull the docker:dind image and then print a container ID confirming the container started.

Seeing a network conflict error?

If Docker says the network already exists, you may have created it in a previous attempt. Run docker network rm jenkins first, then re-run both commands.

If you see a permissions error, make sure Docker Desktop is running.

Help me troubleshoot my Docker network or dind container startup error.

Now you need to build a custom Jenkins image that includes the Docker CLI, the AWS CLI, and the Jenkins plugins your pipeline requires.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir jenkins-cicd-pipeline
cd jenkins-cicd-pipeline
  • Create a jenkins subfolder for the custom Dockerfile by running:
mkdir jenkins
  • Open this project in VS Code. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type VS Code and press Enter to open it.
  • In VS Code, click File in the top menu bar.
  • Click Open Folder.
  • Select the jenkins-cicd-pipeline folder on your Desktop, then click Open.
  • In the VS Code Explorer sidebar, right-click the jenkins folder and select New File.
  • Name the file Dockerfile and press Enter.
  • Paste the following code into jenkins/Dockerfile:
FROM jenkins/jenkins:2.555.3-jdk21

USER root

RUN apt-get update && apt-get install -y \
    lsb-release \
    ca-certificates \
    curl \
    python3 \
    python3-pip \
    python3-venv \
    unzip \
    && install -m 0755 -d /etc/apt/keyrings \
    && curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc \
    && chmod a+r /etc/apt/keyrings/docker.asc \
    && echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] \
    https://download.docker.com/linux/debian $(. /etc/os-release && echo \"$VERSION_CODENAME\") stable" \
    | tee /etc/apt/sources.list.d/docker.list > /dev/null \
    && apt-get update && apt-get install -y docker-ce-cli \
    && apt-get clean && rm -rf /var/lib/apt/lists/*

RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-$(uname -m).zip" -o "awscliv2.zip" \
    && unzip awscliv2.zip \
    && ./aws/install \
    && rm -rf aws awscliv2.zip

USER jenkins

RUN jenkins-plugin-cli --plugins "docker-workflow aws-credentials pipeline-stage-view git workflow-aggregator"

What does this Dockerfile do?

  • Starts from the official Jenkins LTS image (2.555.3 with JDK 21).
  • Switches to root to install system packages including Python 3, the Docker CLI (so Jenkins can talk to the dind container), and the AWS CLI (so Jenkins can push to ECR).
  • Switches back to the jenkins user and pre-installs the Jenkins plugins needed for Docker pipelines, AWS credentials, and Git integration.
  • Save jenkins/Dockerfile.
  • Build the custom Jenkins image by running this command in your terminal (make sure you are in the jenkins-cicd-pipeline folder):
docker build -t myjenkins-blueocean:2.555.3-1 ./jenkins

The build takes 2-3 minutes. You should see each layer completing and a final message showing Successfully tagged myjenkins-blueocean:2.555.3-1.

Build failing or timing out?

Make sure Docker Desktop is running and that you have an active internet connection. The build downloads packages from apt repositories and AWS.

If you see a permissions error, confirm you are running the command from the jenkins-cicd-pipeline folder (not from inside the jenkins subfolder).

Help me fix my Jenkins Docker image build error.

✔️ Awesome, I've got everything!

Great. Double check you have saved jenkins/Dockerfile and that the build completed successfully.

ⓧ I'd like to double check the full code

FROM jenkins/jenkins:2.555.3-jdk21

USER root

RUN apt-get update && apt-get install -y \
    lsb-release \
    ca-certificates \
    curl \
    python3 \
    python3-pip \
    python3-venv \
    unzip \
    && install -m 0755 -d /etc/apt/keyrings \
    && curl -fsSL https://download.docker.com/linux/debian/gpg -o /etc/apt/keyrings/docker.asc \
    && chmod a+r /etc/apt/keyrings/docker.asc \
    && echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] \
    https://download.docker.com/linux/debian $(. /etc/os-release && echo \"$VERSION_CODENAME\") stable" \
    | tee /etc/apt/sources.list.d/docker.list > /dev/null \
    && apt-get update && apt-get install -y docker-ce-cli \
    && apt-get clean && rm -rf /var/lib/apt/lists/*

RUN curl "https://awscli.amazonaws.com/awscli-exe-linux-$(uname -m).zip" -o "awscliv2.zip" \
    && unzip awscliv2.zip \
    && ./aws/install \
    && rm -rf aws awscliv2.zip

USER jenkins

RUN jenkins-plugin-cli --plugins "docker-workflow aws-credentials pipeline-stage-view git workflow-aggregator"
Run Jenkins and complete the setup wizard

With the dind container running and your custom image built, you can now start Jenkins itself.

  • Start the Jenkins container by running this command:
docker run --name jenkins-blueocean --restart=on-failure --detach --network jenkins --env DOCKER_HOST=tcp://docker:2376 --env DOCKER_CERT_PATH=/certs/client --env DOCKER_TLS_VERIFY=1 --publish 8080:8080 --publish 50000:50000 --volume jenkins-data:/var/jenkins_home --volume jenkins-docker-certs:/certs/client:ro myjenkins-blueocean:2.555.3-1

What does this command do?

This launches your custom Jenkins image on the jenkins network. The DOCKER_HOST environment variable tells Jenkins to connect to the dind container at tcp://docker:2376 for building images.

Port 8080 exposes the Jenkins web interface. The shared volumes give Jenkins access to the TLS certificates and persist your Jenkins data between restarts.

  • Retrieve the initial admin password by running:
docker exec jenkins-blueocean cat /var/jenkins_home/secrets/initialAdminPassword

Copy the password string that appears in your terminal. You will paste it in the next step.

  • Open your browser and navigate to http://localhost:8080.
  • Paste the admin password into the Administrator password field and click Continue.
  • On the Customize Jenkins page, click Install suggested plugins. Wait for the plugins to finish installing.
  • When prompted to create a first admin user, fill in a username and password you will remember. Click Save and Continue.
  • On the Instance Configuration page, keep the default URL (http://localhost:8080/) and click Save and Finish.
  • Click Start using Jenkins.

You should now see the Jenkins dashboard with a welcome message.

Page not loading at localhost:8080?

Wait 30-60 seconds after starting the container. Jenkins takes time to boot. If it still does not load, check the container is running with docker ps and look for jenkins-blueocean in the list.

If the container is not listed, run docker logs jenkins-blueocean to see what went wrong.

Help me troubleshoot why Jenkins is not loading at localhost:8080.

Now verify that Jenkins can access both Docker and the AWS CLI from inside a pipeline.

  • From the Jenkins dashboard, click New Item in the left sidebar.
  • Enter test-pipeline as the item name.
  • Select Pipeline and click OK.
  • Scroll down to the Pipeline section.
  • In the Script text area, paste this test pipeline:
pipeline {
    agent any
    stages {
        stage('Verify Tools') {
            steps {
                sh 'docker --version'
                sh 'aws --version'
            }
        }
    }
}
  • Click Save.
  • Click Build Now in the left sidebar.
  • Wait for the build to complete. Click on build #1 in the Build History panel, then click Console Output.

You should see output showing the Docker version and the AWS CLI version, confirming both tools are available to Jenkins.

Build failed with a red icon?

Check the Console Output for the specific error. If you see a Docker connection error, make sure the jenkins-docker container is still running (docker ps). If it stopped, restart it with the dind command from earlier.

If you see a permission denied error for Docker, the TLS certificates may not be shared correctly. Remove both containers and volumes, then re-run the dind and Jenkins commands from the beginning of this substep.

Help me debug my Jenkins test pipeline failure.

  • Once the test passes, delete the test job. Click Dashboard in the top left, hover over test-pipeline, click the dropdown arrow, and select Delete Pipeline.

Your Jenkins server is fully operational with Docker-in-Docker capability. Next up, you will create the Flask application and test script that your pipeline will build and deploy.

Create the Flask App and Bash Test Script

Your Jenkins server is running and ready to execute pipelines. Now you need something for it to build.

In this step, you will create a Python Flask application, containerize it with a Dockerfile, and write a Bash test script that verifies the app works. These files form the codebase that your CI/CD pipeline will build, test, and push in later steps.

In this step, get ready to:
  • Create a Flask health-check API with a Dockerfile.
  • Write a Bash test script that validates the app's health endpoint.
  • Push the initial codebase to GitHub.
Initialize the repository and create the Flask app

Every CI/CD project starts with a version-controlled repository. You will initialize Git in your project folder and build out the application code that Jenkins will eventually automate.

  • Switch back to your terminal and navigate to your jenkins-cicd-pipeline folder (the one you created in Step 1) by running this command:
cd ~/Desktop/jenkins-cicd-pipeline
  • Initialize a Git repository with main as the default branch by running this command:
git init -b main

What does this command do?

The -b main flag sets the initial branch name to main instead of the default master. This keeps your repository aligned with the */main branch that your Jenkinsfile will reference later.

Now create the Flask application. This is a minimal Python web server with a health-check endpoint that your pipeline will test.

  • Create a new file called app.py in the root of your jenkins-cicd-pipeline folder and paste the following code:
from flask import Flask, jsonify
import os

app = Flask(__name__)

@app.route("/health")
def health():
    return jsonify({"status": "healthy", "version": os.getenv("APP_VERSION", "1.0.0")}), 200

@app.route("/")
def home():
    return jsonify({"message": "Flask CI/CD Demo", "endpoints": ["/health"]}), 200

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

What does this code do?

This is a minimal Flask API with two endpoints:

  • /health returns a JSON response with the app's status and version number. The version comes from the APP_VERSION environment variable, defaulting to 1.0.0.
  • / returns a welcome message listing available endpoints.
  • The if __name__ block starts the server on port 5000, listening on all network interfaces so Docker containers can reach it.
  • Save app.py.

Next, create the dependencies file and the Dockerfile that will package your app into a container image.

  • Create a file called requirements.txt in the root of your project folder and paste this single line:
flask==3.1.1
  • Save requirements.txt.
  • Create a new file called Dockerfile (no file extension) in the root of your project folder and paste the following:
FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .

EXPOSE 5000

ENV APP_VERSION=1.0.0

CMD ["python", "app.py"]

What does this Dockerfile do?

This Dockerfile packages your Flask app into a lightweight container image:

  • FROM python:3.12-slim starts from a minimal Python base image.
  • COPY requirements.txt and RUN pip install install Flask before copying the app code. This ordering leverages Docker's layer caching so dependencies are only reinstalled when requirements.txt changes.
  • ENV APP_VERSION=1.0.0 sets the version that the /health endpoint reports.
  • CMD starts the Flask server when the container runs.
  • Save Dockerfile.
  • Confirm all your new files exist by running:
ls

You should see app.py, Dockerfile, requirements.txt, and the jenkins/ folder from Step 1 in the output.

Don't see all the files?

Make sure you created each file in the root of jenkins-cicd-pipeline, not inside the jenkins/ subfolder. The root Dockerfile is for your Flask app. The jenkins/Dockerfile from Step 1 is for your custom Jenkins image.

Help me find my missing files

✔️ Awesome, I've got everything!

Great. Double check you have saved all three files before moving on.

ⓧ I'd like to double check the full code

Here is app.py:

from flask import Flask, jsonify
import os

app = Flask(__name__)

@app.route("/health")
def health():
    return jsonify({"status": "healthy", "version": os.getenv("APP_VERSION", "1.0.0")}), 200

@app.route("/")
def home():
    return jsonify({"message": "Flask CI/CD Demo", "endpoints": ["/health"]}), 200

if __name__ == "__main__":
    app.run(host="0.0.0.0", port=5000)

Here is requirements.txt:

flask==3.1.1

Here is Dockerfile:

FROM python:3.12-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app.py .

EXPOSE 5000

ENV APP_VERSION=1.0.0

CMD ["python", "app.py"]
Write the Bash test script

Your Jenkins pipeline will run automated tests against the Flask app before building the Docker image. You need a test script that starts the app, hits the health endpoint, and reports pass or fail.

  • Create a new file called test.sh in the root of your project folder and paste the following code:
#!/bin/bash
set -e

echo "Starting Flask app for testing..."
python app.py &
APP_PID=$!

sleep 2

echo "Running health check..."
RESPONSE=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:5000/health)

kill $APP_PID

if [ "$RESPONSE" -eq 200 ]; then
    echo "Health check PASSED (HTTP $RESPONSE)"
    exit 0
else
    echo "Health check FAILED (HTTP $RESPONSE)"
    exit 1
fi

What does this script do?

This script automates a health check against your Flask app:

  • set -e makes the script exit immediately if any command fails unexpectedly.
  • python app.py & starts the Flask server in the background. $! captures its process ID so you can stop it later.
  • curl -s -o /dev/null -w "%{http_code}" sends a request to /health and captures only the HTTP status code.
  • After stopping the server with kill, the script checks if the status was 200. It exits with code 0 (pass) or 1 (fail) so Jenkins can detect test failures.
  • Save test.sh.
  • Make the script executable by running this command:
chmod +x test.sh

Now verify the test script works by running it locally. You will need a Python virtual environment with Flask installed.

  • Create a virtual environment and install the project dependencies by running these commands:
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
  • Run the test script to verify everything works:
./test.sh

You should see output ending with Health check PASSED (HTTP 200).

Seeing "command not found" or "connection refused"?

If you see a Python-related error, make sure your virtual environment is activated. You should see (venv) at the start of your terminal prompt.

If you see a curl connection error, the Flask app may not have had time to start. Try increasing the sleep 2 value in test.sh to sleep 4 and run it again.

Help me debug my test.sh script

  • Deactivate the virtual environment now that testing is complete:
deactivate

✔️ Awesome, I've got everything!

Great. Double check you have saved test.sh before continuing.

ⓧ I'd like to double check the full code

Here is the complete test.sh:

#!/bin/bash
set -e

echo "Starting Flask app for testing..."
python app.py &
APP_PID=$!

sleep 2

echo "Running health check..."
RESPONSE=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:5000/health)

kill $APP_PID

if [ "$RESPONSE" -eq 200 ]; then
    echo "Health check PASSED (HTTP $RESPONSE)"
    exit 0
else
    echo "Health check FAILED (HTTP $RESPONSE)"
    exit 1
fi
Push the codebase to GitHub

Jenkins will pull your code from a GitHub repository. You need to create a remote repository and push all your files to it.

  • Open your browser and go to github.com/new.
  • Enter jenkins-cicd-pipeline as the repository name.
  • Leave the repository set to Public.
  • Do not check any boxes for README, .gitignore, or license.
  • Click Create repository.

Why a public repository?

A public repo lets Jenkins clone it without needing a GitHub access token. For a real-world project you would use a private repo with credentials, but public keeps this project focused on the pipeline itself.

  • Back in your terminal, add a .gitignore to keep the virtual environment out of version control by running:
echo "venv/" > .gitignore
  • Stage all files and create your first commit by running these commands:
git add .
git commit -m "Initial commit: Flask app, Dockerfile, and test script"
  • Connect your local repository to GitHub and push by running these commands (replace the URL with your own repository URL from the GitHub page):
git remote add origin [[REPO_URL="https://github.com/your-username/jenkins-cicd-pipeline.git"]]
git push -u origin main

Where do I find my repository URL?

After you clicked Create repository, GitHub shows a page with setup instructions. Look for the HTTPS URL at the top of that page. It follows the pattern https://github.com/your-username/jenkins-cicd-pipeline.git.

  • Refresh your GitHub repository page in the browser.

You should see app.py, Dockerfile, requirements.txt, test.sh, and the jenkins/ folder listed in your repository.

Push rejected or authentication failed?

If Git asks for a password, GitHub no longer accepts passwords for HTTPS pushes. You need a Personal Access Token. Go to Settings > Developer settings > Personal access tokens > Tokens (classic) on GitHub, generate a token with repo scope, and use it as the password when prompted.

Help me push my code to GitHub

Your application code is committed and hosted on GitHub. Next up, you will use Terraform to provision an AWS ECR repository where your Docker images will be stored.

Provision AWS ECR with Terraform

Your Jenkins server is running and your Flask app is pushed to GitHub. Now you need somewhere to store the Docker images that your pipeline will build.

In this step, you will use Terraform to provision an AWS ECR repository as your private container registry. You will also create a dedicated IAM user with scoped permissions so Jenkins can push images securely.

In this step, get ready to:
  • Write Terraform configuration to create an ECR repository with a lifecycle policy.
  • Run Terraform to provision the infrastructure on AWS.
  • Store IAM credentials in Jenkins for pipeline access.
Create the Terraform configuration for ECR

Terraform lets you define cloud infrastructure in code files. You will create a terraform/ directory with two files: one for the ECR repository and one for the IAM user.

  • In your terminal, make sure you are inside your project folder (the one containing app.py).
  • Create the terraform/ directory by running this command:
mkdir terraform
  • Create the terraform/main.tf file in VS Code. This file defines the AWS provider, the ECR repository, a lifecycle policy, and an output for the repository URI.
  • Paste the following code into terraform/main.tf:
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
  required_version = ">= 1.0"
}

provider "aws" {
  region = "us-east-1"
}

resource "aws_ecr_repository" "flask_app" {
  name                 = "flask-cicd-demo"
  image_tag_mutability = "MUTABLE"

  image_scanning_configuration {
    scan_on_push = true
  }

  tags = {
    Project     = "jenkins-cicd-demo"
    ManagedBy   = "terraform"
  }
}

resource "aws_ecr_lifecycle_policy" "flask_app" {
  repository = aws_ecr_repository.flask_app.name

  policy = jsonencode({
    rules = [
      {
        rulePriority = 1
        description  = "Keep only last 5 images"
        selection = {
          tagStatus   = "any"
          countType   = "imageCountMoreThan"
          countNumber = 5
        }
        action = {
          type = "expire"
        }
      }
    ]
  })
}

output "repository_uri" {
  value       = aws_ecr_repository.flask_app.repository_url
  description = "The URI of the ECR repository"
}

What does this code do?

The terraform block pins the AWS provider to version ~> 5.0 and requires Terraform 1.0 or higher.

The aws_ecr_repository resource creates a private container registry named flask-cicd-demo with automatic vulnerability scanning on every push.

The lifecycle policy automatically expires old images once you have more than 5, keeping storage costs at zero within the free tier.

  • Now create a second file terraform/iam.tf for the IAM user and scoped policy.
  • Paste the following code into terraform/iam.tf:
resource "aws_iam_user" "jenkins_ecr" {
  name = "jenkins-ecr-push"
  tags = {
    Project   = "jenkins-cicd-demo"
    ManagedBy = "terraform"
  }
}

resource "aws_iam_user_policy" "jenkins_ecr_policy" {
  name = "jenkins-ecr-push-policy"
  user = aws_iam_user.jenkins_ecr.name

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "ecr:GetAuthorizationToken"
        ]
        Resource = "*"
      },
      {
        Effect = "Allow"
        Action = [
          "ecr:BatchCheckLayerAvailability",
          "ecr:GetDownloadUrlForLayer",
          "ecr:BatchGetImage",
          "ecr:PutImage",
          "ecr:InitiateLayerUpload",
          "ecr:UploadLayerPart",
          "ecr:CompleteLayerUpload",
          "ecr:DescribeRepositories",
          "ecr:ListImages"
        ]
        Resource = aws_ecr_repository.flask_app.arn
      }
    ]
  })
}

resource "aws_iam_access_key" "jenkins_ecr" {
  user = aws_iam_user.jenkins_ecr.name
}

output "jenkins_aws_access_key_id" {
  value       = aws_iam_access_key.jenkins_ecr.id
  description = "Access key ID for Jenkins ECR user"
}

output "jenkins_aws_secret_access_key" {
  value       = aws_iam_access_key.jenkins_ecr.secret
  sensitive   = true
  description = "Secret access key for Jenkins ECR user"
}

What does this code do?

This creates an IAM user named jenkins-ecr-push with an inline policy scoped to only the flask-cicd-demo ECR repository.

The first statement allows ecr:GetAuthorizationToken on all resources (required for Docker login). The second statement restricts push/pull actions to your specific repository ARN.

The access key outputs give you the credentials to store in Jenkins. The secret key is marked sensitive so Terraform redacts it from normal output.

Why infrastructure as code instead of the AWS Console?

Creating resources through the AWS Console is quick for exploration, but it leaves no record of what you built or how to reproduce it. Terraform configuration files serve as documentation, can be version-controlled, and let you tear down everything with a single command.

In production, teams use infrastructure as code to ensure consistent environments across dev, staging, and production.

  • Save both terraform/main.tf and terraform/iam.tf.
  • Confirm the files exist by running:
ls terraform/

You should see main.tf and iam.tf listed in the output.

Don't see both files?

Make sure you created the files inside the terraform/ directory, not in the project root. Check that you saved both files in VS Code before running ls.

help me find my Terraform files

Run Terraform to create the ECR repository

With the configuration files ready, you can now initialize Terraform and apply the changes to create your AWS resources.

  • Initialize Terraform by running this command from your project root:
cd terraform
terraform init

What does terraform init do?

This command downloads the AWS provider plugin (specified in your required_providers block) and sets up the .terraform/ directory. You only need to run it once per project, or when you change provider versions.

You should see a message ending with Terraform has been successfully initialized! in your terminal.

Seeing a provider error?

If you see an error about credentials, make sure your AWS CLI is configured with valid access keys. Run aws sts get-caller-identity to verify your AWS CLI is authenticated.

help me fix my Terraform init error

  • Apply the Terraform configuration to create the resources by running:
terraform apply
  • Terraform will show you a plan of the resources it will create. Type yes when prompted to confirm.

What should I see?

After confirming, Terraform creates the ECR repository, lifecycle policy, IAM user, inline policy, and access key. The output includes three values: repository_uri, jenkins_aws_access_key_id, and jenkins_aws_secret_access_key (marked as sensitive).

  • Copy the repository_uri value from the output and save it somewhere. It follows the format <account-id>.dkr.ecr.us-east-1.amazonaws.com/flask-cicd-demo.
  • Retrieve the secret access key by running:
terraform output -raw jenkins_aws_secret_access_key
  • Copy and save both the access key ID (from the terraform apply output) and the secret access key (from the command above). You will need them in the next substep.

Keep these credentials safe

Do not commit these credentials to Git or share them in plain text. You will store them securely in Jenkins in the next substep and never reference them directly in code.

Terraform apply failed?

If you see a permissions error, your AWS CLI user needs IAM and ECR permissions. Check that your configured AWS CLI credentials have admin access or at minimum iam:* and ecr:* permissions.

help me fix my Terraform apply error

Store AWS credentials in Jenkins

Your pipeline needs to authenticate with AWS to push images to ECR. Jenkins stores secrets securely using its built-in credential manager, so you never hardcode keys in your Jenkinsfile.

You will add two credentials: one for the IAM access keys (so Jenkins can push to ECR) and one for your AWS account ID (used to construct the ECR URI).

  • Open Jenkins in your browser at http://localhost:8080.
  • Click **Manage Jenkins** in the left sidebar.
  • Click **Credentials**.
  • Under **Stores scoped to Jenkins**, click **System**.
  • Click **Global credentials (unrestricted)**.
  • Click **Add Credentials**.

You are now in the credential creation form. First you will add the AWS access keys.

  • In the **Kind** dropdown, select **AWS Credentials**.
  • Set **Scope** to **Global**.
  • In the **ID** field, enter aws-ecr-credentials.
  • In the **Description** field, enter Jenkins ECR push credentials.
  • Paste the IAM access key ID from the Terraform output into the **Access Key ID** field.
  • Paste the secret access key into the **Secret Access Key** field.
  • Click **OK** to save.

Why the ID matters

The ID aws-ecr-credentials is what your Jenkinsfile references in the withCredentials block. If the ID does not match exactly, the pipeline will fail to find the credentials at runtime.

Now add a second credential for your AWS account ID.

  • Click **Add Credentials** again from the **Global credentials (unrestricted)** page.
  • In the **Kind** dropdown, select **Secret text**.
  • Set **Scope** to **Global**.
  • In the **Secret** field, paste your 12-digit AWS account ID. You can find this in the repository_uri output (the number before .dkr.ecr).
  • In the **ID** field, enter aws-account-id.
  • In the **Description** field, enter AWS Account ID.
  • Click **OK** to save.

You should now see both aws-ecr-credentials and aws-account-id listed on the Global credentials page.

Don't see AWS Credentials in the Kind dropdown?

The aws-credentials plugin must be installed for this option to appear. Your custom Jenkins image includes it, but if you skipped that step or rebuilt without it, go to **Manage Jenkins** > **Plugins** > **Available plugins**, search for AWS Credentials, and install it.

help me find the AWS Credentials option in Jenkins

Finally, push the Terraform files to GitHub so your repository contains the full infrastructure definition.

  • Navigate back to the project root by running:
cd ..
  • Stage, commit, and push the Terraform files by running:
git add terraform/
git commit -m "Add Terraform configs for ECR and IAM"
git push origin main

Your GitHub repository now contains the full infrastructure-as-code configuration alongside your application code.

Git push rejected?

If you see a rejection about the remote having changes you don't have locally, run git pull origin main --rebase first, then try the push again.

help me fix my git push issue

✔️ Awesome, I've got everything!

Great. Double check that you have saved both Terraform files and pushed them to GitHub.

ⓧ I'd like to double check the full code

Here is the complete terraform/main.tf:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
  required_version = ">= 1.0"
}

provider "aws" {
  region = "us-east-1"
}

resource "aws_ecr_repository" "flask_app" {
  name                 = "flask-cicd-demo"
  image_tag_mutability = "MUTABLE"

  image_scanning_configuration {
    scan_on_push = true
  }

  tags = {
    Project     = "jenkins-cicd-demo"
    ManagedBy   = "terraform"
  }
}

resource "aws_ecr_lifecycle_policy" "flask_app" {
  repository = aws_ecr_repository.flask_app.name

  policy = jsonencode({
    rules = [
      {
        rulePriority = 1
        description  = "Keep only last 5 images"
        selection = {
          tagStatus   = "any"
          countType   = "imageCountMoreThan"
          countNumber = 5
        }
        action = {
          type = "expire"
        }
      }
    ]
  })
}

output "repository_uri" {
  value       = aws_ecr_repository.flask_app.repository_url
  description = "The URI of the ECR repository"
}

Here is the complete terraform/iam.tf:

resource "aws_iam_user" "jenkins_ecr" {
  name = "jenkins-ecr-push"
  tags = {
    Project   = "jenkins-cicd-demo"
    ManagedBy = "terraform"
  }
}

resource "aws_iam_user_policy" "jenkins_ecr_policy" {
  name = "jenkins-ecr-push-policy"
  user = aws_iam_user.jenkins_ecr.name

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "ecr:GetAuthorizationToken"
        ]
        Resource = "*"
      },
      {
        Effect = "Allow"
        Action = [
          "ecr:BatchCheckLayerAvailability",
          "ecr:GetDownloadUrlForLayer",
          "ecr:BatchGetImage",
          "ecr:PutImage",
          "ecr:InitiateLayerUpload",
          "ecr:UploadLayerPart",
          "ecr:CompleteLayerUpload",
          "ecr:DescribeRepositories",
          "ecr:ListImages"
        ]
        Resource = aws_ecr_repository.flask_app.arn
      }
    ]
  })
}

resource "aws_iam_access_key" "jenkins_ecr" {
  user = aws_iam_user.jenkins_ecr.name
}

output "jenkins_aws_access_key_id" {
  value       = aws_iam_access_key.jenkins_ecr.id
  description = "Access key ID for Jenkins ECR user"
}

output "jenkins_aws_secret_access_key" {
  value       = aws_iam_access_key.jenkins_ecr.secret
  sensitive   = true
  description = "Secret access key for Jenkins ECR user"
}

Your ECR repository is live and Jenkins has the credentials it needs to push images. Next up, you will write the Jenkinsfile that ties everything together into an automated pipeline.

Write the Jenkinsfile and Run the Pipeline

You have a Flask app, a Docker image definition, a test script, and an ECR repository ready to receive images. The missing piece is the automation that ties everything together.

A Jenkinsfile is a text file that defines your entire pipeline as code. Instead of clicking through a UI to configure build steps, you describe every stage (lint, test, build, push) in a single file that lives in your repository alongside your application code.

In this step, get ready to:
  • Create a Jenkinsfile defining a multi-stage declarative pipeline.
  • Configure a Jenkins Pipeline job to pull from your GitHub repository.
  • Run the pipeline and verify your Docker image appears in ECR.
Create the Jenkinsfile

The Jenkinsfile defines four stages that run in sequence: Lint checks code style, Test runs your health check script, Build creates and tags the Docker image, and Push authenticates to ECR and uploads the image.

  • In VS Code, create a new file called Jenkinsfile in the root of your project (the same level as app.py and Dockerfile).
  • Add the following pipeline definition to Jenkinsfile:
pipeline {
    agent any

    environment {
        AWS_ACCOUNT_ID = credentials('aws-account-id')
        AWS_DEFAULT_REGION = 'us-east-1'
        IMAGE_REPO_NAME = 'flask-cicd-demo'
        IMAGE_TAG = "${env.BUILD_NUMBER}"
        REPOSITORY_URI = "${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${IMAGE_REPO_NAME}"
    }

    stages {
        stage('Lint') {
            steps {
                sh 'pip install flake8 && flake8 app.py --max-line-length=120'
            }
        }

        stage('Test') {
            steps {
                sh 'pip install -r requirements.txt'
                sh 'chmod +x test.sh && ./test.sh'
            }
        }

        stage('Build') {
            steps {
                sh "docker build -t ${IMAGE_REPO_NAME}:${IMAGE_TAG} ."
                sh "docker tag ${IMAGE_REPO_NAME}:${IMAGE_TAG} ${REPOSITORY_URI}:${IMAGE_TAG}"
                sh "docker tag ${IMAGE_REPO_NAME}:${IMAGE_TAG} ${REPOSITORY_URI}:latest"
            }
        }

        stage('Push') {
            steps {
                withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', credentialsId: 'aws-ecr-credentials']]) {
                    sh "aws ecr get-login-password --region ${AWS_DEFAULT_REGION} | docker login --username AWS --password-stdin ${REPOSITORY_URI}"
                    sh "docker push ${REPOSITORY_URI}:${IMAGE_TAG}"
                    sh "docker push ${REPOSITORY_URI}:latest"
                }
            }
        }
    }

    post {
        always {
            sh "docker rmi ${IMAGE_REPO_NAME}:${IMAGE_TAG} || true"
            sh "docker rmi ${REPOSITORY_URI}:${IMAGE_TAG} || true"
            sh "docker rmi ${REPOSITORY_URI}:latest || true"
        }
    }
}

What does this pipeline do?

The environment block pulls your AWS account ID from the Jenkins credential you stored earlier and constructs the full ECR repository URI dynamically. Each build gets a unique tag from Jenkins' built-in BUILD_NUMBER variable.

  • Lint installs flake8 and checks app.py for style issues.
  • Test installs Flask, then runs your test.sh health check script.
  • Build creates the Docker image and tags it with both the build number and latest.
  • Push authenticates to ECR using the aws-ecr-credentials credential, then pushes both tags.
  • post always cleans up local Docker images after every run to save disk space. The || true ensures cleanup never fails the build.
  • Save Jenkinsfile.
  • Confirm the file exists at the root of your project by running:
ls Jenkinsfile

You should see Jenkinsfile listed in the output.

Don't see the file?

Make sure you created the file in your project root directory, not inside another folder like terraform/ or jenkins/.

help me find where my Jenkinsfile ended up

✔️ Awesome, I've got everything!

Great. Double check you have saved Jenkinsfile before moving on.

ⓧ I'd like to double check the full code

pipeline {
    agent any

    environment {
        AWS_ACCOUNT_ID = credentials('aws-account-id')
        AWS_DEFAULT_REGION = 'us-east-1'
        IMAGE_REPO_NAME = 'flask-cicd-demo'
        IMAGE_TAG = "${env.BUILD_NUMBER}"
        REPOSITORY_URI = "${AWS_ACCOUNT_ID}.dkr.ecr.${AWS_DEFAULT_REGION}.amazonaws.com/${IMAGE_REPO_NAME}"
    }

    stages {
        stage('Lint') {
            steps {
                sh 'pip install flake8 && flake8 app.py --max-line-length=120'
            }
        }

        stage('Test') {
            steps {
                sh 'pip install -r requirements.txt'
                sh 'chmod +x test.sh && ./test.sh'
            }
        }

        stage('Build') {
            steps {
                sh "docker build -t ${IMAGE_REPO_NAME}:${IMAGE_TAG} ."
                sh "docker tag ${IMAGE_REPO_NAME}:${IMAGE_TAG} ${REPOSITORY_URI}:${IMAGE_TAG}"
                sh "docker tag ${IMAGE_REPO_NAME}:${IMAGE_TAG} ${REPOSITORY_URI}:latest"
            }
        }

        stage('Push') {
            steps {
                withCredentials([[$class: 'AmazonWebServicesCredentialsBinding', credentialsId: 'aws-ecr-credentials']]) {
                    sh "aws ecr get-login-password --region ${AWS_DEFAULT_REGION} | docker login --username AWS --password-stdin ${REPOSITORY_URI}"
                    sh "docker push ${REPOSITORY_URI}:${IMAGE_TAG}"
                    sh "docker push ${REPOSITORY_URI}:latest"
                }
            }
        }
    }

    post {
        always {
            sh "docker rmi ${IMAGE_REPO_NAME}:${IMAGE_TAG} || true"
            sh "docker rmi ${REPOSITORY_URI}:${IMAGE_TAG} || true"
            sh "docker rmi ${REPOSITORY_URI}:latest || true"
        }
    }
}
  • Push the Jenkinsfile to GitHub by running these commands:
git add Jenkinsfile
git commit -m "Add Jenkinsfile with CI/CD pipeline"
git push origin main

Why push before creating the Jenkins job?

The Jenkins Pipeline job you are about to create pulls code from GitHub. The Jenkinsfile needs to be on the main branch before Jenkins can find and execute it.

Push rejected or authentication error?

Make sure you have push access to the repository and your Git credentials are configured. If using HTTPS, you may need a personal access token instead of a password.

help me fix my git push authentication error

Create the Jenkins Pipeline job

Now you will tell Jenkins where to find your Jenkinsfile so it can execute the pipeline automatically.

  • Open Jenkins in your browser at http://localhost:8080.
  • Click New Item in the left sidebar.
  • Enter flask-cicd-pipeline as the item name.
  • Select Pipeline as the project type.
  • Click OK.

This opens the Pipeline configuration page.

  • Scroll down to the Pipeline section at the bottom of the page.
  • In the Definition dropdown, select Pipeline script from SCM.
  • In the SCM dropdown, select Git.
  • In the Repository URL field, paste your GitHub repository URL (the HTTPS URL ending in .git).
  • In the Branch Specifier field, change it to */main.
  • In the Script Path field, ensure it says Jenkinsfile.
  • Click Save.

What is Pipeline script from SCM?

SCM stands for Source Code Management. This setting tells Jenkins to look inside your Git repository for a file called Jenkinsfile and execute whatever pipeline it defines.

This is the pipeline-as-code pattern. Your pipeline definition lives in the same repository as your application, versioned together.

  • Click Build Now in the left sidebar to trigger the pipeline.

Don't see Build Now?

Make sure you clicked Save on the configuration page first. The Build Now option only appears on the job's main page after saving.

help me find the Build Now button in Jenkins

Monitor the pipeline and verify the image in ECR

After clicking Build Now, Jenkins fetches your Jenkinsfile from GitHub and starts executing each stage in order.

  • Watch the build progress in the Stage View on the job page. Each stage (Lint, Test, Build, Push) shows green when it passes.
  • If a stage fails, click the red box and then click Console Output to see the full log.

How long does the first build take?

The first run takes longer because Jenkins downloads dependencies and Docker pulls the base image. Subsequent builds are faster thanks to Docker layer caching.

  • Once all four stages show green, verify the image landed in ECR by running this command in your local terminal:
aws ecr list-images --repository-name flask-cicd-demo --region us-east-1

You should see output listing image tags including 1 (your first build number) and latest.

Pipeline failing at a specific stage?

  • Lint fails: Check Console Output for specific flake8 errors in app.py. Fix the style issues, commit, push, and re-run.
  • Test fails: The Flask app may not have started in time. Check that test.sh has a sleep 2 before the curl command.
  • Build fails: Verify the Docker-in-Docker container (jenkins-docker) is still running with docker ps.
  • Push fails: Check that the aws-ecr-credentials credential ID in Jenkins matches exactly and that the IAM user has the correct ECR permissions.

help me debug my Jenkins pipeline failure

Your CI/CD pipeline is fully operational. Every time you push code to GitHub and click Build Now, Jenkins lints, tests, builds, and pushes a fresh Docker image to ECR. Next up, you will extend this pipeline with security scanning in the Secret Mission.

Secret mission

Add Docker Image Security Scanning

Your pipeline builds and pushes images to ECR, but how do you know those images are safe? In production, teams add security scanning stages that catch vulnerabilities before they reach the container registry. Add Trivy to your pipeline to block unsafe images from being pushed.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. Jenkins runs locally in Docker and ECR stays within the free tier for this project, so there are no ongoing costs unless your AWS free tier has expired.

Cost warning

AWS ECR provides 500 MB of free storage per month for private repositories during your first year. After that, storage costs $0.10 per GB per month. The images from this project use minimal storage, but if your free tier year has passed, delete the ECR repository to avoid charges.

Resources you used:

  • AWS ECR repository (flask-cicd-demo) with pushed Docker images.
  • IAM user (jenkins-ecr-push) with scoped ECR policy and access key.
  • Local Docker containers (jenkins-blueocean and jenkins-docker).
  • Docker volumes (jenkins-data and jenkins-docker-certs).
  • Docker network (jenkins).
  • Custom Jenkins Docker image (myjenkins-blueocean:2.555.3-1).
  • Local project files on your Desktop.

Keep everything running

No action needed. Choose this if you are still actively building or plan to extend the pipeline soon.

  • Jenkins stays available at http://localhost:8080 as long as the containers are running.
  • ECR costs nothing within free tier limits (500 MB per month for one year).
  • Your Terraform state file in the terraform/ directory tracks all AWS resources.

Pause - I'll come back to this later

Shut down running processes to free up memory. Your files and AWS resources stay intact for when you return.

  • Stop the Jenkins containers by running:
docker stop jenkins-blueocean jenkins-docker
  • Restart them anytime by running:
docker start jenkins-docker jenkins-blueocean
  • Your ECR repository and images remain in AWS at no cost within the free tier.
  • Your project files and Terraform state stay on disk, ready for you to resume.

Delete - I don't want to use this again

Remove all project resources and start fresh. This is permanent.

First, destroy the AWS resources (ECR repository and IAM user) using Terraform:

  • Navigate to the terraform/ directory inside your project folder and run:
cd terraform
terraform destroy
  • Type yes when prompted to confirm. This removes the ECR repository, all images inside it, the IAM user, and the IAM policy.

Next, remove the Jenkins Docker containers and volumes:

  • Force-remove the containers by running:
docker rm -f jenkins-blueocean jenkins-docker
  • Remove the Docker volumes by running:
docker volume rm jenkins-data jenkins-docker-certs
  • Remove the Docker network by running:
docker network rm jenkins
  • Remove the custom Jenkins image by running:
docker rmi myjenkins-blueocean:2.555.3-1

Finally, remove the local project files:

  • Delete your project folder from the Desktop by running:
cd ~/Desktop
rm -rf jenkins-cicd-pipeline

Verify everything is gone

Run docker ps -a to confirm no Jenkins containers remain. Run docker volume ls to check that the volumes are gone. In the AWS Console, navigate to ECR and confirm the flask-cicd-demo repository no longer appears.

That's a Wrap!

That's a Wrap!

Nice work! You just built a production-grade CI/CD pipeline from scratch, proving you can design, implement, and operate a container-based delivery workflow end to end.

You've learned how to:

  • Set up Jenkins in Docker with Docker-in-Docker capability to build container images inside a CI pipeline.
  • Provisioned an AWS ECR repository and scoped IAM credentials using Terraform for infrastructure as code.
  • Wrote a declarative Jenkinsfile defining a multi-stage CI/CD pipeline that lints, tests, builds, and pushes Docker images to ECR.
  • Secret Mission: Added container security scanning with Trivy to catch vulnerabilities before deployment.

Ready to quiz yourself?