Scale Jenkins with Spot Instance Agents

Dynamically provision EC2 Spot Instance workers for Jenkins CI/CD pipelines.

Introduction

30 Second Summary

Every time a developer pushes code, a CI/CD pipeline kicks off to build and test it. Most teams run these builds on servers that sit idle 90% of the time, burning money around the clock.

In this project, you will set up a Jenkins Controller-Agent architecture on AWS that dynamically spins up EC2 Spot Instance workers only when a pipeline job needs them. When the job finishes, the workers terminate automatically, saving up to 90% on compute costs.

What You'll Build

You will build a Jenkins controller that uses the EC2 Fleet Plugin to provision Spot Instance agents through an Auto Scaling Group, then run a Declarative Pipeline on those dynamically provisioned agents.

By the end of this project, you'll have:

A Jenkins Controller running on a free-tier EC2 instance that dynamically provisions worker agents.
An Auto Scaling Group with a Launch Template configured to spin up Spot Instance workers with Java pre-installed.
A Declarative Jenkins Pipeline that triggers Spot agent provisioning, runs build and test stages, and scales back to zero when done.
A Secret Mission to diversify instance types across multiple families and verify your agents are actually running on Spot Instances.

Are there any prerequisites?

You will need an AWS account with free tier credits remaining. A basic understanding of Jenkins (what pipelines are) and AWS (navigating the console) will help you follow along.

Launch the Jenkins Controller EC2 Instance

Every Jenkins setup needs a server to run on. In this project, that server is an EC2 instance in AWS that acts as your Jenkins controller, the brain that orchestrates all your CI/CD pipelines.

Before you can install Jenkins, the instance needs proper networking rules so you can access it from your browser and so future worker agents can communicate with it. You'll set all of that up in this step.

In this step, get ready to:

Create a key pair and security group for your Jenkins controller.
Launch an EC2 instance running Amazon Linux 2023.
Install Jenkins and complete the initial setup wizard.

Create a key pair and security group

A key pair lets you securely SSH into your EC2 instance. A security group controls which network traffic is allowed in and out of your instance.

Open the AWS Console in your browser and sign in.
Search for EC2 in the top search bar and select EC2.
In the left sidebar, click Key Pairs under the Network & Security section.
Click Create key pair.
Enter jenkins-controller-key as the Name.
Select .pem as the Private key file format.
Click Create key pair.

Your browser automatically downloads a jenkins-controller-key.pem file. Keep this file in your Downloads folder. You will need it to SSH into your instance.

Why .pem format?

The .pem format works with OpenSSH, which is built into both macOS and Windows 10+. This means you can SSH into your instance directly from your terminal without installing extra software.

Now create the security group that controls network access to your Jenkins controller.

In the left sidebar, click Security Groups under the Network & Security section.
Click Create security group.
Enter jenkins-controller-sg as the Security group name.
Enter Security group for Jenkins controller in the Description field.
Leave the VPC set to your default VPC.

The security group needs three inbound rules. Each one opens a specific port for a different purpose.

Under Inbound rules, click Add rule.
Set the Type to SSH. This auto-fills port 22.
Set the Source to My IP.
Click Add rule again.
Set the Type to Custom TCP.
Enter 8080 in the Port range field.
Set the Source to Anywhere-IPv4 (0.0.0.0/0).
Click Add rule one more time.
Set the Type to Custom TCP.
Enter 50000 in the Port range field.
Set the Source to Custom and enter 172.31.0.0/16.
Click Create security group.

What do these three ports do?

Port 22 (SSH) lets you connect to the instance from your computer's terminal.
Port 8080 is where the Jenkins web UI runs. Opening it to 0.0.0.0/0 lets you access Jenkins from your browser.
Port 50000 is used for JNLP agent communication. Restricting it to 172.31.0.0/16 (your default VPC range) means only instances inside your AWS network can use it.

Not sure what your VPC CIDR is?

172.31.0.0/16 is the default CIDR range for every AWS account's default VPC. If you have never modified your VPC settings, this value is correct. If you have customized your VPC, you can find your CIDR by going to the VPC service in the AWS Console and checking the IPv4 CIDR column on your default VPC.

Help me find my VPC CIDR range in the AWS Console.

Launch the Jenkins controller instance

With your key pair and security group ready, you can launch the EC2 instance that will run your Jenkins controller.

In the left sidebar, click Instances.
Click Launch instances.
Enter jenkins-controller in the Name field.
Under Application and OS Images (AMI), select Amazon Linux 2023. It should be marked as Free tier eligible.
Under Instance type, select t3.micro (free tier eligible).
Under Key pair (login), select jenkins-controller-key from the dropdown.
Under Network settings, click Select existing security group.
Select jenkins-controller-sg from the list.
Leave all other settings at their defaults.
Click Launch instance.

Click View all instances.
Wait until the Instance state column shows Running and the Status check column shows 2/2 checks passed.

Pro tip: grab your public IP now

Click on your jenkins-controller instance to open its details. Copy the Public IPv4 DNS value. You will need it for both SSH and the Jenkins web UI.

SSH into the controller and install Jenkins

Your EC2 instance is running, but it is a blank server. You need to SSH in and install Jenkins on it.

Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.

Before connecting, you need to set the correct permissions on your key file so SSH accepts it.

macOS

Set the correct file permissions by running this command:

chmod 400 ~/Downloads/jenkins-controller-key.pem

Connect to your EC2 instance by running this command:

ssh -i ~/Downloads/jenkins-controller-key.pem ec2-user@[[EC2_PUBLIC_DNS="your EC2 public DNS"]]

When prompted to confirm the connection, type yes and press Enter.

Windows

Connect to your EC2 instance by running this command:

ssh -i "$env:USERPROFILE\Downloads\jenkins-controller-key.pem" ec2-user@[[EC2_PUBLIC_DNS="your EC2 public DNS"]]

When prompted to confirm the connection, type yes and press Enter.

You should see a prompt that starts with [ec2-user@ip-...]. This means you are inside your EC2 instance.

SSH connection refused or timed out?

Make sure you are using the Public IPv4 DNS (not the private IP) from your instance details page.
Check that your jenkins-controller-sg security group has port 22 open to My IP. If your IP address has changed since you created the rule, update it.
On macOS, verify you ran chmod 400 on the .pem file. SSH refuses keys with overly permissive file permissions.
My SSH connection to EC2 is failing. Help me troubleshoot.

Now that you are inside the instance, you can install Jenkins. Jenkins runs on Java, so you will install Java first, then Jenkins itself.

Update the system packages and add the Jenkins repository by running these commands:

sudo dnf update -y
sudo wget -O /etc/yum.repos.d/jenkins.repo https://pkg.jenkins.io/rpm-stable/jenkins.repo
sudo rpm --import https://pkg.jenkins.io/rpm-stable/jenkins.io-2026.key
sudo dnf upgrade -y

What do these commands do?

dnf update updates your system's package list to the latest versions available.
The wget command downloads the official Jenkins package repository definition so your system knows where to find Jenkins.
rpm --import imports the Jenkins GPG signing key so your system can verify the packages are authentic.
dnf upgrade applies any available upgrades, including the newly added Jenkins repo metadata.

Install Java and Jenkins by running these commands:

sudo dnf install java-21-amazon-corretto -y
sudo dnf install jenkins -y

Why Java 21?

Jenkins LTS requires Java 21 or later. Amazon Corretto is Amazon's free, production-ready distribution of the OpenJDK, and it comes pre-configured for Amazon Linux 2023.

Start Jenkins and set it to launch automatically on boot by running these commands:

sudo systemctl enable jenkins
sudo systemctl start jenkins
sudo systemctl status jenkins

You should see output showing Active: active (running) in green text. This confirms Jenkins is running on your instance.

Jenkins status shows failed or inactive?

Run java -version to confirm Java installed successfully. You should see a version starting with 21.
Check the Jenkins logs for more details by running sudo journalctl -u jenkins.
If Java is missing, re-run the install command and then restart Jenkins with sudo systemctl restart jenkins.
Jenkins is showing as failed or inactive after starting. Help me debug.

Jenkins generates a one-time admin password during its first startup. You need this password to unlock the Jenkins web UI.

Retrieve the initial admin password by running this command:

sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Copy the long string of characters that appears. This is your Jenkins admin password.

Now open the Jenkins web UI in your browser to complete the setup wizard.

Open a new browser tab.
Navigate to your Jenkins instance by entering this URL, replacing the placeholder with your instance's public IP:

http://[[EC2_PUBLIC_IP="your EC2 public IPv4 address"]]:8080

Paste the admin password you copied into the Administrator password field.
Click Continue.
Click Install suggested plugins.

Jenkins will download and install a set of commonly used plugins. This takes a few minutes.

Once the plugins finish installing, Jenkins prompts you to create your admin user.
Fill in a username, password, full name, and email address of your choice.
Click Save and Continue.
On the Instance Configuration page, leave the Jenkins URL as the default.
Click Save and Finish.
Click Start using Jenkins.

You should see the Jenkins dashboard with a Welcome to Jenkins! message. Your controller is ready.

Can't reach the Jenkins page in your browser?

Make sure you are using http:// and not https://. Jenkins does not use HTTPS by default.
Check that your jenkins-controller-sg security group has port 8080 open to 0.0.0.0/0.
Confirm Jenkins is running by going back to your SSH session and running sudo systemctl status jenkins.
I cannot access Jenkins on port 8080 in my browser. Help me troubleshoot.

Your Jenkins controller is up and running on EC2. Next, you'll create the IAM role and credentials that let the EC2 Fleet Plugin manage Spot Instance workers on your behalf.

Create the IAM Role and Credentials for the EC2 Fleet Plugin

Your Jenkins controller is installed and running on its EC2 instance. Now it's time to give Jenkins the permissions it needs to manage EC2 instances in your AWS account.

The EC2 Fleet Plugin communicates directly with AWS APIs to spin up and tear down Spot Instance agents. Without the right IAM permissions, the plugin can't see your Auto Scaling Group or launch any workers.

In this step, get ready to:

Create an IAM policy scoped to the EC2 Fleet Plugin's API needs.
Create an IAM user with programmatic access for Jenkins.
Create an IAM role for the Spot worker instances.

Create an IAM policy for the EC2 Fleet Plugin

The EC2 Fleet Plugin needs specific AWS permissions to describe, launch, and terminate EC2 instances on your behalf. You'll define these permissions in a custom IAM policy.

Open the AWS Console and search for IAM in the top search bar.
Click Policies in the left sidebar.
Click Create policy.
Click the JSON tab in the policy editor.
Clear any existing content in the editor, then paste the following policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeSpotFleetInstances",
        "ec2:ModifySpotFleetRequest",
        "ec2:CreateTags",
        "ec2:DescribeRegions",
        "ec2:DescribeInstances",
        "ec2:TerminateInstances",
        "ec2:DescribeInstanceStatus",
        "ec2:DescribeSpotFleetRequests",
        "ec2:DescribeFleets",
        "ec2:DescribeFleetInstances",
        "ec2:ModifyFleet",
        "ec2:DescribeInstanceTypes"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "autoscaling:DescribeAutoScalingGroups",
        "autoscaling:TerminateInstanceInAutoScalingGroup",
        "autoscaling:UpdateAutoScalingGroup"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "iam:ListInstanceProfiles",
        "iam:ListRoles"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": "iam:PassRole",
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "iam:PassedToService": "ec2.amazonaws.com"
        }
      }
    }
  ]
}

What does this policy allow?

This policy has four permission groups, each scoped to what the EC2 Fleet Plugin actually needs:

EC2 actions let the plugin discover, launch, tag, and terminate worker instances.
Auto Scaling actions let the plugin read your ASG configuration and adjust its desired capacity to spin workers up or down.
IAM list actions let the plugin enumerate available roles and instance profiles when you configure the cloud in Jenkins.
iam:PassRole lets the plugin assign an IAM role to newly launched EC2 instances. The condition restricts this to EC2 only.

Click Next.
Enter JenkinsFleetPluginPolicy in the Policy name field.
Click Create policy.

You should see a green success banner confirming that JenkinsFleetPluginPolicy has been created.

Seeing a JSON validation error?

Make sure you cleared all the default content before pasting. Leftover braces or text will break the JSON.
Check that you copied the entire policy, including the opening and closing curly braces.

Help me fix my IAM policy JSON.

Create an IAM user with programmatic access

Jenkins needs a set of AWS credentials (an access key ID and secret access key) to authenticate with the AWS APIs. You'll create a dedicated IAM user for this purpose.

In the IAM Console, click Users in the left sidebar.
Click Create user.
Enter jenkins-fleet-user in the User name field.
Click Next.
Select Attach policies directly.
Search for JenkinsFleetPluginPolicy and check the box next to it.
Click Next, then click Create user.

Now you need to generate an access key so Jenkins can authenticate as this user.

Click on jenkins-fleet-user in the users list to open the user's summary page.
Click the Security credentials tab.
Scroll down to Access keys and click Create access key.
Select Application running outside AWS as the use case.
Click Next, then click Create access key.

Save your credentials now

Copy both the Access Key ID and the Secret Access Key to a secure location (such as a password manager or a local text file). The Secret Access Key will not be shown again after you leave this page.

You will enter these credentials into Jenkins in a later step. Do not share them or commit them to version control.

Click Done.

You should see the access key listed under the Security credentials tab with a status of Active.

Don't see the policy attached to your user?

Click the Permissions tab on the user's summary page. You should see JenkinsFleetPluginPolicy listed.
If the policy is missing, click Add permissions, select Attach policies directly, search for JenkinsFleetPluginPolicy, and attach it.

Help me attach my IAM policy to the jenkins-fleet-user.

Create an IAM role for the EC2 worker instances

Each Spot worker instance needs an IAM role so the Launch Template can reference an instance profile. For this project, the role doesn't need any attached policies.

In the IAM Console, click Roles in the left sidebar.
Click Create role.
Select AWS service as the trusted entity type.
Select EC2 as the use case.
Click Next.
Skip the permissions step. Do not attach any policies. Click Next again.
Enter jenkins-agent-role in the Role name field.
Click Create role.

You should see a success banner confirming that jenkins-agent-role has been created.

Why does this role have no policies?

The role serves as an identity for your worker instances. AWS automatically creates an instance profile with the same name as the role (jenkins-agent-role), and your Launch Template will reference that instance profile in the next step.

Since the worker agents only run Jenkins build commands and don't call any AWS APIs themselves, no additional permissions are needed.

Seeing an error when creating the role?

If you see a "role already exists" error, a role named jenkins-agent-role was already created in your account. Search for it in the roles list and confirm it has EC2 as the trusted entity.
Make sure you selected AWS service (not Custom trust policy) as the trusted entity type.

Help me create the jenkins-agent-role IAM role.

Your IAM permissions are locked in. Next up, you'll create the Launch Template and Auto Scaling Group that define how your Spot worker instances are configured and launched.

Create the Launch Template and Auto Scaling Group

Your IAM credentials are locked and loaded. Now it's time to define what your worker agents look like and how AWS scales them up and down.

A Launch Template is a blueprint that tells AWS exactly how to configure each worker instance. An Auto Scaling Group reads that blueprint and spins instances up or down based on demand.

In this step, get ready to:

Create a security group that lets the controller SSH into worker agents.
Build a Launch Template that defines the worker agent configuration with Java pre-installed.
Set up an Auto Scaling Group configured for Spot Instances across multiple instance types.

Create a security group for worker agents

The EC2 Fleet Plugin connects to worker agents over SSH. The worker agents need their own security group that only allows SSH connections from the Jenkins controller.

In the AWS Console, search for EC2 and open the EC2 dashboard.
In the left sidebar, click Security Groups under Network & Security.
Click Create security group.
Enter jenkins-agent-sg in the Security group name field.
Enter Security group for Jenkins Spot agents in the Description field.
Keep the default VPC selected.
Under Inbound rules, click Add rule.
For Type, select SSH.
For Source, select Custom.
In the search box, type jenkins-controller-sg and select the matching security group from the dropdown.
Leave the default outbound rule (all traffic) as is.
Click Create security group.

Why SSH from the controller security group?

By setting the inbound source to jenkins-controller-sg instead of an IP address, you're telling AWS: "only allow SSH connections from instances that belong to the controller's security group." This is more secure and more flexible than hardcoding an IP.

If the controller's IP changes, the rule still works because it references the security group, not the address.

You should see a success banner confirming that jenkins-agent-sg was created.

Don't see jenkins-controller-sg in the source dropdown?

Make sure you're creating this security group in the same VPC and region as your Jenkins controller instance from Step 1. Security groups from different VPCs won't appear in the dropdown.

Check the region selector in the top-right corner of the AWS Console. It should match the region where you launched your jenkins-controller instance.

Still stuck? Help me find my controller security group in the source dropdown.

Create the Launch Template

A Launch Template defines every detail about an EC2 instance: the AMI, the key pair, the security group, and any scripts that should run at boot. The Auto Scaling Group reads this blueprint whenever it needs to spin up a new worker.

In the EC2 Console left sidebar, click Launch Templates under Instances.
Click Create launch template.
Enter jenkins-spot-agent in the Launch template name field.
Under Application and OS Images (Amazon Machine Image), search for and select Amazon Linux 2023 (the same AMI your controller uses).
Leave the Instance type field empty. The Auto Scaling Group will handle instance type selection in the next substep.
Under Key pair (login), select jenkins-controller-key. This is the same key pair your controller uses, so Jenkins can SSH into agents.
Under Network settings, click Select existing security group.
Choose jenkins-agent-sg from the dropdown.
Expand the Advanced details section at the bottom of the page.
For IAM instance profile, select jenkins-agent-role.
Scroll all the way down to the User data field and paste the following script:

#!/bin/bash
sudo dnf install java-21-amazon-corretto -y

Why does User Data install Java?

Jenkins agents need Java to run the Jenkins agent process. Without Java pre-installed, the controller can SSH into the worker but can't start the agent software.

The User Data script runs automatically the first time an instance boots. By the time Jenkins tries to connect, Java is already installed and ready.

Click Create launch template.

You should see a success message confirming that jenkins-spot-agent was created.

Don't see jenkins-agent-role in the IAM instance profile dropdown?

The IAM role jenkins-agent-role was created in Step 2. If it doesn't appear, go back to the IAM Console and confirm the role exists with EC2 as the trusted entity.

Need help? I can't find jenkins-agent-role in the IAM instance profile dropdown when creating my Launch Template.

Create the Auto Scaling Group with Spot Instances

The Auto Scaling Group is the engine that actually creates and terminates EC2 instances. You'll configure it to use Spot Instances, which cost up to 90% less than On-Demand pricing.

You'll also specify multiple instance types so AWS can pull from a wider pool of spare capacity. This increases the chance that a Spot Instance is available when Jenkins needs one.

In the EC2 Console left sidebar, click Auto Scaling Groups under Auto Scaling.
Click Create Auto Scaling group.
Enter jenkins-spot-agents-asg in the Auto Scaling group name field.
For Launch template, select jenkins-spot-agent.
Click Next.

You're now on the instance launch options page. This is where you configure Spot pricing and instance type diversification.

Under Instance type requirements, select Override launch template.
Add t3.medium as the first instance type.
Click Add instance type and add t3a.medium as the second instance type.

Why multiple instance types?

Spot Instances draw from AWS's unused capacity. If you only request one instance type, there might not be any spare capacity available for that specific type.

By listing both t3.medium and t3a.medium, you double the pool of spare capacity AWS can pull from. The Secret Mission takes this further with even more instance families.

Under Instance purchase options, select Combine purchase options and instances.
Set On-Demand base capacity to 0.
Set On-Demand percentage above base to 0%.
For Spot allocation strategy, select price-capacity-optimized.

What does price-capacity-optimized mean?

This allocation strategy balances the lowest price with the highest available capacity. It reduces the chance of your Spot Instance getting interrupted mid-build.

Setting both On-Demand values to 0 means every instance will be a Spot Instance. No On-Demand instances will be launched.

Under Network, select at least two availability zones and subnets in your region.
Click Next.
On the Configure advanced options page, leave everything as default and click Next.

You're now on the group size and scaling page. This controls how many instances the ASG manages.

Set Desired capacity to 0.
Set Minimum capacity to 0.
Set Maximum capacity to 2.

Why start at 0 desired capacity?

The EC2 Fleet Plugin manages scaling, not the ASG's built-in scaling policies. When a Jenkins job enters the queue, the plugin automatically increases the ASG's desired capacity to spin up a worker.

When the worker goes idle, the plugin brings the capacity back to 0. This means you pay nothing when no builds are running.

Do not add any scaling policies. Click Next through the remaining pages.
On the Review page, confirm your settings and click Create Auto Scaling group.

You should see jenkins-spot-agents-asg appear in your Auto Scaling Groups list with 0 instances running.

Seeing an error when creating the ASG?

If you see a "Launch template or version not found" error, make sure you selected the correct launch template (jenkins-spot-agent) on the first page of the wizard.

If you get a "No default VPC" error, you need to select a VPC manually in the network section. Choose the same default VPC that your Jenkins controller is running in.

Need more help? I'm getting an error when creating my Auto Scaling Group.

Your infrastructure is ready. You have a Launch Template that knows how to build a worker and an Auto Scaling Group that knows when to spin one up. Next up, you'll install the EC2 Fleet Plugin in Jenkins and connect it to this Auto Scaling Group.

Install and Configure the EC2 Fleet Plugin in Jenkins

Your Launch Template and Auto Scaling Group are in place, ready to spin up Spot Instance workers whenever capacity is needed. The missing piece is the connection between Jenkins and your ASG.

The EC2 Fleet Plugin bridges this gap. It monitors the Jenkins build queue and automatically adjusts your ASG's desired capacity to match demand.

In this step, get ready to:

Install the EC2 Fleet Plugin and AWS Credentials plugin in Jenkins.
Add AWS and SSH credentials so Jenkins can manage instances and connect to agents.
Configure the EC2 Fleet Cloud to point at your Auto Scaling Group.

Install the EC2 Fleet Plugin

Jenkins uses plugins to extend its capabilities. The EC2 Fleet Plugin adds the ability to use AWS Auto Scaling Groups as a source of build agents.

In your Jenkins dashboard, click Manage Jenkins in the left sidebar.
Click Plugins.
Select the Available plugins tab on the left.
Type EC2 Fleet in the search bar.
Check the box next to Amazon EC2 Fleet Cloud.

You also need the AWS Credentials plugin to store your AWS access keys securely in Jenkins.

Clear the search bar and type AWS Credentials.
If AWS Credentials appears in the results, check its box too.
Click Install at the top of the page.
On the installation progress page, check Restart Jenkins when installation is complete and no jobs are running.
Wait about a minute for Jenkins to restart, then log back in with your admin credentials.

Pro tip: already installed?

If AWS Credentials doesn't appear under Available plugins, it's already installed. Just install the EC2 Fleet plugin and continue.

Can't find the EC2 Fleet plugin?

Make sure you're on the Available plugins tab, not Installed plugins. Try searching for just EC2 Fleet without quotes. The plugin requires Jenkins 2.492.3 or later. If your Jenkins version is older, you'll need to upgrade first.

Help me find and install the EC2 Fleet plugin in Jenkins.

Add AWS credentials to Jenkins

Jenkins needs two sets of credentials to work with your cloud agents. AWS credentials let the EC2 Fleet Plugin call AWS APIs to manage your Auto Scaling Group. SSH credentials let Jenkins connect to newly provisioned agents and start the agent process.

Start by adding the AWS access keys you created for the jenkins-fleet-user IAM user in Step 2.

Click Manage Jenkins in the left sidebar.
Click Credentials under the Security section.
Click System in the Stores scoped to Jenkins table.
Click Global credentials (unrestricted).
Click Add Credentials in the top right.
Select AWS Credentials from the Kind dropdown.
Enter jenkins-aws-credentials in the ID field.
Paste the Access Key ID you saved from Step 2 into the Access Key ID field.
Paste the Secret Access Key you saved from Step 2 into the Secret Access Key field.
Click Create.

Now add the SSH credentials so Jenkins can connect to new agents.

Click Add Credentials again.
Select SSH Username with private key from the Kind dropdown.
Enter jenkins-agent-ssh-key in the ID field.
Enter ec2-user in the Username field.
Under Private Key, select Enter directly.
Click Add.
Open the jenkins-controller-key.pem file on your computer in a text editor (Notepad on Windows, TextEdit on macOS).
Select all the text in the file and copy it.
Paste the entire key contents into the text area in Jenkins.
Click Create.

Why two sets of credentials?

AWS Credentials give the EC2 Fleet Plugin permission to call AWS APIs. These are the access keys for the jenkins-fleet-user IAM user you created in Step 2.
SSH credentials let Jenkins connect to each new agent over SSH and start the Jenkins agent process. The private key matches the key pair attached to your Launch Template.

You should now see both jenkins-aws-credentials and jenkins-agent-ssh-key listed on the Global credentials page.

Don't see AWS Credentials in the Kind dropdown?

The AWS Credentials plugin might not be installed. Go back to Manage Jenkins > Plugins > Available plugins and search for AWS Credentials. Install it, restart Jenkins, then try adding credentials again.

Help me troubleshoot the AWS Credentials option not showing up in Jenkins.

Configure the EC2 Fleet Cloud

With the plugins installed and credentials stored, you can now create a Cloud configuration that connects Jenkins to your ASG. This is where you tell the plugin which ASG to manage, how to connect to new agents, and when to scale down idle workers.

Click Manage Jenkins in the left sidebar.
Click Clouds under the System Configuration section.
Click New cloud.
Enter spot-agents in the Cloud name field.
Select Amazon EC2 Fleet as the type.
Click Create.

The cloud configuration form opens. Fill in each section with the following settings.

In the AWS Credentials dropdown, select jenkins-aws-credentials.
Select your AWS region from the Region dropdown (e.g. us-east-1). This must match the region where you created your ASG.
In the EC2 Fleet dropdown, select jenkins-spot-agents-asg.

Don't see your ASG in the dropdown?

The dropdown populates after you select both the credentials and the region. Make sure your Region matches where you created the Auto Scaling Group. It may take a few seconds to load the list.

Now configure the launcher so Jenkins knows how to connect to new agents.

Under Launcher, select Launch agents via SSH.
In the Credentials dropdown that appears, select jenkins-agent-ssh-key.
Set the Host Key Verification Strategy to Non verifying Verification Strategy.

Why non-verifying?

Every time the ASG launches a new Spot Instance, it gets a brand new SSH host key. The Non verifying strategy tells Jenkins to accept any host key, which is necessary for ephemeral cloud instances that come and go. Without this, SSH connections to new agents will fail.

Finally, configure the scaling behavior and agent label.

Enter ec2-fleet in the Label field.
Set Max Idle Minutes Before Scaledown to 3.
Set Minimum Cluster Size to 0.
Set Maximum Cluster Size to 2.
Set Number of Executors to 1.
Check the No Delay Provision Strategy checkbox.
Click Save.

What do these settings control?

The ec2-fleet label tells Jenkins which jobs should run on these cloud agents. Your pipeline will target this label in the next step.
Max Idle Minutes Before Scaledown of 3 means agents that sit idle for 3 minutes get terminated. This keeps costs low while giving a small buffer for back-to-back builds.
No Delay Provision Strategy speeds up provisioning by responding to queue demand immediately instead of waiting for the next polling cycle.

You should now see spot-agents listed on the Clouds page with the type Amazon EC2 Fleet.

Cloud saved but something looks wrong?

If the EC2 Fleet dropdown is empty, double-check that your Region matches the region where you created the ASG in Step 3.
If you see a credentials error, verify the Access Key ID and Secret Access Key are correct by comparing them with the values you recorded in Step 2.
Make sure the jenkins-fleet-user IAM user has the JenkinsFleetPluginPolicy policy attached.

Help me troubleshoot my EC2 Fleet Cloud configuration in Jenkins.

Your Jenkins controller is now connected to the Auto Scaling Group and ready to provision Spot workers on demand. Next up, you'll create a pipeline that triggers this entire system into action.

Create and Run a Pipeline on Spot Agents

Your Jenkins controller is now connected to your Auto Scaling Group through the EC2 Fleet Plugin.

Time to put the whole system to the test. You will create a Declarative Pipeline that targets your Spot Instance fleet, then watch as an agent spins up on demand, runs the build, and disappears when done.

In this step, get ready to:

Create a new Pipeline job in Jenkins.
Write a Declarative Pipeline that targets the Spot agent fleet.
Run the pipeline and watch dynamic agent provisioning in action.

Create a new pipeline job

A Pipeline job in Jenkins defines your build process as code. Instead of clicking through a series of UI forms, you describe every stage of your build in a script called a Jenkinsfile.

From the Jenkins dashboard, click New Item in the left sidebar.
Enter spot-agent-demo in the Enter an item name field.
Select Pipeline from the list of job types.
Click OK.

Jenkins opens the job configuration page. You will add the pipeline script in the next substep.

Write the Declarative Pipeline

A Declarative Pipeline uses a structured Groovy syntax to define stages, steps, and post-build actions. The key line is the agent directive, which tells Jenkins where to run the pipeline.

Scroll down to the Pipeline section at the bottom of the configuration page.
In the Definition dropdown, select Pipeline script.
Paste the following Jenkinsfile into the Script text area:

pipeline {
    // Target agents with the 'ec2-fleet' label (your Spot worker fleet)
    agent { label 'ec2-fleet' }

    stages {
        // Print info about the agent running this build
        stage('Verify Agent') {
            steps {
                sh 'echo "Running on agent: $(hostname)"'
                sh 'echo "Java version:" && java -version'
                sh 'echo "Operating System:" && cat /etc/os-release | head -2'
            }
        }

        // Simulate creating build artifacts
        stage('Simulate Build') {
            steps {
                sh 'echo "Starting build process..."'
                sh 'mkdir -p build-output'
                sh 'echo "Build artifact created at $(date)" > build-output/result.txt'
                sh 'cat build-output/result.txt'
            }
        }

        // Simulate running a test suite
        stage('Simulate Tests') {
            steps {
                sh 'echo "Running test suite..."'
                sh 'sleep 5'
                sh 'echo "All 42 tests passed!"'
            }
        }
    }

    // Run this block regardless of whether the pipeline succeeded or failed
    post {
        always {
            sh 'echo "Pipeline completed on agent: $(hostname)"'
        }
    }
}

What does this pipeline do?

agent { label 'ec2-fleet' } tells Jenkins to run this pipeline only on nodes with the ec2-fleet label. This matches the label you set on the EC2 Fleet cloud in Step 4, so Jenkins knows to provision a Spot agent.
The Verify Agent stage prints the hostname, Java version, and OS info. This confirms the pipeline is running on a remote Spot worker, not on the controller.
The Simulate Build stage creates a mock build artifact. In a real project, this is where compilation or bundling would happen.
The Simulate Tests stage mimics a test suite with a short delay.
The post { always { ... } } block runs after the pipeline finishes, regardless of success or failure. It logs the hostname one final time.

Click Save at the bottom of the configuration page.

You should see the spot-agent-demo job page with a Build Now link in the left sidebar.

Seeing a script error when you save?

Make sure you copied the entire script, including the opening pipeline { and the final closing }.
Check that you selected Pipeline script in the Definition dropdown, not Pipeline script from SCM.
help me fix a Jenkinsfile syntax error

Awesome, I've got everything!

Double check that you clicked Save and you can see the spot-agent-demo job page.

I'd like to double check the full code

pipeline {
    agent { label 'ec2-fleet' }

    stages {
        stage('Verify Agent') {
            steps {
                sh 'echo "Running on agent: $(hostname)"'
                sh 'echo "Java version:" && java -version'
                sh 'echo "Operating System:" && cat /etc/os-release | head -2'
            }
        }

        stage('Simulate Build') {
            steps {
                sh 'echo "Starting build process..."'
                sh 'mkdir -p build-output'
                sh 'echo "Build artifact created at $(date)" > build-output/result.txt'
                sh 'cat build-output/result.txt'
            }
        }

        stage('Simulate Tests') {
            steps {
                sh 'echo "Running test suite..."'
                sh 'sleep 5'
                sh 'echo "All 42 tests passed!"'
            }
        }
    }

    post {
        always {
            sh 'echo "Pipeline completed on agent: $(hostname)"'
        }
    }
}

Run the pipeline and watch dynamic provisioning

This is the moment everything comes together. When you trigger a build, the EC2 Fleet Plugin detects the queued job, increases the ASG's desired capacity, and a Spot Instance launches to handle the work.

Click Build Now in the left sidebar of the spot-agent-demo job page.

You should see the build appear in the Build History section with a blinking indicator. The build will stay queued while the EC2 Fleet Plugin requests a new Spot Instance from the ASG.

What is happening behind the scenes?

Jenkins sees a job that needs an agent with the ec2-fleet label, but no agents are online.
The EC2 Fleet Plugin increases the ASG's desired capacity from 0 to 1.
AWS launches a Spot Instance using your Launch Template. The User Data script installs Java automatically.
Once the instance is ready, the plugin connects via SSH and registers it as a Jenkins agent.
The queued job starts running on the new agent.

This whole process typically takes 2-3 minutes. Most of that time is the EC2 instance booting and installing Java.

Watch the Jenkins dashboard. After a few minutes, a new node should appear in the Build Executor Status panel on the left sidebar.
Once the build finishes, click on build #1 in the Build History section.
Click Console Output in the left sidebar to see the full build log.

You should see output from all three stages: the agent's hostname, Java version, OS info, the build artifact message, and the test results line reading All 42 tests passed!.

Open the AWS EC2 Console in a new browser tab.
Click Instances in the left sidebar.
Look for a new instance launched by the ASG. Its name may be blank or match the ASG naming pattern.

If the build just finished, you should see the Spot worker instance in a Running state. After the 3-minute idle timeout passes, the EC2 Fleet Plugin terminates it and reduces the ASG desired capacity back to 0.

Pro tip: watch the instance disappear

Keep the EC2 Console open and refresh after about 3 minutes. You should see the Spot worker transition to a Terminated state. This confirms the idle scaledown is working. Your ASG is back at 0 instances, costing you nothing.

Build stuck in the queue?

Click on the queued build and check the reason Jenkins shows. If it says Waiting for next available executor, the plugin is trying to provision an agent.
Verify the ec2-fleet label in your pipeline matches the label you set in the EC2 Fleet cloud configuration in Step 4. A mismatch means Jenkins cannot find a matching cloud.
Check the AWS credentials you entered in Step 4. Go to Manage Jenkins > Clouds > spot-agents and confirm the ASG name and region are correct.
If the build is still stuck after 5 minutes, try restarting Jenkins. There is a known issue where provisioning can stall after scaling down to 0 instances. Restart by navigating to http://<your-controller-ip>:8080/restart in your browser.
my Jenkins pipeline build is stuck in the queue and no Spot agent is provisioning

Agent comes online but the build fails?

Check the console output for a java: command not found error. This means the User Data script did not finish installing Java before Jenkins connected. Wait a minute and try Build Now again.
Verify that the jenkins-agent-sg security group allows SSH (port 22) inbound from the jenkins-controller-sg security group. Without this, the controller cannot SSH into the agent.
my Jenkins Spot agent comes online but the pipeline build fails

Secret mission

Diversify Instance Types and Verify Spot Status

Your current setup uses only two instance types from the same family. In production, this limits Spot availability and increases the chance of interruptions. Take your setup to production-grade by diversifying across instance families and proving your agents actually run on Spot Instances.

Before you go

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. The Jenkins controller runs on a free-tier eligible t3.micro instance, and the Auto Scaling Group sits at 0 instances when idle, so ongoing costs are minimal if your account still has free tier credits.

Cost warning

If your AWS free tier has expired, the jenkins-controller EC2 instance costs approximately $0.0104/hour (about $7.50/month) while running. Spot worker instances only run during pipeline execution and auto-terminate, so their cost is fractions of a cent per build. If you are no longer experimenting, stop or terminate the controller to avoid charges.

Resources you used:

EC2 instance: jenkins-controller (t3.micro, free-tier eligible).
Auto Scaling Group: jenkins-spot-agents-asg.
Launch Template: jenkins-spot-agent.
Security groups: jenkins-controller-sg and jenkins-agent-sg.
Key pair: jenkins-controller-key.
IAM user: jenkins-fleet-user (with programmatic access key).
IAM role: jenkins-agent-role.
IAM policy: JenkinsFleetPluginPolicy.

Keep everything running

No action needed. Choose this if you are still actively building or experimenting with Jenkins pipelines.

The jenkins-controller instance stays running. This is free-tier eligible if your account is within the 12-month window and under 750 hours/month.
The Auto Scaling Group sits at 0 desired capacity. No Spot Instances are running, so there is no cost while idle.
Spot workers only spin up when you trigger a build and auto-terminate after the 3-minute idle timeout.

Pause - I'll come back to this later

Stop the controller instance to eliminate compute charges while keeping all your configuration intact.

In the EC2 Console, select the jenkins-controller instance.
Click Instance state, then Stop instance.
The Auto Scaling Group remains at 0 desired capacity. No changes needed there.
When you are ready to resume, start the instance again from the EC2 Console. Note that the public IP address may change when you restart.

Delete - I don't want to use this again

Remove all project resources completely. Follow these steps in order to avoid dependency errors.

In the EC2 Console, select the jenkins-controller instance. Click Instance state, then Terminate instance.
Go to Auto Scaling Groups and delete jenkins-spot-agents-asg.
Go to Launch Templates and delete jenkins-spot-agent.
Go to Security Groups and delete jenkins-agent-sg first, then jenkins-controller-sg. You may need to wait for the controller instance to finish terminating before the security groups can be deleted.
Go to Key Pairs and delete jenkins-controller-key. Also delete the .pem file from your local computer.
In the IAM Console, go to Users and delete jenkins-fleet-user. This also removes its access key.
In IAM, go to Roles and delete jenkins-agent-role.
In IAM, go to Policies and delete JenkinsFleetPluginPolicy.
Remove the AWS credentials you stored in Jenkins from your records, if you saved them anywhere locally.

Mission accomplished

That's a Wrap!

Nice work! You just built a production-grade Jenkins Controller-Agent architecture that dynamically provisions EC2 Spot Instances to run CI/CD pipelines. This is the same pattern real companies use to save 70-90% on their build infrastructure costs.

Here's what you learned:

Set up a Jenkins Controller on EC2 and configured the EC2 Fleet Plugin to dynamically provision Spot Instance agents through an Auto Scaling Group.
Built the full AWS infrastructure for dynamic agents, including a Launch Template with User Data, an Auto Scaling Group with Spot Instance diversification, and a custom IAM policy scoped to the exact permissions the plugin needs.
Wrote a Declarative Jenkins Pipeline that targets the ec2-fleet label, triggers on-demand agent provisioning, and watches agents terminate automatically after idle timeout.
Diversified instance types across multiple families and used EC2 Instance Metadata (IMDSv2) to verify agents are running on Spot Instances.

Ready to quiz yourself?