Automate Orphaned Resource Cleanup

Introduction

30 Second Summary

Every month, cloud teams discover they are paying for resources nobody is using. These orphaned resources silently drain budgets until someone manually tracks them down.

In this project, you will build an end-to-end orphaned resource detection and auto-remediation system on AWS. You will intentionally create wasteful resources, then watch them get detected, flagged, and automatically cleaned up.

What You'll Build

You will run a Python script that instantly surfaces wasted resources in your account, then watch as AWS Config flags them and SSM Automation deletes them on its own.

By the end of this project, you'll have:

• A Python script and Bash script you can run to scan your AWS account for unattached EBS volumes and expiring certificates, with estimated monthly cost for each.
• AWS Config rules that continuously monitor your resources and flag them as NON_COMPLIANT the moment they become orphaned.
• An SSM Automation pipeline that automatically snapshots and deletes non-compliant volumes without any human intervention.
• Secret Mission: Wire up SNS email notifications so you receive an alert every time auto-remediation fires.

Are there any prerequisites?

You need an AWS account (free tier works for most steps), Python 3, and the AWS CLI installed and configured. The Trusted Advisor section in Step 5 requires a Business Support plan or higher, but you can follow along with the console walkthrough if you do not have one.

Before We Start

Before we dive in, let's get clear on what you're about to build and why it matters. You'll create an automated pipeline that detects orphaned AWS resources and remediates them without manual intervention.

Set Up and Create Orphaned Resources

To build an automated cleanup pipeline, you first need something to clean up. Creating orphaned resources intentionally lets you observe the full detection-to-remediation lifecycle in a controlled environment.

In this step, you will create two kinds of waste that are common in real AWS accounts: an unattached EBS volume (storage not connected to any server) and a near-expiry ACM certificate (an SSL certificate about to become invalid).

In this step, get ready to:

• Create an unattached EBS volume via the AWS CLI.
• Generate a near-expiry self-signed certificate using Python and import it into ACM.
• Verify both orphaned resources exist in your AWS account.

Create an unattached EBS volume

An EBS volume in "available" state means it exists but is not attached to any EC2 instance. It costs money every month even though nothing is using it.

• Move to your Desktop by running this command:

cd ~/Desktop

• Create your project folder and move into it by running these commands:

mkdir aws-orphan-cleanup
cd aws-orphan-cleanup

• Create a 1 GB unattached EBS volume in us-east-1a by running this command:

aws ec2 create-volume --availability-zone us-east-1a --size 1 --volume-type gp3

What does this command do?

• --availability-zone us-east-1a places the volume in a specific availability zone.
• --size 1 creates a tiny 1 GB volume to minimize cost.
• --volume-type gp3 uses the general-purpose SSD type.
• Because you never attach this volume to an EC2 instance, it becomes "orphaned" immediately.

You should see JSON output containing a VolumeId field (like vol-0abc123def456789) and "State": "creating".

• Copy your VolumeId value and save it somewhere. You will need it later.

Seeing an error about the availability zone?

Make sure your AWS CLI is configured for the us-east-1 region. Run aws configure get region to check. If it shows a different region, run aws configure set region us-east-1 and try the create-volume command again.

Help me debug this

Generate and import a near-expiry certificate

ACM cannot import an already-expired certificate. The workaround is to generate a certificate that expires in 1 day, which is still technically valid at import time but immediately triggers any "expiring soon" checks.

• Create a file called generate_expiring_cert.py in your aws-orphan-cleanup folder and paste the following code:

from cryptography import x509
from cryptography.x509.oid import NameOID
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from datetime import datetime, timedelta, timezone

def generate_expiring_certificate():
    # Generate a 2048-bit RSA private key
    key = rsa.generate_private_key(public_exponent=65537, key_size=2048)

    # Define certificate subject and issuer (self-signed, so they match)
    subject = issuer = x509.Name([
        x509.NameAttribute(NameOID.COUNTRY_NAME, "US"),
        x509.NameAttribute(NameOID.STATE_OR_PROVINCE_NAME, "Test"),
        x509.NameAttribute(NameOID.ORGANIZATION_NAME, "TestOrg"),
        x509.NameAttribute(NameOID.COMMON_NAME, "test.orphan-detection.example.com"),
    ])

    # Build the certificate: valid from yesterday, expires in 1 day
    now = datetime.now(timezone.utc)
    cert = (
        x509.CertificateBuilder()
        .subject_name(subject)
        .issuer_name(issuer)
        .public_key(key.public_key())
        .serial_number(x509.random_serial_number())
        .not_valid_before(now - timedelta(days=1))
        .not_valid_after(now + timedelta(days=1))
        .sign(key, hashes.SHA256())
    )

What does this code do?

• The imports pull in the cryptography library's x509 module for building certificates.
• rsa.generate_private_key() creates a fresh RSA key pair that will sign the certificate.
• The subject and issuer are identical because this is a self-signed certificate. The Common Name uses a test domain so it is clearly identifiable in ACM.
• .not_valid_before(now - timedelta(days=1)) backdates validity to yesterday so the cert is technically valid right now.
• .not_valid_after(now + timedelta(days=1)) sets expiry to tomorrow. This keeps it importable but immediately triggers any "expiring within 14 days" check.

• Directly below the code you just pasted, add this section that serializes the certificate, writes the PEM files, and provides the import command:

    # Serialize the certificate and key to PEM format
    cert_pem = cert.public_bytes(serialization.Encoding.PEM)
    key_pem = key.private_bytes(
        serialization.Encoding.PEM,
        serialization.PrivateFormat.TraditionalOpenSSL,
        serialization.NoEncryption()
    )

    # Write PEM files to disk
    with open("test_cert.pem", "wb") as f:
        f.write(cert_pem)
    with open("test_key.pem", "wb") as f:
        f.write(key_pem)

    print("Generated test_cert.pem (expires in 1 day)")
    print("Generated test_key.pem (unencrypted private key)")
    print("\nImport with:")
    print("  aws acm import-certificate --certificate fileb://test_cert.pem --private-key fileb://test_key.pem")

if __name__ == "__main__":
    generate_expiring_certificate()

What does this code do?

• cert.public_bytes() and key.private_bytes() convert the certificate and key objects into PEM-encoded byte strings.
• The key is exported unencrypted (NoEncryption()) because ACM requires an unencrypted private key at import time.
• The if __name__ block runs the function when you execute the script directly.

• Save generate_expiring_cert.py.
• Run the script to generate your certificate files:

python generate_expiring_cert.py

You should see output confirming both PEM files were generated and showing the import command.

• Confirm the files were created by running:

ls

You should see generate_expiring_cert.py, test_cert.pem, and test_key.pem listed.

Seeing a ModuleNotFoundError for cryptography?

The cryptography library needs to be installed. Run pip install cryptography and try the script again.

Help me debug this error

• Import the certificate into ACM by running:

aws acm import-certificate --certificate fileb://test_cert.pem --private-key fileb://test_key.pem

You should see JSON output containing a CertificateArn field. This is the ARN of your imported certificate.

• Copy the CertificateArn value and save it alongside your Volume ID from earlier.

Why fileb:// instead of file://?

The fileb:// prefix tells the AWS CLI to read the file as raw binary bytes. PEM certificates must be sent as binary data to ACM, not as a UTF-8 text string.

Getting an InvalidParameterException?

ACM requires the current time to be between the certificate's NotBefore and NotAfter fields. If the certificate has already expired (you waited more than a day), re-run python generate_expiring_cert.py to generate a fresh one, then retry the import.

Help me troubleshoot my certificate import

✔️ Awesome, I've got everything!

Great. Double check you saved generate_expiring_cert.py and that you have both your Volume ID and Certificate ARN noted down for later steps.

ⓧ I'd like to double check the full code

Scroll up and confirm your generate_expiring_cert.py contains both chunks in order:

• Chunk 1: Imports, the generate_expiring_certificate() function definition, key generation, subject/issuer definition, and the CertificateBuilder chain.
• Chunk 2: PEM serialization with cert.public_bytes() and key.private_bytes(), file writing, print statements, and the if __name__ == "__main__" block.

If anything is missing, copy the relevant chunk from above and paste it into the correct position in your file.

Verify both orphaned resources exist

Before you move on, confirm that both resources are live in your account and in the expected state.

• Before you run the next command, what do you expect it to print? Think about the state of the volume you created.
• Verify your unattached EBS volume by running:

aws ec2 describe-volumes --filters Name=status,Values=available --query "Volumes[].VolumeId" --output text

You should see your volume ID (like vol-0abc123def456789) printed. This confirms the volume exists and is in "available" (unattached) state.

Not seeing any output?

The volume starts in "creating" state and transitions to "available" within a few seconds. Wait 10 seconds and run the command again. Also confirm your CLI region is set to us-east-1 since the volume was created in us-east-1a.

Help me find my EBS volume

Your orphaned resources are live and costing money. Next up, you will write detection scripts that find these resources automatically and calculate their monthly waste.

Detect Orphaned Resources with Scripts

You now have orphaned resources sitting in your AWS account. An unattached EBS volume is quietly costing money, and a certificate is about to expire with no one watching.

Before automating anything, you need to understand what "orphaned" looks like from the API level. Writing detection scripts manually shows you the exact API calls and filters that identify wasted resources. Once you understand the mechanics, the automation you build in later steps will make complete sense.

In this step, get ready to:

• Write a Python detection script that finds unattached EBS volumes and expiring certificates.
• Write an equivalent Bash detection script using AWS CLI commands.
• Run both scripts and confirm they find the same orphaned resources.

Write the Python detection script

The Python script uses boto3 to call the EC2 and ACM APIs directly. It filters for volumes in the available state (meaning unattached) and certificates expiring within 14 days.

• Create a new file called detect_orphans.py in the same project folder on your Desktop.
• Add the imports, the volume detection function, and the cost estimation helper by pasting this code:

import boto3
from datetime import datetime, timezone

def get_unattached_volumes():
    ec2 = boto3.client("ec2")
    response = ec2.describe_volumes(
        Filters=[{"Name": "status", "Values": ["available"]}]
    )
    volumes = []
    for vol in response["Volumes"]:
        monthly_cost = estimate_ebs_cost(vol["Size"], vol["VolumeType"])
        volumes.append({
            "VolumeId": vol["VolumeId"],
            "Size": vol["Size"],
            "VolumeType": vol["VolumeType"],
            "CreateTime": vol["CreateTime"].strftime("%Y-%m-%d"),
            "EstimatedMonthlyCost": monthly_cost
        })
    return volumes

def estimate_ebs_cost(size_gb, volume_type):
    pricing = {"gp3": 0.08, "gp2": 0.10, "io1": 0.125, "st1": 0.045, "sc1": 0.015}
    rate = pricing.get(volume_type, 0.10)
    return round(size_gb * rate, 2)

What does this code do?

The describe_volumes call uses a server-side filter to return only volumes with status: available. In AWS, a volume that is "available" is not attached to any EC2 instance. It exists, consumes storage, and costs money every month.

The estimate_ebs_cost function multiplies the volume size in GB by a per-GB monthly rate for each volume type. A 1 GB gp3 volume costs roughly $0.08/month.

• Now add the certificate detection function below the estimate_ebs_cost function in the same file:

def get_expiring_certificates(days=14):
    acm = boto3.client("acm")
    response = acm.list_certificates(CertificateStatuses=["ISSUED"])
    expiring = []
    now = datetime.now(timezone.utc)
    for cert_summary in response["CertificateSummaryList"]:
        cert_detail = acm.describe_certificate(
            CertificateArn=cert_summary["CertificateArn"]
        )
        cert = cert_detail["Certificate"]
        not_after = cert.get("NotAfter")
        if not_after:
            days_left = (not_after - now).days
            if days_left <= days:
                expiring.append({
                    "CertificateArn": cert["CertificateArn"],
                    "DomainName": cert["DomainName"],
                    "DaysUntilExpiry": days_left,
                    "Type": cert.get("Type", "UNKNOWN")
                })
    return expiring

What does this code do?

This function lists all ISSUED certificates in ACM, then checks each one's NotAfter date. If a certificate expires within 14 days, it gets flagged. Your test certificate expires in 1 day, so it will definitely appear.

• Save detect_orphans.py and verify the detection functions work by running this quick test:

python -c "from detect_orphans import get_unattached_volumes; print(get_unattached_volumes())"

You should see a Python list containing a dictionary with your volume ID, size 1, type gp3, and an estimated monthly cost of 0.08.

Seeing an empty list or ModuleNotFoundError?

• Make sure you saved detect_orphans.py and that your terminal is in the same directory as the file.
• If you see ModuleNotFoundError: No module named 'boto3', confirm boto3 is installed with pip install boto3.
• An empty list means no volumes are in available state. Confirm your volume from Step 1 still exists by running aws ec2 describe-volumes --filters Name=status,Values=available.

Help me debug why my detect_orphans.py is not finding my unattached volume.

• Now add the main function at the bottom of detect_orphans.py to produce a formatted report:

def main():
    print("=" * 60)
    print("ORPHANED RESOURCE DETECTION REPORT")
    print("=" * 60)

    print("\n--- Unattached EBS Volumes ---")
    volumes = get_unattached_volumes()
    if volumes:
        total_cost = 0
        for vol in volumes:
            print(f"  {vol['VolumeId']} | {vol['Size']}GB {vol['VolumeType']} | "
                  f"Created: {vol['CreateTime']} | ${vol['EstimatedMonthlyCost']}/mo")
            total_cost += vol["EstimatedMonthlyCost"]
        print(f"  TOTAL estimated waste: ${total_cost}/month")
    else:
        print("  None found.")

    print("\n--- Expiring ACM Certificates (within 14 days) ---")
    certs = get_expiring_certificates()
    if certs:
        for cert in certs:
            print(f"  {cert['DomainName']} | {cert['DaysUntilExpiry']} days left | "
                  f"Type: {cert['Type']}")
            print(f"    ARN: {cert['CertificateArn']}")
    else:
        print("  None found.")

    print("\n" + "=" * 60)

if __name__ == "__main__":
    main()

What does this code do?

The main function calls both detection functions and prints a formatted report. It shows each orphaned resource with identifying details and estimated cost. The if __name__ == "__main__" guard ensures the report only prints when you run the file directly.

• Save detect_orphans.py.
• Before you run the full script, what do you expect to see in the report? Think about which resources you created in Step 1.
• Run the detection script:

python detect_orphans.py

You should see a report listing your unattached volume ID (1GB gp3, $0.08/mo) and your expiring certificate for test.orphan-detection.example.com with 0 or 1 days left.

Seeing "None found" for certificates?

• If your certificate was imported more than 24 hours ago, it may have already expired past the NotAfter date. The days_left calculation will return a negative number, which is still less than 14, so it should still appear.
• Confirm the certificate exists by running aws acm list-certificates --certificate-statuses ISSUED. If it shows no results, your certificate may have been deleted or its status changed.

Help me understand why my ACM certificate is not showing up in my detection script.

✔️ Awesome, I've got everything!

Great. Double check you have saved detect_orphans.py and that both sections of the report show your resources.

ⓧ I'd like to double check the full code

Scroll up and confirm your detect_orphans.py file contains these three sections in order:

• Imports, get_unattached_volumes, and estimate_ebs_cost (first chunk).
• get_expiring_certificates (second chunk).
• main function and the if __name__ == "__main__" guard (third chunk).

Write the equivalent Bash detection script

Having a Bash version gives you the same detection capability using only the AWS CLI. This is useful on systems where Python is not available, or in CI/CD pipelines where a shell script is simpler to run.

• Create a new file called detect_orphans.sh in the same project folder.
• Add the script header and the EBS volumes detection section:

#!/bin/bash
echo "============================================================"
echo "ORPHANED RESOURCE DETECTION REPORT"
echo "============================================================"

echo ""
echo "--- Unattached EBS Volumes ---"
VOLUMES=$(aws ec2 describe-volumes \
    --filters Name=status,Values=available \
    --query "Volumes[].{ID:VolumeId,Size:Size,Type:VolumeType,Created:CreateTime}" \
    --output json)

if [ "$(echo "$VOLUMES" | python -c "import sys,json; print(len(json.load(sys.stdin)))" )" -gt 0 ]; then
    echo "$VOLUMES" | python -c "
import sys, json
volumes = json.load(sys.stdin)
pricing = {'gp3': 0.08, 'gp2': 0.10, 'io1': 0.125, 'st1': 0.045, 'sc1': 0.015}
total = 0
for v in volumes:
    cost = v['Size'] * pricing.get(v['Type'], 0.10)
    total += cost
    print(f\"  {v['ID']} | {v['Size']}GB {v['Type']} | Created: {v['Created'][:10]} | \${cost:.2f}/mo\")
print(f'  TOTAL estimated waste: \${total:.2f}/month')
"
else
    echo "  None found."
fi

What does this code do?

The aws ec2 describe-volumes command uses the same status=available filter as the Python script, but adds a --query parameter to extract only the fields you need. The JSON output is then piped into an inline Python script for formatting and cost calculation.

• Add the ACM certificates section header and data retrieval below the fi statement:

echo ""
echo "--- Expiring ACM Certificates (within 14 days) ---"
CERTS=$(aws acm list-certificates \
    --certificate-statuses ISSUED \
    --query "CertificateSummaryList[].CertificateArn" \
    --output json)

What does this code do?

This retrieves all ISSUED certificate ARNs as a JSON array using the same aws acm list-certificates command your Python script calls via boto3. The --query flag extracts only the ARN field from each certificate summary.

• Add the certificate processing logic and closing lines below the CERTS assignment:

echo "$CERTS" | python -c "
import sys, json, subprocess
from datetime import datetime, timezone
arns = json.load(sys.stdin)
found = False
for arn in arns:
    result = subprocess.run(
        ['aws', 'acm', 'describe-certificate', '--certificate-arn', arn, '--output', 'json'],
        capture_output=True, text=True
    )
    cert = json.loads(result.stdout)['Certificate']
    not_after = cert.get('NotAfter')
    if not_after:
        expiry = datetime.fromisoformat(not_after.replace('Z', '+00:00'))
        days_left = (expiry - datetime.now(timezone.utc)).days
        if days_left <= 14:
            found = True
            print(f\"  {cert['DomainName']} | {days_left} days left | Type: {cert.get('Type', 'UNKNOWN')}\")
            print(f\"    ARN: {arn}\")
if not found:
    print('  None found.')
"

echo ""
echo "============================================================"

What does this code do?

This section pipes the certificate ARNs into inline Python that calls aws acm describe-certificate for each ARN. It parses the expiration date and checks if the certificate expires within 14 days. This mirrors the exact same logic as your Python script's get_expiring_certificates function.

• Save detect_orphans.sh.
• Run the Bash script:

bash detect_orphans.sh

You should see the same report format as the Python version, listing your unattached volume and expiring certificate with matching details.

Getting a permission denied or syntax error?

• If you see a permission error, make sure you are running with bash detect_orphans.sh (not ./detect_orphans.sh).
• On Windows, run this inside Git Bash or WSL. The standard Command Prompt cannot execute Bash scripts.
• If you see a Python syntax error, check that the escaped quotes inside the inline Python blocks are preserved. Copy the code block again if needed.

Help me fix the error I am getting when running detect_orphans.sh.

✔️ Awesome, I've got everything!

Great. Double check you have saved detect_orphans.sh and that the output shows both your volume and certificate.

ⓧ I'd like to double check the full code

Scroll up and confirm your detect_orphans.sh file contains these three sections in order:

• The shebang line, header echoes, VOLUMES assignment, and the if/else block for EBS volumes (first chunk).
• The echo "" separator, ACM header echo, and CERTS assignment (second chunk).
• The inline Python processing, if not found fallback, and closing echo lines (third chunk).

Compare outputs from both scripts

Both scripts query the same AWS APIs with the same filters. The outputs should list identical resource IDs and cost figures.

• Compare the volume ID printed by each script. They should match exactly.
• Compare the certificate ARN and days-until-expiry values. They should match (or differ by at most 1 day if you ran them at different times).
• Confirm the estimated monthly cost for your 1 GB gp3 volume is $0.08 in both outputs.

Why write both Python and Bash versions?

In production, teams pick the language that fits their toolchain. Python scripts integrate well with larger automation frameworks and provide rich data structures. Bash scripts are lightweight and run anywhere with the AWS CLI installed.

Having both proves you understand the underlying API calls, not just a single SDK.

You now have two working scripts that can scan any AWS account for orphaned resources. The scripts work, but you have to remember to run them. In the next step, you will set up AWS Config to do this detection continuously and automatically, flagging resources the moment they become non-compliant.

Flag Resources as NON_COMPLIANT with AWS Config

Your Python and Bash scripts found the orphaned resources. But scripts only work when you remember to run them.

If you forget to run your script for a week, orphaned volumes pile up unnoticed. AWS Config solves this by monitoring continuously. It flags resources the moment they become non-compliant, without anyone triggering a scan.

In this step, get ready to:

• Enable AWS Config recording for EC2 volumes and ACM certificates.
• Create two managed Config rules that detect orphaned resources.
• See both resources flagged as NON_COMPLIANT in the Config console.

Enable AWS Config recording in your region

AWS Config needs a configuration recorder running before it can evaluate any rules. The recorder tracks changes to the resource types you specify and stores them in an S3 bucket.

• Open the AWS Management Console and navigate to the AWS Config service. Type Config in the top search bar and select Config from the results.
• If this is your first time using Config in this region, you will see a setup page. Click Get started.
• On the Settings page, under Recording method, select Specific resource types.
• Search for and add AWS::EC2::Volume and AWS::ACM::Certificate as the resource types to record. Set both to Continuous recording.
• Under AWS Config role, leave the default option to create a service-linked role. This gives Config permission to read your resource configurations.
• Under Delivery method, choose Create a bucket and leave the default bucket name. Config needs an S3 bucket to store configuration snapshots.
• Click Next.
• On the Rules page, skip adding rules for now. You will add them via CLI in the next substep. Click Next.
• Review your settings and click Confirm.

What is AWS Config recording?

The configuration recorder watches for changes to the resource types you selected. Every time an EC2 volume or ACM certificate is created, modified, or deleted, Config captures a snapshot called a configuration item (CI).

These snapshots are what Config rules evaluate against. Without the recorder running, rules have nothing to check.

• Verify the recorder is active by running this command in your terminal:

aws configservice describe-configuration-recorder-status

You should see "recording": true in the output. This confirms Config is actively tracking your resources.

Seeing recording as false?

If the recorder status shows "recording": false, the setup may not have completed. Return to the Config console, go to Settings, and confirm the recorder is turned on.

If you get a permissions error, make sure your AWS CLI is configured with credentials that have configservice permissions.

Help me troubleshoot my AWS Config recorder status

Create Config rules to detect orphaned resources

With the recorder running, you can now create rules that evaluate your resources against compliance criteria. You will create two AWS-managed rules: one for unattached EBS volumes and one for expiring certificates.

• Create the EBS volume rule by running this command:

aws configservice put-config-rule --config-rule '{
  "ConfigRuleName": "ec2-volume-inuse-check",
  "Description": "Checks whether EBS volumes are attached to EC2 instances.",
  "Scope": {
    "ComplianceResourceTypes": ["AWS::EC2::Volume"]
  },
  "Source": {
    "Owner": "AWS",
    "SourceIdentifier": "EC2_VOLUME_INUSE_CHECK"
  }
}'

What does this command do?

• ConfigRuleName is a friendly name you give this rule in your account.
• Scope limits evaluation to AWS::EC2::Volume resources only.
• Source.Owner set to AWS means this is a managed (pre-built) rule, not a custom Lambda function.
• SourceIdentifier of EC2_VOLUME_INUSE_CHECK tells Config to use its built-in logic that marks a volume as NON_COMPLIANT if it is not attached to any EC2 instance.

• The command produces no output on success. Confirm the rule exists by running:

aws configservice describe-config-rules --config-rule-names ec2-volume-inuse-check

You should see the rule details returned, including "ConfigRuleState": "ACTIVE".

• Now create the ACM certificate expiration rule by running:

aws configservice put-config-rule --config-rule '{
  "ConfigRuleName": "acm-certificate-expiration-check",
  "Description": "Checks whether ACM certificates expire within 14 days.",
  "Scope": {
    "ComplianceResourceTypes": ["AWS::ACM::Certificate"]
  },
  "Source": {
    "Owner": "AWS",
    "SourceIdentifier": "ACM_CERTIFICATE_EXPIRATION_CHECK"
  },
  "InputParameters": "{\"daysToExpiration\":\"14\"}"
}'

What does the daysToExpiration parameter do?

The InputParameters field passes configuration to the managed rule. Setting daysToExpiration to 14 means any certificate expiring within 14 days is flagged NON_COMPLIANT. Your test certificate expires in 1 day, so it will be caught immediately.

• Confirm this rule also exists by running:

aws configservice describe-config-rules --config-rule-names acm-certificate-expiration-check

You should see "ConfigRuleState": "ACTIVE" in the output for this rule as well.

Getting a permissions error when creating rules?

If you see an error about insufficient permissions, your Config recorder's IAM role may not have the required access. Make sure the Config service-linked role was created during setup. You can check by running:

aws configservice describe-configuration-recorders

Confirm a roleARN is listed. If no recorder appears, revisit the Config console setup.

Help me fix my Config recorder permissions

Verify resources are flagged NON_COMPLIANT

Config evaluates rules against recorded resources. For change-triggered rules like these, evaluation typically completes within a few minutes of the rule being created, because Config already has a configuration snapshot of your existing resources.

• Wait 3-5 minutes for Config to evaluate your resources against the new rules.
• Before you check the results, think about what you expect to see. Your EBS volume is unattached and your certificate expires in 1 day. What compliance status do you predict for each?
• Check the evaluation results for the EBS volume rule by running:

aws configservice get-compliance-details-by-config-rule --config-rule-name ec2-volume-inuse-check --compliance-types NON_COMPLIANT

You should see your unattached volume ID listed as a NON_COMPLIANT resource.

• Check the certificate rule as well:

aws configservice get-compliance-details-by-config-rule --config-rule-name acm-certificate-expiration-check --compliance-types NON_COMPLIANT

You should see your test.orphan-detection.example.com certificate ARN listed as NON_COMPLIANT.

Why is this better than your scripts?

Your Python and Bash scripts require someone to run them manually. If nobody runs them for a month, orphaned volumes accumulate silently.

Config evaluates continuously. The moment a volume is detached or a certificate nears expiration, Config flags it within minutes. No cron jobs, no human memory required.

• Open the AWS Config console in your browser. Click Rules in the left sidebar.
• You should see both ec2-volume-inuse-check and acm-certificate-expiration-check listed with a Noncompliant status.
• Click on ec2-volume-inuse-check to see your specific volume ID flagged as the non-compliant resource.

Both rules are detecting the problem. But notice what has NOT happened. Your unattached EBS volume still exists. The expiring certificate is still there. Config identified the issues but did nothing to fix them.

The gap you will close next

Detection without remediation is only half the story. In a real environment, NON_COMPLIANT resources sit in the console until a human notices and acts. That delay is exactly the problem FinOps teams face.

In the next step, you will wire SSM Automation to Config so that non-compliant resources are automatically fixed. Detection becomes remediation.

Rules still showing no evaluation results?

If after 5 minutes your rules show no evaluation results, Config may not have recorded the resources yet. Try triggering a manual re-evaluation:

aws configservice start-config-rules-evaluation --config-rule-names ec2-volume-inuse-check acm-certificate-expiration-check

Wait another 2-3 minutes and re-check. If the resources were created before Config recording started, Config needs to discover them first.

Help me troubleshoot my Config rule evaluations

Config is now your always-on watchdog, continuously detecting non-compliant resources. Next up, you will give it teeth by wiring SSM Automation to automatically remediate what Config flags.

Wire SSM Automation for Auto-Remediation

Your AWS Config rules are now flagging orphaned resources as NON_COMPLIANT. But detection alone does not fix anything. If you check the console tomorrow, those same resources will still be there, still costing money.

SSM Automation documents define what to DO about non-compliant resources. By wiring Config remediation to an SSM document, you close the loop: detect, then automatically fix. This is the production-grade pattern used by FinOps teams.

In this step, get ready to:

• Create an IAM role that gives SSM Automation permission to snapshot and delete EBS volumes.
• Attach a remediation configuration to the Config rule with automatic remediation enabled.
• Create a new orphaned volume and watch it get automatically snapshotted and deleted.

Create an IAM role for SSM Automation

The SSM Automation document needs permission to interact with your EBS volumes. You create a dedicated IAM role that SSM can assume, with only the minimum permissions required for the snapshot-and-delete workflow.

• Create a trust policy file that allows the SSM service to assume this role by running:

cat > trust-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ssm.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

What does this trust policy do?

A trust policy defines WHO can assume the role. By setting the principal to ssm.amazonaws.com, you allow the Systems Manager Automation service to use this role when running remediation actions on your behalf.

• Create the IAM role using the trust policy by running:

aws iam create-role \
  --role-name SSMAutomationEBSRemediation \
  --assume-role-policy-document file://trust-policy.json

• You should see JSON output showing the role details including an Arn field. Copy the ARN value. It looks like arn:aws:iam::123456789012:role/SSMAutomationEBSRemediation.

• Create a permissions policy file that grants the actions the SSM document needs by running:

cat > ebs-remediation-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ssm:StartAutomationExecution",
        "ssm:GetAutomationExecution",
        "ec2:CreateSnapshot",
        "ec2:DeleteVolume",
        "ec2:DescribeSnapshots",
        "ec2:DescribeVolumes"
      ],
      "Resource": "*"
    }
  ]
}
EOF

Why these six permissions?

• ssm:StartAutomationExecution and ssm:GetAutomationExecution let the document launch and track its own automation steps.
• ec2:CreateSnapshot creates a safety backup before deletion.
• ec2:DeleteVolume removes the orphaned volume.
• ec2:DescribeVolumes lets the document verify the volume exists and is unattached.
• ec2:DescribeSnapshots lets the document confirm the snapshot completed before deleting.

• Attach the permissions policy to the role by running:

aws iam put-role-policy \
  --role-name SSMAutomationEBSRemediation \
  --policy-name EBSRemediationPermissions \
  --policy-document file://ebs-remediation-policy.json

• Verify the role exists and retrieve the ARN by running:

aws iam get-role --role-name SSMAutomationEBSRemediation --query "Role.Arn" --output text

• You should see the full role ARN printed. Save this value for the next substep.

Seeing an access denied error?

Make sure your AWS CLI user has iam:CreateRole and iam:PutRolePolicy permissions. If you are using an admin-level IAM user or role, this should work without issues.

Help me troubleshoot IAM role creation errors

Attach the remediation configuration to the Config rule

Now you connect your Config rule to an AWS-managed SSM document that handles the snapshot-and-delete workflow. When you enable automatic remediation, Config triggers this document every time it finds a non-compliant volume.

• Attach the remediation configuration to your ec2-volume-inuse-check rule by running this command. Replace the account ID placeholder with your 12-digit AWS account ID:

aws configservice put-remediation-configurations \
  --remediation-configurations '[{
    "ConfigRuleName": "ec2-volume-inuse-check",
    "TargetType": "SSM_DOCUMENT",
    "TargetId": "AWSConfigRemediation-DeleteUnusedEBSVolume",
    "Parameters": {
      "AutomationAssumeRole": {
        "StaticValue": {
          "Values": ["arn:aws:iam::[[AWS_ACCOUNT_ID="your 12-digit AWS account ID"]]:role/SSMAutomationEBSRemediation"]
        }
      },
      "VolumeId": {
        "ResourceValue": {
          "Value": "RESOURCE_ID"
        }
      },
      "CreateSnapshot": {
        "StaticValue": {
          "Values": ["true"]
        }
      }
    },
    "Automatic": true,
    "MaximumAutomaticAttempts": 5,
    "RetryAttemptSeconds": 60
  }]'

What does this configuration do?

• TargetId points to AWSConfigRemediation-DeleteUnusedEBSVolume, an AWS-managed SSM document that verifies a volume is unused, optionally snapshots it, then deletes it.
• CreateSnapshot is set to true so you keep a backup before any deletion.
• VolumeId uses RESOURCE_ID which dynamically resolves to the non-compliant resource ID at runtime.
• Automatic: true means Config triggers the remediation without human intervention.

• Verify the remediation configuration is attached by running:

aws configservice describe-remediation-configurations \
  --config-rule-names ec2-volume-inuse-check

• You should see JSON output showing AWSConfigRemediation-DeleteUnusedEBSVolume as the TargetId and "Automatic": true.

Getting an InvalidParameterValueException?

Double-check that the role ARN matches the exact ARN from the previous substep. Also confirm the Config rule name is ec2-volume-inuse-check (the display name you chose when creating the rule in Step 3).

Help me fix my put-remediation-configurations command

Trigger auto-remediation with a new unattached volume

Your original unattached volume may have already been remediated by the time you enabled auto-remediation. To reliably observe the full cycle, create a fresh unattached volume and watch it go through detection, flagging, and automatic cleanup.

• Create a new 1 GB unattached EBS volume by running:

aws ec2 create-volume \
  --availability-zone us-east-1a \
  --size 1 \
  --volume-type gp3 \
  --tag-specifications 'ResourceType=volume,Tags=[{Key=Purpose,Value=remediation-test}]'

• Note the VolumeId from the output. You will use it to confirm remediation happened.

Before you check the result, think about what you expect to happen. Do you think this volume will still exist in a few minutes, or will it be gone?

What happens behind the scenes?

Config evaluates the new volume against the ec2-volume-inuse-check rule. Because the volume is unattached, Config marks it NON_COMPLIANT. The auto-remediation triggers the SSM document, which creates a snapshot and then deletes the volume.

This process takes 3-5 minutes. Config needs to detect the new resource, evaluate it, and then trigger the remediation.

• Wait 3-5 minutes, then check if the volume still exists by running:

aws ec2 describe-volumes \
  --filters Name=status,Values=available \
  --query "Volumes[].VolumeId" \
  --output text

• If auto-remediation has fired, the new volume will no longer appear in this list. If it still shows up, wait another 2-3 minutes and run the command again.

• Confirm the SSM Automation execution completed successfully by running:

aws ssm describe-automation-executions \
  --filters Key=DocumentNamePrefix,Values=AWSConfigRemediation-DeleteUnusedEBSVolume \
  --query "AutomationExecutionMetadataList[0].{Status:AutomationExecutionStatus,StartTime:ExecutionStartTime}" \
  --output table

• You should see a table with Status: Success confirming the automation ran and completed.

• Open the AWS Management Console.
• Search for Systems Manager in the top search bar and open it.
• Click Automation in the left navigation under Change Management.
• Confirm you see a completed execution of AWSConfigRemediation-DeleteUnusedEBSVolume with a green Success status.

Volume still showing as available after 10 minutes?

• Check the Config rule evaluation. In the AWS Config console, select your rule and confirm the new volume appears as NON_COMPLIANT.
• Check Automation executions in Systems Manager. If an execution shows Failed, click into it to see which step failed. The most common cause is the IAM role missing permissions.
• You can manually trigger a re-evaluation of the rule by running:

aws configservice start-config-rules-evaluation \
  --config-rule-names ec2-volume-inuse-check

Help me debug why my Config auto-remediation is not firing

You now have a fully automated detect-and-fix loop. Config continuously watches for unattached EBS volumes, and SSM Automation snapshots and deletes them without any human intervention. Next up, you will see how Trusted Advisor and Cost Explorer surface these same problems from a cost reporting angle.

Explore Trusted Advisor and Cost Explorer

Your automation pipeline now detects and remediates orphaned resources without human intervention. Scripts find them, Config flags them, and SSM deletes them.

But there is one audience this pipeline does not yet serve: leadership. Engineers care about compliance. Managers care about dollars. Trusted Advisor and Cost Explorer surface the same orphaned-resource problems from a billing optimization lens, showing estimated monthly savings and actual spend over time.

In this step, get ready to:

• Review Trusted Advisor Cost Optimization checks for underutilized EBS volumes.
• View daily EBS spend in Cost Explorer filtered by EC2 - Other.
• (Bonus) Call the Trusted Advisor API programmatically with boto3.

Review Trusted Advisor Cost Optimization checks

Trusted Advisor inspects your AWS environment and surfaces recommendations to save money. The Cost Optimization category includes checks for underutilized EBS volumes, idle load balancers, and more.

Business Support plan required

Cost Optimization checks in Trusted Advisor require a Business, Enterprise On-Ramp, or Enterprise Support plan. If you are on the free tier or Developer Support, you will only see Service Limits and a few Security checks. You can still follow along with the screenshots and skip to the Cost Explorer substep.

• In the AWS Management Console, search for Trusted Advisor in the top search bar and select it.
• In the navigation pane on the left, choose Cost optimization.
• Scroll through the list of checks and find Underutilized Amazon EBS volumes.
• Click on the check name to open it.

You should see a table listing any flagged EBS volumes in your account. Each row shows the volume's region, ID, and estimated monthly savings. If your SSM automation already deleted the test volumes, this check may show no flagged resources. That is expected.

Why might the check show no results?

Trusted Advisor checks refresh automatically once per day. If your unattached volumes were created and deleted within a single day, the check may never have captured them. The Idle Load Balancers check (check ID hjLMh88uM8) only covers Classic Load Balancers, not ALB or NLB.

The key takeaway is that in a production account with many teams, this page would surface dozens of orphaned resources with concrete dollar amounts attached, giving leadership the justification to invest in the automation you just built.

• Note the Potential monthly savings figure shown on the Cost Optimization summary page. This is the number you would present to leadership.

View EBS spend in Cost Explorer

Cost Explorer shows actual dollar spend over time. It does not detect orphaned resources on its own. Instead, it answers the question: how much have these orphaned volumes been costing us?

• In the AWS Console search bar, search for Cost Explorer and select Cost Explorer under AWS Cost Management.
• If this is your first time opening Cost Explorer, it may take up to 24 hours to populate data. If you see a prompt to enable it, click Enable Cost Explorer.
• In the navigation pane, choose Cost Explorer, then Cost & Usage.
• In the Filters pane, click Service.
• Select EC2 - Other from the list. This service category includes EBS volume charges, EBS snapshots, and data transfer.
• Set the date range to the last 7 days.
• Use the Group by dropdown and select Usage Type to break the chart down by EBS volume hours, snapshot storage, and other sub-categories.

You should see a bar chart showing your daily EBS-related costs. Look for usage types that include volume charges. Even for a 1 GB gp3 volume that existed for a few hours, you will see a small cost appear.

What does Cost Explorer tell you that Config does not?

Config tells you a resource is non-compliant right now. Cost Explorer tells you how much that non-compliance has cost you over days, weeks, or months. Together, they give both the engineering signal (fix it) and the business signal (here is how much it costs to not fix it).

For programmatic access, the GetCostAndUsageWithResources API can pull this same data into a script or dashboard. It requires filtering by ResourceId and supports daily granularity for EC2 resources.

Cost Explorer shows no data?

Cost Explorer takes up to 24 hours to populate after first activation. If you enabled it earlier today, data may not appear until tomorrow. Check back then. The EBS charges from your project will show up under the EC2 - Other service dimension.

Help me understand why Cost Explorer is empty even though I created EBS volumes.

(Bonus) Query Trusted Advisor API with boto3

If you have a Business Support plan or higher, you can pull Trusted Advisor check results programmatically. This is useful for building automated cost-reporting dashboards.

What if I do not have Business Support?

The Trusted Advisor API returns a SubscriptionRequiredException if your account lacks Business Support or higher. If that happens, read through the code to understand the pattern, then move on to the next section. The API itself requires the us-east-1 endpoint regardless of where your resources live.

• Run the following boto3 snippet to retrieve the Underutilized Amazon EBS Volumes check results:

import boto3

# Trusted Advisor API requires the us-east-1 endpoint
support = boto3.client("support", region_name="us-east-1")

# Retrieve the Underutilized Amazon EBS Volumes check
result = support.describe_trusted_advisor_check_result(
    checkId="DAvU99Dc4C",
    language="en"
)

# Print flagged resources
flagged = result["result"]["flaggedResources"]
print(f"Flagged EBS volumes: {len(flagged)}")
for resource in flagged:
    print(f"  Resource ID: {resource['resourceId']}")
    print(f"  Metadata: {resource['metadata']}")
    print()

What does this code do?

• `boto3.client("support", region_name="us-east-1")` creates a client for the AWS Support API, which hosts Trusted Advisor. This endpoint only works in us-east-1.
• `describe_trusted_advisor_check_result` fetches the results for a specific check by its ID. The check ID DAvU99Dc4C corresponds to the Underutilized Amazon EBS Volumes check.
• The response contains a flaggedResources list with each flagged volume's resourceId and metadata (region, volume ID, size, monthly cost estimate).

✔️ I see flagged volumes listed

Your Business Support plan gives you full access to Trusted Advisor programmatically. The metadata fields contain the same information you saw in the console: region, volume ID, and estimated savings.

ⓧ I see SubscriptionRequiredException

This error means your account is on the free tier or Developer Support. The Trusted Advisor API is not available without Business Support or higher. You can still use the Config + SSM automation you built. It provides the same detection and remediation without needing the Trusted Advisor API.

Before you move on, think about what you have just seen. What do you expect would be the biggest difference between a production account's Trusted Advisor page and what you saw in your learning account?

In a production account, these checks surface hundreds of orphaned resources totalling thousands of dollars per month. The automation you built in the previous steps catches these before they accumulate.

You now have a complete picture: scripts for ad-hoc scanning, Config for continuous compliance, SSM for auto-remediation, and Trusted Advisor plus Cost Explorer for cost reporting. Next up, the Secret Mission adds real-time email notifications when remediation fires.

Secret mission

Add SNS Email Notifications for Auto-Remediation

Auto-remediation silently deletes resources behind the scenes. In production, that is dangerous without visibility. Your challenge: wire up SNS and EventBridge so you receive an email alert every time SSM Automation fires a remediation action.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project uses AWS Config recording and EBS volumes that incur ongoing charges if left active.

Cost warning

AWS Config charges approximately $0.003 per configuration item recorded. With rules active and the recorder running, expect $2-5/month depending on the number of resources in your account. Any remaining EBS snapshots also incur storage charges.

If you are not planning to continue building on this project, choose Pause or Delete to avoid unexpected charges.

Resources you used:

• AWS Config recorder and two Config rules (ec2-volume-inuse-check and acm-certificate-expiration-check).
• EBS snapshots created by SSM Automation.
• Imported ACM certificate (test.orphan-detection.example.com).
• IAM role SSMAutomationEBSRemediation with inline policy.
• SNS topic OrphanedResourceRemediationAlerts (Secret Mission).
• EventBridge rule SSMRemediationSuccessAlert (Secret Mission).
• Local project files (detect_orphans.py, detect_orphans.sh, generate_expiring_cert.py, test_cert.pem, test_key.pem).

Keep everything running

No action needed. Choose this if you are still actively building on this project or plan to extend it further.

• Your Config rules will continue monitoring for orphaned resources.
• Auto-remediation remains active and will snapshot-and-delete any new unattached EBS volumes.
• Expect approximately $2-5/month in Config recording costs depending on resource count.

Pause - I'll come back to this later

Stop the Config recorder so no new evaluations are recorded. Your rules and automation stay configured but inactive.

• Stop the Config recorder by running:
• This pauses all recording and rule evaluations without deleting your setup.

aws configservice stop-configuration-recorder --configuration-recorder-name default

• Your Config rules, IAM role, SNS topic, and EventBridge rule remain in place for when you return.
• To resume later, run aws configservice start-configuration-recorder --configuration-recorder-name default.

Delete - I don't want to use this again

Remove all project resources from your AWS account and local machine. Follow these steps in order.

• Delete both Config rules by running:

aws configservice delete-config-rule --config-rule-name ec2-volume-inuse-check
aws configservice delete-config-rule --config-rule-name acm-certificate-expiration-check

• Stop and delete the Config recorder by running:

aws configservice stop-configuration-recorder --configuration-recorder-name default
aws configservice delete-configuration-recorder --configuration-recorder-name default

• Delete any remaining EBS snapshots. List them first, then delete each one:

aws ec2 describe-snapshots --owner-ids self --query "Snapshots[].SnapshotId" --output text
aws ec2 delete-snapshot --snapshot-id your-snapshot-id

• Delete the imported ACM certificate. Find the ARN first, then delete it:

aws acm list-certificates --query "CertificateSummaryList[?DomainName=='test.orphan-detection.example.com'].CertificateArn" --output text
aws acm delete-certificate --certificate-arn your-certificate-arn

• Delete the IAM role. Remove its inline policy first, then delete the role:

aws iam list-role-policies --role-name SSMAutomationEBSRemediation
aws iam delete-role-policy --role-name SSMAutomationEBSRemediation --policy-name your-policy-name
aws iam delete-role --role-name SSMAutomationEBSRemediation

• Delete the EventBridge rule (Secret Mission). Remove targets first, then delete the rule:

aws events remove-targets --rule "SSMRemediationSuccessAlert" --ids "1"
aws events delete-rule --name "SSMRemediationSuccessAlert"

• Delete the SNS topic (Secret Mission). Replace the ARN with your topic ARN:

aws sns delete-topic --topic-arn your-sns-topic-arn

• Delete local project files by running:

rm detect_orphans.py detect_orphans.sh generate_expiring_cert.py test_cert.pem test_key.pem

• Optionally uninstall the Python packages you installed for this project:

pip uninstall cryptography boto3 -y

Nice Work!

Nice Work!

Nice work! You just built a production-grade automated compliance pipeline that detects orphaned AWS resources and remediates them without any human intervention.

You've learned how to:

• Built Python and Bash detection scripts that scan an AWS account for unattached EBS volumes and expiring ACM certificates, reporting estimated monthly waste for each.
• Configured AWS Config managed rules (ec2-volume-inuse-check and acm-certificate-expiration-check) for continuous compliance monitoring, and wired SSM Automation to auto-remediate non-compliant resources with a snapshot-and-delete sequence.
• Explored Trusted Advisor cost optimization checks and Cost Explorer for billing visibility, showing how leadership quantifies the savings your automation delivers.
• (Secret Mission) Connected EventBridge and SNS to send email alerts every time auto-remediation fires, ensuring your team has full visibility into automated resource cleanup.

Ready to quiz yourself?