Automated AWS Threat Detection Pipeline

Build a pipeline that detects threats and auto-isolates compromised EC2 instances.

Introduction

30 Second Summary

Think about how a smoke detector works. It monitors for danger 24/7, and the moment it detects smoke, it triggers an alarm automatically without you needing to be in the room.

In this project, you will build the cloud security equivalent: an automated threat detection and response pipeline on AWS. When a threat is detected, your system will automatically isolate the compromised resource in seconds.

What You'll Build

A fully automated security operations pipeline that detects threats with Amazon GuardDuty, routes findings through Amazon EventBridge, and triggers an AWS Lambda function to quarantine compromised resources.

By the end of this project, you'll have:

  • A GuardDuty detector continuously monitoring your AWS environment for threats, with AWS Security Hub aggregating findings into a unified dashboard.
  • An EventBridge rule that filters for high-severity findings and triggers automated remediation.
  • A Python Lambda function that automatically isolates compromised EC2 instances by swapping their security groups with a quarantine group that blocks all traffic.
  • Secret Mission: Extend the pipeline with SNS email notifications so you get alerted the moment your system auto-remediates a threat.

Are there any prerequisites?

You need an AWS account with admin access, the AWS CLI installed and configured, and Python 3 installed on your machine. Basic familiarity with AWS services (like navigating the console) is helpful but not required.

Before We Start

Before We Start...

Enable GuardDuty and Security Hub

Before you can detect and respond to threats automatically, you need the services that do the detecting.

In this step, you will enable Amazon GuardDuty (your threat detector) and AWS Security Hub (your unified security dashboard). Together, they form the eyes of your automated pipeline.

In this step, get ready to:
  • Enable Amazon GuardDuty to monitor your AWS environment for threats.
  • Enable AWS Security Hub to aggregate security findings.
  • Verify both services are active and integrated.
Enable Amazon GuardDuty

GuardDuty is a threat detection service that continuously monitors your AWS account for malicious activity. It analyzes CloudTrail management events, VPC Flow Logs, and DNS query logs to identify threats.

  • Enable GuardDuty by running this command in your terminal:
aws guardduty create-detector --enable

What does this command do?

This creates a GuardDuty detector in your current AWS region. A detector is the resource that represents the GuardDuty service. There is one detector per account per region.

The --enable flag activates the detector immediately so it starts analyzing your environment.

You should see output containing a DetectorId value. This is a unique identifier for your detector.

  • Copy and save the DetectorId value somewhere accessible (a text file or sticky note). You will need it in Step 5 when generating sample findings.

Seeing a detector already exists error?

If GuardDuty was previously enabled in this region, you already have a detector. Run the following command to find your existing detector ID:

aws guardduty list-detectors

Copy the ID from the output and continue to the next substep.

Help me troubleshoot my GuardDuty error

What about the free trial?

GuardDuty includes a 30-day free trial with full features and detections for new accounts. You will not be charged for this project.

Enable AWS Security Hub

Security Hub aggregates findings from GuardDuty and other services into a single, prioritized dashboard. When both services are enabled in the same account and region, the integration happens automatically.

  • Enable Security Hub by running this command:
aws securityhub enable-security-hub

What does this command do?

This enables Security Hub in your current region. It also automatically enables default security standards including AWS Foundational Security Best Practices and CIS AWS Foundations Benchmark.

Because you already enabled GuardDuty in the same region, Security Hub automatically starts importing GuardDuty findings. No additional configuration is needed for this integration.

This command produces no output on success. No news is good news here.

Seeing a ResourceConflictException error?

This means Security Hub is already enabled in your region. You can safely continue to the verification step below.

Help me troubleshoot my Security Hub error

What about the Security Hub free trial?

Security Hub includes a 30-day unlimited free trial for the Essentials plan. You will not be charged for this project.

Verify both services are active and integrated

Now confirm that both services are running and that GuardDuty findings will flow into Security Hub.

  • Verify GuardDuty is active by running:
aws guardduty list-detectors

You should see a JSON response with a DetectorIds array containing your detector ID.

  • Verify Security Hub is active by running:
aws securityhub describe-hub

You should see a JSON response that includes a HubArn and a SubscribedAt timestamp. This confirms Security Hub is enabled.

How does the GuardDuty and Security Hub integration work?

When you enable both GuardDuty and Security Hub in the same account and region, the integration is enabled automatically. GuardDuty starts sending all generated findings to Security Hub without any manual configuration.

New findings typically appear in Security Hub within five minutes of being generated.

Not seeing a HubArn in the response?

If describe-hub returns an error, Security Hub may not have finished enabling. Wait 30 seconds and try the command again.

Help me troubleshoot my Security Hub verification

Your detection layer is live. Next up, you will create the quarantine security group and IAM role that your Lambda function needs to isolate compromised instances.

Create the Quarantine Security Group and IAM Role

With GuardDuty watching for threats and Security Hub aggregating findings, you need a way to automatically respond when something bad happens.

In this step, you will build the isolation mechanism and the permissions your Lambda function needs. The quarantine security group blocks all traffic to and from a compromised instance. The IAM role gives your Lambda function permission to perform the isolation.

In this step, get ready to:
  • Create a quarantine security group that blocks all traffic.
  • Create an IAM role with least-privilege permissions for the Lambda function.
  • Attach the trust and permissions policies to the role.
Create the quarantine security group

A standard incident response technique is to replace a compromised instance's security groups with a restrictive "quarantine" group that blocks all traffic. This preserves the instance for forensic investigation while immediately containing the threat.

  • Create the quarantine security group by running this command:
aws ec2 create-security-group \
    --group-name quarantine-sg \
    --description "Quarantine security group - no traffic allowed"

What does this command do?

This creates a new security group named quarantine-sg in your default VPC. By default, AWS adds an outbound rule that allows all traffic out. You will remove that rule next so the group blocks everything.

  • Copy the GroupId value from the output (it looks like sg-0abc123def456789). You will need this later as your QUARANTINE_SG_ID.
  • Remove the default outbound rule so the group allows no traffic at all by running this command:
aws ec2 revoke-security-group-egress \
    --group-id [[QUARANTINE_SG_ID="your quarantine security group ID"]] \
    --ip-permissions '[{"IpProtocol":"-1","IpRanges":[{"CidrIp":"0.0.0.0/0"}]}]'

Why remove the outbound rule?

Every new security group comes with a default rule allowing all outbound traffic. Revoking it ensures the quarantine group has zero inbound and zero outbound rules. A compromised instance attached to this group cannot communicate with anything.

  • Verify the security group has no rules by running:
aws ec2 describe-security-groups \
    --group-ids [[QUARANTINE_SG_ID="your quarantine security group ID"]] \
    --query "SecurityGroups[0].{Inbound:IpPermissions,Outbound:IpPermissionsEgress}"

You should see empty arrays for both Inbound and Outbound.

Still seeing an outbound rule?

Make sure you replaced the placeholder with your actual security group ID from the create command output. The --ip-permissions JSON must match the default rule exactly (protocol -1, CIDR 0.0.0.0/0).

Help me troubleshoot why my revoke-security-group-egress command did not remove the default outbound rule from my quarantine security group.

Create the IAM role and policies

Your Lambda function needs permission to modify EC2 security groups and write logs. You will create a role with two policies: a trust policy (who can assume the role) and a permissions policy (what the role can do).

  • Create the trust policy file by running this command:
cat > trust-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

What is a trust policy?

A trust policy defines which AWS service or principal is allowed to assume (use) this role. Here, you are granting the Lambda service permission to assume the role. Without this, Lambda cannot execute with the role's permissions.

  • Create the permissions policy file by running this command:
cat > lambda-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeInstances",
        "ec2:ModifyInstanceAttribute",
        "ec2:DescribeSecurityGroups"
      ],
      "Resource": "*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*"
    }
  ]
}
EOF

What does this policy allow?

The first statement grants the three EC2 permissions the Lambda needs to find an instance and swap its security groups. The second statement grants CloudWatch Logs permissions so the Lambda can write execution logs for debugging and auditing.

This follows the principle of least privilege. The function only gets the exact permissions it needs to do its job.

  • Confirm both files were created by running:
ls trust-policy.json lambda-policy.json

You should see both filenames listed in the output.

  • Create the IAM role using the trust policy by running this command:
aws iam create-role \
    --role-name guardduty-remediation-role \
    --assume-role-policy-document file://trust-policy.json

What does this command do?

This creates a new IAM role named guardduty-remediation-role with the Lambda trust policy attached. The output includes the role's ARN (Amazon Resource Name). Copy it for later use.

  • Copy the Arn value from the output. It looks like arn:aws:iam::123456789012:role/guardduty-remediation-role.
  • Attach the permissions policy to the role by running this command:
aws iam put-role-policy \
    --role-name guardduty-remediation-role \
    --policy-name guardduty-remediation-policy \
    --policy-document file://lambda-policy.json

This command produces no output on success.

What is an inline policy?

The put-role-policy command embeds the permissions policy directly into the role. This is called an inline policy. It travels with the role and cannot be accidentally shared with other roles.

  • Verify the policy is attached by running:
aws iam list-role-policies --role-name guardduty-remediation-role

You should see guardduty-remediation-policy listed in the PolicyNames array.

Seeing an error when creating the role?

If you see EntityAlreadyExists, it means a role with that name already exists in your account. Either delete the existing role first or use a different name.

If you see a MalformedPolicyDocument error, check that your trust-policy.json file has valid JSON with no trailing commas or missing brackets.

Help me troubleshoot an error I received when trying to create the IAM role guardduty-remediation-role with my trust policy.

Your isolation infrastructure is ready. You have a quarantine security group that blocks all traffic and an IAM role with the exact permissions Lambda needs. Next up, you will write the Python function that uses these resources to automatically isolate compromised instances.

Write and Deploy the Auto-Remediation Lambda Function

Your IAM role and quarantine security group are ready. Now you need the code that actually performs the remediation.

In this step, you will write a Python function that receives a GuardDuty finding from EventBridge, identifies the compromised EC2 instance, and swaps its security groups with the quarantine group. Then you will package and deploy it to AWS Lambda.

In this step, get ready to:
  • Write the Python Lambda function that isolates compromised EC2 instances.
  • Package and deploy the function to AWS Lambda.
  • Verify the function is deployed correctly.
Write the auto-remediation Lambda function

The Lambda function needs to parse the GuardDuty finding event, extract the instance ID, and replace its security groups with the quarantine group.

  • In your code editor, create a new file called lambda_function.py in your project directory (the same folder where trust-policy.json and lambda-policy.json already exist).
  • Paste the following code into lambda_function.py:
import json
import boto3
import logging

# Set up logging so we can see what the function does in CloudWatch
logger = logging.getLogger()
logger.setLevel(logging.INFO)

# Create an EC2 client to modify security groups
ec2_client = boto3.client('ec2')

# The security group ID that blocks all traffic
QUARANTINE_SG_ID = 'your-quarantine-sg-id-here'

What does this code do?

This sets up the building blocks your function needs. boto3 is the AWS SDK for Python, which lets your code interact with AWS services like EC2.

The QUARANTINE_SG_ID variable stores the ID of the quarantine security group you created in the previous step. You will replace the placeholder with your actual security group ID shortly.

  • Below the imports, add the handler function that parses the GuardDuty event:
def lambda_handler(event, context):
    logger.info(f"Received event: {json.dumps(event)}")

    # Extract finding details from the EventBridge event
    detail = event.get('detail', {})
    finding_type = detail.get('type', 'Unknown')
    severity = detail.get('severity', 0)

    logger.info(f"Finding type: {finding_type}, Severity: {severity}")

    # Check if the finding involves an EC2 instance
    resource = detail.get('resource', {})
    resource_type = resource.get('resourceType', '')

    if resource_type != 'Instance':
        logger.info(f"Resource type is '{resource_type}', not an EC2 instance. Skipping.")
        return {
            'statusCode': 200,
            'body': json.dumps('No EC2 instance to remediate.')
        }

    # Get the instance ID from the finding
    instance_details = resource.get('instanceDetails', {})
    instance_id = instance_details.get('instanceId', '')

    if not instance_id:
        logger.error("No instance ID found in finding.")
        return {
            'statusCode': 400,
            'body': json.dumps('No instance ID found.')
        }

What does this code do?

The lambda_handler function is the entry point Lambda calls when triggered. It receives the EventBridge event payload containing the GuardDuty finding.

The code first checks if the finding involves an EC2 instance. GuardDuty can detect threats across multiple resource types, but this function only remediates EC2 instances. If the resource is not an instance, it exits gracefully.

  • Finally, add the isolation logic that swaps the instance's security groups:
    logger.info(f"Isolating instance: {instance_id}")

    try:
        # Look up the instance to get its current security groups
        response = ec2_client.describe_instances(InstanceIds=[instance_id])
        reservations = response.get('Reservations', [])
        if not reservations:
            logger.error(f"Instance {instance_id} not found.")
            return {
                'statusCode': 404,
                'body': json.dumps(f'Instance {instance_id} not found.')
            }

        instance = reservations[0]['Instances'][0]
        vpc_id = instance.get('VpcId', '')
        original_sgs = [sg['GroupId'] for sg in instance.get('SecurityGroups', [])]

        logger.info(f"Original security groups: {original_sgs}")
        logger.info(f"Applying quarantine security group: {QUARANTINE_SG_ID}")

        # Replace all security groups with the quarantine group
        ec2_client.modify_instance_attribute(
            InstanceId=instance_id,
            Groups=[QUARANTINE_SG_ID]
        )

        logger.info(f"Successfully isolated instance {instance_id}")

        return {
            'statusCode': 200,
            'body': json.dumps({
                'message': f'Instance {instance_id} isolated successfully.',
                'finding_type': finding_type,
                'severity': severity,
                'original_security_groups': original_sgs,
                'quarantine_security_group': QUARANTINE_SG_ID
            })
        }

    except Exception as e:
        logger.error(f"Error isolating instance {instance_id}: {str(e)}")
        return {
            'statusCode': 500,
            'body': json.dumps(f'Error: {str(e)}')
        }

What does this code do?

This is where the actual remediation happens. The function calls describe_instances to look up the compromised instance, then calls modify_instance_attribute to replace ALL of its security groups with the quarantine group.

Once the quarantine group is applied, no traffic can reach or leave the instance. The instance stays running (preserving forensic evidence), but it is completely network-isolated.

  • Now replace the placeholder your-quarantine-sg-id-here with the actual quarantine security group ID you recorded in the previous step.
  • Find this line near the top of lambda_function.py:
QUARANTINE_SG_ID = 'your-quarantine-sg-id-here'
  • Replace your-quarantine-sg-id-here with your actual security group ID (it looks like sg-0abc1234def56789), so the line looks like this:
QUARANTINE_SG_ID = 'sg-0abc1234def56789'
  • Save lambda_function.py.

Why hardcode the security group ID?

For this project, hardcoding the ID keeps things simple. In production, you would typically pass it as a Lambda environment variable so you can change it without redeploying code.

✔️ Awesome, I've got everything!

Great. Double check you have saved lambda_function.py with your actual quarantine security group ID.

ⓧ I'd like to double check the full code

Here is the full lambda_function.py file. Make sure yours matches (except with your actual security group ID in place of the placeholder):

import json
import boto3
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

ec2_client = boto3.client('ec2')

QUARANTINE_SG_ID = 'your-quarantine-sg-id-here'

def lambda_handler(event, context):
    logger.info(f"Received event: {json.dumps(event)}")

    detail = event.get('detail', {})
    finding_type = detail.get('type', 'Unknown')
    severity = detail.get('severity', 0)

    logger.info(f"Finding type: {finding_type}, Severity: {severity}")

    resource = detail.get('resource', {})
    resource_type = resource.get('resourceType', '')

    if resource_type != 'Instance':
        logger.info(f"Resource type is '{resource_type}', not an EC2 instance. Skipping.")
        return {
            'statusCode': 200,
            'body': json.dumps('No EC2 instance to remediate.')
        }

    instance_details = resource.get('instanceDetails', {})
    instance_id = instance_details.get('instanceId', '')

    if not instance_id:
        logger.error("No instance ID found in finding.")
        return {
            'statusCode': 400,
            'body': json.dumps('No instance ID found.')
        }

    logger.info(f"Isolating instance: {instance_id}")

    try:
        response = ec2_client.describe_instances(InstanceIds=[instance_id])
        reservations = response.get('Reservations', [])
        if not reservations:
            logger.error(f"Instance {instance_id} not found.")
            return {
                'statusCode': 404,
                'body': json.dumps(f'Instance {instance_id} not found.')
            }

        instance = reservations[0]['Instances'][0]
        vpc_id = instance.get('VpcId', '')
        original_sgs = [sg['GroupId'] for sg in instance.get('SecurityGroups', [])]

        logger.info(f"Original security groups: {original_sgs}")
        logger.info(f"Applying quarantine security group: {QUARANTINE_SG_ID}")

        ec2_client.modify_instance_attribute(
            InstanceId=instance_id,
            Groups=[QUARANTINE_SG_ID]
        )

        logger.info(f"Successfully isolated instance {instance_id}")

        return {
            'statusCode': 200,
            'body': json.dumps({
                'message': f'Instance {instance_id} isolated successfully.',
                'finding_type': finding_type,
                'severity': severity,
                'original_security_groups': original_sgs,
                'quarantine_security_group': QUARANTINE_SG_ID
            })
        }

    except Exception as e:
        logger.error(f"Error isolating instance {instance_id}: {str(e)}")
        return {
            'statusCode': 500,
            'body': json.dumps(f'Error: {str(e)}')
        }
Package and deploy the Lambda function

Lambda expects your code as a .zip file. Since this function only uses boto3 (which is included in the Lambda Python runtime), you just need to zip the single file.

  • Create the deployment package by running this command:
zip lambda.zip lambda_function.py

What does this command do?

The zip command packages lambda_function.py into a file called lambda.zip. Lambda will unzip this archive and look for the handler function inside.

  • Confirm lambda.zip was created by running:
ls lambda.zip

You should see lambda.zip listed in the output.

  • Deploy the Lambda function by running this command:
aws lambda create-function \
  --function-name guardduty-auto-remediation \
  --runtime python3.12 \
  --handler lambda_function.lambda_handler \
  --role [[ROLE_ARN="arn:aws:iam::123456789012:role/guardduty-remediation-role"]] \
  --zip-file fileb://lambda.zip

What does this command do?

  • --function-name names the function guardduty-auto-remediation.
  • --runtime python3.12 tells Lambda to use the Python 3.12 runtime (which includes boto3).
  • --handler lambda_function.lambda_handler tells Lambda the entry point is the lambda_handler function inside lambda_function.py.
  • --role is the ARN of the IAM role you created earlier.
  • --zip-file fileb://lambda.zip uploads the deployment package from your local machine. The fileb:// prefix tells the CLI to read binary file content.

You should see a JSON response containing details about the newly created function, including the FunctionArn.

  • Copy the FunctionArn value from the output and save it somewhere. You will need it in the next step to connect the function to EventBridge.

Seeing an error about the role?

If you see an error about the role not being found or not being assumable, wait 10 seconds and try again. IAM role propagation can take a few seconds after creation.

Make sure you are using the full ARN of the guardduty-remediation-role (it starts with arn:aws:iam::).

help me debug my Lambda create-function error

Verify the function is deployed correctly
  • Confirm the function exists and is active by running:
aws lambda get-function --function-name guardduty-auto-remediation

You should see a JSON response with the function configuration. Look for "State": "Active" in the output to confirm the function is ready.

Function state is Pending?

If you see "State": "Pending", wait a few seconds and run the get-function command again. Lambda needs a moment to provision resources after creation.

help me check if my Lambda function deployed correctly

Your Lambda function is deployed and ready to receive events. Next up, you will wire EventBridge to route high-severity GuardDuty findings to this function.

Create the EventBridge Rule and Connect the Pipeline

Your Lambda function is deployed and ready to isolate compromised instances. But right now, nothing tells it when to run.

In this step, you will create the routing layer that connects GuardDuty findings to your Lambda function. Amazon EventBridge acts as the event bus that listens for GuardDuty findings and forwards them to your remediation function.

In this step, get ready to:
  • Create an EventBridge rule that filters for high-severity GuardDuty findings.
  • Add your Lambda function as the rule's target.
  • Grant EventBridge permission to invoke your Lambda function.
Create the EventBridge rule

EventBridge rules evaluate incoming events against a pattern you define. Only events that match the pattern get forwarded to the target. You want to filter specifically for high-severity GuardDuty findings (severity 7 or above).

  • Create the EventBridge rule by running this command:
aws events put-rule \
  --name guardduty-high-severity-findings \
  --event-pattern '{"source":["aws.guardduty"],"detail-type":["GuardDuty Finding"],"detail":{"severity":[{"numeric":[">",7]}]}}' \
  --description "Routes high-severity GuardDuty findings to auto-remediation Lambda"

What does this command do?

This creates a rule on the default event bus named guardduty-high-severity-findings.

  • The --event-pattern filters for events where the source is aws.guardduty, the detail-type is GuardDuty Finding, and the severity is greater than 7 (meaning 7+ on the GuardDuty scale, which represents high-severity threats).
  • The numeric operator lets EventBridge compare the severity value instead of matching an exact number.
  • You should see output containing a RuleArn value. Copy this ARN and save it somewhere. You will need it in the next substep.

Seeing an invalid event pattern error?

  • Make sure the JSON in --event-pattern is wrapped in single quotes on the outside and uses double quotes inside. Mixing these up causes parsing failures.
  • Check that there are no trailing commas or missing brackets in the JSON pattern.

Help me debug my EventBridge put-rule command for GuardDuty findings.

Add the Lambda function as the rule target

The rule knows which events to match, but it does not know where to send them yet. You need to add your Lambda function as the target so EventBridge forwards matching findings to it.

  • Add your Lambda function as the target of the rule by running this command:
aws events put-targets \
  --rule guardduty-high-severity-findings \
  --targets Id=1,Arn=[[LAMBDA_ARN="arn:aws:lambda:REGION:ACCOUNT_ID:function/guardduty-auto-remediation"]]

What does this command do?

The put-targets command attaches one or more targets to an existing rule. Here you are telling EventBridge to send all matched events to your guardduty-auto-remediation Lambda function.

The Id=1 is a unique identifier for this target within the rule. The Arn is your Lambda function's ARN that you recorded when you deployed it in the previous step.

  • You should see output with "FailedEntryCount": 0 confirming the target was added successfully.

Seeing FailedEntryCount: 1?

  • Double-check that your Lambda function ARN is correct. You can find it by running aws lambda get-function --function-name guardduty-auto-remediation and looking for the FunctionArn field.
  • Make sure the rule name guardduty-high-severity-findings matches exactly what you used in the previous command.

Help me fix my EventBridge put-targets command.

Grant EventBridge permission to invoke your Lambda function

Even though EventBridge knows to send events to your Lambda, the Lambda function itself needs to allow EventBridge to invoke it. This is a resource-based policy on the Lambda side.

  • Grant EventBridge permission to invoke your Lambda function by running this command:
aws lambda add-permission \
  --function-name guardduty-auto-remediation \
  --statement-id AllowEventBridgeInvoke \
  --action lambda:InvokeFunction \
  --principal events.amazonaws.com \
  --source-arn [[RULE_ARN="arn:aws:events:REGION:ACCOUNT_ID:rule/guardduty-high-severity-findings"]]

What does this command do?

This adds a resource-based policy to your Lambda function. The policy says: only the EventBridge service (events.amazonaws.com) is allowed to invoke this function, and only from your specific rule.

The --source-arn restricts the permission to your exact rule. Without it, any EventBridge rule in your account could invoke this function.

  • You should see a JSON response containing a Statement field with the permission details. This confirms the policy was added.

Seeing a ResourceConflictException?

  • This means a permission with the same --statement-id already exists. If you ran the command before, the permission is already in place and you can proceed.
  • If you need to update it, first remove it with aws lambda remove-permission --function-name guardduty-auto-remediation --statement-id AllowEventBridgeInvoke and then re-run the add-permission command.

Help me fix my Lambda add-permission command.

Why a resource-based policy instead of an IAM role?

For Lambda and SNS targets, EventBridge relies on resource-based policies rather than IAM roles. The resource-based policy lives on the Lambda function itself and explicitly allows the EventBridge service to call it.

This is the least privilege approach. Your Lambda only accepts invocations from your specific rule ARN, not from any event source.

Your automated pipeline is fully wired. GuardDuty findings flow into EventBridge, your rule filters for high-severity threats, and your Lambda function receives those events to perform the isolation. Next up, you will test the entire pipeline with sample findings.

Test the Pipeline with Sample Findings

Your pipeline is fully wired. GuardDuty detects threats. EventBridge routes high-severity findings to your Lambda function, which performs the isolation.

But how do you know it actually works without a real attack? GuardDuty provides a way to generate sample findings that simulate threats. These sample findings flow through EventBridge just like real ones, letting you validate the entire pipeline end-to-end.

In this step, get ready to:
  • Generate sample GuardDuty findings using the AWS CLI.
  • Verify the EventBridge rule triggers the Lambda function.
  • Check CloudWatch Logs and Security Hub to confirm end-to-end execution.
Generate sample GuardDuty findings

GuardDuty's sample findings feature creates realistic-looking threat events without requiring an actual attack. These findings trigger EventBridge events just like real detections do.

  • Retrieve your GuardDuty detector ID by running this command:
aws guardduty list-detectors

You should see output containing your unique detector ID:

{
    "DetectorIds": [
        "12abc34d567e8fa901bc2d34eexample"
    ]
}
  • Copy the detector ID from the output. You will use it in the next command.
  • Generate sample findings that simulate EC2-based threats by running this command:
aws guardduty create-sample-findings \
    --detector-id [[DETECTOR_ID="your GuardDuty detector ID"]] \
    --finding-types UnauthorizedAccess:EC2/TorClient UnauthorizedAccess:EC2/TorRelay

What does this command do?

The create-sample-findings command tells GuardDuty to generate simulated threat findings. The two finding types simulate an EC2 instance communicating with the Tor anonymity network, which is a common indicator of compromise.

These sample findings have [SAMPLE] prefixed in their titles and contain fictional instance IDs. They still trigger EventBridge events, which makes them perfect for pipeline testing.

  • The command produces no output if successful. No output means the findings were generated.

Seeing an error about the detector ID?

Make sure you replaced the placeholder with the actual detector ID from the list-detectors output. The ID is a long alphanumeric string.

If you see a permissions error, confirm your AWS CLI is configured with admin credentials.

Help me troubleshoot the create-sample-findings command

Verify the Lambda function was triggered

GuardDuty publishes new findings to EventBridge within about five minutes. Your EventBridge rule matches high-severity findings and invokes the Lambda function automatically.

  • Wait about 5 minutes for the findings to propagate through EventBridge to your Lambda function.
  • In the AWS Console, search for CloudWatch in the top search bar and select it.
  • In the left sidebar, expand Logs and click Log groups.
  • Find and click the log group named /aws/lambda/guardduty-auto-remediation.
  • Click the most recent log stream (the one at the top of the list).

You should see log entries showing that your Lambda function received and processed the GuardDuty finding events.

What should I see in the logs?

Look for these log entries that prove your pipeline is working:

  • "Received event:" followed by the full GuardDuty finding JSON.
  • "Finding type: UnauthorizedAccess:EC2/TorClient" with a severity value.
  • "Isolating instance:" followed by a fictional instance ID.
  • "Error isolating instance" because the sample finding references a non-existent instance.

The error at the end is expected. Sample findings use fictional instance IDs that do not exist in your account. The important thing is that the Lambda received the event and attempted the isolation. With a real threat targeting a real instance, the quarantine would succeed.

Don't see the log group?

Logs may take 5 to 10 minutes to appear after a function invocation. Wait a few more minutes and refresh the Log groups page.

If the log group still does not appear after 10 minutes, the EventBridge rule may not have matched the sample findings. Confirm the rule guardduty-high-severity-findings is enabled in the EventBridge console under Rules.

Help me troubleshoot why my Lambda is not being triggered

Review findings in Security Hub

Security Hub automatically aggregates GuardDuty findings into a unified dashboard. Since you enabled both services in the same account and region, the integration is automatic.

  • In the AWS Console search bar, search for Security Hub and select it.
  • In the left sidebar, click Findings.
  • You should see the sample GuardDuty findings listed with their severity ratings and finding types.

What is Security Hub showing me?

Security Hub presents all your security findings in a single prioritized view. Each finding shows the severity level, affected resource, finding type, and detection time.

In a production environment, this is where your security team would triage incidents. Combined with your automated Lambda remediation, they would arrive to find the compromised instance already isolated and ready for forensic investigation.

Filter for GuardDuty findings

If you have many findings, use the filter bar at the top. Add a filter for Product name equals GuardDuty to see only the findings from this project.

Your automated threat detection and response pipeline is fully verified. GuardDuty detects threats, EventBridge routes them, Lambda remediates automatically, and Security Hub gives you the unified view. Next up, try the Secret Mission to add email notifications so you know the moment a remediation happens.

Secret mission

Add Email Notifications via SNS

Your pipeline detects and isolates threats automatically. But what if you're away from the console when it happens? In this secret mission, you'll wire up Amazon SNS to send you an email notification every time the Lambda quarantines an instance. Real-time alerting closes the loop between automated remediation and human awareness.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. GuardDuty and Security Hub both have 30-day free trials. After the trial ends, charges apply based on usage.

Cost warning

GuardDuty and Security Hub are free for the first 30 days. After that, GuardDuty charges based on log volume (CloudTrail events, VPC Flow Logs, DNS queries) and Security Hub charges per resource evaluated.

If you are not actively using these services after the trial, disable or delete them to avoid unexpected charges.

Resources you used:

  • GuardDuty detector (free for 30 days, then pay-as-you-go).
  • Security Hub (free for 30 days, then per-resource pricing).
  • Lambda function: guardduty-auto-remediation (free tier covers 1M requests/month).
  • EventBridge rule: guardduty-high-severity-findings (free for AWS service events).
  • IAM role: guardduty-remediation-role with inline policy (free).
  • Security group: quarantine-sg (free).
  • SNS topic: guardduty-alerts (free tier).
  • Local project files: lambda_function.py, trust-policy.json, lambda-policy.json.

Keep everything running

No action needed. Choose this if you want GuardDuty to keep monitoring your environment and the auto-remediation pipeline to stay active.

  • GuardDuty continues detecting threats in near real time.
  • EventBridge continues routing findings to your Lambda function.
  • Security Hub continues aggregating findings into a unified dashboard.
  • After the 30-day free trial, GuardDuty and Security Hub charges apply based on usage volume.

Pause - I'll come back to this later

Disable the EventBridge rule to stop automated remediation while keeping detection active. GuardDuty still monitors your environment, but no Lambda actions fire automatically.

  • Disable the EventBridge rule by running:
aws events disable-rule --name guardduty-high-severity-findings
  • GuardDuty and Security Hub remain enabled. Findings still accumulate in the Security Hub dashboard.
  • To resume the pipeline later, re-enable the rule by running:
aws events enable-rule --name guardduty-high-severity-findings

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow these steps in order.

  • Remove the EventBridge rule target, then delete the rule:
aws events remove-targets --rule guardduty-high-severity-findings --ids 1
aws events delete-rule --name guardduty-high-severity-findings
  • Delete the Lambda function:
aws lambda delete-function --function-name guardduty-auto-remediation
  • Delete the SNS topic (if you completed the Secret Mission):
aws sns delete-topic --topic-arn [[SNS_TOPIC_ARN="your SNS topic ARN"]]
  • List the inline policies on the IAM role, then delete the policy and the role:
aws iam list-role-policies --role-name guardduty-remediation-role
  • Use the policy name from the output above in this command:
aws iam delete-role-policy --role-name guardduty-remediation-role --policy-name [[POLICY_NAME="your inline policy name from the output above"]]
aws iam delete-role --role-name guardduty-remediation-role
  • Delete the quarantine security group:
aws ec2 delete-security-group --group-id [[QUARANTINE_SG_ID="your quarantine security group ID"]]
  • Disable Security Hub:
aws securityhub disable-security-hub
  • Delete the GuardDuty detector:
aws guardduty delete-detector --detector-id [[DETECTOR_ID="your GuardDuty detector ID"]]
  • Delete your local project files:
rm lambda_function.py trust-policy.json lambda-policy.json lambda_function.zip

Nice Work!

Nice Work!

Nice work! You just built a production-grade automated threat detection and response pipeline on AWS from scratch.

You've learned how to:

  • Built an event-driven security pipeline using GuardDuty, EventBridge, and Lambda that detects threats and remediates them automatically.
  • Created a Python Lambda function that isolates compromised EC2 instances by replacing their security groups with a quarantine security group that blocks all traffic.
  • Wired EventBridge rules to filter high-severity GuardDuty findings and tested the full pipeline end-to-end using sample findings verified in CloudWatch Logs and Security Hub.
  • Extended the pipeline with SNS email notifications that alert you in real time when the Lambda auto-remediates a threat.

Ready to quiz yourself?