Build a Highly Available E-Commerce on AWS

Introduction

30 Second Summary

Every time you buy something online, there is an entire infrastructure behind the scenes handling your clicks, securing your payment, and keeping the site running even during a flash sale. Building that infrastructure from scratch is the ultimate test of a cloud architect's skills.

In this project, you will build a production-grade e-commerce infrastructure called "CloudShop" on AWS, wiring together 30+ services across networking, compute, databases, serverless pipelines, analytics, and security into one cohesive system.

What You'll Build

A multi-tier, multi-AZ AWS infrastructure with a custom VPC, Auto Scaling compute layer, relational and NoSQL databases, caching, CDN delivery, serverless event-driven pipelines, real-time analytics, security hardening, and full Infrastructure as Code documentation in CloudFormation.

By the end of this project, you'll have:

• A multi-AZ VPC with public and private subnets, a NAT Gateway, security groups, and an Application Load Balancer fronting an Auto Scaling Group of EC2 instances connected to RDS, DynamoDB, and ElastiCache.
• A serverless event-driven pipeline using Lambda, SQS, SNS, and Step Functions for order processing, plus a real-time analytics pipeline with Kinesis Data Firehose and Athena.
• A complete Architecture Decision Record documenting every design choice with GCP-to-AWS mental model mappings, and a CloudFormation template that can recreate the entire infrastructure.
• Secret Mission: Add an ECS Fargate microservice alongside the EC2 tier with path-based routing, and configure automated budget alerts.

Are there any prerequisites?

This project assumes you have an AWS account (Paid plan recommended), the AWS CLI installed, and a solid understanding of cloud architecture concepts. If you hold a GCP Professional Cloud Architect certification or equivalent experience, you are ready to go.

The project uses the us-east-1 region throughout. Make sure your console is set to this region before starting.

Before We Start

Building a production-grade AWS infrastructure requires a properly configured local environment. You need your terminal to talk securely to AWS services before you can create any cloud resources.

In this step, you will set up the AWS CLI with a dedicated project profile, create an IAM user with the right permissions, and initialize your project workspace with an Architecture Decision Record.

In this step, get ready to:

• Verify AWS CLI v2 and configure a named profile for the project.
• Create an IAM user with AdministratorAccess and enable MFA.
• Set up the project directory and initialize an Architecture Decision Record (ADR).

Install and verify AWS CLI v2

The AWS CLI is the command-line tool that lets you interact with every AWS service from your terminal. This project requires version 2, which supports all the latest features you will use throughout the build.

• Open your terminal by pressing Cmd+Space (macOS) to open Spotlight, then type Terminal and press Enter.
• Check your installed AWS CLI version by running this command:

aws --version

✔️ I see aws-cli/2.x.x

You have AWS CLI v2 installed and ready to go. Move on to the next instruction below.

ⓧ I see aws-cli/1.x.x

You have an older version installed. You need to upgrade to version 2.

• Download the official AWS CLI v2 installer by running:

curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"

• Run the installer:

sudo installer -pkg AWSCLIV2.pkg -target /

• Close and reopen your terminal, then run aws --version again to confirm you see aws-cli/2.x.x.

ⓧ Command not found

AWS CLI is not installed yet. Install it using the official macOS package.

• Download the installer by running:

curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"

• Run the installer:

sudo installer -pkg AWSCLIV2.pkg -target /

• Close and reopen your terminal, then run aws --version to confirm you see aws-cli/2.x.x.

Now configure a dedicated named profile for the CloudShop project. This keeps your project credentials separate from any other AWS work on your machine.

• Configure the cloudshop profile by running this command:

aws configure --profile cloudshop

What does this command do?

The aws configure --profile cloudshop command creates a named profile called cloudshop in your AWS configuration files. It will prompt you for four values in order.

Named profiles let you manage multiple AWS accounts or projects from the same machine without mixing credentials.

The command will prompt you for four values. Enter them as follows:

• For AWS Access Key ID: enter the access key from your IAM user (you will create this in the next substep if you do not have one yet).
• For AWS Secret Access Key: enter your secret key.
• For Default region name: enter us-east-1.
• For Default output format: enter json.

Why us-east-1?

The us-east-1 region (N. Virginia) has the widest service availability and is the default region for many AWS features. For a project that uses 30+ services, this ensures you never hit a "service not available in this region" problem.

Create an IAM user with AdministratorAccess

You need an IAM user dedicated to this project so you can track API calls and manage permissions cleanly. If you already have an IAM user with AdministratorAccess and access keys, skip to the MFA step below.

• Open the AWS Management Console in your browser and sign in.
• Search for IAM in the top search bar and select the IAM service.
• Click Users in the left sidebar.
• Click Create user.
• Enter cloudshop-admin as the user name.
• Click Next.
• Select Attach policies directly.
• Search for AdministratorAccess and check the box next to it.
• Click Next.
• Click Create user.

Now create access keys for this user:

• Click on the cloudshop-admin user you just created.
• Click the Security credentials tab.
• Scroll down to Access keys and click Create access key.
• Select Command Line Interface (CLI) as the use case.
• Check the confirmation box at the bottom.
• Click Next.
• Click Create access key.
• Copy the Access key ID and Secret access key and store them securely. You will not be able to see the secret key again after closing this page.

Save your credentials securely

Copy these credentials to a secure location (a password manager is ideal). Once you close this page, AWS will never show the secret access key again. If you lose it, you will need to create a new key pair.

If you have not yet run aws configure --profile cloudshop, do so now and enter these credentials when prompted. If you already ran it with placeholder values, run it again to update.

Now enable MFA for your IAM user:

• On the cloudshop-admin user page, click the Security credentials tab.
• Under Multi-factor authentication (MFA), click Assign MFA device.
• Enter a device name (e.g. cloudshop-mfa) and select Authenticator app.
• Click Next.
• Scan the QR code with your authenticator app (Google Authenticator, Authy, or similar).
• Enter two consecutive MFA codes from your authenticator app.
• Click Add MFA.

Finally, verify that your CLI can authenticate successfully:

• Verify your identity by running this command:

aws sts get-caller-identity --profile cloudshop

What should I see?

You should see a JSON response containing your UserId, Account number, and Arn (Amazon Resource Name) for the cloudshop-admin user. This confirms your credentials are working.

Seeing an error?

If you see an "Unable to locate credentials" error, make sure you ran aws configure --profile cloudshop and entered both your access key ID and secret access key.

If you see an "InvalidClientTokenId" error, double-check that you copied the access key ID correctly from the IAM console.

Help me troubleshoot my AWS CLI configuration

Create the project directory and Architecture Decision Record

Every decision you make in this project is worth recording. An Architecture Decision Record (ADR) captures what you chose, why you chose it, and how it maps to the GCP services you already know from your PCA certification.

• Create the project directory and move into it by running these commands:

mkdir ~/cloudshop
cd ~/cloudshop

• Confirm the directory was created by running:

ls ~/cloudshop

The output should be empty since you just created the folder.

• Create the ADR document by running this command:

cat > ~/cloudshop/ADR.md << 'EOF'
# CloudShop Architecture Decision Record

## Template
- **Decision**: 
- **Context**: 
- **Rationale**: 
- **GCP Equivalent**: 
- **SAA-C03 Exam Relevance**: 
- **Alternatives Considered**: 

EOF

What is an Architecture Decision Record?

An ADR is a document that captures important architectural decisions along with their context and consequences. Cloud engineers use ADRs on real teams to communicate design rationale.

Your ADR template includes a GCP Equivalent field because you already understand GCP from your PCA certification. Mapping each AWS service to its GCP counterpart accelerates your learning. The SAA-C03 Exam Relevance field links each decision to the specific exam domain it covers.

• Verify the file was created correctly by running:

cat ~/cloudshop/ADR.md

You should see the full ADR template printed in your terminal.

File is empty or missing?

Make sure you copied the entire cat > ~/cloudshop/ADR.md << 'EOF' block including the EOF at the end. The heredoc syntax requires EOF on its own line to close the file.

Help me fix my ADR file creation

✔️ Awesome, I've got everything!

Great. Make sure your ~/.aws/config file and ~/cloudshop/ADR.md file are both saved.

ⓧ I'd like to double check the full code

Here is what your ~/.aws/config file should contain for the cloudshop profile:

[profile cloudshop]
region = us-east-1
output = json

Here is what your ~/cloudshop/ADR.md file should contain:

# CloudShop Architecture Decision Record

## Template
- **Decision**: 
- **Context**: 
- **Rationale**: 
- **GCP Equivalent**: 
- **SAA-C03 Exam Relevance**: 
- **Alternatives Considered**: 

Your environment is ready. Next up, you will design the VPC and network foundation that everything else runs on.

Set Up VPC and Network Foundation

Set Up Your VPC and Network Foundation

In the previous step, you configured your AWS CLI and initialized the Architecture Decision Record. Now it's time to build the network foundation that everything else will live on.

Every production system needs network isolation between its tiers. A web server accepting public traffic should not sit on the same network segment as a database holding customer data. In this step, you'll build a multi-AZ VPC with public, private-app, and private-data subnets, wire up internet access through an Internet Gateway and NAT Gateway, and lock down traffic with security groups and NACLs.

In this step, get ready to:

• Create a custom VPC with 6 subnets across 2 Availability Zones.
• Configure internet and NAT gateways with route tables for public and private traffic.
• Set up security groups, Network ACLs, and a VPC Gateway Endpoint for S3.

Create your VPC and 6 subnets

AWS VPC is regional, meaning your subnets are tied to specific Availability Zones within a region. This differs from GCP where VPCs are global. You'll create a 3-tier subnet layout: public subnets for load balancers, private-app subnets for compute, and private-data subnets for databases.

• Return to the AWS Management Console in your browser.
• In the search bar at the top, type VPC and select VPC from the results.
• On the VPC dashboard, click Create VPC.
• For Resources to create, select VPC only.
• In the Name tag field, enter CloudShop-VPC.
• For IPv4 CIDR block, enter 10.0.0.0/16.
• Leave IPv6 CIDR block set to No IPv6 CIDR block.
• Leave Tenancy set to Default.
• Click Create VPC.

You should see a green success banner confirming the VPC was created.

Why VPC only instead of VPC and more?

The "VPC and more" wizard creates subnets automatically, but it only supports public and private tiers. CloudShop needs a 3-tier layout (public, private-app, private-data), so you'll create each subnet manually for full control over naming and CIDR allocation.

• Now enable DNS hostnames on the VPC. Select your new CloudShop-VPC from the VPC list.
• Click Actions, then Edit VPC settings.
• Check the box for Enable DNS hostnames.
• Click Save.

Why enable DNS hostnames?

DNS hostnames allow EC2 instances to receive public DNS names. This is required for services like RDS endpoints and for instances to be addressable by hostname within the VPC.

Next, create the 6 subnets. You'll create them one at a time with specific CIDR blocks and Availability Zone assignments.

• In the left navigation pane, click Subnets.
• Click Create subnet.
• For VPC ID, select CloudShop-VPC from the dropdown.

You can add multiple subnets on this page. Configure the first subnet with these values:

• Set Subnet name to CloudShop-public-1a.
• Set Availability Zone to us-east-1a.
• Set IPv4 subnet CIDR block to 10.0.1.0/24.
• Click Add new subnet to add the next one.

Repeat for the remaining 5 subnets using these values:

• Subnet 2: Name CloudShop-public-1b, AZ us-east-1b, CIDR 10.0.2.0/24.
• Subnet 3: Name CloudShop-private-app-1a, AZ us-east-1a, CIDR 10.0.3.0/24.
• Subnet 4: Name CloudShop-private-app-1b, AZ us-east-1b, CIDR 10.0.4.0/24.
• Subnet 5: Name CloudShop-private-data-1a, AZ us-east-1a, CIDR 10.0.5.0/24.
• Subnet 6: Name CloudShop-private-data-1b, AZ us-east-1b, CIDR 10.0.6.0/24.

• After entering all 6 subnets, click Create subnet.

You should see all 6 subnets listed in the Subnets page with their correct CIDR blocks and AZ assignments.

What is the 3-tier subnet design?

Public subnets hold resources that need direct internet access (load balancers, NAT gateways). Private-app subnets hold compute resources that receive traffic from the load balancer but don't face the internet directly. Private-data subnets hold databases and caches that should only be reachable from the app tier.

Spreading each tier across two AZs (us-east-1a and us-east-1b) gives you high availability. If one AZ goes down, the other keeps your application running.

• Enable auto-assign public IPv4 on your public subnets. Select CloudShop-public-1a from the subnet list.
• Click Actions, then Edit subnet settings.
• Check Enable auto-assign public IPv4 address.
• Click Save.
• Repeat for CloudShop-public-1b.

Don't see your VPC in the subnet dropdown?

Make sure you're in the us-east-1 region. Check the region selector in the top-right corner of the console. If you created the VPC in a different region, switch back.

help me troubleshoot why my VPC isn't showing up in the subnet creation dropdown

Set up internet and NAT gateways with route tables

Subnets alone don't have internet access. You need an Internet Gateway for public subnets and a NAT Gateway for private subnets to reach the internet (for software updates, API calls, etc.) without being directly reachable from outside.

• In the left navigation pane, click Internet gateways.
• Click Create internet gateway.
• For Name tag, enter CloudShop-IGW.
• Click Create internet gateway.
• On the success page, click Attach to a VPC from the banner at the top.
• Select CloudShop-VPC and click Attach internet gateway.

Now create a NAT Gateway so private subnets can reach the internet. A NAT Gateway needs an Elastic IP address.

• In the left navigation pane, click Elastic IPs.
• Click Allocate Elastic IP address.
• Leave the defaults and click Allocate.
• Note the Allocation ID that appears. You'll need it in the next step.

• In the left navigation pane, click NAT gateways.
• Click Create NAT gateway.
• For Name, enter CloudShop-NAT.
• For Subnet, select CloudShop-public-1a.
• For Elastic IP allocation ID, select the Elastic IP you just allocated.
• Click Create NAT gateway.

Why only one NAT Gateway?

NAT Gateways cost $0.045 per hour plus $0.045 per GB of data processed. For a production environment, you'd place one NAT Gateway in each AZ for full high availability. For this non-production lab, a single NAT Gateway saves money while still demonstrating the pattern.

The NAT Gateway takes a few minutes to reach the "Available" state. While it provisions, set up the route tables.

• In the left navigation pane, click Route tables.
• Click Create route table.
• For Name, enter CloudShop-Public-RT.
• For VPC, select CloudShop-VPC.
• Click Create route table.

• With the new route table selected, click the Routes tab, then click Edit routes.
• Click Add route.
• For Destination, enter 0.0.0.0/0.
• For Target, select Internet Gateway, then choose CloudShop-IGW.
• Click Save changes.

• Now associate the public route table with both public subnets. Click the Subnet associations tab.
• Click Edit subnet associations.
• Select CloudShop-public-1a and CloudShop-public-1b.
• Click Save associations.

Now create the private route table that routes outbound traffic through the NAT Gateway.

• Click Create route table again.
• For Name, enter CloudShop-Private-RT.
• For VPC, select CloudShop-VPC.
• Click Create route table.

• With the private route table selected, click the Routes tab, then click Edit routes.
• Click Add route.
• For Destination, enter 0.0.0.0/0.
• For Target, select NAT Gateway, then choose CloudShop-NAT.
• Click Save changes.

• Click the Subnet associations tab, then click Edit subnet associations.
• Select all 4 private subnets: CloudShop-private-app-1a, CloudShop-private-app-1b, CloudShop-private-data-1a, and CloudShop-private-data-1b.
• Click Save associations.

NAT Gateway not appearing as a target?

The NAT Gateway must be in the "Available" state before it appears as a route target. Go to NAT gateways in the left navigation and check the status. It typically takes 1-2 minutes to provision.

help me troubleshoot why my NAT Gateway is not available as a route target

Configure security groups, NACLs, VPC endpoint, and update the ADR

Security groups act as virtual firewalls at the instance level. Network ACLs add a second layer of defense at the subnet level. Together, they implement defense in depth.

• In the left navigation pane, click Security groups.
• Click Create security group.
• For Security group name, enter CloudShop-ALB-SG.
• For Description, enter Allow HTTP/HTTPS from internet.
• For VPC, select CloudShop-VPC.

• Under Inbound rules, click Add rule.
• Set Type to HTTP, Source to 0.0.0.0/0.
• Click Add rule again. Set Type to HTTPS, Source to 0.0.0.0/0.
• Click Create security group.

Now create the App-tier security group that only accepts traffic from the ALB.

• Click Create security group again.
• For Security group name, enter CloudShop-App-SG.
• For Description, enter Allow HTTP/HTTPS from ALB only.
• For VPC, select CloudShop-VPC.
• Under Inbound rules, click Add rule. Set Type to HTTP, Source to the CloudShop-ALB-SG security group.
• Click Add rule again. Set Type to HTTPS, Source to the CloudShop-ALB-SG security group.
• Click Create security group.

Finally, create the Data-tier security group that only accepts database and cache traffic from the app tier.

• Click Create security group again.
• For Security group name, enter CloudShop-Data-SG.
• For Description, enter Allow MySQL and Redis from App tier only.
• For VPC, select CloudShop-VPC.
• Under Inbound rules, click Add rule. Set Type to MYSQL/Aurora (port 3306), Source to the CloudShop-App-SG security group.
• Click Add rule again. Set Type to Custom TCP, Port range to 6379, Source to the CloudShop-App-SG security group.
• Click Create security group.

What is least-privilege in security groups?

Each security group only allows the minimum traffic needed. The ALB accepts traffic from the internet. The app tier only accepts traffic from the ALB. The data tier only accepts traffic from the app tier. This means an attacker who compromises the ALB cannot directly reach the database.

Now configure Network ACLs for extra protection on the data-tier subnets.

• In the left navigation pane, click Network ACLs.
• Click Create network ACL.
• For Name, enter CloudShop-Data-NACL.
• For VPC, select CloudShop-VPC.
• Click Create network ACL.

• Select your new CloudShop-Data-NACL, then click the Inbound rules tab.
• Click Edit inbound rules.
• Click Add new rule. Set Rule number to 100, Type to MYSQL/Aurora (3306), Source to 10.0.3.0/24, Allow.
• Click Add new rule. Set Rule number to 110, Type to MYSQL/Aurora (3306), Source to 10.0.4.0/24, Allow.
• Click Add new rule. Set Rule number to 120, Type to Custom TCP, Port range to 6379, Source to 10.0.3.0/24, Allow.
• Click Add new rule. Set Rule number to 130, Type to Custom TCP, Port range to 6379, Source to 10.0.4.0/24, Allow.
• Click Add new rule. Set Rule number to 200, Type to Custom TCP, Port range to 1024-65535, Source to 10.0.3.0/24, Allow. This allows return traffic (ephemeral ports).
• Click Add new rule. Set Rule number to 210, Type to Custom TCP, Port range to 1024-65535, Source to 10.0.4.0/24, Allow.
• Click Save changes.

• Now configure outbound rules. Click the Outbound rules tab, then click Edit outbound rules.
• Click Add new rule. Set Rule number to 100, Type to Custom TCP, Port range to 1024-65535, Destination to 10.0.3.0/24, Allow.
• Click Add new rule. Set Rule number to 110, Type to Custom TCP, Port range to 1024-65535, Destination to 10.0.4.0/24, Allow.
• Click Save changes.

• Associate the NACL with the data subnets. Click the Subnet associations tab.
• Click Edit subnet associations.
• Select CloudShop-private-data-1a and CloudShop-private-data-1b.
• Click Save changes.

NACLs vs Security Groups

Security groups are stateful. If you allow inbound traffic, the response is automatically allowed out. NACLs are stateless. You must explicitly allow both inbound AND outbound (including ephemeral return ports 1024-65535).

The SAA-C03 exam frequently tests this distinction. Security groups operate at the instance/ENI level. NACLs operate at the subnet level and are evaluated before security groups.

Now create the VPC Gateway Endpoint for S3. This allows private subnets to access S3 without routing through the NAT Gateway, saving data processing costs.

• In the left navigation pane, click Endpoints.
• Click Create endpoint.
• For Name tag, enter CloudShop-S3-Endpoint.
• For Service category, select AWS services.
• In the search field, type s3 and select the com.amazonaws.us-east-1.s3 entry with Type Gateway.
• For VPC, select CloudShop-VPC.
• Under Route tables, select CloudShop-Private-RT.
• Leave the policy as Full access.
• Click Create endpoint.

Why a VPC Gateway Endpoint for S3?

Without the endpoint, S3 traffic from private subnets goes through the NAT Gateway, incurring $0.045/GB in data processing charges. The VPC Gateway Endpoint for S3 has no hourly or data processing charges. For a data-heavy application like CloudShop, this is a significant cost optimization.

Now update your Architecture Decision Record with the VPC design decisions. Open your terminal.

• Open ADR.md in your code editor and replace the contents with the following:

# CloudShop Architecture Decision Record

## ADR-001: VPC Design
- **Decision**: Regional VPC with 6 subnets across 2 AZs (public, private-app, private-data tiers)
- **Context**: Need network isolation between tiers and multi-AZ resilience
- **GCP Equivalent**: GCP VPC is global; AWS VPC is regional with AZ-specific subnets
- **SAA-C03 Exam Relevance**: Multi-AZ subnet design, route table configuration, NACLs vs Security Groups
- **Alternatives Considered**: Single AZ (rejected: no HA), fewer subnets (rejected: no tier isolation)
- **Outcome**: 6-subnet design with NAT Gateway for private outbound; VPC Gateway Endpoint for S3 cost optimization

## ADR-002: NAT Gateway
- **Decision**: Single NAT Gateway in one AZ for cost savings in non-production
- **Context**: NAT Gateway costs $0.045/hr + $0.045/GB data processed
- **GCP Equivalent**: Cloud NAT
- **SAA-C03 Exam Relevance**: NAT Gateway vs NAT Instance tradeoffs
- **Outcome**: Single NAT in public-1a; production would use one per AZ for full HA

What does this ADR capture?

ADR-001 records the VPC design pattern and why you chose it over alternatives. It also maps the AWS pattern to its GCP equivalent (global VPC vs regional VPC), which is useful for your GCP PCA mental model.

ADR-002 documents a deliberate cost tradeoff. In production you'd want a NAT Gateway per AZ, but for a lab environment the single-AZ approach saves roughly $1.08/day.

• Save ADR.md.

✔️ Awesome, I've got everything!

Great. Double check you've saved your file.

ⓧ I'd like to double check the full code

# CloudShop Architecture Decision Record

## ADR-001: VPC Design
- **Decision**: Regional VPC with 6 subnets across 2 AZs (public, private-app, private-data tiers)
- **Context**: Need network isolation between tiers and multi-AZ resilience
- **GCP Equivalent**: GCP VPC is global; AWS VPC is regional with AZ-specific subnets
- **SAA-C03 Exam Relevance**: Multi-AZ subnet design, route table configuration, NACLs vs Security Groups
- **Alternatives Considered**: Single AZ (rejected: no HA), fewer subnets (rejected: no tier isolation)
- **Outcome**: 6-subnet design with NAT Gateway for private outbound; VPC Gateway Endpoint for S3 cost optimization

## ADR-002: NAT Gateway
- **Decision**: Single NAT Gateway in one AZ for cost savings in non-production
- **Context**: NAT Gateway costs $0.045/hr + $0.045/GB data processed
- **GCP Equivalent**: Cloud NAT
- **SAA-C03 Exam Relevance**: NAT Gateway vs NAT Instance tradeoffs
- **Outcome**: Single NAT in public-1a; production would use one per AZ for full HA

Your network foundation is complete. You have a multi-AZ VPC with 3-tier subnet isolation, internet connectivity via IGW and NAT, defense-in-depth security, and a cost-optimized S3 endpoint. Next up, you'll build the IAM roles, KMS keys, and Secrets Manager configuration that secure everything running inside this network.

Configure IAM, KMS, and Secrets Manager

With your network foundation in place, the next layer to build is your security foundation. Every service in CloudShop needs credentials to talk to other services, and every piece of data needs encryption at rest.

In this step, you will create dedicated IAM roles for your compute and serverless tiers, a KMS customer-managed key for encryption across the entire stack, and a Secrets Manager secret that safely stores your database credentials with automatic rotation.

In this step, get ready to:

• Create IAM roles and an instance profile for EC2 and Lambda.
• Create a KMS customer-managed key (CMK) for encryption across all services.
• Store database credentials in Secrets Manager with automatic rotation.

Create IAM roles and instance profiles

AWS uses IAM roles to grant temporary credentials to services. Your EC2 instances need access to S3, CloudWatch Logs, and Systems Manager. Your Lambda functions need permission to write logs.

• Switch back to your terminal.
• Navigate to your cloudshop project folder by running:

cd ~/cloudshop

• Create a trust policy file that allows EC2 to assume the role by running:

cat > ec2-trust-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "ec2.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

What is a trust policy?

A trust policy defines which service or principal can assume a role. This JSON tells AWS that only the EC2 service is allowed to use the CloudShop-EC2-Role you are about to create.

In GCP, this is similar to a service account. The difference is that AWS separates the "who can use this identity" (trust policy) from the "what can this identity do" (permission policies).

• Create the EC2 role by running:

aws iam create-role \
    --role-name CloudShop-EC2-Role \
    --assume-role-policy-document file://ec2-trust-policy.json \
    --description "EC2 role for CloudShop instances" \
    --profile cloudshop

• Attach the required policies to the EC2 role by running these three commands:

# Grant read access to S3 buckets
aws iam attach-role-policy \
    --role-name CloudShop-EC2-Role \
    --policy-arn arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess \
    --profile cloudshop

# Grant permission to write CloudWatch Logs
aws iam attach-role-policy \
    --role-name CloudShop-EC2-Role \
    --policy-arn arn:aws:iam::aws:policy/CloudWatchLogsFullAccess \
    --profile cloudshop

# Grant Systems Manager access for remote management
aws iam attach-role-policy \
    --role-name CloudShop-EC2-Role \
    --policy-arn arn:aws:iam::aws:policy/AmazonSSMManagedInstanceCore \
    --profile cloudshop

What do these policies allow?

AmazonS3ReadOnlyAccess lets your EC2 instances read static assets from S3. CloudWatchLogsFullAccess lets them push application logs. AmazonSSMManagedInstanceCore enables Session Manager access so you can connect without SSH.

• Create an instance profile and attach the role by running:

# Create the instance profile container
aws iam create-instance-profile \
    --instance-profile-name CloudShop-EC2-Profile \
    --profile cloudshop

# Add the role to the instance profile
aws iam add-role-to-instance-profile \
    --instance-profile-name CloudShop-EC2-Profile \
    --role-name CloudShop-EC2-Role \
    --profile cloudshop

Why do you need an instance profile?

EC2 instances cannot assume IAM roles directly. An instance profile acts as a wrapper that connects a role to an instance. When you launch an EC2 instance, you attach the instance profile, and the instance automatically receives the role's temporary credentials.

• Create a trust policy for Lambda and then the Lambda execution role by running:

# Create the Lambda trust policy
cat > lambda-trust-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

# Create the Lambda execution role
aws iam create-role \
    --role-name CloudShop-Lambda-Role \
    --assume-role-policy-document file://lambda-trust-policy.json \
    --description "Lambda execution role for CloudShop functions" \
    --profile cloudshop

# Attach basic execution permissions (CloudWatch Logs)
aws iam attach-role-policy \
    --role-name CloudShop-Lambda-Role \
    --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole \
    --profile cloudshop

• Confirm the Auto Scaling service-linked role exists by running:

aws iam get-role \
    --role-name AWSServiceRoleForAutoScaling \
    --profile cloudshop

What is a service-linked role?

AWS automatically creates service-linked roles when you first use certain services. Auto Scaling needs one to manage EC2 instances on your behalf. If you see the role details in the output, it already exists and you are good to go.

• Verify your roles were created successfully by running:

aws iam list-roles \
    --query "Roles[?starts_with(RoleName, 'CloudShop')].[RoleName]" \
    --output table \
    --profile cloudshop

You should see CloudShop-EC2-Role and CloudShop-Lambda-Role listed in the output.

Don't see your roles?

Make sure you are using the --profile cloudshop flag on every command. If you see an access denied error, confirm your IAM user has AdministratorAccess attached.

Help me troubleshoot my IAM role creation in this CloudShop project.

Create a KMS customer-managed key

Every data store in CloudShop (RDS, EBS, S3, Secrets Manager) needs encryption at rest. A single customer-managed key gives you centralized control over who can encrypt and decrypt data across the entire stack.

• Create the KMS CMK by running:

aws kms create-key \
    --description "CloudShop CMK for encryption across all services" \
    --profile cloudshop

What does the output contain?

The response includes a KeyId field. Copy this value. You will use it in the next commands and throughout the project whenever you configure encryption.

• Create a human-friendly alias for your key by running:

aws kms create-alias \
    --alias-name alias/CloudShop-CMK \
    --target-key-id [[KMS_KEY_ID="your KeyId from the previous output"]] \
    --profile cloudshop

What is envelope encryption?

AWS uses envelope encryption. Your data is encrypted with a data key. That data key is then encrypted with your CMK. This means AWS services never send your raw data to KMS. Only the small data key travels to KMS for decryption.

The GCP equivalent is Cloud KMS with CMEK (Customer-Managed Encryption Keys). The concept is identical.

• Verify the alias was created by running:

aws kms list-aliases \
    --query "Aliases[?AliasName=='alias/CloudShop-CMK']" \
    --output table \
    --profile cloudshop

You should see your alias/CloudShop-CMK listed with the target key ID matching what you created.

Alias not showing up?

Double check that you replaced the key ID placeholder with the actual KeyId value from the create-key output. The key ID looks like a UUID (e.g. 12345678-1234-1234-1234-123456789012).

Help me troubleshoot my KMS alias creation for the CloudShop-CMK key.

Create a Secrets Manager secret for database credentials

Your RDS database (created in a later step) needs credentials. Hardcoding passwords in config files or environment variables is a security anti-pattern. Secrets Manager stores credentials encrypted, provides API-based retrieval, and can automatically rotate passwords on a schedule.

• Open the AWS Management Console in your browser.
• Search for Secrets Manager in the top search bar and select it.
• Click Store a new secret.
• Select Other type of secret as the secret type.
• Under Key/value pairs, add two rows:
• First row: Key = username, Value = cloudshop_admin.
• Second row: Key = password, Value: click Generate a password to let Secrets Manager create a strong random password for you.
• Under Encryption key, select your alias/CloudShop-CMK from the dropdown.
• Click Next.

Why use the console here?

The Secrets Manager console has a built-in password generator. Using the CLI for this would mean writing a password to your shell history, which defeats the purpose of a secrets manager. The console keeps the generated password entirely within AWS.

• Set the Secret name to CloudShop-DB-Credentials.
• Add a description: RDS MySQL credentials for CloudShop application.
• Click Next.

• On the Rotation configuration page, toggle Automatic rotation to on.
• Set the Rotation schedule to Days with a value of 30.
• For the rotation function, select Create a new Lambda function if prompted, or skip rotation function setup for now. The rotation Lambda will become functional once your RDS database exists in Step 5.
• Click Next.
• Review your settings and click Store.

Why Secrets Manager over Parameter Store?

AWS offers two credential storage options. Secrets Manager supports automatic rotation and is purpose-built for credentials that change on a schedule. Parameter Store is cheaper but does not natively support rotation.

The GCP equivalent is Secret Manager (same name, slightly different feature set). Both provide versioned, encrypted secret storage with IAM-based access control.

• Verify your secret exists by returning to your terminal and running:

aws secretsmanager describe-secret \
    --secret-id CloudShop-DB-Credentials \
    --profile cloudshop

You should see the secret details including the name, ARN, KMS key ID (matching your CMK), and rotation configuration showing a 30-day schedule.

Secret not found?

Confirm you are in the us-east-1 region in both the console and CLI. Secrets are regional resources. Also verify the secret name is exactly CloudShop-DB-Credentials (case-sensitive).

Help me find my CloudShop-DB-Credentials secret in Secrets Manager.

• Update your Architecture Decision Record with the IAM, KMS, and Secrets Manager decisions. Open ADR.md in your editor and replace the entire contents with:

# CloudShop Architecture Decision Record

## ADR-001: VPC Design
- **Decision**: Regional VPC with 6 subnets across 2 AZs (public, private-app, private-data tiers)
- **GCP Equivalent**: GCP VPC is global; AWS VPC is regional with AZ-specific subnets
- **SAA-C03 Exam Relevance**: Multi-AZ subnet design, route table configuration, NACLs vs Security Groups
- **Outcome**: 6-subnet design with NAT Gateway for private outbound; VPC Gateway Endpoint for S3 cost optimization

## ADR-002: NAT Gateway
- **Decision**: Single NAT Gateway in one AZ for cost savings in non-production
- **Outcome**: Single NAT in public-1a; production would use one per AZ for full HA

## ADR-003: IAM Design
- **Decision**: Least-privilege roles per service tier; instance profiles for EC2
- **GCP Equivalent**: GCP Service Accounts; AWS IAM uses policy evaluation logic (explicit deny > explicit allow > implicit deny)
- **SAA-C03 Exam Relevance**: IAM policy evaluation, instance profiles, service-linked roles
- **Outcome**: Separate roles for EC2, Lambda; no long-term credentials on instances

## ADR-004: KMS CMK
- **Decision**: Single CMK for all encryption (RDS, EBS, S3, Secrets Manager)
- **Context**: Envelope encryption: data encrypted with data key, data key encrypted with CMK
- **GCP Equivalent**: Cloud KMS CMEK
- **Outcome**: CloudShop-CMK used across all services

## ADR-005: Secrets Manager vs Parameter Store
- **Decision**: Secrets Manager for DB credentials; Parameter Store for config values
- **Context**: Secrets Manager supports automatic rotation; Parameter Store is cheaper for non-secret config
- **GCP Equivalent**: GCP Secret Manager
- **Outcome**: DB credentials in Secrets Manager with 30-day rotation

What is IAM policy evaluation logic?

When AWS evaluates whether a request is allowed, it follows this order: explicit deny wins over everything, then explicit allow, then implicit deny (the default). This means a single deny statement in any attached policy blocks access even if another policy allows it.

GCP uses a similar additive model with allow policies, but AWS adds the explicit deny layer. This is a frequently tested concept on the SAA-C03 exam.

• Save ADR.md.

✔️ Awesome, I've got everything!

Great. Double check you have saved your ADR.md file with all five ADR entries.

ⓧ I'd like to double check the full code

Here is the complete ADR.md as it should look at the end of this step:

# CloudShop Architecture Decision Record

## ADR-001: VPC Design
- **Decision**: Regional VPC with 6 subnets across 2 AZs (public, private-app, private-data tiers)
- **GCP Equivalent**: GCP VPC is global; AWS VPC is regional with AZ-specific subnets
- **SAA-C03 Exam Relevance**: Multi-AZ subnet design, route table configuration, NACLs vs Security Groups
- **Outcome**: 6-subnet design with NAT Gateway for private outbound; VPC Gateway Endpoint for S3 cost optimization

## ADR-002: NAT Gateway
- **Decision**: Single NAT Gateway in one AZ for cost savings in non-production
- **Outcome**: Single NAT in public-1a; production would use one per AZ for full HA

## ADR-003: IAM Design
- **Decision**: Least-privilege roles per service tier; instance profiles for EC2
- **GCP Equivalent**: GCP Service Accounts; AWS IAM uses policy evaluation logic (explicit deny > explicit allow > implicit deny)
- **SAA-C03 Exam Relevance**: IAM policy evaluation, instance profiles, service-linked roles
- **Outcome**: Separate roles for EC2, Lambda; no long-term credentials on instances

## ADR-004: KMS CMK
- **Decision**: Single CMK for all encryption (RDS, EBS, S3, Secrets Manager)
- **Context**: Envelope encryption: data encrypted with data key, data key encrypted with CMK
- **GCP Equivalent**: Cloud KMS CMEK
- **Outcome**: CloudShop-CMK used across all services

## ADR-005: Secrets Manager vs Parameter Store
- **Decision**: Secrets Manager for DB credentials; Parameter Store for config values
- **Context**: Secrets Manager supports automatic rotation; Parameter Store is cheaper for non-secret config
- **GCP Equivalent**: GCP Secret Manager
- **Outcome**: DB credentials in Secrets Manager with 30-day rotation

Your security foundation is in place. You have least-privilege roles for every service tier, a centralized encryption key, and credentials stored safely with automatic rotation configured. Next up, you will build the storage layer with S3 buckets and EFS.

Build the Storage Layer

Configure the Storage Layer

With your security foundation in place, your CloudShop infrastructure needs somewhere to store things. Static images, application logs, analytics data, and shared files all need different storage strategies.

In this step, you'll create three S3 buckets with different purposes, configure lifecycle policies to optimize costs automatically, and set up a shared file system for your compute tier.

In this step, get ready to:

• Create S3 buckets for static assets, logs, and analytics with KMS encryption.
• Configure lifecycle policies and Transfer Acceleration on the assets bucket.
• Set up an EFS file system with mount targets for shared storage across EC2 instances.

Create S3 buckets with encryption and access controls

CloudShop needs three separate buckets, each with a different purpose. The assets bucket stores product images and static files served to customers. The logs bucket captures API audit trails from CloudTrail. The analytics bucket holds event data for reporting.

All three buckets use your CloudShop-CMK for server-side encryption, ensuring data at rest is always protected.

• Create the assets bucket by running this command in your terminal:

aws s3api create-bucket \
  --bucket cloudshop-assets-[[YOURNAME="your unique identifier e.g. jsmith"]] \
  --profile cloudshop

Why no --create-bucket-configuration?

When creating a bucket in us-east-1, you omit the --create-bucket-configuration parameter. This region is the S3 default, and specifying it explicitly causes an error.

• Enable versioning on the assets bucket:

aws s3api put-bucket-versioning \
  --bucket cloudshop-assets-[[YOURNAME="your unique identifier e.g. jsmith"]] \
  --versioning-configuration Status=Enabled \
  --profile cloudshop

• Configure SSE-KMS encryption using your CloudShop-CMK:

aws s3api put-bucket-encryption \
  --bucket cloudshop-assets-[[YOURNAME="your unique identifier e.g. jsmith"]] \
  --server-side-encryption-configuration '{
    "Rules": [
      {
        "ApplyServerSideEncryptionByDefault": {
          "SSEAlgorithm": "aws:kms",
          "KMSMasterKeyID": "[[KMS_KEY_ARN="your CloudShop-CMK ARN from Step 3"]]"
        }
      }
    ]
  }' \
  --profile cloudshop

What does this encryption configuration do?

This sets your customer-managed key as the default encryption for all objects uploaded to this bucket. Even if someone forgets to specify encryption during upload, S3 applies SSE-KMS automatically.

This is the envelope encryption pattern. S3 generates a unique data key for each object, encrypts the object with that data key, then encrypts the data key itself with your CMK.

Now you need a bucket policy that explicitly denies any upload that does not specify KMS encryption. This acts as a guardrail on top of the default encryption.

• Create a file called bucket-policy.json in your ~/cloudshop directory with this content:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyUnencryptedUploads",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::cloudshop-assets-[[YOURNAME="your unique identifier e.g. jsmith"]]/*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption": "aws:kms"
        }
      }
    }
  ]
}

• Apply the bucket policy by running:

aws s3api put-bucket-policy \
  --bucket cloudshop-assets-[[YOURNAME="your unique identifier e.g. jsmith"]] \
  --policy file://~/cloudshop/bucket-policy.json \
  --profile cloudshop

Why both default encryption and a deny policy?

Default encryption handles the common case. The deny policy handles the edge case where a client explicitly overrides encryption with a non-KMS algorithm. Together, they form defense in depth.

This is a pattern the SAA-C03 exam tests in Domain 1 (Design Secure Architectures). The condition key s3:x-amz-server-side-encryption checks what encryption the client requested.

• Verify the assets bucket is properly configured by running:

aws s3api get-bucket-versioning \
  --bucket cloudshop-assets-[[YOURNAME="your unique identifier e.g. jsmith"]] \
  --profile cloudshop

You should see "Status": "Enabled" in the output, confirming versioning is active.

Seeing an access denied error?

Make sure you're using the correct --profile cloudshop flag. Also confirm the bucket name matches exactly what you created, including your unique suffix.

If the bucket policy application failed, check that the ARN in bucket-policy.json matches your actual bucket name.

Help me troubleshoot my S3 bucket configuration for the CloudShop project.

Now create the remaining two buckets for logs and analytics.

• Create the CloudTrail logs bucket:

aws s3api create-bucket \
  --bucket cloudshop-logs-[[YOURNAME="your unique identifier e.g. jsmith"]] \
  --profile cloudshop

• Create the analytics data bucket:

aws s3api create-bucket \
  --bucket cloudshop-analytics-[[YOURNAME="your unique identifier e.g. jsmith"]] \
  --profile cloudshop

• Apply SSE-KMS encryption to both new buckets:

# Encrypt the logs bucket
aws s3api put-bucket-encryption \
  --bucket cloudshop-logs-[[YOURNAME="your unique identifier e.g. jsmith"]] \
  --server-side-encryption-configuration '{
    "Rules": [
      {
        "ApplyServerSideEncryptionByDefault": {
          "SSEAlgorithm": "aws:kms",
          "KMSMasterKeyID": "[[KMS_KEY_ARN="your CloudShop-CMK ARN from Step 3"]]"
        }
      }
    ]
  }' \
  --profile cloudshop

# Encrypt the analytics bucket
aws s3api put-bucket-encryption \
  --bucket cloudshop-analytics-[[YOURNAME="your unique identifier e.g. jsmith"]] \
  --server-side-encryption-configuration '{
    "Rules": [
      {
        "ApplyServerSideEncryptionByDefault": {
          "SSEAlgorithm": "aws:kms",
          "KMSMasterKeyID": "[[KMS_KEY_ARN="your CloudShop-CMK ARN from Step 3"]]"
        }
      }
    ]
  }' \
  --profile cloudshop

What does each bucket store?

• cloudshop-assets: Product images, CSS, JavaScript, and other static files served through CloudFront.
• cloudshop-logs: CloudTrail API audit logs for compliance and forensics.
• cloudshop-analytics: Raw event data from Kinesis Firehose for Athena queries.

• Confirm all three buckets exist by running:

aws s3 ls --profile cloudshop | grep cloudshop

You should see all three bucket names listed with their creation dates.

Configure lifecycle policies and Transfer Acceleration

Static assets like product images are accessed frequently when new, but traffic drops off over time. S3 lifecycle policies automatically transition objects to cheaper storage classes as they age, saving money without manual intervention.

• Create a file called lifecycle-policy.json in your ~/cloudshop directory:

{
  "Rules": [
    {
      "ID": "CloudShopAssetsLifecycle",
      "Status": "Enabled",
      "Filter": {},
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "STANDARD_IA"
        },
        {
          "Days": 90,
          "StorageClass": "GLACIER"
        }
      ],
      "Expiration": {
        "Days": 365
      }
    }
  ]
}

• Apply the lifecycle configuration to the assets bucket:

aws s3api put-bucket-lifecycle-configuration \
  --bucket cloudshop-assets-[[YOURNAME="your unique identifier e.g. jsmith"]] \
  --lifecycle-configuration file://~/cloudshop/lifecycle-policy.json \
  --profile cloudshop

What does this lifecycle do?

• Days 1-30: Objects stay in S3 Standard (frequent access, highest cost).
• After 30 days: Objects move to Standard-IA (infrequent access, lower storage cost, retrieval fee applies).
• After 90 days: Objects move to Glacier (archival storage, minutes-to-hours retrieval time).
• After 365 days: Objects are permanently deleted.

This maps directly to SAA-C03 Domain 4 (Cost-Optimized Architectures). The exam tests your ability to select the right storage class for each access pattern.

• Enable S3 Transfer Acceleration on the assets bucket:

aws s3api put-bucket-accelerate-configuration \
  --bucket cloudshop-assets-[[YOURNAME="your unique identifier e.g. jsmith"]] \
  --accelerate-configuration Status=Enabled \
  --profile cloudshop

What is S3 Transfer Acceleration?

Transfer Acceleration uses CloudFront edge locations to speed up uploads to S3. When customers upload product images from distant regions, data routes through the nearest edge location and travels over AWS's optimized backbone network to the bucket.

This is useful for the CloudShop scenario where vendors might upload product images from anywhere in the world.

• Verify the lifecycle configuration is active:

aws s3api get-bucket-lifecycle-configuration \
  --bucket cloudshop-assets-[[YOURNAME="your unique identifier e.g. jsmith"]] \
  --profile cloudshop

You should see your lifecycle rule with the three transitions listed.

• Verify Transfer Acceleration is enabled:

aws s3api get-bucket-accelerate-configuration \
  --bucket cloudshop-assets-[[YOURNAME="your unique identifier e.g. jsmith"]] \
  --profile cloudshop

You should see "Status": "Enabled" in the output.

Transfer Acceleration not enabling?

Transfer Acceleration requires that the bucket name does not contain periods. If your bucket name includes dots, rename it or use a different unique suffix.

Also make sure your bucket name conforms to DNS naming conventions (lowercase letters, numbers, and hyphens only).

Help me fix Transfer Acceleration on my S3 bucket.

Now update your Architecture Decision Record with the storage class decisions.

• Open ~/cloudshop/ADR.md and add the following entry after ADR-005:

## ADR-006: S3 Storage Classes and Lifecycle
- **Decision**: Lifecycle policy transitioning assets through Standard → Standard-IA → Glacier → Expiry
- **Context**: Static assets have diminishing access frequency over time
- **Rationale**: Cost optimization; pay less for infrequently accessed data
- **GCP Equivalent**: GCS Object Lifecycle Management
- **SAA-C03 Exam Relevance**: Domain 4 - Cost-Optimized Architectures; storage class selection
- **Alternatives Considered**: S3 Intelligent-Tiering (valid for unknown access patterns)

Why not S3 Intelligent-Tiering?

S3 Intelligent-Tiering automatically moves objects between access tiers based on actual usage patterns. It's ideal when you don't know how often objects will be accessed.

For CloudShop, you know the access pattern (new images are hot, old images are cold), so explicit lifecycle rules give you predictable costs without the per-object monitoring fee that Intelligent-Tiering charges.

• Save ADR.md.

Create EFS for shared storage across EC2 instances

Your Auto Scaling Group will launch multiple EC2 instances, and they all need access to the same shared files (uploaded content, configuration, session data). EFS provides a managed NFS file system that multiple instances can mount simultaneously.

This is the key difference from EBS volumes, which attach to a single instance. When you scale from one instance to four, EBS cannot share data between them.

First, you need a security group that allows NFS traffic from your application instances to the EFS mount targets.

• Create an EFS security group by running:

# Create a security group for EFS mount targets
aws ec2 create-security-group \
  --group-name EFS-SG \
  --description "Allow NFS from app tier" \
  --vpc-id [[VPC_ID="your CloudShop-VPC ID from Step 2"]] \
  --profile cloudshop

Note the GroupId in the output. You'll need it for the next commands.

• Add an inbound rule allowing NFS (port 2049) from your App-SG:

aws ec2 authorize-security-group-ingress \
  --group-id [[EFS_SG_ID="your EFS-SG group ID from above"]] \
  --protocol tcp \
  --port 2049 \
  --source-group [[APP_SG_ID="your App-SG group ID from Step 2"]] \
  --profile cloudshop

Why port 2049?

Port 2049 is the standard NFS (Network File System) port. EFS uses NFS v4.1 protocol, so your EC2 instances will connect to the mount targets on this port.

By restricting the source to App-SG, only your application instances can access the file system. Instances in other tiers cannot reach it.

• Create the encrypted EFS file system:

aws efs create-file-system \
  --encrypted \
  --kms-key-id [[KMS_KEY_ARN="your CloudShop-CMK ARN from Step 3"]] \
  --tags Key=Name,Value=CloudShop-EFS \
  --profile cloudshop

What does --encrypted do here?

The --encrypted flag enables encryption at rest for all data stored in the file system. Combined with --kms-key-id, EFS uses your CloudShop-CMK instead of the default AWS-managed key.

Once encryption is enabled on an EFS file system, it cannot be disabled. This is a one-time setting made at creation.

Note the FileSystemId from the output (it starts with fs-). You need it to create mount targets.

• Create a mount target in the first app subnet:

aws efs create-mount-target \
  --file-system-id [[EFS_ID="your FileSystemId starting with fs-"]] \
  --subnet-id [[SUBNET_APP_1A="your private-app-1a subnet ID"]] \
  --security-groups [[EFS_SG_ID="your EFS-SG group ID"]] \
  --profile cloudshop

• Create a mount target in the second app subnet:

aws efs create-mount-target \
  --file-system-id [[EFS_ID="your FileSystemId starting with fs-"]] \
  --subnet-id [[SUBNET_APP_1B="your private-app-1b subnet ID"]] \
  --security-groups [[EFS_SG_ID="your EFS-SG group ID"]] \
  --profile cloudshop

Why one mount target per Availability Zone?

Each mount target provides an IP address in its subnet that EC2 instances use to connect to EFS. By placing one in each app-tier AZ, instances in both us-east-1a and us-east-1b can access the file system with low latency.

If an AZ goes down, instances in the surviving AZ still have their own mount target. This is the multi-AZ resilience pattern tested in SAA-C03 Domain 2.

• Verify your EFS file system and mount targets by running:

aws efs describe-mount-targets \
  --file-system-id [[EFS_ID="your FileSystemId starting with fs-"]] \
  --profile cloudshop

You should see two mount targets listed, one in each app-tier subnet, both with a LifeCycleState of available (this may take a minute to transition from creating).

Mount targets stuck in creating state?

Mount targets typically take 1-2 minutes to become available. Run the describe command again after a short wait.

If a mount target fails, check that the subnet ID belongs to your CloudShop-VPC and that the security group is in the same VPC.

Help me debug my EFS mount target creation.

Finally, document the EFS decision in your ADR.

• Open ~/cloudshop/ADR.md and add this entry after ADR-006:

## ADR-007: EFS for Shared Storage
- **Decision**: EFS over EBS for shared storage across EC2 instances
- **Context**: Multiple EC2 instances in ASG need access to shared files
- **Rationale**: EFS is a managed NFS that can be mounted by multiple instances simultaneously; EBS is single-instance
- **GCP Equivalent**: GCP Filestore
- **SAA-C03 Exam Relevance**: Domain 2 - storage selection for multi-instance workloads
- **Alternatives Considered**: EBS (single-attach only), S3 (not a filesystem)

EFS vs EBS vs S3 for the exam

• EFS: Shared file system, multiple instances mount simultaneously, scales automatically.
• EBS: Block storage attached to a single EC2 instance (except io2 Multi-Attach for specific use cases).
• S3: Object storage accessed via API, not a mountable filesystem for applications expecting POSIX file semantics.

The SAA-C03 exam frequently presents scenarios where you must choose between these three. The key differentiator is whether multiple instances need simultaneous read/write access.

• Save ADR.md.

Your storage layer is complete. You have encrypted S3 buckets with lifecycle automation, a shared file system ready for your compute tier, and cost optimization baked in from day one. Next up, you'll build the database layer with RDS and DynamoDB.

Deploy the Database Layer

Deploy the Database Layer with RDS, DynamoDB, and ElastiCache

Your storage layer is in place with S3 buckets and EFS ready to serve your application. Now you need databases to hold the actual application data.

A production e-commerce platform needs different database types for different workloads. Relational data like products and orders goes into RDS MySQL. Session data that needs millisecond-level access goes into DynamoDB. Frequently queried results get cached in ElastiCache Redis to reduce database load.

In this step, get ready to:

• Create an RDS MySQL instance with encryption and automated backups.
• Deploy a DynamoDB table for session management with TTL and streams.
• Launch an ElastiCache Redis cluster for application caching.

Create the RDS subnet group and MySQL instance

RDS requires a subnet group that tells it which subnets to use for the database instance. You will place your database in the private-data subnets so it has no direct internet access.

• In the AWS Console, search for RDS and open the Amazon RDS console.
• In the left navigation pane, choose Subnet groups.
• Click Create DB subnet group.
• For Name, enter cloudshop-db-subnet-group.
• For Description, enter CloudShop private data subnets for RDS.
• For VPC, select your CloudShop-VPC.
• Under Add subnets, select Availability Zones us-east-1a and us-east-1b.
• Select the two private-data subnets (10.0.5.0/24 and 10.0.6.0/24).
• Click Create.

You should see cloudshop-db-subnet-group appear in the subnet groups list.

Now create the RDS MySQL instance itself.

• In the left navigation pane, choose Databases.
• Click Create database.
• For creation method, choose Standard create.
• Under Engine options, choose MySQL.
• For Version, select a MySQL 8.0 version (the latest available 8.0.x is fine).
• Under Templates, choose Free tier or Sandbox (depending on your account type).
• Under Settings, for DB instance identifier, enter cloudshop-mysql.
• For Credential Settings, choose Managed in AWS Secrets Manager.
• Under Select the encryption key, choose your CloudShop-CMK key.
• Under Instance configuration, confirm the instance class is db.t3.micro.
• Under Connectivity, for Virtual private cloud (VPC), select CloudShop-VPC.
• For DB subnet group, select cloudshop-db-subnet-group.
• For VPC security group, choose Choose existing and select your Data-SG security group.
• Set Public access to No.
• Expand Additional configuration.
• For Initial database name, enter cloudshop.
• Confirm Encryption is enabled and your CloudShop-CMK is selected.
• Set Backup retention period to 7 days.
• Click Create database.

The database status shows Creating. It takes 5-10 minutes to become Available.

What is the difference between Multi-AZ and Read Replicas?

Multi-AZ creates a synchronous standby copy of your database in another Availability Zone. If the primary fails, RDS automatically failovers to the standby. This is for high availability.

A Read Replica is an asynchronous copy used for read scaling. Your application sends read queries to the replica, reducing load on the primary. Read Replicas do not provide automatic failover.

For this lab, you use Single-AZ to stay in free tier. The architecture supports promoting to Multi-AZ at any time without downtime.

Database stuck in Creating?

RDS provisioning can take up to 20 minutes depending on load. Confirm you selected db.t3.micro and not a larger instance class, which takes longer.

If creation fails, check that your Data-SG security group belongs to CloudShop-VPC and that the subnet group includes subnets in two different AZs.

help me troubleshoot my RDS instance creation failure

Create the DynamoDB sessions table

Session data needs low-latency key-value access with automatic expiration. DynamoDB is the standard AWS solution for this pattern because it scales horizontally and supports TTL for automatic item deletion.

• In the AWS Console, search for DynamoDB and open the DynamoDB console.
• In the left navigation pane, choose Tables.
• Click Create table.
• For Table name, enter CloudShop-Sessions.
• For Partition key, enter sessionId and keep the type as String.
• Leave Sort key blank (session lookups only need the partition key).
• Under Table settings, choose Customize settings.
• Under Read/write capacity settings, change the capacity mode to On-demand.
• Under Encryption at rest, choose Customer managed key and select your CloudShop-CMK.
• Click Create table.

Wait for the table status to become Active. This usually takes under a minute.

Now enable TTL and DynamoDB Streams on the table.

• Click the CloudShop-Sessions table name to open its details.
• Choose the Additional settings tab.
• In the Time to Live (TTL) section, click Turn on.
• For the TTL attribute name, enter expiresAt.
• Click Turn on TTL.
• Go to the Exports and streams tab.
• Under DynamoDB stream details, click Turn on.
• Select New and old images as the view type.
• Click Turn on stream.

Why DynamoDB instead of RDS for sessions?

Sessions need single-digit millisecond reads by a single key. DynamoDB delivers this at any scale without connection pooling or read replicas.

TTL automatically expires stale sessions without a cleanup job. On-demand capacity means you never over-provision for traffic spikes. This is the SAA-C03 exam pattern for session stores, shopping carts, and leaderboards.

Can't find the TTL option?

Make sure you are on the Additional settings tab of the table, not the Overview tab. TTL is only accessible after the table reaches Active status.

help me find and enable TTL on my DynamoDB table

Create the ElastiCache Redis cluster

Caching database query results reduces read load on RDS and speeds up response times. ElastiCache Redis provides an in-memory data store that sits between your application and the database.

• In the AWS Console, search for ElastiCache and open the ElastiCache console.
• In the left navigation pane, choose Subnet Groups.
• Click Create Subnet Group.
• For Name, enter cloudshop-cache-subnet-group.
• For Description, enter CloudShop private data subnets for ElastiCache.
• For VPC ID, select CloudShop-VPC.
• Select both private-data subnets (10.0.5.0/24 in us-east-1a and 10.0.6.0/24 in us-east-1b) and click Add.
• Click Create.

Now create the Redis cluster.

• In the navigation pane, choose Redis OSS caches.
• Click Create Redis OSS cache.
• Choose Configure and create a new cluster.
• For Cluster mode, choose Disabled (single-node is sufficient for this lab).
• For Name, enter cloudshop-redis.
• For Node type, select cache.t3.micro.
• For Number of replicas, set to 0 (single node for cost).
• Under Connectivity, for Subnet groups, select cloudshop-cache-subnet-group.
• Under Advanced Redis OSS settings, find the Security section.
• Enable Encryption at rest.
• Enable Encryption in transit.
• For Security groups, select your Data-SG security group.
• Click Next, review your settings, then click Create.

The cluster status shows Creating and takes 5-10 minutes to become Available.

What is lazy loading in caching?

Lazy loading means your application checks the cache first. On a cache miss, it reads from the database and writes the result into the cache for future requests.

The alternative is write-through, where every database write also updates the cache. Lazy loading is simpler and avoids caching data that is never read. For SAA-C03, know both patterns and when each fits.

ElastiCache free tier

If your account was created before July 15, 2025, you get 750 hours per month of cache.t3.micro free for 12 months. Accounts created after that date receive $100-200 in credits. The cache.t3.micro node provides 0.5 GiB of memory.

Encryption options greyed out?

Encryption at rest must be enabled during cluster creation. You cannot add it to an existing cluster. If the option appears disabled, confirm you are creating a new cluster (not modifying an existing one).

help me enable encryption on my ElastiCache Redis cluster

While the Redis cluster provisions, update your Architecture Decision Record.

• Open ~/cloudshop/ADR.md in your text editor and add the following entries below the existing content:

## ADR-008: RDS Multi-AZ vs Read Replica
- **Decision**: Single-AZ RDS for cost in lab; architecture supports Multi-AZ promotion
- **Context**: Multi-AZ = synchronous standby for HA; Read Replica = async copy for read scaling
- **GCP Equivalent**: Cloud SQL with HA and read replicas
- **SAA-C03 Exam Relevance**: RDS Multi-AZ for HA, Read Replicas for read scaling, not HA
- **Outcome**: db.t3.micro Single-AZ with CMK encryption and Secrets Manager credentials

## ADR-009: DynamoDB vs RDS for Sessions
- **Decision**: DynamoDB for session storage; RDS for transactional/relational data
- **Context**: Sessions need low-latency key-value access, TTL, and horizontal scale
- **GCP Equivalent**: Firestore or Bigtable for NoSQL; Cloud SQL for relational
- **SAA-C03 Exam Relevance**: DynamoDB use cases: session store, shopping cart, leaderboards
- **Outcome**: CloudShop-Sessions table with on-demand capacity and TTL

## ADR-010: Caching Strategy
- **Decision**: ElastiCache Redis with lazy loading for database query results
- **Context**: Lazy loading: read from cache, on miss read DB and populate cache
- **GCP Equivalent**: Memorystore for Redis
- **SAA-C03 Exam Relevance**: ElastiCache use cases, lazy loading vs write-through
- **Outcome**: Single-node Redis cluster in private-data subnets

What do these ADR entries capture?

ADR-008 documents why Single-AZ is acceptable for a lab and how Multi-AZ differs from Read Replicas. ADR-009 explains why DynamoDB fits the session use case better than putting sessions in RDS. ADR-010 records the caching strategy choice with the GCP equivalent (Memorystore).

• Save ADR.md.

✔️ Awesome, I've got everything!

Great. Double check you have saved your ADR file with all three new entries.

ⓧ I'd like to double check the full code

Here is what your ADR.md should look like at this point (new entries at the bottom):

# CloudShop Architecture Decision Record

## ADR-001 through ADR-007: [Previous entries retained]

## ADR-008: RDS Multi-AZ vs Read Replica
- **Decision**: Single-AZ RDS for cost in lab; architecture supports Multi-AZ promotion
- **Context**: Multi-AZ = synchronous standby for HA; Read Replica = async copy for read scaling
- **GCP Equivalent**: Cloud SQL with HA and read replicas
- **SAA-C03 Exam Relevance**: RDS Multi-AZ for HA, Read Replicas for read scaling, not HA
- **Outcome**: db.t3.micro Single-AZ with CMK encryption and Secrets Manager credentials

## ADR-009: DynamoDB vs RDS for Sessions
- **Decision**: DynamoDB for session storage; RDS for transactional/relational data
- **Context**: Sessions need low-latency key-value access, TTL, and horizontal scale
- **GCP Equivalent**: Firestore or Bigtable for NoSQL; Cloud SQL for relational
- **SAA-C03 Exam Relevance**: DynamoDB use cases: session store, shopping cart, leaderboards
- **Outcome**: CloudShop-Sessions table with on-demand capacity and TTL

## ADR-010: Caching Strategy
- **Decision**: ElastiCache Redis with lazy loading for database query results
- **Context**: Lazy loading: read from cache, on miss read DB and populate cache
- **GCP Equivalent**: Memorystore for Redis
- **SAA-C03 Exam Relevance**: ElastiCache use cases, lazy loading vs write-through
- **Outcome**: Single-node Redis cluster in private-data subnets

Your database layer is now complete with three complementary data stores. Next, you will build the compute layer with EC2 instances, an Application Load Balancer, and Auto Scaling to serve your web application.

Launch the Compute Layer

Launch the Compute Layer with ALB and Auto Scaling

Your database, cache, and storage layers are all running and ready to serve requests. The missing piece is the compute layer that actually runs your application code and handles incoming traffic from users.

In this step, you'll create an EC2 Launch Template that defines how every instance in your fleet gets configured, then deploy an Application Load Balancer to distribute traffic, and finally wire it all together with an Auto Scaling Group that keeps your fleet elastic.

In this step, get ready to:

• Create an EC2 Launch Template with a user data script that bootstraps a Node.js web app.
• Set up an Application Load Balancer with path-based routing rules.
• Configure an Auto Scaling Group with target tracking and scheduled scaling.

Create the EC2 Launch Template

A Launch Template is a versioned blueprint that tells EC2 exactly how to configure each instance. It locks down the AMI, instance type, IAM role, security groups, and the user data script that bootstraps your application on first boot.

• In the AWS Console, navigate to the EC2 service.
• In the left navigation pane under Instances, click Launch Templates.
• Click Create launch template.
• For Launch template name, enter CloudShop-LaunchTemplate.
• For Template version description, enter Initial CloudShop app server.

Now configure the instance settings:

• Under Application and OS Images, select Amazon Linux 2023 AMI from the Quick Start tab.
• For Instance type, choose t3.micro.
• Skip the Key pair section (you'll use Session Manager instead of SSH).
• Under Network settings, click Select existing security group and choose App-SG.

Next, expand the Advanced details section to configure the IAM profile, monitoring, and user data:

• For IAM instance profile, select CloudShop-EC2InstanceProfile.
• For Detailed CloudWatch monitoring, choose Enable. This gives 1-minute metric granularity for your scaling policies.
• Scroll down to the User data field and paste the following script:

#!/bin/bash
# Update system packages
dnf update -y

# Install Node.js 18 LTS
dnf install -y nodejs

# Install CloudWatch agent
dnf install -y amazon-cloudwatch-agent

# Mount EFS file system for shared storage
dnf install -y amazon-efs-utils
mkdir -p /mnt/efs
mount -t efs [[EFS_ID="your EFS file system ID"]]:/ /mnt/efs
echo "[[EFS_ID="your EFS file system ID"]]:/ /mnt/efs efs defaults,_netdev 0 0" >> /etc/fstab

# Create application directory
mkdir -p /opt/cloudshop
cd /opt/cloudshop

# Create a simple Node.js web application
cat > package.json << 'EOF'
{
  "name": "cloudshop-app",
  "version": "1.0.0",
  "dependencies": {
    "express": "^4.18.0",
    "mysql2": "^3.6.0",
    "redis": "^4.6.0"
  }
}
EOF

npm install

# Create main application file
cat > app.js << 'EOF'
const express = require('express');
const mysql = require('mysql2/promise');
const { createClient } = require('redis');

const app = express();
const PORT = 80;

// Database connection pool
const dbPool = mysql.createPool({
  host: process.env.DB_HOST,
  user: process.env.DB_USER,
  password: process.env.DB_PASS,
  database: 'cloudshop'
});

// Redis cache connection
const redisClient = createClient({
  url: `redis://${process.env.REDIS_HOST}:6379`
});
redisClient.connect();

// Health check endpoint for ALB
app.get('/health', (req, res) => {
  res.status(200).json({ status: 'healthy' });
});

// API endpoint with caching
app.get('/api/products', async (req, res) => {
  const cached = await redisClient.get('products');
  if (cached) return res.json(JSON.parse(cached));
  const [rows] = await dbPool.query('SELECT * FROM products');
  await redisClient.set('products', JSON.stringify(rows), { EX: 300 });
  res.json(rows);
});

app.listen(PORT, () => console.log(`CloudShop running on port ${PORT}`));
EOF

# Retrieve database credentials from Secrets Manager
export DB_HOST=$(aws secretsmanager get-secret-value --secret-id CloudShop/DB/Credentials --query 'SecretString' --output text | python3 -c "import sys,json; print(json.loads(sys.stdin.read())['host'])")
export DB_USER=$(aws secretsmanager get-secret-value --secret-id CloudShop/DB/Credentials --query 'SecretString' --output text | python3 -c "import sys,json; print(json.loads(sys.stdin.read())['username'])")
export DB_PASS=$(aws secretsmanager get-secret-value --secret-id CloudShop/DB/Credentials --query 'SecretString' --output text | python3 -c "import sys,json; print(json.loads(sys.stdin.read())['password'])")
export REDIS_HOST=[[REDIS_ENDPOINT="your ElastiCache Redis endpoint"]]

# Start the application
node app.js &

# Start CloudWatch agent with default config
/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \
  -a fetch-config -m ec2 -s

What does this user data script do?

The script runs once when each EC2 instance first boots. It handles the entire application stack setup automatically:

• Installs Node.js and the CloudWatch agent using dnf (the Amazon Linux 2023 package manager).
• Mounts EFS so all instances share the same filesystem for static uploads.
• Creates a simple Express.js app with a /health endpoint (for ALB health checks) and a /api/products endpoint that queries RDS through the Redis cache.
• Retrieves database credentials securely from Secrets Manager at runtime, so no passwords are baked into the template.

• Click Create launch template.

You should see a success banner confirming CloudShop-LaunchTemplate was created.

Launch template creation failed?

• Make sure you selected App-SG under security groups, not the default VPC security group.
• Verify CloudShop-EC2InstanceProfile appears in the IAM instance profile dropdown. If it does not, return to the IAM console and confirm the instance profile is associated with CloudShop-EC2Role.

Help me troubleshoot my EC2 Launch Template creation in the CloudShop project.

Create the Application Load Balancer with path-based routing

The ALB sits in your public subnets and distributes incoming HTTP traffic across EC2 instances in the private app subnets. You'll configure path-based routing so different URL patterns reach different backends.

• In the EC2 console left navigation, click Load Balancers under Load Balancing.
• Click Create load balancer.
• Under Application Load Balancer, click Create.

Configure the basic settings:

• For Load balancer name, enter CloudShop-ALB.
• For Scheme, keep Internet-facing.
• Under Network mapping, select CloudShop-VPC.
• Select the us-east-1a and us-east-1b Availability Zones.
• For each AZ, choose the corresponding public subnet (public-1a and public-1b).
• Under Security groups, remove the default and select ALB-SG.

Now create the target group that the ALB will route traffic to:

• Under Listeners and routing, for the HTTP:80 listener, click Create target group (this opens a new tab).
• For Target type, keep Instances.
• For Target group name, enter CloudShop-AppTG.
• Keep Protocol as HTTP and Port as 80.
• Select CloudShop-VPC as the VPC.

Configure the health check so the ALB only sends traffic to healthy instances:

• Under Health checks, for Health check path, enter /health.
• Expand Advanced health check settings.
• Set Healthy threshold to 2.
• Set Interval to 30 seconds.
• Click Next, skip the register targets page (Auto Scaling will handle registration), and click Create target group.

Why /health as the health check path?

The ALB sends periodic GET requests to each instance's /health endpoint. If the instance responds with HTTP 200 twice in a row (healthy threshold of 2), the ALB considers it healthy and routes traffic to it. A dedicated health endpoint keeps health checks cheap and fast, separate from your business logic.

Return to the ALB creation tab and finish the setup:

• Click the refresh icon next to the target group dropdown.
• Select CloudShop-AppTG as the default action for the HTTP:80 listener.
• Click Create load balancer.

Wait for the ALB State to change from provisioning to active. This takes 2-3 minutes.

Now add path-based routing rules to the listener:

• Select CloudShop-ALB in the load balancers list.
• Click the Listeners and rules tab.
• Click the HTTP:80 link to open the listener detail page.
• Click Add rule to create the first path-based rule.

Create a rule that forwards API traffic to your app servers:

• For Name, enter api-forward.
• Click Next.
• Click Add condition, select Path, and enter /api/*.
• Click Confirm, then Next.
• For Action, choose Forward to target groups and select CloudShop-AppTG.
• Set the priority to 1.
• Click Create.

Now create a second rule that redirects static asset requests to your future CloudFront distribution:

• Click Add rule again.
• For Name, enter static-redirect.
• Click Next.
• Click Add condition, select Path, and enter /static/*.
• Click Confirm, then Next.
• For Action, choose Redirect to URL.
• Under URI parts, set Protocol to HTTPS, Port to 443, and keep the path as /#{path}. Set Status code to HTTP 301 - Permanently moved.
• Set the priority to 2.
• Click Create.

What is path-based routing?

Path-based routing lets a single load balancer route requests to different targets based on the URL path. Requests to /api/* hit your EC2 app servers, while /static/* gets redirected to CloudFront for edge caching.

This is a common microservices pattern tested heavily on the SAA-C03 exam. It lets you support multiple application tiers behind a single entry point.

Create the Auto Scaling Group with scaling policies

The Auto Scaling Group maintains a fleet of EC2 instances across both Availability Zones. It automatically launches new instances when demand spikes and terminates them when load drops. You'll configure target tracking on CPU utilization so the fleet adjusts without manual intervention.

• In the EC2 console left navigation, click Auto Scaling Groups at the bottom of the navigation pane.
• Click Create Auto Scaling group.
• For Auto Scaling group name, enter CloudShop-ASG.
• For Launch template, select CloudShop-LaunchTemplate.
• Click Next.

Configure where your instances will launch:

• Under Network, select CloudShop-VPC.
• For Availability Zones and subnets, select private-app-1a and private-app-1b.
• Click Next.

Why private subnets for compute?

Your EC2 instances live in private subnets and are not directly reachable from the internet. All inbound traffic flows through the ALB (which sits in public subnets). Outbound traffic from the instances routes through the NAT Gateway. This is a defense-in-depth pattern that limits your attack surface.

Attach the ALB so the ASG registers new instances with the target group automatically:

• Under Load balancing, select Attach to an existing load balancer.
• Choose Choose from your load balancer target groups.
• Select CloudShop-AppTG.
• Under Health checks, check Turn on Elastic Load Balancing health checks.
• Click Next.

Set the group size and scaling policy:

• Under Group size, set Desired capacity to 2.
• Under Scaling, set Min desired capacity to 1 and Max desired capacity to 4.
• Under Automatic scaling, select Target tracking scaling policy.
• For Metric type, choose Average CPU utilization.
• For Target value, enter 60.
• Click Next through the remaining pages (notifications, tags).
• On the Review page, click Create Auto Scaling group.

What is target tracking scaling?

Target tracking works like a thermostat. You set a target (60% CPU utilization), and AWS automatically creates CloudWatch alarms that trigger scaling actions. When average CPU across instances exceeds 60%, the group launches more instances. When it drops below, it terminates extras.

The predefined metric ASGAverageCPUUtilization aggregates CPU across all instances in the group. It scales inversely with capacity, making it ideal for target tracking.

Now add a scheduled scaling action for predictable traffic patterns:

• Select CloudShop-ASG in the Auto Scaling groups list.
• Click the Automatic scaling tab.
• Under Scheduled actions, click Create scheduled action.
• For Name, enter scale-up-business-hours.
• Set Desired capacity to 3, Min to 2, Max to 4.
• Set Recurrence to a cron expression for weekday mornings (e.g., 0 8 * * MON-FRI).
• Click Create.

Verify your ASG is launching instances by checking the Activity tab. You should see 2 instances launching (matching your desired capacity of 2).

Instances stuck in pending or terminating?

• Check that private-app-1a and private-app-1b have a route to the NAT Gateway. Without outbound internet, the user data script cannot download packages.
• Verify the App-SG security group allows inbound traffic on port 80 from ALB-SG.
• If instances fail health checks after launching, connect via Systems Manager Session Manager and check /var/log/cloud-init-output.log for user data script errors.

Help me debug why my Auto Scaling Group instances are failing to launch or pass health checks.

Finally, update your ADR with the Auto Scaling design decision. Open ~/cloudshop/ADR.md and add the following entry:

## ADR-011: Auto Scaling Design
- **Decision**: Target tracking CPU > 60% with min=1, max=4, desired=2
- **Context**: Variable e-commerce traffic requires elastic compute
- **Rationale**: Target tracking is simpler than step scaling; CPU is a reliable proxy for web server load
- **GCP Equivalent**: Managed Instance Group (MIG) with autoscaling
- **SAA-C03 Exam Relevance**: Domain 2 - resilient architectures; scaling policy types
- **Alternatives Considered**: Step scaling (more complex), scheduled only (not responsive to spikes)

What does this ADR entry do?

• Documents why target tracking was chosen over step scaling (simpler, self-managing).
• Maps to GCP's Managed Instance Group with autoscaling, which works similarly.
• Notes the SAA-C03 exam relevance: Auto Scaling patterns are a core topic in Domain 2 (Design Resilient Architectures), which accounts for 26% of the exam.

• Save ADR.md.

Your compute layer is live and elastic. Next up, you'll put a CDN in front of your static assets with CloudFront and configure DNS routing with Route 53.

Configure CDN and DNS

Your compute layer is load-balanced and auto-scaling, but users worldwide still connect directly to your ALB in us-east-1. Static assets like images and CSS files travel the full distance from your S3 bucket every single time.

In this step, you'll add a CloudFront distribution to cache static content at edge locations closer to users. You'll also configure Route 53 with failover routing so DNS automatically redirects traffic to a backup site if your ALB goes down.

In this step, get ready to:

• Create a CloudFront distribution with Origin Access Control for secure S3 delivery.
• Configure Route 53 with health checks and failover routing.
• Document CDN and DNS decisions in your ADR.

Create a CloudFront distribution with Origin Access Control

Origin Access Control (OAC) ensures that your S3 assets bucket can only be accessed through CloudFront. Direct S3 URLs will be blocked, forcing all traffic through your CDN.

• In the AWS Console, search for CloudFront and open the CloudFront service.
• In the left navigation pane, click Origin access.
• Click Create control setting.
• Enter CloudShop-OAC in the Name field.
• Leave Signing behavior set to the default Sign requests (recommended).
• Select S3 from the Origin type dropdown.
• Click Create.

What is Origin Access Control?

OAC is the modern replacement for Origin Access Identity (OAI). It uses AWS Signature Version 4 (SigV4) to sign every request CloudFront sends to your S3 origin.

This means your S3 bucket stays private. Only requests signed by your specific CloudFront distribution are allowed through. OAC is the recommended approach for new distributions.

Now create the distribution itself.

• In the left navigation, click Distributions.
• Click Create distribution.
• For Origin domain, select your CloudShop assets S3 bucket from the dropdown. It will appear as a .s3.us-east-1.amazonaws.com endpoint.
• Under Origin access, select Origin access control settings (recommended).
• From the Origin access control dropdown, select CloudShop-OAC (the OAC you just created).
• Under Default cache behavior, set Viewer protocol policy to HTTPS only.
• Leave the default TTL settings as-is (CloudFront uses the CachingOptimized managed cache policy by default).
• Under Settings, enter CloudShop-CF in the Description field.
• Click Create distribution.

Why HTTPS only?

Setting the viewer protocol policy to HTTPS only means CloudFront rejects any HTTP requests from users. This enforces encryption in transit between the viewer and the edge location, which is a security best practice tested on the SAA-C03 exam.

After creating the distribution, CloudFront will display a banner prompting you to update the S3 bucket policy. You need to grant CloudFront permission to read objects from your bucket.

• Click Copy policy in the yellow banner at the top of the distribution details page.
• Open the S3 console in a new tab.
• Navigate to your CloudShop assets bucket.
• Click the Permissions tab.
• Scroll down to Bucket policy and click Edit.
• Paste the copied policy into the editor. If you already have a policy (the deny-unencrypted-uploads statement from Step 4), merge the new statement into the existing Statement array.
• Click Save changes.

What does the OAC bucket policy do?

The policy allows the cloudfront.amazonaws.com service principal to perform s3:GetObject on your bucket. The condition clause restricts access to only your specific distribution's ARN, preventing other CloudFront distributions from reading your objects.

• Return to the CloudFront console and confirm the distribution Status shows Enabled. The Last modified field will show Deploying while the distribution propagates to edge locations.
• Copy the Distribution domain name (e.g., d1234abcdef8.cloudfront.net). You will use this in the Route 53 configuration.

Distribution stuck in Deploying?

CloudFront distributions typically take 5-10 minutes to deploy globally. If it takes longer than 15 minutes, verify that the OAC was correctly attached by checking the Origins tab on your distribution.

help me troubleshoot my CloudFront distribution deployment

Configure Route 53 with health checks and failover routing

Route 53 provides DNS-level failover. If your primary ALB becomes unhealthy, Route 53 automatically routes traffic to a secondary static website hosted on S3.

• In the AWS Console, search for Route 53 and open the service.
• In the left navigation, click Hosted zones.
• Click Create hosted zone.
• Enter cloudshop.internal as the Domain name.
• Select Public hosted zone as the Type.
• Click Create hosted zone.

Why a public hosted zone for a lab domain?

In production, you would use a registered domain name. For this lab, you are creating a public hosted zone to practice DNS record configuration and routing policies. The records will resolve within Route 53 but will not be publicly reachable without a registered domain.

Next, create a health check for the ALB endpoint so Route 53 can detect failures.

• In the left navigation, click Health checks.
• Click Create health check.
• Enter CloudShop-ALB-Health as the Name.
• Select Endpoint for What to monitor.
• Select Domain name under Specify endpoint by.
• Enter your ALB's DNS name (found in the EC2 console under Load Balancers) as the Domain name value.
• Set Port to 80.
• Enter /health as the Path.
• Click Next.
• On the Get notified page, select No for Create alarm (you will configure alarms separately in a later step).
• Click Create health check.

What does this health check do?

Route 53 sends HTTP requests to your ALB's /health endpoint every 30 seconds from multiple locations worldwide. If the endpoint fails to return a 2xx or 3xx response, Route 53 marks the resource as unhealthy and triggers the failover routing policy.

Now create the DNS records. Start with the primary failover record pointing to the ALB.

• In the left navigation, click Hosted zones.
• Click the cloudshop.internal hosted zone.
• Click Create record.
• Leave Record name blank (this creates a record for the zone apex cloudshop.internal).
• Set Record type to A.
• Toggle Alias to enabled.
• Under Route traffic to, select Alias to Application and Classic Load Balancer.
• Select US East (N. Virginia) as the region.
• Select your CloudShop-ALB from the load balancer dropdown.
• Set Routing policy to Failover.
• Set Failover record type to Primary.
• Select CloudShop-ALB-Health as the Health check.
• Enter primary as the Record ID.
• Click Create records.

Now create the secondary failover record pointing to an S3 static website as the backup.

• Click Create record again.
• Leave Record name blank.
• Set Record type to A.
• Toggle Alias to enabled.
• Under Route traffic to, select Alias to S3 website endpoint.
• Select US East (N. Virginia) as the region.
• Select the S3 bucket configured as a static website (your assets bucket or a separate maintenance page bucket).
• Set Routing policy to Failover.
• Set Failover record type to Secondary.
• Leave Health check unset for the secondary record.
• Enter secondary as the Record ID.
• Click Create records.

Why leave the secondary health check empty?

When Route 53 detects that the primary endpoint is unhealthy, it automatically returns the secondary record. Omitting the health check on the secondary means Route 53 will always fail over to it regardless of its health. This ensures users always get a response, even if it is just a maintenance page.

Finally, create a CNAME record for the CloudFront distribution.

• Click Create record.
• Enter cdn in the Record name field (this creates cdn.cloudshop.internal).
• Set Record type to CNAME.
• In the Value field, paste your CloudFront distribution domain name (e.g., d1234abcdef8.cloudfront.net).
• Set Routing policy to Simple routing.
• Click Create records.

• Verify your hosted zone now shows the two failover A records and the CNAME record alongside the default NS and SOA records.

Route 53 routing policies for the SAA exam

Route 53 supports six routing policies. Each fits a different use case:

• Simple: one resource, no health checks.
• Weighted: distribute traffic by percentage across multiple resources.
• Latency: route to the region with lowest latency for the user.
• Failover: active-passive. Primary handles traffic until unhealthy, then secondary takes over.
• Geolocation: route based on the geographic location of the user.
• Multivalue answer: return multiple healthy records (up to 8) for client-side load balancing.

Health check showing unhealthy?

The ALB security group (ALB-SG) must allow inbound traffic on port 80 from Route 53 health checker IP ranges. Also confirm your target group health check path is /health and that at least one EC2 instance is passing health checks in the target group.

help me troubleshoot my Route 53 health check

Document CDN and DNS decisions in your ADR

Record your CloudFront and Route 53 design decisions with GCP equivalents and SAA exam relevance.

• Open ~/cloudshop/ADR.md in your editor.
• Add the following two entries after ADR-012:

## ADR-013: CloudFront with OAC
- **Decision**: CloudFront distribution with Origin Access Control for S3 assets
- **Context**: OAC replaces OAI; prevents direct S3 access, enables edge caching globally
- **GCP Equivalent**: GCP Cloud CDN with Cloud Storage backend
- **SAA-C03 Exam Relevance**: OAC vs OAI, cache invalidation, HTTPS enforcement, edge locations
- **Alternatives Considered**: Direct S3 public access (rejected: security), OAI (rejected: legacy)
- **Outcome**: CloudFront with OAC; S3 bucket policy restricts access to CloudFront only

## ADR-014: Route 53 Failover Routing
- **Decision**: Failover routing policy with ALB as primary and S3 static website as secondary
- **Context**: Provides automatic DNS failover if ALB health check fails
- **GCP Equivalent**: GCP Cloud DNS with health checks
- **SAA-C03 Exam Relevance**: Route 53 routing policies (simple, weighted, latency, failover, geolocation, multivalue)
- **Alternatives Considered**: Simple routing (rejected: no failover), latency routing (rejected: single region)
- **Outcome**: Failover routing demonstrates HA DNS pattern for SAA exam

What does this ADR capture?

ADR-013 documents why OAC is preferred over the legacy OAI approach. The GCP equivalent is Cloud CDN with a Cloud Storage backend, which similarly restricts origin access.

ADR-014 captures the failover routing decision. In GCP, Cloud DNS provides similar health-check-based routing. The SAA exam frequently tests the differences between Route 53 routing policies, so documenting all six here is valuable study material.

• Save ADR.md.

✔️ Awesome, I've got everything!

Great. Double check you have saved your ADR file with both new entries.

ⓧ I'd like to double check the full code

Here is what the ADR should look like from ADR-013 onward:

## ADR-013: CloudFront with OAC
- **Decision**: CloudFront distribution with Origin Access Control for S3 assets
- **Context**: OAC replaces OAI; prevents direct S3 access, enables edge caching globally
- **GCP Equivalent**: GCP Cloud CDN with Cloud Storage backend
- **SAA-C03 Exam Relevance**: OAC vs OAI, cache invalidation, HTTPS enforcement, edge locations
- **Alternatives Considered**: Direct S3 public access (rejected: security), OAI (rejected: legacy)
- **Outcome**: CloudFront with OAC; S3 bucket policy restricts access to CloudFront only

## ADR-014: Route 53 Failover Routing
- **Decision**: Failover routing policy with ALB as primary and S3 static website as secondary
- **Context**: Provides automatic DNS failover if ALB health check fails
- **GCP Equivalent**: GCP Cloud DNS with health checks
- **SAA-C03 Exam Relevance**: Route 53 routing policies (simple, weighted, latency, failover, geolocation, multivalue)
- **Alternatives Considered**: Simple routing (rejected: no failover), latency routing (rejected: single region)
- **Outcome**: Failover routing demonstrates HA DNS pattern for SAA exam

Your CDN and DNS layer is configured. CloudFront caches static assets at edge locations worldwide, and Route 53 provides automatic DNS failover if your ALB ever goes down. Next up, you'll build the serverless event-driven architecture for order processing.

Build the Serverless Event-Driven Pipeline

Build the Serverless Event-Driven Architecture

Your compute layer is serving traffic, your CDN is caching assets, and Route 53 is routing requests with failover protection. The infrastructure can handle requests, but it has no way to process background work like orders or image uploads asynchronously.

In this step, you'll build an event-driven pipeline that decouples order processing from the web tier. Customers place orders through an API Gateway endpoint, messages flow into an SQS queue, Lambda functions process them, and SNS sends notifications. Step Functions orchestrates the entire workflow with built-in error handling.

In this step, get ready to:

• Create an SQS queue with a dead letter queue and an SNS notification topic.
• Deploy Lambda functions for order processing and image resizing.
• Build a Step Functions state machine and API Gateway REST API to orchestrate and expose the pipeline.

Create the SQS queue and SNS topic

The web tier should never process orders synchronously. If the order processor is slow or fails, it would block the entire request. A message queue decouples the producer (web tier) from the consumer (order processor).

Create the dead letter queue

You need to create the dead letter queue first because the main queue will reference it. A dead letter queue catches messages that fail processing repeatedly, preventing poison messages from blocking the pipeline.

• Open the Amazon SQS console at https://console.aws.amazon.com/sqs/.
• Click Create queue.
• Leave Type set to Standard.
• For Name, enter CloudShop-Orders-DLQ.
• Leave all configuration values at their defaults.
• Expand Encryption. Choose AWS Key Management Service key (SSE-KMS) and select your CloudShop-CMK key.
• Click Create queue.

What is a dead letter queue?

When a message fails processing more times than the configured threshold (called maxReceiveCount), SQS automatically moves it to the dead letter queue. This isolates problematic messages so they don't block healthy ones from processing.

You'll set maxReceiveCount to 3 on the main queue in the next step. After 3 failed processing attempts, the message moves to this DLQ for investigation.

Create the main orders queue

• Return to the SQS console Queues page.
• Click Create queue.
• Leave Type set to Standard.
• For Name, enter CloudShop-Orders.
• Under Configuration, confirm the Message retention period is set to 4 days (the default).
• Expand Encryption. Choose AWS Key Management Service key (SSE-KMS) and select your CloudShop-CMK key.
• Expand Dead-letter queue. Toggle it to Enabled.
• Select CloudShop-Orders-DLQ as the dead letter queue.
• Set Maximum receives to 3.
• Click Create queue.

Why 4 days and maxReceiveCount 3?

The 4-day retention gives you time to investigate and replay failed orders. Setting maxReceiveCount to 3 balances between allowing transient failures to retry and quickly isolating truly broken messages.

Create the SNS notifications topic

• Open the Amazon SNS console at https://console.aws.amazon.com/sns/.
• In the left navigation, click Topics.
• Click Create topic.
• Choose Standard as the type.
• For Name, enter CloudShop-Notifications.
• Expand Encryption and enable encryption with your CloudShop-CMK key.
• Click Create topic.
• On the topic details page, click Create subscription.
• For Protocol, select Email.
• For Endpoint, enter your email address.
• Click Create subscription.
• Check your email inbox and click the Confirm subscription link in the email from AWS Notifications.

Confirm the subscription promptly

SNS subscriptions remain in a pending state until confirmed. If you don't see the email, check your spam folder. The subscription will expire if not confirmed within 3 days.

You should now see two queues in the SQS console (CloudShop-Orders and CloudShop-Orders-DLQ) and one topic in the SNS console (CloudShop-Notifications).

Create and deploy the Lambda functions

Now you'll create two Lambda functions. The first processes orders from the SQS queue, writing to DynamoDB and publishing confirmations to SNS. The second resizes images uploaded to S3.

Write the ProcessOrder function code

• Create the function code file by running these commands in your terminal:

mkdir -p ~/cloudshop/lambda/process_order
cd ~/cloudshop/lambda/process_order

• Create a file called lambda_function.py with the following content:

import json
import boto3
import os
import uuid
from datetime import datetime

# Initialize AWS service clients
dynamodb = boto3.resource('dynamodb')
sns = boto3.client('sns')

# Get environment variables
TABLE_NAME = os.environ.get('TABLE_NAME', 'CloudShop-Sessions')
SNS_TOPIC_ARN = os.environ.get('SNS_TOPIC_ARN')

def lambda_handler(event, context):
    """Process orders from SQS queue."""
    table = dynamodb.Table(TABLE_NAME)

    for record in event['Records']:
        # Parse the order message from SQS
        body = json.loads(record['body'])
        order_id = body.get('orderId', str(uuid.uuid4()))

        # Write order data to DynamoDB
        table.put_item(
            Item={
                'sessionId': f'order-{order_id}',
                'orderData': json.dumps(body),
                'status': 'processed',
                'createdAt': datetime.utcnow().isoformat(),
                'expiresAt': int(datetime.utcnow().timestamp()) + 86400
            }
        )

        # Publish order confirmation to SNS
        if SNS_TOPIC_ARN:
            sns.publish(
                TopicArn=SNS_TOPIC_ARN,
                Subject=f'Order Confirmed: {order_id}',
                Message=json.dumps({
                    'orderId': order_id,
                    'status': 'confirmed',
                    'timestamp': datetime.utcnow().isoformat()
                })
            )

    return {
        'statusCode': 200,
        'body': json.dumps({'message': f'Processed {len(event["Records"])} orders'})
    }

What does this code do?

The handler receives SQS events containing one or more order messages. For each message it parses the order JSON, writes the order record to the CloudShop-Sessions DynamoDB table, and publishes a confirmation to the CloudShop-Notifications SNS topic.

The expiresAt field uses DynamoDB's TTL feature (configured in Step 5) to automatically clean up old order records after 24 hours.

Write the ResizeImage function code

• Create the second function's directory and code file by running:

mkdir -p ~/cloudshop/lambda/resize_image
cd ~/cloudshop/lambda/resize_image

• Create a file called lambda_function.py with the following content:

import json
import boto3
import os
import uuid
from urllib.parse import unquote_plus

# Initialize S3 client
s3_client = boto3.client('s3')

# Target dimensions for resized images
MAX_WIDTH = 800
MAX_HEIGHT = 600

def lambda_handler(event, context):
    """Resize images uploaded to the S3 uploads/ prefix."""
    for record in event['Records']:
        # Get the bucket and key from the S3 event
        bucket = record['s3']['bucket']['name']
        key = unquote_plus(record['s3']['object']['key'])

        # Download the uploaded image to /tmp
        download_path = f'/tmp/{uuid.uuid4()}-original'
        s3_client.download_file(bucket, key, download_path)

        # Create resized filename in processed/ prefix
        filename = key.split('/')[-1]
        resized_key = f'processed/{filename}'

        # For this demo, copy with metadata indicating resize target
        # In production, use Pillow library for actual image processing
        s3_client.copy_object(
            Bucket=bucket,
            CopySource={'Bucket': bucket, 'Key': key},
            Key=resized_key,
            Metadata={
                'resize-target': f'{MAX_WIDTH}x{MAX_HEIGHT}',
                'original-key': key
            },
            MetadataDirective='REPLACE'
        )

    return {
        'statusCode': 200,
        'body': json.dumps({'message': f'Processed {len(event["Records"])} images'})
    }

What does this code do?

This function triggers on S3 PUT events in the uploads/ prefix. It downloads the uploaded file, then copies it to a processed/ prefix with metadata indicating the resize dimensions.

In production you would add the Pillow library to the deployment package for actual image resizing. For this infrastructure project, the copy-with-metadata pattern demonstrates the S3 event trigger and Lambda processing pattern.

Package and deploy the Lambda functions

• Package the ProcessOrder function into a zip file by running:

cd ~/cloudshop/lambda/process_order
zip process_order.zip lambda_function.py

• Package the ResizeImage function into a zip file by running:

cd ~/cloudshop/lambda/resize_image
zip resize_image.zip lambda_function.py

Now deploy both functions through the Lambda console.

Deploy the ProcessOrder function

• Open the Lambda console at https://console.aws.amazon.com/lambda/.
• Click Create function.
• Choose Author from scratch.
• For Function name, enter ProcessOrder.
• For Runtime, choose Python 3.12.
• Expand Change default execution role. Choose Use an existing role and select CloudShop-Lambda-Role.
• Click Create function.
• On the function page, select the Code tab.
• In the Code source pane, click Upload from and choose .zip file.
• Click Upload, select your process_order.zip file, click Open, then click Save.

Configure ProcessOrder environment variables and VPC

• Select the Configuration tab.
• Click Environment variables in the left panel, then click Edit.
• Add a variable with key TABLE_NAME and value CloudShop-Sessions.
• Add a variable with key SNS_TOPIC_ARN and value set to the ARN of your CloudShop-Notifications topic (found on the SNS topic details page).
• Click Save.
• Click VPC in the left panel, then click Edit.
• Select your CloudShop-VPC.
• Select both private-app subnets (10.0.3.0/24 and 10.0.4.0/24).
• Select the App-SG security group.
• Click Save.

Why VPC access for ProcessOrder?

The ProcessOrder function writes to DynamoDB and publishes to SNS. Since it needs to access resources within your VPC (like ElastiCache in future iterations), placing it in the private-app subnets ensures it can reach internal services while the NAT Gateway provides outbound internet for AWS service API calls.

Configure the SQS trigger for ProcessOrder

• On the ProcessOrder function page, click Add trigger in the Function overview section.
• Select SQS as the trigger source.
• Choose the CloudShop-Orders queue.
• Leave batch size at 10 (the default).
• Click Add.

Deploy the ResizeImage function

• Return to the Lambda Functions page and click Create function.
• Choose Author from scratch.
• For Function name, enter ResizeImage.
• For Runtime, choose Python 3.12.
• Expand Change default execution role. Choose Use an existing role and select CloudShop-Lambda-Role.
• Click Create function.
• Select the Code tab. Click Upload from and choose .zip file.
• Upload your resize_image.zip file, then click Save.

Configure the S3 trigger for ResizeImage

• On the ResizeImage function page, click Add trigger.
• Select S3 as the trigger source.
• Choose your cloudshop-assets bucket.
• For Event types, select All object create events.
• For Prefix, enter uploads/.
• Check the acknowledgment box for recursive invocation.
• Click Add.

Both Lambda functions should now show their triggers in the function overview diagram. ProcessOrder shows an SQS trigger and ResizeImage shows an S3 trigger.

Function not showing a trigger?

Make sure you selected the correct queue or bucket. Verify that your CloudShop-Lambda-Role has permissions to read from SQS and access S3. The trigger may take a moment to propagate.

Help me debug my Lambda trigger configuration.

Create the Step Functions state machine and API Gateway

Individual Lambda functions handle single tasks, but a real order workflow involves multiple steps that must execute in sequence with error handling. Step Functions coordinates these steps as a visual workflow.

Create the OrderWorkflow state machine

• Open the Step Functions console at https://console.aws.amazon.com/states/.
• Click Create state machine.
• Choose Write your workflow in code.
• For Type, select Standard.
• Replace the default definition with the following JSON:

{
  "Comment": "CloudShop Order Workflow - validates, reserves, processes, and notifies",
  "StartAt": "ValidatePayment",
  "States": {
    "ValidatePayment": {
      "Type": "Pass",
      "Comment": "Validate payment information",
      "Result": {
        "paymentValid": true
      },
      "ResultPath": "$.paymentResult",
      "Next": "ReserveInventory"
    },
    "ReserveInventory": {
      "Type": "Task",
      "Resource": "arn:aws:states:::dynamodb:putItem",
      "Parameters": {
        "TableName": "CloudShop-Sessions",
        "Item": {
          "sessionId": {
            "S.$": "States.Format('reservation-{}', $.orderId)"
          },
          "status": {
            "S": "reserved"
          },
          "orderData": {
            "S.$": "States.JsonToString($.orderDetails)"
          }
        }
      },
      "ResultPath": "$.reservationResult",
      "Retry": [
        {
          "ErrorEquals": ["States.TaskFailed"],
          "IntervalSeconds": 2,
          "MaxAttempts": 3,
          "BackoffRate": 2.0
        }
      ],
      "Catch": [
        {
          "ErrorEquals": ["States.ALL"],
          "Next": "OrderFailed",
          "ResultPath": "$.error"
        }
      ],
      "Next": "ProcessOrder"
    },
    "ProcessOrder": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "Parameters": {
        "FunctionName": "ProcessOrder",
        "Payload.$": "$"
      },
      "ResultPath": "$.processResult",
      "Retry": [
        {
          "ErrorEquals": ["States.TaskFailed", "Lambda.ServiceException"],
          "IntervalSeconds": 2,
          "MaxAttempts": 3,
          "BackoffRate": 2.0
        }
      ],
      "Catch": [
        {
          "ErrorEquals": ["States.ALL"],
          "Next": "OrderFailed",
          "ResultPath": "$.error"
        }
      ],
      "Next": "SendNotification"
    },
    "SendNotification": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sns:publish",
      "Parameters": {
        "TopicArn": "arn:aws:sns:us-east-1:ACCOUNT_ID:CloudShop-Notifications",
        "Subject": "Order Complete",
        "Message.$": "States.Format('Order {} has been processed successfully.', $.orderId)"
      },
      "ResultPath": "$.notificationResult",
      "Next": "OrderComplete"
    },
    "OrderComplete": {
      "Type": "Succeed"
    },
    "OrderFailed": {
      "Type": "Fail",
      "Error": "OrderProcessingFailed",
      "Cause": "An error occurred during order processing."
    }
  }
}

What does this state machine do?

The workflow executes four steps in sequence: validate payment (a Pass state for demonstration), reserve inventory in DynamoDB, invoke the ProcessOrder Lambda function, and send an SNS notification. Each Task state includes retry logic (3 attempts with exponential backoff) and a catch block that routes failures to the OrderFailed terminal state.

Replace ACCOUNT_ID in the SNS TopicArn with your actual AWS account ID. You can find it in the top-right corner of the AWS console.

• Click Next.
• For State machine name, enter OrderWorkflow.
• Under Permissions, choose Create a new role. Step Functions will create a role with the necessary permissions.
• Click Create state machine.

You should see the state machine's visual workflow with the four steps connected in sequence and error paths branching to the OrderFailed state.

Create the API Gateway REST API

The API Gateway creates an HTTP endpoint that external systems (or the web tier) can call to submit orders. It sends messages directly to the SQS queue, decoupling the API layer from order processing.

• Open the API Gateway console at https://console.aws.amazon.com/apigateway/.
• Click Create API.
• Under REST API (not the private option), click Build.
• For API name, enter CloudShop-API.
• Leave Endpoint Type as Regional.
• Click Create API.
• Click Create Resource.
• For Resource Name, enter orders. The Resource Path auto-fills as /orders.
• Click Create Resource.
• With /orders selected, click Create Method.
• For Method type, select POST.
• For Integration type, select AWS Service.
• For AWS Region, select us-east-1.
• For AWS Service, select Simple Queue Service (SQS).
• For HTTP method, select POST.
• For Action Type, choose Use path override and enter your account ID followed by /CloudShop-Orders (the format is ACCOUNT_ID/CloudShop-Orders).
• For Execution role, enter the ARN of an IAM role that has sqs:SendMessage permission on the CloudShop-Orders queue.
• Click Create Method.
• Click Deploy API.
• For Stage, select New Stage and enter prod as the stage name.
• Click Deploy.

Why API Gateway directly to SQS?

Instead of routing through a Lambda function just to put a message on SQS, API Gateway integrates directly with SQS as an AWS service proxy. This removes a Lambda invocation from the critical path, reducing latency and cost. The actual processing happens asynchronously via the SQS-to-Lambda trigger you configured earlier.

Getting a 403 or 500 error when deploying?

Make sure the execution role ARN you entered has permission to call sqs:SendMessage on your CloudShop-Orders queue. You may need to create a dedicated IAM role for API Gateway with that permission.

Help me set up the IAM role for API Gateway to SQS integration.

Update the Architecture Decision Record

• Open ~/cloudshop/ADR.md and add the following entry after ADR-013:

## ADR-014: Event-Driven Architecture Patterns
- **Decision**: SQS for order decoupling, SNS for fan-out notifications, Step Functions for workflow orchestration
- **Context**: SQS = point-to-point queue with DLQ; SNS = pub/sub fan-out; EventBridge = event bus for cross-service routing
- **GCP Equivalent**: Pub/Sub (SNS+SQS), Cloud Functions (Lambda), Workflows (Step Functions)
- **SAA-C03 Exam Relevance**: SQS vs SNS vs EventBridge decision matrix; DLQ for poison message handling
- **Outcome**: SQS CloudShop-Orders → Lambda ProcessOrder → SNS CloudShop-Notifications; Step Functions OrderWorkflow for complex orchestration; API Gateway POST /orders as entry point

What does this decision record?

This ADR captures the decision matrix between SQS, SNS, and EventBridge. In GCP, Pub/Sub handles both queuing and fan-out in a single service. AWS separates these concerns: SQS for point-to-point guaranteed delivery, SNS for one-to-many fan-out, and EventBridge for event routing between services.

For the SAA-C03 exam, knowing when to use each service is critical. SQS is the answer when you need message retention, ordering guarantees, or dead letter queue handling. SNS is the answer for fan-out to multiple subscribers.

• Save ADR.md.

✔️ Awesome, I've got everything!

Great. Double check you've saved your ADR file with the new entry.

ⓧ I'd like to double check the full code

# CloudShop Architecture Decision Record

## ADR-001 through ADR-013: [Previous entries retained]

## ADR-014: Event-Driven Architecture Patterns
- **Decision**: SQS for order decoupling, SNS for fan-out notifications, Step Functions for workflow orchestration
- **Context**: SQS = point-to-point queue with DLQ; SNS = pub/sub fan-out; EventBridge = event bus for cross-service routing
- **GCP Equivalent**: Pub/Sub (SNS+SQS), Cloud Functions (Lambda), Workflows (Step Functions)
- **SAA-C03 Exam Relevance**: SQS vs SNS vs EventBridge decision matrix; DLQ for poison message handling
- **Outcome**: SQS CloudShop-Orders → Lambda ProcessOrder → SNS CloudShop-Notifications; Step Functions OrderWorkflow for complex orchestration; API Gateway POST /orders as entry point

Your serverless event-driven pipeline is now fully connected. Next up, you'll add monitoring, logging, and compliance checks to give you visibility into everything running across the stack.

Set Up Monitoring, Logging, and Compliance

Your CloudShop infrastructure is fully operational with a serverless event-driven backend processing orders. But without visibility into what is happening across all these services, diagnosing failures or spotting configuration drift becomes guesswork.

In this step you will add observability and governance layers. CloudTrail captures every API call for audit. AWS Config continuously checks resources against compliance rules. CloudWatch ties it all together with dashboards, alarms, and log analysis.

In this step, get ready to:

• Enable CloudTrail, AWS Config, and VPC Flow Logs for audit and compliance.
• Create a CloudWatch dashboard with alarms for critical metrics.
• Configure Logs Insights queries and a composite alarm combining multiple conditions.

Enable CloudTrail, AWS Config, and VPC Flow Logs

Every production environment needs three capabilities: an immutable audit log of who did what, continuous compliance checks, and network-level traffic visibility. You will set up all three now.

Create the CloudTrail multi-region trail

• In the AWS Console, search for CloudTrail and open the service.
• Click Create trail.
• Enter CloudShop-Trail as the trail name.
• Under Storage location, select Use existing S3 bucket and choose your cloudshop-logs bucket (created in Step 4).
• Enable Log file validation. This adds a digest file each hour so you can verify log integrity.
• Under Additional settings, enable Multi-region trail so every region's API calls are captured in one place.
• Click Next.
• On the Choose log events page, keep Management events selected with both Read and Write checked.
• Click Next, review your settings, then click Create trail.

What does CloudTrail do?

CloudTrail records every API call made in your AWS account. This means you can trace who created, modified, or deleted any resource and when.

Log file validation uses SHA-256 hashing so you can prove logs have not been tampered with. This is critical for security audits and compliance frameworks.

Enable AWS Config with compliance rules

• Search for Config in the console and open the service.
• If this is your first time using Config, click Get started.
• On the Settings page, keep Record all current and future resource types selected.
• For the delivery channel, select your cloudshop-logs S3 bucket.
• Click Next.
• On the Rules page, click Add rule and search for ec2-instances-in-vpc. Select it and click Next.
• Accept the defaults and click Add rule.
• Repeat to add the rule encrypted-volumes.
• Repeat once more to add the rule s3-bucket-server-side-encryption-enabled.
• Click Next, review, then click Confirm.

What do these Config rules check?

• ec2-instances-in-vpc verifies that every EC2 instance belongs to a VPC. The rule identifier is INSTANCES_IN_VPC.
• encrypted-volumes checks that attached EBS volumes are encrypted at rest.
• s3-bucket-server-side-encryption-enabled confirms every S3 bucket has default encryption enabled or its bucket policy denies unencrypted uploads.

Enable VPC Flow Logs

• Navigate to VPC in the console.
• Select Your VPCs in the sidebar, then select your CloudShop VPC.
• Click the Flow logs tab, then click Create flow log.
• Set Filter to All (captures accepted and rejected traffic).
• Set Destination to Send to CloudWatch Logs.
• For Destination log group, click Create log group and enter /aws/vpc/cloudshop-flowlogs as the name.
• For IAM role, select or create a role that allows VPC to publish logs to CloudWatch Logs.
• Click Create flow log.

What are VPC Flow Logs?

VPC Flow Logs capture metadata about every network packet entering and leaving your VPC's network interfaces. They record source and destination IPs, ports, protocol, bytes transferred, and whether the traffic was accepted or rejected by security groups or NACLs.

Sending them to CloudWatch Logs lets you query traffic patterns later with Logs Insights.

Verify the trail is active by returning to the CloudTrail console. You should see CloudShop-Trail with status Logging.

Trail not logging?

Check that your cloudshop-logs bucket policy allows CloudTrail to write objects. CloudTrail requires a policy statement granting s3:PutObject from the CloudTrail service principal.

help me fix my CloudTrail trail that won't start logging

Create the CloudWatch dashboard and alarms

A dashboard gives you a single pane of glass across all CloudShop services. Alarms turn passive metrics into active notifications so you know when thresholds are breached.

Build the CloudShop dashboard

• Search for CloudWatch in the console and open the service.
• In the left sidebar, click Dashboards, then click Create dashboard.
• Enter CloudShop-Dashboard as the name and click Create dashboard.
• You will be prompted to add your first widget. Select Line as the widget type and click Next.
• Select Metrics as the data source.

Now add six widgets, one at a time. For each widget, use the metric browser to locate the correct namespace and metric:

• Widget 1: Under ApplicationELB namespace, select RequestCount grouped by LoadBalancer. Title it "ALB Request Count".
• Widget 2: Under EC2 namespace, select CPUUtilization grouped by AutoScalingGroupName. Title it "EC2 CPU Utilization".
• Widget 3: Under RDS namespace, select DatabaseConnections for your CloudShop DB instance. Title it "RDS Connections".
• Widget 4: Under Lambda namespace, select Invocations for ProcessOrder and ResizeImage functions. Title it "Lambda Invocations".
• Widget 5: Under SQS namespace, select ApproximateNumberOfMessagesVisible for CloudShop-Orders. Title it "SQS Queue Depth".
• Widget 6: Under DynamoDB namespace, select ConsumedReadCapacityUnits and ConsumedWriteCapacityUnits for CloudShop-Sessions. Title it "DynamoDB Consumed Capacity".

• After adding all six widgets, click Save dashboard.

Why these six metrics?

Each metric represents a different tier of your architecture. ALB request count shows external traffic volume. EC2 CPU shows compute pressure. RDS connections and DynamoDB capacity show database load. Lambda invocations show serverless activity. SQS depth reveals processing backlog.

Together they give you end-to-end visibility. If SQS depth rises while Lambda invocations stay flat, you know the consumer is stuck.

Create CloudWatch alarms

You will create three metric alarms that notify your existing CloudShop-Notifications SNS topic when thresholds are breached.

• In CloudWatch, click Alarms in the left sidebar, then click Create alarm.
• Click Select metric, navigate to EC2 > By Auto Scaling Group, and select CPUUtilization.
• Set the statistic to Average and the period to 5 minutes.
• Under Conditions, select Greater than and enter 80 as the threshold.
• Click Next. Under Notification, select Select an existing SNS topic and choose CloudShop-Notifications.
• Name the alarm CloudShop-HighCPU and click Create alarm.

Now create the second alarm for 5XX errors:

• Click Create alarm again.
• Select metric under ApplicationELB > Per AppELB Metrics, and choose HTTPCode_ELB_5XX_Count.
• Set the statistic to Sum and the period to 1 minute.
• Set condition to Greater than with threshold 10.
• Select the CloudShop-Notifications SNS topic for notification.
• Name the alarm CloudShop-5XXErrors and click Create alarm.

Finally, create the third alarm for SQS queue depth:

• Click Create alarm again.
• Select metric under SQS > Queue Metrics, and choose ApproximateNumberOfMessagesVisible for CloudShop-Orders.
• Set the statistic to Average and the period to 5 minutes.
• Set condition to Greater than with threshold 100.
• Select the CloudShop-Notifications SNS topic for notification.
• Name the alarm CloudShop-QueueDepth and click Create alarm.

You should now see three alarms listed in the CloudWatch Alarms console. Their initial state will be Insufficient data until enough data points are collected.

Alarm stuck in Insufficient data?

This is normal for newly created alarms. CloudWatch needs at least one evaluation period of data before transitioning to OK or ALARM state. Wait 5-10 minutes and the state will update.

help me understand why my CloudWatch alarm is stuck in Insufficient data state

Configure Logs Insights, composite alarm, and update the ADR

Individual alarms tell you about isolated problems. A composite alarm tells you when multiple things go wrong simultaneously. CloudWatch Logs Insights lets you run SQL-like queries across your log data to find patterns.

Run CloudWatch Logs Insights queries

• In the CloudWatch console, click Logs in the sidebar, then click Logs Insights.
• In the Select log group(s) dropdown, choose /aws/vpc/cloudshop-flowlogs.

Run the following query to see rejected traffic in your VPC:

fields @timestamp, srcAddr, dstAddr, dstPort, action
| filter action = "REJECT"
| sort @timestamp desc
| limit 20

What does this query do?

This query filters VPC Flow Log records to show only rejected traffic. Rejected packets reveal security group or NACL blocks in action. Seeing repeated rejects from the same source IP could indicate a port scan or misconfigured client.

• Click Run query. You should see results if any traffic has been rejected by your security groups or NACLs.
• Click Save and name the query CloudShop-RejectedTraffic.

Now run a second query for traffic volume analysis:

stats sum(bytes) as totalBytes by dstPort
| sort totalBytes desc
| limit 10

• Click Run query. This shows which destination ports handle the most bytes.
• Click Save and name this query CloudShop-TrafficByPort.

Create the composite alarm

• In CloudWatch, click Alarms, then click All alarms.
• Select the checkboxes next to CloudShop-HighCPU, CloudShop-5XXErrors, and CloudShop-QueueDepth.
• Click Create composite alarm.
• Under Specify composite alarm conditions, verify the rule expression uses ALARM for all three alarms combined with OR logic. The expression should read:

ALARM(CloudShop-HighCPU) OR ALARM(CloudShop-5XXErrors) OR ALARM(CloudShop-QueueDepth)

Why OR instead of AND?

Using OR means the composite fires if any single condition is breached. This makes sense for a "system health" alarm where any one failure deserves attention. An AND composite would only fire when all three are simultaneously failing. That is useful for reducing noise during partial issues, but for CloudShop you want early warning.

• Click Next.
• Under Configure actions, select Select an existing SNS topic and choose CloudShop-Notifications.
• Click Next.
• Name the alarm CloudShop-SystemHealth and add a description: "Composite alarm fires when any critical CloudShop metric is breached."
• Click Next, review, then click Create composite alarm.

You should now see four alarms total in the CloudWatch Alarms console. Three metric alarms and one composite alarm labeled CloudShop-SystemHealth.

Update the Architecture Decision Record

• Switch back to your terminal and open ~/cloudshop/ADR.md in your editor.
• Add the following entry after the existing ADR-014 section:

## ADR-015: Observability Strategy
- **Decision**: CloudWatch for metrics/logs/alarms; CloudTrail for API audit; AWS Config for compliance
- **Context**: CloudWatch covers operational monitoring; CloudTrail provides immutable API audit trail; Config tracks resource configuration drift
- **GCP Equivalent**: Cloud Monitoring (CloudWatch), Cloud Logging (CloudWatch Logs), Cloud Audit Logs (CloudTrail)
- **SAA-C03 Exam Relevance**: CloudWatch vs CloudTrail distinction; composite alarms; Config rules for compliance
- **Outcome**: Unified dashboard, three alarms with composite, CloudTrail to S3, Config with 3 managed rules, VPC Flow Logs to CloudWatch

CloudWatch vs CloudTrail on the SAA-C03 exam

A common exam question asks you to distinguish between CloudWatch and CloudTrail. CloudWatch monitors performance (metrics, logs, alarms). CloudTrail monitors API activity (who called what, when, from where). They are complementary, not interchangeable.

Composite alarms are also testable. The exam may present a scenario where you need to reduce alarm noise by combining multiple metric alarms into one notification pathway.

• Save ADR.md.

✔️ Awesome, I've got everything!

Great. Double check you have saved ADR.md with ADR-015 appended after the existing ADR-014 entry.

ⓧ I'd like to double check the full code

Here is the complete ADR.md content for this step:

# CloudShop Architecture Decision Record

## ADR-001 through ADR-014: [Previous entries retained]

## ADR-015: Observability Strategy
- **Decision**: CloudWatch for metrics/logs/alarms; CloudTrail for API audit; AWS Config for compliance
- **Context**: CloudWatch covers operational monitoring; CloudTrail provides immutable API audit trail; Config tracks resource configuration drift
- **GCP Equivalent**: Cloud Monitoring (CloudWatch), Cloud Logging (CloudWatch Logs), Cloud Audit Logs (CloudTrail)
- **SAA-C03 Exam Relevance**: CloudWatch vs CloudTrail distinction; composite alarms; Config rules for compliance
- **Outcome**: Unified dashboard, three alarms with composite, CloudTrail to S3, Config with 3 managed rules, VPC Flow Logs to CloudWatch

Config rules showing non-compliant?

If ec2-instances-in-vpc or encrypted-volumes shows non-compliant results, this is expected for any default-VPC resources or unencrypted EBS volumes outside your CloudShop setup. Config evaluates all resources in the account, not just yours.

help me interpret my AWS Config non-compliant results

Your infrastructure now has full observability. Every API call is audited, resources are checked against compliance rules, network traffic is logged, and critical metrics trigger alerts. Next up, you will build a real-time analytics pipeline to extract business insights from user activity data.

Build the Analytics Pipeline

Your CloudShop infrastructure now has full observability with CloudWatch dashboards, alarms, and compliance checking. The next piece is understanding how customers use the platform.

In this step, you will build a serverless analytics pipeline that captures user activity events, buffers them into columnar Parquet files on S3, and makes them queryable with standard SQL. This is the data lake pattern tested heavily on the SAA-C03 exam.

In this step, get ready to:

• Create a Kinesis Data Firehose delivery stream that converts JSON events to Parquet and delivers them to S3.
• Deploy a Lambda function that simulates user activity and generates sample data.
• Query the analytics data lake with Athena and save reusable named queries.

Create the Firehose delivery stream

A Kinesis Data Firehose delivery stream is a fully managed service that captures, transforms, and loads streaming data into destinations like S3. By enabling Data Format Conversion, Firehose automatically converts incoming JSON records into Parquet columnar format before writing to S3.

• Create a JSON configuration file for the Firehose delivery stream by running this command:

cat > ~/cloudshop/firehose-config.json << 'EOF'
{
  "RoleARN": "arn:aws:iam::YOUR_ACCOUNT_ID:role/CloudShop-Firehose-Role",
  "BucketARN": "arn:aws:s3:::cloudshop-analytics-YOUR_ACCOUNT_ID",
  "Prefix": "raw/year=!{timestamp:yyyy}/month=!{timestamp:MM}/day=!{timestamp:dd}/",
  "ErrorOutputPrefix": "errors/",
  "BufferingHints": {
    "SizeInMBs": 64,
    "IntervalInSeconds": 60
  },
  "EncryptionConfiguration": {
    "KMSEncryptionConfig": {
      "AWSKMSKeyARN": "arn:aws:kms:us-east-1:YOUR_ACCOUNT_ID:key/YOUR_CMK_KEY_ID"
    }
  },
  "DataFormatConversionConfiguration": {
    "Enabled": true,
    "SchemaConfiguration": {
      "DatabaseName": "cloudshop_analytics",
      "TableName": "user_activity",
      "Region": "us-east-1",
      "RoleARN": "arn:aws:iam::YOUR_ACCOUNT_ID:role/CloudShop-Firehose-Role"
    },
    "InputFormatConfiguration": {
      "Deserializer": {
        "OpenXJsonSerDe": {}
      }
    },
    "OutputFormatConfiguration": {
      "Serializer": {
        "ParquetSerDe": {}
      }
    }
  }
}
EOF

What does this configuration do?

This JSON defines the Extended S3 destination configuration for your Firehose stream.

• BufferingHints sets a 60-second interval, meaning Firehose delivers data to S3 at least every 60 seconds (or when the buffer reaches 64 MB).
• DataFormatConversionConfiguration tells Firehose to deserialize incoming JSON using OpenXJsonSerDe and serialize output using ParquetSerDe.
• SchemaConfiguration points to the Glue Data Catalog table that defines the schema for conversion.
• The KMS encryption ensures all data written to S3 is encrypted at rest with your CloudShop-CMK.

• Before creating the stream, you need the Glue database and table in place (Firehose references the Glue schema for Parquet conversion). Create the Glue database by running:

aws glue create-database \
  --database-input '{"Name": "cloudshop_analytics", "Description": "CloudShop analytics data lake"}' \
  --profile cloudshop

• Create the Glue table that defines the schema for user activity events by running:

aws glue create-table \
  --database-name cloudshop_analytics \
  --table-input '{
    "Name": "user_activity",
    "StorageDescriptor": {
      "Columns": [
        {"Name": "event_id", "Type": "string"},
        {"Name": "user_id", "Type": "string"},
        {"Name": "event_type", "Type": "string"},
        {"Name": "product_id", "Type": "string"},
        {"Name": "product_name", "Type": "string"},
        {"Name": "category", "Type": "string"},
        {"Name": "timestamp", "Type": "string"},
        {"Name": "session_id", "Type": "string"}
      ],
      "Location": "s3://cloudshop-analytics-YOUR_ACCOUNT_ID/raw/",
      "InputFormat": "org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat",
      "OutputFormat": "org.apache.hadoop.hive.ql.io.parquet.MapredParquetOutputFormat",
      "SerdeInfo": {
        "SerializationLibrary": "org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe"
      }
    }
  }' \
  --profile cloudshop

What is AWS Glue Data Catalog?

The AWS Glue Data Catalog is a centralized metadata repository. It stores table definitions (column names, data types, S3 locations) so that services like Firehose, Athena, and EMR can share a single schema definition.

In GCP, the equivalent role is played by BigQuery's dataset metadata or Dataflow schema definitions.

• Now create the Firehose delivery stream itself by running:

aws firehose create-delivery-stream \
  --delivery-stream-name CloudShop-Firehose \
  --delivery-stream-type DirectPut \
  --extended-s3-destination-configuration file://~/cloudshop/firehose-config.json \
  --profile cloudshop

• Verify the delivery stream is active by running:

aws firehose describe-delivery-stream \
  --delivery-stream-name CloudShop-Firehose \
  --query 'DeliveryStreamDescription.DeliveryStreamStatus' \
  --output text \
  --profile cloudshop

You should see ACTIVE in the output. If you see CREATING, wait 30 seconds and run the command again.

Stream status shows CREATING_FAILED?

This usually means the IAM role ARN or the Glue table reference is incorrect. Check that cloudshop_analytics.user_activity exists in the Glue catalog and that the Firehose role has permission to access Glue and S3.

If the stream failed, you must delete it before creating a new one with the same name. Help me troubleshoot my Firehose CREATING_FAILED status

Generate sample analytics data with Lambda

With the delivery stream active, you need data flowing through it. You will create a Lambda function that simulates user activity events (page views, add-to-cart actions, and purchases) and sends them to the Firehose stream.

• Create the Lambda function code by running:

cat > ~/cloudshop/simulate_activity.py << 'EOF'
import json
import random
import uuid
import boto3
from datetime import datetime

# Initialize Firehose client
firehose = boto3.client('firehose')

# Sample product catalog
PRODUCTS = [
    {"id": "PROD-001", "name": "Wireless Headphones", "category": "Electronics"},
    {"id": "PROD-002", "name": "Running Shoes", "category": "Sports"},
    {"id": "PROD-003", "name": "Coffee Maker", "category": "Kitchen"},
    {"id": "PROD-004", "name": "Laptop Stand", "category": "Office"},
    {"id": "PROD-005", "name": "Yoga Mat", "category": "Sports"},
    {"id": "PROD-006", "name": "Bluetooth Speaker", "category": "Electronics"},
    {"id": "PROD-007", "name": "Water Bottle", "category": "Sports"},
    {"id": "PROD-008", "name": "Desk Lamp", "category": "Office"}
]

# Event type weights (page_view is most common)
EVENT_TYPES = ["page_view", "page_view", "page_view", "add_to_cart", "add_to_cart", "purchase"]

def lambda_handler(event, context):
    # Generate 50 simulated user activity events
    records = []
    for _ in range(50):
        product = random.choice(PRODUCTS)
        activity = {
            "event_id": str(uuid.uuid4()),
            "user_id": f"USER-{random.randint(1000, 9999)}",
            "event_type": random.choice(EVENT_TYPES),
            "product_id": product["id"],
            "product_name": product["name"],
            "category": product["category"],
            "timestamp": datetime.utcnow().isoformat(),
            "session_id": str(uuid.uuid4())[:8]
        }
        records.append({"Data": json.dumps(activity) + "\n"})

    # Send records to Firehose in batch
    response = firehose.put_record_batch(
        DeliveryStreamName="CloudShop-Firehose",
        Records=records
    )

    return {
        "statusCode": 200,
        "body": f"Sent {len(records)} events. Failed: {response['FailedPutCount']}"
    }
EOF

What does this Lambda do?

Each invocation generates 50 random user activity events covering three event types: page views, add-to-cart, and purchases.

The weighted EVENT_TYPES list makes page views 3x more common than purchases, simulating a realistic conversion funnel. The function uses put_record_batch to send all 50 records to Firehose in a single API call.

• Package and deploy the Lambda function by running:

cd ~/cloudshop && zip simulate_activity.zip simulate_activity.py

aws lambda create-function \
  --function-name SimulateActivity \
  --runtime python3.12 \
  --handler simulate_activity.lambda_handler \
  --role arn:aws:iam::YOUR_ACCOUNT_ID:role/CloudShop-Lambda-Role \
  --zip-file fileb://simulate_activity.zip \
  --timeout 30 \
  --profile cloudshop

• Invoke the function three times to generate sample data by running:

aws lambda invoke --function-name SimulateActivity --profile cloudshop /tmp/out1.json
aws lambda invoke --function-name SimulateActivity --profile cloudshop /tmp/out2.json
aws lambda invoke --function-name SimulateActivity --profile cloudshop /tmp/out3.json
cat /tmp/out3.json

You should see a response containing "statusCode": 200 and a body reporting 50 events sent with 0 failures.

Why invoke three times?

Each invocation sends 50 events, giving you 150 total records. With the 60-second buffer interval, Firehose will batch these into one or two Parquet files in your analytics S3 bucket within a minute or two.

• Wait about 90 seconds for Firehose to flush the buffer, then verify data landed in S3 by running:

aws s3 ls s3://cloudshop-analytics-YOUR_ACCOUNT_ID/raw/ --recursive --profile cloudshop

You should see one or more .parquet files listed under the date-partitioned prefix.

No files in the analytics bucket?

Wait a full two minutes and try again. Firehose buffers data for the configured interval (60 seconds) before writing. Also check the errors/ prefix in the same bucket for any conversion failures.

Help me debug why my Firehose stream is not delivering data to S3

Query the data lake with Athena

With Parquet data in S3 and a Glue table defining the schema, Amazon Athena can query the data using standard SQL without any servers to manage. Athena reads the schema from the Glue Data Catalog and scans only the columns your query needs (thanks to Parquet's columnar format).

• First, configure an Athena query results location by running:

aws athena start-query-execution \
  --query-string "SELECT * FROM cloudshop_analytics.user_activity LIMIT 5" \
  --result-configuration "OutputLocation=s3://cloudshop-analytics-YOUR_ACCOUNT_ID/athena-results/" \
  --profile cloudshop

You should see a QueryExecutionId in the response, confirming the query was submitted.

• Save the Top Products Viewed query as a named query by running:

aws athena create-named-query \
  --name "TopProductsViewed" \
  --description "Top 10 most viewed products" \
  --database cloudshop_analytics \
  --query-string "SELECT product_name, product_id, COUNT(*) as view_count FROM user_activity WHERE event_type = 'page_view' GROUP BY product_name, product_id ORDER BY view_count DESC LIMIT 10" \
  --profile cloudshop

• Save the Conversion Funnel query as a named query by running:

aws athena create-named-query \
  --name "ConversionFunnel" \
  --description "Conversion funnel from page views to purchases" \
  --database cloudshop_analytics \
  --query-string "SELECT event_type, COUNT(*) as event_count FROM user_activity GROUP BY event_type ORDER BY event_count DESC" \
  --profile cloudshop

• Save the Activity By Hour query as a named query by running:

aws athena create-named-query \
  --name "ActivityByHour" \
  --description "User activity distribution by hour" \
  --database cloudshop_analytics \
  --query-string "SELECT SUBSTR(timestamp, 12, 2) as hour, COUNT(*) as events FROM user_activity GROUP BY SUBSTR(timestamp, 12, 2) ORDER BY hour" \
  --profile cloudshop

What are named queries?

Named queries are saved SQL statements stored in Athena's workspace. They let your team rerun common analytics without rewriting SQL each time.

The response from each command includes a NamedQueryId that you can use to retrieve or execute the query later.

• Run the Top Products Viewed query to verify results by running:

aws athena start-query-execution \
  --query-string "SELECT product_name, product_id, COUNT(*) as view_count FROM cloudshop_analytics.user_activity WHERE event_type = 'page_view' GROUP BY product_name, product_id ORDER BY view_count DESC LIMIT 10" \
  --result-configuration "OutputLocation=s3://cloudshop-analytics-YOUR_ACCOUNT_ID/athena-results/" \
  --profile cloudshop

Check the query results in the Athena console or by downloading the CSV from the athena-results/ prefix in your analytics bucket.

Why Firehose instead of Kinesis Data Streams?

Kinesis Data Streams is designed for real-time processing where consumers read records within seconds. Firehose is a delivery-focused service that buffers and batches records before writing to a destination.

For analytics workloads where you query data minutes or hours after it arrives, Firehose is simpler and cheaper. You do not need to manage shards, consumers, or checkpointing. On the SAA-C03 exam, the key distinction is batch delivery (Firehose) versus real-time processing (Data Streams).

• Update your ADR with the analytics pipeline decision by running:

cat >> ~/cloudshop/ADR.md << 'EOF'

## ADR-018: Analytics Data Lake
- **Decision**: Kinesis Firehose → S3 (Parquet) → Athena for analytics
- **Context**: Serverless analytics pipeline; Parquet columnar format reduces Athena query costs
- **GCP Equivalent**: GCP Dataflow (Firehose), GCP BigQuery (Athena), GCP Cloud Storage (S3)
- **SAA-C03 Exam Relevance**: Firehose vs Kinesis Data Streams (batch delivery vs real-time), Athena serverless querying, Glue catalog
- **Alternatives Considered**: Kinesis Data Streams (rejected: real-time processing not needed), Redshift (rejected: cost for lab)
- **Outcome**: Firehose for buffered delivery; Athena for ad-hoc SQL on S3 data lake
EOF

• Verify the ADR was updated by running:

tail -8 ~/cloudshop/ADR.md

You should see the ADR-018 entry with the analytics data lake decision.

Athena query returns zero results?

Make sure the Glue table's Location points to the same S3 prefix where Firehose delivers data. If you used a date-partitioned prefix in the Firehose config, the table location should match the root prefix (s3://cloudshop-analytics-YOUR_ACCOUNT_ID/raw/).

Also confirm that Firehose has had time to flush at least one batch (90 seconds minimum). Help me debug why Athena returns no results from my user_activity table

✔️ Awesome, I've got everything!

Great. Double check you have saved your ADR.md updates and confirmed all three named queries were created.

ⓧ I'd like to double check the full code

Here is the complete ADR-018 entry that should now appear at the bottom of your ~/cloudshop/ADR.md file:

## ADR-018: Analytics Data Lake
- **Decision**: Kinesis Firehose → S3 (Parquet) → Athena for analytics
- **Context**: Serverless analytics pipeline; Parquet columnar format reduces Athena query costs
- **GCP Equivalent**: GCP Dataflow (Firehose), GCP BigQuery (Athena), GCP Cloud Storage (S3)
- **SAA-C03 Exam Relevance**: Firehose vs Kinesis Data Streams (batch delivery vs real-time), Athena serverless querying, Glue catalog
- **Alternatives Considered**: Kinesis Data Streams (rejected: real-time processing not needed), Redshift (rejected: cost for lab)
- **Outcome**: Firehose for buffered delivery; Athena for ad-hoc SQL on S3 data lake

Your analytics pipeline is complete. You now have a serverless data lake that captures user events, stores them in cost-efficient Parquet format, and makes them queryable with SQL. Next up, you will harden the entire infrastructure with WAF rules and Systems Manager automation.

Harden Security with WAF and Systems Manager

Your CloudShop infrastructure is live, monitored, and producing analytics. The final defensive layer protects the application at the network edge and ensures secure management access to private instances.

In this step, you will add AWS WAF to inspect and filter HTTP traffic before it reaches the ALB. You will also configure Systems Manager Parameter Store for encrypted application settings and use Session Manager to access private instances without opening SSH ports.

In this step, get ready to:

• Create a WAF WebACL with rate limiting, managed rule groups, and a geo-restriction rule.
• Configure Systems Manager Parameter Store and connect to a private instance via Session Manager.
• Run Systems Manager Automation for patching and document security decisions in the ADR.

Create WAF WebACL and associate it with the ALB

AWS WAF operates at Layer 7, inspecting HTTP requests before they hit your application. Security Groups filter at Layer 3/4 only, so WAF adds critical protection against SQL injection, cross-site scripting, and volumetric abuse.

• Create a rules JSON file for the WAF WebACL by running this command:

cat > ~/cloudshop/waf-rules.json << 'EOF'
[
  {
    "Name": "RateLimitRule",
    "Priority": 1,
    "Statement": {
      "RateBasedStatement": {
        "Limit": 1000,
        "AggregateKeyType": "IP"
      }
    },
    "Action": {
      "Block": {}
    },
    "VisibilityConfig": {
      "SampledRequestsEnabled": true,
      "CloudWatchMetricsEnabled": true,
      "MetricName": "RateLimitRule"
    }
  },
  {
    "Name": "AWSManagedCommonRuleSet",
    "Priority": 2,
    "Statement": {
      "ManagedRuleGroupStatement": {
        "VendorName": "AWS",
        "Name": "AWSManagedRulesCommonRuleSet"
      }
    },
    "OverrideAction": {
      "None": {}
    },
    "VisibilityConfig": {
      "SampledRequestsEnabled": true,
      "CloudWatchMetricsEnabled": true,
      "MetricName": "AWSManagedCommonRuleSet"
    }
  },
  {
    "Name": "AWSManagedSQLiRuleSet",
    "Priority": 3,
    "Statement": {
      "ManagedRuleGroupStatement": {
        "VendorName": "AWS",
        "Name": "AWSManagedRulesSQLiRuleSet"
      }
    },
    "OverrideAction": {
      "None": {}
    },
    "VisibilityConfig": {
      "SampledRequestsEnabled": true,
      "CloudWatchMetricsEnabled": true,
      "MetricName": "AWSManagedSQLiRuleSet"
    }
  },
  {
    "Name": "GeoRestrictionRule",
    "Priority": 4,
    "Statement": {
      "NotStatement": {
        "Statement": {
          "GeoMatchStatement": {
            "CountryCodes": ["US", "AU", "GB", "CA"]
          }
        }
      }
    },
    "Action": {
      "Block": {}
    },
    "VisibilityConfig": {
      "SampledRequestsEnabled": true,
      "CloudWatchMetricsEnabled": true,
      "MetricName": "GeoRestrictionRule"
    }
  }
]
EOF

What does this rules file define?

• RateLimitRule blocks any single IP that sends more than 1000 requests in a 5-minute window.
• AWSManagedRulesCommonRuleSet covers OWASP Top 10 vulnerabilities including cross-site scripting and path traversal.
• AWSManagedRulesSQLiRuleSet detects and blocks SQL injection patterns in request bodies and query strings.
• GeoRestrictionRule blocks traffic from any country not in the allowed list. Adjust the country codes to match your use case.

• Create the WAF WebACL by running this command:

aws wafv2 create-web-acl \
    --name CloudShop-WAF \
    --scope REGIONAL \
    --default-action Allow={} \
    --visibility-config SampledRequestsEnabled=true,CloudWatchMetricsEnabled=true,MetricName=CloudShopWAF \
    --rules file://~/cloudshop/waf-rules.json \
    --region us-east-1 \
    --profile cloudshop

What does this command do?

This creates a regional WebACL named CloudShop-WAF with a default action of Allow. Requests that do not match any rule pass through. Requests matching a rule get blocked.

The --scope REGIONAL flag is required for ALB associations. CloudFront distributions use CLOUDFRONT scope instead.

• Note the WebACL ARN from the output. You will need it for the next command.
• Retrieve the ALB ARN by running:

aws elbv2 describe-load-balancers \
    --names CloudShop-ALB \
    --query "LoadBalancers[0].LoadBalancerArn" \
    --output text \
    --profile cloudshop \
    --region us-east-1

• Associate the WebACL with the ALB by running this command (replace the ARN placeholders with the values from the previous outputs):

aws wafv2 associate-web-acl \
    --web-acl-arn [[WAF_ACL_ARN="your WebACL ARN from the create-web-acl output"]] \
    --resource-arn [[ALB_ARN="your ALB ARN from the describe-load-balancers output"]] \
    --region us-east-1 \
    --profile cloudshop

• Verify the association succeeded by running:

aws wafv2 get-web-acl-for-resource \
    --resource-arn [[ALB_ARN="your ALB ARN"]] \
    --region us-east-1 \
    --profile cloudshop

You should see the CloudShop-WAF WebACL details in the output, confirming it is attached to your ALB.

Not seeing the WebACL in the output?

• Confirm you used the correct ALB ARN (it starts with arn:aws:elasticloadbalancing:).
• Make sure the WebACL scope is REGIONAL. A CLOUDFRONT scope WebACL cannot attach to an ALB.
• Check that both resources are in the same region (us-east-1).

Why isn't my WAF WebACL associating with my ALB?

Configure Parameter Store and connect via Session Manager

Systems Manager Parameter Store provides a central location for application configuration. Using SecureString parameters encrypted with your KMS key keeps sensitive values out of code and environment variables.

• Create a SecureString parameter for a feature flag by running:

aws ssm put-parameter \
    --name "/cloudshop/config/feature-new-checkout" \
    --value "enabled" \
    --type SecureString \
    --key-id alias/CloudShop-CMK \
    --profile cloudshop \
    --region us-east-1

• Create a SecureString parameter for an API endpoint by running:

aws ssm put-parameter \
    --name "/cloudshop/config/api-endpoint" \
    --value "https://api.cloudshop.internal/v1" \
    --type SecureString \
    --key-id alias/CloudShop-CMK \
    --profile cloudshop \
    --region us-east-1

Why SecureString over plain String?

SecureString parameters are encrypted at rest with your KMS key. Applications decrypt them at runtime using IAM permissions. This prevents accidental exposure in logs, console screenshots, or CloudFormation outputs.

The --key-id alias/CloudShop-CMK flag encrypts with your customer-managed key rather than the default AWS-managed aws/ssm key. Your customer-managed key gives you full control over rotation and access policies.

• Verify your EC2 instances are registered as SSM managed instances by running:

aws ssm describe-instance-information \
    --profile cloudshop \
    --region us-east-1

You should see your EC2 instances listed with PingStatus of Online. This confirms the SSM Agent is running and your instance profile grants the required permissions.

• Connect to one of your private instances using Session Manager by running (replace the instance ID with one from the previous output):

aws ssm start-session \
    --target [[INSTANCE_ID="your instance ID from describe-instance-information"]] \
    --profile cloudshop \
    --region us-east-1

You should see a shell prompt from the remote instance. Type exit to close the session.

Why Session Manager instead of SSH?

Session Manager eliminates the need for open inbound ports, bastion hosts, or SSH key management. All sessions are logged to CloudWatch and S3 for audit purposes.

The connection tunnels through the SSM Agent already running on the instance. Your private instances never need a public IP or port 22 open in the security group.

Session fails to start?

• Confirm your instance profile includes the AmazonSSMManagedInstanceCore policy (or equivalent permissions).
• Make sure the SSM Agent is running. Amazon Linux 2023 AMI comes with SSM Agent pre-installed.
• Check that your private subnet route table has a route to the NAT Gateway so the agent can reach the SSM endpoint.

Why can't I start a Session Manager session to my private EC2 instance?

Run Automation, review Trusted Advisor, and update ADR

Systems Manager Automation provides pre-built runbooks for common operations tasks. Patching instances through Automation is auditable and avoids manual SSH sessions.

• Run the AWS-managed patch automation on your instances by running:

aws ssm start-automation-execution \
    --document-name "AWS-RunPatchBaseline" \
    --parameters '{"Operation":["Scan"]}' \
    --profile cloudshop \
    --region us-east-1

What does this automation do?

The AWS-RunPatchBaseline document scans instances against the default patch baseline for their operating system. Using Scan mode reports missing patches without installing them. Change the operation to Install when you are ready to apply patches.

• In the AWS Management Console, navigate to the Trusted Advisor dashboard.
• Review the Security and Cost Optimization categories.
• Note any recommendations related to your CloudShop resources (open security groups, idle resources, or underutilized instances).

What is Trusted Advisor?

Trusted Advisor inspects your AWS environment and provides recommendations across five categories: cost optimization, performance, security, fault tolerance, and service limits. Some checks require a Business or Enterprise support plan.

• Update your ADR with the security hardening decisions by running:

cat >> ~/cloudshop/ADR.md << 'EOF'

## ADR-019: Defense in Depth Security
- **Decision**: WAF on ALB + KMS encryption + IAM least privilege + Secrets Manager + SSM (no SSH)
- **Context**: Multiple security layers; WAF at edge, encryption at rest/transit, no direct SSH access
- **GCP Equivalent**: GCP Cloud Armor (WAF), GCP OS Config (SSM Patch Manager), GCP IAP (Session Manager)
- **SAA-C03 Exam Relevance**: SAA Domain 1 (Design Secure Architectures), WAF rule priorities, SSM vs SSH, defense in depth
- **Alternatives Considered**: Security Groups only (rejected: no L7 protection), SSH with bastion (rejected: attack surface)
- **Outcome**: WAF + SSM Session Manager eliminates SSH exposure; managed rules cover OWASP Top 10
EOF

What does this ADR entry capture?

This records the defense in depth strategy you implemented. WAF provides Layer 7 filtering, KMS handles encryption, IAM enforces least privilege, and SSM replaces SSH entirely. The GCP equivalents help you map these patterns to your existing knowledge.

For the SAA-C03 exam, understand that Domain 1 (Design Secure Architectures) accounts for 30% of the exam. WAF rule priority ordering, managed rule groups, and the SSM vs bastion tradeoff are common exam topics.

• Verify the ADR was updated by running:

tail -10 ~/cloudshop/ADR.md

You should see the ADR-019 entry with the defense in depth security decisions.

✔️ Awesome, I've got everything!

Great. Double check that you have saved all files in your ~/cloudshop directory.

ⓧ I'd like to double check the full ADR entry

Here is the complete ADR-019 entry that should appear at the end of your ADR.md file:

## ADR-019: Defense in Depth Security
- **Decision**: WAF on ALB + KMS encryption + IAM least privilege + Secrets Manager + SSM (no SSH)
- **Context**: Multiple security layers; WAF at edge, encryption at rest/transit, no direct SSH access
- **GCP Equivalent**: GCP Cloud Armor (WAF), GCP OS Config (SSM Patch Manager), GCP IAP (Session Manager)
- **SAA-C03 Exam Relevance**: SAA Domain 1 (Design Secure Architectures), WAF rule priorities, SSM vs SSH, defense in depth
- **Alternatives Considered**: Security Groups only (rejected: no L7 protection), SSH with bastion (rejected: attack surface)
- **Outcome**: WAF + SSM Session Manager eliminates SSH exposure; managed rules cover OWASP Top 10

Your CloudShop infrastructure now has multiple security layers working together. Next up, you will codify this entire architecture in CloudFormation templates for repeatable deployment.

Codify Infrastructure with CloudFormation

You've built, secured, and monitored an entire multi-tier architecture by hand. Every subnet, every security group, every scaling policy exists because you clicked through the console or ran individual CLI commands.

The problem with manual infrastructure is repeatability. If you needed to recreate this environment in another region or recover from a disaster, you'd spend hours retracing your steps. CloudFormation solves this by letting you declare your entire infrastructure in YAML templates that AWS can provision automatically.

In this step, get ready to:

• Write a networking stack template defining your VPC, subnets, gateways, and security groups.
• Write an application stack template defining your ALB, Launch Template, and Auto Scaling Group.
• Deploy both stacks via the AWS CLI and verify they match your manual infrastructure.

Write the networking stack CloudFormation template

A well-designed Infrastructure as Code project separates concerns into multiple stacks. Networking resources (VPC, subnets, route tables) change infrequently compared to application resources (instances, scaling policies). Splitting them into two stacks means you can update one without risking the other.

The networking stack will also export key resource IDs (like the VPC ID and subnet IDs) so the application stack can reference them without hardcoding values.

• Create the networking template file by running this command in your ~/cloudshop directory:

touch cloudshop-network.yaml

• Open cloudshop-network.yaml in your editor and paste the following template:

AWSTemplateFormatVersion: '2010-09-09'
Description: CloudShop Networking Stack
Parameters:
  EnvironmentName:
    Type: String
    Default: cloudshop
  VpcCIDR:
    Type: String
    Default: 10.0.0.0/16
  PublicSubnet1CIDR:
    Type: String
    Default: 10.0.1.0/24
  PublicSubnet2CIDR:
    Type: String
    Default: 10.0.2.0/24
  PrivateAppSubnet1CIDR:
    Type: String
    Default: 10.0.3.0/24
  PrivateAppSubnet2CIDR:
    Type: String
    Default: 10.0.4.0/24
  PrivateDataSubnet1CIDR:
    Type: String
    Default: 10.0.5.0/24
  PrivateDataSubnet2CIDR:
    Type: String
    Default: 10.0.6.0/24
Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VpcCIDR
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName}-VPC
  InternetGateway:
    Type: AWS::EC2::InternetGateway
  VPCGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref VPC
      InternetGatewayId: !Ref InternetGateway
  PublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: !Ref PublicSubnet1CIDR
      AvailabilityZone: !Select [0, !GetAZs '']
      MapPublicIpOnLaunch: true
  PublicSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: !Ref PublicSubnet2CIDR
      AvailabilityZone: !Select [1, !GetAZs '']
      MapPublicIpOnLaunch: true
  # ... additional subnet, route table, NAT Gateway, security group resources
Outputs:
  VPCId:
    Value: !Ref VPC
    Export:
      Name: !Sub ${EnvironmentName}-VPCId
  PublicSubnet1Id:
    Value: !Ref PublicSubnet1
    Export:
      Name: !Sub ${EnvironmentName}-PublicSubnet1Id
  PublicSubnet2Id:
    Value: !Ref PublicSubnet2
    Export:
      Name: !Sub ${EnvironmentName}-PublicSubnet2Id

What does this template do?

The Parameters section makes the template reusable. You can deploy the same template with different CIDR ranges or environment names without editing the file itself.

The Resources section declares the VPC, Internet Gateway, and public subnets. Each resource uses !Ref and !Sub intrinsic functions to wire parameters into resource properties dynamically.

The Outputs section with Export makes resource IDs available to other stacks via cross-stack references. This is how the application stack will find the VPC and subnets without hardcoding IDs.

• Save cloudshop-network.yaml.
• Confirm the file exists by running:

ls cloudshop-network.yaml

You should see cloudshop-network.yaml listed in the output.

File not found?

Make sure you're in the ~/cloudshop directory. Run cd ~/cloudshop and try again.

Help me find my file

Write the application stack CloudFormation template

The application stack defines the compute layer: the ALB, Launch Template, Auto Scaling Group, and target group. It consumes the networking stack's exports using Fn::ImportValue to place resources in the correct subnets and VPC.

• Create the application template file by running:

touch cloudshop-app.yaml

• Open cloudshop-app.yaml in your editor and paste the following template:

AWSTemplateFormatVersion: '2010-09-09'
Description: CloudShop Application Stack
Parameters:
  EnvironmentName:
    Type: String
    Default: cloudshop
  InstanceType:
    Type: String
    Default: t3.micro
Resources:
  ALB:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Name: !Sub ${EnvironmentName}-ALB
      Subnets:
        - Fn::ImportValue: !Sub ${EnvironmentName}-PublicSubnet1Id
        - Fn::ImportValue: !Sub ${EnvironmentName}-PublicSubnet2Id
      SecurityGroups:
        - Fn::ImportValue: !Sub ${EnvironmentName}-ALBSecurityGroupId
  AppTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: !Sub ${EnvironmentName}-AppTG
      VpcId:
        Fn::ImportValue: !Sub ${EnvironmentName}-VPCId
      Port: 80
      Protocol: HTTP
      HealthCheckPath: /health
      HealthCheckIntervalSeconds: 30
      HealthyThresholdCount: 2
  LaunchTemplate:
    Type: AWS::EC2::LaunchTemplate
    Properties:
      LaunchTemplateName: !Sub ${EnvironmentName}-LaunchTemplate
      LaunchTemplateData:
        InstanceType: !Ref InstanceType
        ImageId: '{{resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64}}'
  AutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      LaunchTemplate:
        LaunchTemplateId: !Ref LaunchTemplate
        Version: !GetAtt LaunchTemplate.LatestVersionNumber
      MinSize: 1
      MaxSize: 4
      DesiredCapacity: 2
      TargetGroupARNs:
        - !Ref AppTargetGroup
      VPCZoneIdentifier:
        - Fn::ImportValue: !Sub ${EnvironmentName}-PrivateAppSubnet1Id
        - Fn::ImportValue: !Sub ${EnvironmentName}-PrivateAppSubnet2Id

How does Fn::ImportValue work?

When CloudFormation sees Fn::ImportValue: !Sub ${EnvironmentName}-VPCId, it looks for a stack that exported a value with that exact name. The networking stack's Outputs section exports cloudshop-VPCId, which resolves to the actual VPC ID at deploy time.

This pattern enforces deployment order. You must deploy the networking stack first so its exports exist before the application stack tries to import them.

What is the resolve:ssm pattern?

The ImageId uses a dynamic reference to Systems Manager Parameter Store. AWS maintains the path /aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64 with the latest Amazon Linux 2023 AMI ID for the region. This means your template always launches the newest patched AMI without you updating the ID manually.

• Save cloudshop-app.yaml.
• Confirm both template files exist by running:

ls cloudshop-network.yaml cloudshop-app.yaml

You should see both files listed.

One file missing?

Check that you saved both files in the same ~/cloudshop directory. If your editor saved to a different location, move the file with mv.

Help me locate my missing file

✔️ Awesome, I've got everything!

Great. Double check you've saved both files before moving on.

ⓧ I'd like to double check the full code

Here is the full cloudshop-network.yaml:

AWSTemplateFormatVersion: '2010-09-09'
Description: CloudShop Networking Stack
Parameters:
  EnvironmentName:
    Type: String
    Default: cloudshop
  VpcCIDR:
    Type: String
    Default: 10.0.0.0/16
  PublicSubnet1CIDR:
    Type: String
    Default: 10.0.1.0/24
  PublicSubnet2CIDR:
    Type: String
    Default: 10.0.2.0/24
  PrivateAppSubnet1CIDR:
    Type: String
    Default: 10.0.3.0/24
  PrivateAppSubnet2CIDR:
    Type: String
    Default: 10.0.4.0/24
  PrivateDataSubnet1CIDR:
    Type: String
    Default: 10.0.5.0/24
  PrivateDataSubnet2CIDR:
    Type: String
    Default: 10.0.6.0/24
Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VpcCIDR
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName}-VPC
  InternetGateway:
    Type: AWS::EC2::InternetGateway
  VPCGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      VpcId: !Ref VPC
      InternetGatewayId: !Ref InternetGateway
  PublicSubnet1:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: !Ref PublicSubnet1CIDR
      AvailabilityZone: !Select [0, !GetAZs '']
      MapPublicIpOnLaunch: true
  PublicSubnet2:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: !Ref PublicSubnet2CIDR
      AvailabilityZone: !Select [1, !GetAZs '']
      MapPublicIpOnLaunch: true
  # ... additional subnet, route table, NAT Gateway, security group resources
Outputs:
  VPCId:
    Value: !Ref VPC
    Export:
      Name: !Sub ${EnvironmentName}-VPCId
  PublicSubnet1Id:
    Value: !Ref PublicSubnet1
    Export:
      Name: !Sub ${EnvironmentName}-PublicSubnet1Id
  PublicSubnet2Id:
    Value: !Ref PublicSubnet2
    Export:
      Name: !Sub ${EnvironmentName}-PublicSubnet2Id

And here is the full cloudshop-app.yaml:

AWSTemplateFormatVersion: '2010-09-09'
Description: CloudShop Application Stack
Parameters:
  EnvironmentName:
    Type: String
    Default: cloudshop
  InstanceType:
    Type: String
    Default: t3.micro
Resources:
  ALB:
    Type: AWS::ElasticLoadBalancingV2::LoadBalancer
    Properties:
      Name: !Sub ${EnvironmentName}-ALB
      Subnets:
        - Fn::ImportValue: !Sub ${EnvironmentName}-PublicSubnet1Id
        - Fn::ImportValue: !Sub ${EnvironmentName}-PublicSubnet2Id
      SecurityGroups:
        - Fn::ImportValue: !Sub ${EnvironmentName}-ALBSecurityGroupId
  AppTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: !Sub ${EnvironmentName}-AppTG
      VpcId:
        Fn::ImportValue: !Sub ${EnvironmentName}-VPCId
      Port: 80
      Protocol: HTTP
      HealthCheckPath: /health
      HealthCheckIntervalSeconds: 30
      HealthyThresholdCount: 2
  LaunchTemplate:
    Type: AWS::EC2::LaunchTemplate
    Properties:
      LaunchTemplateName: !Sub ${EnvironmentName}-LaunchTemplate
      LaunchTemplateData:
        InstanceType: !Ref InstanceType
        ImageId: '{{resolve:ssm:/aws/service/ami-amazon-linux-latest/al2023-ami-kernel-default-x86_64}}'
  AutoScalingGroup:
    Type: AWS::AutoScaling::AutoScalingGroup
    Properties:
      LaunchTemplate:
        LaunchTemplateId: !Ref LaunchTemplate
        Version: !GetAtt LaunchTemplate.LatestVersionNumber
      MinSize: 1
      MaxSize: 4
      DesiredCapacity: 2
      TargetGroupARNs:
        - !Ref AppTargetGroup
      VPCZoneIdentifier:
        - Fn::ImportValue: !Sub ${EnvironmentName}-PrivateAppSubnet1Id
        - Fn::ImportValue: !Sub ${EnvironmentName}-PrivateAppSubnet2Id

Deploy both stacks and update the ADR

With both templates written, you can deploy them using the AWS CLI. The networking stack must be deployed first because the application stack depends on its exports.

• Deploy the networking stack by running this command:

aws cloudformation create-stack \
  --stack-name cloudshop-network \
  --template-body file://cloudshop-network.yaml \
  --profile cloudshop

What does this command do?

• --stack-name cloudshop-network names the stack so you can reference it later.
• --template-body file://cloudshop-network.yaml reads the template from your local file. The file:// prefix tells the CLI to load the file contents.
• --profile cloudshop uses your project-specific AWS credentials.

• Wait for the networking stack to finish creating. Check its status by running:

aws cloudformation describe-stacks \
  --stack-name cloudshop-network \
  --query "Stacks[0].StackStatus" \
  --output text \
  --profile cloudshop

You should see CREATE_COMPLETE. If you see CREATE_IN_PROGRESS, wait a minute and run the command again.

• Once the networking stack shows CREATE_COMPLETE, deploy the application stack by running:

aws cloudformation create-stack \
  --stack-name cloudshop-app \
  --template-body file://cloudshop-app.yaml \
  --profile cloudshop

• Check the application stack status by running:

aws cloudformation describe-stacks \
  --stack-name cloudshop-app \
  --query "Stacks[0].StackStatus" \
  --output text \
  --profile cloudshop

You should see CREATE_COMPLETE once both stacks finish provisioning.

Seeing CREATE_FAILED or ROLLBACK_COMPLETE?

Run the following to see which resource failed and why:

aws cloudformation describe-stack-events \
  --stack-name cloudshop-network \
  --profile cloudshop

Common causes include a resource name conflict with your existing manually-created resources (the stack tries to create a resource with a name that already exists) or insufficient IAM permissions.

Help me debug my failed stack

• Update your ADR with the IaC decision and final architecture summary. Open ADR.md and add the following at the end:

## ADR-020: Infrastructure as Code with CloudFormation
- **Decision**: Two-stack CloudFormation approach: networking stack + application stack with cross-stack references
- **Context**: Separation of concerns; networking changes less frequently than application; cross-stack references via Fn::ImportValue
- **GCP Equivalent**: GCP Deployment Manager or Terraform (Terraform is cloud-agnostic)
- **SAA-C03 Exam Relevance**: CloudFormation stack dependencies, nested stacks vs cross-stack references, cfn-init, drift detection
- **Alternatives Considered**: Single monolithic stack (rejected: too large, slow updates), Terraform (valid but CloudFormation is AWS-native and SAA in-scope)
- **Outcome**: Two stacks with exported outputs; demonstrates IaC best practices

## Final Architecture: 30+ Services Summary
VPC, 6 Subnets, IGW, NAT Gateway, NACLs, Security Groups, VPC Endpoint, IAM Roles, Instance Profile, KMS CMK, Secrets Manager, S3 (3 buckets), EFS, RDS MySQL, DynamoDB, ElastiCache Redis, ALB, EC2 Launch Template, Auto Scaling Group, CloudFront, Route 53, SQS + DLQ, SNS, Lambda (3 functions), Step Functions, API Gateway, Kinesis Firehose, Glue, Athena, CloudTrail, AWS Config, VPC Flow Logs, CloudWatch, WAF, SSM Parameter Store, CloudFormation (2 stacks)

Why CloudFormation over Terraform for this project?

Both are valid IaC tools. Terraform is cloud-agnostic and widely used in industry. However, CloudFormation is AWS-native and explicitly listed as an in-scope service on the SAA-C03 exam. Understanding stack dependencies, cross-stack references, and drift detection maps directly to exam questions.

On the job, the choice depends on whether your organization is multi-cloud (Terraform) or AWS-only (CloudFormation). Knowing both patterns makes you a stronger candidate.

• Save ADR.md.

Your entire CloudShop infrastructure is now codified. Next up, you'll review the cost of everything you've built and learn optimization strategies that map directly to SAA-C03 Domain 4.

Review Cost Optimization

Your infrastructure is fully codified in CloudFormation templates. You can recreate the entire CloudShop environment from scratch with two CLI commands.

Now it is time to look at the other side of the coin: how much does all of this cost? AWS charges for many services by the hour, and leaving resources running without understanding the spend adds up quickly. In this step, you will analyze your actual costs, set up a budget alert as a safety net, and document optimization recommendations that map directly to SAA-C03 Domain 4.

In this step, get ready to:

• Review actual spend in Cost Explorer and identify the most expensive services.
• Create a budget with an alert threshold to prevent surprise bills.
• Evaluate cost optimization patterns and document them in your ADR.

Analyze spend in Cost Explorer and create a budget

Cost Explorer visualizes your AWS costs over time. It shows which services consume the most money and forecasts future spend based on your usage patterns.

• In the AWS Management Console, open the Billing and Cost Management console at https://console.aws.amazon.com/costmanagement/.
• In the navigation pane, choose Cost Explorer.
• Choose Cost & Usage to open the report view.
• Set the time range to the current month and group by Service.
• Review the chart to identify your top cost drivers. Note which services appear as the largest bars.

What should I see?

Your top cost drivers will likely be the NAT Gateway ($0.045/hr) and the Application Load Balancer ($0.0225/hr). EC2, RDS, and ElastiCache follow depending on how long your infrastructure has been running.

If your account is new (under 12 months), some services like RDS and ElastiCache may show $0 due to free tier coverage. The NAT Gateway has no free tier and will always incur charges.

With your spend data in hand, a budget alert acts as a safety net against surprise bills.

• In the navigation pane, choose Budgets.
• At the top of the page, choose Create budget.
• Under Budget setup, choose Customize (advanced).
• Under Budget types, choose Cost budget.
• Choose Next.
• Enter CloudShop-Budget as the budget name.
• Set the budgeted amount to 25 (dollars per month).
• Choose Next.
• Add an alert threshold at 80% of the budgeted amount ($20). Enter your email address for the notification.
• Choose Next, review your settings, then choose Create budget.

Why set the alert at 80%?

An 80% threshold ($20 of a $25 budget) gives you a warning before you hit the limit. This lets you decide whether to tear down resources or keep going.

AWS Budgets updates up to three times a day, so the alert fires within hours of crossing the threshold.

Finally, review the purchasing recommendations that AWS generates based on your usage history.

• Return to Cost Explorer in the left navigation pane.
• In the left pane, look for the reservation recommendations section. Review the Reserved Instance recommendations for RDS and EC2.
• Review the Savings Plans recommendations for consistent compute usage.
• Note the projected savings percentages. Do NOT purchase anything. This is for exam context only.

Reserved Instances vs Savings Plans

Reserved Instances commit you to a specific instance type in a specific region for 1 or 3 years (up to 72% savings). Savings Plans commit you to a dollar-per-hour spend level for 1 or 3 years but allow flexibility across instance families (up to 66% savings).

For the SAA-C03 exam, know that Reserved Instances offer the deepest discount for predictable workloads, while Savings Plans offer more flexibility. Both are covered in Domain 4.

Cost Explorer showing no data?

Cost Explorer can take up to 24 hours to populate data after first being enabled. If you see blank charts, wait and return later. Your budget will still be created and active.

Why is my Cost Explorer empty even though I have resources running?

Review architecture for cost optimization opportunities

Beyond monitoring spend, a Solutions Architect identifies structural changes that reduce costs without sacrificing availability. This is the core of SAA-C03 Domain 4 (Design Cost-Optimized Architectures, 20% of the exam).

• Open the Compute Optimizer console at https://console.aws.amazon.com/compute-optimizer/.
• Choose EC2 instances in the navigation pane.
• Review the finding classifications for your CloudShop instances. Note whether they are classified as over-provisioned, under-provisioned, or optimized.
• Choose EC2 Auto Scaling groups in the navigation pane to see recommendations for your ASG.

What if Compute Optimizer shows no recommendations?

Compute Optimizer needs at least 30 hours of metric data to generate recommendations. If your infrastructure is new, it may show "insufficient data" for your instances. The recommendations will appear after the look-back period.

Now evaluate the structural cost optimization patterns in your architecture.

• Identify Spot Instance candidates: your Auto Scaling Group runs non-critical web servers. In production, you could set a mixed instances policy with Spot Instances for up to 90% cost savings on interruptible capacity.
• Evaluate the NAT Gateway cost: your CloudShop uses one NAT Gateway at $0.045/hr ($32/month). For traffic to S3, the VPC Gateway Endpoint you created eliminates data processing charges entirely. For production, evaluate whether additional VPC endpoints for other services (DynamoDB, SQS) could reduce NAT Gateway data processing costs.
• Review right-sizing: your t3.micro instances are the smallest general-purpose option. In production with real traffic, Compute Optimizer would recommend the correct size based on actual CPU and memory utilization.

Why one NAT Gateway instead of two?

A production architecture places one NAT Gateway per Availability Zone for high availability. If the AZ hosting a single NAT Gateway goes down, private subnets in the other AZ lose internet access.

For this lab, a single NAT Gateway saves $32/month. In production, you would accept the additional $32/month for resilience. The SAA-C03 exam tests this tradeoff: cost savings vs availability in multi-AZ designs.

Document cost optimization in ADR-021

Your Architecture Decision Record captures every design choice. This final entry documents the cost implications of your architecture and what you would change for a production deployment.

• Open ADR.md in your editor.
• Add the following ADR-021 entry at the end of the file:

## ADR-021: Cost Optimization Strategy
- **Decision**: Single NAT Gateway, t3.micro instances, on-demand DynamoDB, single-AZ ElastiCache for lab
- **Context**: Lab environment prioritizes learning over production resilience; cost ~$3-5/day
- **GCP Equivalent**: GCP Committed Use Discounts (Reserved Instances), Preemptible VMs (Spot Instances)
- **SAA-C03 Exam Relevance**: SAA Domain 4 (Cost-Optimized Architectures), Reserved vs Spot vs On-Demand, S3 Intelligent-Tiering, right-sizing
- **Production Recommendations**:
  - Reserved Instances for RDS (1-year, ~40% savings)
  - Savings Plans for consistent EC2 compute (~30% savings)
  - Two NAT Gateways for HA (accept cost for production)
  - S3 Intelligent-Tiering for assets with unknown access patterns
  - Spot Instances for ASG non-critical capacity (up to 90% savings)
- **Cost Breakdown (approximate)**:
  - NAT Gateway: $0.045/hr = ~$32/month
  - ALB: $0.0225/hr = ~$16/month
  - RDS t3.micro: ~$15/month
  - ElastiCache t3.micro: ~$12/month
  - EC2 t3.micro x2: ~$15/month
  - Total: ~$90-100/month (lab teardown after use reduces to ~$15-20 for 2-3 days)

What does this entry capture?

The cost breakdown shows the hourly rates from AWS pricing (NAT Gateway at $0.045/hr, ALB at $0.0225/hr) converted to monthly totals. The production recommendations list what you would change if this were a real e-commerce platform: commit to Reserved Instances for always-on databases, add Spot Instances for burst capacity, and duplicate the NAT Gateway for availability.

The GCP equivalents (Committed Use Discounts, Preemptible VMs) help you map these AWS cost patterns to concepts you already know from your GCP PCA certification.

• Save ADR.md.
• Verify the entry by scrolling to the bottom of the file. You should see ADR-021 with six sections: Decision, Context, GCP Equivalent, SAA-C03 Exam Relevance, Production Recommendations, and Cost Breakdown.

Numbers don't match your Cost Explorer?

The cost breakdown in ADR-021 shows approximate monthly rates for a continuously running infrastructure. Your actual costs will be lower because you are tearing down resources after a few days. The ADR documents what the architecture would cost at steady state for interview discussions.

Help me understand the difference between my actual bill and the ADR estimates.

✔️ Awesome, I've got everything!

Great. Double check you have saved ADR.md with the new ADR-021 entry.

ⓧ I'd like to double check the full code

Here is the complete ADR-021 entry that should appear at the end of your ADR.md file:

## ADR-021: Cost Optimization Strategy
- **Decision**: Single NAT Gateway, t3.micro instances, on-demand DynamoDB, single-AZ ElastiCache for lab
- **Context**: Lab environment prioritizes learning over production resilience; cost ~$3-5/day
- **GCP Equivalent**: GCP Committed Use Discounts (Reserved Instances), Preemptible VMs (Spot Instances)
- **SAA-C03 Exam Relevance**: SAA Domain 4 (Cost-Optimized Architectures), Reserved vs Spot vs On-Demand, S3 Intelligent-Tiering, right-sizing
- **Production Recommendations**:
  - Reserved Instances for RDS (1-year, ~40% savings)
  - Savings Plans for consistent EC2 compute (~30% savings)
  - Two NAT Gateways for HA (accept cost for production)
  - S3 Intelligent-Tiering for assets with unknown access patterns
  - Spot Instances for ASG non-critical capacity (up to 90% savings)
- **Cost Breakdown (approximate)**:
  - NAT Gateway: $0.045/hr = ~$32/month
  - ALB: $0.0225/hr = ~$16/month
  - RDS t3.micro: ~$15/month
  - ElastiCache t3.micro: ~$12/month
  - EC2 t3.micro x2: ~$15/month
  - Total: ~$90-100/month (lab teardown after use reduces to ~$15-20 for 2-3 days)

You now have a complete cost optimization review documented in your ADR. This entry maps directly to SAA-C03 Domain 4 exam questions about choosing the right purchasing option, right-sizing compute, and evaluating cost tradeoffs in multi-AZ architectures.

Your CloudShop infrastructure is complete. Next up, tackle the Secret Mission to extend your architecture with an ECS Fargate microservice and automated budget controls.

Secret mission

Add ECS Fargate Microservice and Budget Alerts

Your CloudShop infrastructure runs entirely on EC2 instances managed by Auto Scaling. In production, modern architectures mix compute types to match workload characteristics. Can you add a containerized microservice on ECS Fargate and wire up automated cost controls that stop instances when your budget is exceeded?

Clean Up Your Resources

Clean Up Your Resources

This project creates AWS resources that cost money while provisioned. The primary cost drivers are the NAT Gateway ($0.045/hr) and the Application Load Balancer ($0.0225/hr). Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Cost warning

If you leave all resources running, expect to spend approximately $3-5 per day. The NAT Gateway alone costs about $1.08/day, and the ALB costs about $0.54/day. ElastiCache also incurs charges if you are no longer within free tier eligibility.

If you are not actively building on this project, choose Pause or Delete to avoid unexpected charges on your AWS bill.

Resources you used:

• CloudFormation stacks (cloudshop-network, cloudshop-app).
• VPC, subnets, Internet Gateway, NAT Gateway, Elastic IP, route tables, NACLs, security groups, VPC Gateway Endpoint.
• Application Load Balancer, target groups, Auto Scaling Group, Launch Template, EC2 instances.
• RDS MySQL instance, DynamoDB table (CloudShop-Sessions), ElastiCache Redis cluster.
• EFS file system.
• S3 buckets (static assets, CloudTrail logs, analytics data).
• CloudFront distribution.
• Route 53 hosted zone and records.
• Lambda functions (ProcessOrder, ResizeImage, analytics simulator).
• SQS queues (CloudShop-Orders, dead letter queue), SNS topic (CloudShop-Notifications).
• Step Functions state machine (OrderWorkflow), API Gateway REST API.
• Kinesis Data Firehose delivery stream, Glue database and table, Athena named queries.
• CloudWatch dashboard, alarms, log groups. CloudTrail trail. AWS Config rules. VPC Flow Logs.
• WAF WebACL. Systems Manager Parameter Store values.
• IAM roles, policies, instance profile. KMS customer-managed key. Secrets Manager secret.
• ECS cluster (CloudShop-ECS), Fargate service, task definition. AWS Budgets.
• Local project directory (~/cloudshop).

Keep everything running

No action needed. Choose this if you are still actively building or experimenting with the infrastructure over the next few days.

• Your resources will continue running and incurring charges (approximately $3-5/day).
• Monitor your spend in the AWS Cost Explorer or via the Budget alert you configured at $20.
• Your local project directory at ~/cloudshop remains on your machine with all CloudFormation templates and the ADR document.

Pause - I'll come back to this later

Shut down running compute instances to reduce charges. Note that NAT Gateway and ALB still incur hourly charges while provisioned.

• Stop your EC2 instances from the EC2 console. Select each instance, then choose Instance state > Stop instance.
• Set your Auto Scaling Group desired and minimum capacity to 0 so stopped instances are not replaced.
• The NAT Gateway ($0.045/hr) and ALB ($0.0225/hr) will continue to charge while provisioned. Consider deleting these if you will be away for more than a day or two.
• Your RDS instance, ElastiCache cluster, and ECS Fargate service will also continue charging while active.
• Your local ~/cloudshop directory remains intact for when you return.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow the steps below in order, as some resources depend on others being removed first.

Delete CloudFormation stacks first

If you deployed the CloudFormation stacks in Step 12, delete them first. They will remove the networking and application resources they manage.

• Delete the application stack by running:
• Wait for that deletion to complete before deleting the network stack.

aws cloudformation delete-stack --stack-name cloudshop-app --profile cloudshop
aws cloudformation delete-stack --stack-name cloudshop-network --profile cloudshop

Delete ECS resources (Secret Mission)

• In the ECS console, navigate to Clusters > CloudShop-ECS.
• Select the CloudShop-RecommendationsService service and choose Delete service.
• After the service is deleted, select the cluster and choose Delete cluster.
• Navigate to Task definitions and deregister the cloudshop-recommendations task definition.

Delete ElastiCache cluster

• Open the ElastiCache console and select your Redis cluster.
• Choose Actions > Delete and confirm.

Delete RDS instance

• Delete the RDS MySQL instance (skip final snapshot for a lab environment) by running:

aws rds delete-db-instance \
    --db-instance-identifier cloudshop-db \
    --skip-final-snapshot \
    --profile cloudshop

Delete NAT Gateway and release Elastic IP

• In the VPC console, navigate to NAT gateways.
• Select your NAT Gateway, choose Actions > Delete NAT gateway, and confirm.
• Navigate to Elastic IPs, select the Elastic IP you allocated for the NAT Gateway, and choose Actions > Release Elastic IP addresses.

Delete ALB, target groups, and Auto Scaling Group

• In the EC2 console, navigate to Load Balancers. Select your ALB and choose Actions > Delete load balancer.
• Navigate to Target Groups and delete all CloudShop target groups.
• Navigate to Auto Scaling Groups, select your ASG, and choose Delete. This terminates associated EC2 instances.
• Delete the Launch Template from the Launch Templates section.

Delete S3 buckets

• Empty each S3 bucket first (you cannot delete a bucket with objects in it). In the S3 console, select the bucket, choose Empty, and confirm.
• After emptying, select the bucket and choose Delete.
• Repeat for all three buckets (static assets, CloudTrail logs, analytics data).

Delete CloudFront distribution

• In the CloudFront console, select your distribution and choose Disable.
• Wait approximately 15 minutes for the distribution to finish disabling.
• Once the status shows Disabled, select it and choose Delete.

Delete Route 53 records and hosted zone

• In the Route 53 console, navigate to your hosted zone.
• Delete all records except the default NS and SOA records.
• Delete the health check.
• Delete the hosted zone itself.

Delete serverless and event-driven resources

• Delete the API Gateway REST API from the API Gateway console.
• Delete the Step Functions state machine (OrderWorkflow) from the Step Functions console.
• Delete all Lambda functions (ProcessOrder, ResizeImage, and the analytics simulator) from the Lambda console.
• Delete the SQS queues (CloudShop-Orders and its dead letter queue) from the SQS console.
• Delete the SNS topic (CloudShop-Notifications) from the SNS console.
• Delete the Kinesis Data Firehose delivery stream from the Firehose console.
• Delete the Glue database from the Glue console.

Delete monitoring and compliance resources

• Delete the CloudWatch dashboard and all CloudWatch alarms from the CloudWatch console.
• Delete the CloudTrail trail from the CloudTrail console.
• Delete the AWS Config rules from the Config console.
• Delete VPC Flow Logs from the VPC console (under Your VPCs > Flow logs tab).
• Delete the CloudWatch Log Groups created for Flow Logs and application logs.

Delete security resources

• Delete the WAF WebACL from the WAF console (disassociate from the ALB first if it still exists).
• Delete the Secrets Manager secret from the Secrets Manager console (choose Delete secret and set the waiting period to the minimum).
• Delete the Systems Manager Parameter Store values from the Systems Manager console.
• Schedule the KMS key for deletion (minimum 7-day waiting period) from the KMS console.
• Delete the IAM roles, policies, and instance profile from the IAM console.

Delete EFS and DynamoDB

• Delete the EFS file system from the EFS console (delete mount targets first).
• Delete the DynamoDB table (CloudShop-Sessions) from the DynamoDB console.

Delete VPC and networking

• In the VPC console, select your CloudShop VPC and choose Actions > Delete VPC. This deletes the VPC and its associated subnets, route tables, internet gateway, network ACLs, and security groups.
• If the VPC deletion fails, manually delete any remaining ENIs (Elastic Network Interfaces) that may be attached to resources you already deleted, then retry.

Delete AWS Budgets

• In the AWS Billing console, navigate to Budgets.
• Select your CloudShop budget and choose Delete.

Verify with Resource Groups Tag Editor

• Open the AWS Resource Groups Tag Editor in the console.
• Search for any resources tagged with your project tags to confirm nothing was missed.

Delete local project files

• Remove the local project directory by running:

rm -rf ~/cloudshop

That's a Wrap!

That's a Wrap!

Congratulations! You've built a complete production-grade AWS infrastructure from the ground up. CloudShop is now a fully documented, multi-tier e-commerce architecture spanning 30+ SAA-C03 services wired together as one cohesive system.

You've learned how to:

• Architect a multi-AZ VPC with public and private subnets, deploy a highly available compute tier with ALB and Auto Scaling, and configure relational and NoSQL databases with ElastiCache caching.
• Build a serverless event-driven pipeline with Lambda, SQS, SNS, Step Functions, and API Gateway, plus a real-time analytics pipeline with Kinesis Firehose and Athena.
• Implement security best practices (IAM, KMS, WAF, Secrets Manager), configure full observability with CloudWatch and CloudTrail, and codify the entire stack with CloudFormation and an Architecture Decision Record documenting every design choice with GCP comparisons.
• Added an ECS Fargate microservice with path-based ALB routing and configured AWS Budgets with automated cost controls.

Ready to quiz yourself?