AWS Multi-Region Edge Failover Sprint
Build a global edge routing architecture with automated DNS failover.
Introduction
30 Second Summary
When a payments API goes down, every second of downtime translates directly into lost transactions and SLA penalties. For fintech companies processing billions annually, a single misconfigured DNS record can cost millions.
In this project, you will build a multi-region edge failover architecture on AWS by leading four parallel AI agents through a 90-minute crisis-response sprint. You will deliver validated infrastructure, executive documentation, and a clean teardown that leaves zero orphaned resources.
What You'll Build
A two-region web property with sub-60-second automated failover, static Anycast IPs for regulated banking partners, and edge compute for security. All managed through Terraform and validated exclusively with AWS CLI.
By the end of this project, you'll have:
- A two-region failover architecture with Lambda Function URL origins, CloudFront origin groups, and Route 53 health checks proving sub-60-second recovery.
- A Global Accelerator static-IP variant solving the regulated banking requirement for firewall allowlists (PCI-DSS/SOC2 compliance).
- A leadership-ready package with 4 ADRs, a FinOps report, Mermaid topology, and an HTML presentation for both business and technical audiences.
- Secret Mission: Add WAF edge rules to rate-limit by geography and block banned user-agents at the edge without touching application code.
Are there any prerequisites?
You will need an AWS account with admin access, familiarity with Terraform, and a basic understanding of DNS and HTTP. You should also have Cursor IDE installed with access to Composer for running parallel agents.
Before We Start
It is 8:15 AM. The VP of Engineering just walked into your office. JPMorgan's CISO called your CEO because VaultStream's us-east-1 origin went unreachable for 47 minutes. Route 53 should have failed over. It did not.
Your assignment: deliver a production-grade, two-region failover reference architecture by 2:00 PM. Fully automated with Terraform. Fully documented. Tested. Presentable to the board. In this step, you will install your tools, brief your four parallel AI agents, and scaffold the project so you can hit the ground running.
In this step, get ready to:
- Install and verify Terraform 1.15.3 and AWS CLI v2.
- Brief your squad (Roy, Nova, Atlas, Sage) in Cursor Composer.
- Create the project scaffold and run terraform init to cache the AWS provider.
Install Terraform and verify AWS CLI
Two tools power this entire sprint. Terraform manages every AWS resource as code. The AWS CLI validates what Terraform deploys. Both must be ready before anything else happens.
- Open your terminal. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
- Type Terminal (macOS) or PowerShell (Windows) and press Enter to open it.
- Check whether Terraform is already installed by running:
terraform version✔️ I see version 1.15.3 or higher
Terraform is ready. Skip ahead to the AWS CLI check below.
ⓧ I see an older version
You need 1.15.3 or higher. Download the latest binary from the official install page and replace your existing installation.
macOS
- Upgrade via Homebrew by running these commands:
brew tap hashicorp/tap
brew install hashicorp/tap/terraform- Close and reopen your terminal, then run terraform version again to confirm 1.15.3 or higher.
Windows
- Download the latest Windows AMD64 binary from releases.hashicorp.com/terraform.
- Extract terraform.exe from the zip file.
- Replace your existing terraform.exe with the new one, or move it to a folder in your system PATH.
- Close and reopen PowerShell, then run terraform version again to confirm 1.15.3 or higher.
ⓧ Command not found
Terraform is not installed yet. Follow the instructions for your operating system below.
macOS
- Install Terraform via Homebrew by running these commands:
brew tap hashicorp/tap
brew install hashicorp/tap/terraform- Close and reopen your terminal, then run terraform version to confirm it prints 1.15.3 or higher.
Windows
- Download the latest Windows AMD64 binary from releases.hashicorp.com/terraform.
- Extract terraform.exe from the zip file.
- Move terraform.exe to a folder in your system PATH (e.g. C:\terraform\), then add that folder to your PATH environment variable via System Properties > Environment Variables.
- Close and reopen PowerShell, then run terraform version to confirm it prints 1.15.3 or higher.
Now check the AWS CLI. Run this command:
aws --version✔️ I see aws-cli/2.x
Your AWS CLI v2 is ready. Continue below.
ⓧ I see aws-cli/1.x
You have version 1 installed. You need version 2 for this project.
- Uninstall version 1, then install version 2 using the instructions in the next tab.
- Follow the official AWS CLI v2 install guide at docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html.
- After installing, close and reopen your terminal, then run aws --version to confirm it shows aws-cli/2.x.
ⓧ Command not found
AWS CLI is not installed. Install version 2 for your platform.
macOS
- Download the macOS installer package from https://awscli.amazonaws.com/AWSCLIV2.pkg.
- Open the downloaded .pkg file and follow the on-screen instructions.
- Close and reopen your terminal, then run aws --version to confirm it shows aws-cli/2.x.
Windows
- Download the MSI installer from https://awscli.amazonaws.com/AWSCLIV2.msi.
- Run the .msi file and follow the prompts.
- Close and reopen PowerShell, then run aws --version to confirm it shows aws-cli/2.x.
Now verify that your CLI credentials are configured and pointing at the correct account. Run this command:
aws sts get-caller-identityWhat should I see?
You should see a JSON response containing your Account, UserId, and Arn. This confirms your credentials are valid and your CLI can reach AWS.
- Confirm your default region is us-east-1 by running:
aws configure get regionSet your default region
If this returns blank or a different region, set it by running aws configure set region us-east-1. This project deploys the primary origin to us-east-1.
💡 What about a domain for Route 53?
You will need either a registered domain or a subdomain you already control via NS delegation. If you do not have one, you can register a cheap domain ($3-12) via Route 53 later. For now, just confirm your tools work.
🙋♀️ AWS CLI returning an error?
If you see a credentials error, run aws configure and enter your Access Key ID, Secret Access Key, default region (us-east-1), and output format (json).
If you see an expired token error, refresh your session credentials in your ~/.aws/credentials file or re-run aws configure.
Help me troubleshoot my AWS CLI credentials
Brief your squad in Cursor Composer
You are not coding alone. You are running four AI agents in parallel, each with a clear role and clear boundaries. This is how principal architects deliver under pressure.
- Open Cursor. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
- Type Cursor and press Enter to open it.
- Open Composer by pressing Cmd+I (macOS) or Ctrl+I (Windows).
- You will brief each agent with its role. Paste the following into Composer as your first message:
# Squad Briefing - VaultStream Edge Failover Sprint
We are building a two-region failover reference architecture for VaultStream.
Everything is Terraform-managed. One state file. One `terraform destroy`.
## Agent Roles:
**Roy (Infrastructure Lead):**
You own the origins and DNS. Two Lambda functions with Function URLs.
Route 53 zone. All 5 routing policies. Health checks.
Nothing manual. Everything in Terraform. You deploy FIRST because Nova and Atlas depend on your Function URL outputs.
**Nova (Edge/CDN Specialist):**
You own the edge. CloudFront distribution, origin group with failover,
CloudFront Function for the legacy header, Lambda@Edge for signed URLs.
Sub-5-second origin failover or we fail the SLA.
You WAIT for Roy's Lambda URL outputs before applying.
**Atlas (Network/Compliance):**
You own the network layer and security. Global Accelerator with static IPs
for the banking partners. NLBs as intermediaries (GA cannot target Lambda URLs directly).
WAF rules. The auditors will look at your work first.
You WAIT for Roy's Lambda ARNs for your NLB target groups.
**Sage (Staff Engineer/Docs):**
You own the story. Zero dependencies on infrastructure. Start immediately.
ADRs, Mermaid diagram, FinOps report, implementation plan, and presentation
can all begin now. Pick up Terraform outputs for the presentation as they land.What is this squad briefing doing?
You are establishing parallel workstreams with explicit dependency ordering. Roy goes first because Nova and Atlas both consume his Lambda outputs. Sage has zero dependencies and starts immediately. This mirrors how a real platform team ships under time pressure.
In Cursor Composer, you can open multiple agent threads. Each thread acts as one of your squad members, working on its assigned module directory.
Create the project scaffold and initialize Terraform
Every resource in this sprint lives in a single Terraform state. The directory structure separates concerns by agent ownership: Roy gets modules/origin/, Nova gets modules/edge/, Atlas gets modules/accelerator/.
- Move to your Desktop by running this command:
cd ~/Desktop- Create your project folder and move into it by running these commands:
mkdir vaultstream-edge
cd vaultstream-edge- Create the full directory structure by running:
mkdir -p modules/origin modules/edge modules/accelerator functions scripts docs/adr
touch docs/presentation.html- Confirm the directories exist by running:
lsYou should see docs, functions, modules, and scripts in the output.
- Create the providers.tf file. This file tells Terraform which cloud provider to use and pins the version. Open providers.tf in Cursor and paste the following code:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.80"
}
}
}
provider "aws" {
alias = "us_east_1"
region = "us-east-1"
}
provider "aws" {
alias = "eu_west_1"
region = "eu-west-1"
}What does this code do?
The required_providers block pins the AWS provider to version ~> 5.80. This means Terraform will use any 5.x version at or above 5.80 but never 6.0.
The two provider blocks create aliased configurations for us-east-1 and eu-west-1. This lets you deploy resources to both regions from a single Terraform state file.
- Save providers.tf.
- Initialize Terraform to download the AWS provider by running:
terraform initWhy run terraform init now?
The first terraform init downloads the AWS provider plugin (around 300MB). This takes 30-60 seconds. Running it now means the provider is cached locally when Roy, Nova, and Atlas start writing modules.
You should see output ending with a message confirming Terraform has been successfully initialized and a note about the installed provider version.
Terraform init failed?
If you see a network error, check your internet connection. Terraform needs to reach registry.terraform.io to download the provider.
If you see a version constraint error, make sure your providers.tf file matches the code above exactly. A typo in the version string will cause the download to fail.
Help me troubleshoot terraform init
✔️ Awesome, I've got everything!
Great. Double check you have saved providers.tf and that terraform init completed successfully.
ⓧ I'd like to double check the full code
Here is the complete providers.tf file. Make sure yours matches exactly:
terraform {
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 5.80"
}
}
}
provider "aws" {
alias = "us_east_1"
region = "us-east-1"
}
provider "aws" {
alias = "eu_west_1"
region = "eu-west-1"
}Your environment is locked and loaded. Terraform is cached. The squad is briefed. The clock is running. Next up, Roy deploys the two-region Lambda origins that everything else depends on.
Set Up Two-Region Origins and Route 53 DNS
Your squad is briefed and the Terraform provider is cached. Now it is time to build the foundation that everything else depends on.
Roy deploys first because Nova and Atlas cannot start their work until Roy's Lambda Function URL outputs exist. In this step, Roy creates two Lambda origins (one per region), configures Route 53 health checks against them, and wires up all five routing policies that map directly to SAP-C02 exam scenarios.
In this step, get ready to:
- Deploy two Lambda functions with Function URLs in us-east-1 and eu-west-1.
- Configure Route 53 health checks and five routing policy subdomains.
- Validate with AWS CLI and unblock Nova and Atlas.
Instruct Roy to define the Lambda origins
Roy's job is to create two identical Lambda functions, one in each region. Each function returns a JSON health payload with its region name. The Function URL gives each Lambda a public HTTPS endpoint at zero cost.
- In Cursor Composer, open a new agent tab for Roy.
- Give Roy the following prompt to create the origin module variables file at modules/origin/variables.tf:
Create modules/origin/variables.tf with a single variable:
- domain_name (type string, description "The Route 53 domain name for the hosted zone")Roy produces this file:
variable "domain_name" {
type = string
description = "The Route 53 domain name for the hosted zone"
}Why a separate variables file?
Terraform modules accept input via variables. Keeping variables.tf separate from main.tf makes the module interface obvious to anyone reading the code. Nova and Atlas will reference this module's outputs without needing to read its internals.
- Now instruct Roy to create modules/origin/main.tf with the two Lambda functions and their Function URLs. Give Roy this prompt:
In modules/origin/main.tf, create:
1. Two Lambda functions (Python 3.12, inline code via archive_file data source):
- "edge-origin-east" in us-east-1 (provider aws.us_east_1)
- "edge-origin-west" in eu-west-1 (provider aws.eu_west_1)
Each handler returns HTTP 200 with JSON: {"region": "<region>", "timestamp": "<iso>", "status": "healthy"}
2. Function URLs for both (authorization_type = NONE)
3. IAM execution roles for both regions
4. archive_file data sources to package the inline Python
5. Tag all resources: Project = "vaultstream-edge"Roy generates the Lambda resources. Review that the key section matches this pattern:
# Lambda function: us-east-1
resource "aws_lambda_function" "origin_east" {
provider = aws.us_east_1
function_name = "edge-origin-east"
runtime = "python3.12"
handler = "index.handler"
role = aws_iam_role.lambda_exec_east.arn
filename = data.archive_file.lambda_zip_east.output_path
tags = { Project = "vaultstream-edge" }
}
resource "aws_lambda_function_url" "origin_east" {
provider = aws.us_east_1
function_name = aws_lambda_function.origin_east.function_name
authorization_type = "NONE"
}
# Lambda function: eu-west-1
resource "aws_lambda_function" "origin_west" {
provider = aws.eu_west_1
function_name = "edge-origin-west"
runtime = "python3.12"
handler = "index.handler"
role = aws_iam_role.lambda_exec_west.arn
filename = data.archive_file.lambda_zip_west.output_path
tags = { Project = "vaultstream-edge" }
}
resource "aws_lambda_function_url" "origin_west" {
provider = aws.eu_west_1
function_name = aws_lambda_function.origin_west.function_name
authorization_type = "NONE"
}What does this code do?
Each aws_lambda_function deploys a Python 3.12 function to a specific region via the provider alias. The archive_file data source zips the inline handler code. Roy also creates IAM execution roles so Lambda has permission to run.
The aws_lambda_function_url resource creates a dedicated HTTPS endpoint for each function. Setting authorization_type = "NONE" makes the endpoint publicly accessible. This is what Route 53 health checks will hit.
💡 Why Lambda Function URLs instead of ALBs?
An Application Load Balancer costs approximately $0.0225/hour ($1.08/day for two). Lambda Function URLs are included in the standard Lambda free tier of 1M requests/month. For a simple health-check endpoint returning JSON, Function URLs deliver the same failover signal at zero cost.
This maps directly to SAP-C02 Domain 3.1 (cost optimization). The exam tests whether you know that ALBs are NOT required for CloudFront custom origins when a simpler option exists.
Instruct Roy to configure Route 53 health checks and routing policies
With the Lambda origins defined, Roy now adds the DNS layer. Route 53 health checks monitor each Function URL. Five routing-policy subdomains each demonstrate a different SAP-C02 scenario.
- Give Roy this prompt to add the Route 53 resources to modules/origin/main.tf:
In the same modules/origin/main.tf, add:
1. Route 53 hosted zone for var.domain_name (tagged Project = "vaultstream-edge")
2. Two health checks (HTTPS, resource_path: "/", interval: 10s, failure_threshold: 3,
regions: ["us-east-1", "eu-west-1", "ap-southeast-1"])
Extract FQDN from Function URL using regex("https://([^/]+)", url)[0]
3. Five routing policy subdomains (all CNAME, TTL 60):
- failover.<domain>: active-passive (PRIMARY with east health check, SECONDARY with west)
- latency.<domain>: nearest-region routing (us-east-1, eu-west-1)
- weighted.<domain>: 70/30 split (east 70, west 30)
- geo.<domain>: geolocation (continent NA -> east, EU -> west)
- multivalue.<domain>: multivalue answer with health checks on bothRoy generates the Route 53 resources. Review that the health checks match this pattern:
# Route 53 hosted zone
resource "aws_route53_zone" "main" {
name = var.domain_name
tags = { Project = "vaultstream-edge" }
}
# Health checks
resource "aws_route53_health_check" "east" {
fqdn = regex("https://([^/]+)", aws_lambda_function_url.origin_east.function_url)[0]
type = "HTTPS"
resource_path = "/"
request_interval = 10
failure_threshold = 3
regions = ["us-east-1", "eu-west-1", "ap-southeast-1"]
tags = { Project = "vaultstream-edge" }
}
resource "aws_route53_health_check" "west" {
fqdn = regex("https://([^/]+)", aws_lambda_function_url.origin_west.function_url)[0]
type = "HTTPS"
resource_path = "/"
request_interval = 10
failure_threshold = 3
regions = ["us-east-1", "eu-west-1", "ap-southeast-1"]
tags = { Project = "vaultstream-edge" }
}What does this code do?
The regex function extracts the bare FQDN from the Function URL (stripping https:// and the trailing /). Route 53 health checkers from three geographically spread regions hit each endpoint every 10 seconds.
After 3 consecutive failures (30 seconds), Route 53 marks the endpoint unhealthy. This is the signal that drives DNS-level failover for the failover. subdomain.
Review the failover routing policy. This is the pattern that caused VaultStream's $2.3M outage when evaluate_target_health was set incorrectly:
# Failover routing policy
resource "aws_route53_record" "failover_primary" {
zone_id = aws_route53_zone.main.zone_id
name = "failover.${var.domain_name}"
type = "CNAME"
ttl = 60
records = [regex("https://([^/]+)", aws_lambda_function_url.origin_east.function_url)[0]]
set_identifier = "primary"
failover_routing_policy { type = "PRIMARY" }
health_check_id = aws_route53_health_check.east.id
}
resource "aws_route53_record" "failover_secondary" {
zone_id = aws_route53_zone.main.zone_id
name = "failover.${var.domain_name}"
type = "CNAME"
ttl = 60
records = [regex("https://([^/]+)", aws_lambda_function_url.origin_west.function_url)[0]]
set_identifier = "secondary"
failover_routing_policy { type = "SECONDARY" }
health_check_id = aws_route53_health_check.west.id
}Why is this the most critical routing policy?
Failover routing is active-passive. Route 53 returns the PRIMARY record when healthy, and switches to SECONDARY when the primary's health check fails. The health_check_id on each record is what makes this automatic.
SAP-C02 Domain 2.4 trap: candidates confuse failover with multivalue answer. Multivalue returns up to 8 healthy IPs for approximate load distribution. Failover returns only the healthy primary OR the secondary. They solve fundamentally different problems.
- Confirm Roy also generated the remaining four routing policies: latency, weighted, geo, and multivalue.
- Instruct Roy to create modules/origin/outputs.tf exposing the Function URLs, Lambda ARNs, and hosted zone ID:
output "function_url_east" {
value = aws_lambda_function_url.origin_east.function_url
}
output "function_url_west" {
value = aws_lambda_function_url.origin_west.function_url
}
output "lambda_arn_east" {
value = aws_lambda_function.origin_east.arn
}
output "lambda_arn_west" {
value = aws_lambda_function.origin_west.arn
}
output "hosted_zone_id" {
value = aws_route53_zone.main.zone_id
}Why do these outputs matter?
Nova needs function_url_east and function_url_west to configure CloudFront custom origins. Atlas needs lambda_arn_east and lambda_arn_west to set up NLB target groups for Global Accelerator. Without these outputs, both agents are blocked.
- Save all files in modules/origin/.
- Apply the module by running this command in your terminal:
terraform apply -auto-approveWhat should I see?
Terraform creates the IAM roles, Lambda functions, Function URLs, Route 53 zone, health checks, and all routing policy records. You should see output like Apply complete! Resources: N added, 0 changed, 0 destroyed followed by your output values.
The apply typically takes 30-60 seconds. Route 53 health checks may take an additional 30 seconds to reach a healthy state.
🙋♀️ Seeing an error during apply?
If you see an error about the IAM role not existing, ensure Roy's code includes aws_iam_role resources for both regions with the lambda.amazonaws.com trust policy.
If the Route 53 zone fails, confirm you have a registered domain and that your AWS account has permission to create hosted zones.
help me debug my Terraform apply error for the origin module
✔️ Awesome, I've got everything!
Great. Double check you saved all three files in modules/origin/.
ⓧ I'd like to double check the full code
variable "domain_name" {
type = string
description = "The Route 53 domain name for the hosted zone"
}# Lambda function: us-east-1
resource "aws_lambda_function" "origin_east" {
provider = aws.us_east_1
function_name = "edge-origin-east"
runtime = "python3.12"
handler = "index.handler"
role = aws_iam_role.lambda_exec_east.arn
filename = data.archive_file.lambda_zip_east.output_path
tags = { Project = "vaultstream-edge" }
}
resource "aws_lambda_function_url" "origin_east" {
provider = aws.us_east_1
function_name = aws_lambda_function.origin_east.function_name
authorization_type = "NONE"
}
# Lambda function: eu-west-1
resource "aws_lambda_function" "origin_west" {
provider = aws.eu_west_1
function_name = "edge-origin-west"
runtime = "python3.12"
handler = "index.handler"
role = aws_iam_role.lambda_exec_west.arn
filename = data.archive_file.lambda_zip_west.output_path
tags = { Project = "vaultstream-edge" }
}
resource "aws_lambda_function_url" "origin_west" {
provider = aws.eu_west_1
function_name = aws_lambda_function.origin_west.function_name
authorization_type = "NONE"
}
# Route 53 hosted zone
resource "aws_route53_zone" "main" {
name = var.domain_name
tags = { Project = "vaultstream-edge" }
}
# Health checks
resource "aws_route53_health_check" "east" {
fqdn = regex("https://([^/]+)", aws_lambda_function_url.origin_east.function_url)[0]
type = "HTTPS"
resource_path = "/"
request_interval = 10
failure_threshold = 3
regions = ["us-east-1", "eu-west-1", "ap-southeast-1"]
tags = { Project = "vaultstream-edge" }
}
resource "aws_route53_health_check" "west" {
fqdn = regex("https://([^/]+)", aws_lambda_function_url.origin_west.function_url)[0]
type = "HTTPS"
resource_path = "/"
request_interval = 10
failure_threshold = 3
regions = ["us-east-1", "eu-west-1", "ap-southeast-1"]
tags = { Project = "vaultstream-edge" }
}
# Failover routing policy
resource "aws_route53_record" "failover_primary" {
zone_id = aws_route53_zone.main.zone_id
name = "failover.${var.domain_name}"
type = "CNAME"
ttl = 60
records = [regex("https://([^/]+)", aws_lambda_function_url.origin_east.function_url)[0]]
set_identifier = "primary"
failover_routing_policy { type = "PRIMARY" }
health_check_id = aws_route53_health_check.east.id
}
resource "aws_route53_record" "failover_secondary" {
zone_id = aws_route53_zone.main.zone_id
name = "failover.${var.domain_name}"
type = "CNAME"
ttl = 60
records = [regex("https://([^/]+)", aws_lambda_function_url.origin_west.function_url)[0]]
set_identifier = "secondary"
failover_routing_policy { type = "SECONDARY" }
health_check_id = aws_route53_health_check.west.id
}
# Latency routing policy
resource "aws_route53_record" "latency_east" {
zone_id = aws_route53_zone.main.zone_id
name = "latency.${var.domain_name}"
type = "CNAME"
ttl = 60
records = [regex("https://([^/]+)", aws_lambda_function_url.origin_east.function_url)[0]]
set_identifier = "east"
latency_routing_policy { region = "us-east-1" }
health_check_id = aws_route53_health_check.east.id
}
resource "aws_route53_record" "latency_west" {
zone_id = aws_route53_zone.main.zone_id
name = "latency.${var.domain_name}"
type = "CNAME"
ttl = 60
records = [regex("https://([^/]+)", aws_lambda_function_url.origin_west.function_url)[0]]
set_identifier = "west"
latency_routing_policy { region = "eu-west-1" }
health_check_id = aws_route53_health_check.west.id
}
# Weighted routing policy (70/30)
resource "aws_route53_record" "weighted_east" {
zone_id = aws_route53_zone.main.zone_id
name = "weighted.${var.domain_name}"
type = "CNAME"
ttl = 60
records = [regex("https://([^/]+)", aws_lambda_function_url.origin_east.function_url)[0]]
set_identifier = "east-70"
weighted_routing_policy { weight = 70 }
}
resource "aws_route53_record" "weighted_west" {
zone_id = aws_route53_zone.main.zone_id
name = "weighted.${var.domain_name}"
type = "CNAME"
ttl = 60
records = [regex("https://([^/]+)", aws_lambda_function_url.origin_west.function_url)[0]]
set_identifier = "west-30"
weighted_routing_policy { weight = 30 }
}
# Geolocation routing policy
resource "aws_route53_record" "geo_na" {
zone_id = aws_route53_zone.main.zone_id
name = "geo.${var.domain_name}"
type = "CNAME"
ttl = 60
records = [regex("https://([^/]+)", aws_lambda_function_url.origin_east.function_url)[0]]
set_identifier = "north-america"
geolocation_routing_policy { continent = "NA" }
}
resource "aws_route53_record" "geo_eu" {
zone_id = aws_route53_zone.main.zone_id
name = "geo.${var.domain_name}"
type = "CNAME"
ttl = 60
records = [regex("https://([^/]+)", aws_lambda_function_url.origin_west.function_url)[0]]
set_identifier = "europe"
geolocation_routing_policy { continent = "EU" }
}
# Multivalue routing policy
resource "aws_route53_record" "multivalue_east" {
zone_id = aws_route53_zone.main.zone_id
name = "multivalue.${var.domain_name}"
type = "CNAME"
ttl = 60
records = [regex("https://([^/]+)", aws_lambda_function_url.origin_east.function_url)[0]]
set_identifier = "mv-east"
multivalue_answer_routing_policy = true
health_check_id = aws_route53_health_check.east.id
}
resource "aws_route53_record" "multivalue_west" {
zone_id = aws_route53_zone.main.zone_id
name = "multivalue.${var.domain_name}"
type = "CNAME"
ttl = 60
records = [regex("https://([^/]+)", aws_lambda_function_url.origin_west.function_url)[0]]
set_identifier = "mv-west"
multivalue_answer_routing_policy = true
health_check_id = aws_route53_health_check.west.id
}output "function_url_east" {
value = aws_lambda_function_url.origin_east.function_url
}
output "function_url_west" {
value = aws_lambda_function_url.origin_west.function_url
}
output "lambda_arn_east" {
value = aws_lambda_function.origin_east.arn
}
output "lambda_arn_west" {
value = aws_lambda_function.origin_west.arn
}
output "hosted_zone_id" {
value = aws_route53_zone.main.zone_id
}Validate with AWS CLI and unblock the squad
The infrastructure is live. Now you verify every resource exists and confirm the outputs that Nova and Atlas depend on.
- Check the health checks are registered by running:
aws route53 list-health-checks --output table- You should see two health checks with Type: HTTPS and RequestInterval: 10.
- Verify the east Lambda Function URL is responding by running:
aws lambda get-function-url-config --function-name edge-origin-east- You should see AuthorizationType: NONE and a FunctionUrl value starting with https://.
- Check the Route 53 record sets by running:
aws route53 list-resource-record-sets --hosted-zone-id [[HOSTED_ZONE_ID="your Route 53 hosted zone ID from terraform output"]]- You should see CNAME records for failover., latency., weighted., geo., and multivalue. subdomains.
What is the SAP-C02 trap with multivalue routing?
Multivalue answer returns up to 8 healthy records. The client picks one at random. This is DNS-level load spreading, not failover. If you need deterministic active-passive behavior, use failover routing.
The exam loves to present multivalue as a failover substitute. It is not. Multivalue is an alternative to a load balancer for simple use cases where client-side random selection is acceptable.
🙋♀️ Health checks stuck in unknown state?
Route 53 health checks can take 30-60 seconds to report their first status. Wait a minute and check the status again. If they remain unhealthy, verify your Lambda Function URLs return HTTP 200 by curling one directly.
help me debug Route 53 health checks that are stuck in unhealthy or unknown state
Roy's work is done. Nova now has the Function URL outputs to configure CloudFront origins. Atlas has the Lambda ARNs for NLB target groups. Time to build the edge layer.
Build CloudFront Distribution with Origin Failover and Edge Compute
Roy's Lambda origins and Route 53 health checks are live. Now it is time for Nova to build the edge layer that sits in front of those origins and makes sub-5-second request-level failover possible.
DNS-level failover (Route 53) operates on a 30-60 second cycle because of TTL propagation. CloudFront origin groups solve a different problem: when a single request hits a failing primary origin, CloudFront immediately retries on the secondary origin within the same viewer connection. Combined with edge compute functions that strip legacy headers and gate premium content with signed URLs, this step transforms VaultStream's raw Lambda endpoints into a production-grade edge network.
In this step, get ready to:
- Define a CloudFront distribution with origin failover (us-east-1 primary, eu-west-1 secondary).
- Deploy a CloudFront Function to strip legacy client headers at the viewer-request edge.
- Create a Lambda@Edge function that validates signed URLs at the origin-request stage.
Define the CloudFront distribution with origin failover
Nova needs Roy's Function URL outputs as origin domain names. The distribution uses a custom origin configuration pointing to each Lambda Function URL, grouped into an origin group that triggers failover on 5xx status codes.
- Create the file modules/edge/variables.tf to accept the Function URL outputs from Roy's module:
variable "function_url_east" {
type = string
}
variable "function_url_west" {
type = string
}What do these variables do?
These two input variables receive the Lambda Function URL outputs from Roy's origin module. Nova's CloudFront distribution extracts the domain portion from each URL to configure its custom origins.
- Create modules/edge/main.tf and add the CloudFront distribution resource. This is the largest resource in Nova's module. Instruct Nova to write the following:
# CloudFront distribution
resource "aws_cloudfront_distribution" "main" {
provider = aws.us_east_1
enabled = true
comment = "VaultStream edge distribution"
tags = { Project = "vaultstream-edge" }
origin {
origin_id = "lambda-east"
domain_name = regex("https://([^/]+)", var.function_url_east)[0]
custom_origin_config {
http_port = 80
https_port = 443
origin_protocol_policy = "https-only"
origin_ssl_protocols = ["TLSv1.2"]
origin_read_timeout = 3
origin_keepalive_timeout = 3
}
}
origin {
origin_id = "lambda-west"
domain_name = regex("https://([^/]+)", var.function_url_west)[0]
custom_origin_config {
http_port = 80
https_port = 443
origin_protocol_policy = "https-only"
origin_ssl_protocols = ["TLSv1.2"]
origin_read_timeout = 3
origin_keepalive_timeout = 3
}
}
origin_group {
origin_id = "lambda-failover-group"
failover_criteria {
status_codes = [500, 502, 503, 504]
}
member { origin_id = "lambda-east" }
member { origin_id = "lambda-west" }
}
default_cache_behavior {
target_origin_id = "lambda-failover-group"
viewer_protocol_policy = "redirect-to-https"
allowed_methods = ["GET", "HEAD"]
cached_methods = ["GET", "HEAD"]
compress = true
forwarded_values {
query_string = true
cookies { forward = "none" }
}
function_association {
event_type = "viewer-request"
function_arn = aws_cloudfront_function.strip_legacy_header.arn
}
lambda_function_association {
event_type = "origin-request"
lambda_arn = aws_lambda_function.signed_url_validator.qualified_arn
include_body = false
}
}
restrictions {
geo_restriction { restriction_type = "none" }
}
viewer_certificate {
cloudfront_default_certificate = true
}
}What does this distribution do?
Two custom origins point to Roy's Lambda Function URLs. The origin_group bundles them together with failover criteria: if the primary (us-east-1) returns 500, 502, 503, or 504, CloudFront retries the same request against the secondary (eu-west-1) within the same viewer connection.
The origin_read_timeout of 3 seconds combined with the origin group's automatic retry means a failing origin triggers failover in under 5 seconds. By default, CloudFront would wait 30 seconds (3 attempts of 10 seconds each). We reduced this to a single 3-second window.
The function_association and lambda_function_association blocks reference edge compute resources we define next. This is valid because Terraform resolves references via its dependency graph, not file order.
💡 Why CloudFront Functions on viewer-request and Lambda@Edge on origin-request?
CloudFront restricts combining both function types on the same event type within the same cache behavior. However, placing a CloudFront Function on viewer-request and Lambda@Edge on origin-request is an allowed combination. This gives us submillisecond header stripping at the edge PLUS full Node.js runtime for cryptographic signature validation at the origin.
Create edge compute functions
Nova needs two edge functions: a lightweight CloudFront Function that strips the x-legacy-client header before it ever hits the origin, and a Lambda@Edge function that validates signed URL parameters to gate premium API content.
- Add the CloudFront Function resource at the top of modules/edge/main.tf (above the distribution resource):
# CloudFront Function: strip legacy header
resource "aws_cloudfront_function" "strip_legacy_header" {
provider = aws.us_east_1
name = "strip-legacy-header"
runtime = "cloudfront-js-2.0"
publish = true
code = <<-EOF
function handler(event) {
var request = event.request;
delete request.headers['x-legacy-client'];
return request;
}
EOF
}What does this CloudFront Function do?
VaultStream's legacy banking partners send an X-Legacy-Client header that the origin no longer needs. Instead of modifying 200+ partner integrations, we strip the header at the edge in submillisecond time. This is a SAP-C02 D4.2 pattern: migrate workload behavior at the edge without touching application code.
CloudFront Functions run at every edge location with a 10KB code limit and no network access. They are purpose-built for simple request/response manipulations like this.
- Add the Lambda@Edge resource below the CloudFront Function (still above the distribution) in modules/edge/main.tf:
# Lambda@Edge: signed URL validator (must be us-east-1)
resource "aws_lambda_function" "signed_url_validator" {
provider = aws.us_east_1
function_name = "signed-url-validator"
runtime = "nodejs20.x"
handler = "index.handler"
role = aws_iam_role.lambda_edge_exec.arn
filename = data.archive_file.signed_url_zip.output_path
publish = true
tags = { Project = "vaultstream-edge" }
}Why must Lambda@Edge deploy to us-east-1?
AWS replicates Lambda@Edge functions globally from us-east-1. Regardless of where your viewers connect, the function must be authored and published in us-east-1. The publish = true flag creates a numbered version (not $LATEST), which is required for CloudFront association.
- Create the handler code at functions/signed_url_validator/index.js:
'use strict';
exports.handler = async (event) => {
const request = event.Records[0].cf.request;
const querystring = require('querystring');
const params = querystring.parse(request.querystring);
if (!params.Expires || !params.Signature) {
return {
status: '403',
statusDescription: 'Forbidden',
headers: { 'content-type': [{ key: 'Content-Type', value: 'application/json' }] },
body: JSON.stringify({ error: 'Unauthorized', reason: 'Invalid or expired signature' })
};
}
const now = Math.floor(Date.now() / 1000);
if (parseInt(params.Expires) < now) {
return {
status: '403',
statusDescription: 'Forbidden',
headers: { 'content-type': [{ key: 'Content-Type', value: 'application/json' }] },
body: JSON.stringify({ error: 'Unauthorized', reason: 'Invalid or expired signature' })
};
}
return request;
};What does this Lambda@Edge function validate?
On every origin-request, this function checks for Expires and Signature query parameters. If either is missing, or if the expiration timestamp is in the past, the function short-circuits with a 403 JSON response. The request never reaches the Lambda origin.
This is a SAP-C02 D2.2 pattern: security controls at the edge without touching application code. A CloudFront Function cannot perform this validation because it has no access to the request body and is limited to 10KB of code. Lambda@Edge has full Node.js runtime, network access, and up to 30 seconds of execution time.
- Save both modules/edge/main.tf and functions/signed_url_validator/index.js.
Nova also needs supporting resources
The Lambda@Edge resource references aws_iam_role.lambda_edge_exec and data.archive_file.signed_url_zip. Instruct Nova to create the IAM execution role (with edgelambda.amazonaws.com and lambda.amazonaws.com as trusted principals) and the archive data source that zips functions/signed_url_validator/. These are standard boilerplate that Nova will generate in the same module.
🙋♀️ Terraform says resource reference is invalid?
If you see errors about aws_cloudfront_function.strip_legacy_header or aws_lambda_function.signed_url_validator not existing, confirm that all three resources (CloudFront Function, Lambda@Edge, Distribution) are in the same module directory. Terraform resolves dependencies within a module regardless of file order.
Help me debug reference errors in my edge module
Apply and validate the edge module
With all edge resources defined, Nova can now apply and verify that CloudFront, the CloudFront Function, and Lambda@Edge are all live.
- Create modules/edge/outputs.tf to expose the distribution ID and domain name for other modules:
output "cloudfront_distribution_id" {
value = aws_cloudfront_distribution.main.id
}
output "cloudfront_domain_name" {
value = aws_cloudfront_distribution.main.domain_name
}Why expose these outputs?
Sage needs the CloudFront domain name for the validation scripts (curl testing). Atlas needs the distribution ID to associate the WAF WebACL. These outputs make the edge module composable within the single Terraform state.
- Apply the edge module by running:
terraform apply -auto-approveWhat should I expect?
CloudFront distributions take 5-15 minutes to deploy globally. Terraform will show the apply as complete once AWS accepts the configuration, but the distribution status will be InProgress for several minutes. You can proceed to validation commands immediately. They will work as soon as the status changes to Deployed.
- Validate the CloudFront distribution origins by running:
aws cloudfront get-distribution --id $(terraform output -raw cloudfront_distribution_id) --query "Distribution.DistributionConfig.Origins"- You should see both lambda-east and lambda-west origins listed with their Function URL domain names.
- Validate the CloudFront Function is deployed to LIVE stage by running:
aws cloudfront describe-function --name strip-legacy-header --stage LIVE- You should see the function metadata including Status and Runtime confirming it is live.
- Validate the Lambda@Edge function has a published version by running:
aws lambda get-function --function-name signed-url-validator --query "Configuration.Version"- You should see a version number (not $LATEST). This confirms the function has a published numbered version that CloudFront can associate with.
Distribution still showing InProgress?
CloudFront global propagation takes 5-15 minutes. The get-distribution command will still return origin configuration even while status is InProgress. If the command returns an error about the distribution not existing, check that terraform apply completed successfully and that you are using the correct distribution ID from your Terraform output.
Help me troubleshoot my edge module deployment
✔️ Awesome, I've got everything!
Great. Double check you have saved all three files in the edge module.
ⓧ I'd like to double check the full code
Here is the complete modules/edge/main.tf:
# CloudFront Function: strip legacy header
resource "aws_cloudfront_function" "strip_legacy_header" {
provider = aws.us_east_1
name = "strip-legacy-header"
runtime = "cloudfront-js-2.0"
publish = true
code = <<-EOF
function handler(event) {
var request = event.request;
delete request.headers['x-legacy-client'];
return request;
}
EOF
}
# Lambda@Edge: signed URL validator (must be us-east-1)
resource "aws_lambda_function" "signed_url_validator" {
provider = aws.us_east_1
function_name = "signed-url-validator"
runtime = "nodejs20.x"
handler = "index.handler"
role = aws_iam_role.lambda_edge_exec.arn
filename = data.archive_file.signed_url_zip.output_path
publish = true
tags = { Project = "vaultstream-edge" }
}
# CloudFront distribution
resource "aws_cloudfront_distribution" "main" {
provider = aws.us_east_1
enabled = true
comment = "VaultStream edge distribution"
tags = { Project = "vaultstream-edge" }
origin {
origin_id = "lambda-east"
domain_name = regex("https://([^/]+)", var.function_url_east)[0]
custom_origin_config {
http_port = 80
https_port = 443
origin_protocol_policy = "https-only"
origin_ssl_protocols = ["TLSv1.2"]
origin_read_timeout = 3
origin_keepalive_timeout = 3
}
}
origin {
origin_id = "lambda-west"
domain_name = regex("https://([^/]+)", var.function_url_west)[0]
custom_origin_config {
http_port = 80
https_port = 443
origin_protocol_policy = "https-only"
origin_ssl_protocols = ["TLSv1.2"]
origin_read_timeout = 3
origin_keepalive_timeout = 3
}
}
origin_group {
origin_id = "lambda-failover-group"
failover_criteria {
status_codes = [500, 502, 503, 504]
}
member { origin_id = "lambda-east" }
member { origin_id = "lambda-west" }
}
default_cache_behavior {
target_origin_id = "lambda-failover-group"
viewer_protocol_policy = "redirect-to-https"
allowed_methods = ["GET", "HEAD"]
cached_methods = ["GET", "HEAD"]
compress = true
forwarded_values {
query_string = true
cookies { forward = "none" }
}
function_association {
event_type = "viewer-request"
function_arn = aws_cloudfront_function.strip_legacy_header.arn
}
lambda_function_association {
event_type = "origin-request"
lambda_arn = aws_lambda_function.signed_url_validator.qualified_arn
include_body = false
}
}
restrictions {
geo_restriction { restriction_type = "none" }
}
viewer_certificate {
cloudfront_default_certificate = true
}
}Here is modules/edge/variables.tf:
variable "function_url_east" {
type = string
}
variable "function_url_west" {
type = string
}Here is modules/edge/outputs.tf:
output "cloudfront_distribution_id" {
value = aws_cloudfront_distribution.main.id
}
output "cloudfront_domain_name" {
value = aws_cloudfront_distribution.main.domain_name
}Here is functions/signed_url_validator/index.js:
'use strict';
exports.handler = async (event) => {
const request = event.Records[0].cf.request;
const querystring = require('querystring');
const params = querystring.parse(request.querystring);
if (!params.Expires || !params.Signature) {
return {
status: '403',
statusDescription: 'Forbidden',
headers: { 'content-type': [{ key: 'Content-Type', value: 'application/json' }] },
body: JSON.stringify({ error: 'Unauthorized', reason: 'Invalid or expired signature' })
};
}
const now = Math.floor(Date.now() / 1000);
if (parseInt(params.Expires) < now) {
return {
status: '403',
statusDescription: 'Forbidden',
headers: { 'content-type': [{ key: 'Content-Type', value: 'application/json' }] },
body: JSON.stringify({ error: 'Unauthorized', reason: 'Invalid or expired signature' })
};
}
return request;
};Nova's edge layer is live. CloudFront is now fronting VaultStream's two-region origins with sub-5-second request-level failover, legacy header stripping, and signed URL validation. Next up, Atlas builds the Global Accelerator layer to give regulated banking partners the static Anycast IPs they need for their firewall allowlists.
Deploy Global Accelerator and WAF
Roy and Nova have delivered the origins and edge layer. Now Atlas faces the compliance problem that triggered this entire sprint: JPMorgan's CISO needs static IP addresses for their firewall allowlist.
CloudFront uses dynamic Anycast IPs that change constantly. Global Accelerator provides two permanent static IPv4 addresses per accelerator. The catch is that GA cannot target Lambda Function URLs directly. Atlas must build an NLB bridge in each region to connect GA to Lambda.
In this step, get ready to:
- Define Global Accelerator with NLB intermediaries bridging to Lambda in both regions.
- Deploy a WAF WebACL protecting CloudFront with rate-limiting and bot-blocking rules.
- Validate static IPs and WAF rules with AWS CLI commands.
Define the accelerator module with NLB bridge and Global Accelerator
GA endpoint types are limited to NLBs, ALBs, EC2 instances, and Elastic IP addresses. Since Roy's origins are Lambda Function URLs, Atlas needs a minimal NLB in each region to bridge the gap. Each NLB needs a target group (type lambda), a Lambda permission granting elasticloadbalancing.amazonaws.com invoke access, and a TCP/443 listener.
- Create the module variables file by instructing Atlas to add modules/accelerator/variables.tf with this code:
variable "lambda_arn_east" {
type = string
}
variable "lambda_arn_west" {
type = string
}
variable "subnet_ids_east" {
type = list(string)
}
variable "subnet_ids_west" {
type = list(string)
}What do these variables do?
The module takes Roy's Lambda ARNs (for NLB target group registration) and subnet IDs (for NLB placement in each VPC). The root module passes these values from Roy's outputs and your default VPC data sources.
- Save modules/accelerator/variables.tf.
- Now instruct Atlas to create modules/accelerator/main.tf starting with the NLB bridge for us-east-1:
# NLB east
resource "aws_lb" "nlb_east" {
provider = aws.us_east_1
name = "vaultstream-nlb-east"
load_balancer_type = "network"
internal = false
subnets = var.subnet_ids_east
tags = { Project = "vaultstream-edge" }
}
resource "aws_lb_target_group" "lambda_east" {
provider = aws.us_east_1
name = "vaultstream-tg-east"
target_type = "lambda"
tags = { Project = "vaultstream-edge" }
}
resource "aws_lambda_permission" "nlb_east" {
provider = aws.us_east_1
statement_id = "AllowNLBEast"
action = "lambda:InvokeFunction"
function_name = var.lambda_arn_east
principal = "elasticloadbalancing.amazonaws.com"
source_arn = aws_lb_target_group.lambda_east.arn
}
resource "aws_lb_target_group_attachment" "lambda_east" {
provider = aws.us_east_1
target_group_arn = aws_lb_target_group.lambda_east.arn
target_id = var.lambda_arn_east
depends_on = [aws_lambda_permission.nlb_east]
}
resource "aws_lb_listener" "nlb_east" {
provider = aws.us_east_1
load_balancer_arn = aws_lb.nlb_east.arn
port = 443
protocol = "TCP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.lambda_east.arn
}
}What does this NLB bridge do?
Five resources form the east-region bridge. The NLB receives TCP/443 traffic from Global Accelerator. The target group (type lambda) registers Roy's Lambda function as the backend.
The aws_lambda_permission is critical. Without it, the NLB cannot invoke Lambda and requests return 502. The depends_on ensures the permission exists before Terraform attaches Lambda to the target group.
- Below the east NLB resources, add the mirrored NLB bridge for eu-west-1:
# NLB west
resource "aws_lb" "nlb_west" {
provider = aws.eu_west_1
name = "vaultstream-nlb-west"
load_balancer_type = "network"
internal = false
subnets = var.subnet_ids_west
tags = { Project = "vaultstream-edge" }
}
resource "aws_lb_target_group" "lambda_west" {
provider = aws.eu_west_1
name = "vaultstream-tg-west"
target_type = "lambda"
tags = { Project = "vaultstream-edge" }
}
resource "aws_lambda_permission" "nlb_west" {
provider = aws.eu_west_1
statement_id = "AllowNLBWest"
action = "lambda:InvokeFunction"
function_name = var.lambda_arn_west
principal = "elasticloadbalancing.amazonaws.com"
source_arn = aws_lb_target_group.lambda_west.arn
}
resource "aws_lb_target_group_attachment" "lambda_west" {
provider = aws.eu_west_1
target_group_arn = aws_lb_target_group.lambda_west.arn
target_id = var.lambda_arn_west
depends_on = [aws_lambda_permission.nlb_west]
}
resource "aws_lb_listener" "nlb_west" {
provider = aws.eu_west_1
load_balancer_arn = aws_lb.nlb_west.arn
port = 443
protocol = "TCP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.lambda_west.arn
}
}Why mirror the NLB?
Global Accelerator routes to the nearest healthy endpoint group. If the east endpoint goes unhealthy, GA automatically shifts traffic to the west endpoint group in about 30 seconds. Without both NLBs, there is no failover path for the static-IP traffic.
- Below the west NLB resources, add the Global Accelerator itself:
# Global Accelerator
resource "aws_globalaccelerator_accelerator" "main" {
name = "vaultstream-ga"
ip_address_type = "IPV4"
enabled = true
tags = { Project = "vaultstream-edge" }
}
resource "aws_globalaccelerator_listener" "main" {
accelerator_arn = aws_globalaccelerator_accelerator.main.id
protocol = "TCP"
port_range {
from_port = 443
to_port = 443
}
}
resource "aws_globalaccelerator_endpoint_group" "east" {
listener_arn = aws_globalaccelerator_listener.main.id
endpoint_group_region = "us-east-1"
endpoint_configuration {
endpoint_id = aws_lb.nlb_east.arn
weight = 100
}
}
resource "aws_globalaccelerator_endpoint_group" "west" {
listener_arn = aws_globalaccelerator_listener.main.id
endpoint_group_region = "eu-west-1"
endpoint_configuration {
endpoint_id = aws_lb.nlb_west.arn
weight = 100
}
}What does this Global Accelerator configuration do?
The accelerator provisions two static Anycast IPv4 addresses. The listener accepts TCP/443 traffic on those IPs. Two endpoint groups (one per region) point to the NLBs, each with weight 100 (equal distribution).
GA monitors endpoint health every 10-30 seconds. If the east NLB becomes unhealthy, GA routes all traffic to the west endpoint group. Recovery takes approximately 30 seconds.
- Save modules/accelerator/main.tf.
Seeing provider configuration errors?
Global Accelerator is a global service but its API lives in us-west-2. Terraform's AWS provider handles this internally, so you do not need a us-west-2 provider alias. If you see errors about the GA resources, confirm your module block passes the required provider aliases correctly.
help me debug Global Accelerator provider configuration in Terraform
Add WAF WebACL and capture static IPs
Atlas now adds edge security. A WAF WebACL with scope CLOUDFRONT must be created in us-east-1. It attaches to the CloudFront distribution and enforces two rules: geo-scoped rate limiting and bot User-Agent blocking.
- At the bottom of modules/accelerator/main.tf, add the WAF WebACL resource:
# WAF WebACL (scope=CLOUDFRONT must be us-east-1)
resource "aws_wafv2_web_acl" "edge" {
provider = aws.us_east_1
name = "vaultstream-edge-waf"
scope = "CLOUDFRONT"
tags = { Project = "vaultstream-edge" }
default_action { allow {} }
rule {
name = "RateLimitGeoXX"
priority = 1
action { block {} }
statement {
rate_based_statement {
limit = 1000
aggregate_key_type = "IP"
scope_down_statement {
geo_match_statement {
country_codes = ["XX"]
}
}
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "RateLimitGeoXX"
sampled_requests_enabled = true
}
}
rule {
name = "BlockBadBot"
priority = 2
action { block {} }
statement {
byte_match_statement {
search_string = "badbot/1.0"
field_to_match {
single_header { name = "user-agent" }
}
text_transformation {
priority = 0
type = "LOWERCASE"
}
positional_constraint = "CONTAINS"
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "BlockBadBot"
sampled_requests_enabled = true
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "VaultStreamEdgeWAF"
sampled_requests_enabled = true
}
}What do these WAF rules do?
Rule 1 (RateLimitGeoXX): blocks any single IP that exceeds 1000 requests in a 5-minute window, scoped only to country code XX. The scope_down_statement ensures the rate limit only applies to traffic from that geography.
Rule 2 (BlockBadBot): inspects the user-agent header after applying a LOWERCASE transformation. Any request containing badbot/1.0 (case-insensitive) gets blocked immediately. This proves D2.2 security controls without touching application code.
- Now create modules/accelerator/outputs.tf to expose the static IPs and ARNs:
output "ga_static_ips" {
value = aws_globalaccelerator_accelerator.main.ip_sets[0].ip_addresses
}
output "ga_arn" {
value = aws_globalaccelerator_accelerator.main.id
}
output "waf_web_acl_arn" {
value = aws_wafv2_web_acl.edge.arn
}Why capture ga_static_ips?
The ga_static_ips output contains the two Anycast IPv4 addresses that go directly into JPMorgan's firewall allowlist. These IPs are permanently assigned to the accelerator for its lifetime. Sage will also reference them in the FinOps report and the board presentation.
- Save both modules/accelerator/main.tf and modules/accelerator/outputs.tf.
- Apply the accelerator module by running:
terraform apply -auto-approve- After apply completes, capture the static IPs by running:
terraform output ga_static_ipsYou should see two IPv4 addresses in the output. These are the permanent Anycast IPs for VaultStream's banking partners.
Apply failing on NLB or GA resources?
Confirm your subnet_ids_east and subnet_ids_west variables are populated with valid subnet IDs from your default VPCs. NLBs require at least one subnet. Also confirm Roy's Lambda ARNs are being passed correctly from module.origin.lambda_arn_east and module.origin.lambda_arn_west.
help me debug Terraform apply errors for NLB and Global Accelerator
Validate Global Accelerator and WAF with AWS CLI
Atlas confirms every resource is live by running AWS CLI commands. GA commands require --region us-west-2 because that is where the GA API lives. WAF CLOUDFRONT-scoped commands require --region us-east-1.
- List all Global Accelerator resources by running:
aws globalaccelerator list-accelerators --region us-west-2What should I see?
You should see an accelerator named vaultstream-ga with Status: DEPLOYED and an IpSets array containing two IPv4 addresses.
- Verify the static IPs match your Terraform output by running:
aws globalaccelerator describe-accelerator --accelerator-arn [[GA_ARN="your Global Accelerator ARN from terraform output"]] --query "Accelerator.IpSets" --region us-west-2- Confirm the IpSets IpAddresses match the two IPs from terraform output ga_static_ips exactly.
- List the WAF WebACLs scoped to CloudFront by running:
aws wafv2 list-web-acls --scope CLOUDFRONT --region us-east-1You should see vaultstream-edge-waf in the WebACLs list with its ID and ARN.
Why different regions for GA and WAF?
Global Accelerator's control plane lives exclusively in us-west-2. Every GA CLI command must target that region. WAF with scope=CLOUDFRONT must be created and queried in us-east-1 because CloudFront is a global service administered from that region.
🙋♀️ Commands returning empty results?
Double-check the --region flag. Running GA commands against us-east-1 or WAF commands against us-west-2 returns empty lists. Also confirm terraform apply completed without errors. Resources that fail mid-apply will not appear in CLI queries.
help me troubleshoot empty AWS CLI results for Global Accelerator or WAF
✔️ Awesome, I've got everything!
Great. Double check that terraform output ga_static_ips shows two IPs and that the WAF list command shows vaultstream-edge-waf.
ⓧ I'd like to double check the full code
Here is the complete modules/accelerator/main.tf:
# NLB east
resource "aws_lb" "nlb_east" {
provider = aws.us_east_1
name = "vaultstream-nlb-east"
load_balancer_type = "network"
internal = false
subnets = var.subnet_ids_east
tags = { Project = "vaultstream-edge" }
}
resource "aws_lb_target_group" "lambda_east" {
provider = aws.us_east_1
name = "vaultstream-tg-east"
target_type = "lambda"
tags = { Project = "vaultstream-edge" }
}
resource "aws_lambda_permission" "nlb_east" {
provider = aws.us_east_1
statement_id = "AllowNLBEast"
action = "lambda:InvokeFunction"
function_name = var.lambda_arn_east
principal = "elasticloadbalancing.amazonaws.com"
source_arn = aws_lb_target_group.lambda_east.arn
}
resource "aws_lb_target_group_attachment" "lambda_east" {
provider = aws.us_east_1
target_group_arn = aws_lb_target_group.lambda_east.arn
target_id = var.lambda_arn_east
depends_on = [aws_lambda_permission.nlb_east]
}
resource "aws_lb_listener" "nlb_east" {
provider = aws.us_east_1
load_balancer_arn = aws_lb.nlb_east.arn
port = 443
protocol = "TCP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.lambda_east.arn
}
}
# NLB west
resource "aws_lb" "nlb_west" {
provider = aws.eu_west_1
name = "vaultstream-nlb-west"
load_balancer_type = "network"
internal = false
subnets = var.subnet_ids_west
tags = { Project = "vaultstream-edge" }
}
resource "aws_lb_target_group" "lambda_west" {
provider = aws.eu_west_1
name = "vaultstream-tg-west"
target_type = "lambda"
tags = { Project = "vaultstream-edge" }
}
resource "aws_lambda_permission" "nlb_west" {
provider = aws.eu_west_1
statement_id = "AllowNLBWest"
action = "lambda:InvokeFunction"
function_name = var.lambda_arn_west
principal = "elasticloadbalancing.amazonaws.com"
source_arn = aws_lb_target_group.lambda_west.arn
}
resource "aws_lb_target_group_attachment" "lambda_west" {
provider = aws.eu_west_1
target_group_arn = aws_lb_target_group.lambda_west.arn
target_id = var.lambda_arn_west
depends_on = [aws_lambda_permission.nlb_west]
}
resource "aws_lb_listener" "nlb_west" {
provider = aws.eu_west_1
load_balancer_arn = aws_lb.nlb_west.arn
port = 443
protocol = "TCP"
default_action {
type = "forward"
target_group_arn = aws_lb_target_group.lambda_west.arn
}
}
# Global Accelerator
resource "aws_globalaccelerator_accelerator" "main" {
name = "vaultstream-ga"
ip_address_type = "IPV4"
enabled = true
tags = { Project = "vaultstream-edge" }
}
resource "aws_globalaccelerator_listener" "main" {
accelerator_arn = aws_globalaccelerator_accelerator.main.id
protocol = "TCP"
port_range {
from_port = 443
to_port = 443
}
}
resource "aws_globalaccelerator_endpoint_group" "east" {
listener_arn = aws_globalaccelerator_listener.main.id
endpoint_group_region = "us-east-1"
endpoint_configuration {
endpoint_id = aws_lb.nlb_east.arn
weight = 100
}
}
resource "aws_globalaccelerator_endpoint_group" "west" {
listener_arn = aws_globalaccelerator_listener.main.id
endpoint_group_region = "eu-west-1"
endpoint_configuration {
endpoint_id = aws_lb.nlb_west.arn
weight = 100
}
}
# WAF WebACL (scope=CLOUDFRONT must be us-east-1)
resource "aws_wafv2_web_acl" "edge" {
provider = aws.us_east_1
name = "vaultstream-edge-waf"
scope = "CLOUDFRONT"
tags = { Project = "vaultstream-edge" }
default_action { allow {} }
rule {
name = "RateLimitGeoXX"
priority = 1
action { block {} }
statement {
rate_based_statement {
limit = 1000
aggregate_key_type = "IP"
scope_down_statement {
geo_match_statement {
country_codes = ["XX"]
}
}
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "RateLimitGeoXX"
sampled_requests_enabled = true
}
}
rule {
name = "BlockBadBot"
priority = 2
action { block {} }
statement {
byte_match_statement {
search_string = "badbot/1.0"
field_to_match {
single_header { name = "user-agent" }
}
text_transformation {
priority = 0
type = "LOWERCASE"
}
positional_constraint = "CONTAINS"
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "BlockBadBot"
sampled_requests_enabled = true
}
}
visibility_config {
cloudwatch_metrics_enabled = true
metric_name = "VaultStreamEdgeWAF"
sampled_requests_enabled = true
}
}Here is modules/accelerator/variables.tf:
variable "lambda_arn_east" {
type = string
}
variable "lambda_arn_west" {
type = string
}
variable "subnet_ids_east" {
type = list(string)
}
variable "subnet_ids_west" {
type = list(string)
}Here is modules/accelerator/outputs.tf:
output "ga_static_ips" {
value = aws_globalaccelerator_accelerator.main.ip_sets[0].ip_addresses
}
output "ga_arn" {
value = aws_globalaccelerator_accelerator.main.id
}
output "waf_web_acl_arn" {
value = aws_wafv2_web_acl.edge.arn
}Atlas has delivered. Two static Anycast IPs are locked in for JPMorgan's allowlist. WAF is live at the edge. Next up, Sage writes the validation scripts, leadership documentation, and the board presentation.
Write Validation Scripts and Leadership Documentation
Roy, Nova, and Atlas have delivered the infrastructure. Every module is applied, every output is available, and the architecture is live across two regions.
Now Sage takes the stage. The director meeting is at 2:00 PM, and leadership does not read Terraform files. Sage produces the validation scripts that prove sub-60-second failover, the ADRs that satisfy the auditors, and the HTML presentation that tells the complete story to both the CFO and the CISO in one tab each.
In this step, get ready to:
- Write PowerShell validation scripts that kill the primary region, measure failover timing, and restore service.
- Produce ADRs, Mermaid topology, FinOps report, and implementation plan for the leadership package.
- Build an HTML presentation with dual-audience tabs for the 2:00 PM director briefing.
Write the failover validation scripts
The VP of Engineering will not accept "it probably fails over." You need a script that kills the primary region, loops requests with cache-busting, and measures the exact number of seconds until traffic shifts.
- Instruct Sage to create scripts/validate.ps1 by providing this prompt in Cursor Composer:
Sage: Write scripts/validate.ps1 (PowerShell). It must:
1. Get the CloudFront domain from `terraform output -raw cloudfront_domain_name`
2. Kill primary by running: aws lambda put-function-concurrency --function-name edge-origin-east --reserved-concurrent-executions 0
3. Loop requests to https://<cloudfront>/?t=<unix_timestamp> every 2 seconds (cache-busting query param)
4. Detect when response.region flips from us-east-1 to eu-west-1
5. Log the failover timestamp and calculate delta
6. Restore primary: aws lambda delete-function-concurrency --function-name edge-origin-east
7. PASS if delta < 60s, FAIL otherwise
8. Timeout after 120 seconds if no failover detectedSage produces the following script. Verify it matches:
$cloudfrontUrl = (terraform output -raw cloudfront_domain_name)
$functionNameEast = "edge-origin-east"
$maxWaitSeconds = 120
$failoverDetected = $false
$failoverTime = $null
Write-Host "Starting failover validation..."
Write-Host "Killing primary region (us-east-1)..."
aws lambda put-function-concurrency --function-name $functionNameEast --reserved-concurrent-executions 0
$startTime = Get-Date
while ((Get-Date) - $startTime).TotalSeconds -lt $maxWaitSeconds {
$t = [int][double]::Parse((Get-Date -UFormat %s))
try {
$response = Invoke-RestMethod -Uri "https://$cloudfrontUrl/?t=$t" -Method GET
Write-Host "$(Get-Date -Format 'HH:mm:ss') Region: $($response.region)"
if ($response.region -eq "eu-west-1" -and -not $failoverDetected) {
$failoverDetected = $true
$failoverTime = (Get-Date) - $startTime
Write-Host "FAILOVER DETECTED at $([math]::Round($failoverTime.TotalSeconds, 1))s"
}
} catch {
Write-Host "$(Get-Date -Format 'HH:mm:ss') Request failed: $_"
}
Start-Sleep -Seconds 2
}
Write-Host "Restoring primary region..."
aws lambda delete-function-concurrency --function-name $functionNameEast
if ($failoverDetected) {
$delta = [math]::Round($failoverTime.TotalSeconds, 1)
Write-Host "Failover duration: ${delta}s"
if ($delta -lt 60) {
Write-Host "PASS: Failover < 60s SLA target"
} else {
Write-Host "FAIL: Failover exceeded 60s SLA target"
}
} else {
Write-Host "FAIL: No failover detected within ${maxWaitSeconds}s"
}What does this script do?
The script automates the entire failover test lifecycle in one run:
- It kills the primary Lambda by setting reserved concurrency to 0. This makes every invocation throttle, which returns 5xx to CloudFront.
- The ?t=<timestamp> query parameter busts the CloudFront edge cache. Without it, you would test cached responses, not live origin failover.
- When the response region flips to eu-west-1, the script records the exact failover duration. Target is under 60 seconds.
- After detection (or timeout), it restores the primary by removing the concurrency limit.
- Now instruct Sage to create the standalone kill and restore scripts. Create scripts/kill-primary.ps1 with this content:
Write-Host "Killing primary Lambda (us-east-1) by setting concurrency to 0..."
aws lambda put-function-concurrency --function-name edge-origin-east --reserved-concurrent-executions 0
Write-Host "Primary killed. Monitor CloudFront for failover to eu-west-1."- Create scripts/restore-primary.ps1 with this content:
Write-Host "Restoring primary Lambda (us-east-1) by removing concurrency limit..."
aws lambda delete-function-concurrency --function-name edge-origin-east
Write-Host "Primary restored."Why separate kill and restore scripts?
The full validate.ps1 runs the complete cycle automatically. The standalone scripts let you kill and restore independently during the integration test in the next step, or when red-teaming with Roy.
- Save all three files in your scripts/ directory.
- Confirm the scripts exist by running:
ls scripts/You should see validate.ps1, kill-primary.ps1, and restore-primary.ps1 in the output.
Don't see the scripts?
- Make sure you created the scripts/ directory first. Run mkdir scripts if it does not exist.
- Verify Sage saved the files to the correct path. The files should be in scripts/ at the project root, not inside a module directory.
Help me debug why my PowerShell scripts are not appearing in the scripts directory.
Write the ADRs and operational documentation
Auditors do not trust verbal explanations. Every architectural decision needs a written record in Nygard ADR format: Title, Status, Context, Decision, Consequences.
- Instruct Sage to produce four ADRs in docs/adr/. Provide this prompt:
Sage: Write 4 ADRs in docs/adr/ using Nygard format (Title, Status [Accepted], Context, Decision, Consequences):
- ADR-001: CloudFront vs Global Accelerator for Static IP (banking partners need allowlisted IPs, PCI-DSS Req 1.3, SOC2 CC6.6)
- ADR-002: Lambda Function URL vs ALB as CloudFront Origin (FinOps: $0 vs $1.08/day per region)
- ADR-003: CloudFront Functions vs Lambda@Edge for Header Manipulation (submillisecond vs 5-15min propagation)
- ADR-004: Route 53 Failover vs Multivalue for Primary Use Case (SAP-C02 D2.4 trap pattern)Sage produces ADR-001.md as the first decision record. Verify it matches:
# ADR-001: CloudFront vs Global Accelerator for Static IP
**Status:** Accepted
**Context:** Banking partners (e.g., JPMorgan) require static IP addresses for firewall allowlisting under PCI-DSS Requirement 1.3 and SOC2 CC6.6. CloudFront uses dynamic Anycast IPs that change without notice. Global Accelerator provides two static Anycast IPv4 addresses that never change.
**Decision:** Deploy Global Accelerator with NLB intermediaries (GA cannot target Lambda Function URLs directly) to provide static IPs for regulated banking partners. CloudFront remains the primary edge for caching and edge compute. GA is the compliance-driven overlay.
**Consequences:** +$18/month for GA. NLB bridge adds 12 Terraform resources. Static IPs satisfy banking partner allowlist requirements. GA provides ~30s health-check-driven failover independent of CloudFront.What is the Nygard ADR format?
An Architecture Decision Record captures the "why" behind a design choice. The Nygard format has five sections: Title (what was decided), Status (accepted/superseded/deprecated), Context (forces at play), Decision (what you chose), and Consequences (trade-offs).
ADRs are what auditors look for during SOC2 reviews. They prove you made deliberate choices, not accidental ones.
- Verify Sage also produces ADR-002.md, ADR-003.md, and ADR-004.md in the same docs/adr/ directory.
- Now instruct Sage to produce the Mermaid topology. Create docs/topology.md with this content:
```mermaid
flowchart TD
Viewer(["Browser / API Client"])
CF["CloudFront Distribution"]
OG["Origin Group\n(Failover)"]
LE["Lambda East\nus-east-1"]
LW["Lambda West\neu-west-1"]
R53["Route 53\nHosted Zone"]
HC1["Health Check\nus-east-1"]
HC2["Health Check\neu-west-1"]
GA["Global Accelerator\n(Static IPs)"]
NLBE["NLB East"]
NLBW["NLB West"]
WAF["WAF WebACL"]
CFF["CF Function\nstrip-legacy-header"]
LAE["Lambda@Edge\nsigned-url-validator"]
Viewer --> WAF --> CF
CF --> CFF
CFF --> LAE
LAE --> OG
OG -->|Primary| LE
OG -->|Failover| LW
R53 --> HC1 --> LE
R53 --> HC2 --> LW
Viewer --> GA
GA --> NLBE --> LE
GA --> NLBW --> LW
```Why Mermaid for the topology?
Mermaid diagrams are text-based, git-trackable, and render in GitHub, VS Code, and Mermaid Live. No Visio license, no binary file, no stale screenshot. Paste this into mermaid.live to verify it renders correctly.
- Next, instruct Sage to produce the FinOps report. Create docs/finops.md with this content:
# VaultStream Edge — FinOps Report
| Resource | Free Tier Type | Monthly Cost | Lab Cost (90 min) |
|---|---|---|---|
| Lambda (2 functions) | Always Free (1M req/mo) | $0 | $0 |
| Lambda Function URLs | Always Free (included) | $0 | $0 |
| CloudFront | Always Free (1TB/10M req) | $0 | $0 |
| CloudFront Functions | Always Free (2M invocations) | $0 | $0 |
| Lambda@Edge | NO free tier | ~$0.60/1M requests | $0.01 |
| Route 53 zone | None | $0.50 | $0.50 |
| Route 53 health checks (2) | None | $1.00 | $0.05 |
| Global Accelerator | None | $18 ($0.025/hr) | $0.04 |
| NLB (2, GA bridge) | 12-Month Free (750 hrs) | $0 if first year | $0 |
| WAF WebACL | None | $5 + $1/rule | $0.01 |
| **TOTAL** | | **$24.50/month** | **$0.61** |
**Key insight:** Lambda Function URLs replace ALBs ($64.80/month savings). CloudFront free tier covers 1TB transfer + 10M requests. Total lab cost $0.61 — teardown immediately after validation.What does the CFO see here?
The free-tier classification column is the key. It shows the CFO which resources cost nothing because of AWS always-free tiers, which are covered by 12-month trial, and which have ongoing charges. The $0.61 lab cost proves the PoC budget is zero.
- Finally, instruct Sage to produce the implementation plan. Create docs/implementation-plan.md with this content:
# Implementation Plan — 90-Minute Sprint
## Swim Lanes
| Time | Roy (Origins/DNS) | Nova (Edge/CDN) | Atlas (Network/Compliance) | Sage (Docs) |
|---|---|---|---|---|
| 0-10 | Setup, init, brief squad | Setup | Setup | Setup |
| 10-18 | Write + apply Lambda + Function URLs | Write CF module (no apply) | Write GA+NLB skeleton (no apply) | Start ADRs + Mermaid |
| 18-35 | Route 53 zone + 5 policies + health checks | Apply CF after Roy outputs | Apply GA+NLB after Roy ARNs | Continue ADRs |
| 35-50 | Validate + outputs for Sage | Validate CloudFront | Validate GA + WAF | FinOps report |
| 50-75 | Red-team kill test | Confirm origin group intact | Confirm GA health transition | Presentation HTML |
| 75-90 | Integration test | Integration test | Integration test | Final review |
## Critical Path
Roy (Lambda deploy) → Nova (CF apply) + Atlas (GA apply) → Integration → Teardown
## Dependency Arrows
- Roy outputs function_url_east/west → Nova CF origins
- Roy outputs lambda_arn_east/west → Atlas NLB target groups
- All Terraform outputs → Sage presentation.html
- Integration test pass → terraform destroy- Save all documentation files.
- Confirm the docs structure by running:
ls docs/ docs/adr/You should see topology.md, finops.md, implementation-plan.md, and presentation.html in docs/. The docs/adr/ directory should contain ADR-001.md through ADR-004.md.
Missing ADR files?
- Ensure the docs/adr/ nested directory exists. Run mkdir -p docs/adr to create both levels.
- Check that Sage saved each ADR as a separate file, not all content in one file.
Help me figure out why my docs directory structure is wrong.
Build the director briefing presentation
The director opens one file at 2:00 PM. It must speak CFO (cost, SLA math) on one tab and VP Engineering (topology, tech debt, hardening) on another. No PowerPoint. One HTML file that opens in any browser.
- Instruct Sage to create docs/presentation.html as a single-page app with tabbed Business/Technical navigation:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>VaultStream Edge Failover — Director Briefing</title>
<script src="https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js"></script>
<style>
body { font-family: sans-serif; max-width: 960px; margin: 0 auto; padding: 2rem; }
.tabs { display: flex; gap: 1rem; margin-bottom: 1rem; }
.tab-btn { padding: 0.5rem 1.5rem; cursor: pointer; border: 1px solid #ccc; background: #f5f5f5; }
.tab-btn.active { background: #0066cc; color: white; border-color: #0066cc; }
.tab-content { display: none; }
.tab-content.active { display: block; }
table { border-collapse: collapse; width: 100%; }
th, td { border: 1px solid #ddd; padding: 0.5rem; text-align: left; }
th { background: #f0f0f0; }
</style>
</head>
<body>
<h1>VaultStream Edge Failover Reference Architecture</h1>
<p><em>Director Briefing — 2:00 PM</em></p>
<div class="tabs">
<button class="tab-btn active" onclick="showTab('business')">Business</button>
<button class="tab-btn" onclick="showTab('technical')">Technical</button>
</div>
<div id="business" class="tab-content active">
<h2>Executive Summary</h2>
<p>A 47-minute outage caused by a single misconfigured DNS setting cost VaultStream $2.3M in SLA penalties. This sprint delivers a fully automated, IaC-managed multi-region failover architecture that recovers in under 60 seconds, costs $0.61 to prove, and $24.50/month in staging. Banking partners have their static IPs. Auditors have their ADRs.</p>
<h2>Cost</h2>
<table>
<tr><th>Environment</th><th>Cost</th></tr>
<tr><td>Lab (90 min, destroyed)</td><td>$0.61</td></tr>
<tr><td>Staging (monthly)</td><td>$24.50</td></tr>
<tr><td>After teardown</td><td>$0</td></tr>
</table>
<h2>SLA Math</h2>
<p>99.95% uptime = 21.9 minutes allowed downtime/month. Proven recovery: <60 seconds = single incident consumes <4.6% of monthly budget. Original incident: 47 minutes = 214% of monthly budget consumed in one event.</p>
<h2>Compliance</h2>
<ul>
<li>2 static Anycast IPs captured for banking partner allowlists (PCI-DSS Req 1.3)</li>
<li>WAF WebACL active at edge (SOC2 CC6.6)</li>
<li>100% IaC-managed — zero console drift, full audit trail</li>
<li>ADRs document every architectural decision</li>
</ul>
</div>
<div id="technical" class="tab-content">
<h2>Topology</h2>
<div class="mermaid">
flowchart TD
Viewer(["Browser / API Client"])
CF["CloudFront Distribution"]
OG["Origin Group (Failover)"]
LE["Lambda East us-east-1"]
LW["Lambda West eu-west-1"]
GA["Global Accelerator (Static IPs)"]
WAF["WAF WebACL"]
Viewer --> WAF --> CF --> OG
OG -->|Primary| LE
OG -->|Failover| LW
Viewer --> GA --> LE
GA --> LW
</div>
<h2>Failover Timing</h2>
<p>CloudFront origin group: 3s timeout, 1 attempt, failover on 5xx. DNS failover (Route 53): 10s health check interval, 3 failures = 30s detection. Total measured recovery: <60s.</p>
<h2>Tech Debt Register</h2>
<ol>
<li>Replace Lambda Function URLs with ALBs for production (connection draining, WAF at origin)</li>
<li>Add Route 53 ARC for readiness checks and zonal shift</li>
<li>Enable CloudFront Origin Shield</li>
<li>Implement signed cookies for session-based access</li>
<li>Add CloudWatch Synthetics canaries (5+ geo locations)</li>
<li>Enable AWS Config conformance packs (PCI-DSS, SOC2)</li>
<li>Migrate Terraform state to S3 + DynamoDB locking</li>
<li>Add Terraform workspace separation or Terragrunt</li>
<li>Enable Shield Advanced on GA static IPs</li>
<li>CloudFront real-time logging to Kinesis Firehose -> S3</li>
</ol>
</div>
<script>
mermaid.initialize({ startOnLoad: true });
function showTab(name) {
document.querySelectorAll('.tab-content').forEach(el => el.classList.remove('active'));
document.querySelectorAll('.tab-btn').forEach(el => el.classList.remove('active'));
document.getElementById(name).classList.add('active');
event.target.classList.add('active');
}
</script>
</body>
</html>What does this presentation do?
The Business tab shows cost, SLA math, and compliance posture. The CFO sees $0.61. The CISO sees PCI-DSS Req 1.3 and SOC2 CC6.6 satisfied.
The Technical tab renders the Mermaid topology diagram live in the browser, shows failover timing measurements, and lists all 10 tech debt items with the production hardening roadmap. The VP Engineering sees depth.
- Save docs/presentation.html.
- Open docs/presentation.html in your browser. You should see the title "VaultStream Edge Failover Reference Architecture" with Business and Technical tab buttons.
- Click the Technical tab. The Mermaid diagram should render showing the full topology with CloudFront, Origin Group, Lambda East/West, Global Accelerator, and WAF nodes.
Mermaid diagram not rendering?
- The Mermaid CDN script requires an internet connection. If you are offline, the diagram div will remain as raw text.
- Check your browser console for errors. The most common issue is an ad blocker preventing the CDN script from loading.
- Verify the Mermaid syntax is correct by pasting the flowchart block into mermaid.live directly.
Help me debug why the Mermaid diagram is not rendering in my presentation.
✔️ Awesome, I've got everything!
Great. Double check you have saved all files in scripts/ and docs/.
ⓧ I'd like to double check the full code
Here is scripts/validate.ps1 in full:
$cloudfrontUrl = (terraform output -raw cloudfront_domain_name)
$functionNameEast = "edge-origin-east"
$maxWaitSeconds = 120
$failoverDetected = $false
$failoverTime = $null
Write-Host "Starting failover validation..."
Write-Host "Killing primary region (us-east-1)..."
aws lambda put-function-concurrency --function-name $functionNameEast --reserved-concurrent-executions 0
$startTime = Get-Date
while ((Get-Date) - $startTime).TotalSeconds -lt $maxWaitSeconds {
$t = [int][double]::Parse((Get-Date -UFormat %s))
try {
$response = Invoke-RestMethod -Uri "https://$cloudfrontUrl/?t=$t" -Method GET
Write-Host "$(Get-Date -Format 'HH:mm:ss') Region: $($response.region)"
if ($response.region -eq "eu-west-1" -and -not $failoverDetected) {
$failoverDetected = $true
$failoverTime = (Get-Date) - $startTime
Write-Host "FAILOVER DETECTED at $([math]::Round($failoverTime.TotalSeconds, 1))s"
}
} catch {
Write-Host "$(Get-Date -Format 'HH:mm:ss') Request failed: $_"
}
Start-Sleep -Seconds 2
}
Write-Host "Restoring primary region..."
aws lambda delete-function-concurrency --function-name $functionNameEast
if ($failoverDetected) {
$delta = [math]::Round($failoverTime.TotalSeconds, 1)
Write-Host "Failover duration: ${delta}s"
if ($delta -lt 60) {
Write-Host "PASS: Failover < 60s SLA target"
} else {
Write-Host "FAIL: Failover exceeded 60s SLA target"
}
} else {
Write-Host "FAIL: No failover detected within ${maxWaitSeconds}s"
}Here is scripts/kill-primary.ps1:
Write-Host "Killing primary Lambda (us-east-1) by setting concurrency to 0..."
aws lambda put-function-concurrency --function-name edge-origin-east --reserved-concurrent-executions 0
Write-Host "Primary killed. Monitor CloudFront for failover to eu-west-1."Here is scripts/restore-primary.ps1:
Write-Host "Restoring primary Lambda (us-east-1) by removing concurrency limit..."
aws lambda delete-function-concurrency --function-name edge-origin-east
Write-Host "Primary restored."Here is docs/adr/ADR-001.md:
# ADR-001: CloudFront vs Global Accelerator for Static IP
**Status:** Accepted
**Context:** Banking partners (e.g., JPMorgan) require static IP addresses for firewall allowlisting under PCI-DSS Requirement 1.3 and SOC2 CC6.6. CloudFront uses dynamic Anycast IPs that change without notice. Global Accelerator provides two static Anycast IPv4 addresses that never change.
**Decision:** Deploy Global Accelerator with NLB intermediaries (GA cannot target Lambda Function URLs directly) to provide static IPs for regulated banking partners. CloudFront remains the primary edge for caching and edge compute. GA is the compliance-driven overlay.
**Consequences:** +$18/month for GA. NLB bridge adds 12 Terraform resources. Static IPs satisfy banking partner allowlist requirements. GA provides ~30s health-check-driven failover independent of CloudFront.Here is docs/adr/ADR-002.md:
# ADR-002: Lambda Function URL vs ALB as CloudFront Origin
**Status:** Accepted
**Context:** CloudFront requires an HTTP origin. Traditional pattern uses ALB ($1.08/day per region = $64.80/month for two regions). Lambda Function URLs provide HTTPS endpoints at $0 cost (always-free tier).
**Decision:** Use Lambda Function URLs as CloudFront custom origins. Saves $64.80/month in staging, eliminates ALB config overhead, and provides identical failover signal (5xx on unhealthy origin triggers CloudFront origin group failover).
**Consequences:** No connection draining, no target group health checks at origin level. Acceptable for proof-of-concept and staging. Production hardening item: replace with ALB for connection draining and WAF at origin.Here is docs/adr/ADR-003.md:
# ADR-003: CloudFront Functions vs Lambda@Edge for Header Manipulation
**Status:** Accepted
**Context:** Legacy banking clients send X-Legacy-Client header that must be stripped before reaching origins. Two options: CloudFront Functions (JS, submillisecond, viewer-request/response only, 10KB limit, no network access) or Lambda@Edge (Node.js, 5-15min propagation, origin-request/response + viewer-request/response, full Node.js runtime).
**Decision:** Use CloudFront Function for X-Legacy-Client header stripping (viewer-request). Use Lambda@Edge for signed URL validation (origin-request). CF Functions are purpose-built for simple header manipulation: zero propagation delay, submillisecond execution, always deployed.
**Consequences:** CF Function + Lambda@Edge on same behavior is allowed when on different event types. CF Functions cannot validate signed URLs (no crypto library, no network access). Lambda@Edge cannot be used for viewer-request when CF Function is already assigned to that event.Here is docs/adr/ADR-004.md:
# ADR-004: Route 53 Failover vs Multivalue for Primary Use Case
**Status:** Accepted
**Context:** SAP-C02 trap: candidates confuse multivalue answer routing with failover routing. Multivalue returns up to 8 healthy IPs for DNS-level load distribution. Failover returns PRIMARY record unless health check fails, then returns SECONDARY. They are not interchangeable.
**Decision:** Use Route 53 failover routing policy for active-passive DR (failover.domain). Use multivalue for DNS-level spreading demonstration only (multivalue.domain). The $2.3M VaultStream outage was caused by evaluate_target_health=false on the alias record, not by choosing the wrong routing policy.
**Consequences:** Failover policy with evaluate_target_health=true is mandatory for automated DNS-level failover. Multivalue is NOT a failover mechanism. This distinction is tested on SAP-C02 D2.4.Here is docs/topology.md:
```mermaid
flowchart TD
Viewer(["Browser / API Client"])
CF["CloudFront Distribution"]
OG["Origin Group\n(Failover)"]
LE["Lambda East\nus-east-1"]
LW["Lambda West\neu-west-1"]
R53["Route 53\nHosted Zone"]
HC1["Health Check\nus-east-1"]
HC2["Health Check\neu-west-1"]
GA["Global Accelerator\n(Static IPs)"]
NLBE["NLB East"]
NLBW["NLB West"]
WAF["WAF WebACL"]
CFF["CF Function\nstrip-legacy-header"]
LAE["Lambda@Edge\nsigned-url-validator"]
Viewer --> WAF --> CF
CF --> CFF
CFF --> LAE
LAE --> OG
OG -->|Primary| LE
OG -->|Failover| LW
R53 --> HC1 --> LE
R53 --> HC2 --> LW
Viewer --> GA
GA --> NLBE --> LE
GA --> NLBW --> LW
```Here is docs/finops.md:
# VaultStream Edge — FinOps Report
| Resource | Free Tier Type | Monthly Cost | Lab Cost (90 min) |
|---|---|---|---|
| Lambda (2 functions) | Always Free (1M req/mo) | $0 | $0 |
| Lambda Function URLs | Always Free (included) | $0 | $0 |
| CloudFront | Always Free (1TB/10M req) | $0 | $0 |
| CloudFront Functions | Always Free (2M invocations) | $0 | $0 |
| Lambda@Edge | NO free tier | ~$0.60/1M requests | $0.01 |
| Route 53 zone | None | $0.50 | $0.50 |
| Route 53 health checks (2) | None | $1.00 | $0.05 |
| Global Accelerator | None | $18 ($0.025/hr) | $0.04 |
| NLB (2, GA bridge) | 12-Month Free (750 hrs) | $0 if first year | $0 |
| WAF WebACL | None | $5 + $1/rule | $0.01 |
| **TOTAL** | | **$24.50/month** | **$0.61** |
**Key insight:** Lambda Function URLs replace ALBs ($64.80/month savings). CloudFront free tier covers 1TB transfer + 10M requests. Total lab cost $0.61 — teardown immediately after validation.Here is docs/implementation-plan.md:
# Implementation Plan — 90-Minute Sprint
## Swim Lanes
| Time | Roy (Origins/DNS) | Nova (Edge/CDN) | Atlas (Network/Compliance) | Sage (Docs) |
|---|---|---|---|---|
| 0-10 | Setup, init, brief squad | Setup | Setup | Setup |
| 10-18 | Write + apply Lambda + Function URLs | Write CF module (no apply) | Write GA+NLB skeleton (no apply) | Start ADRs + Mermaid |
| 18-35 | Route 53 zone + 5 policies + health checks | Apply CF after Roy outputs | Apply GA+NLB after Roy ARNs | Continue ADRs |
| 35-50 | Validate + outputs for Sage | Validate CloudFront | Validate GA + WAF | FinOps report |
| 50-75 | Red-team kill test | Confirm origin group intact | Confirm GA health transition | Presentation HTML |
| 75-90 | Integration test | Integration test | Integration test | Final review |
## Critical Path
Roy (Lambda deploy) → Nova (CF apply) + Atlas (GA apply) → Integration → Teardown
## Dependency Arrows
- Roy outputs function_url_east/west → Nova CF origins
- Roy outputs lambda_arn_east/west → Atlas NLB target groups
- All Terraform outputs → Sage presentation.html
- Integration test pass → terraform destroyHere is docs/presentation.html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>VaultStream Edge Failover — Director Briefing</title>
<script src="https://cdn.jsdelivr.net/npm/mermaid/dist/mermaid.min.js"></script>
<style>
body { font-family: sans-serif; max-width: 960px; margin: 0 auto; padding: 2rem; }
.tabs { display: flex; gap: 1rem; margin-bottom: 1rem; }
.tab-btn { padding: 0.5rem 1.5rem; cursor: pointer; border: 1px solid #ccc; background: #f5f5f5; }
.tab-btn.active { background: #0066cc; color: white; border-color: #0066cc; }
.tab-content { display: none; }
.tab-content.active { display: block; }
table { border-collapse: collapse; width: 100%; }
th, td { border: 1px solid #ddd; padding: 0.5rem; text-align: left; }
th { background: #f0f0f0; }
</style>
</head>
<body>
<h1>VaultStream Edge Failover Reference Architecture</h1>
<p><em>Director Briefing — 2:00 PM</em></p>
<div class="tabs">
<button class="tab-btn active" onclick="showTab('business')">Business</button>
<button class="tab-btn" onclick="showTab('technical')">Technical</button>
</div>
<div id="business" class="tab-content active">
<h2>Executive Summary</h2>
<p>A 47-minute outage caused by a single misconfigured DNS setting cost VaultStream $2.3M in SLA penalties. This sprint delivers a fully automated, IaC-managed multi-region failover architecture that recovers in under 60 seconds, costs $0.61 to prove, and $24.50/month in staging. Banking partners have their static IPs. Auditors have their ADRs.</p>
<h2>Cost</h2>
<table>
<tr><th>Environment</th><th>Cost</th></tr>
<tr><td>Lab (90 min, destroyed)</td><td>$0.61</td></tr>
<tr><td>Staging (monthly)</td><td>$24.50</td></tr>
<tr><td>After teardown</td><td>$0</td></tr>
</table>
<h2>SLA Math</h2>
<p>99.95% uptime = 21.9 minutes allowed downtime/month. Proven recovery: <60 seconds = single incident consumes <4.6% of monthly budget. Original incident: 47 minutes = 214% of monthly budget consumed in one event.</p>
<h2>Compliance</h2>
<ul>
<li>2 static Anycast IPs captured for banking partner allowlists (PCI-DSS Req 1.3)</li>
<li>WAF WebACL active at edge (SOC2 CC6.6)</li>
<li>100% IaC-managed — zero console drift, full audit trail</li>
<li>ADRs document every architectural decision</li>
</ul>
</div>
<div id="technical" class="tab-content">
<h2>Topology</h2>
<div class="mermaid">
flowchart TD
Viewer(["Browser / API Client"])
CF["CloudFront Distribution"]
OG["Origin Group (Failover)"]
LE["Lambda East us-east-1"]
LW["Lambda West eu-west-1"]
GA["Global Accelerator (Static IPs)"]
WAF["WAF WebACL"]
Viewer --> WAF --> CF --> OG
OG -->|Primary| LE
OG -->|Failover| LW
Viewer --> GA --> LE
GA --> LW
</div>
<h2>Failover Timing</h2>
<p>CloudFront origin group: 3s timeout, 1 attempt, failover on 5xx. DNS failover (Route 53): 10s health check interval, 3 failures = 30s detection. Total measured recovery: <60s.</p>
<h2>Tech Debt Register</h2>
<ol>
<li>Replace Lambda Function URLs with ALBs for production (connection draining, WAF at origin)</li>
<li>Add Route 53 ARC for readiness checks and zonal shift</li>
<li>Enable CloudFront Origin Shield</li>
<li>Implement signed cookies for session-based access</li>
<li>Add CloudWatch Synthetics canaries (5+ geo locations)</li>
<li>Enable AWS Config conformance packs (PCI-DSS, SOC2)</li>
<li>Migrate Terraform state to S3 + DynamoDB locking</li>
<li>Add Terraform workspace separation or Terragrunt</li>
<li>Enable Shield Advanced on GA static IPs</li>
<li>CloudFront real-time logging to Kinesis Firehose -> S3</li>
</ol>
</div>
<script>
mermaid.initialize({ startOnLoad: true });
function showTab(name) {
document.querySelectorAll('.tab-content').forEach(el => el.classList.remove('active'));
document.querySelectorAll('.tab-btn').forEach(el => el.classList.remove('active'));
document.getElementById(name).classList.add('active');
event.target.classList.add('active');
}
</script>
</body>
</html>Sage has delivered everything the director needs. The validation scripts are ready to prove failover timing. The ADRs satisfy the auditors. The FinOps report satisfies the CFO. The presentation tells the complete story in one HTML file. Next up, you run the full integration test and execute the clean teardown.
Run Integration Tests and Verify Failover
Everything is deployed. Roy's origins are live, Nova's edge distribution is active, Atlas's Global Accelerator is routing, and Sage's documentation is complete.
Now comes the moment of truth. The entire point of this architecture is sub-60-second failover. If it does not fail over in time, the banking partners eat 5xx errors and the CISO calls the CEO again. You will kill the primary region, measure the recovery delta, and prove to the director that this architecture works under pressure.
In this step, get ready to:
- Run the end-to-end failover validation script with cache-busting enabled.
- Confirm CloudFront origin group and Global Accelerator health transitions during the kill test.
- Verify zero orphaned resources across Terraform state and AWS resource tags.
Run the end-to-end failover validation
The validation script kills the primary Lambda, polls CloudFront with a cache-busting query parameter on every request, and measures how long until the response flips from us-east-1 to eu-west-1.
Without the cache-bust parameter, you would test the CloudFront edge cache instead of the actual origin failover. Every request must include a unique ?t= timestamp to force a fresh origin fetch.
- Run the full validation script by executing this command in your terminal:
./scripts/validate.ps1What does this script do?
- Reads your CloudFront domain name from Terraform outputs.
- Kills the primary Lambda (edge-origin-east) by setting its reserved concurrency to 0.
- Polls https://<cloudfront>/?t=<unix_timestamp> every 2 seconds, forcing a fresh origin fetch each time.
- Detects when the response region flips from us-east-1 to eu-west-1 and records the failover duration.
- Restores the primary Lambda by removing the concurrency limit.
- Reports PASS if failover completed in under 60 seconds, FAIL otherwise.
- Watch the terminal output. You should see timestamps with region responses scrolling, then a FAILOVER DETECTED message followed by the measured delta in seconds.
- Confirm the final output reads PASS: Failover < 60s SLA target.
Script shows FAIL or no failover detected?
- CloudFront may still be serving cached responses. Confirm the ?t= parameter is present on every request in the script. If you modified validate.ps1, ensure the timestamp changes on each loop iteration.
- The CloudFront distribution may not have finished deploying. Check aws cloudfront get-distribution --id <id> --query "Distribution.Status" returns Deployed.
- If the failover takes longer than 60 seconds but does eventually happen, your CloudFront origin timeout settings may differ. Verify connection timeout is 3s and connection attempts is 1 in your Terraform config.
Help me debug my failover validation script
Confirm infrastructure behavior during the kill test
A successful failover test is not enough on its own. You need to prove that the failover was transparent (CloudFront did not lose an origin from its config) and that Global Accelerator independently detected the health change.
- While the primary is still killed (or after re-killing it with the dedicated script), verify that CloudFront's origin group still lists both origins by running:
# Kill primary again for this check
./scripts/kill-primary.ps1
# Confirm CloudFront origin group still has both origins
aws cloudfront get-distribution --id [[CF_DIST_ID="your CloudFront distribution ID"]] --query "Distribution.DistributionConfig.OriginGroups"Why check the origin group during a failure?
CloudFront origin failover is transparent. It does not remove the unhealthy origin from the configuration. It simply routes to the secondary when the primary returns a configured error code (500, 502, 503, 504). The origin group must still show both origins, confirming no manual intervention occurred.
- You should see a JSON response showing both your us-east-1 and eu-west-1 Lambda Function URL origins listed in the OriginGroups object.
- Next, confirm that Global Accelerator detected the primary endpoint as unhealthy by running:
# Check GA endpoint group health status (requires the endpoint group ARN from Terraform output)
aws globalaccelerator describe-endpoint-group --endpoint-group-arn [[GA_ENDPOINT_GROUP_ARN="your GA endpoint group ARN from terraform output"]] --region us-west-2What should I see?
The response includes an EndpointDescriptions array. Each endpoint has a HealthState field. The primary NLB endpoint (us-east-1) should show UNHEALTHY because its target Lambda has zero concurrency and returns errors.
Global Accelerator runs its own health checks independently of CloudFront. This proves both failover paths (CDN-level and network-level) detected the outage.
- Restore the primary Lambda by running:
./scripts/restore-primary.ps1- Wait 30 seconds, then re-run the describe-endpoint-group command. The primary endpoint HealthState should return to HEALTHY.
GA endpoint still shows UNHEALTHY after restore?
Global Accelerator health checks take approximately 30 seconds to detect recovery. Wait a full minute and check again. If it persists, verify that the Lambda can actually respond by curling the Function URL directly.
Why is my GA endpoint still showing unhealthy after I restored the Lambda?
Verify zero orphaned resources
The auditors will look at this first. Every resource must be accounted for in Terraform state. If the count of resources in Terraform does not match the count of tagged AWS resources, something was created manually and will not be destroyed cleanly.
- Count the total resources managed by Terraform by running:
# Count all resources in Terraform state
terraform state list | Measure-Object- Note the Count value from the output. This is your expected resource count.
- Cross-reference against AWS by querying all resources tagged with your project tag:
# Cross-reference with AWS resource tags
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=vaultstream-edgeWhy cross-reference Terraform state with AWS tags?
Terraform state tells you what Terraform thinks it manages. The tagging API tells you what actually exists in AWS with your project tag. If these counts diverge, either a resource was created outside Terraform (manual console change) or Terraform lost track of something. Both are audit findings.
Note that some Terraform resources (like IAM policies or provider-level configurations) may not support tagging. A small difference is acceptable. A large difference means orphans.
- Finally, confirm Terraform can cleanly destroy everything by running a dry-run destroy plan:
# Dry-run destroy to confirm clean teardown is possible
terraform plan -destroy- Verify the output shows Plan: 0 to add, 0 to change, N to destroy where N matches your terraform state list count exactly.
- If the numbers match, you have zero orphans. One terraform destroy will remove everything cleanly.
Terraform plan shows unexpected changes?
If you see resources to add or change (not just destroy), something drifted. Run terraform plan (without -destroy) to identify what changed. Common causes: a CloudFront distribution status updated during your test, or a health check threshold incremented.
Help me understand why my terraform plan -destroy shows unexpected changes
The integration test passed. Sub-60-second failover, proven with cache-busted requests. CloudFront origin group intact. Global Accelerator health transitions confirmed. Zero orphans in Terraform state. The leadership package is ready, the director meeting is at 2:00 PM, and you have the numbers to back up every claim. Sprint delivered.
Secret mission
Validate WAF Edge Rules
Atlas deployed WAF rules that rate-limit by geography and block a banned user-agent at the edge. Can you prove they actually work? Validate that your edge security controls block malicious traffic without touching a single line of application code. This is SAP-C02 D2.2 in action.
Clean Up Your Resources
Clean Up Your Resources
Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project costs approximately $25/month if left running, primarily due to Global Accelerator ($18/month) and Route 53 health checks ($1/month).
Cost warning
Global Accelerator alone costs $0.025/hour ($18/month). Route 53 hosted zones cost $0.50/month. WAF adds $5 + $1/rule/month. If you do not intend to keep iterating on this architecture, run terraform destroy -auto-approve immediately.
The total monthly cost if left running is approximately $25. The lab cost for 90 minutes was only $0.61 because you tear down immediately.
Resources you used:
- Lambda functions (2) with Function URLs (us-east-1 and eu-west-1).
- Route 53 hosted zone, health checks (2), and 5 routing policy record sets.
- CloudFront distribution with origin group, CloudFront Function, and Lambda@Edge.
- Global Accelerator with 2 NLBs (intermediary endpoints).
- WAF WebACL with rate-limit and string-match rules.
- Local project files (Terraform configs, scripts, docs, functions).
Keep everything running
No action needed. Choose this if you are still actively building on this architecture or plan to present it at the director meeting.
- Your AWS resources will continue running and incurring costs (~$25/month).
- Your local project files remain in the vaultstream-edge folder on your Desktop.
- Terraform state is stored locally. Do not delete the .terraform directory or terraform.tfstate file, or you will lose the ability to manage these resources.
Pause - I'll come back to this later
Since all resources are in a single Terraform state, the cleanest pause is to destroy and re-apply later. Terraform makes this trivial.
- Run terraform destroy -auto-approve to remove all AWS resources and stop billing.
- Keep your local project files. Your .tf files, scripts, and docs are your complete blueprint.
- When you are ready to resume, run terraform init followed by terraform apply to recreate everything from scratch.
Why destroy instead of stopping?
Most of these resources (Global Accelerator, Route 53 zones, WAF) do not have a "stopped" state. They incur costs simply by existing. Destroying and re-applying is the standard IaC pattern for cost control.
Delete - I don't want to use this again
Remove all project resources and start fresh. This is the expected end state for the lab.
- Navigate to your project directory and run the destroy command:
terraform destroy -auto-approveWhat if destroy takes a while?
CloudFront distributions can take 10-15 minutes to fully disable before deletion. Terraform handles this wait automatically, but your terminal may appear to hang. This is normal.
- Verify the state is empty:
terraform state listThis should return nothing. If any resources remain, run terraform destroy -auto-approve again.
- Confirm AWS resources are gone with these verification commands:
aws cloudfront list-distributions --query "DistributionList.Items[].Id"
aws globalaccelerator list-accelerators --region us-west-2
aws route53 list-hosted-zonesCloudFront and Global Accelerator should return empty results. Route 53 should show only pre-existing zones (if any).
- Delete the local project files (macOS/Linux):
cd ~/Desktop
rm -rf vaultstream-edge- Delete the local project files (Windows):
cd ~/Desktop
Remove-Item -Recurse -Force vaultstream-edgeZero orphans is the only acceptable end state
If terraform state list returns anything after destroy, you have an orphaned resource that will continue incurring costs. Investigate and remove it manually before closing this lab.
That's a Wrap!
That's a Wrap!
The 2:00 PM meeting went well. You delivered a production-grade multi-region edge failover architecture for VaultStream in 90 minutes for $0.61, proved sub-60-second recovery, and walked out with zero orphaned resources.
You've learned how to:
- Deploy a two-region failover architecture with Lambda Function URLs, CloudFront origin groups, and Route 53 health checks that recovers in under 60 seconds.
- Deliver a leadership-ready package including 4 ADRs, a FinOps report, and a single-page HTML presentation for both business and technical audiences.
- Provision Global Accelerator with static Anycast IPs for regulated banking partners requiring PCI-DSS-compliant firewall allowlists.
- Secret Mission: Added WAF edge rules to rate-limit by geography and block banned user-agents at the CloudFront edge without touching application code.
Ready to quiz yourself?