AWS FinOps Command Center

Introduction

30 Second Summary

Imagine opening your electricity bill and seeing it jumped 38% with no explanation. Now multiply that feeling by a million dollars. That is exactly what happens when a Fortune 10 company loses visibility into cloud spending.

In this project, you will build an enterprise FinOps command center that automatically detects cost anomalies, stops runaway spending, enforces tagging governance, and presents executive-ready dashboards. You will deploy the entire system as infrastructure-as-code and assemble a CFO-ready pitch deck.

What You'll Build

A fully automated AWS cost intelligence pipeline with per-customer chargeback dashboards, anomaly-driven remediation, and spend guardrails, all managed via Terraform and visualized in Grafana.

By the end of this project, you'll have:

• A CUR 2.0 data pipeline flowing through Glue and Athena into Grafana executive dashboards with FinOps KPIs, Savings Plans coverage, and 12-month projections.
• An automated remediation loop using Cost Anomaly Detection, EventBridge, and Lambda that stops non-production EC2 instances when spending spikes.
• A complete set of principal-level pre-artifacts (ADRs, architecture diagrams, risk register, operating cadence) and an executive pitch deck with Well-Architected Cost Pillar alignment.
• Secret Mission: Add a carbon footprint overlay panel and an automated weekly FinOps cadence email report.

Are there any prerequisites?

You need an AWS account with Organizations enabled and management account access (required for cost allocation tags and tag policies). You should also have basic Terraform knowledge and familiarity with AWS console navigation.

Set Up Your Environment

Before you can build a cost intelligence pipeline, you need the right tools in place. Your FinOps command center relies on three core tools: AWS CLI for managing cloud resources, Terraform for infrastructure-as-code, and Docker for running Grafana locally.

In this step, you will verify each tool, spin up a Grafana OSS container with the Athena plugin pre-installed, and scaffold the Terraform project that will hold your entire FinOps infrastructure.

In this step, get ready to:

• Verify and configure AWS CLI, Terraform, and Docker Desktop.
• Launch a Grafana OSS container with the Athena datasource plugin.
• Create the Terraform project structure and initialize the AWS provider.

Verify and install required tools

Your FinOps pipeline depends on three CLI tools working together. Let's verify each one is installed and at the right version.

• Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
• Type Terminal and press Enter to open it.
• Check your AWS CLI version by running this command:

aws --version

✔️ I see version 2.x.x

Your AWS CLI is version 2, which is what this project requires. Move on to the Terraform check below.

ⓧ I see version 1.x.x

AWS CLI version 1 is missing features this project needs (Organizations support, modern authentication). You need to upgrade to version 2.

• Follow the official AWS CLI v2 install guide to upgrade: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
• After installing, close and reopen your terminal.
• Run aws --version again to confirm you see version 2.

ⓧ Command not found

AWS CLI is not installed yet.

• Install AWS CLI v2 from the official guide: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html
• After installing, close and reopen your terminal.
• Run aws --version to confirm it shows version 2.

• Check your Terraform version by running:

terraform --version

✔️ I see version 1.5.0 or higher

Your Terraform version meets the minimum requirement. Move on to the Docker check below.

ⓧ I see an older version

This project requires Terraform 1.5.0 or higher for the features used in the configuration.

• Upgrade Terraform using the official install guide: https://developer.hashicorp.com/terraform/install
• After installing, close and reopen your terminal.
• Run terraform --version again to confirm you see 1.5.0 or higher.

ⓧ Command not found

Terraform is not installed yet.

• Install Terraform from the official guide: https://developer.hashicorp.com/terraform/install
• After installing, close and reopen your terminal.
• Run terraform --version to confirm it shows 1.5.0 or higher.

• Check your Docker version by running:

docker --version

✔️ I see a version number

Docker is installed and ready. Make sure Docker Desktop is running (you should see the Docker whale icon in your system tray or menu bar).

ⓧ Command not found

Docker Desktop is not installed yet.

• Download and install Docker Desktop from the official site: https://www.docker.com/products/docker-desktop/
• After installing, launch Docker Desktop and wait for it to start (the whale icon appears in your system tray or menu bar).
• Run docker --version to confirm it returns a version number.

Now that all three tools are confirmed, configure AWS CLI with your management account credentials.

• Configure your AWS CLI credentials by running:

aws configure

What will this command ask?

You will be prompted for four values. Enter them one at a time:

• AWS Access Key ID - your management account access key.
• AWS Secret Access Key - your management account secret key.
• Default region name - enter us-east-1 (required for CUR).
• Default output format - enter json.

📣 Keep your credentials safe

Never share your Access Key ID or Secret Access Key. These credentials grant full management account access to your AWS Organization.

• Verify your Organization access by running:

aws organizations describe-organization

What should I see?

You should see a JSON response containing your Organization ID, the management account email, and available policy types. This confirms your CLI has the right permissions to manage cost allocation tags and tag policies later in the project.

🙋‍♀️ Seeing an access denied error?

Make sure you are using credentials from the management account of your Organization, not a member account. Member accounts cannot call the Organizations API.

If you recently created the credentials, wait 30 seconds and try again. New IAM credentials can take a moment to propagate.

help me troubleshoot the aws organizations access denied error

Pull and start the Grafana OSS container

Grafana OSS is the dashboard platform that will visualize your CUR data. Running it in Docker means zero install hassle and $0 licensing cost.

• Start the Grafana container with the Athena plugin pre-installed by running this command:

docker run -d --name grafana -p 3000:3000 -e GF_INSTALL_PLUGINS=grafana-athena-datasource grafana/grafana-oss:latest

What does this command do?

• -d runs the container in the background so your terminal stays free.
• --name grafana gives the container a friendly name you can reference later.
• -p 3000:3000 maps port 3000 on your machine to port 3000 inside the container.
• GF_INSTALL_PLUGINS=grafana-athena-datasource tells Grafana to install the Athena datasource plugin on startup.
• grafana/grafana-oss:latest is the official Grafana OSS Docker image.

• Open your browser and navigate to http://localhost:3000.
• Log in with username admin and password admin.
• When prompted to change the password, you can skip for now or set a new one.
• In the left sidebar, click Connections.
• Click Add new connection.
• Search for Athena. You should see Amazon Athena in the results.

Seeing the Athena plugin listed confirms the grafana-athena-datasource plugin installed correctly during container startup.

Don't see the Athena plugin?

The plugin installs during container startup, which can take 30-60 seconds. Refresh the page and check again.

If it still does not appear, check the container logs for errors:

docker logs grafana

Look for lines mentioning plugin installation. If the download failed, your machine may have a network issue reaching the Grafana plugin registry.

help me fix the Grafana Athena plugin not showing up

Create the Terraform project structure

Your entire FinOps infrastructure will live in a single Terraform project. CUR must be created in us-east-1, so the AWS provider is configured for that region.

• Move to your Desktop by running this command:

cd ~/Desktop

• Create your project folder with all required subdirectories by running these commands:

mkdir finops-command-center
cd finops-command-center
mkdir modules pre-artifacts dashboards lambda

• Confirm the directories exist by running:

ls

You should see dashboards, lambda, modules, and pre-artifacts listed in the output.

• Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
• Type Cursor and press Enter to open it.
• In Cursor, click File in the top menu bar.
• Click Open Folder.
• Navigate to your Desktop, select the finops-command-center folder, and click Open.
• Create a new file called main.tf in the root of finops-command-center by right-clicking in the Explorer sidebar and selecting New File.
• Paste the following code into main.tf:

terraform {
  required_version = ">= 1.5.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.0.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

data "aws_caller_identity" "current" {}
data "aws_organizations_organization" "current" {}

locals {
  account_id   = data.aws_caller_identity.current.account_id
  org_id       = data.aws_organizations_organization.current.id
  org_root_id  = data.aws_organizations_organization.current.roots[0].id
  project_name = "finops-command-center"
}

What does this code do?

• required_version = ">= 1.5.0" ensures anyone on your team uses a compatible Terraform version.
• provider "aws" sets the region to us-east-1 because CUR 2.0 can only be created in that region.
• data "aws_caller_identity" fetches your AWS account ID dynamically so you never hardcode it.
• data "aws_organizations_organization" retrieves your Organization ID and root ID for tag policy attachment later.
• The locals block stores these values as reusable references across all modules.

• Save main.tf.
• Back in your terminal (still inside finops-command-center), initialize Terraform by running:

terraform init

You should see "Terraform has been successfully initialized!" confirming the AWS provider downloaded correctly.

Seeing a provider error?

Make sure you saved main.tf before running terraform init. Terraform reads the file from disk, not from your editor buffer.

If you see a network error, check your internet connection. Terraform needs to download the AWS provider plugin from the HashiCorp registry.

help me fix terraform init failing

✔️ Awesome, I've got everything!

Great. Double check you have saved main.tf and that terraform init completed without errors.

ⓧ I'd like to double check the full code

Here is the complete main.tf file for comparison:

terraform {
  required_version = ">= 1.5.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 5.0.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

data "aws_caller_identity" "current" {}
data "aws_organizations_organization" "current" {}

locals {
  account_id   = data.aws_caller_identity.current.account_id
  org_id       = data.aws_organizations_organization.current.id
  org_root_id  = data.aws_organizations_organization.current.roots[0].id
  project_name = "finops-command-center"
}

Your environment is locked and loaded. Next up, you will write the pre-artifacts that document your architectural decisions and implementation plan before touching any infrastructure code.

Create Pre-Artifacts

Before writing any infrastructure code, a principal architect documents the why behind each decision. These pre-artifacts are the deliverables that distinguish a principal hire from a senior engineer.

In this step, you will write Architecture Decision Records (ADRs), a system architecture diagram, and an implementation plan. These documents serve double duty: they guide the build AND they become slides in the CFO presentation later.

In this step, get ready to:

• Write three ADRs covering Grafana vs QuickSight, CUR 2.0 vs Cost Explorer API, and Tag Policies vs manual tagging.
• Create a Mermaid architecture diagram showing data flow, control flow, governance, and operational cadence.
• Write the requirements document, implementation plan, risk register, and FinOps operating cadence template.

Write the Architecture Decision Records

An ADR captures a single architectural decision with its context, alternatives considered, and trade-offs accepted. Each ADR below includes a SAP-C02 trap-pattern callout that highlights common exam pitfalls related to the decision.

• Create a new file called ADR-001-grafana-over-quicksight.md inside the pre-artifacts/ directory, then paste in the following content:

# ADR-001: Grafana OSS over Amazon QuickSight for FinOps Dashboards

## Status
Accepted

## Context
Apex Global requires a cost intelligence dashboard platform that provides:
- Per-customer chargeback visualization
- Automated alerting on cost anomalies
- Template variables for dynamic OU/tag switching
- $0 licensing cost for the platform team
- Dashboard-as-code for version control and CI/CD

**SAP-C02 Trap-Pattern Callout:**
The exam tests whether candidates confuse Cost Anomaly Detection with CloudWatch
alarms for spend monitoring. CloudWatch monitors resource-level metrics (CPU, memory,
network). Cost Anomaly Detection uses ML on billing data (Cost Explorer data feed)
and runs approximately 3 times daily. They serve completely different purposes.
QuickSight dashboards cannot natively alert on cost anomalies. Grafana with the
Athena plugin can query CUR data AND fire alerts, bridging both visibility and action.

## Decision
Use Grafana OSS (self-hosted via Docker) with the grafana-athena-datasource plugin
(v3.2.0) as the primary dashboard platform. QuickSight is not used.

## Consequences
- Positive: $0 cost, native alerting on Athena queries, template variables,
  dashboard-as-code via JSON provisioning, portable across cloud providers
- Positive: Pre-built "Cost Usage Report Monitoring" dashboard ships with plugin
- Negative: Self-managed (Docker container), no managed SLA
- Negative: QuickSight ML Insights (forecasting) not available natively
- Mitigation: Grafana alerting + Athena SQL projections replace ML Insights

What does this ADR cover?

The Context section lays out the five requirements that drove the decision. The SAP-C02 callout highlights a critical exam distinction: Cost Anomaly Detection monitors billing data with ML, while CloudWatch monitors resource-level metrics. They are completely different tools for different purposes.

The Decision section names the exact plugin version (v3.2.0) so the team can reproduce the environment. The Consequences section balances positives against trade-offs with a mitigation strategy.

• Save ADR-001-grafana-over-quicksight.md.

• Now create ADR-002-cur2-athena-over-cost-explorer-api.md inside pre-artifacts/ with this content:

# ADR-002: CUR 2.0 + Athena over Cost Explorer API

## Status
Accepted

## Context
The CFO mandate requires per-customer chargeback with resource-level granularity.
Two approaches exist:
1. Cost Explorer API (GetCostAndUsage): Limited to 13 months history, 12 filters
   max, no resource-level IDs, throttled at 5 requests/second.
2. CUR 2.0 + Athena: Full resource-level granularity, unlimited history, standard
   SQL, per-resource line items, cost-effective at scale.

**SAP-C02 Trap-Pattern Callout:**
The exam distinguishes CUR + Athena from CloudWatch dashboards for cost visibility.
CloudWatch dashboards show resource metrics (CPU, disk, network), NOT costs.
CUR 2.0 delivers billing data. The SQL table name is COST_AND_USAGE_REPORT.
CUR 2.0 has a fixed schema (unlike legacy CUR which varied monthly).
Key gotcha: CUR takes up to 24 hours for first delivery after creation.
Another trap: CUR must be created in us-east-1 regardless of workload region.

## Decision
Use CUR 2.0 (Data Exports) in Parquet format, delivered to S3, cataloged by Glue,
and queried via Athena. Cost Explorer is used only for quick ad-hoc console checks.

## Consequences
- Positive: Resource-level granularity (line_item_resource_id)
- Positive: Standard SQL, unlimited query complexity
- Positive: Parquet format reduces Athena scan costs by 90%+ vs CSV
- Negative: 24-hour initial delivery delay
- Negative: Requires Glue Crawler setup for schema management
- Mitigation: Crawler runs daily on schedule; delay is one-time setup cost

What does this ADR cover?

This ADR documents why CUR 2.0 with Athena beats the Cost Explorer API for chargeback. The key SAP-C02 traps here: CUR must be created in us-east-1 regardless of where your workloads run, the SQL table name is COST_AND_USAGE_REPORT, and CUR 2.0 has a fixed schema unlike legacy CUR.

• Save ADR-002-cur2-athena-over-cost-explorer-api.md.

• Create the final ADR file ADR-003-tag-policies-over-manual-tagging.md inside pre-artifacts/ with this content:

# ADR-003: Tag Policies with Required Keys over Manual Tagging

## Status
Accepted

## Context
Apex Global has 0% tag enforcement. Resources are created without CostCenter,
Environment, or Owner tags, making chargeback impossible for 60%+ of spend.

**SAP-C02 Trap-Pattern Callout:**
The exam tests the management-account-only restriction for cost allocation tag
activation. Only the management account (or standalone accounts not in an org)
can activate cost allocation tags. Member accounts cannot. This is the #1 gotcha
for tag-related questions.

Additional trap: Tag Policies (Organizations) vs Cost Allocation Tags (Billing)
are different mechanisms.
- Tag Policies enforce tag KEY and VALUE compliance at resource creation.
- Cost Allocation Tags make existing tags visible in CUR/Cost Explorer.
You need BOTH: Tag Policies to enforce, Cost Allocation Tags to report.

Tag Policies with "Required tag keys" now enforce via Terraform AWS Provider
6.22.0+ with tag policy validation enabled. This means Terraform plan will fail
if required tags are missing, catching non-compliance before deployment.

## Decision
Deploy Organizations Tag Policy requiring CostCenter, Environment, and Owner
tags on EC2, S3, RDS, and Lambda resources. Activate cost allocation tags from
management account. Use SCP as a fallback guardrail for console-created resources.

## Consequences
- Positive: 100% tag compliance at creation time for IaC deployments
- Positive: Proactive enforcement (fail before create) vs reactive (find after)
- Positive: Single policy definition works across Terraform, CloudFormation, Pulumi
- Negative: Tag Policies do not enforce on resources created WITHOUT tags
  (untagged resources are not evaluated). SCP needed as complement.
- Negative: Up to 24 hours for tag activation to take effect in billing reports

What does this ADR cover?

This is the governance ADR. The critical SAP-C02 distinction: Tag Policies (Organizations) enforce compliance at creation time. Cost Allocation Tags (Billing) make tags visible in reports. You need both mechanisms working together.

The management-account-only restriction for activating cost allocation tags is the number one exam gotcha for tag-related questions. Member accounts cannot activate them.

• Save ADR-003-tag-policies-over-manual-tagging.md.
• Confirm all three files exist by running:

ls pre-artifacts/

You should see ADR-001-grafana-over-quicksight.md, ADR-002-cur2-athena-over-cost-explorer-api.md, and ADR-003-tag-policies-over-manual-tagging.md in the output.

Files not showing up?

Make sure you are running ls from the finops-command-center/ project root. If you saved files in the wrong directory, move them into pre-artifacts/ using your file explorer or mv.

help me find where my ADR files were saved

Create the architecture diagram

A Mermaid diagram renders as a visual from plain text. This architecture diagram shows four layers of the FinOps command center: data flow, control flow, governance, and operational cadence.

• Create architecture.md inside the pre-artifacts/ directory with the following content:

# FinOps Command Center - Architecture

## System Architecture Diagram

```mermaid
flowchart TD
    subgraph DataFlow["Data Flow - Inform"]
        direction LR
        CUR["CUR 2.0\n(Data Exports)"] --> S3["S3 Bucket\n(Parquet)"]
        S3 --> Glue["Glue Catalog\n(Crawler)"]
        Glue --> Athena["Athena\n(SQL Views)"]
        Athena --> Grafana["Grafana OSS\n(Dashboards)"]
    end

    subgraph ControlFlow["Control Flow - Operate"]
        direction LR
        CAD["Cost Anomaly\nDetection"] --> EB["EventBridge"]
        EB --> Lambda["Lambda\n(Remediator)"]
        Lambda --> EC2Stop["EC2 Stop\n(Non-Prod)"]
        Lambda --> SNSApproval["SNS\n(Prod Approval)"]
        Budgets["Budget Actions"] --> IAMDeny["IAM Deny\n(ec2:RunInstances)"]
        Budgets --> SSMStop["SSM Stop\n(EC2 Instances)"]
    end

    subgraph Governance["Governance Overlay - Optimize"]
        direction LR
        TagPolicy["Tag Policies\n(Organizations)"] --> Required["Required Tags:\nCostCenter\nEnvironment\nOwner"]
        CATags["Cost Allocation Tags\n(Billing)"] --> CURVisibility["Tags Visible\nin CUR/Reports"]
    end

    subgraph Cadence["Operational Cadence"]
        direction LR
        Weekly["Weekly Review\n(Spend vs Budget)"]
        Monthly["Monthly Optimization\n(Savings Plans, Rightsizing)"]
        Quarterly["Quarterly Renewal\n(Commitments, Forecasts)"]
        Weekly --> Monthly --> Quarterly
    end

    DataFlow --> ControlFlow
    ControlFlow --> Governance
    Governance --> Cadence
```

## Flow Descriptions

### Data Flow (Inform Phase)
CUR 2.0 delivers hourly billing data in Parquet format to S3. A Glue Crawler
catalogs the data for Athena SQL queries. Grafana connects to Athena for
real-time dashboard visualization with template variables and alerting.

### Control Flow (Operate Phase)
Cost Anomaly Detection identifies spend deviations using ML on billing data.
EventBridge routes anomaly events to a Lambda function that performs cross-account
remediation (stops non-prod EC2 instances automatically, requests approval for prod).
Budget Actions provide a second guardrail layer with IAM deny and SSM stop.

### Governance Overlay (Optimize Phase)
Tag Policies enforce required cost allocation tags (CostCenter, Environment, Owner)
at resource creation time. Cost Allocation Tags (activated from management account)
make these tags visible in CUR and Cost Explorer for chargeback reporting.

### Operational Cadence
- Weekly: Spend vs budget review, top anomalies, tag coverage percentage
- Monthly: Savings Plans coverage, commitment purchases, waste elimination
- Quarterly: Commitment renewals, forecast updates, FinOps maturity assessment

What does this diagram show?

The four subgraphs map to the FinOps lifecycle phases. The Data Flow (Inform) gives you visibility. The Control Flow (Operate) automates responses. The Governance Overlay (Optimize) prevents waste proactively. The Operational Cadence defines the human review rhythm.

This diagram becomes a key slide in the executive presentation. It shows the CFO that every dollar of spend is visible, governed, and actively managed.

• Save architecture.md.
• To preview the Mermaid diagram, paste the code block between the ```mermaid fences into the Mermaid Live Editor at https://mermaid.live. You should see four connected subgraphs rendering as a top-down flowchart.

Diagram not rendering?

Make sure you copied everything between the ```mermaid and closing ``` fences (not including the fences themselves). Mermaid is whitespace-sensitive, so check that indentation is preserved.

help me debug why my Mermaid architecture diagram is not rendering

Write the requirements document and implementation plan

The final pre-artifact combines four documents into a single reference file. These translate the CFO's mandate into actionable deliverables with timelines, risks, and a repeatable operating cadence.

• Create requirements-and-plan.md inside pre-artifacts/ with the following content:

# FinOps Command Center - Requirements and Implementation Plan

## 1. Requirements Document

### Business Context
The CFO called an emergency leadership meeting after 38% quarter-over-quarter
spend growth with zero revenue increase. The mandate: deliver end-to-end cost
intelligence with per-customer chargeback, automated anomaly remediation, and
spend guardrails by month-end.

### Functional Requirements
- FR-01: Per-customer chargeback reporting by CostCenter tag
- FR-02: Per-account monthly spend breakdown with historical trend
- FR-03: Automated detection of spend anomalies (ML-based, not threshold-only)
- FR-04: Automated remediation for non-production environments
- FR-05: Manual approval workflow for production remediation
- FR-06: Budget guardrails that prevent new resource launches at 80% threshold
- FR-07: Tag enforcement at resource creation time (not after-the-fact)
- FR-08: Executive dashboard with KPIs, projections, and Savings Plans coverage
- FR-09: Daily automated data refresh (no manual exports)
- FR-10: $0 platform licensing cost

### Non-Functional Requirements
- NFR-01: Detection-to-remediation latency < 15 minutes for non-prod
- NFR-02: Dashboard query response < 10 seconds for executive panels
- NFR-03: Infrastructure-as-code (reproducible, version-controlled)
- NFR-04: Cross-account reach (management account controls member accounts)
- NFR-05: Audit trail for all remediation actions

## 2. Implementation Plan

### Parallel Agent Structure
Three workstreams execute in parallel after shared pre-artifacts are complete:

| Agent | Scope | Steps |
|-------|-------|-------|
| Agent A | Data Pipeline | CUR 2.0, Glue, Athena views, Savings Plans analysis |
| Agent B | Visualization | Grafana datasource, dashboards, KPIs, projections |
| Agent C | Governance | Anomaly Detection, Lambda, Budget Actions, Tag Policy |

### Timeline
- Day 1: Pre-artifacts (ADRs, architecture, this plan)
- Day 2-3: Agent A deploys data pipeline (CUR, Glue, Athena)
- Day 2-3: Agent B builds Grafana dashboards (datasource, panels, alerts)
- Day 2-3: Agent C deploys governance (Anomaly Detection, Lambda, Budgets, Tags)
- Day 4: Integration testing (spike test, end-to-end validation)
- Day 5: Executive presentation assembly and delivery

### Dependencies
- Agent B depends on Agent A (Athena workgroup and Glue database must exist)
- Agent C depends on Agent A (SNS topic shared for anomaly alerts)
- Spike test depends on all three agents completing

## 3. Risk Register

| Risk | Impact | Likelihood | Mitigation |
|------|--------|------------|------------|
| False-positive remediation stops production instance | Critical | Low | approval_model = MANUAL for prod accounts; Lambda checks Environment tag before stopping |
| CUR 24-hour delivery delay blocks dashboard testing | Medium | Certain | Use synthetic Athena data for initial dashboard development; wait for real data on Day 3 |
| Anomaly Detection 10-day training window for new services | Medium | Certain | Deploy monitors immediately; use Budget Actions as interim guardrail during training period |
| Tag Policy blocks legitimate deployment | High | Medium | Deploy in Warn mode for first 30 days; switch to Fail mode after baseline established |
| Glue Crawler schema drift on CUR 2.0 update | Low | Low | CUR 2.0 has fixed schema (unlike legacy CUR); set delete_behavior = LOG to detect changes |

## 4. FinOps Operating Cadence Template

### Weekly Review Meeting
- Attendees: FinOps lead, engineering leads, finance partner
- Agenda:
  1. Spend vs budget (actual vs forecast)
  2. Top anomalies detected this week
  3. Tag coverage percentage (target: 95%+)
  4. Rightsizing backlog (instances to resize)
  5. Action items from previous week

### Monthly Optimization Cycle
- Savings Plans coverage review (target: 70%+ of eligible spend)
- Commitment purchase decisions (1-year vs 3-year trade-offs)
- Waste elimination (unattached EBS, idle instances, unused Elastic IPs)
- Unit economics update (cost per customer, cost per transaction)

### Quarterly Commitment Renewal
- Savings Plans expiry review (90-day lookahead)
- Forecast update based on business growth projections
- FinOps maturity assessment (Crawl -> Walk -> Run progression)
- Budget re-baseline for next quarter

### Escalation Path
1. Anomaly detected -> Slack alert (immediate)
2. Non-prod: auto-remediation within 15 minutes
3. Prod: manual approval required -> SNS notification to on-call
4. If unresolved in 1 hour: escalate to engineering lead
5. Post-incident review within 48 hours (update runbook)

What does this document cover?

The Requirements section translates the CFO mandate into numbered requirements that each component of the system maps back to. The Implementation Plan shows the 3-agent parallel structure that makes a 5-day delivery possible.

The Risk Register is critical. The top risk (false-positive remediation of production) is mitigated by the approval_model = MANUAL pattern you will implement in the Lambda function later. The 24-hour CUR delay and 10-day training window are both documented AWS behaviors that cannot be shortened.

The Operating Cadence template defines what happens after deployment. FinOps is not a one-time build. It is a continuous practice with weekly, monthly, and quarterly rhythms.

• Save requirements-and-plan.md.
• Verify all pre-artifacts are in place by running:

ls pre-artifacts/

You should see five files: the three ADRs, architecture.md, and requirements-and-plan.md.

Missing a file?

Double-check that you saved each file with the exact filename shown (including hyphens, not underscores). All five files must be inside the pre-artifacts/ subdirectory, not at the project root.

help me find my missing pre-artifact files

✔️ Awesome, I've got everything!

Great. Double check you have saved all five files in the pre-artifacts/ directory.

ⓧ I'd like to double check the full code

Here are the three ADR files for comparison. The architecture.md and requirements-and-plan.md should match what you pasted from the code blocks above.

# ADR-001: Grafana OSS over Amazon QuickSight for FinOps Dashboards

## Status
Accepted

## Context
Apex Global requires a cost intelligence dashboard platform that provides:
- Per-customer chargeback visualization
- Automated alerting on cost anomalies
- Template variables for dynamic OU/tag switching
- $0 licensing cost for the platform team
- Dashboard-as-code for version control and CI/CD

**SAP-C02 Trap-Pattern Callout:**
The exam tests whether candidates confuse Cost Anomaly Detection with CloudWatch
alarms for spend monitoring. CloudWatch monitors resource-level metrics (CPU, memory,
network). Cost Anomaly Detection uses ML on billing data (Cost Explorer data feed)
and runs approximately 3 times daily. They serve completely different purposes.
QuickSight dashboards cannot natively alert on cost anomalies. Grafana with the
Athena plugin can query CUR data AND fire alerts, bridging both visibility and action.

## Decision
Use Grafana OSS (self-hosted via Docker) with the grafana-athena-datasource plugin
(v3.2.0) as the primary dashboard platform. QuickSight is not used.

## Consequences
- Positive: $0 cost, native alerting on Athena queries, template variables,
  dashboard-as-code via JSON provisioning, portable across cloud providers
- Positive: Pre-built "Cost Usage Report Monitoring" dashboard ships with plugin
- Negative: Self-managed (Docker container), no managed SLA
- Negative: QuickSight ML Insights (forecasting) not available natively
- Mitigation: Grafana alerting + Athena SQL projections replace ML Insights

# ADR-002: CUR 2.0 + Athena over Cost Explorer API

## Status
Accepted

## Context
The CFO mandate requires per-customer chargeback with resource-level granularity.
Two approaches exist:
1. Cost Explorer API (GetCostAndUsage): Limited to 13 months history, 12 filters
   max, no resource-level IDs, throttled at 5 requests/second.
2. CUR 2.0 + Athena: Full resource-level granularity, unlimited history, standard
   SQL, per-resource line items, cost-effective at scale.

**SAP-C02 Trap-Pattern Callout:**
The exam distinguishes CUR + Athena from CloudWatch dashboards for cost visibility.
CloudWatch dashboards show resource metrics (CPU, disk, network), NOT costs.
CUR 2.0 delivers billing data. The SQL table name is COST_AND_USAGE_REPORT.
CUR 2.0 has a fixed schema (unlike legacy CUR which varied monthly).
Key gotcha: CUR takes up to 24 hours for first delivery after creation.
Another trap: CUR must be created in us-east-1 regardless of workload region.

## Decision
Use CUR 2.0 (Data Exports) in Parquet format, delivered to S3, cataloged by Glue,
and queried via Athena. Cost Explorer is used only for quick ad-hoc console checks.

## Consequences
- Positive: Resource-level granularity (line_item_resource_id)
- Positive: Standard SQL, unlimited query complexity
- Positive: Parquet format reduces Athena scan costs by 90%+ vs CSV
- Negative: 24-hour initial delivery delay
- Negative: Requires Glue Crawler setup for schema management
- Mitigation: Crawler runs daily on schedule; delay is one-time setup cost

# ADR-003: Tag Policies with Required Keys over Manual Tagging

## Status
Accepted

## Context
Apex Global has 0% tag enforcement. Resources are created without CostCenter,
Environment, or Owner tags, making chargeback impossible for 60%+ of spend.

**SAP-C02 Trap-Pattern Callout:**
The exam tests the management-account-only restriction for cost allocation tag
activation. Only the management account (or standalone accounts not in an org)
can activate cost allocation tags. Member accounts cannot. This is the #1 gotcha
for tag-related questions.

Additional trap: Tag Policies (Organizations) vs Cost Allocation Tags (Billing)
are different mechanisms.
- Tag Policies enforce tag KEY and VALUE compliance at resource creation.
- Cost Allocation Tags make existing tags visible in CUR/Cost Explorer.
You need BOTH: Tag Policies to enforce, Cost Allocation Tags to report.

Tag Policies with "Required tag keys" now enforce via Terraform AWS Provider
6.22.0+ with tag policy validation enabled. This means Terraform plan will fail
if required tags are missing, catching non-compliance before deployment.

## Decision
Deploy Organizations Tag Policy requiring CostCenter, Environment, and Owner
tags on EC2, S3, RDS, and Lambda resources. Activate cost allocation tags from
management account. Use SCP as a fallback guardrail for console-created resources.

## Consequences
- Positive: 100% tag compliance at creation time for IaC deployments
- Positive: Proactive enforcement (fail before create) vs reactive (find after)
- Positive: Single policy definition works across Terraform, CloudFormation, Pulumi
- Negative: Tag Policies do not enforce on resources created WITHOUT tags
  (untagged resources are not evaluated). SCP needed as complement.
- Negative: Up to 24 hours for tag activation to take effect in billing reports

Your pre-artifacts are complete. These documents ground every infrastructure decision in the remaining steps. Next up, you will deploy the data pipeline that brings CUR 2.0 billing data into Athena for querying.

Deploy the Data Pipeline

Your pre-artifacts established the architectural decisions and operating cadence for Apex Global's FinOps program. Now it's time to build the actual data pipeline that feeds those dashboards and reports.

The pipeline follows the flow you documented in ADR-002: CUR 2.0 delivers billing data to S3, a Glue Crawler catalogs the schema, and Athena provides the SQL interface for querying cost data.

In this step, get ready to:

• Deploy the CUR 2.0 export module with S3 bucket and billing report definition.
• Deploy the Glue catalog database, crawler, and Athena workgroup module.
• Create Athena views for chargeback analysis and activate cost allocation tags.

Deploy the CUR 2.0 export module

CUR 2.0 is the foundation of your cost intelligence pipeline. It delivers hourly billing data in Parquet format to an S3 bucket, where downstream services can catalog and query it.

• Create the file modules/cur/main.tf inside your finops-command-center project folder.
• Paste the following Terraform configuration into it:

resource "aws_s3_bucket" "cur_bucket" {
  bucket = "${var.project_name}-cur-${var.account_id}"
}

resource "aws_s3_bucket_policy" "cur_bucket_policy" {
  bucket = aws_s3_bucket.cur_bucket.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          Service = "billingreports.amazonaws.com"
        }
        Action   = ["s3:GetBucketAcl", "s3:GetBucketPolicy"]
        Resource = aws_s3_bucket.cur_bucket.arn
        Condition = {
          StringEquals = {
            "aws:SourceAccount" = var.account_id
            "aws:SourceArn"     = "arn:aws:cur:us-east-1:${var.account_id}:definition/*"
          }
        }
      },
      {
        Effect = "Allow"
        Principal = {
          Service = "billingreports.amazonaws.com"
        }
        Action   = "s3:PutObject"
        Resource = "${aws_s3_bucket.cur_bucket.arn}/*"
        Condition = {
          StringEquals = {
            "aws:SourceAccount" = var.account_id
            "aws:SourceArn"     = "arn:aws:cur:us-east-1:${var.account_id}:definition/*"
          }
        }
      }
    ]
  })
}

resource "aws_cur_report_definition" "apex_global_cur" {
  report_name                = "apex-global-cur"
  time_unit                  = "HOURLY"
  format                     = "Parquet"
  compression                = "Parquet"
  additional_schema_elements = ["RESOURCES", "SPLIT_COST_ALLOCATION_DATA"]
  s3_bucket                  = aws_s3_bucket.cur_bucket.id
  s3_prefix                  = "cur-reports"
  s3_region                  = "us-east-1"
  additional_artifacts       = ["ATHENA"]
  report_versioning          = "OVERWRITE_REPORT"
  refresh_closed_reports     = true
}

variable "project_name" {
  type = string
}

variable "account_id" {
  type = string
}

output "cur_bucket_name" {
  value = aws_s3_bucket.cur_bucket.id
}

output "cur_report_name" {
  value = aws_cur_report_definition.apex_global_cur.report_name
}

What does this module do?

This module creates three resources that work together:

• An S3 bucket to receive billing data deliveries from the CUR 2.0 service.
• A bucket policy granting billingreports.amazonaws.com permission to write report files. The SourceArn condition locks access to only your account's CUR definitions.
• The CUR report definition itself, configured for hourly Parquet delivery with resource-level granularity (RESOURCES) and split cost allocation data.

The ATHENA additional artifact and OVERWRITE_REPORT versioning are required together. This tells CUR to deliver data in a format optimized for Athena queries.

💡 Why must CUR be in us-east-1?

The aws_cur_report_definition resource only works with the provider set to us-east-1. This is a hard constraint from AWS regardless of where your workloads run.

CUR also takes up to 24 hours for its first delivery after creation. You will not see data in the S3 bucket immediately.

• Save modules/cur/main.tf.

Seeing a provider region error?

If Terraform reports that the CUR resource is not available in your configured region, confirm your main.tf provider block uses region = "us-east-1". CUR definitions can only be created from us-east-1.

help me fix a CUR provider region error in Terraform

Deploy the Glue catalog and Athena workgroup module

Once CUR delivers Parquet files to S3, you need a way to catalog that data and query it with SQL. The Glue Crawler automatically detects the schema and registers it in the Glue Data Catalog. Athena then uses that catalog to run standard SQL queries against the billing data.

• Create the file modules/glue-athena/main.tf inside your finops-command-center project folder.
• Paste the following Terraform configuration into it:

resource "aws_glue_catalog_database" "finops" {
  name = "${var.project_name}-database"
}

resource "aws_iam_role" "glue_crawler_role" {
  name = "${var.project_name}-glue-crawler-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Principal = { Service = "glue.amazonaws.com" }
      Action = "sts:AssumeRole"
    }]
  })
}

resource "aws_iam_role_policy_attachment" "glue_service" {
  role       = aws_iam_role.glue_crawler_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole"
}

resource "aws_iam_role_policy" "glue_s3_access" {
  name = "${var.project_name}-glue-s3-access"
  role = aws_iam_role.glue_crawler_role.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect   = "Allow"
      Action   = ["s3:GetObject", "s3:ListBucket"]
      Resource = [
        "arn:aws:s3:::${var.cur_bucket_name}",
        "arn:aws:s3:::${var.cur_bucket_name}/*"
      ]
    }]
  })
}

resource "aws_glue_crawler" "cur_crawler" {
  database_name = aws_glue_catalog_database.finops.name
  name          = "${var.project_name}-cur-crawler"
  role          = aws_iam_role.glue_crawler_role.arn

  s3_target {
    path = "s3://${var.cur_bucket_name}/cur-reports/apex-global-cur/apex-global-cur/data/"
  }

  schedule = "cron(0 6 * * ? *)"

  schema_change_policy {
    delete_behavior = "LOG"
    update_behavior = "UPDATE_IN_DATABASE"
  }
}

resource "aws_s3_bucket" "athena_results" {
  bucket = "${var.project_name}-athena-results-${var.account_id}"
}

resource "aws_athena_workgroup" "finops" {
  name = var.project_name

  configuration {
    result_configuration {
      output_location = "s3://${aws_s3_bucket.athena_results.id}/query-results/"
    }
  }
}

variable "project_name" {
  type = string
}

variable "cur_bucket_name" {
  type = string
}

variable "account_id" {
  type = string
}

output "glue_database_name" {
  value = aws_glue_catalog_database.finops.name
}

output "athena_workgroup" {
  value = aws_athena_workgroup.finops.name
}

What does this module do?

This module sets up the query layer for your cost data:

• A Glue Catalog database (finops-command-center-database) that stores table metadata for Athena to reference.
• An IAM role with the managed AWSGlueServiceRole policy plus an inline policy granting S3 read access to the CUR bucket.
• A Glue Crawler that scans the CUR data path daily at 6:00 AM UTC and registers partitions (one per billing period) in the catalog.
• An Athena workgroup (finops-command-center) with its own results bucket. Workgroups let you isolate FinOps queries from other analytics workloads.

• Save modules/glue-athena/main.tf.

Now deploy both modules together using Terraform.

• In your terminal, navigate to the finops-command-center root directory and re-initialize Terraform by running:

terraform init

• Apply the CUR module first by running:

terraform apply -target=module.cur

• Once the CUR module completes, apply the Glue and Athena module by running:

terraform apply -target=module.glue_athena

How are these modules connected to main.tf?

Terraform modules need to be called from the root configuration via module blocks. Your main.tf uses the locals block to define shared values like account_id and project_name, which get passed as variables to each module.

The CUR module outputs cur_bucket_name, which the Glue-Athena module consumes as its cur_bucket_name variable input. This is how Terraform chains module dependencies.

• Verify the deployment succeeded by checking the Terraform output shows both S3 buckets and the CUR report created.
• Confirm in the AWS console under AWS Glue > Databases that finops-command-center-database exists.

Terraform apply failing?

If you see an error about the CUR resource, confirm your AWS credentials have management account access. CUR report definitions can only be created from the management account in an organization.

If the Glue module fails with an S3 permission error, make sure the CUR module was applied first so the bucket exists.

help me debug a Terraform apply failure for my FinOps CUR and Glue modules

✔️ Awesome, I've got everything!

Great. Double check you have saved both module files.

ⓧ I'd like to double check the full code

resource "aws_s3_bucket" "cur_bucket" {
  bucket = "${var.project_name}-cur-${var.account_id}"
}

resource "aws_s3_bucket_policy" "cur_bucket_policy" {
  bucket = aws_s3_bucket.cur_bucket.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Principal = {
          Service = "billingreports.amazonaws.com"
        }
        Action   = ["s3:GetBucketAcl", "s3:GetBucketPolicy"]
        Resource = aws_s3_bucket.cur_bucket.arn
        Condition = {
          StringEquals = {
            "aws:SourceAccount" = var.account_id
            "aws:SourceArn"     = "arn:aws:cur:us-east-1:${var.account_id}:definition/*"
          }
        }
      },
      {
        Effect = "Allow"
        Principal = {
          Service = "billingreports.amazonaws.com"
        }
        Action   = "s3:PutObject"
        Resource = "${aws_s3_bucket.cur_bucket.arn}/*"
        Condition = {
          StringEquals = {
            "aws:SourceAccount" = var.account_id
            "aws:SourceArn"     = "arn:aws:cur:us-east-1:${var.account_id}:definition/*"
          }
        }
      }
    ]
  })
}

resource "aws_cur_report_definition" "apex_global_cur" {
  report_name                = "apex-global-cur"
  time_unit                  = "HOURLY"
  format                     = "Parquet"
  compression                = "Parquet"
  additional_schema_elements = ["RESOURCES", "SPLIT_COST_ALLOCATION_DATA"]
  s3_bucket                  = aws_s3_bucket.cur_bucket.id
  s3_prefix                  = "cur-reports"
  s3_region                  = "us-east-1"
  additional_artifacts       = ["ATHENA"]
  report_versioning          = "OVERWRITE_REPORT"
  refresh_closed_reports     = true
}

variable "project_name" {
  type = string
}

variable "account_id" {
  type = string
}

output "cur_bucket_name" {
  value = aws_s3_bucket.cur_bucket.id
}

output "cur_report_name" {
  value = aws_cur_report_definition.apex_global_cur.report_name
}

resource "aws_glue_catalog_database" "finops" {
  name = "${var.project_name}-database"
}

resource "aws_iam_role" "glue_crawler_role" {
  name = "${var.project_name}-glue-crawler-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Principal = { Service = "glue.amazonaws.com" }
      Action = "sts:AssumeRole"
    }]
  })
}

resource "aws_iam_role_policy_attachment" "glue_service" {
  role       = aws_iam_role.glue_crawler_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole"
}

resource "aws_iam_role_policy" "glue_s3_access" {
  name = "${var.project_name}-glue-s3-access"
  role = aws_iam_role.glue_crawler_role.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect   = "Allow"
      Action   = ["s3:GetObject", "s3:ListBucket"]
      Resource = [
        "arn:aws:s3:::${var.cur_bucket_name}",
        "arn:aws:s3:::${var.cur_bucket_name}/*"
      ]
    }]
  })
}

resource "aws_glue_crawler" "cur_crawler" {
  database_name = aws_glue_catalog_database.finops.name
  name          = "${var.project_name}-cur-crawler"
  role          = aws_iam_role.glue_crawler_role.arn

  s3_target {
    path = "s3://${var.cur_bucket_name}/cur-reports/apex-global-cur/apex-global-cur/data/"
  }

  schedule = "cron(0 6 * * ? *)"

  schema_change_policy {
    delete_behavior = "LOG"
    update_behavior = "UPDATE_IN_DATABASE"
  }
}

resource "aws_s3_bucket" "athena_results" {
  bucket = "${var.project_name}-athena-results-${var.account_id}"
}

resource "aws_athena_workgroup" "finops" {
  name = var.project_name

  configuration {
    result_configuration {
      output_location = "s3://${aws_s3_bucket.athena_results.id}/query-results/"
    }
  }
}

variable "project_name" {
  type = string
}

variable "cur_bucket_name" {
  type = string
}

variable "account_id" {
  type = string
}

output "glue_database_name" {
  value = aws_glue_catalog_database.finops.name
}

output "athena_workgroup" {
  value = aws_athena_workgroup.finops.name
}

Create Athena views and activate cost allocation tags

With the infrastructure deployed, you need two more things before Grafana can visualize costs: SQL views that shape the raw CUR data into chargeback-friendly formats, and activated cost allocation tags so tag-based grouping appears in billing data.

• Open the Amazon Athena console.
• Select the finops-command-center workgroup from the Workgroup dropdown.
• Select finops-command-center-database from the Database dropdown.
• Create the chargeback-by-tag view by running this SQL in the Athena query editor:

CREATE OR REPLACE VIEW v_chargeback_by_tag AS
SELECT
  bill_billing_period_start_date,
  resource_tags['CostCenter'] AS cost_center,
  line_item_product_code AS service,
  SUM(line_item_unblended_cost) AS total_cost
FROM cost_and_usage_report
WHERE line_item_line_item_type != 'Tax'
GROUP BY
  bill_billing_period_start_date,
  resource_tags['CostCenter'],
  line_item_product_code;

• Create the chargeback-by-account view by running:

CREATE OR REPLACE VIEW v_chargeback_by_account AS
SELECT
  bill_billing_period_start_date,
  line_item_usage_account_id,
  line_item_usage_account_name,
  line_item_product_code AS service,
  SUM(line_item_unblended_cost) AS total_cost
FROM cost_and_usage_report
WHERE line_item_line_item_type != 'Tax'
GROUP BY
  bill_billing_period_start_date,
  line_item_usage_account_id,
  line_item_usage_account_name,
  line_item_product_code;

• Create the daily spend trend view by running:

CREATE OR REPLACE VIEW v_daily_spend_trend AS
SELECT
  DATE(line_item_usage_start_date) AS usage_date,
  line_item_product_code AS service,
  SUM(line_item_unblended_cost) AS daily_cost
FROM cost_and_usage_report
WHERE line_item_line_item_type != 'Tax'
GROUP BY
  DATE(line_item_usage_start_date),
  line_item_product_code;

• Create the Savings Plans coverage view by running:

CREATE OR REPLACE VIEW v_savings_plans_coverage AS
SELECT
  bill_billing_period_start_date,
  SUM(savings_plan_used_commitment) AS used_commitment,
  SUM(savings_plan_total_commitment_to_date) AS total_commitment,
  CASE
    WHEN SUM(savings_plan_total_commitment_to_date) > 0
    THEN SUM(savings_plan_used_commitment) / SUM(savings_plan_total_commitment_to_date) * 100
    ELSE 0
  END AS utilization_percent
FROM cost_and_usage_report
WHERE savings_plan_savings_plan_effective_cost > 0
GROUP BY bill_billing_period_start_date;

What do these views provide?

• v_chargeback_by_tag groups costs by CostCenter tag, billing period, and service. This powers per-customer chargeback panels in Grafana.
• v_chargeback_by_account groups by linked account ID and name. Useful for multi-account organizations.
• v_daily_spend_trend creates a time-series of daily costs per service for Grafana line charts.
• v_savings_plans_coverage calculates Savings Plans utilization as a percentage. This feeds the executive KPI gauge panel.

These views will only return data once CUR delivers its first report (up to 24 hours after creation). Until then, queries return empty results.

Next, activate cost allocation tags so tag-based costs appear in the CUR data.

• Open the AWS Billing and Cost Management console.
• Select Cost allocation tags from the left navigation.
• Under User-defined cost allocation tags, search for and select: CostCenter, Environment, Owner, and Project.
• Click Activate.

Why does this require the management account?

Cost allocation tags can only be activated from the management account in an AWS Organization. Member accounts cannot access this page.

Activated tags take up to 24 hours to appear in CUR data. Tag Policies (deployed later) enforce that these tags exist at resource creation time. Cost allocation tag activation makes them visible in billing reports.

• Finally, enable Compute Optimizer for your organization. Open the AWS Compute Optimizer console and click Opt in.
• Select All accounts in this organization to enable recommendations across member accounts.

What is Compute Optimizer?

Compute Optimizer analyzes your EC2, EBS, Lambda, and ECS resource usage to recommend rightsizing changes. It requires 14 days of CloudWatch metrics before generating accurate recommendations.

Enabling it now means recommendations will be ready when you build the executive dashboard panels in a later step.

Your data pipeline infrastructure is deployed and waiting for CUR's first delivery. Next up, you will build the Grafana dashboards that visualize this cost data for Apex Global's leadership.

Build the Grafana Dashboards

Your data pipeline is complete. CUR 2.0 is delivering billing data to S3, Glue is cataloging the schema, and Athena views are ready to query per-tag chargeback, account spend, and Savings Plans coverage.

Now you need to turn that raw data into something the CFO can act on. In this step, you will connect Grafana to your Athena pipeline, build executive dashboards with per-customer chargeback panels, and configure alerting rules that fire when spend deviates from the baseline.

In this step, get ready to:

• Create and mount the Grafana Athena datasource provisioning YAML.
• Build the main chargeback dashboard with template variables and alerting.
• Build executive KPI and 12-month projection dashboards with SAP-C02 reference panels.

Create the Athena datasource provisioning YAML

Grafana uses YAML provisioning files to configure datasources at startup. This eliminates manual UI configuration and makes the setup reproducible as infrastructure-as-code.

• Create the provisioning file at dashboards/grafana-provisioning.yaml by adding the following content:

apiVersion: 1
datasources:
  - name: AthenaFinOps
    type: grafana-athena-datasource
    access: proxy
    isDefault: true
    jsonData:
      authType: keys
      defaultRegion: us-east-1
      catalog: AwsDataCatalog
      database: finops-command-center-database
      workgroup: finops-command-center
    secureJsonData:
      accessKey: your-access-key-here
      secretKey: your-secret-key-here

What does this configuration do?

This YAML tells Grafana to provision an Athena datasource named AthenaFinOps on startup. The jsonData block configures the Athena connection to point to your Glue database and workgroup.

The secureJsonData section holds your AWS credentials. Replace your-access-key-here and your-secret-key-here with your actual AWS access key and secret key. These are the same credentials you configured with aws configure in Step 1.

• Save dashboards/grafana-provisioning.yaml.
• Stop and remove the existing Grafana container, then restart it with the provisioning file mounted by running these commands:

docker stop grafana && docker rm grafana
docker run -d --name grafana -p 3000:3000 \
  -e GF_INSTALL_PLUGINS=grafana-athena-datasource \
  -v $(pwd)/dashboards/grafana-provisioning.yaml:/etc/grafana/provisioning/datasources/athena.yaml \
  grafana/grafana-oss:latest

Why restart instead of reload?

Grafana reads provisioning files at startup. By removing and recreating the container with a volume mount, you ensure the YAML file is available inside the container at the path Grafana expects: /etc/grafana/provisioning/datasources/.

• Open your browser and navigate to http://localhost:3000.
• Log in with username admin and password admin.
• Click the hamburger menu icon in the top left.
• Select Connections, then Data sources.
• Confirm that AthenaFinOps appears in the list with a green checkmark indicating a successful connection.

Don't see AthenaFinOps in the list?

• Verify the volume mount path is correct. Run docker inspect grafana and check that Mounts shows the YAML file mapped to /etc/grafana/provisioning/datasources/athena.yaml.
• Check that your AWS credentials in the YAML are valid. An invalid key causes a silent connection failure.
• Check the container logs with docker logs grafana for any provisioning errors on startup.

help me troubleshoot my Grafana Athena datasource not appearing after provisioning

Build the main chargeback dashboard with template variables and alerting

The chargeback dashboard gives the CFO per-customer cost visibility. Template variables let executives switch between cost centers and accounts without editing any queries.

• In Grafana, click the + icon in the left sidebar.
• Select New dashboard.
• Click the gear icon to open Dashboard settings.
• Set the title to FinOps Chargeback Dashboard.
• Navigate to the Variables tab and click Add variable.

Create three query variables that pull dynamic values from your CUR data:

• Create a variable named costcenter with type Query, datasource AthenaFinOps, and query:

SELECT DISTINCT resource_tags_user_cost_center
FROM v_chargeback_by_tag
WHERE resource_tags_user_cost_center IS NOT NULL

• Create a variable named account with the same type and datasource, and query:

SELECT DISTINCT line_item_usage_account_id
FROM v_chargeback_by_account

• Create a variable named timerange with type Custom and values: 7d,14d,30d,90d.
• Click Save dashboard to save the variable configuration.

What are template variables?

Template variables appear as dropdown selectors at the top of your dashboard. When an executive selects a different cost center from the $costcenter dropdown, every panel automatically re-queries with that filter applied.

This is what makes Grafana superior to QuickSight for FinOps. QuickSight cannot natively alert on CUR data or provide this level of dynamic filtering.

Now add the five core chargeback panels. For each panel, click + Add visualization, select AthenaFinOps as the datasource, and configure as follows:

• Panel 1: Total Monthly Spend (Gauge visualization). Query: SELECT SUM(line_item_unblended_cost) FROM v_chargeback_by_tag WHERE $__dateFilter(billing_period).
• Panel 2: Spend by CostCenter (Bar chart, stacked). Query uses v_chargeback_by_tag grouped by resource_tags_user_cost_center with a WHERE clause filtering on $costcenter.
• Panel 3: Spend by Account (Table visualization). Query uses v_chargeback_by_account filtered by $account.
• Panel 4: Daily Spend Trend (Time series visualization). Query uses v_daily_spend_trend with $__timeFilter(billing_period) for dynamic time range filtering.
• Panel 5: Top 10 Services (Pie chart visualization). Query groups by service and limits to top 10 by cost.

What is the anomaly band on the time-series panel?

On Panel 4, add a second query that computes the 7-day rolling average multiplied by 2. Display it as a threshold band. This visually highlights days where spend exceeds the expected baseline, making anomalies immediately visible to executives.

Next, configure alerting so Grafana fires a notification when daily spend deviates:

• On the Daily Spend Trend panel, click the Alert tab.
• Click Create alert rule from this panel.
• Set the condition: when the query result is greater than 2x the 7-day average.
• Under Notifications, configure a contact point for email or your SNS topic ARN.
• Click Save rule.

Finally, add annotations to mark remediation events on the dashboard:

• Go to Dashboard settings, then the Annotations tab.
• Click Add annotation query.
• Select AthenaFinOps as the datasource.
• Write a query that reads from the DynamoDB remediation log table (via an Athena federated query or DynamoDB connector) to show when anomaly events triggered auto-remediation.
• Save the dashboard.

Why Grafana alerting on CUR data matters

This is the capability that makes Grafana win over QuickSight for FinOps. QuickSight cannot natively alert on cost data. Grafana can run scheduled Athena queries and fire alerts when results cross a threshold. This closes the loop between visibility and action.

• Verify your dashboard by selecting different values from the $costcenter dropdown at the top. Panels should re-render with filtered data.

Panels showing no data?

• CUR data takes up to 24 hours for first delivery. If your CUR export was just created, panels will be empty until data arrives.
• Check that the Glue crawler has run at least once. Run it manually from the AWS console under AWS Glue if needed.
• Verify the database name in your panel queries matches finops-command-center-database.

help me debug why my Grafana Athena panels show no data

Build executive KPI and projection dashboards

The executive dashboard translates raw cost data into the KPIs that matter for a CFO presentation. These panels map directly to the Well-Architected Cost Pillar practices you will reference in the pitch deck.

• Create a new dashboard named FinOps Executive KPIs.
• Add the following seven KPI panels using the AthenaFinOps datasource:

• Panel 1: FinOps Maturity Score (Stat panel). Display where Apex Global sits on the Crawl/Walk/Run maturity model.
• Panel 2: Tag Coverage Rate (Gauge). Query calculates the percentage of spend that has a CostCenter tag attached.
• Panel 3: Waste Score (Stat panel). Aggregates unattached EBS volumes, idle EC2 instances, and unused Elastic IPs from Compute Optimizer data.
• Panel 4: Remediation MTTR (Stat panel). Calculates mean time from anomaly detection to Lambda remediation action using the DynamoDB log.
• Panel 5: Savings Plans Coverage % (Gauge with alert). Query uses v_savings_plans_coverage view. Set a threshold alert when utilization drops below 80%.
• Panel 6: Savings Plans Net Savings (Stat panel). Shows dollar value saved versus on-demand pricing.
• Panel 7: Non-allocable Spend % (Gauge). Shows the percentage of total spend that has no cost allocation tags. This is the metric that proves tag policy enforcement is working.

Why these seven KPIs?

Each KPI maps to a specific Well-Architected Cost Pillar practice. Tag Coverage maps to COST01 (governance). Waste Score maps to COST06 (resource type selection). Savings Plans Coverage maps to COST09 (demand management). The CFO does not care about technical details. They care about money saved, money wasted, and money untracked.

Next, add the 12-month projection panels to the same dashboard:

• Panel 8: Monthly Burn Rate Trend (Time series with linear projection). Query historical monthly totals from CUR and extend with a linear regression line.
• Panel 9: Cost Reduction Waterfall (Bar chart). Shows the projected impact of rightsizing, anomaly prevention, tag coverage improvement, and Savings Plans optimization.
• Panel 10: Savings Plans Recommendation Impact (Bar chart). Compares on-demand pricing versus committed spend with a break-even timeline.
• Panel 11: Quarter-over-Quarter Unit Economics (Table). Shows cost per customer or transaction over time to demonstrate efficiency improvements.

Finally, add the SAP-C02 Trap-Pattern Reference panel as a study aid for the presentation:

• Add a Text panel to the dashboard.
• Set the content mode to Markdown.
• Enter the following five key exam traps as a reference card:

• Cost Anomaly Detection uses billing data with ML, runs approximately 3 times daily. It is NOT CloudWatch metrics.
• Budget Actions can stop EC2 via SSM (STOP_EC2_INSTANCES), apply IAM policy, or apply SCP. Budgets alert on thresholds. Anomaly Detection alerts on deviations.
• Cost allocation tags can ONLY be activated from the management account.
• EC2 Instance Savings Plans cover one instance family in one region. Compute Savings Plans cover any instance family, Lambda, and Fargate across regions.
• Compute Optimizer requires 14 days of CloudWatch metrics. Trusted Advisor checks are immediate but less granular.

• Save both dashboards.
• Export each dashboard as JSON by clicking the Share icon, then Export, and save the files into your dashboards/ directory.

Why export as JSON?

Storing dashboard JSON files in your project directory enables dashboard-as-code. You can version control them in Git, diff changes between deployments, and provision them automatically in future environments. This is the principal-level approach to Grafana management.

• Verify the Executive KPI dashboard by opening it and confirming all 11 panels render without errors. The Savings Plans Coverage gauge should show the alert threshold line at 80%.

Savings Plans panels showing zero?

• Savings Plans data only appears in CUR if you have active Savings Plans. For a lab environment without commitments, the v_savings_plans_coverage view will return empty results. This is expected.
• The panels will populate once the CUR includes Savings Plans line items. For your presentation, you can use hardcoded sample data in the panel query to demonstrate the visualization.

help me understand why my Savings Plans panels show no data in Grafana

✔️ Awesome, I've got everything!

Great. Double check you have saved both dashboards and exported the JSON files to your dashboards/ directory.

ⓧ I'd like to double check the full code

Here is the complete provisioning YAML that should exist in your project:

apiVersion: 1
datasources:
  - name: AthenaFinOps
    type: grafana-athena-datasource
    access: proxy
    isDefault: true
    jsonData:
      authType: keys
      defaultRegion: us-east-1
      catalog: AwsDataCatalog
      database: finops-command-center-database
      workgroup: finops-command-center
    secureJsonData:
      accessKey: your-access-key-here
      secretKey: your-secret-key-here

Your dashboards are ready. Next up, you will build the governance and automated remediation layer that turns these insights into action.

Deploy Anomaly Detection and Automated Remediation

Your data pipeline and dashboards give you visibility into spend. Visibility alone does not stop runaway costs.

In this step, you will deploy the automated governance layer: Cost Anomaly Detection monitors that identify unusual spend, a Lambda-based remediator that auto-stops non-production instances, and layered guardrails (Budget Actions + Tag Policies) that prevent uncontrolled spend at the source.

In this step, get ready to:

• Deploy Cost Anomaly Detection monitors with SNS alerting.
• Write and deploy a Lambda remediator with cross-account reach and EventBridge integration.
• Deploy Budget Actions and an Organizations Tag Policy as layered guardrails.

Deploy Cost Anomaly Detection monitors and SNS alerting

Cost Anomaly Detection uses machine learning on billing data to identify spend deviations. It runs approximately 3 times daily after billing data is processed.

• Create the modules/anomaly-detection/ directory inside your finops-command-center project folder.
• Create a file called main.tf inside modules/anomaly-detection/ and paste the following code:

# SNS topic for anomaly alert delivery
resource "aws_sns_topic" "cost_anomaly_alerts" {
  name = "${var.project_name}-anomaly-alerts"
}

# Allow Cost Anomaly Detection service to publish to SNS
resource "aws_sns_topic_policy" "cost_anomaly_publish" {
  arn = aws_sns_topic.cost_anomaly_alerts.arn
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Sid       = "AWSAnomalyDetectionSNSPublishingPermissions"
      Effect    = "Allow"
      Principal = { Service = "costalerts.amazonaws.com" }
      Action    = "SNS:Publish"
      Resource  = aws_sns_topic.cost_anomaly_alerts.arn
      Condition = {
        StringEquals = {
          "aws:SourceAccount" = var.account_id
        }
      }
    }]
  })
}

# DIMENSIONAL monitor - automatically tracks ALL services
resource "aws_ce_anomaly_monitor" "service_monitor" {
  name              = "ApexGlobal-ServiceMonitor"
  monitor_type      = "DIMENSIONAL"
  monitor_dimension = "SERVICE"
}

# CUSTOM monitor - scoped to non-prod environments only
resource "aws_ce_anomaly_monitor" "nonprod_monitor" {
  name         = "ApexGlobal-NonProdMonitor"
  monitor_type = "CUSTOM"
  monitor_specification = jsonencode({
    Tags = {
      Key    = "Environment"
      Values = ["Development", "Test", "Staging"]
    }
  })
}

# Subscription fires IMMEDIATELY when anomaly impact >= $50
resource "aws_ce_anomaly_subscription" "immediate_alerts" {
  name      = "ApexGlobal-ImmediateAlerts"
  frequency = "IMMEDIATE"

  monitor_arn_list = [
    aws_ce_anomaly_monitor.service_monitor.arn,
    aws_ce_anomaly_monitor.nonprod_monitor.arn,
  ]

  subscriber {
    type    = "SNS"
    address = aws_sns_topic.cost_anomaly_alerts.arn
  }

  threshold_expression {
    dimension {
      key           = "ANOMALY_TOTAL_IMPACT_ABSOLUTE"
      match_options = ["GREATER_THAN_OR_EQUAL"]
      values        = ["50"]
    }
  }
}

variable "project_name" {
  type = string
}

variable "account_id" {
  type = string
}

output "sns_topic_arn" {
  value = aws_sns_topic.cost_anomaly_alerts.arn
}

output "service_monitor_arn" {
  value = aws_ce_anomaly_monitor.service_monitor.arn
}

What does this code do?

This module deploys the detection layer of your FinOps governance system.

• The SNS topic with costalerts.amazonaws.com publish permission is the delivery channel for anomaly alerts.
• The DIMENSIONAL monitor (SERVICE) automatically tracks spend deviations across all AWS services without maintenance.
• The CUSTOM monitor scopes specifically to non-production environments, catching dev/test spend spikes independently.
• The subscription triggers IMMEDIATELY when the absolute dollar impact of an anomaly reaches $50 or more, using ANOMALY_TOTAL_IMPACT_ABSOLUTE as the threshold key.

💡 Why two monitors instead of one?

The DIMENSIONAL monitor gives you breadth. It watches every service automatically. The CUSTOM monitor gives you depth. It focuses specifically on non-production environments where runaway spend is most likely and auto-remediation is safest.

This combination ensures you catch both broad service-level anomalies and targeted non-prod spikes.

• Save modules/anomaly-detection/main.tf.

Seeing a provider error?

Cost Anomaly Detection resources (aws_ce_anomaly_monitor) require the AWS provider 5.0.0+ which you configured in Step 1. If Terraform complains about an unknown resource type, run terraform init -upgrade to update your provider.

help me debug a Terraform provider version error for aws_ce_anomaly_monitor

Write the Lambda remediator and deploy the remediation module

The Lambda function is the action arm of your governance loop. It receives anomaly events from EventBridge, identifies the affected account, and takes automated action based on a safety rule: AUTOMATIC remediation for non-production, MANUAL approval for production.

• Create the lambda/ directory inside your finops-command-center project folder.
• Create a file called remediator.py inside lambda/ and paste the following code:

import json
import os
import boto3
from datetime import datetime, timezone

# Initialize AWS SDK clients
dynamodb = boto3.resource("dynamodb")
sns = boto3.client("sns")


def get_ec2_client(account_id=None):
    # If a different account is specified, assume cross-account role
    if account_id and account_id != boto3.client("sts").get_caller_identity()["Account"]:
        role_name = os.environ["CROSS_ACCOUNT_ROLE_NAME"]
        role_arn = f"arn:aws:iam::{account_id}:role/{role_name}"
        sts = boto3.client("sts")
        credentials = sts.assume_role(
            RoleArn=role_arn,
            RoleSessionName="finops-remediator"
        )["Credentials"]
        return boto3.client(
            "ec2",
            aws_access_key_id=credentials["AccessKeyId"],
            aws_secret_access_key=credentials["SecretAccessKey"],
            aws_session_token=credentials["SessionToken"],
        )
    # Default: use the Lambda execution role's local credentials
    return boto3.client("ec2")


def lambda_handler(event, context):
    # Load configuration from environment variables
    table_name = os.environ["REMEDIATION_TABLE"]
    non_prod_envs = os.environ["NON_PROD_ENVS"].split(",")
    approval_topic = os.environ["APPROVAL_SNS_TOPIC"]
    table = dynamodb.Table(table_name)

    # Parse the anomaly event from EventBridge
    detail = event.get("detail", {})
    anomaly_id = detail.get("anomalyId", "unknown")
    impact = detail.get("impact", {})
    total_impact = impact.get("totalImpact", 0)
    root_causes = detail.get("rootCauses", [])

    stopped_instances = []
    action_taken = "NONE"

    # Identify the affected account from root causes
    target_account = None
    if root_causes:
        target_account = root_causes[0].get("linkedAccount")

    ec2 = get_ec2_client(target_account)

    # Find running non-prod instances
    response = ec2.describe_instances(
        Filters=[
            {"Name": "tag:Environment", "Values": non_prod_envs},
            {"Name": "instance-state-name", "Values": ["running"]},
        ]
    )

    instance_ids = []
    for reservation in response["Reservations"]:
        for instance in reservation["Instances"]:
            instance_ids.append(instance["InstanceId"])

    if instance_ids:
        # Non-prod: auto-stop running instances
        ec2.stop_instances(InstanceIds=instance_ids)
        stopped_instances = instance_ids
        action_taken = "STOP_NON_PROD_EC2"
    else:
        # Prod or no instances: send approval request for manual review
        sns.publish(
            TopicArn=approval_topic,
            Subject=f"[FinOps] Anomaly {anomaly_id} requires manual review",
            Message=json.dumps({
                "anomaly_id": anomaly_id,
                "total_impact": total_impact,
                "root_causes": root_causes,
                "action_required": "Review and manually remediate if needed",
            }, indent=2),
        )
        action_taken = "APPROVAL_REQUESTED"

    # Log every action to DynamoDB for Grafana annotation queries
    timestamp = datetime.now(timezone.utc).isoformat()
    table.put_item(
        Item={
            "anomaly_id": anomaly_id,
            "timestamp": timestamp,
            "total_impact": str(total_impact),
            "instances_stopped": stopped_instances,
            "root_causes": json.dumps(root_causes),
            "action": action_taken,
            "target_account": target_account or "local",
        }
    )

    return {
        "statusCode": 200,
        "body": json.dumps({
            "anomaly_id": anomaly_id,
            "action": action_taken,
            "instances_stopped": len(stopped_instances),
            "instance_ids": stopped_instances,
        }),
    }

What does this Lambda function do?

This is the principal-level safety pattern: AUTOMATIC for non-prod, MANUAL (human approval) for prod.

• get_ec2_client handles cross-account reach by assuming a role in member accounts via sts:AssumeRole.
• lambda_handler parses the EventBridge payload (anomalyId, rootCauses, impact), finds running non-prod instances, and stops them.
• If no non-prod instances are found (indicating a production anomaly), it publishes an approval request to SNS instead of auto-remediating.
• Every action is logged to DynamoDB so Grafana can query remediation events as dashboard annotations.

• Save lambda/remediator.py.
• Now create the Terraform module that deploys this Lambda function with all its supporting infrastructure. Create the modules/remediation/ directory and a main.tf file inside it with the following code:

# IAM trust policy allowing Lambda service to assume this role
data "aws_iam_policy_document" "lambda_assume" {
  statement {
    effect  = "Allow"
    principals {
      type        = "Service"
      identifiers = ["lambda.amazonaws.com"]
    }
    actions = ["sts:AssumeRole"]
  }
}

resource "aws_iam_role" "remediator_role" {
  name               = "${var.project_name}-remediator-role"
  assume_role_policy = data.aws_iam_policy_document.lambda_assume.json
}

# Permissions: cross-account assume-role, EC2 stop, logs, DynamoDB, SNS
resource "aws_iam_role_policy" "remediator_permissions" {
  name = "${var.project_name}-remediator-policy"
  role = aws_iam_role.remediator_role.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid      = "AssumeRoleInMemberAccounts"
        Effect   = "Allow"
        Action   = "sts:AssumeRole"
        Resource = "arn:aws:iam::*:role/${var.project_name}-member-remediator"
      },
      {
        Sid      = "LocalEC2Remediation"
        Effect   = "Allow"
        Action   = ["ec2:DescribeInstances", "ec2:StopInstances"]
        Resource = "*"
        Condition = {
          StringEquals = {
            "aws:ResourceTag/Environment" = ["Development", "Test", "Staging"]
          }
        }
      },
      {
        Sid      = "DescribeAllInstances"
        Effect   = "Allow"
        Action   = ["ec2:DescribeInstances"]
        Resource = "*"
      },
      {
        Sid      = "CloudWatchLogs"
        Effect   = "Allow"
        Action   = ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"]
        Resource = "arn:aws:logs:*:*:*"
      },
      {
        Sid      = "DynamoDBLog"
        Effect   = "Allow"
        Action   = ["dynamodb:PutItem"]
        Resource = aws_dynamodb_table.remediation_log.arn
      },
      {
        Sid      = "SNSApproval"
        Effect   = "Allow"
        Action   = ["sns:Publish"]
        Resource = var.sns_topic_arn
      }
    ]
  })
}

# DynamoDB table for remediation action logging (Grafana reads this)
resource "aws_dynamodb_table" "remediation_log" {
  name         = "${var.project_name}-remediation-log"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "anomaly_id"
  range_key    = "timestamp"

  attribute {
    name = "anomaly_id"
    type = "S"
  }

  attribute {
    name = "timestamp"
    type = "S"
  }
}

# Package the Lambda function from the lambda/ directory
data "archive_file" "remediator_zip" {
  type        = "zip"
  source_file = "${path.module}/../../lambda/remediator.py"
  output_path = "${path.module}/../../lambda/remediator.zip"
}

resource "aws_lambda_function" "remediator" {
  function_name    = "${var.project_name}-remediator"
  filename         = data.archive_file.remediator_zip.output_path
  source_code_hash = data.archive_file.remediator_zip.output_base64sha256
  handler          = "remediator.lambda_handler"
  runtime          = "python3.12"
  role             = aws_iam_role.remediator_role.arn
  timeout          = 60

  environment {
    variables = {
      REMEDIATION_TABLE       = aws_dynamodb_table.remediation_log.name
      NON_PROD_ENVS           = "Development,Test,Staging"
      CROSS_ACCOUNT_ROLE_NAME = "${var.project_name}-member-remediator"
      APPROVAL_SNS_TOPIC      = var.sns_topic_arn
    }
  }
}

# EventBridge rule matching Cost Anomaly Detection events
resource "aws_cloudwatch_event_rule" "anomaly_detected" {
  name        = "${var.project_name}-anomaly-detected"
  description = "Triggers Lambda when Cost Anomaly Detection fires"

  event_pattern = jsonencode({
    source      = ["aws.ce"]
    detail-type = ["Anomaly Detected"]
  })
}

# Connect EventBridge to Lambda
resource "aws_cloudwatch_event_target" "lambda_target" {
  rule = aws_cloudwatch_event_rule.anomaly_detected.name
  arn  = aws_lambda_function.remediator.arn
}

# Grant EventBridge permission to invoke the Lambda
resource "aws_lambda_permission" "eventbridge_invoke" {
  statement_id  = "AllowEventBridgeInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.remediator.function_name
  principal     = "events.amazonaws.com"
  source_arn    = aws_cloudwatch_event_rule.anomaly_detected.arn
}

variable "project_name" {
  type = string
}

variable "sns_topic_arn" {
  type = string
}

output "lambda_function_arn" {
  value = aws_lambda_function.remediator.arn
}

output "remediation_table_name" {
  value = aws_dynamodb_table.remediation_log.name
}

What does this module do?

This module creates the entire automated remediation pipeline.

• The IAM role grants the Lambda least-privilege permissions. The StopInstances action is conditioned on the Environment tag, so production instances can never be auto-stopped.
• The DynamoDB table logs every remediation action for audit trail and Grafana annotation queries.
• The EventBridge rule matches source = "aws.ce" and detail-type = "Anomaly Detected" to connect Cost Anomaly Detection directly to the Lambda.
• The archive_file data source zips your Python file for Lambda deployment without a separate build step.

• Save modules/remediation/main.tf.

Lambda packaging error?

The archive_file data source references a relative path to lambda/remediator.py. If Terraform cannot find the file, confirm your directory structure has lambda/ at the project root (same level as modules/).

help me fix a Terraform archive_file path error for my Lambda function

Deploy Budget Actions and Tag Policy guardrails

Budget Actions and Tag Policies are your static guardrails. They complement the dynamic Lambda remediator by providing threshold-based enforcement and proactive tag compliance at resource creation time.

• Create the modules/budget-actions/ directory and a main.tf file inside it with the following code:

# Monthly cost budget scoped to non-production environments
resource "aws_budgets_budget" "nonprod_monthly" {
  name         = "${var.project_name}-nonprod-monthly"
  budget_type  = "COST"
  limit_amount = "500"
  limit_unit   = "USD"
  time_unit    = "MONTHLY"

  cost_filter {
    name   = "TagKeyValue"
    values = ["user:Environment$Development", "user:Environment$Test", "user:Environment$Staging"]
  }
}

# IAM role that Budget Actions assumes to execute enforcement
resource "aws_iam_role" "budget_action_role" {
  name = "${var.project_name}-budget-action-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Principal = { Service = "budgets.amazonaws.com" }
      Action = "sts:AssumeRole"
    }]
  })
}

resource "aws_iam_role_policy" "budget_action_permissions" {
  name = "${var.project_name}-budget-action-policy"
  role = aws_iam_role.budget_action_role.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect   = "Allow"
        Action   = ["iam:AttachUserPolicy", "iam:AttachGroupPolicy", "iam:AttachRolePolicy"]
        Resource = "*"
      },
      {
        Effect   = "Allow"
        Action   = ["ssm:StartAutomationExecution"]
        Resource = "*"
      },
      {
        Effect   = "Allow"
        Action   = ["ec2:StopInstances", "rds:StopDBInstance"]
        Resource = "*"
      }
    ]
  })
}

# The deny policy that gets applied at 80% threshold
resource "aws_iam_policy" "deny_ec2_launch" {
  name = "${var.project_name}-deny-ec2-launch"
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect   = "Deny"
      Action   = ["ec2:RunInstances"]
      Resource = "*"
    }]
  })
}

# At 80% forecasted spend, attach the deny policy automatically
resource "aws_budgets_budget_action" "deny_at_80_percent" {
  budget_name        = aws_budgets_budget.nonprod_monthly.name
  action_type        = "APPLY_IAM_POLICY"
  approval_model     = "AUTOMATIC"
  notification_type  = "FORECASTED"
  execution_role_arn = aws_iam_role.budget_action_role.arn

  action_threshold {
    action_threshold_type  = "PERCENTAGE"
    action_threshold_value = 80
  }

  definition {
    iam_action_definition {
      policy_arn = aws_iam_policy.deny_ec2_launch.arn
      roles      = [var.target_role_name]
    }
  }

  subscriber {
    address           = var.alert_email
    subscription_type = "EMAIL"
  }
}

variable "project_name" {
  type = string
}

variable "target_role_name" {
  type    = string
  default = "developers"
}

variable "alert_email" {
  type    = string
  default = "finops@apex-global.example.com"
}

Why Lambda AND Budget Actions?

Budget Actions require specific instance IDs upfront. They target named resources, not dynamic queries. This makes them excellent for static guardrails ("never exceed this threshold") but poor at discovering and stopping newly-launched rogue instances.

The Lambda remediator uses dynamic discovery (ec2:DescribeInstances with tag filters) at trigger time. Budget Actions at 80% prevent new launches. Lambda at anomaly time stops existing runners. Together they form a layered defense.

• Save modules/budget-actions/main.tf.
• Now create the Tag Policy module. Create the modules/tag-policy/ directory and a main.tf file inside it with the following code:

# Organizations Tag Policy requiring cost allocation tags
resource "aws_organizations_policy" "cost_allocation_tags" {
  name        = "${var.project_name}-required-tags"
  description = "Enforces CostCenter, Environment, and Owner tags for cost allocation"
  type        = "TAG_POLICY"

  content = jsonencode({
    tags = {
      CostCenter = {
        tag_key = { "@@assign" = "CostCenter" }
        report_required_tag_for = { "@@assign" = ["ec2:instance", "s3:bucket", "rds:db", "lambda:function"] }
      }
      Environment = {
        tag_key = { "@@assign" = "Environment" }
        tag_value = { "@@assign" = ["Production", "Staging", "Development", "Test"] }
        report_required_tag_for = { "@@assign" = ["ec2:ALL_SUPPORTED", "rds:ALL_SUPPORTED", "s3:ALL_SUPPORTED"] }
      }
      Owner = {
        tag_key = { "@@assign" = "Owner" }
        report_required_tag_for = { "@@assign" = ["ec2:ALL_SUPPORTED"] }
      }
    }
  })
}

# Attach the tag policy to the organization root (applies to all accounts)
resource "aws_organizations_policy_attachment" "root_tag_policy" {
  policy_id = aws_organizations_policy.cost_allocation_tags.id
  target_id = var.org_root_id
}

variable "project_name" {
  type = string
}

variable "org_root_id" {
  type        = string
  description = "The ID of the organization root to attach the tag policy to"
}

What does the Tag Policy enforce?

This policy requires three tags for cost allocation: CostCenter, Environment, and Owner.

The Environment tag additionally enforces allowed values (Production, Staging, Development, Test). The @@assign operator means this policy's values apply to all accounts in the organization. By attaching to the organization root, every account inherits the enforcement.

Tag Policies enforce at creation time. Cost allocation tags (activated from the management account) make those tags visible in billing reports. You need both mechanisms working together.

• Save modules/tag-policy/main.tf.
• Deploy all four new modules by running:

terraform init
terraform apply

• Terraform will prompt you to confirm. Type yes and press Enter.
• Wait for the apply to complete. You should see output confirming that the SNS topic, anomaly monitors, Lambda function, EventBridge rule, DynamoDB table, budget, budget action, and tag policy were all created.

What about the 24-hour detection delay?

New Cost Anomaly Detection monitors need 24 hours before they start detecting anomalies. New services tracked by the monitor also need 10 days of historical data for accurate baselines. This is expected behavior.

In the next step, you will validate the remediation chain using a synthetic EventBridge event instead of waiting for a real anomaly.

🙋‍♀️ Tag Policy attachment failing?

The aws_organizations_policy resource requires that Tag Policies are enabled as a policy type in your organization.

• In the AWS console, navigate to Organizations.
• Select Policies, then Tag policies.
• Enable tag policies if they are not already active.

help me enable Tag Policies in my AWS Organization so Terraform can create tag policy resources

✔️ Awesome, I've got everything!

Great. Confirm you have saved all four module files and lambda/remediator.py.

ⓧ I'd like to double check the full code

Here is the complete code for each file created in this step.

modules/anomaly-detection/main.tf

resource "aws_sns_topic" "cost_anomaly_alerts" {
  name = "${var.project_name}-anomaly-alerts"
}

resource "aws_sns_topic_policy" "cost_anomaly_publish" {
  arn = aws_sns_topic.cost_anomaly_alerts.arn
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Sid       = "AWSAnomalyDetectionSNSPublishingPermissions"
      Effect    = "Allow"
      Principal = { Service = "costalerts.amazonaws.com" }
      Action    = "SNS:Publish"
      Resource  = aws_sns_topic.cost_anomaly_alerts.arn
      Condition = {
        StringEquals = {
          "aws:SourceAccount" = var.account_id
        }
      }
    }]
  })
}

resource "aws_ce_anomaly_monitor" "service_monitor" {
  name              = "ApexGlobal-ServiceMonitor"
  monitor_type      = "DIMENSIONAL"
  monitor_dimension = "SERVICE"
}

resource "aws_ce_anomaly_monitor" "nonprod_monitor" {
  name         = "ApexGlobal-NonProdMonitor"
  monitor_type = "CUSTOM"
  monitor_specification = jsonencode({
    Tags = {
      Key    = "Environment"
      Values = ["Development", "Test", "Staging"]
    }
  })
}

resource "aws_ce_anomaly_subscription" "immediate_alerts" {
  name      = "ApexGlobal-ImmediateAlerts"
  frequency = "IMMEDIATE"

  monitor_arn_list = [
    aws_ce_anomaly_monitor.service_monitor.arn,
    aws_ce_anomaly_monitor.nonprod_monitor.arn,
  ]

  subscriber {
    type    = "SNS"
    address = aws_sns_topic.cost_anomaly_alerts.arn
  }

  threshold_expression {
    dimension {
      key           = "ANOMALY_TOTAL_IMPACT_ABSOLUTE"
      match_options = ["GREATER_THAN_OR_EQUAL"]
      values        = ["50"]
    }
  }
}

variable "project_name" {
  type = string
}

variable "account_id" {
  type = string
}

output "sns_topic_arn" {
  value = aws_sns_topic.cost_anomaly_alerts.arn
}

output "service_monitor_arn" {
  value = aws_ce_anomaly_monitor.service_monitor.arn
}

lambda/remediator.py

import json
import os
import boto3
from datetime import datetime, timezone

dynamodb = boto3.resource("dynamodb")
sns = boto3.client("sns")


def get_ec2_client(account_id=None):
    if account_id and account_id != boto3.client("sts").get_caller_identity()["Account"]:
        role_name = os.environ["CROSS_ACCOUNT_ROLE_NAME"]
        role_arn = f"arn:aws:iam::{account_id}:role/{role_name}"
        sts = boto3.client("sts")
        credentials = sts.assume_role(
            RoleArn=role_arn,
            RoleSessionName="finops-remediator"
        )["Credentials"]
        return boto3.client(
            "ec2",
            aws_access_key_id=credentials["AccessKeyId"],
            aws_secret_access_key=credentials["SecretAccessKey"],
            aws_session_token=credentials["SessionToken"],
        )
    return boto3.client("ec2")


def lambda_handler(event, context):
    table_name = os.environ["REMEDIATION_TABLE"]
    non_prod_envs = os.environ["NON_PROD_ENVS"].split(",")
    approval_topic = os.environ["APPROVAL_SNS_TOPIC"]
    table = dynamodb.Table(table_name)

    detail = event.get("detail", {})
    anomaly_id = detail.get("anomalyId", "unknown")
    impact = detail.get("impact", {})
    total_impact = impact.get("totalImpact", 0)
    root_causes = detail.get("rootCauses", [])

    stopped_instances = []
    action_taken = "NONE"

    target_account = None
    if root_causes:
        target_account = root_causes[0].get("linkedAccount")

    ec2 = get_ec2_client(target_account)

    response = ec2.describe_instances(
        Filters=[
            {"Name": "tag:Environment", "Values": non_prod_envs},
            {"Name": "instance-state-name", "Values": ["running"]},
        ]
    )

    instance_ids = []
    for reservation in response["Reservations"]:
        for instance in reservation["Instances"]:
            instance_ids.append(instance["InstanceId"])

    if instance_ids:
        ec2.stop_instances(InstanceIds=instance_ids)
        stopped_instances = instance_ids
        action_taken = "STOP_NON_PROD_EC2"
    else:
        sns.publish(
            TopicArn=approval_topic,
            Subject=f"[FinOps] Anomaly {anomaly_id} requires manual review",
            Message=json.dumps({
                "anomaly_id": anomaly_id,
                "total_impact": total_impact,
                "root_causes": root_causes,
                "action_required": "Review and manually remediate if needed",
            }, indent=2),
        )
        action_taken = "APPROVAL_REQUESTED"

    timestamp = datetime.now(timezone.utc).isoformat()
    table.put_item(
        Item={
            "anomaly_id": anomaly_id,
            "timestamp": timestamp,
            "total_impact": str(total_impact),
            "instances_stopped": stopped_instances,
            "root_causes": json.dumps(root_causes),
            "action": action_taken,
            "target_account": target_account or "local",
        }
    )

    return {
        "statusCode": 200,
        "body": json.dumps({
            "anomaly_id": anomaly_id,
            "action": action_taken,
            "instances_stopped": len(stopped_instances),
            "instance_ids": stopped_instances,
        }),
    }

modules/remediation/main.tf

data "aws_iam_policy_document" "lambda_assume" {
  statement {
    effect  = "Allow"
    principals {
      type        = "Service"
      identifiers = ["lambda.amazonaws.com"]
    }
    actions = ["sts:AssumeRole"]
  }
}

resource "aws_iam_role" "remediator_role" {
  name               = "${var.project_name}-remediator-role"
  assume_role_policy = data.aws_iam_policy_document.lambda_assume.json
}

resource "aws_iam_role_policy" "remediator_permissions" {
  name = "${var.project_name}-remediator-policy"
  role = aws_iam_role.remediator_role.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid      = "AssumeRoleInMemberAccounts"
        Effect   = "Allow"
        Action   = "sts:AssumeRole"
        Resource = "arn:aws:iam::*:role/${var.project_name}-member-remediator"
      },
      {
        Sid      = "LocalEC2Remediation"
        Effect   = "Allow"
        Action   = ["ec2:DescribeInstances", "ec2:StopInstances"]
        Resource = "*"
        Condition = {
          StringEquals = {
            "aws:ResourceTag/Environment" = ["Development", "Test", "Staging"]
          }
        }
      },
      {
        Sid      = "DescribeAllInstances"
        Effect   = "Allow"
        Action   = ["ec2:DescribeInstances"]
        Resource = "*"
      },
      {
        Sid      = "CloudWatchLogs"
        Effect   = "Allow"
        Action   = ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"]
        Resource = "arn:aws:logs:*:*:*"
      },
      {
        Sid      = "DynamoDBLog"
        Effect   = "Allow"
        Action   = ["dynamodb:PutItem"]
        Resource = aws_dynamodb_table.remediation_log.arn
      },
      {
        Sid      = "SNSApproval"
        Effect   = "Allow"
        Action   = ["sns:Publish"]
        Resource = var.sns_topic_arn
      }
    ]
  })
}

resource "aws_dynamodb_table" "remediation_log" {
  name         = "${var.project_name}-remediation-log"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "anomaly_id"
  range_key    = "timestamp"

  attribute {
    name = "anomaly_id"
    type = "S"
  }

  attribute {
    name = "timestamp"
    type = "S"
  }
}

data "archive_file" "remediator_zip" {
  type        = "zip"
  source_file = "${path.module}/../../lambda/remediator.py"
  output_path = "${path.module}/../../lambda/remediator.zip"
}

resource "aws_lambda_function" "remediator" {
  function_name    = "${var.project_name}-remediator"
  filename         = data.archive_file.remediator_zip.output_path
  source_code_hash = data.archive_file.remediator_zip.output_base64sha256
  handler          = "remediator.lambda_handler"
  runtime          = "python3.12"
  role             = aws_iam_role.remediator_role.arn
  timeout          = 60

  environment {
    variables = {
      REMEDIATION_TABLE       = aws_dynamodb_table.remediation_log.name
      NON_PROD_ENVS           = "Development,Test,Staging"
      CROSS_ACCOUNT_ROLE_NAME = "${var.project_name}-member-remediator"
      APPROVAL_SNS_TOPIC      = var.sns_topic_arn
    }
  }
}

resource "aws_cloudwatch_event_rule" "anomaly_detected" {
  name        = "${var.project_name}-anomaly-detected"
  description = "Triggers Lambda when Cost Anomaly Detection fires"

  event_pattern = jsonencode({
    source      = ["aws.ce"]
    detail-type = ["Anomaly Detected"]
  })
}

resource "aws_cloudwatch_event_target" "lambda_target" {
  rule = aws_cloudwatch_event_rule.anomaly_detected.name
  arn  = aws_lambda_function.remediator.arn
}

resource "aws_lambda_permission" "eventbridge_invoke" {
  statement_id  = "AllowEventBridgeInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.remediator.function_name
  principal     = "events.amazonaws.com"
  source_arn    = aws_cloudwatch_event_rule.anomaly_detected.arn
}

variable "project_name" {
  type = string
}

variable "sns_topic_arn" {
  type = string
}

output "lambda_function_arn" {
  value = aws_lambda_function.remediator.arn
}

output "remediation_table_name" {
  value = aws_dynamodb_table.remediation_log.name
}

modules/budget-actions/main.tf

resource "aws_budgets_budget" "nonprod_monthly" {
  name         = "${var.project_name}-nonprod-monthly"
  budget_type  = "COST"
  limit_amount = "500"
  limit_unit   = "USD"
  time_unit    = "MONTHLY"

  cost_filter {
    name   = "TagKeyValue"
    values = ["user:Environment$Development", "user:Environment$Test", "user:Environment$Staging"]
  }
}

resource "aws_iam_role" "budget_action_role" {
  name = "${var.project_name}-budget-action-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Principal = { Service = "budgets.amazonaws.com" }
      Action = "sts:AssumeRole"
    }]
  })
}

resource "aws_iam_role_policy" "budget_action_permissions" {
  name = "${var.project_name}-budget-action-policy"
  role = aws_iam_role.budget_action_role.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect   = "Allow"
        Action   = ["iam:AttachUserPolicy", "iam:AttachGroupPolicy", "iam:AttachRolePolicy"]
        Resource = "*"
      },
      {
        Effect   = "Allow"
        Action   = ["ssm:StartAutomationExecution"]
        Resource = "*"
      },
      {
        Effect   = "Allow"
        Action   = ["ec2:StopInstances", "rds:StopDBInstance"]
        Resource = "*"
      }
    ]
  })
}

resource "aws_iam_policy" "deny_ec2_launch" {
  name = "${var.project_name}-deny-ec2-launch"
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect   = "Deny"
      Action   = ["ec2:RunInstances"]
      Resource = "*"
    }]
  })
}

resource "aws_budgets_budget_action" "deny_at_80_percent" {
  budget_name        = aws_budgets_budget.nonprod_monthly.name
  action_type        = "APPLY_IAM_POLICY"
  approval_model     = "AUTOMATIC"
  notification_type  = "FORECASTED"
  execution_role_arn = aws_iam_role.budget_action_role.arn

  action_threshold {
    action_threshold_type  = "PERCENTAGE"
    action_threshold_value = 80
  }

  definition {
    iam_action_definition {
      policy_arn = aws_iam_policy.deny_ec2_launch.arn
      roles      = [var.target_role_name]
    }
  }

  subscriber {
    address           = var.alert_email
    subscription_type = "EMAIL"
  }
}

variable "project_name" {
  type = string
}

variable "target_role_name" {
  type    = string
  default = "developers"
}

variable "alert_email" {
  type    = string
  default = "finops@apex-global.example.com"
}

modules/tag-policy/main.tf

resource "aws_organizations_policy" "cost_allocation_tags" {
  name        = "${var.project_name}-required-tags"
  description = "Enforces CostCenter, Environment, and Owner tags for cost allocation"
  type        = "TAG_POLICY"

  content = jsonencode({
    tags = {
      CostCenter = {
        tag_key = { "@@assign" = "CostCenter" }
        report_required_tag_for = { "@@assign" = ["ec2:instance", "s3:bucket", "rds:db", "lambda:function"] }
      }
      Environment = {
        tag_key = { "@@assign" = "Environment" }
        tag_value = { "@@assign" = ["Production", "Staging", "Development", "Test"] }
        report_required_tag_for = { "@@assign" = ["ec2:ALL_SUPPORTED", "rds:ALL_SUPPORTED", "s3:ALL_SUPPORTED"] }
      }
      Owner = {
        tag_key = { "@@assign" = "Owner" }
        report_required_tag_for = { "@@assign" = ["ec2:ALL_SUPPORTED"] }
      }
    }
  })
}

resource "aws_organizations_policy_attachment" "root_tag_policy" {
  policy_id = aws_organizations_policy.cost_allocation_tags.id
  target_id = var.org_root_id
}

variable "project_name" {
  type = string
}

variable "org_root_id" {
  type        = string
  description = "The ID of the organization root to attach the tag policy to"
}

Your governance layer is now live. You have detection (Cost Anomaly Detection), response (Lambda + EventBridge), prevention (Budget Actions), and compliance (Tag Policies) all working as a coordinated system. Next up, you will validate this entire chain end-to-end with a spike test.

Validate the End-to-End Pipeline

You have built every layer of the FinOps command center: data ingestion, anomaly detection, automated remediation, budget guardrails, and tag governance.

Now you need to prove the entire chain works end-to-end. In this step, you will simulate a "forgotten dev fleet" scenario by launching a spike instance, firing a synthetic anomaly event, and confirming that the Lambda stops the instance and logs the action for Grafana.

In this step, get ready to:

• Launch a t3.micro spike instance tagged as non-production to simulate runaway spend.
• Send a synthetic anomaly event to trigger the remediation Lambda.
• Verify the Grafana dashboard reflects the remediation action with annotations and chargeback data.

Launch the validation spike

Cost Anomaly Detection needs 24 hours before new monitors start detecting anomalies. You cannot wait a day to validate your pipeline, so you will use a two-part approach: deploy a real EC2 instance to prove the infrastructure works, then trigger the Lambda directly with a synthetic EventBridge event.

• Create a file called spike.tf in your finops-command-center root directory by pasting the following code:

# Fetch the latest Amazon Linux 2023 AMI for the spike instance
data "aws_ami" "al2023" {
  most_recent = true
  owners      = ["amazon"]

  filter {
    name   = "name"
    values = ["al2023-ami-2023.*-x86_64"]
  }

  filter {
    name   = "virtualization-type"
    values = ["hvm"]
  }
}

# Deploy a t3.micro spike instance to simulate a forgotten dev fleet
resource "aws_instance" "spike_test" {
  ami           = data.aws_ami.al2023.id
  instance_type = "t3.micro"

  tags = {
    Name        = "FinOps-Spike-Test"
    Environment = "Development"
    CostCenter  = "SPIKE-TEST"
    Owner       = "finops-validation"
  }
}

What does this code do?

The data "aws_ami" block finds the latest Amazon Linux 2023 AMI so you do not need to hardcode an AMI ID.

The aws_instance resource launches a t3.micro (free tier eligible) tagged Environment=Development and CostCenter=SPIKE-TEST. This simulates the "forgotten dev fleet" your Lambda is designed to stop.

• Deploy the spike instance by running these commands:

terraform plan -target=aws_instance.spike_test
terraform apply -target=aws_instance.spike_test -auto-approve

Why use -target?

The -target flag tells Terraform to only deploy the spike instance without touching your existing infrastructure. This keeps the validation isolated.

• Verify the instance is running by running this command:

aws ec2 describe-instances --filters "Name=tag:CostCenter,Values=SPIKE-TEST" "Name=instance-state-name,Values=running" --query "Reservations[].Instances[].{ID:InstanceId,State:State.Name,Tags:Tags}" --output table

You should see your t3.micro instance listed with state running and the SPIKE-TEST cost center tag.

Instance not showing as running?

If the instance shows pending, wait 30 seconds and run the command again. EC2 instances take a moment to reach the running state.

If Terraform failed with a permissions error, confirm your AWS CLI credentials belong to the management account with EC2 launch permissions in us-east-1.

Why can't I launch my spike instance?

Send a synthetic anomaly event to trigger the Lambda

Cost Anomaly Detection needs 24 hours before it starts generating real alerts on new monitors. To validate the remediation chain immediately, you will send a synthetic event to EventBridge that matches the documented event schema.

• Create a file called test-event.json in your finops-command-center root directory with the following content:

[
  {
    "Source": "aws.ce",
    "DetailType": "Anomaly Detected",
    "Detail": "{\"anomalyId\":\"test-spike-001\",\"anomalyScore\":{\"maxScore\":95},\"impact\":{\"totalActualSpend\":250,\"totalExpectedSpend\":50,\"totalImpact\":200,\"totalImpactPercentage\":300},\"rootCauses\":[{\"linkedAccount\":null,\"service\":\"Amazon Elastic Compute Cloud - Compute\",\"usageType\":\"USW2-BoxUsage:t3.micro\",\"region\":\"us-east-1\"}]}"
  }
]

What does this mock event contain?

The event mirrors the documented Cost Anomaly Detection EventBridge schema. It includes an anomalyId, an impact block with spend values, and a rootCauses array identifying the service and region.

Setting linkedAccount to null tells the Lambda to use the local account (where your spike instance lives) instead of assuming a cross-account role.

• Send the synthetic event to EventBridge by running this command:

aws events put-events --entries file://test-event.json

You should see a response with FailedEntryCount of 0, confirming the event was accepted.

• Wait 10-15 seconds for the Lambda to execute, then check the CloudWatch Logs by running this command:

aws logs tail /aws/lambda/finops-command-center-remediator --since 5m

You should see log output showing the anomaly ID, the number of instances found, and a STOP_NON_PROD_EC2 action confirming the Lambda stopped your spike instance.

• Confirm the spike instance was stopped by running this command:

aws ec2 describe-instances --filters "Name=tag:CostCenter,Values=SPIKE-TEST" --query "Reservations[].Instances[].{ID:InstanceId,State:State.Name}" --output table

The instance state should now show stopped or stopping. Your automated remediation loop is working.

• Verify the remediation was logged to DynamoDB by running this command:

aws dynamodb scan --table-name finops-command-center-remediation-log --output json

You should see an item with anomaly_id of test-spike-001, action of STOP_NON_PROD_EC2, and the instance ID in instances_stopped.

Lambda did not execute or instance still running?

If the CloudWatch log group does not exist, the Lambda has never been invoked. Confirm the EventBridge rule finops-command-center-anomaly-detected is in the ENABLED state and the Lambda permission resource eventbridge_invoke was applied successfully.

If the Lambda executed but did not stop the instance, check that the instance is tagged Environment=Development (case-sensitive). The Lambda filters on the exact values Development, Test, and Staging.

Why didn't my Lambda stop the spike instance?

Verify the Grafana dashboard reflects the event

The final proof is that your Grafana dashboard shows the remediation event as a visible annotation. The dashboard queries the DynamoDB remediation log table that the Lambda just wrote to.

• Open your browser and navigate to http://localhost:3000.
• Log in with the default credentials (admin/admin) if you have not already changed them.
• Open your executive dashboard from the **Dashboards** menu.
• Verify that an annotation marker appears on the daily spend time-series panel. This annotation is the remediation event your Lambda logged to DynamoDB.
• Check the chargeback panel. The SPIKE-TEST cost center should appear in the breakdown.
• Confirm all KPI panels render without errors. Each panel should show data or placeholder values.

Why annotations matter for the executive presentation

Annotations show leadership exactly when anomalies fired and how fast you remediated them. This is your Remediation MTTR metric in action. The timestamp in DynamoDB minus the event timestamp gives you the time-to-resolution.

• Take screenshots of the following for your executive pitch deck:

• The daily spend panel with the remediation annotation visible.
• The chargeback breakdown showing the SPIKE-TEST cost center.
• The full KPI dashboard view with all panels rendering correctly.

Panels showing 'No data' in Grafana?

If the annotation panel shows no data, confirm the Athena datasource can reach the finops-command-center-remediation-log DynamoDB table. The annotation query must target the correct table name.

If the CUR-based panels (chargeback, spend trends) show no data, this is expected. CUR takes up to 24 hours for first delivery after creation. The annotation from DynamoDB should still appear independently.

Why are my Grafana panels empty?

You have just proved the entire detection-to-remediation loop works in under 15 minutes. Next up, you will assemble these results into an executive pitch deck that communicates the value to Fortune 10 leadership.

Assemble the Executive Pitch Deck

Your FinOps command center is fully operational. The spike test proved every layer works: anomaly detection triggers Lambda, Lambda stops non-prod instances, DynamoDB logs the action, and Grafana shows the annotation.

Now you need to translate this technical achievement into language a CFO understands. In this step, you will assemble the executive pitch deck that ties your infrastructure directly to business outcomes, risk mitigation, and a 12-month optimization roadmap.

In this step, get ready to:

• Assemble the 10-section pitch deck structure from your Grafana dashboards and pre-artifacts.
• Write the Technical Debt Reduction scorecard and Risk & Mitigation Matrix.
• Write the 12-month cost optimization roadmap with Well-Architected Cost Pillar alignment.

Create the pitch deck with Problem Statement, Architecture, and KPIs

A principal-level presentation connects technical decisions to business impact. Each section in this deck maps directly to something you already built or documented.

• Create a new file called executive-pitch-deck.md inside your pre-artifacts/ folder by pasting the following content:

# Executive Pitch Deck: FinOps Command Center
## Apex Global - Cost Intelligence & Governance Platform

---

## 1. Problem Statement

- **38% quarter-over-quarter spend growth** with zero corresponding revenue increase
- No per-customer chargeback capability (60%+ of spend is unallocable)
- Zero automated remediation (anomalies discovered at next billing cycle)
- No tag enforcement (resources created without cost allocation metadata)
- Manual spreadsheet reviews cannot scale with multi-account AWS Organization

**CFO Mandate:** Deliver end-to-end cost intelligence with per-customer chargeback,
automated anomaly remediation, and spend guardrails by month-end.

---

## 2. Solution Architecture

```mermaid
graph TD
    subgraph "Data Flow"
        CUR[CUR 2.0 Export] --> S3[S3 Bucket]
        S3 --> Glue[Glue Catalog]
        Glue --> Athena[Amazon Athena]
        Athena --> Grafana[Grafana OSS]
    end

    subgraph "Control Flow"
        CAD[Cost Anomaly Detection] --> EB[EventBridge]
        EB --> Lambda[Lambda Remediator]
        Lambda --> EC2Stop[EC2 Stop Non-Prod]
        Budget[Budget Actions] --> IAMDeny[IAM Deny Policy]
    end

    subgraph "Governance Overlay"
        TagPolicy[Tag Policies] --> Enforce[CostCenter/Environment/Owner]
    end
```

References: ADR-001 (Grafana over QuickSight), ADR-002 (CUR 2.0 + Athena over
Cost Explorer API), ADR-003 (Tag Policies over manual tagging).

---

## 3. Live Dashboard Demo

- Grafana OSS running at http://localhost:3000
- Athena datasource querying CUR 2.0 data via Glue Catalog
- Template variables for dynamic CostCenter, Account, and Time Range switching
- Alerting rules: daily spend exceeds 2x 7-day average
- Annotations: Cost Anomaly Detection events from DynamoDB remediation log

**Key panels:** Total monthly spend gauge, Spend by CostCenter stacked bar,
Spend by account table, Daily spend time-series with anomaly band, Top 10
services pie chart.

---

## 4. KPI Scorecard

| KPI | Current | Target (90 days) |
|-----|---------|-------------------|
| FinOps Maturity Score | Crawl | Walk |
| Tag Coverage Rate (% spend with CostCenter) | ~40% | 95% |
| Waste Score (idle EC2 + unattached EBS + unused EIPs) | Unknown | <5% of spend |
| Remediation MTTR | Days (manual) | <15 minutes (automated) |
| Savings Plans Coverage % | 0% | 70% |
| Savings Plans Net Savings vs On-Demand | $0 | Projected from coverage |
| Non-allocable Spend % | 60%+ | <5% |

---

## 5. Savings Plans Coverage and Optimization Path

- **Current state:** 0% Savings Plans coverage (all On-Demand)
- **Recommendation:** Compute Savings Plans (covers EC2, Lambda, Fargate across regions)
- **Target:** 70% coverage within 6 months
- **Break-even timeline:** Compute SP 1-year No Upfront breaks even at 7% utilization
- **Monitoring:** Grafana panel tracks utilization % with alert at <80%
- **SAP-C02 distinction:** EC2 Instance SP covers one family/one region.
  Compute SP covers any family, Lambda, and Fargate across all regions.

---

## 6. 12-Month Projections (Well-Architected Cost Pillar)

[Detailed in Section 6 below]

---

## 7. Technical Debt Reduction Scorecard

[Detailed in Section 7 below]

---

## 8. Risk & Mitigation Matrix

[Detailed in Section 8 below]

---

## 9. SAP-C02 Domain Mapping

| Domain | Practice | Implementation |
|--------|----------|----------------|
| D1.5 | Cost Visibility | CUR 2.0 + Athena + Grafana chargeback dashboards |
| D2.6 | Cost-Optimized Compute | Compute Optimizer export + rightsizing panels |
| D3.5 | Cost Optimization Opportunities | Anomaly Detection + automated remediation |
| D1.1 | Organizational Complexity | Tag Policies + cross-account assume-role |
| D4.1 | Automation | EventBridge + Lambda + Budget Actions |

---

## 10. FinOps Operating Cadence

### Weekly Review (Every Monday)
- **Attendees:** FinOps lead, engineering leads, finance partner
- **Agenda:** Spend vs budget, top anomalies, tag coverage %, rightsizing backlog
- **Output:** Action items assigned with owners and deadlines

### Monthly Optimization Cycle
- Savings Plans coverage review and commitment purchase decisions
- Waste elimination sprint (idle resources identified by Compute Optimizer)
- Tag compliance audit and remediation

### Quarterly Commitment Renewal
- Savings Plans expiry review and renewal/adjustment
- FinOps maturity assessment (Crawl/Walk/Run progression)
- Budget reforecasting based on business growth projections

### Escalation Path
Anomaly detected -> Slack alert -> 15-min auto-remediation (non-prod) ->
Manual approval (prod) -> Post-incident review

What does this deck structure accomplish?

The 10 sections follow a deliberate narrative arc. Sections 1-3 establish the problem and your solution. Sections 4-6 prove quantifiable value. Sections 7-8 address executive risk concerns. Sections 9-10 demonstrate ongoing governance.

This structure maps directly to how Fortune 10 CFOs evaluate platform investments: problem, solution, measurable impact, risk, and operational sustainability.

• Save pre-artifacts/executive-pitch-deck.md.
• Open the file in your editor's preview mode. Confirm you can see all 10 section headings with content in sections 1-5, 9, and 10.

Mermaid diagram not rendering?

The Mermaid diagram in Section 2 requires a Markdown renderer that supports Mermaid (like GitHub, or Cursor's built-in preview). If you see raw text instead of a diagram, the content is still correct for presentation tools that support Mermaid.

Write the Technical Debt Reduction scorecard and Risk Matrix

Executives evaluate platform investments through the lens of risk reduction. The scorecard quantifies how each component reduces operational risk, while the matrix addresses the CFO's likely objections.

• In executive-pitch-deck.md, find the placeholder text [Detailed in Section 7 below] and replace it with the following content:

### Technical Debt Reduction Scorecard

| Capability | Before (Manual) | After (Automated) | Improvement |
|-----------|----------------|-------------------|-------------|
| Detection Latency | Next billing cycle | <24 hours (Cost Anomaly Detection) | ~30x faster |
| Remediation MTTR | Days (ticket-based) | <15 minutes (EventBridge + Lambda) | ~100x faster |
| Tag Compliance | 0% enforced | 100% at creation time (Tag Policies) | Proactive vs reactive |
| Chargeback Delivery | Never (manual spreadsheets) | Daily automated refresh (CUR 2.0 + Grafana) | Continuous |
| Guardrail Coverage | None | Automated non-prod spend caps (Budget Actions) | Full coverage |

• Now find the placeholder text [Detailed in Section 8 below] and replace it with the Risk & Mitigation Matrix:

### Risk & Mitigation Matrix

| # | Risk | Impact | Likelihood | Mitigation |
|---|------|--------|------------|------------|
| 1 | False-positive stops production instance | Critical | Low | approval_model = MANUAL for prod accounts. Lambda only auto-remediates resources tagged Environment=Development/Test/Staging. Production requires human approval via SNS. |
| 2 | CUR 24-hour delay misses intra-day spike | High | Medium | Budget Actions provide real-time threshold guardrail as second layer. 80% forecasted threshold triggers IAM deny before overspend occurs. |
| 3 | Tag Policy blocks legitimate deployment | Medium | Medium | Deploy in Warn mode for first 30 days before enabling Fail mode. Engineering teams notified via weekly tag compliance report. Escalation path defined in operating cadence. |

Why separate detection from remediation risk?

Risk 1 addresses the remediation layer (Lambda). Risk 2 addresses the detection layer (CUR delay). Risk 3 addresses the governance layer (Tag Policies). Each layer has independent failure modes, and each mitigation is specific to that layer.

This separation demonstrates to leadership that you have thought through failure at every level, not just the happy path.

• Save executive-pitch-deck.md.
• Confirm that sections 7 and 8 now show formatted tables instead of placeholder text.

Tables not rendering correctly?

Markdown tables require the header separator row (the row with dashes and pipes). Check that each table has the |---|---|---| row between the header and body rows.

Write the 12-month cost optimization roadmap

The 12-month roadmap maps each phase to specific Well-Architected Cost Pillar practices. This proves to leadership that your plan follows AWS's own framework, not ad-hoc decisions.

• In executive-pitch-deck.md, find the placeholder text [Detailed in Section 6 below] and replace it with the following roadmap:

### Month 1-3: Crawl Phase (COST01-COST03)

**Well-Architected Practices:** COST01 (Governance), COST02 (Controls), COST03 (Monitoring)

- Governance deployed: Tag Policies enforcing CostCenter/Environment/Owner
- Cost allocation tags activated from management account
- Anomaly detection active with automated non-prod remediation
- Baseline KPIs established in Grafana executive dashboard
- Budget Actions guardrails deployed (80% IAM deny, 100% SSM stop)
- Tag compliance tracked weekly (target: 95% within 90 days)

### Month 4-6: Walk Phase (COST06-COST09)

**Well-Architected Practices:** COST06 (Resource Type Selection), COST09 (Demand Management)

- Rightsizing recommendations from Compute Optimizer executed
- Savings Plans purchased based on coverage analysis (target: 70% coverage)
- Unit economics tracked per customer via chargeback views
- Resource type optimization: Graviton migration candidates identified
- Idle resource cleanup sprint (unattached EBS, unused Elastic IPs)
- Savings Plans utilization monitored with <80% alert threshold

### Month 7-12: Run Phase (COST10-COST11)

**Well-Architected Practices:** COST10 (Evaluate Cost Over Time), COST11 (Sustainability)

- Predictive anomaly thresholds tuned (reduce false positives to <20%)
- Carbon footprint optimization via region-aware scheduling
- Multi-account federated dashboards for business unit self-service
- FinOps team trained and operating independently
- Quarterly commitment renewal process automated
- Cost reduction waterfall: rightsizing + anomaly prevention + tag coverage + SP optimization

### Projected Savings Summary

| Phase | Primary Driver | Estimated Impact |
|-------|---------------|------------------|
| Crawl | Anomaly prevention + tag visibility | Baseline established |
| Walk | Rightsizing + Savings Plans | 20-35% reduction vs On-Demand |
| Run | Continuous optimization + automation | Sustained <10% QoQ growth |

Why map phases to Well-Architected practices?

The Well-Architected Cost Pillar (COST01 through COST11) is AWS's official framework for cost optimization maturity. Mapping your roadmap to specific COST practices demonstrates that your plan is grounded in industry-standard methodology.

In SAP-C02 exam scenarios, this is how principal-level architects frame recommendations. It also gives leadership confidence that the plan is repeatable and auditable.

• Save executive-pitch-deck.md.
• Scroll through the full document. Confirm all 10 sections now have content with no remaining placeholder text.

Still seeing placeholder text?

Search the file for the bracket character [ to find any remaining placeholders. Each of the three placeholders should have been replaced with the content from substeps above.

✔️ Awesome, I've got everything!

Your executive pitch deck is complete. Make sure you have saved pre-artifacts/executive-pitch-deck.md with all 10 sections populated.

ⓧ I'd like to double check the full code

Here is the complete executive pitch deck file for comparison.

# Executive Pitch Deck: FinOps Command Center
## Apex Global - Cost Intelligence & Governance Platform

---

## 1. Problem Statement

- **38% quarter-over-quarter spend growth** with zero corresponding revenue increase
- No per-customer chargeback capability (60%+ of spend is unallocable)
- Zero automated remediation (anomalies discovered at next billing cycle)
- No tag enforcement (resources created without cost allocation metadata)
- Manual spreadsheet reviews cannot scale with multi-account AWS Organization

**CFO Mandate:** Deliver end-to-end cost intelligence with per-customer chargeback,
automated anomaly remediation, and spend guardrails by month-end.

---

## 2. Solution Architecture

```mermaid
graph TD
    subgraph "Data Flow"
        CUR[CUR 2.0 Export] --> S3[S3 Bucket]
        S3 --> Glue[Glue Catalog]
        Glue --> Athena[Amazon Athena]
        Athena --> Grafana[Grafana OSS]
    end

    subgraph "Control Flow"
        CAD[Cost Anomaly Detection] --> EB[EventBridge]
        EB --> Lambda[Lambda Remediator]
        Lambda --> EC2Stop[EC2 Stop Non-Prod]
        Budget[Budget Actions] --> IAMDeny[IAM Deny Policy]
    end

    subgraph "Governance Overlay"
        TagPolicy[Tag Policies] --> Enforce[CostCenter/Environment/Owner]
    end
```

References: ADR-001 (Grafana over QuickSight), ADR-002 (CUR 2.0 + Athena over
Cost Explorer API), ADR-003 (Tag Policies over manual tagging).

---

## 3. Live Dashboard Demo

- Grafana OSS running at http://localhost:3000
- Athena datasource querying CUR 2.0 data via Glue Catalog
- Template variables for dynamic CostCenter, Account, and Time Range switching
- Alerting rules: daily spend exceeds 2x 7-day average
- Annotations: Cost Anomaly Detection events from DynamoDB remediation log

**Key panels:** Total monthly spend gauge, Spend by CostCenter stacked bar,
Spend by account table, Daily spend time-series with anomaly band, Top 10
services pie chart.

---

## 4. KPI Scorecard

| KPI | Current | Target (90 days) |
|-----|---------|-------------------|
| FinOps Maturity Score | Crawl | Walk |
| Tag Coverage Rate (% spend with CostCenter) | ~40% | 95% |
| Waste Score (idle EC2 + unattached EBS + unused EIPs) | Unknown | <5% of spend |
| Remediation MTTR | Days (manual) | <15 minutes (automated) |
| Savings Plans Coverage % | 0% | 70% |
| Savings Plans Net Savings vs On-Demand | $0 | Projected from coverage |
| Non-allocable Spend % | 60%+ | <5% |

---

## 5. Savings Plans Coverage and Optimization Path

- **Current state:** 0% Savings Plans coverage (all On-Demand)
- **Recommendation:** Compute Savings Plans (covers EC2, Lambda, Fargate across regions)
- **Target:** 70% coverage within 6 months
- **Break-even timeline:** Compute SP 1-year No Upfront breaks even at 7% utilization
- **Monitoring:** Grafana panel tracks utilization % with alert at <80%
- **SAP-C02 distinction:** EC2 Instance SP covers one family/one region.
  Compute SP covers any family, Lambda, and Fargate across all regions.

---

## 6. 12-Month Projections (Well-Architected Cost Pillar)

### Month 1-3: Crawl Phase (COST01-COST03)

**Well-Architected Practices:** COST01 (Governance), COST02 (Controls), COST03 (Monitoring)

- Governance deployed: Tag Policies enforcing CostCenter/Environment/Owner
- Cost allocation tags activated from management account
- Anomaly detection active with automated non-prod remediation
- Baseline KPIs established in Grafana executive dashboard
- Budget Actions guardrails deployed (80% IAM deny, 100% SSM stop)
- Tag compliance tracked weekly (target: 95% within 90 days)

### Month 4-6: Walk Phase (COST06-COST09)

**Well-Architected Practices:** COST06 (Resource Type Selection), COST09 (Demand Management)

- Rightsizing recommendations from Compute Optimizer executed
- Savings Plans purchased based on coverage analysis (target: 70% coverage)
- Unit economics tracked per customer via chargeback views
- Resource type optimization: Graviton migration candidates identified
- Idle resource cleanup sprint (unattached EBS, unused Elastic IPs)
- Savings Plans utilization monitored with <80% alert threshold

### Month 7-12: Run Phase (COST10-COST11)

**Well-Architected Practices:** COST10 (Evaluate Cost Over Time), COST11 (Sustainability)

- Predictive anomaly thresholds tuned (reduce false positives to <20%)
- Carbon footprint optimization via region-aware scheduling
- Multi-account federated dashboards for business unit self-service
- FinOps team trained and operating independently
- Quarterly commitment renewal process automated
- Cost reduction waterfall: rightsizing + anomaly prevention + tag coverage + SP optimization

### Projected Savings Summary

| Phase | Primary Driver | Estimated Impact |
|-------|---------------|------------------|
| Crawl | Anomaly prevention + tag visibility | Baseline established |
| Walk | Rightsizing + Savings Plans | 20-35% reduction vs On-Demand |
| Run | Continuous optimization + automation | Sustained <10% QoQ growth |

---

## 7. Technical Debt Reduction Scorecard

### Technical Debt Reduction Scorecard

| Capability | Before (Manual) | After (Automated) | Improvement |
|-----------|----------------|-------------------|-------------|
| Detection Latency | Next billing cycle | <24 hours (Cost Anomaly Detection) | ~30x faster |
| Remediation MTTR | Days (ticket-based) | <15 minutes (EventBridge + Lambda) | ~100x faster |
| Tag Compliance | 0% enforced | 100% at creation time (Tag Policies) | Proactive vs reactive |
| Chargeback Delivery | Never (manual spreadsheets) | Daily automated refresh (CUR 2.0 + Grafana) | Continuous |
| Guardrail Coverage | None | Automated non-prod spend caps (Budget Actions) | Full coverage |

---

## 8. Risk & Mitigation Matrix

### Risk & Mitigation Matrix

| # | Risk | Impact | Likelihood | Mitigation |
|---|------|--------|------------|------------|
| 1 | False-positive stops production instance | Critical | Low | approval_model = MANUAL for prod accounts. Lambda only auto-remediates resources tagged Environment=Development/Test/Staging. Production requires human approval via SNS. |
| 2 | CUR 24-hour delay misses intra-day spike | High | Medium | Budget Actions provide real-time threshold guardrail as second layer. 80% forecasted threshold triggers IAM deny before overspend occurs. |
| 3 | Tag Policy blocks legitimate deployment | Medium | Medium | Deploy in Warn mode for first 30 days before enabling Fail mode. Engineering teams notified via weekly tag compliance report. Escalation path defined in operating cadence. |

---

## 9. SAP-C02 Domain Mapping

| Domain | Practice | Implementation |
|--------|----------|----------------|
| D1.5 | Cost Visibility | CUR 2.0 + Athena + Grafana chargeback dashboards |
| D2.6 | Cost-Optimized Compute | Compute Optimizer export + rightsizing panels |
| D3.5 | Cost Optimization Opportunities | Anomaly Detection + automated remediation |
| D1.1 | Organizational Complexity | Tag Policies + cross-account assume-role |
| D4.1 | Automation | EventBridge + Lambda + Budget Actions |

---

## 10. FinOps Operating Cadence

### Weekly Review (Every Monday)
- **Attendees:** FinOps lead, engineering leads, finance partner
- **Agenda:** Spend vs budget, top anomalies, tag coverage %, rightsizing backlog
- **Output:** Action items assigned with owners and deadlines

### Monthly Optimization Cycle
- Savings Plans coverage review and commitment purchase decisions
- Waste elimination sprint (idle resources identified by Compute Optimizer)
- Tag compliance audit and remediation

### Quarterly Commitment Renewal
- Savings Plans expiry review and renewal/adjustment
- FinOps maturity assessment (Crawl/Walk/Run progression)
- Budget reforecasting based on business growth projections

### Escalation Path
Anomaly detected -> Slack alert -> 15-min auto-remediation (non-prod) ->
Manual approval (prod) -> Post-incident review

Secret mission

Add Carbon Footprint Tracking and Automated Weekly FinOps Reporting

You have built the cost intelligence loop. Now extend it with sustainability metrics and automated operational cadence. These two capabilities are what separate a principal architect from a senior engineer in real enterprise engagements.

Clean Up Your Resources

Clean Up Your Resources

This project runs almost entirely within AWS Free Tier, so ongoing costs are negligible. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Cost warning

If you leave everything deployed, CUR 2.0 delivers data to S3 daily and the Glue Crawler runs once per day (~$0.01/run). Over time, S3 storage accumulates. At lab scale this is pennies per month, but if left indefinitely it adds up.

If you are not actively using the dashboards, choose Pause or Delete below.

Resources you used:

• CUR 2.0 export and S3 buckets (CUR data bucket, Athena results bucket).
• Glue Catalog database and Crawler.
• Athena workgroup and named queries.
• Cost Anomaly Detection monitors and subscription.
• Lambda function, DynamoDB table, and EventBridge rule.
• Budget and Budget Actions.
• Organizations Tag Policy.
• SNS topic and IAM roles.
• t3.micro EC2 spike instance (should already be stopped).
• Grafana OSS Docker container (local).
• Local project files (finops-command-center/ directory).

Keep everything running

No action needed. Choose this if you are still actively building or plan to extend the dashboards soon.

• CUR will continue delivering daily to S3. Cost: effectively $0/month at lab scale (S3 storage only).
• Cost Anomaly Detection, Budgets, Tag Policy, EventBridge, and SNS all cost nothing when idle.
• Stop the Grafana Docker container when you are not using it to free up local memory. Run:

docker stop grafana

You can restart it later with:

docker start grafana

Pause - I'll come back to this later

Shut down running processes to free up local resources while keeping your Terraform state and AWS resources intact. The AWS-side resources cost nothing when idle.

• Stop the Grafana container by running:

docker stop grafana

• Confirm the t3.micro spike instance is stopped. If it is still running, stop it from the EC2 console or run:

aws ec2 stop-instances --instance-ids $(aws ec2 describe-instances --filters "Name=tag:CostCenter,Values=SPIKE-TEST" "Name=instance-state-name,Values=running" --query "Reservations[].Instances[].InstanceId" --output text)

• Leave your Terraform state files and local project directory intact. When you return, run docker start grafana and you are back in business.

Delete - I don't want to use this again

Remove all project resources and start fresh. This is a full teardown.

• Navigate to your project directory by running:

cd ~/Desktop/finops-command-center

• Destroy all Terraform-managed AWS resources by running:

terraform destroy

• Type yes when prompted to confirm the destruction.
• If Terraform fails to delete an S3 bucket because it is not empty (CUR delivers data daily), empty the bucket first by running:

aws s3 rm s3://finops-command-center-cur-$(aws sts get-caller-identity --query Account --output text) --recursive
aws s3 rm s3://finops-command-center-athena-results-$(aws sts get-caller-identity --query Account --output text) --recursive

• Then run terraform destroy again to remove the now-empty buckets.
• Stop and remove the Grafana Docker container by running:

docker stop grafana && docker rm grafana

• Delete the local project files. On macOS/Linux, run:

rm -rf ~/Desktop/finops-command-center

• On Windows, run:

Remove-Item -Recurse -Force ~\Desktop\finops-command-center

• Deactivate cost allocation tags if desired. In the AWS console, navigate to Billing > Cost allocation tags and deactivate the user-defined tags (CostCenter, Environment, Owner). This is optional because inactive tags cost nothing.

That's a Wrap!

That's a Wrap!

Nice work. You just delivered what most platform teams take a quarter to build.

In 90 minutes, you stood up an enterprise-grade FinOps command center with automated cost governance and presented it to Fortune 10 leadership.

You've learned how to:

• Deploy a complete CUR 2.0 data pipeline through Glue and Athena into Grafana executive dashboards with per-customer chargeback views, Savings Plans coverage analysis, KPIs, alerting, and 12-month projections.
• Implement automated anomaly remediation using EventBridge and Lambda with cross-account assume-role that stops runaway non-prod spend in under 15 minutes, with manual approval for production.
• Produce principal-level artifacts including ADRs, architecture diagrams, a FinOps operating cadence, Budget Actions guardrails, Organizations Tag Policies, and a CFO-ready executive presentation with Well-Architected Cost Pillar alignment.
• Secret Mission: Added a carbon footprint overlay panel and an automated weekly FinOps cadence email report via SNS with EventBridge Scheduler.

Ready to quiz yourself?