Multi-Cloud Claims Analytics with Terraform
Build HIPAA-compliant AWS and GCP infrastructure to query claims data in place.
Introduction
30 Second Summary
Every time you visit a doctor, your health insurance company processes a claim containing sensitive personal details. Keeping that data secure while still letting analysts extract insights is one of the hardest infrastructure challenges in healthcare.
In this project, you will build a multi-cloud infrastructure using Terraform that stores encrypted claims data in AWS and enables GCP BigQuery to query it in place, without ever moving protected health information across cloud boundaries.
What You'll Build
A production-grade Terraform repository that provisions HIPAA-compliant healthcare claims infrastructure spanning AWS and GCP, connected by BigQuery Omni's cross-cloud analytics.
By the end of this project, you'll have:
- A modular Terraform codebase with isolated state per cloud provider, KMS-encrypted S3 storage, and CloudTrail audit logging.
- A working cross-cloud connection that queries AWS S3 data directly from GCP BigQuery without transferring PHI.
- A compliance-validated infrastructure with Checkov policy-as-code scans, Architecture Decision Records, and a HIPAA control mapping matrix.
- Secret Mission: Custom governance policies and FinOps cost estimation that demonstrate operational maturity.
Are there any prerequisites?
This project assumes you have an AWS account with admin access, a GCP account with project owner access, Terraform installed, and both the AWS CLI and gcloud CLI configured. You will also need Python 3 installed for Checkov.
Before We Start
A multi-cloud infrastructure project needs a solid foundation before any resources are provisioned. Without a clear directory structure and automation tooling, managing Terraform across two cloud providers becomes chaotic fast.
In this step, you will set up the repository skeleton, install a compliance scanner, and create a Makefile that automates every Terraform operation for both AWS and GCP.
In this step, get ready to:
- Initialize a Git repository with an enterprise-grade directory structure.
- Install Checkov for policy-as-code compliance scanning.
- Create a Makefile to automate Terraform operations across both cloud providers.
Create the project repository and directory structure
Enterprise multi-cloud repositories separate infrastructure code by provider. This prevents a failed deployment in one cloud from corrupting state in another.
- Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
- Type Terminal and press Enter to open it.
- Move to your Desktop by running this command:
cd ~/Desktop
- Create your project folder and move into it by running these commands:
mkdir multi-cloud-claims-analytics
cd multi-cloud-claims-analytics
- Initialize a Git repository and create the directory structure by running these commands:
git init
mkdir -p aws gcp modules policies docs/adr
What do these directories do?
- aws/ holds all AWS-specific Terraform configuration (S3, KMS, CloudTrail, IAM).
- gcp/ holds all GCP-specific Terraform configuration (BigQuery, connections).
- modules/ is reserved for reusable Terraform modules you may extract later.
- policies/ stores custom Checkov policy checks (used in the Secret Mission).
- docs/adr/ holds Architecture Decision Records explaining key design choices.
- Confirm the directories were created by running:
ls
You should see aws, docs, gcp, modules, and policies listed in the output.
Not seeing all five directories?
Make sure you ran the mkdir -p command from inside the multi-cloud-claims-analytics folder. Run pwd to check your current location.
Help me figure out what went wrong with my directory structure.
Install Checkov
Checkov is a policy-as-code tool that scans Terraform files for security misconfigurations before you deploy. You will use it to validate HIPAA Technical Safeguards throughout this project.
- Install Checkov by running this command:
pip3 install checkov
What does this command do?
pip3 install checkov downloads and installs the Checkov package from PyPI. It includes built-in policies for AWS, GCP, Azure, and Kubernetes that check for common security issues.
- Verify the installation succeeded by running:
checkov --version
You should see a version number printed in the terminal, confirming Checkov is ready to use.
Seeing command not found?
If your terminal cannot find checkov, the Python scripts directory may not be on your PATH. Try running python3 -m checkov --version instead.
Help me fix my PATH so checkov works.
Create the Makefile
A Makefile gives you short, memorable commands for every Terraform operation instead of typing long commands with directory paths. It also documents the correct deployment ordering for your team.
- Create a file called Makefile in the root of your multi-cloud-claims-analytics folder.
- Open Makefile in your text editor and paste the following code:
.PHONY: init-aws init-gcp plan-aws plan-gcp apply-aws apply-gcp destroy-aws destroy-gcp validate lint
# AWS targets
init-aws:
cd aws && terraform init
plan-aws:
cd aws && terraform plan -out=tfplan
apply-aws:
cd aws && terraform apply tfplan
destroy-aws:
cd aws && terraform destroy
# GCP targets
init-gcp:
cd gcp && terraform init
plan-gcp:
cd gcp && terraform plan -out=tfplan
apply-gcp:
cd gcp && terraform apply tfplan
destroy-gcp:
cd gcp && terraform destroy
# Validation targets
validate:
cd aws && terraform validate
cd gcp && terraform validate
lint:
checkov -d aws/ --framework terraform --output cli
checkov -d gcp/ --framework terraform --output cli
# Full deployment (order matters)
deploy: apply-aws apply-gcp
@echo "Run 'make apply-aws' again after updating bigquery_omni_identity variable with GCP output"
# Full teardown (order matters: GCP first, then AWS)
teardown: destroy-gcp destroy-aws
What does this Makefile do?
The Makefile organizes Terraform operations into four groups:
- AWS targets (init-aws, plan-aws, apply-aws, destroy-aws) run Terraform commands scoped to the aws/ directory.
- GCP targets follow the same pattern for the gcp/ directory.
- Validation targets run terraform validate and Checkov scans across both providers.
- Deployment targets enforce the correct ordering. AWS deploys first, then GCP. Teardown reverses the order (GCP first, then AWS) because of cross-cloud dependencies.
- Save Makefile.
- Confirm the file exists by running:
ls
You should see Makefile listed alongside your aws, docs, gcp, modules, and policies directories.
Makefile not appearing?
Make sure the file is named exactly Makefile with a capital M and no file extension. Some editors may add a .txt extension by default.
Help me troubleshoot my Makefile.
✔️ Awesome, I've got everything!
Great. Double check you have saved your Makefile and that all six directories exist.
ⓧ I'd like to double check the full code
Here is the complete Makefile for comparison:
.PHONY: init-aws init-gcp plan-aws plan-gcp apply-aws apply-gcp destroy-aws destroy-gcp validate lint
# AWS targets
init-aws:
cd aws && terraform init
plan-aws:
cd aws && terraform plan -out=tfplan
apply-aws:
cd aws && terraform apply tfplan
destroy-aws:
cd aws && terraform destroy
# GCP targets
init-gcp:
cd gcp && terraform init
plan-gcp:
cd gcp && terraform plan -out=tfplan
apply-gcp:
cd gcp && terraform apply tfplan
destroy-gcp:
cd gcp && terraform destroy
# Validation targets
validate:
cd aws && terraform validate
cd gcp && terraform validate
lint:
checkov -d aws/ --framework terraform --output cli
checkov -d gcp/ --framework terraform --output cli
# Full deployment (order matters)
deploy: apply-aws apply-gcp
@echo "Run 'make apply-aws' again after updating bigquery_omni_identity variable with GCP output"
# Full teardown (order matters: GCP first, then AWS)
teardown: destroy-gcp destroy-aws
Your directory structure should look like this:
multi-cloud-claims-analytics/
├── Makefile
├── aws/
├── docs/
│ └── adr/
├── gcp/
├── modules/
└── policies/
Your project skeleton is ready. Next up, you will build the AWS foundation with a KMS encryption key and a HIPAA-compliant S3 bucket for claims data.
Set Up Project Structure and Tooling
Your repository structure and Makefile are ready. Now you need to configure Terraform to actually talk to both AWS and GCP.
Each cloud provider needs its own provider configuration and a remote state backend. Keeping state files separate per cloud is a deliberate enterprise pattern that limits blast radius. A failed AWS apply cannot corrupt GCP state, and vice versa.
In this step, get ready to:
- Create the AWS Terraform provider configuration with HIPAA-required default tags.
- Create the GCP Terraform provider configuration for BigQuery Omni analytics.
- Set up isolated remote state backends and initialize both Terraform workspaces.
Create the AWS provider configuration
The AWS provider block tells Terraform which region to use and which default tags to apply to every resource. For HIPAA workloads, tagging every resource with its data classification and compliance framework is essential for auditing.
- Create a new file called aws/providers.tf and paste the following configuration:
terraform {
required_version = ">= 1.5.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 6.49"
}
}
backend "s3" {
bucket = "claims-analytics-tf-state"
key = "aws/terraform.tfstate"
region = "us-east-1"
dynamodb_table = "claims-analytics-tf-lock"
encrypt = true
}
}
provider "aws" {
region = var.aws_region
default_tags {
tags = {
Project = "claims-analytics"
Environment = var.environment
ManagedBy = "terraform"
DataClassification = "phi"
Compliance = "hipaa"
}
}
}
What does this configuration do?
The terraform block pins the provider version to ~> 6.49 so future runs produce consistent results. The backend "s3" block stores your state file in a remote S3 bucket with DynamoDB-based locking to prevent concurrent modifications.
The default_tags block is the most important HIPAA compliance pattern here. Every AWS resource Terraform creates automatically gets tagged with DataClassification = "phi" and Compliance = "hipaa". Auditors can query these tags to find all PHI-bearing resources.
- Save aws/providers.tf.
- Now create aws/variables.tf and paste the following variable definitions:
variable "aws_region" {
description = "AWS region for claims data infrastructure"
type = string
default = "us-east-1"
}
variable "environment" {
description = "Deployment environment"
type = string
default = "production"
}
variable "project_name" {
description = "Project identifier used in resource naming"
type = string
default = "claims-analytics"
}
variable "bigquery_omni_identity" {
description = "Google-owned identity from BigQuery Omni connection (set after GCP deployment)"
type = string
default = ""
}
What does bigquery_omni_identity do?
This variable starts empty because it creates a chicken-and-egg situation. The GCP BigQuery Omni connection outputs a Google-owned identity. That identity must be placed into the AWS IAM trust policy so BigQuery can assume the role. You will fill this variable after deploying the GCP resources in a later step.
- Save aws/variables.tf.
Seeing a syntax error?
Make sure every opening brace { has a matching closing brace }. HCL is strict about block nesting. Check that default_tags is inside the provider "aws" block, not outside it.
Help me debug my aws/providers.tf file
Create the GCP provider configuration
The GCP side needs its own provider configuration pointing at BigQuery Omni's colocated region. GCP state lives in a separate GCS bucket to maintain the blast radius isolation.
- Create a new file called gcp/providers.tf and paste the following:
terraform {
required_version = ">= 1.5.0"
required_providers {
google = {
source = "hashicorp/google"
version = "~> 7.35"
}
}
backend "gcs" {
bucket = "claims-analytics-tf-state-gcp"
prefix = "gcp/terraform.tfstate"
}
}
provider "google" {
project = var.gcp_project_id
region = var.gcp_region
}
Why a separate state backend for GCP?
Isolated state per cloud means independent team workflows. The networking team managing AWS infrastructure and the analytics team managing GCP resources can operate in parallel without risking cross-cloud state corruption. It also creates clean audit boundaries: an AWS compliance audit does not need to inspect GCP state.
- Save gcp/providers.tf.
- Now create gcp/variables.tf and paste the following:
variable "gcp_project_id" {
description = "GCP project ID for analytics infrastructure"
type = string
}
variable "gcp_region" {
description = "GCP region for non-Omni resources"
type = string
default = "us-east4"
}
variable "project_name" {
description = "Project identifier used in resource naming"
type = string
default = "claims-analytics"
}
variable "aws_omni_role_arn" {
description = "ARN of the AWS IAM role for BigQuery Omni (from AWS deployment output)"
type = string
}
variable "claims_bucket_name" {
description = "Name of the AWS S3 claims data bucket (from AWS deployment output)"
type = string
}
Why do some variables have no default?
Variables like gcp_project_id, aws_omni_role_arn, and claims_bucket_name have no default because their values come from your specific environment or from the outputs of the AWS deployment. Terraform will prompt you for them at plan time, or you will set them in a terraform.tfvars file later.
- Save gcp/variables.tf.
✔️ Awesome, I've got everything!
Great. Double check you have saved all four files before moving on.
ⓧ I'd like to double check the full code
Here is the full content of each file for comparison.
terraform {
required_version = ">= 1.5.0"
required_providers {
aws = {
source = "hashicorp/aws"
version = "~> 6.49"
}
}
backend "s3" {
bucket = "claims-analytics-tf-state"
key = "aws/terraform.tfstate"
region = "us-east-1"
dynamodb_table = "claims-analytics-tf-lock"
encrypt = true
}
}
provider "aws" {
region = var.aws_region
default_tags {
tags = {
Project = "claims-analytics"
Environment = var.environment
ManagedBy = "terraform"
DataClassification = "phi"
Compliance = "hipaa"
}
}
}
variable "aws_region" {
description = "AWS region for claims data infrastructure"
type = string
default = "us-east-1"
}
variable "environment" {
description = "Deployment environment"
type = string
default = "production"
}
variable "project_name" {
description = "Project identifier used in resource naming"
type = string
default = "claims-analytics"
}
variable "bigquery_omni_identity" {
description = "Google-owned identity from BigQuery Omni connection (set after GCP deployment)"
type = string
default = ""
}
terraform {
required_version = ">= 1.5.0"
required_providers {
google = {
source = "hashicorp/google"
version = "~> 7.35"
}
}
backend "gcs" {
bucket = "claims-analytics-tf-state-gcp"
prefix = "gcp/terraform.tfstate"
}
}
provider "google" {
project = var.gcp_project_id
region = var.gcp_region
}
variable "gcp_project_id" {
description = "GCP project ID for analytics infrastructure"
type = string
}
variable "gcp_region" {
description = "GCP region for non-Omni resources"
type = string
default = "us-east4"
}
variable "project_name" {
description = "Project identifier used in resource naming"
type = string
default = "claims-analytics"
}
variable "aws_omni_role_arn" {
description = "ARN of the AWS IAM role for BigQuery Omni (from AWS deployment output)"
type = string
}
variable "claims_bucket_name" {
description = "Name of the AWS S3 claims data bucket (from AWS deployment output)"
type = string
}
Set up remote state backends and initialize Terraform
Before terraform init can succeed, the state backends must already exist. You will create the S3 bucket and DynamoDB lock table in AWS, plus the GCS bucket in GCP, using CLI commands.
- Create the S3 state bucket by running:
aws s3api create-bucket --bucket claims-analytics-tf-state --region us-east-1
What does this command do?
This creates an S3 bucket named claims-analytics-tf-state in us-east-1. For us-east-1 specifically, you do not need a LocationConstraint. This bucket will hold your AWS Terraform state file.
- Create the DynamoDB lock table by running:
aws dynamodb create-table \
--table-name claims-analytics-tf-lock \
--attribute-definitions AttributeName=LockID,AttributeType=S \
--key-schema AttributeName=LockID,KeyType=HASH \
--billing-mode PAY_PER_REQUEST \
--region us-east-1
What does the lock table do?
Terraform uses this DynamoDB table to acquire a lock before writing state. If two team members run terraform apply simultaneously, only one will acquire the lock. The other will wait, preventing state corruption. The LockID partition key is what Terraform expects by convention.
- Now create the GCS state bucket for GCP by running:
gcloud storage buckets create gs://claims-analytics-tf-state-gcp --location=us-east4
- You should see Creating gs://claims-analytics-tf-state-gcp/... confirming the bucket was created.
Bucket name already exists?
Both S3 and GCS bucket names are globally unique. If you see a conflict error, append your initials or a short random suffix to the bucket name. Then update the matching bucket field in your providers.tf backend blocks to match.
Help me fix my state bucket naming conflict
- Initialize the AWS Terraform workspace by running:
cd aws && terraform init
- You should see Terraform has been successfully initialized! confirming the S3 backend is connected.
- Initialize the GCP Terraform workspace by running:
cd ../gcp && terraform init
- You should see the same Terraform has been successfully initialized! message confirming the GCS backend is connected.
- Return to the project root by running:
cd ..
Terraform init failing with access denied?
Confirm your AWS CLI is configured with credentials that have S3 and DynamoDB permissions. Run aws sts get-caller-identity to check your active identity. For GCP, run gcloud auth list to confirm you are authenticated with a principal that has storage admin access.
Help me fix my terraform init access error
Your Terraform workspaces are initialized and talking to their remote backends. Next up, you will provision the KMS encryption key and the claims data S3 bucket that form the foundation of your HIPAA-compliant AWS layer.
Build the AWS KMS and Encrypted Claims Data Bucket
Your Terraform backend is ready and your providers are configured. Now it's time to provision the first real infrastructure: the encryption layer that protects claims data at rest.
HIPAA's Technical Safeguards require encryption of Protected Health Information at rest and in transit. In this step, you'll build a KMS key for encryption and an S3 bucket hardened with five separate compliance controls.
In this step, get ready to:
- Create a KMS encryption key with automatic annual rotation.
- Provision a HIPAA-compliant S3 bucket with SSE-KMS encryption, versioning, public access blocking, and SSL enforcement.
- Plan and apply the AWS infrastructure using the Makefile.
Create the KMS encryption key
AWS KMS provides managed encryption keys that automatically handle key material, rotation schedules, and access policies. For HIPAA compliance, you need a dedicated key with automatic rotation enabled.
- Create a new file called kms.tf inside your aws/ directory and add the following code:
resource "aws_kms_key" "claims_encryption" {
description = "KMS key for encrypting PHI claims data at rest"
deletion_window_in_days = 30
enable_key_rotation = true
rotation_period_in_days = 365
tags = {
Name = "${var.project_name}-claims-key"
}
}
resource "aws_kms_alias" "claims_encryption" {
name = "alias/${var.project_name}-claims"
target_key_id = aws_kms_key.claims_encryption.key_id
}
What does this code do?
This file defines two resources that work together:
- aws_kms_key creates the encryption key itself. deletion_window_in_days = 30 ensures the key cannot be accidentally deleted. It enters a waiting period for 30 days before permanent deletion.
- enable_key_rotation = true with rotation_period_in_days = 365 satisfies the HIPAA requirement for periodic key rotation. AWS automatically rotates the key material annually while keeping the same key ID.
- aws_kms_alias gives the key a human-readable name (alias/claims-analytics-claims) so you can reference it by name in the AWS console.
- Save aws/kms.tf.
Why a 30-day deletion window?
HIPAA requires that encryption keys used for PHI are retained for audit purposes. The 30-day window prevents accidental data loss. If someone deletes the key, you have 30 days to cancel the deletion before the encrypted data becomes permanently unreadable.
Provision the encrypted S3 claims data bucket
The S3 bucket needs five separate compliance controls layered on top of each other: KMS encryption at rest, versioning for integrity, a public access block, an SSL-only bucket policy, and a dynamic bucket name tied to your AWS account.
- Create a new file called s3.tf inside your aws/ directory and add the bucket resource with its data source:
resource "aws_s3_bucket" "claims_data" {
bucket = "${var.project_name}-claims-data-${data.aws_caller_identity.current.account_id}"
tags = {
Name = "${var.project_name}-claims-data"
DataClassification = "phi"
}
}
data "aws_caller_identity" "current" {}
What does this code do?
The aws_s3_bucket resource creates your claims data bucket. The bucket name includes your AWS account ID (pulled from the aws_caller_identity data source) to guarantee global uniqueness.
The DataClassification = "phi" tag marks this bucket as containing Protected Health Information. This tag is what your Checkov policies will validate later.
- Below the data "aws_caller_identity" "current" {} line, add the versioning configuration. Place it above the data source by finding the closing brace of aws_s3_bucket and adding this code after it:
resource "aws_s3_bucket_versioning" "claims_data" {
bucket = aws_s3_bucket.claims_data.id
versioning_configuration {
status = "Enabled"
}
}
Why enable versioning?
HIPAA §164.312(c)(1) requires integrity controls. Versioning means every change to a claims file creates a new version rather than overwriting the original. If a file is accidentally modified or deleted, you can recover any previous version.
- Add the server-side encryption configuration below the versioning resource:
resource "aws_s3_bucket_server_side_encryption_configuration" "claims_data" {
bucket = aws_s3_bucket.claims_data.id
rule {
apply_server_side_encryption_by_default {
kms_master_key_id = aws_kms_key.claims_encryption.arn
sse_algorithm = "aws:kms"
}
bucket_key_enabled = true
}
}
What does this code do?
This links your KMS key to the bucket. Every object uploaded is automatically encrypted with your claims_encryption key using the aws:kms algorithm.
bucket_key_enabled = true reduces KMS API calls by generating a bucket-level key derived from your CMK. This cuts costs significantly when you have many objects.
- Add the public access block below the encryption configuration:
resource "aws_s3_bucket_public_access_block" "claims_data" {
bucket = aws_s3_bucket.claims_data.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}
Why all four block settings?
Each setting blocks a different path to public exposure. Together they form a defense-in-depth barrier that prevents claims data from ever being publicly accessible, even if someone attaches a permissive ACL or policy.
- Add the bucket policy that enforces SSL-only access below the public access block:
resource "aws_s3_bucket_policy" "claims_data_enforce_ssl" {
bucket = aws_s3_bucket.claims_data.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "EnforceSSLOnly"
Effect = "Deny"
Principal = "*"
Action = "s3:*"
Resource = [
aws_s3_bucket.claims_data.arn,
"${aws_s3_bucket.claims_data.arn}/*"
]
Condition = {
Bool = {
"aws:SecureTransport" = "false"
}
}
}
]
})
}
What does this policy enforce?
This policy denies ALL S3 operations (s3:*) when the request does not use HTTPS. The aws:SecureTransport condition evaluates to false for any non-TLS connection.
This satisfies HIPAA §164.312(e)(1) which requires transmission security. Any unencrypted request to this bucket is rejected at the API level.
- Save aws/s3.tf.
Seeing indentation issues?
HCL is whitespace-sensitive in its formatting conventions. Make sure each nested block is indented by two spaces from its parent. You can run terraform fmt from within the aws/ directory to auto-format your files.
help me fix my s3.tf formatting
✔️ Awesome, I've got everything!
Great. Make sure you've saved aws/s3.tf with all five resources and the data source.
ⓧ I'd like to double check the full code
Here is the complete aws/s3.tf file:
resource "aws_s3_bucket" "claims_data" {
bucket = "${var.project_name}-claims-data-${data.aws_caller_identity.current.account_id}"
tags = {
Name = "${var.project_name}-claims-data"
DataClassification = "phi"
}
}
resource "aws_s3_bucket_versioning" "claims_data" {
bucket = aws_s3_bucket.claims_data.id
versioning_configuration {
status = "Enabled"
}
}
resource "aws_s3_bucket_server_side_encryption_configuration" "claims_data" {
bucket = aws_s3_bucket.claims_data.id
rule {
apply_server_side_encryption_by_default {
kms_master_key_id = aws_kms_key.claims_encryption.arn
sse_algorithm = "aws:kms"
}
bucket_key_enabled = true
}
}
resource "aws_s3_bucket_public_access_block" "claims_data" {
bucket = aws_s3_bucket.claims_data.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}
resource "aws_s3_bucket_policy" "claims_data_enforce_ssl" {
bucket = aws_s3_bucket.claims_data.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "EnforceSSLOnly"
Effect = "Deny"
Principal = "*"
Action = "s3:*"
Resource = [
aws_s3_bucket.claims_data.arn,
"${aws_s3_bucket.claims_data.arn}/*"
]
Condition = {
Bool = {
"aws:SecureTransport" = "false"
}
}
}
]
})
}
data "aws_caller_identity" "current" {}
Create outputs and deploy the infrastructure
Outputs expose key values from your infrastructure so other Terraform configurations (like your GCP layer) can reference them. You'll also run your first real deployment using the Makefile targets.
- Create a new file called outputs.tf inside your aws/ directory and add the following:
output "claims_bucket_arn" {
description = "ARN of the claims data S3 bucket"
value = aws_s3_bucket.claims_data.arn
}
output "claims_bucket_name" {
description = "Name of the claims data S3 bucket"
value = aws_s3_bucket.claims_data.id
}
output "kms_key_arn" {
description = "ARN of the KMS key used for PHI encryption"
value = aws_kms_key.claims_encryption.arn
}
What does this code do?
These outputs export three values you'll need later: the bucket ARN and name (for the GCP BigQuery Omni connection) and the KMS key ARN (for CloudTrail encryption in the next step).
You'll add a fourth output (bigquery_omni_role_arn) in the next step when you create the IAM role.
- Save aws/outputs.tf.
- Validate your Terraform configuration by running:
make validate
- You should see Success! The configuration is valid. for the aws directory.
- Generate the execution plan by running:
make plan-aws
What should I see?
Terraform shows a plan to create 7 resources: the KMS key, the KMS alias, the S3 bucket, versioning, encryption configuration, public access block, and bucket policy. Review the plan to confirm each resource matches your expectations.
- Apply the infrastructure by running:
make apply-aws
- Type yes when Terraform prompts for confirmation.
- Wait for the apply to complete. You should see Apply complete! Resources: 7 added, 0 changed, 0 destroyed. along with your three output values.
Seeing an access denied error?
Make sure your AWS CLI credentials have admin access. Run aws sts get-caller-identity to confirm you're authenticated as the correct IAM user or role.
If the error mentions the state bucket or DynamoDB table, verify you created those resources in Step 1 with the exact names claims-analytics-tf-state and claims-analytics-tf-lock.
help me debug my Terraform apply access denied error
Why does KMS cost $1 per month?
AWS charges $1/month for each customer-managed KMS key, regardless of usage. This is the minimum cost of HIPAA-compliant encryption. The key cannot be immediately deleted. It has a minimum 7-day waiting period (yours is set to 30 days).
Your encryption foundation and claims data bucket are live in AWS. Next up, you'll add CloudTrail audit logging and the IAM role that BigQuery Omni will assume to read this data.
Add AWS CloudTrail Audit Logging and IAM Role
Your claims data bucket is encrypted and locked down. But encryption alone does not satisfy HIPAA. Auditors need to prove that every access to Protected Health Information is logged, timestamped, and tamper-proof.
In this step, you will provision CloudTrail to record every S3 object-level event on your claims bucket. You will also create the IAM role that BigQuery Omni will eventually assume to query the data without moving it out of AWS.
In this step, get ready to:
- Provision a CloudTrail trail with a dedicated, encrypted audit log bucket and S3 data event selectors.
- Create the IAM role and least-privilege policy for BigQuery Omni cross-cloud access.
- Upload synthetic claims data to the encrypted S3 bucket for testing.
Provision the CloudTrail audit log bucket
CloudTrail needs its own dedicated S3 bucket to store log files. This audit bucket also needs KMS encryption and a bucket policy that grants the CloudTrail service write access.
- Create a new file called aws/cloudtrail.tf in your aws/ directory.
- Add the audit log bucket with its encryption, public access block, and CloudTrail bucket policy by pasting the following code:
resource "aws_s3_bucket" "audit_logs" {
bucket = "${var.project_name}-audit-logs-${data.aws_caller_identity.current.account_id}"
tags = {
Name = "${var.project_name}-audit-logs"
DataClassification = "internal"
}
}
resource "aws_s3_bucket_server_side_encryption_configuration" "audit_logs" {
bucket = aws_s3_bucket.audit_logs.id
rule {
apply_server_side_encryption_by_default {
kms_master_key_id = aws_kms_key.claims_encryption.arn
sse_algorithm = "aws:kms"
}
bucket_key_enabled = true
}
}
resource "aws_s3_bucket_public_access_block" "audit_logs" {
bucket = aws_s3_bucket.audit_logs.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}
resource "aws_s3_bucket_policy" "audit_logs" {
bucket = aws_s3_bucket.audit_logs.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "AWSCloudTrailAclCheck"
Effect = "Allow"
Principal = { Service = "cloudtrail.amazonaws.com" }
Action = "s3:GetBucketAcl"
Resource = aws_s3_bucket.audit_logs.arn
},
{
Sid = "AWSCloudTrailWrite"
Effect = "Allow"
Principal = { Service = "cloudtrail.amazonaws.com" }
Action = "s3:PutObject"
Resource = "${aws_s3_bucket.audit_logs.arn}/cloudtrail/*"
Condition = {
StringEquals = {
"s3:x-amz-acl" = "bucket-owner-full-control"
}
}
}
]
})
}
What does this code do?
This creates a dedicated audit log bucket with the same security controls as your claims bucket. The bucket policy grants only two specific permissions to the CloudTrail service:
- GetBucketAcl lets CloudTrail verify it has write access before attempting to deliver logs.
- PutObject allows CloudTrail to write log files under the cloudtrail/ prefix.
- The DataClassification = "internal" tag marks this as non-PHI data. Audit logs contain metadata about access, not the claims data itself.
Add the CloudTrail trail resource
The trail itself configures what gets logged and where the logs go. You need S3 data event selectors to capture every read and write on your claims bucket.
- Add the CloudTrail resource below the bucket policy in aws/cloudtrail.tf:
resource "aws_cloudtrail" "claims_audit" {
depends_on = [aws_s3_bucket_policy.audit_logs]
name = "${var.project_name}-claims-audit"
s3_bucket_name = aws_s3_bucket.audit_logs.id
s3_key_prefix = "cloudtrail"
include_global_service_events = true
is_multi_region_trail = false
enable_log_file_validation = true
kms_key_id = aws_kms_key.claims_encryption.arn
event_selector {
read_write_type = "All"
include_management_events = true
data_resource {
type = "AWS::S3::Object"
values = ["${aws_s3_bucket.claims_data.arn}/"]
}
}
tags = {
Name = "${var.project_name}-claims-audit"
}
}
What does this code do?
The depends_on ensures Terraform creates the bucket policy before the trail. Without it, CloudTrail would fail to validate write access during creation.
- enable_log_file_validation = true produces digest files that detect log tampering. This satisfies HIPAA's integrity control (§164.312(c)(1)).
- The event_selector block with data_resource type AWS::S3::Object logs every object-level event (GetObject, PutObject, DeleteObject) on the claims bucket.
- kms_key_id encrypts the trail logs with the same KMS key, so audit data is protected at rest too.
- Save aws/cloudtrail.tf.
- Validate the syntax by running:
cd aws && terraform validate
You should see Success! The configuration is valid. in the output.
Seeing a validation error?
- Check that you are referencing aws_kms_key.claims_encryption.arn and aws_s3_bucket.claims_data.arn exactly as they appear in aws/kms.tf and aws/s3.tf.
- Make sure you have the data "aws_caller_identity" "current" {} block in aws/s3.tf from the previous step.
- help me debug a Terraform validation error in my cloudtrail.tf
✔️ Awesome, I've got everything!
Great. Make sure you have saved aws/cloudtrail.tf before moving on.
ⓧ I'd like to double check the full code
Here is the complete aws/cloudtrail.tf file:
resource "aws_s3_bucket" "audit_logs" {
bucket = "${var.project_name}-audit-logs-${data.aws_caller_identity.current.account_id}"
tags = {
Name = "${var.project_name}-audit-logs"
DataClassification = "internal"
}
}
resource "aws_s3_bucket_server_side_encryption_configuration" "audit_logs" {
bucket = aws_s3_bucket.audit_logs.id
rule {
apply_server_side_encryption_by_default {
kms_master_key_id = aws_kms_key.claims_encryption.arn
sse_algorithm = "aws:kms"
}
bucket_key_enabled = true
}
}
resource "aws_s3_bucket_public_access_block" "audit_logs" {
bucket = aws_s3_bucket.audit_logs.id
block_public_acls = true
block_public_policy = true
ignore_public_acls = true
restrict_public_buckets = true
}
resource "aws_s3_bucket_policy" "audit_logs" {
bucket = aws_s3_bucket.audit_logs.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Sid = "AWSCloudTrailAclCheck"
Effect = "Allow"
Principal = { Service = "cloudtrail.amazonaws.com" }
Action = "s3:GetBucketAcl"
Resource = aws_s3_bucket.audit_logs.arn
},
{
Sid = "AWSCloudTrailWrite"
Effect = "Allow"
Principal = { Service = "cloudtrail.amazonaws.com" }
Action = "s3:PutObject"
Resource = "${aws_s3_bucket.audit_logs.arn}/cloudtrail/*"
Condition = {
StringEquals = {
"s3:x-amz-acl" = "bucket-owner-full-control"
}
}
}
]
})
}
resource "aws_cloudtrail" "claims_audit" {
depends_on = [aws_s3_bucket_policy.audit_logs]
name = "${var.project_name}-claims-audit"
s3_bucket_name = aws_s3_bucket.audit_logs.id
s3_key_prefix = "cloudtrail"
include_global_service_events = true
is_multi_region_trail = false
enable_log_file_validation = true
kms_key_id = aws_kms_key.claims_encryption.arn
event_selector {
read_write_type = "All"
include_management_events = true
data_resource {
type = "AWS::S3::Object"
values = ["${aws_s3_bucket.claims_data.arn}/"]
}
}
tags = {
Name = "${var.project_name}-claims-audit"
}
}
Create the BigQuery Omni IAM role
BigQuery Omni queries S3 data by assuming an AWS IAM role via OIDC federation. The role's trust policy restricts access to a specific Google-owned identity that you will obtain from GCP in a later step.
- Create a new file called aws/iam.tf in your aws/ directory.
- Add the IAM role and its read-only S3 policy by pasting the following code:
resource "aws_iam_role" "bigquery_omni_access" {
name = "${var.project_name}-bigquery-omni-role"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Principal = {
Federated = "accounts.google.com"
}
Action = "sts:AssumeRoleWithWebIdentity"
Condition = {
StringEquals = {
"accounts.google.com:sub" = var.bigquery_omni_identity
}
}
}
]
})
tags = {
Name = "${var.project_name}-bigquery-omni-role"
}
}
resource "aws_iam_role_policy" "bigquery_omni_s3_read" {
name = "${var.project_name}-bigquery-omni-s3-read"
role = aws_iam_role.bigquery_omni_access.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Action = [
"s3:GetObject"
]
Resource = "${aws_s3_bucket.claims_data.arn}/*"
},
{
Effect = "Allow"
Action = [
"s3:ListBucket"
]
Resource = aws_s3_bucket.claims_data.arn
}
]
})
}
What does this code do?
The IAM role uses sts:AssumeRoleWithWebIdentity to allow a Google-owned identity to federate into AWS without long-lived credentials. The var.bigquery_omni_identity is currently an empty string placeholder. You will fill it with the actual Google identity after deploying the GCP side.
The attached policy grants the minimum permissions BigQuery Omni needs: s3:GetObject to read individual claims files and s3:ListBucket to enumerate the bucket contents. No write access is granted.
Why OIDC over long-lived credentials?
Traditional cross-cloud access uses static access keys stored in the consuming service. These keys never expire and can be leaked. OIDC federation eliminates stored credentials entirely. Google assumes the AWS role on-demand, and AWS CloudTrail logs every assume-role event for your audit trail.
- Save aws/iam.tf.
- Validate the configuration again by running:
terraform validate
You should see Success! The configuration is valid. confirming both new files parse correctly.
Getting a reference error for bigquery_omni_identity?
- Make sure aws/variables.tf contains the bigquery_omni_identity variable with a default of "". This was created in an earlier step.
- help me fix a Terraform variable reference error in my IAM role configuration
✔️ Awesome, I've got everything!
Great. Make sure you have saved aws/iam.tf before moving on.
ⓧ I'd like to double check the full code
Here is the complete aws/iam.tf file:
resource "aws_iam_role" "bigquery_omni_access" {
name = "${var.project_name}-bigquery-omni-role"
assume_role_policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Principal = {
Federated = "accounts.google.com"
}
Action = "sts:AssumeRoleWithWebIdentity"
Condition = {
StringEquals = {
"accounts.google.com:sub" = var.bigquery_omni_identity
}
}
}
]
})
tags = {
Name = "${var.project_name}-bigquery-omni-role"
}
}
resource "aws_iam_role_policy" "bigquery_omni_s3_read" {
name = "${var.project_name}-bigquery-omni-s3-read"
role = aws_iam_role.bigquery_omni_access.id
policy = jsonencode({
Version = "2012-10-17"
Statement = [
{
Effect = "Allow"
Action = [
"s3:GetObject"
]
Resource = "${aws_s3_bucket.claims_data.arn}/*"
},
{
Effect = "Allow"
Action = [
"s3:ListBucket"
]
Resource = aws_s3_bucket.claims_data.arn
}
]
})
}
Upload sample claims data and deploy
Before deploying, you need synthetic test data in your claims bucket. This data simulates what a real healthcare payer would store: individual insurance claims with member identifiers, procedure codes, and dollar amounts.
- Create a file called sample_claims.csv in the root of your project directory by pasting the following:
claim_id,member_id,procedure_code,amount,date
CLM-001,MBR-1001,99213,150.00,2024-01-15
CLM-002,MBR-1002,99214,250.00,2024-01-16
CLM-003,MBR-1001,90834,175.00,2024-01-17
CLM-004,MBR-1003,99215,350.00,2024-01-18
CLM-005,MBR-1002,99213,150.00,2024-01-19
What is this data?
This is entirely synthetic. No real patient information is included. The procedure codes (99213, 99214, etc.) are standard CPT codes for office visits. In production, this bucket would hold millions of rows with actual PHI.
- Save sample_claims.csv.
- Deploy the CloudTrail and IAM resources by running the plan:
make plan-aws
- Review the plan output. You should see resources to add for the audit log bucket, its configuration, the CloudTrail trail, and the IAM role.
- Apply the plan by running:
make apply-aws
Terraform will prompt you to confirm. Type yes and press Enter.
- After the apply succeeds, upload the sample data to your claims bucket by running:
aws s3 cp sample_claims.csv s3://$(cd aws && terraform output -raw claims_bucket_name)/claims/sample_claims.csv
What does this command do?
The subshell $(cd aws && terraform output -raw claims_bucket_name) dynamically fetches the actual bucket name from Terraform state. This avoids hardcoding your AWS account ID. The file is uploaded under a claims/ prefix to keep the bucket organized.
You should see an upload confirmation showing the file copied to your S3 bucket.
Upload failed or bucket not found?
- Make sure make apply-aws completed successfully before uploading. The bucket must exist first.
- Confirm your AWS CLI is configured for the same account and region (us-east-1) as your Terraform deployment.
- If you see a permissions error, verify your AWS CLI credentials have s3:PutObject permission on the claims bucket.
- help me debug an S3 upload failure for my claims data
Your AWS audit layer is complete. Every time anyone accesses the claims bucket, CloudTrail records who did it, when, and from where. Next up, you will build the GCP side of the equation so BigQuery can query this data in place.
Build the GCP BigQuery Dataset and Omni Connection
Your AWS layer is fully provisioned. Claims data sits in an encrypted S3 bucket with CloudTrail watching every access event. Now you need to give the analytics team a way to query that data without moving it.
In this step, you will build the GCP side of the architecture. You will provision a BigQuery dataset in the Omni region that maps to your AWS resources, create the cross-cloud connection, and then complete the two-phase deployment that wires up IAM trust between the clouds.
In this step, get ready to:
- Define the BigQuery dataset and Omni connection in Terraform.
- Create GCP outputs that surface the Google-owned identity needed for cross-cloud trust.
- Deploy GCP resources and complete the IAM federation loop back to AWS.
Create the BigQuery dataset and enable the Connection API
BigQuery Omni lets GCP query data that lives in AWS without copying it. For this to work, you need a BigQuery dataset placed in the special aws-us-east-1 region and the BigQuery Connection API enabled in your project.
- Create a new file called gcp/bigquery.tf in VS Code.
- Add the API enablement resource and the BigQuery dataset by pasting this code:
resource "google_project_service" "bigquery_connection_api" {
service = "bigqueryconnection.googleapis.com"
disable_on_destroy = false
}
resource "google_bigquery_dataset" "claims_analytics" {
dataset_id = "claims_analytics"
friendly_name = "Claims Analytics"
description = "Cross-cloud analytics dataset for healthcare claims data via BigQuery Omni"
location = "aws-us-east-1"
labels = {
project = var.project_name
data_classification = "phi"
compliance = "hipaa"
}
}
What does this code do?
The google_project_service resource enables the BigQuery Connection API in your GCP project. Setting disable_on_destroy = false prevents Terraform from disabling the API if you later destroy this resource.
The google_bigquery_dataset resource creates your analytics dataset. The location = "aws-us-east-1" setting is what makes this a BigQuery Omni dataset. It tells BigQuery that the data lives in AWS us-east-1 and queries should execute there.
The data_classification = "phi" label tracks that this dataset handles Protected Health Information, satisfying HIPAA audit requirements.
Add the BigQuery Omni connection resource
The Omni connection is the bridge between GCP and AWS. It tells BigQuery which AWS IAM role to assume when reading data from S3.
- Below the dataset resource in gcp/bigquery.tf, add the connection resource:
resource "google_bigquery_connection" "aws_s3_connection" {
depends_on = [google_project_service.bigquery_connection_api]
connection_id = "${var.project_name}-aws-connection"
location = "aws-us-east-1"
friendly_name = "AWS Claims Data Connection"
description = "BigQuery Omni connection to AWS S3 claims data bucket"
aws {
access_role {
iam_role_id = var.aws_omni_role_arn
}
}
}
What does this code do?
The google_bigquery_connection resource creates the cross-cloud link. The aws.access_role.iam_role_id field points to the IAM role you created in the previous step. BigQuery will use sts:AssumeRoleWithWebIdentity to assume this role when querying S3.
The depends_on ensures the Connection API is enabled before Terraform tries to create the connection. Without it, the apply would fail with an API-not-enabled error.
Why BigQuery Omni over an ETL pipeline?
An ETL pipeline would copy claims data from AWS into GCP native storage. That means PHI crosses cloud boundaries, which requires a separate Business Associate Agreement covering the transfer and creates a second copy to secure.
With BigQuery Omni, queries execute in AWS where the data already lives. Only the query results cross clouds. This eliminates cross-cloud PHI transfer entirely and keeps your HIPAA compliance scope smaller.
- Save gcp/bigquery.tf.
✔️ Awesome, I've got everything!
Great. Double check you have saved the file.
ⓧ I'd like to double check the full code
resource "google_project_service" "bigquery_connection_api" {
service = "bigqueryconnection.googleapis.com"
disable_on_destroy = false
}
resource "google_bigquery_dataset" "claims_analytics" {
dataset_id = "claims_analytics"
friendly_name = "Claims Analytics"
description = "Cross-cloud analytics dataset for healthcare claims data via BigQuery Omni"
location = "aws-us-east-1"
labels = {
project = var.project_name
data_classification = "phi"
compliance = "hipaa"
}
}
resource "google_bigquery_connection" "aws_s3_connection" {
depends_on = [google_project_service.bigquery_connection_api]
connection_id = "${var.project_name}-aws-connection"
location = "aws-us-east-1"
friendly_name = "AWS Claims Data Connection"
description = "BigQuery Omni connection to AWS S3 claims data bucket"
aws {
access_role {
iam_role_id = var.aws_omni_role_arn
}
}
}
Define the GCP outputs
When Terraform creates the BigQuery Omni connection, GCP generates a Google-owned service account identity. You need to capture this identity and feed it back to AWS so the IAM role knows which principal to trust.
- Create a new file called gcp/outputs.tf and add the following output definitions:
output "bigquery_omni_identity" {
description = "Google-owned identity to configure in AWS IAM trust policy"
value = google_bigquery_connection.aws_s3_connection.aws[0].access_role[0].identity
}
output "bigquery_dataset_id" {
description = "BigQuery dataset ID for claims analytics"
value = google_bigquery_dataset.claims_analytics.dataset_id
}
output "bigquery_connection_id" {
description = "BigQuery Omni connection ID"
value = google_bigquery_connection.aws_s3_connection.connection_id
}
What does this code do?
The bigquery_omni_identity output extracts the Google-owned service account that BigQuery uses to call sts:AssumeRoleWithWebIdentity on your AWS IAM role. This value only exists after GCP creates the connection, which is why the deployment requires two phases.
The other two outputs (bigquery_dataset_id and bigquery_connection_id) provide references you will use in documentation and future queries.
- Save gcp/outputs.tf.
Unsure about the .aws[0].access_role[0] syntax?
Terraform represents nested blocks as lists, even when there is only one element. The [0] index accesses the first (and only) element in the aws and access_role blocks. help me understand Terraform nested block indexing
Deploy GCP and complete the cross-cloud IAM trust
This is the two-phase deployment pattern. First you deploy GCP to create the connection and get the Google identity. Then you feed that identity back to AWS so the IAM role trusts the correct principal.
- Before deploying, you need to provide variable values for the GCP configuration. Create a gcp/terraform.tfvars file with your project-specific values:
gcp_project_id = "[[GCP_PROJECT_ID="your-gcp-project-id"]]"
aws_omni_role_arn = "[[AWS_OMNI_ROLE_ARN="arn:aws:iam::123456789012:role/claims-analytics-bigquery-omni-role"]]"
claims_bucket_name = "[[CLAIMS_BUCKET_NAME="claims-analytics-claims-data-123456789012"]]"
Where do I find these values?
Your gcp_project_id is visible in the GCP Console project selector dropdown or in the URL bar.
For the AWS values, switch to your terminal in the aws/ directory and run terraform output to see all outputs from your AWS deployment. Use the bigquery_omni_role_arn and claims_bucket_name values.
- Retrieve your AWS output values by running this from the project root:
cd aws && terraform output -raw bigquery_omni_role_arn && echo "" && terraform output -raw claims_bucket_name && cd ..
- Copy those values into your gcp/terraform.tfvars file.
- Initialize the GCP Terraform configuration by running:
make init-gcp
- Plan the GCP deployment to preview what Terraform will create:
make plan-gcp
You should see Terraform planning to create 3 resources: the API enablement, the dataset, and the connection.
- Apply the GCP deployment by running:
make apply-gcp
Terraform will create the resources and display the outputs. Look for the bigquery_omni_identity value in the output. This is the Google-owned service account you need for the AWS trust policy.
- Copy the bigquery_omni_identity value from the apply output. You can also retrieve it by running:
cd gcp && terraform output -raw bigquery_omni_identity && cd ..
- Now complete the trust loop. Update your aws/terraform.tfvars to include the bigquery_omni_identity variable with the value you just copied:
bigquery_omni_identity = "[[BIGQUERY_OMNI_IDENTITY="the Google-owned identity string from GCP output"]]"
- Re-plan and re-apply the AWS configuration to update the IAM trust policy:
make plan-aws
make apply-aws
Terraform will detect that the aws_iam_role.bigquery_omni_access trust policy changed (the accounts.google.com:sub condition now has the real identity instead of an empty string) and update it in place.
What is the two-phase deployment pattern?
When two cloud providers depend on each other's outputs, you cannot deploy them in a single pass. GCP needs the AWS role ARN to create the connection. AWS needs the Google identity from that connection to finalize its trust policy.
The solution is to deploy AWS first with an empty trust identity, deploy GCP to generate the identity, then re-apply AWS with the real value. This is a standard enterprise pattern for OIDC cross-cloud federation.
Seeing a permission denied or API not enabled error?
Make sure the BigQuery Connection API (bigqueryconnection.googleapis.com) is enabled. If make apply-gcp failed on the connection resource, re-run it. The API enablement can take a few seconds to propagate.
If AWS re-apply fails, verify that the bigquery_omni_identity value in aws/terraform.tfvars matches the GCP output exactly. help me debug my cross-cloud IAM trust configuration
The cross-cloud bridge is live. BigQuery can now assume your AWS IAM role and read from the encrypted S3 claims bucket. Next, you will run policy-as-code validation to confirm the entire infrastructure meets HIPAA Technical Safeguards.
Run Policy-as-Code Validation and Build the Compliance Matrix
Your multi-cloud infrastructure is live. Both the AWS claims data layer and the GCP analytics layer are deployed and connected via OIDC federation.
Now you need to prove it is actually compliant. In this step, you will run Checkov policy-as-code scans against your Terraform to validate HIPAA Technical Safeguards, then document your design decisions in Architecture Decision Records.
In this step, get ready to:
- Run Checkov scans against both the AWS and GCP Terraform directories and review findings.
- Create a HIPAA compliance matrix mapping Terraform resources to Security Rule sections.
- Write Architecture Decision Records documenting key design choices.
Run Checkov against the AWS and GCP directories
Checkov is a static analysis tool that scans infrastructure-as-code for security misconfigurations. It checks your Terraform against hundreds of built-in policies covering encryption, access control, logging, and more.
- Run the Checkov scan against both cloud directories by running:
make lint
Checkov scans each .tf file and outputs a summary showing passed, failed, and skipped checks for every resource.
What should I see?
The output lists each check with a PASSED or FAILED result, the affected resource, and the file location. Your infrastructure already includes encryption, versioning, public access blocks, and SSL enforcement, so many checks will pass.
You may see some failures for checks like S3 access logging (your claims bucket logs to a separate audit bucket via CloudTrail, not via S3 access logs). These are acceptable trade-offs in this architecture. Review each finding and confirm you understand why it passed or failed.
How do I suppress a check I have intentionally decided against?
Add a comment inside the resource block using the format:
# checkov:skip=CKV_AWS_20:The bucket is a public static content host
Replace the check ID and reason with your own. Checkov then reports the check as SKIPPED instead of FAILED. Only suppress checks you have deliberately addressed through an alternative control.
Seeing command not found?
Make sure Checkov is installed in your active Python environment. Run checkov --version to confirm. If it is not found, reinstall with pip3 install checkov.
If make itself is not found, run the Checkov commands directly from the Makefile's lint target.
help me debug why Checkov is not running from my Makefile
Create the HIPAA compliance matrix
A compliance matrix maps each infrastructure resource to the specific regulatory control it satisfies. Auditors and compliance officers use this document to verify that every HIPAA Technical Safeguard has a corresponding technical implementation.
- Create the docs directory by running:
mkdir -p docs
- Create a new file called docs/hipaa-compliance-matrix.md and paste the following content:
# HIPAA Compliance Matrix
| HIPAA Section | Control | Terraform Resource | Implementation |
|---|---|---|---|
| §164.312(a)(1) | Access Control | `aws_iam_role`, `aws_iam_role_policy` | Least-privilege IAM role for BigQuery Omni with read-only S3 access |
| §164.312(a)(2)(iv) | Encryption and Decryption | `aws_kms_key`, `aws_s3_bucket_server_side_encryption_configuration` | AES-256 via KMS with automatic annual rotation |
| §164.312(b) | Audit Controls | `aws_cloudtrail` | S3 data event logging with log file validation |
| §164.312(c)(1) | Integrity | `aws_s3_bucket_versioning`, CloudTrail `enable_log_file_validation` | Object versioning and log digest validation |
| §164.312(e)(1) | Transmission Security | `aws_s3_bucket_policy` (SSL enforcement) | Deny policy for non-HTTPS requests |
| §164.312(e)(2)(ii) | Encryption in Transit | OIDC federation, S3 SSL policy | TLS-only access enforced at bucket level |
| §164.308(a)(1)(ii)(D) | Information System Activity Review | `aws_cloudtrail` with data events | All S3 object-level access logged and encrypted |
What does this matrix cover?
Each row maps a specific section of the HIPAA Security Rule (45 CFR Part 164) to the Terraform resource that implements that control. The sections span access control, encryption at rest, audit logging, data integrity, and transmission security.
This is the document an auditor reviews during a HIPAA compliance assessment. It proves that your infrastructure code enforces the required technical safeguards programmatically rather than relying on manual configuration.
- Save docs/hipaa-compliance-matrix.md.
- Confirm the file exists by running:
ls docs/
You should see hipaa-compliance-matrix.md listed in the output.
Don't see the file?
Make sure you created the docs/ directory first with mkdir -p docs. If you are in the wrong directory, navigate back to your project root.
help me find where my docs directory was created
Write Architecture Decision Records
Architecture Decision Records (ADRs) document the reasoning behind key technical choices. They capture what was decided, why, and what trade-offs were accepted. Enterprise teams use ADRs so that future engineers understand the rationale without hunting through old Slack threads.
- Create the ADR directory by running:
mkdir -p docs/adr
- Create the first ADR file docs/adr/001-bigquery-omni-over-etl.md and paste this content:
# ADR 001: BigQuery Omni Over ETL Replication
## Status
Accepted
## Context
Claims data resides in AWS S3 (industry standard for payer operations). The analytics team requires BigQuery for complex queries and ML integration. Two approaches exist:
1. ETL pipeline to replicate data from S3 to BigQuery native storage
2. BigQuery Omni to query S3 data in place
## Decision
Use BigQuery Omni to query S3 data in place.
## Consequences
- PHI never leaves AWS, simplifying HIPAA compliance (no cross-cloud PHI transfer)
- No egress costs for data movement (only query results cross clouds)
- Analytics team uses familiar BigQuery SQL
- Trade-off: query performance is slightly lower than native BigQuery storage
- Trade-off: requires maintaining cross-cloud IAM trust relationship
Why is this decision important?
The alternative (ETL replication) would copy PHI from AWS into GCP. That doubles your compliance surface area because you now need a Business Associate Agreement with both clouds covering the same data. BigQuery Omni eliminates this by querying data in place.
- Save docs/adr/001-bigquery-omni-over-etl.md.
- Create the second ADR file docs/adr/002-oidc-over-long-lived-credentials.md and paste this content:
# ADR 002: OIDC Federation Over Long-Lived Credentials
## Status
Accepted
## Context
BigQuery Omni needs to read from an AWS S3 bucket. Authentication options:
1. Long-lived AWS access keys stored in GCP
2. OIDC-based web identity federation (Google assumes AWS IAM role)
## Decision
Use OIDC web identity federation via sts:AssumeRoleWithWebIdentity.
## Consequences
- No long-lived credentials to rotate or leak
- Least-privilege: Google identity is scoped to a specific connection
- AWS CloudTrail logs every assume-role event for audit
- Trade-off: initial setup is more complex (chicken-and-egg between connection identity and trust policy)
- Mitigation: two-phase deployment pattern documented in Makefile
What does this ADR capture?
Long-lived access keys are the most common source of cloud breaches in healthcare. OIDC federation means there are zero static credentials to rotate, leak, or accidentally commit to Git. CloudTrail captures every single assume-role event for your audit trail.
- Save docs/adr/002-oidc-over-long-lived-credentials.md.
- Create the third ADR file docs/adr/003-isolated-state-per-cloud.md and paste this content:
# ADR 003: Isolated State Files Per Cloud Provider
## Status
Accepted
## Context
Multi-cloud Terraform can use a single state file or separate state files per provider.
## Decision
Maintain separate state files: S3+DynamoDB backend for AWS resources, GCS backend for GCP resources.
## Consequences
- Blast radius isolation: a failed AWS apply cannot corrupt GCP state
- Independent team workflows: networking and analytics teams can operate in parallel
- Clear ownership boundaries aligned with cloud-specific IAM
- Trade-off: cross-state references require explicit output/variable passing
- Trade-off: deployment ordering must be documented (AWS first, GCP second, then AWS trust update)
Why isolated state instead of one big file?
A single state file for both clouds means a corrupted terraform apply on one provider can take down the other. Separate backends create blast radius isolation. This also aligns with audit boundaries. An AWS compliance audit does not need to inspect GCP state.
- Save docs/adr/003-isolated-state-per-cloud.md.
- Confirm all documentation files exist by running:
ls docs/ docs/adr/
You should see hipaa-compliance-matrix.md under docs/ and three ADR files under docs/adr/.
- Commit all documentation to Git by running:
git add docs/
git commit -m "Add HIPAA compliance matrix and Architecture Decision Records"
Git commit failing?
If you see a message about no changes staged, make sure you ran git add docs/ first. If Git is not initialized, run git init from your project root before adding files.
help me fix my git commit for the compliance documentation
✔️ Awesome, I've got everything!
Great. Double check you have saved all files and committed them to Git.
ⓧ I'd like to double check the full code
Here is the full content for each documentation file.
docs/hipaa-compliance-matrix.md:
# HIPAA Compliance Matrix
| HIPAA Section | Control | Terraform Resource | Implementation |
|---|---|---|---|
| §164.312(a)(1) | Access Control | `aws_iam_role`, `aws_iam_role_policy` | Least-privilege IAM role for BigQuery Omni with read-only S3 access |
| §164.312(a)(2)(iv) | Encryption and Decryption | `aws_kms_key`, `aws_s3_bucket_server_side_encryption_configuration` | AES-256 via KMS with automatic annual rotation |
| §164.312(b) | Audit Controls | `aws_cloudtrail` | S3 data event logging with log file validation |
| §164.312(c)(1) | Integrity | `aws_s3_bucket_versioning`, CloudTrail `enable_log_file_validation` | Object versioning and log digest validation |
| §164.312(e)(1) | Transmission Security | `aws_s3_bucket_policy` (SSL enforcement) | Deny policy for non-HTTPS requests |
| §164.312(e)(2)(ii) | Encryption in Transit | OIDC federation, S3 SSL policy | TLS-only access enforced at bucket level |
| §164.308(a)(1)(ii)(D) | Information System Activity Review | `aws_cloudtrail` with data events | All S3 object-level access logged and encrypted |
docs/adr/001-bigquery-omni-over-etl.md:
# ADR 001: BigQuery Omni Over ETL Replication
## Status
Accepted
## Context
Claims data resides in AWS S3 (industry standard for payer operations). The analytics team requires BigQuery for complex queries and ML integration. Two approaches exist:
1. ETL pipeline to replicate data from S3 to BigQuery native storage
2. BigQuery Omni to query S3 data in place
## Decision
Use BigQuery Omni to query S3 data in place.
## Consequences
- PHI never leaves AWS, simplifying HIPAA compliance (no cross-cloud PHI transfer)
- No egress costs for data movement (only query results cross clouds)
- Analytics team uses familiar BigQuery SQL
- Trade-off: query performance is slightly lower than native BigQuery storage
- Trade-off: requires maintaining cross-cloud IAM trust relationship
docs/adr/002-oidc-over-long-lived-credentials.md:
# ADR 002: OIDC Federation Over Long-Lived Credentials
## Status
Accepted
## Context
BigQuery Omni needs to read from an AWS S3 bucket. Authentication options:
1. Long-lived AWS access keys stored in GCP
2. OIDC-based web identity federation (Google assumes AWS IAM role)
## Decision
Use OIDC web identity federation via sts:AssumeRoleWithWebIdentity.
## Consequences
- No long-lived credentials to rotate or leak
- Least-privilege: Google identity is scoped to a specific connection
- AWS CloudTrail logs every assume-role event for audit
- Trade-off: initial setup is more complex (chicken-and-egg between connection identity and trust policy)
- Mitigation: two-phase deployment pattern documented in Makefile
docs/adr/003-isolated-state-per-cloud.md:
# ADR 003: Isolated State Files Per Cloud Provider
## Status
Accepted
## Context
Multi-cloud Terraform can use a single state file or separate state files per provider.
## Decision
Maintain separate state files: S3+DynamoDB backend for AWS resources, GCS backend for GCP resources.
## Consequences
- Blast radius isolation: a failed AWS apply cannot corrupt GCP state
- Independent team workflows: networking and analytics teams can operate in parallel
- Clear ownership boundaries aligned with cloud-specific IAM
- Trade-off: cross-state references require explicit output/variable passing
- Trade-off: deployment ordering must be documented (AWS first, GCP second, then AWS trust update)
Your infrastructure is now validated by policy-as-code and backed by professional compliance documentation. Next up, you will write the project README and prepare the repository for release.
Write the README and Publish the Repository
Your infrastructure is deployed, validated, and fully documented with compliance mappings and Architecture Decision Records.
In this final step, you will tie everything together with a professional README that gives anyone visiting your repository the full picture. Then you will push the repository to GitHub and create a versioned release.
In this step, get ready to:
- Write a README with project overview, architecture diagram, prerequisites, and usage instructions.
- Document the Makefile targets and two-phase deployment ordering.
- Tag the repository with a semantic version and create a GitHub release.
Write the professional README
A README is the first thing anyone sees when they visit your repository. For an infrastructure project, it needs to communicate what the system does, how the pieces connect, and how to operate it.
- Create a file called README.md in the root of your project by running:
touch README.md
- Open README.md in VS Code and paste the following content:
# HIPAA-Compliant Multi-Cloud Claims Analytics Platform
Multi-cloud infrastructure (AWS + GCP) that enables healthcare payer analytics teams to query claims data stored in AWS S3 using GCP BigQuery Omni, without moving Protected Health Information across cloud boundaries.
## Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ GCP (Analytics) │
│ │
│ ┌──────────────────┐ ┌────────────────────────────────┐ │
│ │ BigQuery Dataset │────>│ BigQuery Omni Connection │ │
│ │ (claims_analytics)│ │ (OIDC Federation) │ │
│ └──────────────────┘ └──────────────┬─────────────────┘ │
│ │ │
│ State: GCS bucket │ AssumeRoleWith │
└───────────────────────────────────────────┼─WebIdentity─────────┘
│
┌───────────────────────────────────────────┼─────────────────────┐
│ AWS (Data Layer) │ │
│ v │
│ ┌──────────────────┐ ┌────────────────────────────────┐ │
│ │ KMS Key │────>│ S3 Bucket (claims data) │ │
│ │ (auto-rotation) │ │ SSE-KMS, versioning, SSL-only │ │
│ └──────────────────┘ └────────────────────────────────┘ │
│ │
│ ┌──────────────────┐ ┌────────────────────────────────┐ │
│ │ CloudTrail │────>│ S3 Audit Logs Bucket │ │
│ │ (data events) │ │ (KMS-encrypted) │ │
│ └──────────────────┘ └────────────────────────────────┘ │
│ │
│ State: S3 + DynamoDB │
└─────────────────────────────────────────────────────────────────┘
```
## Prerequisites
- Terraform >= 1.5.0
- AWS account with admin access
- GCP account with project owner access
- AWS CLI configured (`aws configure`)
- gcloud CLI configured (`gcloud auth application-default login`)
- Python 3.8+ (for Checkov)
- Checkov (`pip3 install checkov`)
What does this README section cover?
The project title and one-line summary tell visitors exactly what this infrastructure does.
The text-based architecture diagram shows the two-cloud layout (GCP analytics on top, AWS data layer on bottom) with the OIDC trust relationship connecting them. Text diagrams render in any terminal or Markdown viewer without needing external images.
- Save README.md.
- Preview the file in VS Code by pressing Cmd+Shift+V (macOS) or Ctrl+Shift+V (Windows). You should see a rendered heading, the box-drawing architecture diagram, and a bullet list of prerequisites.
Diagram looks broken in preview?
The box-drawing characters require a monospace font. In VS Code's Markdown preview, they render correctly by default. If lines look misaligned, check that you did not accidentally delete any spaces inside the code fence.
help me fix my README architecture diagram alignment
Add Makefile usage and deployment ordering
The most important operational information for this project is how to run the Makefile targets and the correct deployment sequence. Without this, another engineer would not know that AWS must go first.
- In README.md, add the following content below the Prerequisites section:
## Usage
### Makefile Targets
| Target | Description |
|--------|-------------|
| `make init-aws` | Initialize AWS Terraform backend |
| `make init-gcp` | Initialize GCP Terraform backend |
| `make plan-aws` | Generate AWS execution plan |
| `make plan-gcp` | Generate GCP execution plan |
| `make apply-aws` | Apply AWS infrastructure |
| `make apply-gcp` | Apply GCP infrastructure |
| `make destroy-aws` | Destroy all AWS resources |
| `make destroy-gcp` | Destroy all GCP resources |
| `make validate` | Validate Terraform syntax for both clouds |
| `make lint` | Run Checkov compliance scans |
| `make deploy` | Deploy both clouds (AWS first, then GCP) |
| `make teardown` | Destroy both clouds (GCP first, then AWS) |
### Deployment Order
This project requires a **two-phase AWS deployment** due to the cross-cloud identity dependency:
1. **Phase 1 - AWS Foundation**: Deploy AWS resources with an empty `bigquery_omni_identity` variable.
```bash
make init-aws
make plan-aws
make apply-aws
```
2. **Phase 2 - GCP Analytics**: Deploy GCP resources, passing the AWS IAM role ARN as input.
```bash
make init-gcp
make plan-gcp
make apply-gcp
```
3. **Phase 3 - Cross-Cloud Trust Update**: Copy the `bigquery_omni_identity` output from GCP and update `aws/terraform.tfvars`, then re-apply AWS.
```bash
cd gcp && terraform output bigquery_omni_identity
# Update aws/terraform.tfvars with the identity value
make plan-aws
make apply-aws
```
### Cost Estimates
| Resource | Monthly Cost |
|----------|-------------|
| AWS KMS Key | ~$1.00 |
| S3 Storage | < $0.10 (demo data) |
| CloudTrail Data Events | ~$0.10 per 100K events |
| BigQuery Omni Connection | Free when idle |
| BigQuery Queries | On-demand (first 1TB/month free) |
**Total ongoing cost**: ~$1-2/month when idle.
### Teardown
Order matters due to cross-cloud dependencies:
```bash
make teardown
```
This runs `destroy-gcp` first (removes BigQuery connection), then `destroy-aws` (removes S3, KMS, CloudTrail, IAM).
> **Note**: KMS keys have a minimum 7-day deletion window and will not be removed immediately.
## Compliance
- See [`docs/hipaa-compliance-matrix.md`](docs/hipaa-compliance-matrix.md) for the HIPAA Security Rule control mapping.
- See [`docs/adr/`](docs/adr/) for Architecture Decision Records.
## License
MIT
Why document deployment ordering?
The two-phase deployment is a chicken-and-egg problem. AWS needs the Google identity to complete the trust policy, but that identity only exists after GCP creates the BigQuery Omni connection. Documenting this ordering prevents another engineer from running make deploy and hitting a broken trust relationship.
This is the same pattern enterprise teams solve with CI/CD pipelines in production.
- Save README.md.
- Preview again with Cmd+Shift+V (macOS) or Ctrl+Shift+V (Windows). You should now see the Makefile targets table, the three deployment phases, and the cost estimates table.
Table not rendering correctly?
Markdown tables require a header separator row (the line with dashes and pipes). Make sure the |--------|-------------| row is present between the header and the data rows.
help me fix my Markdown table formatting
✔️ Awesome, I've got everything!
Great. Double check you have saved README.md with both sections.
ⓧ I'd like to double check the full code
Your complete README.md should contain all sections from above combined into a single file, starting with the # HIPAA-Compliant Multi-Cloud Claims Analytics Platform heading and ending with the ## License section.
Tag the repository and create a GitHub release
Semantic versioning marks a stable, reviewable snapshot of your infrastructure code. An annotated Git tag records who created the release and when. A GitHub release makes that tag discoverable with release notes attached.
- Stage and commit all files by running:
git add .
git commit -m "Add professional README and complete documentation"
- Create a new repository on GitHub. Go to github.com/new in your browser.
- Enter a repository name (e.g. hipaa-multi-cloud-claims-analytics).
- Leave it as Public so it is visible in your portfolio.
- Do not initialize with a README, .gitignore, or license (you already have these locally).
- Click Create repository.
- Copy the remote URL GitHub gives you, then add the remote and push by running:
git remote add origin [[REPO_URL="https://github.com/your-username/hipaa-multi-cloud-claims-analytics.git"]]
git push -u origin main
- Create an annotated tag for your first release by running:
git tag -a v1.0.0 -m "Initial release: HIPAA-compliant multi-cloud claims analytics platform"
What is an annotated tag?
An annotated tag is a full Git object that stores the tagger's name, email, date, and a message. Lightweight tags are simple pointers with no metadata. Annotated tags are the standard for releases because they provide an audit trail of who created the release.
- Push the tag to GitHub by running:
git push origin v1.0.0
- You should see output confirming the new tag was pushed, including a line like * [new tag] v1.0.0 -> v1.0.0.
Now create a GitHub release from that tag using the web interface.
- Go to your repository page on GitHub.
- Click Releases on the right side of the file list.
- Click Draft a new release.
- In the Choose a tag dropdown, select v1.0.0.
- Set the release title to v1.0.0.
- In the description box, summarize what the release includes. For example:
## What's Included
- HIPAA-compliant AWS claims data layer (KMS, S3, CloudTrail)
- GCP BigQuery Omni cross-cloud analytics connection
- Cross-cloud OIDC federation (no long-lived credentials)
- Checkov policy-as-code validation
- HIPAA compliance matrix and Architecture Decision Records
- Makefile-driven workflow with documented deployment ordering
- Click Publish release.
- You should now see the release page with your v1.0.0 tag, release notes, and a source code download.
Don't see your tag in the dropdown?
Make sure you ran git push origin v1.0.0 before navigating to the Releases page. GitHub only shows tags that exist on the remote, not your local repository.
help me push my git tag to GitHub
Your repository is live and versioned. Anyone visiting it can see the architecture, understand the compliance controls, and deploy the infrastructure themselves using the documented Makefile workflow.
Secret mission
Add FinOps Cost Estimation and Custom Governance Policies
Your infrastructure is deployed and validated. But in enterprise healthcare, two questions always follow: "How much does this cost?" and "Are we enforcing our own tagging standards?" Build Infracost integration for cost visibility and a custom Checkov policy that enforces your organization's data classification standard.
Clean Up Your Resources
Clean Up Your Resources
Decide whether to keep your resources running, pause them to come back later, or delete them entirely.
This project provisions cloud resources on both AWS and GCP that incur ongoing charges. The primary cost driver is the AWS KMS key at approximately $1 per month, with S3 storage and CloudTrail data events adding small amounts depending on usage.
Cost warning
Your AWS KMS key charges approximately $1 per month and cannot be deleted immediately. Terraform schedules it for deletion with a 30-day waiting period. If you want to stop charges as soon as possible, run the teardown commands in the Delete tab below.
Your total cost for this session is approximately $2-5. Leaving resources running without tearing down costs approximately $1-2 per month.
Resources you used:
- AWS KMS key (claims-analytics-claims alias) with automatic rotation.
- AWS S3 buckets (claims data bucket and audit logs bucket), both KMS-encrypted.
- AWS CloudTrail trail with S3 data event logging.
- AWS IAM role for BigQuery Omni cross-cloud access.
- GCP BigQuery dataset and Omni connection to AWS S3.
- Terraform state backends: S3 bucket (claims-analytics-tf-state), DynamoDB table (claims-analytics-tf-lock), and GCS bucket (claims-analytics-tf-state-gcp).
- Local project directory (claims-analytics) on your Desktop.
- Checkov Python package (installed via pip).
Keep everything running
No action needed. Choose this if you want to demonstrate live cross-cloud queries to interviewers or continue iterating on the infrastructure.
- Your ongoing cost is approximately $1-2 per month, primarily from the KMS key.
- BigQuery Omni connections are free when idle. You only pay for queries you run.
- Infracost and Checkov remain installed on your machine. They consume no cloud resources.
Pause - I'll come back to this later
Shut down the active data and query layers to reduce costs while keeping the core infrastructure for quick re-deployment.
- Delete the sample claims data from S3 by running:
aws s3 rm s3://$(cd aws && terraform output -raw claims_bucket_name)/ --recursive
- This removes the uploaded sample_claims.csv and stops any CloudTrail data event charges from further access.
- Your Terraform infrastructure stays intact. When you are ready to resume, re-upload sample data and run queries.
- The KMS key still costs approximately $1 per month while paused.
Delete - I don't want to use this again
Remove all cloud resources and local files. Order matters because GCP resources depend on the AWS IAM role.
Destroy all Terraform-managed resources by running these commands in order:
make destroy-gcp
make destroy-aws
Why GCP first?
The GCP BigQuery Omni connection references the AWS IAM role. Destroying AWS first would leave GCP resources pointing at a deleted role, which can cause Terraform errors during cleanup.
What about the KMS key?
Terraform schedules the KMS key for deletion with a 30-day waiting period (set by deletion_window_in_days = 30 in your code). The key is disabled immediately but the $1/month charge continues until the deletion completes after 30 days.
Delete the Terraform state backends (these are not managed by Terraform itself):
aws s3 rb s3://claims-analytics-tf-state --force
aws dynamodb delete-table --table-name claims-analytics-tf-lock
gsutil rm -r gs://claims-analytics-tf-state-gcp
Remove the local project directory:
cd ~/Desktop
rm -rf claims-analytics
On Windows, use this instead:
cd ~\Desktop
Remove-Item -Recurse -Force claims-analytics
Uninstall Checkov:
pip3 uninstall checkov
Nice Work!
Nice Work!
You've built exactly what healthcare payer infrastructure teams build in production. Your multi-cloud repository provisions HIPAA-compliant claims data infrastructure spanning AWS and GCP, complete with cross-cloud analytics, policy-as-code validation, and professional documentation.
You've learned how to:
- Build a HIPAA-compliant multi-cloud infrastructure using enterprise Terraform patterns with isolated state per cloud provider, KMS encryption, and CloudTrail audit logging.
- Implement cross-cloud OIDC federation so BigQuery Omni queries S3 data without moving PHI across cloud boundaries.
- Create policy-as-code validation with Checkov that enforces HIPAA Technical Safeguards before deployment, paired with Architecture Decision Records and a compliance mapping matrix.
- Added FinOps cost estimation with Infracost and custom governance policies that enforce data classification tagging across both cloud providers.
Ready to quiz yourself?