AWS Bedrock Multi-Tenant Routing

Introduction

30 Second Summary

Every quarter a new foundation model drops and suddenly your AI applications are out of date. When model IDs are hardcoded across a dozen services, swapping one model becomes a cross-team sprint with its own incident tickets.

In this project, you will build a Terraform substrate that gives four tenant segments config-driven access to any model on AWS Bedrock. One config edit swaps the model. One circuit breaker contains the blast radius. One dashboard shows the cost.

What You'll Build

You edit one config line, deploy in sixty seconds, and the same API endpoint responds from a completely different foundation model while the dashboard shows zero downtime and every dollar attributed to the segment that spent it.

By the end of this project, you'll have:

• A live API endpoint where you swap the underlying model from Claude to Nova by editing one AppConfig line, deploy in under sixty seconds, hit the endpoint, and see the new model respond with zero code changes and no CI/CD trigger.
• A Step Functions Express circuit breaker you can trip on command. Watch it detect a throttle, open per-segment, and route to a fallback model within two seconds while every other segment keeps serving.
• Per-segment cost attribution flowing to AWS Cost Explorer via application inference profiles. The healthcare segment stays pinned to HIPAA-eligible US regions at zero violations, validated by an audit script you run on demand.
• Secret Mission: Add Intelligent Prompt Routing so simple hospitality queries route to a cheaper model automatically, cutting costs by up to 30% with per-request traceability on the dashboard.

Are there any prerequisites?

This is a principal-architect-level build. You need a working AWS account with Bedrock model access enabled for Anthropic Claude and Amazon Nova, plus Terraform, AWS CLI v2, and Python 3.12 installed.

Familiarity with Terraform modules, IAM, Lambda, and Step Functions is expected. If you have not worked with distributed systems patterns at scale before, start with a lighter Bedrock project first.

Before We Start

Before any Terraform is written or any AWS resource provisioned, you need to lock in what this substrate prevents and why the architecture review board cares. This step is where you commit to the two incidents that triggered this work: hard-coded model IDs that caused missed model churn, and an uncontained throttle blast radius that degraded the entire platform.

Set Up and Run Preflight Verification

The substrate depends on Terraform, AWS CLI, Bedrock model access, SageMaker, and Python.

The Bedrock model catalog and feature availability drift between regions and accounts. A preflight that confirms the exact model IDs and feature state saves thirty minutes of debugging downstream and ensures the substrate is built on verified ground.

In this step, get ready to:

• Verify Terraform and AWS CLI versions and credentials.
• Preflight the Bedrock model catalog and Intelligent Prompt Routing API.
• Create the project directory structure and initialize Terraform.

Verify Terraform and AWS CLI access

Before building anything, you need to confirm the tools are at the right versions and your credentials can reach the services the substrate uses.

• Open the integrated terminal in Cursor by pressing Ctrl+` (backtick) on both macOS and Windows.
• Confirm your Terraform version by running:

terraform version

You should see a version number of 1.5.0 or higher printed.

Seeing a version below 1.5.0?

Download the latest Terraform binary from the official HashiCorp releases page and replace your existing install. The substrate's required_version constraint will block terraform init if your binary is too old.

help me upgrade Terraform to the latest version on my operating system

• Confirm your AWS CLI credentials are configured and can reach your account by running:

aws sts get-caller-identity

You should see a JSON response containing your Account, UserId, and Arn.

• Confirm the credentials can reach Bedrock by listing the first page of foundation models:

aws bedrock list-foundation-models --output table | head -20

You should see a table with model names and IDs. This confirms your credentials have Bedrock access in the current region.

Getting AccessDeniedException?

Your IAM identity needs permissions for bedrock:ListFoundationModels, bedrock:InvokeModel, and sagemaker:ListDomains at minimum. Attach the AmazonBedrockFullAccess and AmazonSageMakerFullAccess managed policies for this lab, or scope down to the specific actions.

help me troubleshoot AWS CLI access denied errors for Bedrock and SageMaker

Preflight the Bedrock model catalog

The substrate routes to specific model IDs. If model access has not been granted in your account, invocations will fail silently downstream.

• Confirm the Claude Haiku model is accessible by running:

aws bedrock list-foundation-models --query "modelSummaries[?modelId=='anthropic.claude-3-haiku-20240307-v1:0'].modelId" --output text

You should see anthropic.claude-3-haiku-20240307-v1:0 printed.

• Confirm the Nova Lite model is accessible by running:

aws bedrock list-foundation-models --query "modelSummaries[?modelId=='amazon.nova-lite-v1:0'].modelId" --output text

You should see amazon.nova-lite-v1:0 printed.

One or both model IDs missing from the output?

If either command returns blank, you need to request model access in the Bedrock console.

• Open the AWS console and navigate to Amazon Bedrock.
• In the left sidebar, click Model access.
• Request access for Anthropic Claude 3 Haiku and Amazon Nova Lite.
• Wait for the status to show Access granted, then re-run the commands above.

Access approval is typically instant for Amazon models and may take a few minutes for Anthropic models.

• Confirm the Intelligent Prompt Routing API is available in your region by running:

aws bedrock list-prompt-routers

An empty list is fine. The command returning without error confirms the API is available in-region.

• Confirm SageMaker access by running:

aws sagemaker list-domains

An empty list or a list of existing domains confirms your credentials have SageMaker access.

Why verify all of this upfront?

The Bedrock model catalog and feature availability drift between regions, accounts, and time. A model that was accessible last month may have been revoked, or a new API like Intelligent Prompt Routing may not be available in every region.

Catching access gaps now saves thirty minutes of cryptic AccessDeniedException debugging in the middle of the Terraform apply.

Create the project directory and initialize Terraform

With all access confirmed, you can scaffold the project directory. The directory structure separates Terraform modules by responsibility and keeps Lambda source alongside the infrastructure code.

• Move to your Desktop by running this command:

cd ~/Desktop

• Create the project directory and all subdirectories by running:

mkdir -p kai-substrate/modules/routing \
  kai-substrate/modules/circuit-breaker \
  kai-substrate/modules/model-registry \
  kai-substrate/lambda/router \
  kai-substrate/docs/adr

• Confirm the structure was created by running:

ls kai-substrate/

You should see docs, lambda, and modules listed.

• Move into the project directory:

cd kai-substrate

• Create the root main.tf with the provider block by running:

cat > main.tf << 'EOF'
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.54"
    }
  }
  required_version = ">= 1.5.0"
}

provider "aws" {
  region = "us-east-1"
}
EOF

What does this provider block do?

The required_providers block pins the AWS provider to version ~> 6.54, which means any 6.x release at 6.54.0 or higher. This ensures everyone on the team uses a compatible provider version.

The required_version constraint blocks terraform init if the Terraform binary is below 1.5.0. The provider block sets us-east-1 as the default region for all resources.

Before you run the next command, what do you expect terraform init to do with this provider block?

• Initialize the Terraform working directory by running:

terraform init

You should see output ending with Terraform has been successfully initialized! confirming the AWS provider was downloaded and the backend is ready.

Seeing a version constraint error?

If terraform init fails with a version constraint message, your Terraform binary is below 1.5.0. Upgrade Terraform and re-run the command.

If you see a network error downloading the provider, check your internet connection and confirm no corporate proxy is blocking registry.terraform.io.

help me fix terraform init failures related to version constraints or provider downloads

✔️ Awesome, I've got everything!

Great. Double check you have saved main.tf and that terraform init completed without errors.

ⓧ I'd like to double check the full code

Your kai-substrate/main.tf should contain exactly this:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.54"
    }
  }
  required_version = ">= 1.5.0"
}

provider "aws" {
  region = "us-east-1"
}

Your environment is verified and the Terraform root is initialized. Next up, you will build the routing core: a Lambda router that speaks the Converse API, backed by AppConfig for config-driven model selection, with your first two architecture decision records documenting the design choices.

Build the Routing Core

The substrate exists to prove one contract: the model ID lives in config, not in code. A config-driven swap means the next twelve months of model churn never touches application code.

In this step you write the first two architecture decision records (design precedes build in a reviewed substrate), build the Lambda router that calls the Bedrock Converse API, and stand up AppConfig with an API Gateway endpoint. By the end, one curl returns a model response with the modelId proving traceability.

In this step, get ready to:

• Write ADR-001 (Converse API) and ADR-002 (AppConfig over SSM) and the Lambda router function.
• Terraform AppConfig, API Gateway, Lambda, and IAM into a routing module.
• Deploy and invoke the endpoint to prove the Converse API contract works end-to-end.

Write the architecture decision records and Lambda router

Design precedes build. ADR-001 captures why the Converse API is the single integration contract across all model families. ADR-002 captures why AppConfig beats SSM Parameter Store for runtime config swaps.

• Create docs/adr/adr-001-converse-api.md with the following content:

# ADR-001: Converse API as the Model-Agnostic Contract

## Status
Accepted

## Context
The substrate must support multiple model families (Claude, Nova, Titan, Llama, Mistral) without per-model payload formatting in application code.

## Decision
Use the Bedrock Converse API as the single integration contract for all message-capable models.

## Consequences
One integration works across all supported models. Model swaps require only a config change. No per-model payload formatting in application or routing code.

• Create docs/adr/adr-002-appconfig.md with the following content:

# ADR-002: AppConfig over SSM Parameter Store

## Status
Accepted

## Context
Model configuration (modelId, temperature, maxTokens) must be swappable at runtime with zero application redeploy, with rollback capability.

## Decision
Use AWS AppConfig with hosted configuration profiles and the Lambda extension for cached reads.

## Consequences
Deployment strategies with rollback-on-alarm. Validation hooks before deployment. Lambda extension provides ~5ms cached reads. Config changes deploy in under 60 seconds with no application code change.

• Confirm both ADRs exist by running:

ls docs/adr/

You should see adr-001-converse-api.md and adr-002-appconfig.md listed.

Why write ADRs before code?

In a reviewed substrate, every structural choice carries a rationale the next architect can read. ADRs prevent tribal knowledge. They also give the architecture review board a paper trail without scheduling a meeting.

Now build the Lambda function that reads config from AppConfig and calls the Bedrock Converse API.

• Create lambda/router/handler.py and add the imports, client setup, and config-fetching function:

import json
import boto3
import os
import urllib.request

appconfig_client = boto3.client('appconfig')
bedrock_client = boto3.client('bedrock-runtime')

APPCONFIG_EXTENSION_PORT = 2772

def get_model_config(segment: str) -> dict:
    app = os.environ['APPCONFIG_APP']
    env = os.environ['APPCONFIG_ENV']
    profile = segment
    url = f'http://localhost:{APPCONFIG_EXTENSION_PORT}/applications/{app}/environments/{env}/configurations/{profile}'
    with urllib.request.urlopen(url) as response:
        return json.loads(response.read().decode())

What does this code do?

The AppConfig Lambda extension runs as a sidecar on port 2772. Instead of calling the AppConfig API directly (which adds latency and requires token management), the function fetches config from localhost at ~5ms.

The segment variable (hospitality, retail, healthcare, eu-sovereign) maps directly to an AppConfig configuration profile name. Each segment gets its own model config independently.

• Below the get_model_config function, add the main Lambda handler:

def handler(event, context):
    segment = event.get('pathParameters', {}).get('segment', 'hospitality')
    body = json.loads(event.get('body', '{}'))
    prompt = body.get('prompt', '')

    config = get_model_config(segment)
    model_id = config['modelId']
    temperature = config.get('temperature', 0.5)
    max_tokens = config.get('maxTokens', 512)

    messages = [{'role': 'user', 'content': [{'text': prompt}]}]

    response = bedrock_client.converse(
        modelId=model_id,
        messages=messages,
        inferenceConfig={'maxTokens': max_tokens, 'temperature': temperature}
    )

    output_text = response['output']['message']['content'][0]['text']
    usage = response.get('usage', {})

    return {
        'statusCode': 200,
        'body': json.dumps({
            'response': output_text,
            'modelId': model_id,
            'segment': segment,
            'usage': usage
        })
    }

What does the handler do?

It extracts the segment from the URL path, fetches that segment's config from AppConfig, and calls bedrock_client.converse() with the model ID and inference parameters from config.

The response includes modelId alongside the model output. This proves traceability: the caller always knows which model served the request.

• Save handler.py and confirm the file exists by running:

ls lambda/router/

You should see handler.py in the output.

Don't see handler.py?

Make sure you are in the kai-substrate root directory when running ls. The path is relative to project root.

help me find where my handler.py file ended up

Terraform the routing module

The routing module provisions AppConfig, IAM, Lambda, and API Gateway as one reusable unit. You will build it in chunks, verifying with terraform plan after the config layer is complete.

• Create modules/routing/main.tf and add the AppConfig application, environment, and segment config:

resource "aws_appconfig_application" "kai" {
  name = "Kai Concierge Cloud"
}

resource "aws_appconfig_environment" "production" {
  name           = "production"
  application_id = aws_appconfig_application.kai.id
}

locals {
  segments = ["hospitality", "retail", "healthcare", "eu-sovereign"]
  default_config = jsonencode({
    modelId         = "anthropic.claude-3-haiku-20240307-v1:0"
    temperature     = 0.5
    maxTokens       = 512
    fallbackModelId = "amazon.nova-lite-v1:0"
  })
}

What does this config do?

One AppConfig application ("Kai Concierge Cloud") with one environment ("production") serves all four segments. The default_config local defines the initial model routing: Claude Haiku as primary, Nova Lite as fallback.

Each segment starts with identical config. Later steps diverge them (geographic profiles for healthcare, inference profiles per segment).

• Below the locals block, add the configuration profiles and hosted versions:

resource "aws_appconfig_configuration_profile" "segments" {
  for_each       = toset(local.segments)
  application_id = aws_appconfig_application.kai.id
  name           = each.key
  location_uri   = "hosted"
}

resource "aws_appconfig_hosted_configuration_version" "segments" {
  for_each               = toset(local.segments)
  application_id         = aws_appconfig_application.kai.id
  configuration_profile_id = aws_appconfig_configuration_profile.segments[each.key].configuration_profile_id
  content_type           = "application/json"
  content                = local.default_config
}

What does for_each do here?

The for_each = toset(local.segments) iterates over all four segment names, creating one configuration profile and one hosted version per segment. Each segment gets its own AppConfig profile that the Lambda router addresses by name.

• Below the hosted versions, add the deployment strategy and deployments:

resource "aws_appconfig_deployment_strategy" "instant" {
  name                           = "instant"
  deployment_duration_in_minutes = 0
  final_bake_time_in_minutes     = 0
  growth_factor                  = 100
  replicate_to                   = "NONE"
}

resource "aws_appconfig_deployment" "segments" {
  for_each                 = toset(local.segments)
  application_id           = aws_appconfig_application.kai.id
  environment_id           = aws_appconfig_environment.production.environment_id
  configuration_profile_id = aws_appconfig_configuration_profile.segments[each.key].configuration_profile_id
  configuration_version    = aws_appconfig_hosted_configuration_version.segments[each.key].version_number
  deployment_strategy_id   = aws_appconfig_deployment_strategy.instant.id
}

What does the deployment strategy do?

The "instant" strategy deploys config changes immediately (zero growth duration, zero bake time). This is what enables the under-60-second swap contract. In production you would add a bake time with a CloudWatch alarm for rollback-on-error.

• Update main.tf in the project root to reference the routing module. Find the provider "aws" block and add the module reference below it:

module "routing" {
  source = "./modules/routing"
}

• Initialize the module and run a plan by running:

terraform init -upgrade && terraform plan

You should see Terraform discovering the routing module and reporting resources to create (AppConfig application, environment, profiles, versions, strategy, and deployments).

Seeing module errors?

Confirm modules/routing/main.tf exists in the correct path relative to your root main.tf. The source = "./modules/routing" path must match your directory structure exactly.

help me debug my Terraform module path error

Add Lambda, API Gateway, and deploy

The AppConfig layer stores config. Now you add the compute and networking layer: an IAM role, the Lambda function with the AppConfig extension, and an HTTP API Gateway to expose the endpoint.

• In modules/routing/main.tf, below the deployment resources, add the IAM role for the Lambda:

resource "aws_iam_role" "lambda_router" {
  name = "kai-lambda-router-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action    = "sts:AssumeRole"
      Effect    = "Allow"
      Principal = { Service = "lambda.amazonaws.com" }
    }]
  })
}

• Below the role, add the IAM policy granting Bedrock invoke, AppConfig read, and CloudWatch logs permissions:

resource "aws_iam_role_policy" "lambda_router" {
  role = aws_iam_role.lambda_router.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect   = "Allow"
        Action   = ["bedrock:InvokeModel", "bedrock:Converse"]
        Resource = "*"
      },
      {
        Effect   = "Allow"
        Action   = ["appconfig:GetLatestConfiguration", "appconfig:StartConfigurationSession"]
        Resource = "*"
      },
      {
        Effect   = "Allow"
        Action   = ["logs:CreateLogGroup", "logs:CreateLogStream", "logs:PutLogEvents"]
        Resource = "*"
      }
    ]
  })
}

What permissions does the router need?

Three permission groups: bedrock:InvokeModel and bedrock:Converse for model calls, appconfig:GetLatestConfiguration and appconfig:StartConfigurationSession for the AppConfig extension, and CloudWatch Logs for observability.

• Run a quick plan to confirm the IAM resources are valid:

terraform plan | grep -c "will be created"

You should see the count increasing as Terraform picks up the new IAM resources alongside the earlier AppConfig resources.

• Below the IAM policy, add the archive data source and Lambda function:

data "archive_file" "router" {
  type        = "zip"
  source_file = "${path.root}/lambda/router/handler.py"
  output_path = "${path.root}/lambda/router/handler.zip"
}

resource "aws_lambda_function" "router" {
  function_name    = "kai-router"
  role             = aws_iam_role.lambda_router.arn
  handler          = "handler.handler"
  runtime          = "python3.12"
  timeout          = 30
  memory_size      = 256
  filename         = data.archive_file.router.output_path
  source_code_hash = data.archive_file.router.output_base64sha256

  environment {
    variables = {
      APPCONFIG_APP = aws_appconfig_application.kai.id
      APPCONFIG_ENV = aws_appconfig_environment.production.environment_id
    }
  }

  layers = ["arn:aws:lambda:us-east-1:027255383542:layer:AWS-AppConfig-Extension:128"]
}

What does the layers argument do?

The layers argument attaches the AWS-managed AppConfig Lambda extension as a sidecar. This is what serves config on localhost:2772 inside the Lambda execution environment. The extension handles caching, token management, and background polling for config updates.

• Below the Lambda function, add the API Gateway HTTP API, integration, and route:

resource "aws_apigatewayv2_api" "kai" {
  name          = "kai-substrate-api"
  protocol_type = "HTTP"
}

resource "aws_apigatewayv2_integration" "router" {
  api_id             = aws_apigatewayv2_api.kai.id
  integration_type   = "AWS_PROXY"
  integration_uri    = aws_lambda_function.router.invoke_arn
  integration_method = "POST"
}

resource "aws_apigatewayv2_route" "invoke" {
  api_id    = aws_apigatewayv2_api.kai.id
  route_key = "POST /{segment}/invoke"
  target    = "integrations/${aws_apigatewayv2_integration.router.id}"
}

What does the route capture?

The route POST /{segment}/invoke captures the segment name from the URL path and passes it to the Lambda as pathParameters.segment. This is how one endpoint serves all four tenant segments.

• Finally, add the stage, Lambda permission, and output:

resource "aws_apigatewayv2_stage" "default" {
  api_id      = aws_apigatewayv2_api.kai.id
  name        = "$default"
  auto_deploy = true
}

resource "aws_lambda_permission" "apigw" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.router.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "${aws_apigatewayv2_api.kai.execution_arn}/*/*"
}

output "api_endpoint" {
  value = aws_apigatewayv2_stage.default.invoke_url
}

Why auto_deploy on the stage?

The $default stage with auto_deploy = true means route changes go live immediately. The api_endpoint output gives you the invoke URL for testing.

• Save modules/routing/main.tf and deploy the entire substrate by running:

terraform apply -auto-approve

Terraform provisions all the resources. This takes 1-2 minutes. Watch for the api_endpoint output at the end.

Seeing a zip error or handler not found?

The archive_file data source looks for lambda/router/handler.py relative to the Terraform root. Confirm the file exists at exactly that path from where you run terraform apply.

help me fix my Terraform archive_file path for the Lambda router

Before you test, predict: what model ID do you expect to see in the response? The answer lives in the AppConfig config you just deployed.

• Test the endpoint by running this command (replace the URL with your api_endpoint output):

curl -X POST [[API_ENDPOINT="your api_endpoint output URL"]]/hospitality/invoke \
  -H "Content-Type: application/json" \
  -d '{"prompt": "Hello, what time is checkout?"}'

You should see a JSON response containing "modelId": "anthropic.claude-3-haiku-20240307-v1:0", the model response text, the segment name, and token usage. The modelId in the response proves the Converse API contract: the model came from config, not code.

Getting a 500 or timeout?

Check CloudWatch Logs for the kai-router function. Common causes:

• AppConfig extension not loading: confirm the layer ARN matches your region (us-east-1).
• Bedrock access denied: confirm model access is enabled for Claude Haiku in the Bedrock console Model access page.
• Timeout at 30s: first invocations are cold starts. Wait a moment and retry.

help me debug my kai-router Lambda returning 500

✔️ Awesome, I've got everything!

Great. Double check you have saved both modules/routing/main.tf and lambda/router/handler.py.

ⓧ I'd like to double check the full code

Your routing module is the largest file. Scroll through modules/routing/main.tf and confirm these sections appear in order:

• AppConfig application and environment resources.
• Locals block with segments list and default_config.
• Configuration profiles and hosted versions using for_each.
• Deployment strategy (instant) and four deployments.
• IAM role and policy.
• Archive data source and Lambda function with the AppConfig extension layer.
• API Gateway HTTP API, integration, route, stage, permission, and output.

Your root main.tf should look like this:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.54"
    }
  }
  required_version = ">= 1.5.0"
}

provider "aws" {
  region = "us-east-1"
}

module "routing" {
  source = "./modules/routing"
}

Your lambda/router/handler.py should contain the get_model_config function and the handler function that calls bedrock_client.converse() and returns modelId in the response body.

The routing core is live. One endpoint, one model, one call, config-driven. Next up, you will layer in inference profiles for per-segment cost attribution, HIPAA region pinning for healthcare, and a SageMaker Model Registry for the fine-tuned voice model.

Add Inference Profiles, HIPAA Pinning, and Model Registry

In Step 2 you proved the routing contract works: one endpoint, one model call, config from AppConfig. But the CTO cannot attribute that spend to any segment on the monthly bill.

The hospitality cost overrun happened because no team could be charged for the token spend. Application inference profiles attach per-segment cost-allocation tags so spend flows natively to Cost Explorer without log reconciliation. The healthcare segment must never route outside HIPAA-eligible US regions. The fine-tuned hospitality voice model needs version governance with known rollback versions and a warm endpoint at a predictable hourly cost.

In this step, get ready to:

• Provision application inference profiles with cost-allocation tags for all four segments and pin the healthcare segment to HIPAA-eligible US regions.
• Stand up the SageMaker Model Registry with three approved versions and a warm real-time endpoint.
• Wire per-request cost attribution into the Lambda router and write the region-eligibility audit script.

Terraform inference profiles and HIPAA region pinning

Before building, capture the design decision that drives this substep. ADR-003 records why you use application inference profiles over raw model IDs.

• Create the file docs/adr/adr-003-inference-profiles.md with the following content:

# ADR-003: Application Inference Profiles over Raw Model IDs

## Status
Accepted

## Context
Per-segment cost attribution was missing, causing unattributed Bedrock spend on the monthly bill.

## Decision
Use application inference profiles with cost-allocation tags (Segment, CostCenter, Environment) instead of raw model IDs.

## Consequences
Native cost attribution flows to Cost Explorer and CUR 2.0 without log-based reconciliation. Inference profiles also provide an indirection layer for model swaps.

What does ADR-003 capture?

Without inference profiles, attributing Bedrock spend to a segment requires parsing CUR logs, matching by IAM principal, and reconciling against invocation timestamps. Inference profiles make attribution native: each profile carries tags that flow directly to Cost Explorer line items.

Now add the infrastructure. The inference profiles need two maps: one for cost centers and one for model ARNs (the healthcare segment uses the US geographic profile ID).

• In modules/routing/main.tf, find the existing locals block that contains segments and default_config.
• Remove the default_config line entirely (you will replace it with per-segment configs shortly).
• Add a new locals block below the existing one with these maps:

locals {
  segment_cost_centers = {
    hospitality  = "CC-001"
    retail       = "CC-002"
    healthcare   = "CC-003"
    eu-sovereign = "CC-004"
  }
  segment_model_arns = {
    hospitality  = "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-haiku-20240307-v1:0"
    retail       = "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-haiku-20240307-v1:0"
    healthcare   = "arn:aws:bedrock:us-east-1::foundation-model/us.anthropic.claude-3-haiku-20240307-v1:0"
    eu-sovereign = "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-haiku-20240307-v1:0"
  }
}

Why does healthcare use a different model ARN?

The us.anthropic.claude-3-haiku-20240307-v1:0 ID is the US geographic cross-region inference profile. When you create an application inference profile with this as the source, destination regions are restricted to us-east-1, us-west-2, and us-east-2. All three are under the AWS BAA for HIPAA eligibility.

• Add the inference profile resource below the new locals block:

resource "aws_bedrock_inference_profile" "segments" {
  for_each = local.segment_cost_centers
  name     = "kai-${each.key}-profile"
  model_source {
    copy_from = local.segment_model_arns[each.key]
  }
  tags = {
    Segment     = each.key
    CostCenter  = each.value
    Environment = "production"
  }
}

What does this resource do?

Each aws_bedrock_inference_profile creates an APPLICATION-type profile that wraps the source model. The tags block attaches cost-allocation metadata (Segment, CostCenter, Environment) that flows to Cost Explorer and CUR 2.0 on every invocation through this profile.

The healthcare segment needs a defense-in-depth control: an IAM deny that blocks invocations through any global inference profile.

• Add this IAM deny policy below the existing aws_iam_role_policy.lambda_router resource:

resource "aws_iam_role_policy" "healthcare_deny" {
  role = aws_iam_role.lambda_router.id
  name = "healthcare-global-profile-deny"
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect   = "Deny"
        Action   = "bedrock:InvokeModel"
        Resource = "arn:aws:bedrock:*:*:inference-profile/global.*"
      }
    ]
  })
}

Why an explicit Deny?

The geographic profile restricts destination regions, but an operator could accidentally invoke a global profile (which routes to any region worldwide). The IAM deny makes that impossible: any attempt to invoke through a global.* profile ARN returns AccessDenied regardless of other Allow statements.

• Also add cloudwatch:PutMetricData to the existing aws_iam_role_policy.lambda_router Allow statements. Find the Statement array and add this entry after the logs statement:

      {
        Effect   = "Allow"
        Action   = ["cloudwatch:PutMetricData"]
        Resource = "*"
      }

Now replace the AppConfig hosted configuration versions. The old for_each pattern used a single default_config for all segments. Each segment now needs its own config with the inferenceProfileArn field pointing to its inference profile.

• Remove the existing aws_appconfig_hosted_configuration_version.segments resource (the one using for_each and local.default_config).
• Replace it with this hospitality config version:

resource "aws_appconfig_hosted_configuration_version" "hospitality" {
  application_id           = aws_appconfig_application.kai.id
  configuration_profile_id = aws_appconfig_configuration_profile.segments["hospitality"].configuration_profile_id
  content_type             = "application/json"
  content = jsonencode({
    modelId            = "anthropic.claude-3-haiku-20240307-v1:0"
    inferenceProfileArn = aws_bedrock_inference_profile.segments["hospitality"].id
    temperature        = 0.5
    maxTokens          = 512
    fallbackModelId    = "amazon.nova-lite-v1:0"
  })
}

• Add the retail config version following the same pattern (change hospitality to retail in both the resource name and the profile reference):

resource "aws_appconfig_hosted_configuration_version" "retail" {
  application_id           = aws_appconfig_application.kai.id
  configuration_profile_id = aws_appconfig_configuration_profile.segments["retail"].configuration_profile_id
  content_type             = "application/json"
  content = jsonencode({
    modelId            = "anthropic.claude-3-haiku-20240307-v1:0"
    inferenceProfileArn = aws_bedrock_inference_profile.segments["retail"].id
    temperature        = 0.5
    maxTokens          = 512
    fallbackModelId    = "amazon.nova-lite-v1:0"
  })
}

• Add the healthcare config version. Note the different modelId using the US geographic profile ID:

resource "aws_appconfig_hosted_configuration_version" "healthcare" {
  application_id           = aws_appconfig_application.kai.id
  configuration_profile_id = aws_appconfig_configuration_profile.segments["healthcare"].configuration_profile_id
  content_type             = "application/json"
  content = jsonencode({
    modelId            = "us.anthropic.claude-3-haiku-20240307-v1:0"
    inferenceProfileArn = aws_bedrock_inference_profile.segments["healthcare"].id
    temperature        = 0.5
    maxTokens          = 512
    fallbackModelId    = "amazon.nova-lite-v1:0"
  })
}

• Add the eu-sovereign config version (same pattern as hospitality, with eu-sovereign references):

resource "aws_appconfig_hosted_configuration_version" "eu_sovereign" {
  application_id           = aws_appconfig_application.kai.id
  configuration_profile_id = aws_appconfig_configuration_profile.segments["eu-sovereign"].configuration_profile_id
  content_type             = "application/json"
  content = jsonencode({
    modelId            = "anthropic.claude-3-haiku-20240307-v1:0"
    inferenceProfileArn = aws_bedrock_inference_profile.segments["eu-sovereign"].id
    temperature        = 0.5
    maxTokens          = 512
    fallbackModelId    = "amazon.nova-lite-v1:0"
  })
}

• Similarly, remove the old aws_appconfig_deployment.segments resource (the for_each version) and replace it with four individual deployments. Add the hospitality and retail deployments:

resource "aws_appconfig_deployment" "hospitality" {
  application_id           = aws_appconfig_application.kai.id
  environment_id           = aws_appconfig_environment.production.environment_id
  configuration_profile_id = aws_appconfig_configuration_profile.segments["hospitality"].configuration_profile_id
  configuration_version    = aws_appconfig_hosted_configuration_version.hospitality.version_number
  deployment_strategy_id   = aws_appconfig_deployment_strategy.instant.id
}

resource "aws_appconfig_deployment" "retail" {
  application_id           = aws_appconfig_application.kai.id
  environment_id           = aws_appconfig_environment.production.environment_id
  configuration_profile_id = aws_appconfig_configuration_profile.segments["retail"].configuration_profile_id
  configuration_version    = aws_appconfig_hosted_configuration_version.retail.version_number
  deployment_strategy_id   = aws_appconfig_deployment_strategy.instant.id
}

• Add the healthcare and eu-sovereign deployments:

resource "aws_appconfig_deployment" "healthcare" {
  application_id           = aws_appconfig_application.kai.id
  environment_id           = aws_appconfig_environment.production.environment_id
  configuration_profile_id = aws_appconfig_configuration_profile.segments["healthcare"].configuration_profile_id
  configuration_version    = aws_appconfig_hosted_configuration_version.healthcare.version_number
  deployment_strategy_id   = aws_appconfig_deployment_strategy.instant.id
}

resource "aws_appconfig_deployment" "eu_sovereign" {
  application_id           = aws_appconfig_application.kai.id
  environment_id           = aws_appconfig_environment.production.environment_id
  configuration_profile_id = aws_appconfig_configuration_profile.segments["eu-sovereign"].configuration_profile_id
  configuration_version    = aws_appconfig_hosted_configuration_version.eu_sovereign.version_number
  deployment_strategy_id   = aws_appconfig_deployment_strategy.instant.id
}

Before you apply, do you expect terraform plan to show creates, destroys, or both for the AppConfig resources?

• Run terraform plan to confirm the inference profiles and config replacements are valid:

terraform plan

You should see 4 inference profiles to create, 4 new hosted config versions, 4 new deployments, and the old for_each resources marked for destruction. The healthcare profile creation will reference the US geographic model ARN.

Seeing a cycle error or unknown reference?

Confirm you removed the old aws_appconfig_hosted_configuration_version.segments and aws_appconfig_deployment.segments for_each resources completely. The new individual resources reference the inference profiles by key, so both must exist in the same file.

Also check that local.default_config is fully removed from the locals block. Any lingering reference to it causes an unknown value error.

help me debug my Terraform plan error for the inference profile references

Add SageMaker Model Registry

The fine-tuned hospitality voice model needs version governance: a named current version and two prior versions available for immediate rollback. A warm SageMaker endpoint serves the latest approved version at a predictable ~$0.07/hr on ml.m5.large.

• Create docs/adr/adr-004-warm-endpoint.md to record this decision:

# ADR-004: Warm SageMaker Endpoint with Registry Rollback

## Status
Accepted

## Context
The fine-tuned hospitality voice model needs version governance and a warm serving endpoint with predictable cost.

## Decision
Use SageMaker Model Registry with three approved versions and a single ml.m5.large real-time endpoint (~$0.07/hr).

## Consequences
Known hourly cost. Version-pinned deployment. Rollback is a registry approval change, not a redeploy. Two prior versions available for immediate rollback.

Why a warm endpoint instead of serverless?

A warm real-time endpoint has predictable latency (no cold start) and a fixed hourly cost you can name in the architecture review. At lab scale, one ml.m5.large instance costs ~$0.07/hr (~$1.68/day). You tear it down when not testing.

• Create the directory modules/model-registry/ and add main.tf with the model package group and three versions:

resource "aws_sagemaker_model_package_group" "kai_voice" {
  model_package_group_name = "kai-voice-model"
  model_package_group_description = "Fine-tuned hospitality voice model with version governance"
}

resource "aws_sagemaker_model_package" "v1" {
  model_package_group_name = aws_sagemaker_model_package_group.kai_voice.model_package_group_name
  model_package_description = "Voice model v1 (two versions prior)"
  model_approval_status = "Approved"
}

resource "aws_sagemaker_model_package" "v2" {
  model_package_group_name = aws_sagemaker_model_package_group.kai_voice.model_package_group_name
  model_package_description = "Voice model v2 (one version prior)"
  model_approval_status = "Approved"
}

What does Approved status mean here?

Each aws_sagemaker_model_package with model_approval_status = "Approved" is available for deployment. Rollback is changing which version the endpoint serves, not reverting the approval. Having two prior approved versions means immediate fallback without re-approval workflows.

• Add the current version and the endpoint resources below the v2 package:

resource "aws_sagemaker_model_package" "current" {
  model_package_group_name = aws_sagemaker_model_package_group.kai_voice.model_package_group_name
  model_package_description = "Voice model current (production)"
  model_approval_status = "Approved"
}

resource "aws_sagemaker_endpoint_configuration" "kai_voice" {
  name = "kai-voice-endpoint-config"
  production_variants {
    variant_name           = "primary"
    model_name             = "kai-voice-current"
    initial_instance_count = 1
    instance_type          = "ml.m5.large"
  }
}

resource "aws_sagemaker_endpoint" "kai_voice" {
  name                 = "kai-voice-endpoint"
  endpoint_config_name = aws_sagemaker_endpoint_configuration.kai_voice.name
}

What does this endpoint cost?

The ml.m5.large instance runs at ~$0.07/hr while the endpoint is in service. You pay only while it exists. The Before You Go section covers pausing (deleting the endpoint) to stop the clock.

• Wire the new module into the root kai-substrate/main.tf. Find the existing module "routing" block and add below it:

module "model_registry" {
  source = "./modules/model-registry"
}

• Apply both the inference profiles and the model registry together:

terraform apply -auto-approve

Terraform provisions the four inference profiles, deploys the updated AppConfig configs, creates the SageMaker model package group with three versions, and starts the endpoint. The endpoint takes 3-5 minutes to reach InService status.

Seeing an error about model access or InsufficientCapacity?

If the inference profile creation fails with an access error, confirm you enabled model access for both Claude and Nova in the Bedrock console (Model access page) during Step 1.

If the SageMaker endpoint returns InsufficientCapacity, try a different instance type in the endpoint configuration or a different availability zone. At lab scale this is rare.

help me debug my Terraform apply failure for inference profiles or SageMaker endpoint

Wire cost attribution, requestMetadata, and the audit script

The Lambda router needs two updates: use the inference profile ARN for invocations (so cost tags apply), and publish per-segment token metrics to CloudWatch for the dashboard.

• Open lambda/router/handler.py. Find this line near the top:

appconfig_client = boto3.client('appconfig')
bedrock_client = boto3.client('bedrock-runtime')

• Replace those two lines with:

bedrock_client = boto3.client('bedrock-runtime')
cw_client = boto3.client('cloudwatch')

• Next, find the line that sets model_id in the handler function:

    model_id = config['modelId']

• Replace it with this line that prefers the inference profile ARN when available:

    model_id = config.get('inferenceProfileArn', config['modelId'])

Why fall back to modelId?

The config.get('inferenceProfileArn', config['modelId']) pattern means the router works whether or not an inference profile is configured. If AppConfig returns a profile ARN, it uses that (and cost tags apply). If not, it falls back to the raw model ID.

• Find the bedrock_client.converse( call and add requestMetadata to it. The full call should now look like:

    response = bedrock_client.converse(
        modelId=model_id,
        messages=messages,
        inferenceConfig={'maxTokens': max_tokens, 'temperature': temperature},
        requestMetadata={'segment': segment}
    )

What does requestMetadata do?

The requestMetadata field writes per-request attributes to model invocation logs. It is not a cost-allocation tag (those come from the inference profile). It provides operational attribution: which segment triggered which specific invocation.

• Finally, add the CloudWatch metric publishing after the usage = response.get('usage', {}) line. Add these lines before the return statement:

    input_tokens = usage.get('inputTokens', 0)
    output_tokens = usage.get('outputTokens', 0)
    cw_client.put_metric_data(
        Namespace='KaiSubstrate',
        MetricData=[
            {'MetricName': 'InputTokens', 'Dimensions': [{'Name': 'Segment', 'Value': segment}], 'Value': input_tokens, 'Unit': 'Count'},
            {'MetricName': 'OutputTokens', 'Dimensions': [{'Name': 'Segment', 'Value': segment}], 'Value': output_tokens, 'Unit': 'Count'},
        ]
    )

What do these custom metrics provide?

Publishing InputTokens and OutputTokens per segment to CloudWatch gives the operations team real-time token usage visibility. The dashboard in Step 4 uses these metrics to show per-segment consumption without parsing Bedrock logs.

• Save handler.py and redeploy the Lambda by running:

terraform apply -auto-approve

Terraform detects the changed source hash and updates the Lambda function.

Now write the region-eligibility audit script that validates zero HIPAA violations.

• Create scripts/audit-region.sh with the following content:

#!/bin/bash
# Region eligibility audit for healthcare segment
# Queries CloudTrail for InvokeModel events and asserts all regions are HIPAA-eligible

HIPAA_REGIONS=("us-east-1" "us-west-2" "us-east-2")
VIOLATIONS=0

EVENTS=$(aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventName,AttributeValue=InvokeModel \
  --query 'Events[?contains(CloudTrailEvent, `healthcare`)].CloudTrailEvent' \
  --output text)

• Add the validation loop and result reporting below:

while IFS= read -r event; do
  REGION=$(echo "$event" | python3 -c "import sys,json; print(json.load(sys.stdin).get('awsRegion','unknown'))")
  if [[ ! " ${HIPAA_REGIONS[@]} " =~ " ${REGION} " ]]; then
    echo "VIOLATION: healthcare InvokeModel in non-HIPAA region: $REGION"
    VIOLATIONS=$((VIOLATIONS+1))
  fi
done <<< "$EVENTS"

if [ $VIOLATIONS -eq 0 ]; then
  echo "PASS: Zero HIPAA region violations detected."
else
  echo "FAIL: $VIOLATIONS violation(s) detected."
  exit 1
fi

• Make the script executable:

chmod +x scripts/audit-region.sh

When do you run this audit?

The audit script queries CloudTrail for healthcare-segment InvokeModel events and asserts every one occurred in us-east-1, us-west-2, or us-east-2. You run it in Step 5 as one of the four validation bars. Zero violations is the pass condition.

Cost-allocation tag activation is manual

The Segment, CostCenter, and Environment tags must be activated in the AWS Billing console under Cost Allocation Tags. Activation takes up to 24 hours and is not retroactive. This step is not automatable via Terraform. You will activate them when you run the full validation in Step 5.

Before running the final verification, what do you expect to see in the healthcare profile's models field?

• Verify the healthcare inference profile restricts to US regions by running:

aws bedrock get-inference-profile --inference-profile-identifier $(terraform output -raw healthcare_profile_arn) --query 'models' --output json

You should see a models array where each modelArn contains only US region identifiers (us-east-1, us-west-2, us-east-2). No global or EU region ARNs should appear.

• Verify the Model Registry has three versions by running:

aws sagemaker list-model-packages --model-package-group-name kai-voice-model --query 'ModelPackageSummaryList[].ModelApprovalStatus' --output table

You should see three rows, each showing Approved. That confirms version governance is in place with the current model and two rollback candidates.

Profile ARN not found or list-model-packages returns empty?

If the terraform output command fails, you may need to add an output block for the healthcare profile ARN to your routing module. Alternatively, find the profile ARN from terraform state show module.routing.aws_bedrock_inference_profile.segments["healthcare"] and pass it directly to the CLI command.

If list-model-packages returns an empty list, confirm terraform apply completed without errors on the model-registry module. Check terraform state list for module.model_registry resources.

help me verify my inference profiles and SageMaker model registry are provisioned correctly

Cost attribution is flowing through inference profiles, the healthcare segment is pinned to US regions with a defense-in-depth IAM deny, and the voice model has three governed versions with a warm endpoint. Next up, you will wrap each invocation in a circuit breaker that contains blast radius to one segment and fails over within two seconds.

Build the Circuit Breaker and CloudWatch Dashboard

Your routing substrate delivers config-driven model invocations to four segments. But what happens when Bedrock throttles one segment's model? Without containment, the caller retries indefinitely, latency balloons, and the blast radius spreads to adjacent segments.

This step adds a Step Functions Express circuit breaker that opens per-segment, fails over to the fallback model within two seconds, and heals after a 30-second TTL. You will also provision a CloudWatch dashboard that makes circuit state, latency, and cost posture visible to the operations team.

In this step, get ready to:

• Terraform the DynamoDB circuit state table and Step Functions Express state machine with retry and fallback logic.
• Wire API Gateway to the state machine and validate the circuit opens on throttle within two seconds.
• Provision a CloudWatch dashboard with six per-segment widgets and write the operational runbooks.

Write ADR-005 and create the circuit breaker module

The circuit breaker needs two primitives: a state store that tracks which segments are open, and an orchestrator that checks state before invoking. DynamoDB provides the state store with TTL-based automatic expiry. Step Functions Express provides sub-second orchestration with synchronous API Gateway integration.

• Create the architecture decision record for this choice by creating docs/adr/adr-005-express-sfn.md:

# ADR-005: Step Functions Express over Standard for Circuit Breaker

## Status
Accepted

## Context
The circuit breaker must respond synchronously to API Gateway within the 2-second RTO budget.

## Decision
Use Step Functions Express workflows for sub-second orchestration with synchronous StartSyncExecution API Gateway integration.

## Consequences
Synchronous response to API Gateway. Sub-second state transitions. $1/M transitions (near-zero at lab scale). Express workflows do not support long-running async patterns, which is acceptable for this use case.

Why Express over Standard?

Standard workflows are asynchronous and bill per transition with higher latency. Express workflows respond synchronously to the caller with sub-second state transitions. The circuit breaker sits on the hot path between API Gateway and the model. It must add milliseconds, not seconds.

• Create the directory modules/circuit-breaker/ and start the module by adding the DynamoDB table and IAM role to modules/circuit-breaker/main.tf:

resource "aws_dynamodb_table" "circuit_status" {
  name         = "kai-circuit-status"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "segment_name"

  attribute {
    name = "segment_name"
    type = "S"
  }

  ttl {
    attribute_name = "expire_timestamp"
    enabled        = true
  }
}

resource "aws_iam_role" "sfn_role" {
  name = "kai-sfn-circuit-breaker-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action    = "sts:AssumeRole"
      Effect    = "Allow"
      Principal = { Service = "states.amazonaws.com" }
    }]
  })
}

What does this table do?

Each row represents an open circuit for one segment. The partition key segment_name ensures blast radius stays per-segment. The expire_timestamp TTL attribute auto-deletes the row after 30 seconds, closing the circuit without a manual reset.

• Add the IAM policy that grants Step Functions the permissions it needs. Add this below the IAM role in the same file:

resource "aws_iam_role_policy" "sfn_policy" {
  role = aws_iam_role.sfn_role.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect   = "Allow"
        Action   = ["lambda:InvokeFunction"]
        Resource = var.lambda_arn
      },
      {
        Effect   = "Allow"
        Action   = ["dynamodb:GetItem", "dynamodb:PutItem"]
        Resource = aws_dynamodb_table.circuit_status.arn
      },
      {
        Effect   = "Allow"
        Action   = ["logs:CreateLogGroup", "logs:CreateLogDelivery", "logs:PutLogEvents"]
        Resource = "*"
      }
    ]
  })
}

What does this IAM policy grant?

Three capabilities: invoke the Lambda router (primary and fallback calls), read/write the DynamoDB circuit table (check status and open circuits), and write execution logs for debugging.

• Now add the state machine definition. Start with the resource header and the first two states (GetCircuitStatus and CheckCircuit). Add this below the IAM policy:

resource "aws_sfn_state_machine" "circuit_breaker" {
  name     = "kai-circuit-breaker"
  role_arn = aws_iam_role.sfn_role.arn
  type     = "EXPRESS"

  definition = jsonencode({
    Comment = "Kai circuit breaker with DynamoDB state and fallback routing"
    StartAt = "GetCircuitStatus"
    States = {
      GetCircuitStatus = {
        Type     = "Task"
        Resource = "arn:aws:states:::dynamodb:getItem"
        Parameters = {
          TableName = aws_dynamodb_table.circuit_status.name
          Key = {
            segment_name = { "S.$" = "$.segment" }
          }
        }
        ResultPath = "$.circuitStatus"
        Next       = "CheckCircuit"
      }

What does GetCircuitStatus do?

It reads the DynamoDB table for the current segment. If a row exists (circuit is open and TTL has not expired), the next state will route to fallback. If no row exists, the circuit is closed and the primary model proceeds.

• Continue the state machine definition with the CheckCircuit choice state:

      CheckCircuit = {
        Type = "Choice"
        Choices = [
          {
            Variable      = "$.circuitStatus.Item"
            IsPresent     = true
            Next          = "FallbackInvoke"
          }
        ]
        Default = "InvokePrimary"
      }

• Add the InvokePrimary state with retry and catch logic:

      InvokePrimary = {
        Type     = "Task"
        Resource = "arn:aws:states:::lambda:invoke"
        Parameters = {
          FunctionName = var.lambda_arn
          "Payload.$"  = "$"
        }
        Retry = [
          {
            ErrorEquals     = ["ThrottlingException", "States.TaskFailed"]
            IntervalSeconds = 0.5
            MaxAttempts     = 3
            BackoffRate     = 2.0
          }
        ]
        Catch = [
          {
            ErrorEquals = ["States.ALL"]
            Next        = "OpenCircuit"
            ResultPath  = "$.error"
          }
        ]
        ResultPath = "$.result"
        Next       = "Success"
      }

How does the retry and catch work?

On a ThrottlingException (HTTP 429), Step Functions retries 3 times with 0.5s intervals and a 2.0 backoff rate. If all retries exhaust, the Catch clause routes to OpenCircuit rather than failing the execution. Total retry window is under 2 seconds.

• Add the OpenCircuit state that writes the circuit-open record to DynamoDB:

      OpenCircuit = {
        Type     = "Task"
        Resource = "arn:aws:states:::dynamodb:putItem"
        Parameters = {
          TableName = aws_dynamodb_table.circuit_status.name
          Item = {
            segment_name     = { "S.$" = "$.segment" }
            expire_timestamp = { "N.$" = "States.Format('{}', States.MathAdd(States.StringToJson(States.Format('{}', $$.Execution.StartTime)), 30))" }
          }
        }
        ResultPath = "$.openCircuit"
        Next       = "FallbackInvoke"
      }

• Add the FallbackInvoke and Success states, then close the state machine definition:

      FallbackInvoke = {
        Type     = "Task"
        Resource = "arn:aws:states:::lambda:invoke"
        Parameters = {
          FunctionName = var.lambda_arn
          Payload = {
            "pathParameters.$" = "$.pathParameters"
            "body.$"           = "$.body"
            useFallback        = true
          }
        }
        ResultPath = "$.result"
        Next       = "Success"
      }
      Success = {
        Type = "Succeed"
      }
    }
  })
}

What does the fallback path do?

OpenCircuit writes a DynamoDB item with a 30-second TTL. All subsequent requests for that segment hit CheckCircuit, find the item, and route directly to FallbackInvoke until the TTL expires. FallbackInvoke calls the same Lambda router but with useFallback: true, signaling it to use the fallback model from AppConfig.

• Complete the module by adding the API Gateway integration role, variables, and output at the bottom of the same file:

resource "aws_iam_role" "apigw_sfn" {
  name = "kai-apigw-sfn-role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action    = "sts:AssumeRole"
      Effect    = "Allow"
      Principal = { Service = "apigateway.amazonaws.com" }
    }]
  })
}

resource "aws_iam_role_policy" "apigw_sfn" {
  role = aws_iam_role.apigw_sfn.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect   = "Allow"
      Action   = "states:StartSyncExecution"
      Resource = aws_sfn_state_machine.circuit_breaker.arn
    }]
  })
}

variable "lambda_arn" {}
variable "api_id" {}

output "state_machine_arn" {
  value = aws_sfn_state_machine.circuit_breaker.arn
}

Why does API Gateway need its own role?

API Gateway invokes Step Functions via the StartSyncExecution action. It needs an IAM role that grants exactly that permission on the circuit breaker state machine ARN. This replaces the previous direct Lambda invocation with the full circuit-breaker-wrapped path.

• Save modules/circuit-breaker/main.tf.
• Validate the module syntax by running:

cd modules/circuit-breaker && terraform validate

You should see Success! The configuration is valid. printed. If you see syntax errors, check that all braces and brackets in the state machine definition are balanced.

Seeing a variable error?

If Terraform reports undeclared variables, ensure the variable "lambda_arn" {} and variable "api_id" {} declarations are at the bottom of the file. The validate command confirms syntax only, not that the module is wired into the root yet.

Help me debug my circuit breaker module syntax

Wire the module, update the Lambda, and deploy

The circuit breaker module is defined but not wired into the substrate yet. You need to export outputs from the routing module, reference the circuit breaker module in the root, and update the Lambda handler to support the useFallback flag that FallbackInvoke passes.

• Add the required outputs to the bottom of modules/routing/main.tf:

output "lambda_arn" {
  value = aws_lambda_function.router.arn
}

output "api_id" {
  value = aws_apigatewayv2_api.kai.id
}

• Add the circuit breaker module to kai-substrate/main.tf below the existing modules:

module "circuit_breaker" {
  source       = "./modules/circuit-breaker"
  lambda_arn   = module.routing.lambda_arn
  api_id       = module.routing.api_id
}

• Now update the Lambda router to support fallback mode. In lambda/router/handler.py, find this line:

    prompt = body.get('prompt', '')

• Replace the model selection block that follows it so the section looks like this:

    prompt = body.get('prompt', '')
    use_fallback = event.get('useFallback', False)

    config = get_model_config(segment)
    if use_fallback:
        model_id = config.get('fallbackModelId', config.get('inferenceProfileArn', config['modelId']))
    else:
        model_id = config.get('inferenceProfileArn', config['modelId'])
    temperature = config.get('temperature', 0.5)
    max_tokens = config.get('maxTokens', 512)

• Also update the return statement at the bottom of the handler to include the fallback flag:

    return {
        'statusCode': 200,
        'body': json.dumps({
            'response': output_text,
            'modelId': model_id,
            'segment': segment,
            'usage': usage,
            'fallback': use_fallback
        })
    }

What changed in the Lambda?

The router now checks an useFallback field in the event payload. When Step Functions routes to FallbackInvoke, it sets this to true. The Lambda responds with the fallback model from AppConfig instead of the primary. The response body includes "fallback": true so the caller knows which path was taken.

• Save all files and apply the full infrastructure by running:

cd ~/Desktop/kai-substrate && terraform init && terraform apply -auto-approve

Terraform creates the DynamoDB table, IAM roles, Step Functions state machine, and updates the API Gateway integration. Watch for the state machine ARN in the output.

• Before you test, think about this: if you invoke the hospitality segment with a valid prompt right now, do you think the circuit will be open or closed?
• Test the happy path by running:

curl -X POST $(terraform output -raw api_endpoint)/hospitality/invoke \
  -d '{"prompt":"What time is checkout?"}'

You should see a JSON response with the model answer, "fallback": false, and the primary model ID. The circuit is closed (no DynamoDB row exists), so InvokePrimary handles the request directly.

• Now trigger a circuit open. Temporarily update the hospitality AppConfig profile to use a nonexistent model ID to force failures. After 3 retries exhaust, invoke again and observe the fallback respond.
• Verify the circuit breaker path in the Step Functions console. Open the Step Functions service, select kai-circuit-breaker, and check the CloudWatch Logs group for the execution. You should see the path: GetCircuitStatus > CheckCircuit > InvokePrimary (retries exhausted) > OpenCircuit > FallbackInvoke.

Why not list-executions for Express workflows?

Express state machines do not support the list-executions API. Execution history for Express workflows is written to CloudWatch Logs. Enable logging on the state machine and use CloudWatch Logs Insights or the Step Functions console to inspect execution flow.

• Write the circuit-breaker drill runbook at docs/runbooks/circuit-breaker-drill.md:

# Circuit Breaker Drill Runbook

## Purpose
Validate that the circuit breaker opens on throttle, fallback responds within 2s, and circuit heals after TTL expiry.

## Steps
1. Invoke the retail segment with a prompt that triggers ThrottlingException (or temporarily set the model ID to an invalid value to force failure).
2. Observe Step Functions execution history: confirm GetCircuitStatus > InvokePrimary > OpenCircuit > FallbackInvoke states executed.
3. Measure end-to-end response time: assert < 2 seconds.
4. Confirm fallback model ID appears in the response body.
5. Wait 30 seconds for the DynamoDB TTL to expire the circuit-open record.
6. Invoke again: confirm InvokePrimary succeeds (circuit closed).
7. Check CloudWatch dashboard: circuit breaker state widget should show 0 (all closed).

Circuit not opening on failure?

Check that the Retry ErrorEquals matches the error name Step Functions receives from the Lambda. If your Lambda raises a generic exception rather than a named error, the Catch clause on States.ALL should still fire. Verify in the CloudWatch Logs execution history that the retries actually triggered before the catch engaged.

Help me debug my circuit breaker retry and catch configuration

Provision the CloudWatch dashboard and write the model-swap runbook

The circuit breaker handles resilience. The CloudWatch dashboard makes it observable. Without the dashboard, the substrate is a black box and the next incident has no early warning signal.

• Create modules/circuit-breaker/dashboard.tf with the first three dashboard widgets (invocation count, latency, error rate):

resource "aws_cloudwatch_dashboard" "kai_substrate" {
  dashboard_name = "kai-substrate-operations"
  dashboard_body = jsonencode({
    widgets = [
      {
        type   = "metric"
        x = 0, y = 0, width = 8, height = 6
        properties = {
          title   = "Per-Segment Invocation Count"
          metrics = [["AWS/Lambda", "Invocations", "FunctionName", "kai-router"]]
          period  = 60
          stat    = "Sum"
        }
      },
      {
        type   = "metric"
        x = 8, y = 0, width = 8, height = 6
        properties = {
          title   = "Per-Segment p95 Latency"
          metrics = [["AWS/ApiGateway", "Latency", "ApiId", var.api_id]]
          period  = 60
          stat    = "p95"
        }
      },

• Continue the dashboard definition with the error rate and token usage widgets:

      {
        type   = "metric"
        x = 16, y = 0, width = 8, height = 6
        properties = {
          title   = "Per-Segment Error Rate"
          metrics = [["AWS/ApiGateway", "5xx", "ApiId", var.api_id]]
          period  = 60
          stat    = "Sum"
        }
      },
      {
        type   = "metric"
        x = 0, y = 6, width = 8, height = 6
        properties = {
          title   = "Token Usage Per Segment"
          metrics = [
            ["KaiSubstrate", "InputTokens", "Segment", "hospitality"],
            ["KaiSubstrate", "OutputTokens", "Segment", "hospitality"]
          ]
          period = 60
          stat   = "Sum"
        }
      },

• Add the final two widgets (circuit breaker state and cost reference) and close the resource:

      {
        type   = "metric"
        x = 8, y = 6, width = 8, height = 6
        properties = {
          title   = "Circuit Breaker State"
          metrics = [["AWS/DynamoDB", "ItemCount", "TableName", "kai-circuit-status"]]
          period  = 60
          stat    = "Maximum"
        }
      },
      {
        type   = "text"
        x = 16, y = 6, width = 8, height = 6
        properties = {
          markdown = "## Cost Per Segment\nReview daily in Cost Explorer via inference profile tags."
        }
      }
    ]
  })
}

What do the six widgets show?

Invocation count, p95 latency, and error rate come from API Gateway and Lambda native metrics. Token usage comes from the custom KaiSubstrate namespace the Lambda publishes to. Circuit breaker state shows DynamoDB item count (0 means all circuits closed). Cost-per-segment references Cost Explorer via the inference profile tags activated earlier.

• Apply the dashboard by running:

terraform apply -auto-approve

• Verify the dashboard is live by running:

aws cloudwatch list-dashboards \
  --query "DashboardEntries[?DashboardName=='kai-substrate-operations'].DashboardName" \
  --output text

You should see kai-substrate-operations printed. Open the CloudWatch Dashboards page in the AWS Console to confirm the six widgets render with data from your test invocations.

• Write the model-swap runbook at docs/runbooks/model-swap.md:

# Model Swap Runbook

## Purpose
Swap the active model for any segment with zero application code change in under 60 seconds.

## Steps
1. Navigate to AWS AppConfig console > Kai Concierge Cloud > production > [segment] profile.
2. Create a new hosted configuration version with the updated modelId or inferenceProfileArn.
3. Deploy using the instant deployment strategy.
4. Invoke the segment endpoint and confirm the response body shows the new modelId.
5. Verify on the CloudWatch dashboard that the segment is responding normally.
6. No Lambda redeploy, no CI/CD trigger, no application code change required.

• Save the runbook.

Before you verify the full end-to-end path, predict: if you invoke the hospitality segment right now, will the circuit be open or closed after the earlier throttle test?

• If more than 30 seconds have passed since the throttle test, the DynamoDB TTL has expired the circuit-open record. Invoke again and confirm the primary model responds:

curl -X POST $(terraform output -raw api_endpoint)/hospitality/invoke \
  -d '{"prompt":"Hello"}'

You should see "fallback": false in the response, confirming the circuit healed. Open the CloudWatch dashboard in the AWS Console. The circuit breaker state widget should show 0 (all circuits closed) and the invocation count widget should reflect your test calls.

Dashboard showing no data?

CloudWatch metrics can take 1-2 minutes to populate after the first data point. Refresh the dashboard after waiting. If the custom KaiSubstrate namespace shows no data, confirm the Lambda has cloudwatch:PutMetricData permission and that the metric publish code runs without error in CloudWatch Logs.

Help me debug my CloudWatch dashboard metrics

✔️ Awesome, I've got everything!

Great. Confirm you have saved all files in modules/circuit-breaker/, updated lambda/router/handler.py, and both runbooks exist in docs/runbooks/.

ⓧ I'd like to double check the full code

Scroll up and confirm each of these chunks is present in order:

• modules/circuit-breaker/main.tf: DynamoDB table, SFN IAM role + policy, state machine definition (GetCircuitStatus > CheckCircuit > InvokePrimary > OpenCircuit > FallbackInvoke > Success), API Gateway SFN role, variables, and output.
• modules/circuit-breaker/dashboard.tf: Six widgets (invocation count, p95 latency, error rate, token usage, circuit state, cost text).
• modules/routing/main.tf: lambda_arn and api_id outputs at the bottom.
• kai-substrate/main.tf: module "circuit_breaker" block referencing module.routing.lambda_arn and module.routing.api_id.
• lambda/router/handler.py: use_fallback = event.get('useFallback', False) with conditional model selection and 'fallback': use_fallback in the response.
• docs/adr/adr-005-express-sfn.md, docs/runbooks/circuit-breaker-drill.md, docs/runbooks/model-swap.md.

The substrate now degrades gracefully. A throttle on one segment triggers fallback within two seconds without touching any other segment. The CloudWatch dashboard gives the operations team real-time visibility into circuit state and per-segment health. Next up, you will run the four validation bars, write the plain-language guide, and assemble the architecture-review deck for the board.

Validate the Substrate and Assemble Documentation

The substrate infrastructure is fully deployed. Four segments route through the circuit breaker to Bedrock via inference profiles, with the CloudWatch dashboard live, all five ADRs written, and both runbooks ready.

But deployed infrastructure without validation evidence is an untested assumption. A substrate without documentation is tribal knowledge that leaves the next architect stranded. This step runs the four proof-point bars that prove the substrate works as designed, then assembles the plain-language guide, topology diagram, and architecture-review deck the board needs.

In this step, get ready to:

• Run the four validation bars (zero-code swap, resilience, compliance, latency) and confirm all pass.
• Write the plain-language guide and Mermaid topology diagram.
• Assemble the architecture-review deck and tag the repo release.

Run the zero-code-swap and resilience validation bars

The first validation bar proves the substrate's core contract: swap a model by editing config alone, live in under sixty seconds, with zero Lambda code change.

• Retrieve the AppConfig resource IDs from Terraform output by running:

cd kai-substrate
terraform output -json | python3 -c "
import sys, json
out = json.load(sys.stdin)
print('APP_ID:', out.get('appconfig_app_id', {}).get('value', 'N/A'))
print('ENV_ID:', out.get('appconfig_env_id', {}).get('value', 'N/A'))
print('HOSPITALITY_PROFILE_ID:', out.get('appconfig_hospitality_profile_id', {}).get('value', 'N/A'))
print('STRATEGY_ID:', out.get('appconfig_strategy_id', {}).get('value', 'N/A'))
print('API_ENDPOINT:', out.get('api_endpoint', {}).get('value', 'N/A'))
"

What does this command do?

This extracts the AppConfig IDs and API endpoint from Terraform state. You need these to issue the config swap and invoke the endpoint without touching any application code.

• Note the values printed. You will use them in the next commands.
• Create a JSON file with the Nova model config, then push it as a new AppConfig hosted configuration version by running:

# Write the swap config to a file
echo '{"modelId":"amazon.nova-lite-v1:0","temperature":0.5,"maxTokens":512,"fallbackModelId":"anthropic.claude-3-haiku-20240307-v1:0"}' > /tmp/nova-config.json

# Create new hosted configuration version (content is base64-encoded)
aws appconfig create-hosted-configuration-version \
  --application-id [[APPCONFIG_APP_ID="your AppConfig application ID"]] \
  --configuration-profile-id [[HOSPITALITY_PROFILE_ID="your hospitality profile ID"]] \
  --content fileb:///tmp/nova-config.json \
  --content-type "application/json" \
  /tmp/version-output.json

• Deploy the new configuration version by running:

# Deploy the new version immediately
aws appconfig start-deployment \
  --application-id [[APPCONFIG_APP_ID="your AppConfig application ID"]] \
  --environment-id [[APPCONFIG_ENV_ID="your AppConfig environment ID"]] \
  --deployment-strategy-id [[STRATEGY_ID="your instant deployment strategy ID"]] \
  --configuration-profile-id [[HOSPITALITY_PROFILE_ID="your hospitality profile ID"]] \
  --configuration-version 2

Why version 2?

The original Terraform deployment created version 1 (Claude Haiku). The version you just created is version 2 (Nova Lite). Deploying version 2 swaps the model with no Lambda code change.

Before you invoke the endpoint, what model ID do you expect in the response?

• Invoke the hospitality endpoint to confirm the swap by running:

# Invoke with the new config - should show Nova model ID
curl -s -X POST [[API_ENDPOINT="your API Gateway endpoint"]]/hospitality/invoke \
  -d '{"prompt":"What time is checkout?"}' | python3 -m json.tool

The response body shows amazon.nova-lite-v1:0 in the modelId field. Zero Lambda code was touched. The swap happened in config alone.

Still seeing Claude Haiku in the response?

The AppConfig Lambda extension caches configuration for up to 45 seconds. Wait one minute and invoke again. If the model ID still shows Claude, confirm the deployment completed successfully.

Run aws appconfig list-deployments --application-id <app-id> --environment-id <env-id> to check deployment state.

help me debug why my AppConfig model swap is not taking effect

Validation bar 1 passed: zero-code swap live in under 60 seconds. Now run the resilience bar.

• Trigger the circuit breaker by temporarily setting the retail segment to an invalid model ID, then invoking. Run the following to seed a DynamoDB circuit-open record directly (simulating three failed retries):

# Seed circuit-open state for retail (simulates 3 exhausted retries)
TTL=$(python3 -c "import time; print(int(time.time()) + 30)")
aws dynamodb put-item \
  --table-name kai-circuit-status \
  --item '{"segment_name":{"S":"retail"},"expire_timestamp":{"N":"'$TTL'"}}'

• Now invoke the retail segment and measure the response time:

# Time the fallback invocation - should be < 2 seconds
time curl -s -X POST [[API_ENDPOINT="your API Gateway endpoint"]]/retail/invoke \
  -d '{"prompt":"What are your store hours?"}' | python3 -m json.tool

The response shows "fallback": true and the real time is under 2 seconds. The circuit breaker routed to the fallback model without retrying the primary. Blast radius stayed on the retail segment only.

What just happened?

The Step Functions Express workflow checked DynamoDB, found the circuit-open record for retail, and immediately routed to FallbackInvoke. No retries on the primary model. The 30-second TTL means the circuit will close automatically after the record expires.

Seeing a timeout or > 2 seconds?

Check that the DynamoDB item was written with a future TTL. Run aws dynamodb get-item --table-name kai-circuit-status --key '{"segment_name":{"S":"retail"}}' to verify the record exists.

help me debug why my circuit breaker fallback is not responding within 2 seconds

Run the compliance and latency validation bars

The compliance bar proves healthcare traffic stays in HIPAA-eligible US regions. The latency bar confirms all segments meet the p95 under 3 seconds budget.

• Invoke the healthcare segment to generate a CloudTrail event, then run the audit script:

# Generate a healthcare invocation for CloudTrail
curl -s -X POST [[API_ENDPOINT="your API Gateway endpoint"]]/healthcare/invoke \
  -d '{"prompt":"What is the patient discharge protocol?"}' > /dev/null

# Wait for CloudTrail to register the event (up to 5 min)
echo "Waiting 60 seconds for CloudTrail propagation..."
sleep 60

# Run the region audit script
chmod +x scripts/audit-region.sh
./scripts/audit-region.sh

You should see PASS: Zero HIPAA region violations detected. printed. The geographic inference profile restricted routing to us-east-1, us-west-2, and us-east-2.

Why wait 60 seconds?

CloudTrail events can take up to 5 minutes to appear in lookup-events. A 60-second wait is usually sufficient for the first event. If the script returns no events, wait longer and re-run.

• Run the latency bar by invoking each segment 10 times and checking API Gateway metrics:

# Invoke each segment 10 times
for SEGMENT in hospitality retail healthcare eu-sovereign; do
  echo "Invoking $SEGMENT 10 times..."
  for i in $(seq 1 10); do
    curl -s -X POST [[API_ENDPOINT="your API Gateway endpoint"]]/$SEGMENT/invoke \
      -d '{"prompt":"Hello"}' > /dev/null
  done
done
echo "Done. Check API Gateway metrics for p95 latency."

• Query the p95 latency from API Gateway metrics by running:

# Check p95 latency over the last 10 minutes
aws cloudwatch get-metric-statistics \
  --namespace AWS/ApiGateway \
  --metric-name Latency \
  --dimensions Name=ApiId,Value=[[API_ID="your API Gateway ID"]] \
  --start-time $(date -u -d '10 minutes ago' +%Y-%m-%dT%H:%M:%S) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%S) \
  --period 600 \
  --statistics p95 \
  --output table

The p95 value should be under 3000 milliseconds (3 seconds). All four validation bars now pass.

p95 latency exceeds 3 seconds?

Lambda cold starts can inflate the first invocation. Run the 10-invocation loop again (the functions are now warm) and re-check the metric. If latency remains high, check that provisioned concurrency is not needed for your invocation pattern.

help me investigate why my Bedrock substrate p95 latency exceeds the 3-second budget

Write the plain-language guide and topology diagram

The validation bars prove the substrate works. Now you need documentation artifacts so the next architect can understand, operate, and tear down the substrate without tribal knowledge.

• Create the plain-language guide by writing the first half of docs/guide.md:

# Kai Concierge Cloud Substrate - Plain-Language Guide

## What Is This?
A config-driven Bedrock routing substrate with circuit breaker, region pinning, cost attribution, and model governance for four tenant segments.

## Why Does It Exist?
Two incidents triggered this work:
1. Hard-coded model IDs caused missed model churn - teams had to redeploy applications every time a model was updated or replaced.
2. An uncontained throttle caused platform-wide degradation - one segment's model throttle cascaded to all segments.

## How to Stand It Up
```bash
cd kai-substrate
terraform init
terraform apply
```

• Add the operational sections to complete docs/guide.md (append below the content above):

## How to Swap a Model
Edit the AppConfig hosted configuration version for the target segment (change modelId or inferenceProfileArn), deploy using the instant strategy. See docs/runbooks/model-swap.md. No application code change required. Live in under 60 seconds.

## How the Circuit Breaker Works
Three failures on the primary model open the circuit (DynamoDB record written with 30s TTL). All subsequent requests route to the fallback model. After 30 seconds the TTL expires, the circuit closes, and the primary model is tried again.

## How Region Routing Works
The healthcare segment uses a geographic cross-region inference profile (us.anthropic.claude-3-haiku-20240307-v1:0) that restricts routing to us-east-1, us-west-2, and us-east-2. An IAM deny blocks invocation of global inference profiles for the healthcare Lambda role.

## How Cost Attribution Works
Application inference profiles carry cost-allocation tags (Segment, CostCenter, Environment). After activating tags in the Billing console, spend flows natively to Cost Explorer and CUR 2.0 by segment.

## How to Tear It Down
See the Before You Go section. Run `terraform destroy` and delete the prompt router manually if the Secret Mission was completed.

What does this guide cover?

Each section answers one operational question the next architect will ask: what is this, why does it exist, how do I stand it up, how do I swap a model, how does the circuit breaker heal, how is region pinning enforced, how does cost attribution flow, and how do I tear it all down.

• Save docs/guide.md.
• Create the topology diagram at docs/architecture-diagram.md by writing:

# Kai Substrate Topology

```mermaid
graph LR
    Client([Client]) --> APIGW[API Gateway HTTP API\nPOST /{segment}/invoke]
    APIGW --> SFN[Step Functions Express\nCircuit Breaker]
    SFN --> DDB[(DynamoDB\nkai-circuit-status\nTTL 30s)]
    SFN --> Lambda[Lambda Router\nkai-router\nPython 3.12]
    Lambda --> AppConfig[AppConfig\nSegment Config\nmodelId / inferenceProfileArn]
    Lambda --> Bedrock[Bedrock Converse API\nvia Inference Profile]
    Bedrock --> Model([Claude / Nova / Haiku])
    Lambda --> CW[CloudWatch\nKaiSubstrate Metrics]
    CW --> Dashboard[CloudWatch Dashboard]
    SFN --> DDB
```

## Side Channels
- DynamoDB: circuit breaker state (open/closed per segment, TTL 30s)
- CloudWatch: per-segment invocation count, p95 latency, error rate, token usage, circuit state

Why Mermaid?

Mermaid diagrams render in GitHub, GitLab, and most markdown previewers without any external tooling. The next architect sees a live topology diagram directly in the repo without needing a Lucidchart or Draw.io account.

• Save docs/architecture-diagram.md.
• Confirm both files exist by running:

ls docs/guide.md docs/architecture-diagram.md

Both files should be listed. The guide and topology diagram are now part of the documentation set.

Assemble the architecture-review deck and tag the release

The architecture-review deck is the single artifact the board reads. It summarizes the incidents, the decisions, the proof, and the cost posture in one document.

• Create docs/review-deck.md with the incident context, ADR summaries, and topology reference:

# Kai Concierge Cloud Substrate - Architecture Review Deck

## (a) Incidents That Triggered This Work
1. Hard-coded model IDs: teams redeployed applications on every model update, causing 2-4 hour deployment windows during model churn.
2. Uncontained throttle blast radius: one segment's ThrottlingException cascaded to all segments, causing platform-wide degradation for 8 minutes.

## (b) Five ADRs Summarized
- ADR-001: Converse API as model-agnostic contract - one integration across all message-capable models.
- ADR-002: AppConfig over SSM - deployment strategies, rollback-on-alarm, Lambda extension cached reads.
- ADR-003: Application inference profiles - native cost attribution to Cost Explorer, no log reconciliation.
- ADR-004: Warm SageMaker endpoint with registry rollback - known hourly cost, version-pinned, rollback is a registry change.
- ADR-005: Step Functions Express - sub-second synchronous orchestration for circuit breaker.

## (c) Topology Diagram
See docs/architecture-diagram.md

• Append the validation evidence and cost posture sections to docs/review-deck.md:

## (d) Four Validation Bars
1. Zero-code swap: PASS - hospitality swapped Claude to Nova via AppConfig in < 60s, no Lambda redeploy.
2. Resilience: PASS - retail throttle triggered fallback within 2s, blast radius contained to retail segment.
3. Compliance: PASS - healthcare audit script returned zero non-HIPAA-region InvokeModel events.
4. Latency: PASS - all segments p95 < 3s across 10 invocations each.

## (e) Cost Posture
- Per-segment cost attribution via inference profile tags flowing to Cost Explorer.
- SageMaker endpoint: ~$0.07/hr (~$1.68/day) - only meaningful idle cost.
- Bedrock: pay-per-token (Claude Haiku ~$0.25/M input, Nova Lite ~$0.06/M input).
- Step Functions Express: ~$0.00 at lab scale.
- Everything else: near-zero at lab scale.

• Append the carrying cost, prevention list, and lessons learned to complete docs/review-deck.md:

## (f) Carrying Cost
- Bedrock on-demand: ~$0.25/M input tokens (Haiku), ~$0.06/M (Nova Lite)
- SageMaker endpoint: ~$0.07/hr on ml.m5.large
- Intelligent Prompt Routing (if Secret Mission completed): $1/1,000 requests
- CloudWatch dashboard: $3/month
- All other resources: near-zero at lab scale

## (g) What the Substrate Prevents
- Hard-coded model IDs causing missed model churn
- Missing region constraints allowing healthcare traffic outside HIPAA-eligible regions
- No graceful degradation causing platform-wide throttle cascades
- Unattributed cost making per-segment spend invisible
- No version governance on fine-tuned models

## (h) Lessons Learned and Open Items
- Intelligent Prompt Routing is not yet a Terraform resource (provider v6.54.0); manage via AWS CLI.
- Cost-allocation tag activation takes up to 24 hours and is not retroactive.
- SageMaker endpoint is the only resource with meaningful idle cost; pause it when not testing.
- Next review cycle: evaluate EU-sovereign geographic profile for GDPR compliance.

What makes this deck effective for the board?

It answers the three questions any review board asks: what incidents triggered this investment, what evidence proves it works, and what does it cost to carry. Every section is backed by the validation bars you just ran and the cost figures from the Verified Facts.

• Save docs/review-deck.md.
• Tag the repo release by running:

git add .
git commit -m "Complete substrate validation and documentation"
git tag -a v1.0.0 -m "Kai substrate v1.0.0 - all validation bars passed"

• Confirm the full documentation set exists by running:

ls docs/ docs/adr/ docs/runbooks/ scripts/

Before you check the output, predict what files you expect in each directory.

You should see: docs/ contains guide.md, architecture-diagram.md, review-deck.md, adr/, and runbooks/. The docs/adr/ directory contains five ADRs (001-005). The docs/runbooks/ directory contains circuit-breaker-drill.md and model-swap.md. The scripts/ directory contains audit-region.sh.

Missing any documentation files?

If any files are missing from docs/, check that you saved each file after writing it. Run find docs/ -name '*.md' | sort to list all markdown files recursively.

help me verify that all Kai substrate documentation artifacts are present

Secret mission

Add Intelligent Prompt Routing for Cost Optimization

The hospitality segment handles high-volume simple queries alongside complex ones. Every prompt currently routes to the same model at the same per-token cost. Can you add Intelligent Prompt Routing so simple prompts go to a cheaper model within the same family, cutting costs by up to 30%?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project provisions real AWS resources with varying idle costs.

Cost warning

The SageMaker real-time endpoint charges approximately $0.07/hr (~$1.68/day) while running, even if you send no traffic. The CloudWatch dashboard costs $3/month.

All other resources (Lambda, API Gateway, DynamoDB, AppConfig, Step Functions Express, inference profiles) are near-zero at lab scale. If you are done experimenting, choose Pause or Delete to stop the SageMaker cost immediately.

Resources you used:

• Terraform-managed infrastructure: Lambda function, API Gateway HTTP API, DynamoDB table (kai-circuit-status), AppConfig application and profiles, Step Functions Express state machine, four application inference profiles, SageMaker endpoint and Model Registry, CloudWatch dashboard.
• Bedrock prompt router (kai-hospitality-router), created via AWS CLI (not managed by Terraform).
• Cost-allocation tags (Segment, CostCenter) activated in the Billing console.
• Local project directory: kai-substrate/ on your Desktop.

Keep everything running

No action needed. Choose this if you are still actively testing the substrate, running drills, or iterating on the Secret Mission.

• The SageMaker endpoint continues to charge ~$0.07/hr (~$1.68/day).
• The CloudWatch dashboard charges $3/month.
• All other resources are pay-per-invocation and cost nothing while idle.
• The prompt router is charged only when invoked ($1/1,000 requests).

Pause - I'll come back to this later

Shut down the SageMaker endpoint to stop the hourly charge. Everything else remains intact for when you return.

• Delete only the SageMaker endpoint by running:
• Confirm the endpoint is gone by running:

cd ~/Desktop/kai-substrate
terraform destroy -target=aws_sagemaker_endpoint.kai_voice

Terraform will ask you to confirm. Type yes and press Enter.

aws sagemaker list-endpoints --query "Endpoints[?EndpointName=='kai-voice']"

You should see an empty list, confirming the endpoint is deleted.

• The Model Registry, model packages, Lambda, API Gateway, DynamoDB, AppConfig, Step Functions, inference profiles, prompt router, and CloudWatch dashboard all remain.
• To resume later, re-run terraform apply to recreate the endpoint.

Delete - I don't want to use this again

Remove all project resources and return your AWS account to near-zero carrying cost.

• Navigate to your project directory and destroy all Terraform-managed resources by running:

cd ~/Desktop/kai-substrate
terraform destroy

Terraform will ask you to confirm. Type yes and press Enter. This removes the Lambda, API Gateway, DynamoDB table, AppConfig resources, Step Functions state machine, inference profiles, SageMaker endpoint, SageMaker Model Registry, and CloudWatch dashboard.

• Delete the prompt router (not managed by Terraform) by running:

aws bedrock delete-prompt-router --prompt-router-arn [[PROMPT_ROUTER_ARN="your prompt router ARN from the create-prompt-router output"]]

• Confirm all application inference profiles are gone by running:

aws bedrock list-inference-profiles --type-equals APPLICATION

You should see an empty list.

• Deactivate cost-allocation tags in the Billing console.
• Open the AWS Billing console.
• Navigate to Cost Allocation Tags.
• Select the Segment and CostCenter tags, then click Deactivate.

• Delete the local project directory. On macOS or Linux, run:

cd ~/Desktop
rm -rf kai-substrate

On Windows, run:

cd ~\Desktop
rmdir /s /q kai-substrate

• Confirm the directory is removed by running ls ~/Desktop (macOS/Linux) or dir ~\Desktop (Windows). You should no longer see kai-substrate in the output.

That's a Wrap!

That's a Wrap!

Nice work. You just built a production-grade Terraform substrate that gives Kai Concierge Cloud one door to any model on Bedrock, with config-driven routing, circuit-breaker resilience, HIPAA region pinning, per-segment cost attribution, model version governance, and a documentation suite ready for board review.

You've learned how to:

• Build a config-driven routing substrate with the Converse API as the model-agnostic contract, proving the zero-code-change contract by swapping Claude to Nova via a single AppConfig edit in under sixty seconds.
• Implement a Step Functions Express circuit breaker with DynamoDB state that contains blast radius to one segment and fails over within two seconds, alongside HIPAA-eligible region pinning validated at zero violations using geographic inference profiles and IAM deny.
• Attribute per-segment cost via application inference profiles flowing to Cost Explorer, stand up SageMaker Model Registry with rollback versions on a warm endpoint, and deliver five ADRs, two runbooks, a topology diagram, and an architecture-review deck.
• Secret Mission: Added Intelligent Prompt Routing to the hospitality segment, routing simple prompts to a cheaper model and complex prompts to the capable model, with per-request traceability on the dashboard and named cost ($1/1,000 requests, up to 30% token savings).

Ready to quiz yourself?