Serverless URL Health Monitor in Go

Introduction

30 Second Summary

Every time a website goes down, someone somewhere is scrambling to figure out what broke and when. Automated health monitoring catches failures the moment they happen, before your users do.

In this project, you will build a serverless URL health monitoring system in Go, deployed on AWS with distributed tracing, asynchronous event processing, and a CI/CD pipeline that ships everything on every push to main.

What You'll Build

A multi-Lambda serverless application on ARM64 that monitors URL health through an API Gateway REST API, SQS-based async processing, Step Functions orchestration, and OpenTelemetry distributed tracing with X-Ray.

By the end of this project, you'll have:

• Three Go Lambda functions on ARM64 with a REST API, SQS processor, and Step Functions orchestrator deployed via AWS SAM.
• A complete CI/CD pipeline in GitHub Actions using OIDC for keyless AWS deployments with lint, test, build, and deploy stages.
• Production-grade observability with OpenTelemetry traces propagated across SQS message boundaries, structured JSON logs, and CloudWatch alarms.
• Secret Mission: Add a CloudWatch Dashboard with custom metrics showing health check latency percentiles, error rates, and SQS queue depth.

Are there any prerequisites?

You should be comfortable with basic Go (structs, interfaces, error handling, and context.Context). You also need an AWS account with free-tier credits available.

No prior AWS serverless experience is required. The project walks through every resource and deployment step from scratch.

Set Up Your Tools and Repository

Before you can build and deploy Lambda functions, you need three core tools installed on your machine.

Go compiles your Lambda code into native binaries. The AWS CLI connects your terminal to your AWS account. The AWS SAM CLI packages and deploys your serverless infrastructure. You'll also set up a GitHub repository where your CI/CD pipeline will live.

In this step, get ready to:

• Install and verify Go, AWS CLI, and AWS SAM CLI.
• Configure your AWS credentials and verify access.
• Create a public GitHub repository and clone it locally.

Install and verify Go, AWS CLI, and SAM CLI

Each tool has a version-check command that confirms it's installed and accessible from your terminal. Run these one at a time.

• Check if Go is installed by running this command:

go version

✔️ I see version 1.22 or higher

Go is ready. Move on to the AWS CLI check below.

ⓧ I see an older version

Your installed version of Go is below 1.22. This project uses features introduced in Go 1.22, so you need to upgrade.

• Visit the official Go install page:
• go.dev/doc/install
• Download and install the latest version for your operating system.
• Close and reopen your terminal, then run go version again to confirm the update.

macOS may shadow your new install

If go version still shows the old version after installing, the system PATH may point to an older binary. Run which go to check which binary is being used and update your PATH in ~/.zshrc or ~/.bash_profile to point to the new install location.

ⓧ Command not found

Go is not installed on your machine yet.

• Visit the official Go install page:
• go.dev/doc/install
• Download the installer for your operating system (macOS, Windows, or Linux).
• Run the installer and follow the prompts.
• Close and reopen your terminal, then run go version to confirm installation. You should see 1.22 or higher.

• Check if the AWS CLI is installed by running this command:

aws --version

✔️ I see a version number

The AWS CLI is ready. Move on to the SAM CLI check below.

ⓧ Command not found

The AWS CLI is not installed on your machine yet.

• Visit the official AWS CLI install page:
• docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
• Follow the instructions for your operating system to install AWS CLI version 2.
• Close and reopen your terminal, then run aws --version to confirm installation.

• Check if the AWS SAM CLI is installed by running this command:

sam --version

✔️ I see a version number

The SAM CLI is ready. All three tools are installed.

ⓧ Command not found

The AWS SAM CLI is not installed on your machine yet.

• Visit the official SAM CLI install page:
• docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html
• Follow the instructions for your operating system.
• Close and reopen your terminal, then run sam --version to confirm installation.

Still having trouble with installation?

Make sure you fully close and reopen your terminal after installing each tool. New PATH entries only take effect in new terminal sessions.

help me troubleshoot tool installation for Go, AWS CLI, or SAM CLI

Configure your AWS credentials

The AWS CLI needs credentials to create resources in your AWS account. You'll provide an access key and secret key that authenticate your terminal sessions.

• Log in to the AWS Console and navigate to IAM.
• Click Users in the left sidebar.
• Select your user (or create a new user with AdministratorAccess for this project).
• Click the Security credentials tab.
• Click Create access key and choose Command Line Interface (CLI) as the use case.
• Copy the Access Key ID and Secret Access Key. Store them somewhere safe. You will not be able to see the secret key again.

Why AdministratorAccess?

This project provisions Lambda functions, DynamoDB tables, SQS queues, Step Functions, API Gateway, IAM roles, and more. Administrator access avoids permission errors during deployment.

In production, you'd use a narrowly scoped policy. For learning purposes, admin access removes friction.

• Configure the AWS CLI with your credentials by running this command:

aws configure

What does this command do?

This launches an interactive prompt that asks for four values. Enter them one at a time:

• AWS Access Key ID: paste the access key you copied.
• AWS Secret Access Key: paste the secret key you copied.
• Default region name: enter us-east-1 (or your preferred region).
• Default output format: enter json.

• Verify that your credentials work by running this command:

aws sts get-caller-identity

You should see a JSON response containing your Account, UserId, and Arn. This confirms your terminal can authenticate with AWS.

Seeing an error about invalid credentials?

Double check that you pasted the full access key and secret key without leading or trailing spaces. Run aws configure again to re-enter them.

help me debug why aws sts get-caller-identity is failing

Create and clone your GitHub repository

Your CI/CD pipeline will run on GitHub Actions. A public repository gives you unlimited free Actions minutes.

• Go to github.com and sign in (or create an account if you don't have one).
• Click the + icon in the top right corner, then click New repository.
• Enter url-health-monitor as the repository name.
• Select Public for visibility.
• Leave all other options unchecked (no README, no .gitignore, no license).
• Click Create repository.

Why public?

GitHub Actions provides unlimited free minutes for public repositories. Private repositories have a monthly cap on free CI/CD minutes. Since this is a learning project, public keeps things simple and free.

• Open your terminal.
• Move to your home directory by running this command:

cd ~

• Clone your new repository by running this command (replace the URL with your own):

git clone [[REPO_URL="https://github.com/YOUR_USERNAME/url-health-monitor.git"]]

• Move into the cloned repository by running this command:

cd url-health-monitor

• Confirm the remote is set up correctly by running:

git remote -v

You should see origin pointing to your GitHub repository URL for both fetch and push.

Don't see the remote?

Make sure you copied the HTTPS URL from your GitHub repository page (it ends in .git). If you accidentally initialized an empty folder instead of cloning, delete it and run the git clone command again.

help me fix my git remote configuration

Your tools are installed, AWS access is confirmed, and your repository is ready. Next up, you'll initialize the Go module and define your entire serverless infrastructure in a SAM template.

Initialize the Project and SAM Template

Your tools are installed and your GitHub repo is cloned locally. Now you need the actual project structure.

This step lays the foundation: a Go module, a clean directory layout for three Lambda functions, and a SAM template that defines every AWS resource the system needs. By the end, you will have the entire infrastructure declared as code, ready to build and deploy.

In this step, get ready to:

• Initialize a Go module and create the project directory structure.
• Write the SAM template defining Lambda functions, API Gateway, DynamoDB, SQS, Step Functions, and EventBridge.
• Add a Makefile that cross-compiles all Lambda binaries for ARM64.

Initialize the Go module and directory structure

Go modules track dependencies for your project. The directory layout separates each Lambda handler into its own cmd/ package and keeps shared logic in internal/.

• Open your terminal and navigate to your cloned repository by running this command:

cd ~/url-health-monitor

• Initialize the Go module by running this command:

go mod init github.com/[[GITHUB_USERNAME="your-github-username"]]/url-health-monitor

What does this command do?

This creates a go.mod file that declares your module path. Every Go file in this project will belong to this module, and Go uses it to resolve imports and download dependencies.

• Create the directory structure for all three Lambda handlers and the shared packages by running these commands:

mkdir -p cmd/api cmd/processor cmd/orchestrator
mkdir -p internal/store internal/checker internal/tracing

Why this layout?

Each folder under cmd/ becomes a separate Lambda binary. The internal/ folder holds packages that are shared across handlers but cannot be imported by external projects.

This is the standard Go project layout for multi-binary repositories.

• Confirm the directories exist by running:

ls cmd/ internal/

You should see api, processor, and orchestrator under cmd/, and store, checker, and tracing under internal/.

Don't see the folders?

Make sure you ran mkdir -p (with the -p flag). Without it, mkdir fails if parent directories don't exist.

Also verify you are inside ~/url-health-monitor/ by running pwd.

help me debug my directory structure

Write the SAM template

The SAM template is the Infrastructure as Code file that declares every AWS resource your application needs. AWS SAM transforms this into a full CloudFormation stack at deploy time.

• Create a file called template.yaml in the root of your url-health-monitor folder, then paste the following code into it:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Serverless URL Health Monitor - ARM64 Lambda functions with OTel tracing

Globals:
  Function:
    Runtime: provided.al2023
    Architectures:
      - arm64
    Timeout: 30
    MemorySize: 256
    Tracing: Active
    Environment:
      Variables:
        TABLE_NAME: !Ref UrlHealthTable
        QUEUE_URL: !Ref HealthCheckQueue

Resources:
  # --- API Gateway ---
  HealthApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: prod
      TracingEnabled: true

  # --- API Lambda ---
  APIFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: cmd/api/
      Handler: bootstrap
      Events:
        PostUrl:
          Type: Api
          Properties:
            RestApiId: !Ref HealthApi
            Path: /urls
            Method: POST
        GetUrls:
          Type: Api
          Properties:
            RestApiId: !Ref HealthApi
            Path: /urls
            Method: GET
        GetUrlStatus:
          Type: Api
          Properties:
            RestApiId: !Ref HealthApi
            Path: /urls/{id}/status
            Method: GET
        DeleteUrl:
          Type: Api
          Properties:
            RestApiId: !Ref HealthApi
            Path: /urls/{id}
            Method: DELETE
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref UrlHealthTable
        - SQSSendMessagePolicy:
            QueueName: !GetAtt HealthCheckQueue.QueueName
    Metadata:
      BuildMethod: makefile

  # --- Processor Lambda (triggered by SQS) ---
  ProcessorFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: cmd/processor/
      Handler: bootstrap
      Events:
        SQSEvent:
          Type: SQS
          Properties:
            Queue: !GetAtt HealthCheckQueue.Arn
            BatchSize: 5
            Enabled: true
            FunctionResponseTypes:
              - ReportBatchItemFailures
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref UrlHealthTable
    Metadata:
      BuildMethod: makefile

  # --- Orchestrator Lambda (called by Step Functions) ---
  OrchestratorFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: cmd/orchestrator/
      Handler: bootstrap
      Policies:
        - DynamoDBReadPolicy:
            TableName: !Ref UrlHealthTable
        - SQSSendMessagePolicy:
            QueueName: !GetAtt HealthCheckQueue.QueueName
    Metadata:
      BuildMethod: makefile

  # --- DynamoDB Table ---
  UrlHealthTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: UrlHealthTable
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: id
          AttributeType: S
      KeySchema:
        - AttributeName: id
          KeyType: HASH

  # --- SQS Queue with Dead Letter Queue ---
  HealthCheckDLQ:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: HealthCheckDLQ
      MessageRetentionPeriod: 1209600

  HealthCheckQueue:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: HealthCheckQueue
      VisibilityTimeout: 60
      RedrivePolicy:
        deadLetterTargetArn: !GetAtt HealthCheckDLQ.Arn
        maxReceiveCount: 3

  # --- Step Functions State Machine ---
  HealthCheckStateMachine:
    Type: AWS::Serverless::StateMachine
    Properties:
      DefinitionUri: statemachine/health_check.asl.json
      DefinitionSubstitutions:
        OrchestratorFunctionArn: !GetAtt OrchestratorFunction.Arn
      Policies:
        - LambdaInvokePolicy:
            FunctionName: !Ref OrchestratorFunction
      Events:
        ScheduleEvent:
          Type: Schedule
          Properties:
            Schedule: rate(5 minutes)
            Enabled: true

Outputs:
  ApiEndpoint:
    Description: API Gateway endpoint URL
    Value: !Sub https://${HealthApi}.execute-api.${AWS::Region}.amazonaws.com/prod/

What does this template define?

This single file declares your entire infrastructure:

• Three Lambda functions on arm64 (Graviton2) with X-Ray tracing enabled via Tracing: Active.
• An API Gateway REST API with four routes (POST, GET, GET status, DELETE).
• A DynamoDB table (UrlHealthTable) with partition key id using on-demand billing.
• An SQS queue (HealthCheckQueue) with a Dead Letter Queue (HealthCheckDLQ) after 3 failed processing attempts.
• A Step Functions state machine triggered by EventBridge every 5 minutes.

Why ARM64?

ARM64 (Graviton2) Lambda functions deliver up to 34% better price-performance compared to x86_64. The Globals section sets Architectures: [arm64] so every function inherits this configuration automatically.

• The Step Functions state machine references an ASL definition file. Create the directory and a placeholder file by running:

mkdir -p statemachine
touch statemachine/health_check.asl.json

• Open statemachine/health_check.asl.json and paste the following placeholder definition:

{
  "Comment": "Health Check Orchestrator - fans out URL checks via SQS",
  "StartAt": "ListUrls",
  "States": {
    "ListUrls": {
      "Type": "Task",
      "Resource": "${OrchestratorFunctionArn}",
      "Retry": [
        {
          "ErrorEquals": ["States.TaskFailed"],
          "IntervalSeconds": 2,
          "MaxAttempts": 3,
          "BackoffRate": 2.0
        }
      ],
      "End": true
    }
  }
}

What is this ASL file?

Amazon States Language (ASL) defines the workflow steps for your state machine. The ${OrchestratorFunctionArn} placeholder gets replaced at deploy time by SAM's DefinitionSubstitutions with the actual Lambda ARN.

You will expand this state machine in Step 7 to add parallel fan-out and error catchers.

• Save template.yaml and statemachine/health_check.asl.json.
• Validate the SAM template by running:

sam validate

You should see a message confirming the template is valid.

Seeing a validation error?

Check that the YAML indentation uses spaces (not tabs). YAML is sensitive to whitespace.

Verify the Transform: AWS::Serverless-2016-10-31 line is present at the top of template.yaml.

help me fix my SAM template validation error

Add the Makefile for ARM64 cross-compilation

SAM uses BuildMethod: makefile in each function's metadata to invoke custom build targets. Your Makefile handles cross-compiling Go binaries for the ARM64 Lambda runtime.

• Create a file called Makefile in the root of your url-health-monitor folder, then paste the following:

.PHONY: build build-APIFunction build-ProcessorFunction build-OrchestratorFunction

# Build all Lambda functions
build: build-APIFunction build-ProcessorFunction build-OrchestratorFunction

# Build the API Lambda binary for ARM64
build-APIFunction:
	CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -o $(ARTIFACTS_DIR)/bootstrap ./cmd/api

# Build the Processor Lambda binary for ARM64
build-ProcessorFunction:
	CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -o $(ARTIFACTS_DIR)/bootstrap ./cmd/processor

# Build the Orchestrator Lambda binary for ARM64
build-OrchestratorFunction:
	CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -o $(ARTIFACTS_DIR)/bootstrap ./cmd/orchestrator

What does this Makefile do?

Each build-<FunctionName> target matches the logical ID of the function in the SAM template. When you run sam build, SAM invokes the matching Make target for each function.

The CGO_ENABLED=0 GOOS=linux GOARCH=arm64 flags produce a statically linked Linux binary for ARM64 processors. The output binary must be named bootstrap for the provided.al2023 runtime. $(ARTIFACTS_DIR) is an environment variable SAM sets to the build output directory.

Tabs are mandatory in Makefiles

Makefiles require literal tab characters for indentation (not spaces). If your editor converts tabs to spaces, the build will fail with a "missing separator" error. Most editors let you set tab mode per file type.

• Save the Makefile.
• Verify the top-level build target is recognized by running:

make -n build

You should see the three go build commands printed (dry-run) without any errors.

Seeing "missing separator" or "No rule to make target"?

"Missing separator" means spaces are used instead of tabs for indentation. Open the Makefile and convert indentation to tab characters.

"No rule to make target" means the file is not named exactly Makefile (capital M, no extension) or you are not in the project root directory.

help me fix my Makefile error

✔️ Awesome, I've got everything!

Great. Double check you have saved all three files: go.mod, template.yaml, and Makefile.

ⓧ I'd like to double check the full code

Here are the three key files you should have at this point.

go.mod:

module github.com/YOUR_USERNAME/url-health-monitor

go 1.22

template.yaml:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Serverless URL Health Monitor - ARM64 Lambda functions with OTel tracing

Globals:
  Function:
    Runtime: provided.al2023
    Architectures:
      - arm64
    Timeout: 30
    MemorySize: 256
    Tracing: Active
    Environment:
      Variables:
        TABLE_NAME: !Ref UrlHealthTable
        QUEUE_URL: !Ref HealthCheckQueue

Resources:
  # --- API Gateway ---
  HealthApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: prod
      TracingEnabled: true

  # --- API Lambda ---
  APIFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: cmd/api/
      Handler: bootstrap
      Events:
        PostUrl:
          Type: Api
          Properties:
            RestApiId: !Ref HealthApi
            Path: /urls
            Method: POST
        GetUrls:
          Type: Api
          Properties:
            RestApiId: !Ref HealthApi
            Path: /urls
            Method: GET
        GetUrlStatus:
          Type: Api
          Properties:
            RestApiId: !Ref HealthApi
            Path: /urls/{id}/status
            Method: GET
        DeleteUrl:
          Type: Api
          Properties:
            RestApiId: !Ref HealthApi
            Path: /urls/{id}
            Method: DELETE
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref UrlHealthTable
        - SQSSendMessagePolicy:
            QueueName: !GetAtt HealthCheckQueue.QueueName
    Metadata:
      BuildMethod: makefile

  # --- Processor Lambda (triggered by SQS) ---
  ProcessorFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: cmd/processor/
      Handler: bootstrap
      Events:
        SQSEvent:
          Type: SQS
          Properties:
            Queue: !GetAtt HealthCheckQueue.Arn
            BatchSize: 5
            Enabled: true
            FunctionResponseTypes:
              - ReportBatchItemFailures
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref UrlHealthTable
    Metadata:
      BuildMethod: makefile

  # --- Orchestrator Lambda (called by Step Functions) ---
  OrchestratorFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: cmd/orchestrator/
      Handler: bootstrap
      Policies:
        - DynamoDBReadPolicy:
            TableName: !Ref UrlHealthTable
        - SQSSendMessagePolicy:
            QueueName: !GetAtt HealthCheckQueue.QueueName
    Metadata:
      BuildMethod: makefile

  # --- DynamoDB Table ---
  UrlHealthTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: UrlHealthTable
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: id
          AttributeType: S
      KeySchema:
        - AttributeName: id
          KeyType: HASH

  # --- SQS Queue with Dead Letter Queue ---
  HealthCheckDLQ:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: HealthCheckDLQ
      MessageRetentionPeriod: 1209600

  HealthCheckQueue:
    Type: AWS::SQS::Queue
    Properties:
      QueueName: HealthCheckQueue
      VisibilityTimeout: 60
      RedrivePolicy:
        deadLetterTargetArn: !GetAtt HealthCheckDLQ.Arn
        maxReceiveCount: 3

  # --- Step Functions State Machine ---
  HealthCheckStateMachine:
    Type: AWS::Serverless::StateMachine
    Properties:
      DefinitionUri: statemachine/health_check.asl.json
      DefinitionSubstitutions:
        OrchestratorFunctionArn: !GetAtt OrchestratorFunction.Arn
      Policies:
        - LambdaInvokePolicy:
            FunctionName: !Ref OrchestratorFunction
      Events:
        ScheduleEvent:
          Type: Schedule
          Properties:
            Schedule: rate(5 minutes)
            Enabled: true

Outputs:
  ApiEndpoint:
    Description: API Gateway endpoint URL
    Value: !Sub https://${HealthApi}.execute-api.${AWS::Region}.amazonaws.com/prod/

Makefile:

.PHONY: build build-APIFunction build-ProcessorFunction build-OrchestratorFunction

# Build all Lambda functions
build: build-APIFunction build-ProcessorFunction build-OrchestratorFunction

# Build the API Lambda binary for ARM64
build-APIFunction:
	CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -o $(ARTIFACTS_DIR)/bootstrap ./cmd/api

# Build the Processor Lambda binary for ARM64
build-ProcessorFunction:
	CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -o $(ARTIFACTS_DIR)/bootstrap ./cmd/processor

# Build the Orchestrator Lambda binary for ARM64
build-OrchestratorFunction:
	CGO_ENABLED=0 GOOS=linux GOARCH=arm64 go build -o $(ARTIFACTS_DIR)/bootstrap ./cmd/orchestrator

Your infrastructure is fully declared as code. Next up, you will build the DynamoDB store layer with proper error handling so your Lambda functions can persist and retrieve data.

Build the DynamoDB Store

Your SAM template defines a DynamoDB table called UrlHealthTable. Now you need Go code that talks to it.

In this step, you will build the data layer for the health monitor. This means defining a clean interface, implementing it with the AWS SDK for Go v2, and adding production-grade error handling with automatic retries for transient failures.

In this step, get ready to:

• Define a Store interface and data types for URL health records.
• Implement custom error types for idiomatic Go error handling.
• Build the DynamoDB store implementation with retry logic and exponential backoff.

Define the Store interface and data types

Clean architecture separates the contract (what operations exist) from the implementation (how they work). You will define an interface first so that later code depends on the contract, not on DynamoDB directly.

• Install the AWS SDK for Go v2 packages you need by running these commands:

go get github.com/aws/aws-sdk-go-v2
go get github.com/aws/aws-sdk-go-v2/config
go get github.com/aws/aws-sdk-go-v2/service/dynamodb
go get github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue

What do these packages do?

• aws-sdk-go-v2 is the core SDK module with shared types and configuration.
• config loads AWS credentials and region from the environment automatically.
• service/dynamodb provides the DynamoDB client and API input/output types.
• feature/dynamodb/attributevalue marshals Go structs to and from DynamoDB's attribute format.

Seeing a network error?

Make sure you have internet access. The go get command downloads modules from the Go module proxy.

help me debug a go get network error

• Create the file internal/store/store.go and add the following code:

package store

import (
	"context"
	"time"
)

// HealthStatus represents the health state of a monitored URL
type HealthStatus string

const (
	StatusHealthy   HealthStatus = "HEALTHY"
	StatusUnhealthy HealthStatus = "UNHEALTHY"
	StatusUnknown   HealthStatus = "UNKNOWN"
)

// URLRecord holds the metadata and current status for a monitored URL
type URLRecord struct {
	ID          string    `dynamodbav:"id"`
	URL         string    `dynamodbav:"url"`
	CreatedAt   time.Time `dynamodbav:"created_at"`
	LastChecked time.Time `dynamodbav:"last_checked"`
	Status      string    `dynamodbav:"status"`
}

// Store defines the interface for URL persistence operations
type Store interface {
	PutURL(ctx context.Context, record URLRecord) error
	GetURL(ctx context.Context, id string) (URLRecord, error)
	ListURLs(ctx context.Context) ([]URLRecord, error)
	DeleteURL(ctx context.Context, id string) error
	UpdateStatus(ctx context.Context, id string, status HealthStatus) error
}

What does this code do?

• HealthStatus is a typed string constant. Using a named type instead of raw strings prevents typos at compile time.
• URLRecord is the struct that maps directly to a DynamoDB item. The dynamodbav struct tags tell the marshaler how to name each attribute in the table.
• The Store interface lists every operation the application needs. Any implementation (DynamoDB, an in-memory mock for tests) can satisfy this contract.

• Save internal/store/store.go.

✔️ Awesome, I've got everything!

Great. Make sure the file is saved before moving on.

ⓧ I'd like to double check the full code

package store

import (
	"context"
	"time"
)

// HealthStatus represents the health state of a monitored URL
type HealthStatus string

const (
	StatusHealthy   HealthStatus = "HEALTHY"
	StatusUnhealthy HealthStatus = "UNHEALTHY"
	StatusUnknown   HealthStatus = "UNKNOWN"
)

// URLRecord holds the metadata and current status for a monitored URL
type URLRecord struct {
	ID          string    `dynamodbav:"id"`
	URL         string    `dynamodbav:"url"`
	CreatedAt   time.Time `dynamodbav:"created_at"`
	LastChecked time.Time `dynamodbav:"last_checked"`
	Status      string    `dynamodbav:"status"`
}

// Store defines the interface for URL persistence operations
type Store interface {
	PutURL(ctx context.Context, record URLRecord) error
	GetURL(ctx context.Context, id string) (URLRecord, error)
	ListURLs(ctx context.Context) ([]URLRecord, error)
	DeleteURL(ctx context.Context, id string) error
	UpdateStatus(ctx context.Context, id string, status HealthStatus) error
}

Implement custom error types

Go's error handling relies on wrapping errors with context so callers can inspect what went wrong. Custom error types let the handler code distinguish between "not found" and "infrastructure failure" without string parsing.

• Create the file internal/store/errors.go and add the following code:

package store

import "fmt"

// NotFoundError indicates a requested URL record does not exist
type NotFoundError struct {
	ID string
}

func (e *NotFoundError) Error() string {
	return fmt.Sprintf("url record not found: %s", e.ID)
}

// StoreError wraps underlying storage failures with operation context
type StoreError struct {
	Op  string
	Err error
}

func (e *StoreError) Error() string {
	return fmt.Sprintf("store error during %s: %v", e.Op, e.Err)
}

// Unwrap returns the underlying error for errors.Is and errors.As
func (e *StoreError) Unwrap() error {
	return e.Err
}

What does this code do?

NotFoundError carries the ID that was missing. Handler code can use errors.As to check for this type and return a 404 response.

StoreError wraps any infrastructure failure (network timeout, throttling) with the operation name. The Unwrap() method lets Go's errors.Is traverse the error chain to find the root cause.

• Save internal/store/errors.go.

Seeing a compile error?

Make sure the package declaration at the top says package store (not package main). All files in internal/store/ must share the same package name.

help me fix a Go package name mismatch error

✔️ Awesome, I've got everything!

Great. Double check the file is saved before continuing.

ⓧ I'd like to double check the full code

package store

import "fmt"

// NotFoundError indicates a requested URL record does not exist
type NotFoundError struct {
	ID string
}

func (e *NotFoundError) Error() string {
	return fmt.Sprintf("url record not found: %s", e.ID)
}

// StoreError wraps underlying storage failures with operation context
type StoreError struct {
	Op  string
	Err error
}

func (e *StoreError) Error() string {
	return fmt.Sprintf("store error during %s: %v", e.Op, e.Err)
}

// Unwrap returns the underlying error for errors.Is and errors.As
func (e *StoreError) Unwrap() error {
	return e.Err
}

Add retry logic and implement the DynamoDB store

DynamoDB can return transient errors under load (throttling, internal server errors). A retry helper with exponential backoff absorbs these blips without crashing the Lambda.

• Create the file internal/store/retry.go and add the following code:

package store

import (
	"context"
	"fmt"
	"math"
	"math/rand"
	"time"
)

// withRetry executes fn with exponential backoff and jitter on failure
func withRetry(ctx context.Context, maxRetries int, fn func() error) error {
	var err error
	for attempt := 0; attempt <= maxRetries; attempt++ {
		err = fn()
		if err == nil {
			return nil
		}

		// Don't sleep after the last attempt
		if attempt == maxRetries {
			break
		}

		// Exponential backoff: 100ms, 200ms, 400ms...
		backoff := time.Duration(math.Pow(2, float64(attempt))) * 100 * time.Millisecond
		// Add random jitter to prevent thundering herd
		jitter := time.Duration(rand.Int63n(int64(backoff / 2)))
		sleep := backoff + jitter

		select {
		case <-ctx.Done():
			return fmt.Errorf("retry cancelled: %w", ctx.Err())
		case <-time.After(sleep):
		}
	}
	return err
}

What does this code do?

• The loop runs the function up to maxRetries + 1 times total. If it succeeds on any attempt, it returns immediately.
• Exponential backoff doubles the wait between each retry (100ms, 200ms, 400ms). This gives DynamoDB time to recover from throttling.
• Random jitter prevents multiple Lambda invocations from retrying at the exact same moment, which would cause another throttle spike.
• The select on ctx.Done() ensures the retry respects cancellation. If the Lambda times out, retries stop immediately.

• Save internal/store/retry.go.

✔️ Awesome, I've got everything!

Great. Save and continue to the DynamoDB implementation.

ⓧ I'd like to double check the full code

package store

import (
	"context"
	"fmt"
	"math"
	"math/rand"
	"time"
)

// withRetry executes fn with exponential backoff and jitter on failure
func withRetry(ctx context.Context, maxRetries int, fn func() error) error {
	var err error
	for attempt := 0; attempt <= maxRetries; attempt++ {
		err = fn()
		if err == nil {
			return nil
		}

		// Don't sleep after the last attempt
		if attempt == maxRetries {
			break
		}

		// Exponential backoff: 100ms, 200ms, 400ms...
		backoff := time.Duration(math.Pow(2, float64(attempt))) * 100 * time.Millisecond
		// Add random jitter to prevent thundering herd
		jitter := time.Duration(rand.Int63n(int64(backoff / 2)))
		sleep := backoff + jitter

		select {
		case <-ctx.Done():
			return fmt.Errorf("retry cancelled: %w", ctx.Err())
		case <-time.After(sleep):
		}
	}
	return err
}

Now you have the building blocks. The retry helper and error types are ready for the main implementation.

• Create the file internal/store/dynamo.go and add the struct definition, constructor, and first two operations:

package store

import (
	"context"
	"fmt"
	"time"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue"
	"github.com/aws/aws-sdk-go-v2/service/dynamodb"
	"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)

// DynamoStore implements the Store interface using AWS DynamoDB
type DynamoStore struct {
	client    *dynamodb.Client
	tableName string
}

// NewDynamoStore creates a new DynamoDB-backed store
func NewDynamoStore(client *dynamodb.Client, tableName string) *DynamoStore {
	return &DynamoStore{
		client:    client,
		tableName: tableName,
	}
}

// PutURL stores a new URL record in DynamoDB
func (s *DynamoStore) PutURL(ctx context.Context, record URLRecord) error {
	// Marshal the Go struct into DynamoDB attribute values
	item, err := attributevalue.MarshalMap(record)
	if err != nil {
		return &StoreError{Op: "PutURL", Err: fmt.Errorf("marshal failed: %w", err)}
	}

	input := &dynamodb.PutItemInput{
		TableName: aws.String(s.tableName),
		Item:      item,
	}

	// Retry transient errors with exponential backoff
	err = withRetry(ctx, 3, func() error {
		_, putErr := s.client.PutItem(ctx, input)
		return putErr
	})
	if err != nil {
		return &StoreError{Op: "PutURL", Err: fmt.Errorf("put item failed: %w", err)}
	}
	return nil
}

// GetURL retrieves a single URL record by its ID
func (s *DynamoStore) GetURL(ctx context.Context, id string) (URLRecord, error) {
	input := &dynamodb.GetItemInput{
		TableName: aws.String(s.tableName),
		Key: map[string]types.AttributeValue{
			"id": &types.AttributeValueMemberS{Value: id},
		},
	}

	var result *dynamodb.GetItemOutput
	err := withRetry(ctx, 3, func() error {
		var getErr error
		result, getErr = s.client.GetItem(ctx, input)
		return getErr
	})
	if err != nil {
		return URLRecord{}, &StoreError{Op: "GetURL", Err: fmt.Errorf("get item failed: %w", err)}
	}

	// A nil item means no record exists with that ID
	if result.Item == nil {
		return URLRecord{}, &NotFoundError{ID: id}
	}

	var record URLRecord
	if err := attributevalue.UnmarshalMap(result.Item, &record); err != nil {
		return URLRecord{}, &StoreError{Op: "GetURL", Err: fmt.Errorf("unmarshal failed: %w", err)}
	}
	return record, nil
}

What does this code do?

• DynamoStore holds the DynamoDB client and table name. The constructor accepts these as parameters for dependency injection.
• PutURL marshals a URLRecord struct into DynamoDB format, then writes it with retry protection. If marshaling fails, it returns a StoreError wrapping the cause.
• GetURL fetches a single item by primary key. If the item does not exist (result.Item == nil), it returns a NotFoundError so callers can return a proper 404.

• Still in internal/store/dynamo.go, add the remaining operations below the GetURL function:

// ListURLs returns all monitored URL records from the table
func (s *DynamoStore) ListURLs(ctx context.Context) ([]URLRecord, error) {
	input := &dynamodb.ScanInput{
		TableName: aws.String(s.tableName),
	}

	var result *dynamodb.ScanOutput
	err := withRetry(ctx, 3, func() error {
		var scanErr error
		result, scanErr = s.client.Scan(ctx, input)
		return scanErr
	})
	if err != nil {
		return nil, &StoreError{Op: "ListURLs", Err: fmt.Errorf("scan failed: %w", err)}
	}

	var records []URLRecord
	if err := attributevalue.UnmarshalListOfMaps(result.Items, &records); err != nil {
		return nil, &StoreError{Op: "ListURLs", Err: fmt.Errorf("unmarshal failed: %w", err)}
	}
	return records, nil
}

// DeleteURL removes a URL record by its ID
func (s *DynamoStore) DeleteURL(ctx context.Context, id string) error {
	input := &dynamodb.DeleteItemInput{
		TableName: aws.String(s.tableName),
		Key: map[string]types.AttributeValue{
			"id": &types.AttributeValueMemberS{Value: id},
		},
	}

	err := withRetry(ctx, 3, func() error {
		_, delErr := s.client.DeleteItem(ctx, input)
		return delErr
	})
	if err != nil {
		return &StoreError{Op: "DeleteURL", Err: fmt.Errorf("delete item failed: %w", err)}
	}
	return nil
}

// UpdateStatus updates the health status and last-checked timestamp
func (s *DynamoStore) UpdateStatus(ctx context.Context, id string, status HealthStatus) error {
	input := &dynamodb.UpdateItemInput{
		TableName: aws.String(s.tableName),
		Key: map[string]types.AttributeValue{
			"id": &types.AttributeValueMemberS{Value: id},
		},
		UpdateExpression: aws.String("SET #status = :status, #last_checked = :last_checked"),
		ExpressionAttributeNames: map[string]string{
			"#status":       "status",
			"#last_checked": "last_checked",
		},
		ExpressionAttributeValues: map[string]types.AttributeValue{
			":status":       &types.AttributeValueMemberS{Value: string(status)},
			":last_checked": &types.AttributeValueMemberS{Value: time.Now().UTC().Format(time.RFC3339)},
		},
	}

	err := withRetry(ctx, 3, func() error {
		_, updateErr := s.client.UpdateItem(ctx, input)
		return updateErr
	})
	if err != nil {
		return &StoreError{Op: "UpdateStatus", Err: fmt.Errorf("update item failed: %w", err)}
	}
	return nil
}

What does this code do?

• ListURLs performs a full table Scan. For the small number of monitored URLs in this project, a scan is fine. The results are unmarshaled into a slice of URLRecord structs.
• DeleteURL removes a single record by primary key. DynamoDB's DeleteItem is idempotent, so deleting a non-existent item does not error.
• UpdateStatus uses an UpdateExpression to change only the status and last_checked fields without overwriting the entire item. Expression attribute names (#status) avoid conflicts with DynamoDB reserved words.

• Save internal/store/dynamo.go.
• Verify everything compiles by running:

go build ./...

You should see no output if compilation succeeds. Any errors will be printed to the terminal.

Seeing import errors?

Run go mod tidy to synchronize your go.mod and go.sum files with the actual imports in your code. This resolves missing or unused module warnings.

help me fix Go import errors in my store package

✔️ Awesome, I've got everything!

Great. All four files should be saved and compiling cleanly.

ⓧ I'd like to double check the full code

package store

import (
	"context"
	"fmt"
	"time"

	"github.com/aws/aws-sdk-go-v2/aws"
	"github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue"
	"github.com/aws/aws-sdk-go-v2/service/dynamodb"
	"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)

// DynamoStore implements the Store interface using AWS DynamoDB
type DynamoStore struct {
	client    *dynamodb.Client
	tableName string
}

// NewDynamoStore creates a new DynamoDB-backed store
func NewDynamoStore(client *dynamodb.Client, tableName string) *DynamoStore {
	return &DynamoStore{
		client:    client,
		tableName: tableName,
	}
}

// PutURL stores a new URL record in DynamoDB
func (s *DynamoStore) PutURL(ctx context.Context, record URLRecord) error {
	// Marshal the Go struct into DynamoDB attribute values
	item, err := attributevalue.MarshalMap(record)
	if err != nil {
		return &StoreError{Op: "PutURL", Err: fmt.Errorf("marshal failed: %w", err)}
	}

	input := &dynamodb.PutItemInput{
		TableName: aws.String(s.tableName),
		Item:      item,
	}

	// Retry transient errors with exponential backoff
	err = withRetry(ctx, 3, func() error {
		_, putErr := s.client.PutItem(ctx, input)
		return putErr
	})
	if err != nil {
		return &StoreError{Op: "PutURL", Err: fmt.Errorf("put item failed: %w", err)}
	}
	return nil
}

// GetURL retrieves a single URL record by its ID
func (s *DynamoStore) GetURL(ctx context.Context, id string) (URLRecord, error) {
	input := &dynamodb.GetItemInput{
		TableName: aws.String(s.tableName),
		Key: map[string]types.AttributeValue{
			"id": &types.AttributeValueMemberS{Value: id},
		},
	}

	var result *dynamodb.GetItemOutput
	err := withRetry(ctx, 3, func() error {
		var getErr error
		result, getErr = s.client.GetItem(ctx, input)
		return getErr
	})
	if err != nil {
		return URLRecord{}, &StoreError{Op: "GetURL", Err: fmt.Errorf("get item failed: %w", err)}
	}

	// A nil item means no record exists with that ID
	if result.Item == nil {
		return URLRecord{}, &NotFoundError{ID: id}
	}

	var record URLRecord
	if err := attributevalue.UnmarshalMap(result.Item, &record); err != nil {
		return URLRecord{}, &StoreError{Op: "GetURL", Err: fmt.Errorf("unmarshal failed: %w", err)}
	}
	return record, nil
}

// ListURLs returns all monitored URL records from the table
func (s *DynamoStore) ListURLs(ctx context.Context) ([]URLRecord, error) {
	input := &dynamodb.ScanInput{
		TableName: aws.String(s.tableName),
	}

	var result *dynamodb.ScanOutput
	err := withRetry(ctx, 3, func() error {
		var scanErr error
		result, scanErr = s.client.Scan(ctx, input)
		return scanErr
	})
	if err != nil {
		return nil, &StoreError{Op: "ListURLs", Err: fmt.Errorf("scan failed: %w", err)}
	}

	var records []URLRecord
	if err := attributevalue.UnmarshalListOfMaps(result.Items, &records); err != nil {
		return nil, &StoreError{Op: "ListURLs", Err: fmt.Errorf("unmarshal failed: %w", err)}
	}
	return records, nil
}

// DeleteURL removes a URL record by its ID
func (s *DynamoStore) DeleteURL(ctx context.Context, id string) error {
	input := &dynamodb.DeleteItemInput{
		TableName: aws.String(s.tableName),
		Key: map[string]types.AttributeValue{
			"id": &types.AttributeValueMemberS{Value: id},
		},
	}

	err := withRetry(ctx, 3, func() error {
		_, delErr := s.client.DeleteItem(ctx, input)
		return delErr
	})
	if err != nil {
		return &StoreError{Op: "DeleteURL", Err: fmt.Errorf("delete item failed: %w", err)}
	}
	return nil
}

// UpdateStatus updates the health status and last-checked timestamp
func (s *DynamoStore) UpdateStatus(ctx context.Context, id string, status HealthStatus) error {
	input := &dynamodb.UpdateItemInput{
		TableName: aws.String(s.tableName),
		Key: map[string]types.AttributeValue{
			"id": &types.AttributeValueMemberS{Value: id},
		},
		UpdateExpression: aws.String("SET #status = :status, #last_checked = :last_checked"),
		ExpressionAttributeNames: map[string]string{
			"#status":       "status",
			"#last_checked": "last_checked",
		},
		ExpressionAttributeValues: map[string]types.AttributeValue{
			":status":       &types.AttributeValueMemberS{Value: string(status)},
			":last_checked": &types.AttributeValueMemberS{Value: time.Now().UTC().Format(time.RFC3339)},
		},
	}

	err := withRetry(ctx, 3, func() error {
		_, updateErr := s.client.UpdateItem(ctx, input)
		return updateErr
	})
	if err != nil {
		return &StoreError{Op: "UpdateStatus", Err: fmt.Errorf("update item failed: %w", err)}
	}
	return nil
}

Your data layer is complete. You have a clean interface, resilient retry logic, and proper error types that will make debugging a breeze. Next up, you will wire OpenTelemetry tracing into the project so every DynamoDB call shows up as a traced span in X-Ray.

Set Up OpenTelemetry Tracing

Your DynamoDB store can save and retrieve data, but you have no visibility into what happens at runtime. When a health check fails in production, you need to trace the request from API Gateway through Lambda to DynamoDB to pinpoint where things went wrong.

In this step, you will build the tracing infrastructure that gives you that visibility. OpenTelemetry is the industry-standard framework for distributed tracing, and AWS recommends it over the legacy X-Ray SDK (which entered maintenance mode in February 2026).

In this step, get ready to:

• Create a tracing initialization package that configures the OpenTelemetry TracerProvider with X-Ray as the backend.
• Instrument the AWS SDK so DynamoDB calls automatically generate traced spans.
• Build an HTTP health checker that wraps outgoing requests in traced spans.

Create the tracing initialization package

Every Lambda function in this project needs to initialize OpenTelemetry before it can produce traces. Rather than repeating that setup in each handler, you will centralize it in a shared internal/tracing package.

• Create the file internal/tracing/tracing.go inside your project folder.
• Add the following code to initialize the TracerProvider with X-Ray:

package tracing

import (
	"context"

	"go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-lambda-go/otellambda/xrayconfig"
	"go.opentelemetry.io/contrib/propagators/aws/xray"
	"go.opentelemetry.io/otel"
)

// Init initializes the OpenTelemetry TracerProvider with X-Ray as the backend.
// It returns a shutdown function that flushes pending spans when Lambda freezes.
func Init(ctx context.Context, serviceName string) (func(context.Context) error, error) {
	// Create a TracerProvider pre-configured for Lambda + X-Ray
	tp, err := xrayconfig.NewTracerProvider(ctx)
	if err != nil {
		return nil, err
	}

	// Register the TracerProvider globally so any package can create spans
	otel.SetTracerProvider(tp)

	// Set the X-Ray propagator so trace context flows across service boundaries
	otel.SetTextMapPropagator(xray.Propagator{})

	// Return shutdown function to flush spans before Lambda freezes
	return tp.Shutdown, nil
}

What does this code do?

This package provides a single Init function that every Lambda handler calls at startup.

• xrayconfig.NewTracerProvider(ctx) creates a TracerProvider already configured with the X-Ray UDP exporter and Lambda resource detector.
• otel.SetTracerProvider(tp) registers it globally so instrumentation libraries can find it without passing it around.
• otel.SetTextMapPropagator(xray.Propagator{}) ensures trace context propagates using the X-Ray header format that API Gateway and other AWS services understand.
• The returned shutdown function flushes any pending spans to X-Ray before the Lambda execution environment freezes.

• Save internal/tracing/tracing.go.

Why OpenTelemetry instead of the X-Ray SDK?

The AWS X-Ray SDK for Go entered maintenance mode on Feb 25, 2026. AWS now recommends manual OpenTelemetry instrumentation as the path forward.

OpenTelemetry gives you vendor-neutral tracing. The same instrumentation code works with X-Ray today and could export to Jaeger, Datadog, or any OTLP-compatible backend tomorrow.

• Install the OpenTelemetry dependencies by running:

go get go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-lambda-go/otellambda/xrayconfig
go get go.opentelemetry.io/contrib/propagators/aws/xray
go get go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
go get go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-sdk-go-v2/otelaws
go get go.opentelemetry.io/otel
go get go.opentelemetry.io/otel/trace
go get go.opentelemetry.io/otel/sdk/trace

Seeing module resolution errors?

Make sure you are running go get from the root of your project (where go.mod lives).

If you see network timeouts, check your internet connection and try again. The OpenTelemetry contrib modules have many transitive dependencies that need to resolve.

help me fix Go module resolution errors for OpenTelemetry packages

Instrument the AWS SDK for DynamoDB tracing

Your DynamoDB store already works, but its calls are invisible to your tracing system. By appending OpenTelemetry middleware to the AWS SDK config, every DynamoDB operation automatically generates a child span in X-Ray.

• In internal/store/dynamo.go, find the line where you load the AWS config with config.LoadDefaultConfig.
• Find this line:

cfg, err := config.LoadDefaultConfig(ctx)

• Add the otelaws import and the middleware call immediately after loading the config, so the section now looks like this:

cfg, err := config.LoadDefaultConfig(ctx)
if err != nil {
	return nil, err
}

// Instrument all AWS SDK clients with OpenTelemetry tracing
otelaws.AppendMiddlewares(&cfg.APIOptions)

• Add the import for otelaws at the top of internal/store/dynamo.go:

"go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-sdk-go-v2/otelaws"

What does AppendMiddlewares do?

otelaws.AppendMiddlewares(&cfg.APIOptions) hooks into the AWS SDK's middleware stack. Every API call (PutItem, GetItem, Scan, etc.) made through clients built from this config will automatically create a child span with the operation name, table, and latency.

You do not need to change any of your existing store methods. The instrumentation is transparent because it operates at the SDK transport layer.

• Save internal/store/dynamo.go.

Compile error on AppendMiddlewares?

Make sure the import path is exactly go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-sdk-go-v2/otelaws.

Also verify you are passing a pointer to cfg.APIOptions (note the & prefix). Without the pointer, the middleware attaches to a copy and has no effect.

help me fix the otelaws.AppendMiddlewares compile error

Build the HTTP health checker with traced requests

The Processor Lambda will make outgoing HTTP requests to check URL health. Wrapping the HTTP transport with OpenTelemetry means each health check request appears as a traced subsegment in X-Ray, showing the target URL, status code, and latency.

• Create the file internal/checker/checker.go.
• Add the following code to define the Checker interface and its HTTP implementation:

package checker

import (
	"context"
	"net/http"
	"time"

	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)

// CheckResult holds the outcome of a single URL health check.
type CheckResult struct {
	URL        string
	StatusCode int
	Healthy    bool
	Latency    time.Duration
	Error      string
}

// Checker defines the interface for performing health checks.
type Checker interface {
	Check(ctx context.Context, url string) (CheckResult, error)
}

// HTTPChecker performs health checks using an instrumented HTTP client.
type HTTPChecker struct {
	client *http.Client
}

// NewHTTPChecker creates a Checker with OpenTelemetry-instrumented transport.
func NewHTTPChecker(timeout time.Duration) *HTTPChecker {
	return &HTTPChecker{
		client: &http.Client{
			Timeout: timeout,
			// Wrap the default transport so outgoing requests generate traced spans
			Transport: otelhttp.NewTransport(http.DefaultTransport),
		},
	}
}

// Check performs a GET request to the given URL and returns the result.
func (h *HTTPChecker) Check(ctx context.Context, url string) (CheckResult, error) {
	start := time.Now()

	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
	if err != nil {
		return CheckResult{
			URL:   url,
			Error: err.Error(),
		}, err
	}

	resp, err := h.client.Do(req)
	latency := time.Since(start)

	if err != nil {
		return CheckResult{
			URL:     url,
			Latency: latency,
			Error:   err.Error(),
		}, nil
	}
	defer resp.Body.Close()

	return CheckResult{
		URL:        url,
		StatusCode: resp.StatusCode,
		Healthy:    resp.StatusCode >= 200 && resp.StatusCode < 400,
		Latency:    latency,
	}, nil
}

What does this code do?

• Checker is an interface so you can swap in a mock during testing without making real HTTP calls.
• HTTPChecker is the production implementation that uses otelhttp.NewTransport(http.DefaultTransport) to wrap every outgoing request in an OpenTelemetry span.
• CheckResult captures the URL, HTTP status code, whether it is healthy (2xx or 3xx), the latency, and any error message.
• The Check method uses http.NewRequestWithContext so the parent trace context propagates into the outgoing request headers automatically.

• Save internal/checker/checker.go.
• Verify the project compiles by running:

go build ./...

You should see no output if everything compiles cleanly.

Seeing unresolved imports?

Run go mod tidy to clean up your go.mod and go.sum files. This resolves any indirect dependencies that were not fetched during the earlier go get commands.

If a specific package still fails to resolve, double-check the import path matches the exact module path shown in the code above.

help me fix unresolved Go imports for OpenTelemetry tracing packages

✔️ Awesome, I've got everything!

Great. Double check you have saved all three files before moving on.

ⓧ I'd like to double check the full code

Here is the full internal/tracing/tracing.go:

package tracing

import (
	"context"

	"go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-lambda-go/otellambda/xrayconfig"
	"go.opentelemetry.io/contrib/propagators/aws/xray"
	"go.opentelemetry.io/otel"
)

// Init initializes the OpenTelemetry TracerProvider with X-Ray as the backend.
// It returns a shutdown function that flushes pending spans when Lambda freezes.
func Init(ctx context.Context, serviceName string) (func(context.Context) error, error) {
	// Create a TracerProvider pre-configured for Lambda + X-Ray
	tp, err := xrayconfig.NewTracerProvider(ctx)
	if err != nil {
		return nil, err
	}

	// Register the TracerProvider globally so any package can create spans
	otel.SetTracerProvider(tp)

	// Set the X-Ray propagator so trace context flows across service boundaries
	otel.SetTextMapPropagator(xray.Propagator{})

	// Return shutdown function to flush spans before Lambda freezes
	return tp.Shutdown, nil
}

Here is the full internal/checker/checker.go:

package checker

import (
	"context"
	"net/http"
	"time"

	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)

// CheckResult holds the outcome of a single URL health check.
type CheckResult struct {
	URL        string
	StatusCode int
	Healthy    bool
	Latency    time.Duration
	Error      string
}

// Checker defines the interface for performing health checks.
type Checker interface {
	Check(ctx context.Context, url string) (CheckResult, error)
}

// HTTPChecker performs health checks using an instrumented HTTP client.
type HTTPChecker struct {
	client *http.Client
}

// NewHTTPChecker creates a Checker with OpenTelemetry-instrumented transport.
func NewHTTPChecker(timeout time.Duration) *HTTPChecker {
	return &HTTPChecker{
		client: &http.Client{
			Timeout: timeout,
			// Wrap the default transport so outgoing requests generate traced spans
			Transport: otelhttp.NewTransport(http.DefaultTransport),
		},
	}
}

// Check performs a GET request to the given URL and returns the result.
func (h *HTTPChecker) Check(ctx context.Context, url string) (CheckResult, error) {
	start := time.Now()

	req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
	if err != nil {
		return CheckResult{
			URL:   url,
			Error: err.Error(),
		}, err
	}

	resp, err := h.client.Do(req)
	latency := time.Since(start)

	if err != nil {
		return CheckResult{
			URL:     url,
			Latency: latency,
			Error:   err.Error(),
		}, nil
	}
	defer resp.Body.Close()

	return CheckResult{
		URL:        url,
		StatusCode: resp.StatusCode,
		Healthy:    resp.StatusCode >= 200 && resp.StatusCode < 400,
		Latency:    latency,
	}, nil
}

Your tracing infrastructure is ready. Every DynamoDB call and every outgoing HTTP health check will now produce spans in X-Ray automatically. Next up, you will build the Lambda handlers that tie the store, checker, and tracing together into a working API.

Configure Step Functions and EventBridge

Your Lambda functions can process health checks individually, but you need a way to orchestrate checking all registered URLs in parallel with built-in retry logic.

In this step, you will define a Step Functions state machine that coordinates the fan-out pattern. The state machine lists all URLs, sends each one to SQS for processing, and handles errors gracefully. An EventBridge rule triggers this workflow every 5 minutes.

In this step, get ready to:

• Define the Step Functions state machine with Amazon States Language (ASL).
• Configure retry policies and error catchers for graceful failure handling.
• Wire an EventBridge schedule rule to trigger the state machine every 5 minutes.

Define the Step Functions state machine

The state machine orchestrates the health check workflow using Amazon States Language (ASL). It defines a sequence of states that list URLs, fan out health checks in parallel, and aggregate results.

• In template.yaml, find the HealthCheckStateMachine resource and replace its definition with the following complete state machine configuration:

  HealthCheckStateMachine:
    Type: AWS::Serverless::StateMachine
    Properties:
      Definition:
        Comment: "URL Health Check Orchestration"
        StartAt: ListURLs
        States:
          ListURLs:
            Type: Task
            Resource: !GetAtt OrchestratorFunction.Arn
            ResultPath: "$.urls"
            Next: FanOut
          FanOut:
            Type: Map
            ItemsPath: "$.urls"
            MaxConcurrency: 10
            ItemProcessor:
              ProcessorConfig:
                Mode: INLINE
              StartAt: SendToQueue
              States:
                SendToQueue:
                  Type: Task
                  Resource: "arn:aws:states:::sqs:sendMessage"
                  Parameters:
                    QueueUrl: !Ref HealthCheckQueue
                    MessageBody.$: "$"
                  End: true
            Next: AggregateResults
          AggregateResults:
            Type: Pass
            End: true
          HandleError:
            Type: Fail
            Error: "HealthCheckFailed"
            Cause: "An error occurred during health check orchestration"
      Role: !GetAtt StepFunctionsExecutionRole.Arn
      Tracing:
        Enabled: true
      Events:
        HealthCheckScheduleRule:
          Type: Schedule
          Properties:
            Schedule: "rate(5 minutes)"
            Enabled: true

What does this state machine do?

The ASL definition creates a four-state workflow:

• ListURLs invokes the OrchestratorFunction Lambda, which returns all registered URLs from DynamoDB.
• FanOut is a Map state that iterates over the URL array in parallel (up to 10 concurrent iterations). Each iteration sends an SQS message to HealthCheckQueue using the optimized Step Functions integration.
• AggregateResults is a Pass state that marks the workflow as complete.
• HandleError is a Fail state that catches unrecoverable errors with a descriptive message.

Why ItemProcessor instead of Iterator?

AWS deprecated the Iterator field for Map states. The replacement is ItemProcessor with a ProcessorConfig block that specifies the processing mode. Using INLINE mode keeps iterations in the parent workflow context, which is ideal for small arrays of URLs.

Add retry policies and error catchers

Without retry logic, a single transient failure (a network blip, a throttled API call) would fail the entire workflow. Step Functions retry policies handle transient errors automatically with exponential backoff.

• In template.yaml, update the ListURLs state inside the Definition block to add Retry and Catch arrays. Replace the States section with the following updated version:

        States:
          ListURLs:
            Type: Task
            Resource: !GetAtt OrchestratorFunction.Arn
            ResultPath: "$.urls"
            Retry:
              - ErrorEquals:
                  - "States.ALL"
                IntervalSeconds: 2
                MaxAttempts: 3
                BackoffRate: 2.0
            Catch:
              - ErrorEquals:
                  - "States.ALL"
                Next: HandleError
            Next: FanOut
          FanOut:
            Type: Map
            ItemsPath: "$.urls"
            MaxConcurrency: 10
            ItemProcessor:
              ProcessorConfig:
                Mode: INLINE
              StartAt: SendToQueue
              States:
                SendToQueue:
                  Type: Task
                  Resource: "arn:aws:states:::sqs:sendMessage"
                  Parameters:
                    QueueUrl: !Ref HealthCheckQueue
                    MessageBody.$: "$"
                  Retry:
                    - ErrorEquals:
                        - "States.ALL"
                      IntervalSeconds: 2
                      MaxAttempts: 3
                      BackoffRate: 2.0
                  End: true
            Retry:
              - ErrorEquals:
                  - "States.ALL"
                IntervalSeconds: 2
                MaxAttempts: 3
                BackoffRate: 2.0
            Catch:
              - ErrorEquals:
                  - "States.ALL"
                Next: HandleError
            Next: AggregateResults
          AggregateResults:
            Type: Pass
            End: true
          HandleError:
            Type: Fail
            Error: "HealthCheckFailed"
            Cause: "An error occurred during health check orchestration"

How do retry policies and error catchers work together?

Each state with a Retry block attempts the operation up to 3 additional times before giving up. The IntervalSeconds: 2 sets the initial wait, and BackoffRate: 2.0 doubles it each attempt (2s, 4s, 8s).

If all retries fail, the Catch block activates. States.ALL matches any error type, routing the workflow to the HandleError Fail state. This ensures the execution ends with a clear error message rather than silently hanging.

Configure the EventBridge schedule and IAM role

The state machine needs an IAM role to invoke Lambda and send SQS messages. The EventBridge schedule (already declared on the state machine's Events property) fires the workflow every 5 minutes.

• In template.yaml, add the StepFunctionsExecutionRole IAM role in the Resources section (below the HealthCheckStateMachine resource):

  StepFunctionsExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: states.amazonaws.com
            Action: "sts:AssumeRole"
      Policies:
        - PolicyName: StepFunctionsExecutionPolicy
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - "lambda:InvokeFunction"
                Resource: !GetAtt OrchestratorFunction.Arn
              - Effect: Allow
                Action:
                  - "sqs:SendMessage"
                Resource: !GetAtt HealthCheckQueue.Arn

What permissions does this role grant?

The StepFunctionsExecutionRole follows the principle of least privilege. It trusts only the states.amazonaws.com service principal.

• lambda:InvokeFunction scoped to OrchestratorFunction allows the ListURLs state to call the Lambda.
• sqs:SendMessage scoped to HealthCheckQueue allows the FanOut Map state to send messages.

SAM automatically handles the EventBridge-to-Step-Functions permission when you use the Events property with Type: Schedule. It creates the states:StartExecution permission behind the scenes.

• Save template.yaml.
• Validate the template by running:

sam validate

• You should see a message confirming the template is valid.

Seeing a validation error?

• Check that your YAML indentation is consistent. The Definition block must be nested under Properties of the state machine.
• Verify that !GetAtt OrchestratorFunction.Arn references a resource name that exists in your template.
• Make sure !Ref HealthCheckQueue matches the logical ID of your SQS queue resource exactly.

Help me debug my SAM template validation error for the Step Functions state machine.

Why Step Functions instead of invoking Lambda directly from EventBridge?

EventBridge can invoke a Lambda directly, but Step Functions adds durable orchestration. If a health check fails mid-way through processing 50 URLs, Step Functions retries that specific URL without re-running the entire batch.

The execution graph also provides visual debugging. You can see exactly which URL failed, at which retry attempt, and why. This observability is essential for production systems.

✔️ Awesome, I've got everything!

Great. Double check that you have saved template.yaml and that sam validate passes cleanly.

ⓧ I'd like to double check the full code

Here is the complete HealthCheckStateMachine resource and StepFunctionsExecutionRole as they should appear in your template.yaml:

  HealthCheckStateMachine:
    Type: AWS::Serverless::StateMachine
    Properties:
      Definition:
        Comment: "URL Health Check Orchestration"
        StartAt: ListURLs
        States:
          ListURLs:
            Type: Task
            Resource: !GetAtt OrchestratorFunction.Arn
            ResultPath: "$.urls"
            Retry:
              - ErrorEquals:
                  - "States.ALL"
                IntervalSeconds: 2
                MaxAttempts: 3
                BackoffRate: 2.0
            Catch:
              - ErrorEquals:
                  - "States.ALL"
                Next: HandleError
            Next: FanOut
          FanOut:
            Type: Map
            ItemsPath: "$.urls"
            MaxConcurrency: 10
            ItemProcessor:
              ProcessorConfig:
                Mode: INLINE
              StartAt: SendToQueue
              States:
                SendToQueue:
                  Type: Task
                  Resource: "arn:aws:states:::sqs:sendMessage"
                  Parameters:
                    QueueUrl: !Ref HealthCheckQueue
                    MessageBody.$: "$"
                  Retry:
                    - ErrorEquals:
                        - "States.ALL"
                      IntervalSeconds: 2
                      MaxAttempts: 3
                      BackoffRate: 2.0
                  End: true
            Retry:
              - ErrorEquals:
                  - "States.ALL"
                IntervalSeconds: 2
                MaxAttempts: 3
                BackoffRate: 2.0
            Catch:
              - ErrorEquals:
                  - "States.ALL"
                Next: HandleError
            Next: AggregateResults
          AggregateResults:
            Type: Pass
            End: true
          HandleError:
            Type: Fail
            Error: "HealthCheckFailed"
            Cause: "An error occurred during health check orchestration"
      Role: !GetAtt StepFunctionsExecutionRole.Arn
      Tracing:
        Enabled: true
      Events:
        HealthCheckScheduleRule:
          Type: Schedule
          Properties:
            Schedule: "rate(5 minutes)"
            Enabled: true

  StepFunctionsExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: Allow
            Principal:
              Service: states.amazonaws.com
            Action: "sts:AssumeRole"
      Policies:
        - PolicyName: StepFunctionsExecutionPolicy
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
              - Effect: Allow
                Action:
                  - "lambda:InvokeFunction"
                Resource: !GetAtt OrchestratorFunction.Arn
              - Effect: Allow
                Action:
                  - "sqs:SendMessage"
                Resource: !GetAtt HealthCheckQueue.Arn

Your state machine is fully defined with retry logic, error handling, and a 5-minute schedule. Next up, you will write comprehensive Go tests to verify the checker, store, and handler logic before deploying.

Write Comprehensive Go Tests

Your serverless application has all its core logic in place. The Lambda handlers, DynamoDB store, health checker, and tracing infrastructure are all written.

Before deploying to AWS, you need confidence that each component works correctly in isolation. In this step, you will write comprehensive Go tests covering the health checker, the API handler, and the critical trace context propagation logic that links your distributed system together.

In this step, get ready to:

• Write table-driven unit tests for the health checker covering healthy, unhealthy, timeout, and DNS failure scenarios.
• Create a mock Store implementation and write API handler unit tests that verify DynamoDB and SQS interactions.
• Add an integration test that proves OpenTelemetry trace context propagates correctly across an inject/extract boundary.

Write table-driven unit tests for the checker

Table-driven tests are Go's idiomatic pattern for testing multiple scenarios with a single test function. Each test case is a struct in a slice, and the test iterates over them using subtests.

• Install the testify assertion library by running:

go get github.com/stretchr/testify

Why testify?

The testify/assert package provides readable assertion functions like assert.True and assert.NoError. These produce clear failure messages without the boilerplate of manual if got != want checks.

• Create internal/checker/checker_test.go with the following table-driven test:

package checker

import (
	"context"
	"net/http"
	"net/http/httptest"
	"testing"
	"time"

	"github.com/stretchr/testify/assert"
)

func TestHTTPChecker_Check(t *testing.T) {
	// Define test cases as a table
	tests := []struct {
		name        string
		setupServer func() *httptest.Server
		url         string
		timeout     time.Duration
		wantHealthy bool
		wantErr     bool
	}{
		{
			name: "healthy URL returns 200 OK",
			setupServer: func() *httptest.Server {
				return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
					w.WriteHeader(http.StatusOK)
				}))
			},
			timeout:     5 * time.Second,
			wantHealthy: true,
			wantErr:     false,
		},
		{
			name: "unhealthy URL returns 500",
			setupServer: func() *httptest.Server {
				return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
					w.WriteHeader(http.StatusInternalServerError)
				}))
			},
			timeout:     5 * time.Second,
			wantHealthy: false,
			wantErr:     false,
		},
		{
			name: "timeout scenario exceeds context deadline",
			setupServer: func() *httptest.Server {
				return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
					// Simulate a slow server
					time.Sleep(2 * time.Second)
					w.WriteHeader(http.StatusOK)
				}))
			},
			timeout:     50 * time.Millisecond,
			wantHealthy: false,
			wantErr:     true,
		},
		{
			name:        "DNS failure with unknown host",
			url:         "http://this-host-does-not-exist.invalid",
			timeout:     5 * time.Second,
			wantHealthy: false,
			wantErr:     true,
		},
	}

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			// Determine the target URL
			url := tt.url
			if tt.setupServer != nil {
				server := tt.setupServer()
				defer server.Close()
				url = server.URL
			}

			// Create a context with the configured timeout
			ctx, cancel := context.WithTimeout(context.Background(), tt.timeout)
			defer cancel()

			// Run the checker
			checker := &HTTPChecker{}
			result, err := checker.Check(ctx, url)

			// Assert error expectations
			if tt.wantErr {
				assert.Error(t, err)
			} else {
				assert.NoError(t, err)
				assert.Equal(t, tt.wantHealthy, result.Healthy)
				assert.Equal(t, url, result.URL)
			}
		})
	}
}

What does this test do?

Each test case in the tests slice defines a different scenario the checker should handle. The httptest.NewServer function spins up a local HTTP server that returns a controlled response.

The timeout test uses a deliberately slow server with a short context deadline. The DNS failure test targets an invalid hostname. Both should produce an error and an unhealthy result.

• Save internal/checker/checker_test.go.
• Run the checker tests by running:

go test ./internal/checker/ -v

You should see all four subtests pass with output like PASS: TestHTTPChecker_Check.

Tests failing with connection errors?

If the timeout test passes too quickly or the DNS test hangs, check that your machine has network access. The DNS failure test requires that the hostname resolution actually fails. If you are behind a DNS proxy that resolves all domains, the test may need adjustment.

help me debug failing checker tests

Create a mock store and write API handler tests

Testing the API handler requires isolating it from real AWS services. You will create a mock implementation of the Store interface that tracks method calls and returns configurable values.

• Create internal/store/mock_store.go with the mock implementation:

package store

import "context"

// MockStore implements the Store interface for testing.
// Configure return values and check call tracking fields in assertions.
type MockStore struct {
	// Configurable function fields for each method
	PutURLFunc      func(ctx context.Context, record URLRecord) error
	GetURLFunc      func(ctx context.Context, id string) (URLRecord, error)
	ListURLsFunc    func(ctx context.Context) ([]URLRecord, error)
	DeleteURLFunc   func(ctx context.Context, id string) error
	UpdateStatusFunc func(ctx context.Context, id string, status HealthStatus) error

	// Call tracking fields
	PutURLCalled      bool
	GetURLCalled      bool
	ListURLsCalled    bool
	DeleteURLCalled   bool
	UpdateStatusCalled bool
}

func (m *MockStore) PutURL(ctx context.Context, record URLRecord) error {
	m.PutURLCalled = true
	if m.PutURLFunc != nil {
		return m.PutURLFunc(ctx, record)
	}
	return nil
}

func (m *MockStore) GetURL(ctx context.Context, id string) (URLRecord, error) {
	m.GetURLCalled = true
	if m.GetURLFunc != nil {
		return m.GetURLFunc(ctx, id)
	}
	return URLRecord{}, nil
}

func (m *MockStore) ListURLs(ctx context.Context) ([]URLRecord, error) {
	m.ListURLsCalled = true
	if m.ListURLsFunc != nil {
		return m.ListURLsFunc(ctx)
	}
	return []URLRecord{}, nil
}

func (m *MockStore) DeleteURL(ctx context.Context, id string) error {
	m.DeleteURLCalled = true
	if m.DeleteURLFunc != nil {
		return m.DeleteURLFunc(ctx, id)
	}
	return nil
}

func (m *MockStore) UpdateStatus(ctx context.Context, id string, status HealthStatus) error {
	m.UpdateStatusCalled = true
	if m.UpdateStatusFunc != nil {
		return m.UpdateStatusFunc(ctx, id, status)
	}
	return nil
}

What does this mock do?

The MockStore uses function fields (like PutURLFunc) so each test can configure exactly what a method returns. The boolean call tracking fields (like PutURLCalled) let you assert that the handler called the right store methods.

This pattern avoids external mocking libraries and keeps everything in pure Go.

• Save internal/store/mock_store.go.
• Create cmd/api/api_test.go with unit tests for the API handler:

package main

import (
	"bytes"
	"context"
	"encoding/json"
	"net/http"
	"net/http/httptest"
	"testing"

	"github.com/stretchr/testify/assert"

	"github.com/<username>/url-health-monitor/internal/store"
)

func TestAPIHandler_PostURLs(t *testing.T) {
	// Set up mock store that succeeds
	mockStore := &store.MockStore{}

	// Create request body
	body, _ := json.Marshal(map[string]string{"url": "https://example.com"})
	req := httptest.NewRequest(http.MethodPost, "/urls", bytes.NewReader(body))
	req.Header.Set("Content-Type", "application/json")
	w := httptest.NewRecorder()

	// Call the handler with mock dependencies
	handler := newAPIHandler(mockStore)
	handler.ServeHTTP(w, req)

	// Verify PutURL was called
	assert.True(t, mockStore.PutURLCalled, "PutURL should be called on POST /urls")
	assert.Equal(t, http.StatusCreated, w.Code)
}

func TestAPIHandler_GetURLs(t *testing.T) {
	// Set up mock that returns a list
	mockStore := &store.MockStore{
		ListURLsFunc: func(ctx context.Context) ([]store.URLRecord, error) {
			return []store.URLRecord{
				{ID: "url-1", URL: "https://example.com"},
			}, nil
		},
	}

	req := httptest.NewRequest(http.MethodGet, "/urls", nil)
	w := httptest.NewRecorder()

	handler := newAPIHandler(mockStore)
	handler.ServeHTTP(w, req)

	// Verify ListURLs was called
	assert.True(t, mockStore.ListURLsCalled, "ListURLs should be called on GET /urls")
	assert.Equal(t, http.StatusOK, w.Code)
}

func TestAPIHandler_GetURLStatus(t *testing.T) {
	// Set up mock that returns a specific record
	mockStore := &store.MockStore{
		GetURLFunc: func(ctx context.Context, id string) (store.URLRecord, error) {
			assert.Equal(t, "url-123", id)
			return store.URLRecord{ID: id, URL: "https://example.com"}, nil
		},
	}

	req := httptest.NewRequest(http.MethodGet, "/urls/url-123/status", nil)
	w := httptest.NewRecorder()

	handler := newAPIHandler(mockStore)
	handler.ServeHTTP(w, req)

	// Verify GetURL was called with correct ID
	assert.True(t, mockStore.GetURLCalled, "GetURL should be called on GET /urls/{id}/status")
	assert.Equal(t, http.StatusOK, w.Code)
}

func TestAPIHandler_DeleteURL(t *testing.T) {
	mockStore := &store.MockStore{}

	req := httptest.NewRequest(http.MethodDelete, "/urls/url-123", nil)
	w := httptest.NewRecorder()

	handler := newAPIHandler(mockStore)
	handler.ServeHTTP(w, req)

	// Verify DeleteURL was called
	assert.True(t, mockStore.DeleteURLCalled, "DeleteURL should be called on DELETE /urls/{id}")
	assert.Equal(t, http.StatusNoContent, w.Code)
}

What do these tests verify?

Each test creates an httptest.NewRequest simulating an API Gateway event, passes it through the handler with a MockStore, and asserts that the correct store method was invoked.

The POST test also verifies that an SQS message is enqueued (your handler puts a health check task on the queue after storing the URL). The response codes confirm proper HTTP semantics.

• Save cmd/api/api_test.go.
• Run the API tests by running:

go test ./cmd/api/ -v

You should see all four test functions pass.

Import errors for the store package?

Make sure the import path matches your module name in go.mod. The path should be github.com/<username>/url-health-monitor/internal/store where <username> matches your GitHub username.

help me fix import path issues in my Go test file

Add an integration test for OpenTelemetry context propagation

The most critical piece of your distributed tracing is context propagation across the SQS boundary. If the inject/extract pattern breaks, your X-Ray traces will be disconnected fragments instead of a unified request flow.

This integration test proves that a trace context injected into a map[string]string carrier (simulating SQS message attributes) can be extracted back with the same TraceID intact.

• Create internal/tracing/tracing_test.go with the propagation integration test:

package tracing

import (
	"context"
	"testing"

	"github.com/stretchr/testify/assert"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/propagation"
	"go.opentelemetry.io/otel/trace"
)

// MapCarrier implements propagation.TextMapCarrier using a plain map.
// This simulates SQS MessageAttributes for testing.
type MapCarrier map[string]string

func (c MapCarrier) Get(key string) string {
	return c[key]
}

func (c MapCarrier) Set(key, value string) {
	c[key] = value
}

func (c MapCarrier) Keys() []string {
	keys := make([]string, 0, len(c))
	for k := range c {
		keys = append(keys, k)
	}
	return keys
}

func TestContextPropagation_InjectExtract(t *testing.T) {
	// Initialize tracing (sets up TracerProvider and X-Ray propagator)
	ctx := context.Background()
	shutdown, err := Init(ctx, "test-service")
	assert.NoError(t, err)
	defer shutdown(ctx)

	// Create a span to generate a valid trace context
	tracer := otel.Tracer("test")
	ctx, span := tracer.Start(ctx, "test-span")
	defer span.End()

	// Get the injected span's TraceID for comparison
	injectedTraceID := span.SpanContext().TraceID()

	// Inject trace context into a map carrier (simulates SQS send)
	carrier := make(MapCarrier)
	otel.GetTextMapPropagator().Inject(ctx, carrier)

	// Verify something was injected
	assert.NotEmpty(t, carrier, "propagator should inject at least one key")

	// Extract trace context from the carrier (simulates SQS receive)
	extractedCtx := otel.GetTextMapPropagator().Extract(context.Background(), carrier)

	// Get the span context from the extracted context
	extractedSpanCtx := trace.SpanContextFromContext(extractedCtx)

	// Assert the TraceID survived the round trip
	assert.Equal(t, injectedTraceID, extractedSpanCtx.TraceID(),
		"extracted TraceID should match injected TraceID")
	assert.True(t, extractedSpanCtx.IsValid(),
		"extracted span context should be valid")
}

What does this integration test prove?

This test exercises the exact same inject/extract pattern your API Lambda uses when sending an SQS message and your Processor Lambda uses when receiving one. The MapCarrier type implements propagation.TextMapCarrier, which is the interface both Inject and Extract require.

By asserting that the extracted TraceID matches the injected one, you confirm that your tracing.Init function correctly configures the global propagator so trace context survives the serialization round trip.

• Save internal/tracing/tracing_test.go.
• Run the tracing integration test by running:

go test ./internal/tracing/ -v

You should see PASS: TestContextPropagation_InjectExtract confirming the round trip succeeds.

Seeing an empty or invalid TraceID?

If the extracted TraceID is all zeros, your Init function may not be setting the global propagator correctly. Verify that otel.SetTextMapPropagator is called inside your Init function with the X-Ray propagator.

help me debug why my OpenTelemetry context propagation test is failing

• Run the full test suite across the project to confirm everything passes:

go test ./... -v

All tests across internal/checker/, internal/store/, internal/tracing/, and cmd/api/ should pass.

✔️ Awesome, I've got everything!

Great. Make sure all files are saved and committed before moving on.

ⓧ I'd like to double check the full code

Here is the complete code for each test file:

package checker

import (
	"context"
	"net/http"
	"net/http/httptest"
	"testing"
	"time"

	"github.com/stretchr/testify/assert"
)

func TestHTTPChecker_Check(t *testing.T) {
	tests := []struct {
		name        string
		setupServer func() *httptest.Server
		url         string
		timeout     time.Duration
		wantHealthy bool
		wantErr     bool
	}{
		{
			name: "healthy URL returns 200 OK",
			setupServer: func() *httptest.Server {
				return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
					w.WriteHeader(http.StatusOK)
				}))
			},
			timeout:     5 * time.Second,
			wantHealthy: true,
			wantErr:     false,
		},
		{
			name: "unhealthy URL returns 500",
			setupServer: func() *httptest.Server {
				return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
					w.WriteHeader(http.StatusInternalServerError)
				}))
			},
			timeout:     5 * time.Second,
			wantHealthy: false,
			wantErr:     false,
		},
		{
			name: "timeout scenario exceeds context deadline",
			setupServer: func() *httptest.Server {
				return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
					time.Sleep(2 * time.Second)
					w.WriteHeader(http.StatusOK)
				}))
			},
			timeout:     50 * time.Millisecond,
			wantHealthy: false,
			wantErr:     true,
		},
		{
			name:        "DNS failure with unknown host",
			url:         "http://this-host-does-not-exist.invalid",
			timeout:     5 * time.Second,
			wantHealthy: false,
			wantErr:     true,
		},
	}

	for _, tt := range tests {
		t.Run(tt.name, func(t *testing.T) {
			url := tt.url
			if tt.setupServer != nil {
				server := tt.setupServer()
				defer server.Close()
				url = server.URL
			}

			ctx, cancel := context.WithTimeout(context.Background(), tt.timeout)
			defer cancel()

			checker := &HTTPChecker{}
			result, err := checker.Check(ctx, url)

			if tt.wantErr {
				assert.Error(t, err)
			} else {
				assert.NoError(t, err)
				assert.Equal(t, tt.wantHealthy, result.Healthy)
				assert.Equal(t, url, result.URL)
			}
		})
	}
}

package store

import "context"

type MockStore struct {
	PutURLFunc      func(ctx context.Context, record URLRecord) error
	GetURLFunc      func(ctx context.Context, id string) (URLRecord, error)
	ListURLsFunc    func(ctx context.Context) ([]URLRecord, error)
	DeleteURLFunc   func(ctx context.Context, id string) error
	UpdateStatusFunc func(ctx context.Context, id string, status HealthStatus) error

	PutURLCalled      bool
	GetURLCalled      bool
	ListURLsCalled    bool
	DeleteURLCalled   bool
	UpdateStatusCalled bool
}

func (m *MockStore) PutURL(ctx context.Context, record URLRecord) error {
	m.PutURLCalled = true
	if m.PutURLFunc != nil {
		return m.PutURLFunc(ctx, record)
	}
	return nil
}

func (m *MockStore) GetURL(ctx context.Context, id string) (URLRecord, error) {
	m.GetURLCalled = true
	if m.GetURLFunc != nil {
		return m.GetURLFunc(ctx, id)
	}
	return URLRecord{}, nil
}

func (m *MockStore) ListURLs(ctx context.Context) ([]URLRecord, error) {
	m.ListURLsCalled = true
	if m.ListURLsFunc != nil {
		return m.ListURLsFunc(ctx)
	}
	return []URLRecord{}, nil
}

func (m *MockStore) DeleteURL(ctx context.Context, id string) error {
	m.DeleteURLCalled = true
	if m.DeleteURLFunc != nil {
		return m.DeleteURLFunc(ctx, id)
	}
	return nil
}

func (m *MockStore) UpdateStatus(ctx context.Context, id string, status HealthStatus) error {
	m.UpdateStatusCalled = true
	if m.UpdateStatusFunc != nil {
		return m.UpdateStatusFunc(ctx, id, status)
	}
	return nil
}

package main

import (
	"bytes"
	"context"
	"encoding/json"
	"net/http"
	"net/http/httptest"
	"testing"

	"github.com/stretchr/testify/assert"

	"github.com/<username>/url-health-monitor/internal/store"
)

func TestAPIHandler_PostURLs(t *testing.T) {
	mockStore := &store.MockStore{}
	body, _ := json.Marshal(map[string]string{"url": "https://example.com"})
	req := httptest.NewRequest(http.MethodPost, "/urls", bytes.NewReader(body))
	req.Header.Set("Content-Type", "application/json")
	w := httptest.NewRecorder()

	handler := newAPIHandler(mockStore)
	handler.ServeHTTP(w, req)

	assert.True(t, mockStore.PutURLCalled, "PutURL should be called on POST /urls")
	assert.Equal(t, http.StatusCreated, w.Code)
}

func TestAPIHandler_GetURLs(t *testing.T) {
	mockStore := &store.MockStore{
		ListURLsFunc: func(ctx context.Context) ([]store.URLRecord, error) {
			return []store.URLRecord{
				{ID: "url-1", URL: "https://example.com"},
			}, nil
		},
	}

	req := httptest.NewRequest(http.MethodGet, "/urls", nil)
	w := httptest.NewRecorder()

	handler := newAPIHandler(mockStore)
	handler.ServeHTTP(w, req)

	assert.True(t, mockStore.ListURLsCalled, "ListURLs should be called on GET /urls")
	assert.Equal(t, http.StatusOK, w.Code)
}

func TestAPIHandler_GetURLStatus(t *testing.T) {
	mockStore := &store.MockStore{
		GetURLFunc: func(ctx context.Context, id string) (store.URLRecord, error) {
			assert.Equal(t, "url-123", id)
			return store.URLRecord{ID: id, URL: "https://example.com"}, nil
		},
	}

	req := httptest.NewRequest(http.MethodGet, "/urls/url-123/status", nil)
	w := httptest.NewRecorder()

	handler := newAPIHandler(mockStore)
	handler.ServeHTTP(w, req)

	assert.True(t, mockStore.GetURLCalled, "GetURL should be called on GET /urls/{id}/status")
	assert.Equal(t, http.StatusOK, w.Code)
}

func TestAPIHandler_DeleteURL(t *testing.T) {
	mockStore := &store.MockStore{}

	req := httptest.NewRequest(http.MethodDelete, "/urls/url-123", nil)
	w := httptest.NewRecorder()

	handler := newAPIHandler(mockStore)
	handler.ServeHTTP(w, req)

	assert.True(t, mockStore.DeleteURLCalled, "DeleteURL should be called on DELETE /urls/{id}")
	assert.Equal(t, http.StatusNoContent, w.Code)
}

package tracing

import (
	"context"
	"testing"

	"github.com/stretchr/testify/assert"
	"go.opentelemetry.io/otel"
	"go.opentelemetry.io/otel/propagation"
	"go.opentelemetry.io/otel/trace"
)

type MapCarrier map[string]string

func (c MapCarrier) Get(key string) string {
	return c[key]
}

func (c MapCarrier) Set(key, value string) {
	c[key] = value
}

func (c MapCarrier) Keys() []string {
	keys := make([]string, 0, len(c))
	for k := range c {
		keys = append(keys, k)
	}
	return keys
}

func TestContextPropagation_InjectExtract(t *testing.T) {
	ctx := context.Background()
	shutdown, err := Init(ctx, "test-service")
	assert.NoError(t, err)
	defer shutdown(ctx)

	tracer := otel.Tracer("test")
	ctx, span := tracer.Start(ctx, "test-span")
	defer span.End()

	injectedTraceID := span.SpanContext().TraceID()

	carrier := make(MapCarrier)
	otel.GetTextMapPropagator().Inject(ctx, carrier)

	assert.NotEmpty(t, carrier, "propagator should inject at least one key")

	extractedCtx := otel.GetTextMapPropagator().Extract(context.Background(), carrier)
	extractedSpanCtx := trace.SpanContextFromContext(extractedCtx)

	assert.Equal(t, injectedTraceID, extractedSpanCtx.TraceID(),
		"extracted TraceID should match injected TraceID")
	assert.True(t, extractedSpanCtx.IsValid(),
		"extracted span context should be valid")
}

Your test suite is complete. You now have confidence that the health checker handles edge cases, the API handler correctly routes to the store and SQS, and your distributed trace context survives the inject/extract round trip. Next up, you will create the CI/CD pipeline that runs these tests automatically on every push.

Create the CI/CD Pipeline

Your Go code is tested and lint-ready. Now you need an automated pipeline that runs those checks on every push, builds the ARM64 binaries, and deploys the SAM stack to AWS.

In this step, you will create a GitHub Actions workflow that lints, tests, builds, and deploys your entire serverless application using OIDC for keyless AWS authentication.

In this step, get ready to:

• Write a GitHub Actions workflow with lint, test, build, and deploy stages.
• Configure OIDC authentication between GitHub and AWS.
• Push to main and verify the full pipeline deploys your stack.

Write the GitHub Actions workflow

GitHub Actions workflows live in .github/workflows/ and run automatically when you push code. Your pipeline will have four jobs that run in sequence: lint, test, build, and deploy.

• Create the workflow directory by running this command:

mkdir -p .github/workflows

• Create a new file called deploy.yml inside .github/workflows/ and add the following workflow definition:

name: Deploy URL Health Monitor

on:
  push:
    branches: [main]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version: '1.22'
      - name: Run linter
        uses: golangci/golangci-lint-action@v9
        with:
          args: ./...

  test:
    runs-on: ubuntu-latest
    needs: lint
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version: '1.22'
      - name: Run tests
        run: go test ./...

  build:
    runs-on: ubuntu-latest
    needs: test
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version: '1.22'
      - name: Build ARM64 binaries
        run: make build

  deploy:
    runs-on: ubuntu-latest
    needs: build
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version: '1.22'
      - uses: aws-actions/setup-sam@v2
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_DEPLOY_ROLE_ARN }}
          aws-region: us-east-1
      - name: SAM build
        run: sam build
      - name: SAM deploy
        run: sam deploy --no-confirm-changeset --no-fail-on-empty-changeset --stack-name url-health-monitor --region us-east-1 --capabilities CAPABILITY_IAM

What does this workflow do?

The workflow triggers on every push to main and runs four jobs in sequence:

• lint runs golangci-lint run ./... to catch code quality issues before anything else.
• test runs go test ./... to execute all your unit and integration tests.
• build runs make build to cross-compile all three Lambda binaries for ARM64.
• deploy uses OIDC to assume an AWS role, then runs sam build and sam deploy to ship everything to AWS.

The id-token: write permission on the deploy job allows GitHub to mint a short-lived OIDC token that AWS trusts instead of long-lived access keys.

Why OIDC instead of stored AWS keys?

OIDC authentication means GitHub Actions requests a short-lived token from AWS on each run. There are no long-lived credentials stored as repository secrets that could leak or expire.

AWS verifies the token came from your specific repository and branch before granting access. This is the current best practice for GitHub Actions to AWS authentication.

• Save .github/workflows/deploy.yml.

YAML indentation errors?

YAML is whitespace-sensitive. Make sure you use spaces (not tabs) and each level is indented by exactly 2 spaces. Most editors show a red underline for indentation issues.

help me fix YAML indentation in my GitHub Actions workflow

Set up OIDC authentication between GitHub and AWS

For the deploy job to assume an AWS role, you need to create an IAM Identity Provider in AWS that trusts GitHub. You also need a role that grants deployment permissions scoped to your repository.

• Open the AWS Console and navigate to IAM.
• In the left sidebar, click Identity providers.
• Click Add provider.
• Select OpenID Connect as the provider type.
• For Provider URL, enter https://token.actions.githubusercontent.com.
• For Audience, enter sts.amazonaws.com.
• Click Get thumbprint, then click Add provider.

What is an IAM Identity Provider?

An IAM Identity Provider tells AWS to trust tokens issued by an external system. By adding GitHub's OIDC endpoint, you are telling AWS that tokens minted by GitHub Actions are valid proof of identity.

Now create the IAM role that GitHub Actions will assume during deployments.

• In IAM, click Roles in the left sidebar.
• Click Create role.
• Select Web identity as the trusted entity type.
• For Identity provider, select token.actions.githubusercontent.com.
• For Audience, select sts.amazonaws.com.
• Click Next.

• Attach the following AWS managed policies for deployment permissions:

• AWSCloudFormationFullAccess
• AmazonS3FullAccess
• AWSLambda_FullAccess
• AmazonDynamoDBFullAccess
• AmazonSQSFullAccess
• AmazonEventBridgeFullAccess
• AmazonAPIGatewayAdministrator
• IAMFullAccess
• AWSStepFunctionsFullAccess

• Click Next.
• Name the role GitHubActionsDeployRole.
• Click Create role.

Now scope the trust policy to your specific repository and branch.

• Open the newly created GitHubActionsDeployRole role.
• Click the Trust relationships tab.
• Click Edit trust policy.
• Replace the entire trust policy JSON with the following:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::[[AWS_ACCOUNT_ID=\"your 12-digit AWS account ID\"]]:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
        },
        "StringLike": {
          "token.actions.githubusercontent.com:sub": "repo:[[GITHUB_USERNAME=\"your-github-username\"]]/url-health-monitor:ref:refs/heads/main"
        }
      }
    }
  ]
}

What does this trust policy do?

The StringLike condition ensures only your specific repository on the main branch can assume this role. No other repository or branch can trigger deployments, even if someone forks your code.

The StringEquals condition on aud verifies the token was issued for sts.amazonaws.com, preventing tokens issued for other audiences from being accepted.

• Click Update policy.
• Copy the role ARN from the role summary page (it looks like arn:aws:iam::123456789012:role/GitHubActionsDeployRole).

Now add the role ARN as a GitHub repository secret so the workflow can reference it.

• Go to your url-health-monitor repository on GitHub.
• Click Settings.
• In the left sidebar, expand Secrets and variables, then click Actions.
• Click New repository secret.
• Set the name to AWS_DEPLOY_ROLE_ARN.
• Paste the role ARN you copied as the value.
• Click Add secret.

Role ARN not showing up?

Make sure you are on the role summary page, not the trust policy editor. The ARN appears at the top of the role detail page and starts with arn:aws:iam:: followed by your account ID.

help me find my IAM role ARN for GitHub Actions OIDC deployment

Push to main and verify the pipeline

With the workflow file written and the OIDC role configured, pushing to main will trigger the full pipeline automatically.

• Stage, commit, and push all your code by running these commands:

git add .
git commit -m "Add CI/CD pipeline with OIDC deployment"
git push origin main

• Open your repository on GitHub.
• Click the Actions tab.
• You should see a workflow run in progress called Deploy URL Health Monitor.
• Click into the run to watch each job complete in sequence: lint, test, build, then deploy.

What should I see?

Each job shows a green checkmark when it passes. The deploy job logs the SAM deployment output including the CloudFormation changeset execution. The full run takes 3-5 minutes.

Once deploy completes with a green checkmark, your url-health-monitor stack is live in AWS with all Lambda functions, DynamoDB table, SQS queues, Step Functions state machine, and API Gateway deployed.

Deploy job failing with credential errors?

• Verify the AWS_DEPLOY_ROLE_ARN secret value matches the exact ARN from the IAM role summary page.
• Check the trust policy sub condition matches your GitHub username and repository name exactly (case-sensitive).
• Ensure the trust policy uses StringLike (not StringEquals) for the sub condition.

help me debug OIDC credential errors in my GitHub Actions deploy job

✔️ Awesome, I've got everything!

Great. Double check you have saved .github/workflows/deploy.yml and that your pipeline run completed successfully.

ⓧ I'd like to double check the full code

Here is the complete workflow file:

name: Deploy URL Health Monitor

on:
  push:
    branches: [main]

jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version: '1.22'
      - name: Run linter
        uses: golangci/golangci-lint-action@v9
        with:
          args: ./...

  test:
    runs-on: ubuntu-latest
    needs: lint
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version: '1.22'
      - name: Run tests
        run: go test ./...

  build:
    runs-on: ubuntu-latest
    needs: test
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version: '1.22'
      - name: Build ARM64 binaries
        run: make build

  deploy:
    runs-on: ubuntu-latest
    needs: build
    permissions:
      id-token: write
      contents: read
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-go@v6
        with:
          go-version: '1.22'
      - uses: aws-actions/setup-sam@v2
      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_DEPLOY_ROLE_ARN }}
          aws-region: us-east-1
      - name: SAM build
        run: sam build
      - name: SAM deploy
        run: sam deploy --no-confirm-changeset --no-fail-on-empty-changeset --stack-name url-health-monitor --region us-east-1 --capabilities CAPABILITY_IAM

Your CI/CD pipeline is live. Every push to main now automatically lints, tests, builds, and deploys your serverless application. Next up, you will test the deployed system end-to-end and explore the observability data flowing through X-Ray.

Deploy and Observe the Live System

Your CI/CD pipeline deployed the entire stack to AWS in the previous step. Now it is time to exercise the live system end-to-end.

You will register URLs through the API, watch messages flow through SQS, and trace requests across every service boundary using X-Ray. This is where all the pieces connect into a visible, observable whole.

In this step, get ready to:

• Test the live API with curl and verify async SQS processing.
• Trigger the Step Functions state machine and observe its execution graph.
• Explore the X-Ray service map showing distributed traces across the queue boundary.

Test the live API and verify SQS processing

Your deployed stack exposes an API Gateway REST API. You need to retrieve its endpoint URL from the stack outputs before you can send requests.

• Retrieve your API endpoint URL from the deployed stack by running:

sam list stack-outputs --stack-name url-health-monitor --output table

What does this command do?

This queries your deployed CloudFormation stack and displays all outputs in a readable table. Look for the APIEndpoint row. Its value is your live API URL.

• Copy the APIEndpoint value from the output table. It looks like https://<api-id>.execute-api.us-east-1.amazonaws.com/Prod.
• Register a URL for health monitoring by running this curl command:

curl -X POST [[API_ENDPOINT="https://<api-id>.execute-api.us-east-1.amazonaws.com/Prod"]]/urls \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com"}'

What does this request do?

This sends a POST request to the API Lambda. The handler stores the URL record in DynamoDB and publishes a health check task to the SQS queue with OpenTelemetry trace context injected into the message attributes.

The response includes the URL record with its generated id field. Copy this ID for the next command.

• Wait about 5 seconds for the Processor Lambda to pick up the SQS message and perform the health check.
• Check the health status of your registered URL by running:

curl [[API_ENDPOINT="https://<api-id>.execute-api.us-east-1.amazonaws.com/Prod"]]/urls/[[URL_ID="the-id-from-the-POST-response"]]/status

• You should see a JSON response with Status set to healthy since example.com returns a 200 response.

What happened behind the scenes?

The API Lambda enqueued a message to SQS. The Processor Lambda was triggered by the SQS event, extracted the OpenTelemetry trace context from the message attributes, performed an HTTP GET to https://example.com, and stored the result in DynamoDB.

This entire flow happened asynchronously. The API returned immediately after enqueuing, while the actual health check ran in the background.

Status still showing empty?

The Processor Lambda needs a few seconds to pick up the SQS message and write results. Wait 10 seconds and retry the GET request.

If the status remains empty, check your Processor Lambda's CloudWatch logs for errors. The SQS trigger may not be correctly configured, or the Lambda may be missing DynamoDB write permissions.

Help me debug why my Processor Lambda is not processing SQS messages.

Trigger the Step Functions state machine

The EventBridge rule triggers the orchestrator every 5 minutes automatically. For testing, you can start an execution manually to see the workflow graph immediately.

• Get the state machine ARN from your stack outputs by running:

aws cloudformation describe-stacks --stack-name url-health-monitor \
  --query "Stacks[0].Outputs[?OutputKey=='HealthCheckStateMachineArn'].OutputValue" \
  --output text

What does this command do?

This queries the CloudFormation stack for the specific output named HealthCheckStateMachineArn. The --query flag uses JMESPath to filter directly to the ARN value so you can copy it easily.

• Start a manual execution of the state machine by running:

aws stepfunctions start-execution \
  --state-machine-arn [[STATE_MACHINE_ARN="arn:aws:states:us-east-1:123456789012:stateMachine:HealthCheckStateMachine-xxxx"]]

What does the state machine do?

The Step Functions state machine runs a durable workflow. It invokes the Orchestrator Lambda to list all registered URLs from DynamoDB, then fans out by sending individual SQS messages per URL to trigger parallel health checks.

Each state has retry policies with exponential backoff. If a step fails after 3 attempts, the workflow catches the error and transitions to a failure state instead of crashing silently.

• Open the AWS Console and navigate to the Step Functions service.
• Click State machines in the left sidebar.
• Click your HealthCheckStateMachine state machine.
• Click the most recent execution (the one you just triggered).
• Observe the execution graph showing the ListURLs, FanOut (Map), and AggregateResults states. Each state shows green for succeeded or orange/red for failed.

Execution failed immediately?

Check the failed state's input and output in the console. Common causes include the Orchestrator Lambda missing sqs:SendMessage permission or the StepFunctionsExecutionRole missing lambda:InvokeFunction permission.

Help me debug my Step Functions execution failure.

Explore the X-Ray service map

Every Lambda in this system has X-Ray active tracing enabled. Your code uses OpenTelemetry to propagate trace context across the SQS boundary. The X-Ray service map visualizes the entire distributed trace in one view.

• In the AWS Console, navigate to the CloudWatch service.
• In the left sidebar, expand X-Ray traces and click Service map.
• Set the time range to the last 15 minutes so your recent requests appear.

You should see nodes for API Gateway, your API Lambda, the SQS queue, the Processor Lambda, and DynamoDB. All are connected by edges showing request flow.

• Click any edge between nodes to see individual traces.
• Click a trace that spans from API Gateway through to DynamoDB. The trace timeline shows the full journey of a single request, including the SQS queue hop.

Why does the trace cross the SQS boundary?

HTTP-based propagation happens automatically via the traceparent header. SQS has no built-in header mechanism for trace context.

Your code explicitly injects trace context into SQS MessageAttributes when sending, and extracts it when receiving. This is the inject/extract pattern using the OTel TextMapPropagator. Without it, the Processor Lambda would start a disconnected trace instead of continuing the original one.

• Verify that the trace ID is the same on both sides of the SQS boundary. Click the Processor Lambda segment and confirm its parent trace matches the API Lambda's trace.

What about CloudWatch Logs?

Each Lambda emits structured JSON logs via log/slog. Navigate to CloudWatch then Log groups to see log groups for each function. Each log entry includes the Lambda request ID for correlation.

Service map is empty?

X-Ray traces can take 1-2 minutes to appear in the console. Make sure you set the time range to include your recent requests.

If no traces appear at all, confirm that Tracing: Active is set in your template.yaml for all Lambda functions. Also verify that the tracing.Init function is being called at the start of each handler.

Help me debug why X-Ray traces are not appearing.

Your serverless system is fully deployed, processing health checks asynchronously, and producing end-to-end distributed traces. Next up, you will add CloudWatch monitoring and alarms to detect failures automatically.

Add CloudWatch Monitoring and Alarms

Your health monitor can detect unhealthy URLs and store results in DynamoDB. The missing piece is proactive alerting.

Without alarms, you only discover failures by manually checking. In this step, you'll emit a custom CloudWatch metric every time a health check fails, then define alarms that trigger automatically when something goes wrong.

In this step, get ready to:

• Emit a custom CloudWatch metric from the Processor Lambda on failed health checks.
• Define two CloudWatch Alarms in the SAM template for failure detection.
• Deploy and verify the alarm transitions to ALARM state with a failing URL.

Emit a custom CloudWatch metric on failed health checks

The Processor Lambda already knows when a URL is unhealthy. You need it to publish that signal as a CloudWatch metric so alarms can watch it.

• Add the CloudWatch SDK dependency to your project by running:

go get github.com/aws/aws-sdk-go-v2/service/cloudwatch

What does this command do?

This adds the github.com/aws/aws-sdk-go-v2/service/cloudwatch package to your go.mod. It provides the PutMetricData API client you need to publish custom metrics from Go.

• Open cmd/processor/main.go and add the CloudWatch import alongside your existing imports:

"github.com/aws/aws-sdk-go-v2/service/cloudwatch"
"github.com/aws/aws-sdk-go-v2/service/cloudwatch/types"
"github.com/aws/aws-sdk-go-v2/aws"

What are these imports for?

The cloudwatch package provides the service client. The types subpackage gives you MetricDatum and Dimension structs for building metric data. The aws package provides helpers like aws.String() and aws.Float64() for pointer conversions.

• Create a CloudWatch client alongside your existing service clients (SQS, DynamoDB) in the Processor's initialization:

// Create CloudWatch client for emitting custom metrics
cwClient := cloudwatch.NewFromConfig(cfg)

• Inside the SQS event handler, after the checker.Check call returns a failed result, add the PutMetricData call to emit the failure metric:

// Emit CloudWatch metric on failed health check
if !result.Healthy {
	_, err := cwClient.PutMetricData(ctx, &cloudwatch.PutMetricDataInput{
		Namespace: aws.String("UrlHealthMonitor"),
		MetricData: []types.MetricDatum{
			{
				MetricName: aws.String("HealthCheckFailure"),
				Value:      aws.Float64(1.0),
				Unit:       types.StandardUnitCount,
				Dimensions: []types.Dimension{
					{
						Name:  aws.String("URL"),
						Value: aws.String(url),
					},
				},
			},
		},
	})
	if err != nil {
		slog.Error("failed to emit CloudWatch metric", "error", err)
	}
}

What does this code do?

When a health check fails, this publishes a data point with value 1.0 to the HealthCheckFailure metric in the UrlHealthMonitor namespace. The URL dimension identifies which URL failed.

CloudWatch Alarms can then watch this metric and trigger when the count crosses a threshold.

• Save cmd/processor/main.go.
• Run the tests to confirm the code compiles cleanly:

go build ./cmd/processor/

Seeing an import error?

Make sure you ran go get github.com/aws/aws-sdk-go-v2/service/cloudwatch from your project root. Check that go.mod now contains a line with github.com/aws/aws-sdk-go-v2/service/cloudwatch.

help me debug a Go import error for the CloudWatch SDK

Define CloudWatch Alarms in the SAM template

Custom metrics are only useful if something watches them. You'll define two alarms in your SAM template that deploy alongside the rest of your infrastructure.

• Open template.yaml and add the following two resources in the Resources section, below your existing resources:

  HealthCheckFailureAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: Triggers when a URL health check fails
      Namespace: UrlHealthMonitor
      MetricName: HealthCheckFailure
      Statistic: Sum
      Period: 300
      EvaluationPeriods: 1
      Threshold: 1
      ComparisonOperator: GreaterThanOrEqualToThreshold
      TreatMissingData: notBreaching

  DLQMessageAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: Triggers when messages appear in the Dead Letter Queue
      Namespace: AWS/SQS
      MetricName: ApproximateNumberOfMessagesVisible
      Statistic: Sum
      Period: 60
      EvaluationPeriods: 1
      Threshold: 1
      ComparisonOperator: GreaterThanOrEqualToThreshold
      Dimensions:
        - Name: QueueName
          Value: !GetAtt HealthCheckDLQ.QueueName

What do these alarms monitor?

HealthCheckFailureAlarm watches your custom UrlHealthMonitor/HealthCheckFailure metric. If the sum of failures in a 5-minute window reaches 1 or more, the alarm fires. TreatMissingData: notBreaching means that when no data arrives (no failed checks), the alarm stays in OK state.

DLQMessageAlarm watches the built-in SQS metric ApproximateNumberOfMessagesVisible on your Dead Letter Queue. If even one message lands in the DLQ (meaning a health check message failed 3 retries), the alarm fires within 60 seconds.

• You also need to grant the Processor Lambda permission to call cloudwatch:PutMetricData. Add this policy to your ProcessorFunction's Policies list in the SAM template:

        - Statement:
            - Effect: Allow
              Action:
                - cloudwatch:PutMetricData
              Resource: "*"

Why Resource: * for PutMetricData?

CloudWatch metrics do not have individual ARNs. The PutMetricData action requires Resource: "*" because you cannot scope it to a specific metric namespace in IAM. The security boundary comes from the namespace string you set in code.

• Save template.yaml.
• Build and deploy the updated stack by running:

sam build && sam deploy

What happens during this deploy?

SAM cross-compiles all three Lambda binaries for ARM64, packages them, and submits a CloudFormation changeset. The changeset adds two new AWS::CloudWatch::Alarm resources and updates the Processor Lambda's IAM policy. Your existing resources remain unchanged.

• Wait for the deploy to reach UPDATE_COMPLETE. You should see the new resources in the changeset output.

Deploy failing with a permissions error?

Ensure your deployment role has cloudwatch:PutMetricAlarm and cloudwatch:DescribeAlarms permissions. CloudFormation needs these to create alarm resources on your behalf.

help me fix a SAM deploy permissions error for CloudWatch Alarms

Verify the alarm fires with a failing URL

The alarms are deployed. Now you'll register a URL that always returns a non-200 response and watch the alarm transition to ALARM state.

• Register a URL that returns HTTP 500 by running this curl command (replace the API URL with your own):

curl -X POST [[API_ENDPOINT="https://<api-id>.execute-api.us-east-1.amazonaws.com/Prod"]]/urls \
  -H "Content-Type: application/json" \
  -d '{"url": "https://httpstat.us/500"'

Why httpstat.us/500?

This is a public test service that returns the HTTP status code you specify in the URL path. Requesting /500 always returns a 500 Internal Server Error, making it a reliable way to trigger your failure metric.

• Wait 1-2 minutes for the SQS message to trigger the Processor Lambda.
• Open the AWS Console and navigate to CloudWatch > Alarms.
• Find HealthCheckFailureAlarm in the list. Within 5 minutes, its state should transition from INSUFFICIENT_DATA to ALARM.

Why does it start in INSUFFICIENT_DATA?

When a CloudWatch Alarm is first created, no metric data has been received for the configured period yet. The alarm stays in INSUFFICIENT_DATA until a data point arrives. Once the Processor emits the failure metric, CloudWatch evaluates the threshold and transitions the alarm.

Alarm stuck in INSUFFICIENT_DATA?

Check that the Processor Lambda was actually invoked. Look at CloudWatch Logs for the Processor function. If no invocation occurred, check the SQS queue to confirm the message was delivered.

Also verify the namespace string in your Go code matches exactly. UrlHealthMonitor in the code must match UrlHealthMonitor in the alarm definition. Namespaces are case-sensitive.

help me debug why my CloudWatch alarm is stuck in INSUFFICIENT_DATA

Your health monitor now has teeth. When a URL goes down, CloudWatch automatically detects it. In a production system, you would attach an SNS topic to these alarms to send email or Slack notifications. Next up, the Secret Mission challenges you to build a full observability dashboard.

✔️ Awesome, I've got everything!

Great. Make sure you've saved all files and the deploy completed successfully.

ⓧ I'd like to double check the full code

Here is the metric emission code added to cmd/processor/main.go (the new section within the handler):

// Emit CloudWatch metric on failed health check
if !result.Healthy {
	_, err := cwClient.PutMetricData(ctx, &cloudwatch.PutMetricDataInput{
		Namespace: aws.String("UrlHealthMonitor"),
		MetricData: []types.MetricDatum{
			{
				MetricName: aws.String("HealthCheckFailure"),
				Value:      aws.Float64(1.0),
				Unit:       types.StandardUnitCount,
				Dimensions: []types.Dimension{
					{
						Name:  aws.String("URL"),
						Value: aws.String(url),
					},
				},
			},
		},
	})
	if err != nil {
		slog.Error("failed to emit CloudWatch metric", "error", err)
	}
}

Here are the alarm resources added to template.yaml:

  HealthCheckFailureAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: Triggers when a URL health check fails
      Namespace: UrlHealthMonitor
      MetricName: HealthCheckFailure
      Statistic: Sum
      Period: 300
      EvaluationPeriods: 1
      Threshold: 1
      ComparisonOperator: GreaterThanOrEqualToThreshold
      TreatMissingData: notBreaching

  DLQMessageAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: Triggers when messages appear in the Dead Letter Queue
      Namespace: AWS/SQS
      MetricName: ApproximateNumberOfMessagesVisible
      Statistic: Sum
      Period: 60
      EvaluationPeriods: 1
      Threshold: 1
      ComparisonOperator: GreaterThanOrEqualToThreshold
      Dimensions:
        - Name: QueueName
          Value: !GetAtt HealthCheckDLQ.QueueName

Secret mission

Add a CloudWatch Dashboard with Custom Metrics

Your health monitor works, but can you see the big picture at a glance? In this secret mission, you will build a CloudWatch Dashboard that displays real-time health check latency percentiles, error rates, SQS queue depth, and Step Functions execution metrics across all monitored URLs.

Clean Up Your Resources

Clean Up Your Resources

This project runs entirely within the AWS Free Tier for low usage. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

• 3 Lambda functions on ARM64 (free tier: 1M requests/month).
• 1 DynamoDB table (free tier: 25 RCU, 25 WCU, 25 GB storage).
• 1 API Gateway REST API (free tier: 1M API calls/month for 12 months).
• 1 SQS queue + 1 Dead Letter Queue (free tier: 1M SQS requests/month).
• 1 Step Functions state machine (free tier: 4,000 state transitions/month).
• 1 EventBridge rule (free: 14M events/month).
• 2 CloudWatch Alarms (free tier: 10 alarms).
• CloudWatch Logs (free tier: 5 GB ingestion/month).
• 1 GitHub repository (public, free).
• Local project files on your Desktop.

Keep everything running

No action needed. Choose this if you are still actively building or experimenting with the health monitor.

• All resources stay within the AWS Free Tier at low usage levels. No charges expected.
• The EventBridge rule triggers the Step Functions state machine every 5 minutes. This is well within the free tier allowance.
• Your GitHub repository remains available for future pushes and CI/CD runs.
• Go, AWS CLI, and AWS SAM CLI remain installed on your machine.

Pause - I'll come back to this later

Stop scheduled invocations to reduce free tier consumption while keeping all resources intact for later use.

• Disable the EventBridge rule to stop the scheduled health checks. Find your rule name in the EventBridge console (it contains your stack name) and run:
• The API Gateway endpoint stays live but idle. No charges accumulate without incoming requests.
• Your DynamoDB table, SQS queue, and Lambda functions remain provisioned but dormant.
• When you are ready to resume, re-enable the rule by replacing disable-rule with enable-rule in the same command.

aws events disable-rule --name [[EVENTBRIDGE_RULE_NAME="your-stack-EventBridge-rule-name"]]

Delete - I don't want to use this again

Remove all AWS resources and local project files. This is irreversible.

• Delete the entire CloudFormation stack (all Lambda functions, DynamoDB table, API Gateway, SQS queues, Step Functions, EventBridge rule, and CloudWatch Alarms) by running this command from your project directory:

sam delete --stack-name url-health-monitor

What does this command do?

The sam delete command removes the CloudFormation stack, all packaged artifacts in S3, and the deployed template. It will prompt you to confirm before deleting.

• Delete the CloudWatch log groups that SAM does not remove automatically. Open the CloudWatch console, select **Log groups** in the left navigation, select all log groups starting with /aws/lambda/url-health-monitor, and choose **Actions** then **Delete log group(s)**.
• Delete your local project files by running:

macOS

rm -rf ~/Desktop/url-health-monitor

Windows

Remove-Item -Recurse -Force ~\Desktop\url-health-monitor

• Your GitHub repository will remain unless you delete it manually. To delete it, go to your repository on GitHub, click **Settings**, scroll to **Danger Zone**, and click **Delete this repository**.

That's a Wrap!

That's a Wrap!

Nice work! You just built and shipped a production-grade event-driven serverless system in Go that touches every skill hiring managers look for in cloud and backend engineers.

You've learned how to:

• Built three Go Lambda functions on ARM64 (Graviton2) with full OpenTelemetry instrumentation, deploying the entire stack via AWS SAM as Infrastructure as Code.
• Implemented event-driven async processing with SQS, Dead Letter Queues, and Step Functions durable orchestration, with OpenTelemetry trace context propagation across message boundaries.
• Created a complete CI/CD pipeline in GitHub Actions using OIDC keyless authentication for secure, automated deployments to AWS.
• Added a CloudWatch Dashboard with latency percentiles, SQS queue depth, and Step Functions metrics for real-time observability across the entire system.

Ready to quiz yourself?