Event-Driven Order Pipeline with Terraform

Build a serverless order processing pipeline using Terraform, Step Functions, and EventBridge Pipes.

Introduction

30 Second Summary

When you buy something online, a chain of invisible steps fires off behind the scenes. Your order gets checked, your payment goes through, and someone starts packing your box.

In this project, you will build an order processing pipeline on AWS using Terraform. A single API call triggers a multi-step workflow that validates, charges, and fulfills each order automatically.

What You'll Build

Picture posting a JSON order to a live API Gateway endpoint and watching it cascade through validation, payment, and fulfillment inside the Step Functions console.

By the end of this project, you'll have:

  • A live API endpoint you can POST orders to, triggering an entire processing workflow from validation through fulfillment.
  • A real-time console view of each order moving through processing states in Step Functions, including how the pipeline catches errors and handles them gracefully.
  • A zero-code integration connecting DynamoDB Streams to your workflow via EventBridge Pipes, replacing the typical glue Lambda pattern.
  • Secret Mission: an optional challenge to push your skills further.

Are there any prerequisites?

This project assumes you have an AWS account with the AWS CLI configured and Terraform installed. You should also be comfortable with AWS Lambda and API Gateway basics.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters for your serverless skills.

Set Up the Terraform Project

Your order processing pipeline will span several AWS services, from DynamoDB to Lambda to Step Functions. Before any of those resources can exist in the cloud, Terraform needs a properly configured project directory with your AWS provider settings.

In this step, you'll create the order-pipeline project directory, write four Python Lambda functions for different stages of order processing, and run terraform init to download the AWS provider plugin.

In this step, get ready to:
  • Create the order-pipeline directory with a lambda/ subdirectory.
  • Define Terraform configuration files and four Python Lambda functions.
  • Initialize Terraform and download the AWS provider plugin.
Create the project directory and Terraform files

Every Terraform project lives in its own directory. The .tf files inside define your provider, variables, and outputs.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal and press Enter to open it.
  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder with a lambda subdirectory, then move into it by running these commands:
mkdir -p order-pipeline/lambda
cd order-pipeline
  • Confirm the directory structure by running:
ls

You should see a lambda folder listed in the output.

  • Create a file called main.tf in your order-pipeline folder with the following content:
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
}

provider "aws" {
  region = var.aws_region
}

What does this code do?

  • The required_providers block tells Terraform to download the hashicorp/aws provider. The ~> 6.0 constraint means any version in the 6.x range.
  • The provider block configures AWS to use whatever region you define in var.aws_region. You'll define that variable next.
  • Create a file called variables.tf in the same folder with the following content:
variable "aws_region" {
  description = "AWS region to deploy resources"
  type        = string
  default     = "us-east-1"
}

variable "project_name" {
  description = "Project name used for resource naming"
  type        = string
  default     = "order-pipeline"
}

What does this code do?

  • aws_region defaults to us-east-1. This is the region where all your resources will be created.
  • project_name defaults to order-pipeline. Terraform will use this to prefix resource names so they stay organized.
  • Save both main.tf and variables.tf.
  • Confirm the files exist by running:
ls

You should see lambda, main.tf, and variables.tf listed.

  • Create a file called outputs.tf with the following content:
output "api_endpoint" {
  description = "API Gateway endpoint URL for creating orders"
  value       = "${aws_apigatewayv2_api.orders_api.api_endpoint}/${aws_apigatewayv2_stage.default.name}/orders"
}

output "state_machine_arn" {
  description = "ARN of the Step Functions state machine"
  value       = aws_sfn_state_machine.order_processing.arn
}

output "dynamodb_table_name" {
  description = "Name of the DynamoDB orders table"
  value       = aws_dynamodb_table.orders.name
}

What does this code do?

Outputs expose important values after Terraform applies your configuration. These three outputs will print the API endpoint URL, the Step Functions state machine ARN, and the DynamoDB table name.

The resources these outputs reference (like aws_apigatewayv2_api.orders_api) don't exist yet. That's fine. Terraform only evaluates outputs during terraform apply, not during terraform init.

  • Save outputs.tf.
  • Confirm all three Terraform files exist by running:
ls

You should see lambda, main.tf, outputs.tf, and variables.tf listed.

Missing a file?

  • Make sure you created each file inside the order-pipeline folder, not inside the lambda subfolder. Run pwd to check your current directory.
  • Check that each filename ends with .tf. Terraform only reads files with that extension.

Ask for help: My Terraform files aren't showing up in the right directory. Can you help me figure out where they went?

✔️ Awesome, I've got everything!

Great. Double check you've saved all three files before moving on.

ⓧ I'd like to double check the full code

Here are all three Terraform files. Compare each one against yours.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
}

provider "aws" {
  region = var.aws_region
}
variable "aws_region" {
  description = "AWS region to deploy resources"
  type        = string
  default     = "us-east-1"
}

variable "project_name" {
  description = "Project name used for resource naming"
  type        = string
  default     = "order-pipeline"
}
output "api_endpoint" {
  description = "API Gateway endpoint URL for creating orders"
  value       = "${aws_apigatewayv2_api.orders_api.api_endpoint}/${aws_apigatewayv2_stage.default.name}/orders"
}

output "state_machine_arn" {
  description = "ARN of the Step Functions state machine"
  value       = aws_sfn_state_machine.order_processing.arn
}

output "dynamodb_table_name" {
  description = "Name of the DynamoDB orders table"
  value       = aws_dynamodb_table.orders.name
}
Create the Lambda function source files

Your pipeline processes orders through four stages: creation, validation, payment, and fulfillment. Each stage is handled by its own Lambda function.

  • Create lambda/create_order.py with the following content:
import json
import boto3
import uuid
import os

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])


def handler(event, context):
    body = json.loads(event.get("body", "{}"))

    order_id = str(uuid.uuid4())
    order = {
        "order_id": order_id,
        "customer": body.get("customer", "unknown"),
        "item": body.get("item", "unknown"),
        "quantity": body.get("quantity", 1),
        "status": "RECEIVED",
    }

    table.put_item(Item=order)

    return {
        "statusCode": 200,
        "body": json.dumps({"message": "Order created", "order_id": order_id}),
    }

What does this function do?

  • This function receives POST requests from API Gateway, parses the JSON body, and generates a unique order_id using Python's uuid library.
  • It then writes the order to DynamoDB with a status of RECEIVED and returns the new order_id to the caller.
  • Create lambda/validate_order.py with the following content:
import json


def handler(event, context):
    records = event if isinstance(event, list) else [event]

    for record in records:
        dynamodb_record = record.get("dynamodb", {})
        new_image = dynamodb_record.get("NewImage", {})

        item = new_image.get("item", {}).get("S", "unknown")
        customer = new_image.get("customer", {}).get("S", "unknown")
        quantity = new_image.get("quantity", {}).get("N", "0")

        if item == "INVALID":
            raise ValueError(f"Invalid item in order for customer {customer}")

        if int(quantity) <= 0:
            raise ValueError(f"Invalid quantity {quantity} for customer {customer}")

    return {
        "statusCode": 200,
        "body": json.dumps({"status": "VALIDATED", "item": item, "customer": customer}),
    }

What does this function do?

  • This function receives DynamoDB stream records and extracts the order data from the NewImage field. DynamoDB stores values with type indicators like S for strings and N for numbers.
  • If the item is INVALID or the quantity is zero or negative, it raises a ValueError. You'll use this intentional error path later to test error handling.
  • Save both files.
  • Confirm they exist by running:
ls lambda/

You should see create_order.py and validate_order.py listed.

  • Create lambda/process_payment.py with the following content:
import json


def handler(event, context):
    body = event.get("body")
    if body and isinstance(body, str):
        data = json.loads(body)
    elif isinstance(event, dict) and "customer" in event:
        data = event
    else:
        data = event

    return {
        "statusCode": 200,
        "body": json.dumps({
            "status": "PAYMENT_PROCESSED",
            "message": "Payment successfully processed",
        }),
    }

What does this function do?

This function simulates a payment step. It handles different event formats (from API Gateway or from Step Functions) and returns a PAYMENT_PROCESSED status. In a real system, this is where you'd integrate with a payment provider like Stripe.

  • Create lambda/fulfill_order.py with the following content:
import json


def handler(event, context):
    return {
        "statusCode": 200,
        "body": json.dumps({
            "status": "FULFILLED",
            "message": "Order has been fulfilled and is ready for shipping",
        }),
    }

What does this function do?

This function simulates the final fulfillment stage. It returns a FULFILLED status indicating the order is ready for shipping. Like the payment function, a production version would connect to a real fulfillment system.

  • Save both files.
  • Confirm all four Lambda files exist by running:
ls lambda/

You should see create_order.py, fulfill_order.py, process_payment.py, and validate_order.py listed.

Missing a Lambda file?

  • Make sure you created the files inside the lambda/ subdirectory, not in the root order-pipeline folder.
  • Check that each filename matches exactly, including underscores. For example, create_order.py not createorder.py.

Ask for help: My Lambda files aren't showing up in the lambda/ directory. Can you help me check the file paths?

✔️ Awesome, I've got everything!

Great. Double check you've saved all four Lambda files before moving on.

ⓧ I'd like to double check the full code

Here are all four Lambda function files. Compare each one against yours.

import json
import boto3
import uuid
import os

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])


def handler(event, context):
    body = json.loads(event.get("body", "{}"))

    order_id = str(uuid.uuid4())
    order = {
        "order_id": order_id,
        "customer": body.get("customer", "unknown"),
        "item": body.get("item", "unknown"),
        "quantity": body.get("quantity", 1),
        "status": "RECEIVED",
    }

    table.put_item(Item=order)

    return {
        "statusCode": 200,
        "body": json.dumps({"message": "Order created", "order_id": order_id}),
    }
import json


def handler(event, context):
    records = event if isinstance(event, list) else [event]

    for record in records:
        dynamodb_record = record.get("dynamodb", {})
        new_image = dynamodb_record.get("NewImage", {})

        item = new_image.get("item", {}).get("S", "unknown")
        customer = new_image.get("customer", {}).get("S", "unknown")
        quantity = new_image.get("quantity", {}).get("N", "0")

        if item == "INVALID":
            raise ValueError(f"Invalid item in order for customer {customer}")

        if int(quantity) <= 0:
            raise ValueError(f"Invalid quantity {quantity} for customer {customer}")

    return {
        "statusCode": 200,
        "body": json.dumps({"status": "VALIDATED", "item": item, "customer": customer}),
    }
import json


def handler(event, context):
    body = event.get("body")
    if body and isinstance(body, str):
        data = json.loads(body)
    elif isinstance(event, dict) and "customer" in event:
        data = event
    else:
        data = event

    return {
        "statusCode": 200,
        "body": json.dumps({
            "status": "PAYMENT_PROCESSED",
            "message": "Payment successfully processed",
        }),
    }
import json


def handler(event, context):
    return {
        "statusCode": 200,
        "body": json.dumps({
            "status": "FULFILLED",
            "message": "Order has been fulfilled and is ready for shipping",
        }),
    }
Initialize Terraform

With all your files in place, Terraform needs to scan your configuration and download the AWS provider plugin before you can deploy anything.

Before you run the next command, what do you think terraform init does? Does it create AWS resources, or does it set up your local project?

  • Initialize your Terraform project by running:
terraform init

You should see Terraform has been successfully initialized! in the output. Terraform downloaded the hashicorp/aws provider plugin but did not create any cloud resources.

What did terraform init do?

The terraform init command reads your main.tf file, finds the hashicorp/aws provider requirement, and downloads the provider plugin to a local .terraform directory.

It also creates a .terraform.lock.hcl file that locks the exact provider version. This ensures everyone working on the project uses the same version.

Seeing an error during init?

  • If you see a provider-related error, check that main.tf has source = "hashicorp/aws" spelled exactly as shown. A typo in the provider name will cause the download to fail.
  • If you see a credentials error, confirm your AWS CLI is configured by running aws sts get-caller-identity. You should see your account ID in the output.

Ask for help: I'm getting an error when running terraform init. Can you help me debug it?

Your Terraform project is scaffolded and initialized. Next up, you'll define the DynamoDB table, Lambda functions, and API Gateway resources to get a working API that accepts orders.

Deploy DynamoDB, Lambda, and API Gateway

You have a Terraform project with provider configuration and four Lambda source files ready to go. Now it is time to define the AWS infrastructure those files will run on.

Before any order processing can happen, you need somewhere to store orders and a way for the outside world to send them in. In this step, you will define a DynamoDB table, four Lambda functions, and an API Gateway HTTP API, then deploy everything with a single terraform apply.

In this step, get ready to:
  • Define the DynamoDB table, Lambda functions, and IAM roles in Terraform.
  • Define the API Gateway HTTP API with a POST /orders route.
  • Deploy the infrastructure and test the API with a real order.
Define the DynamoDB table, Lambda functions, and IAM roles

The orders table is the foundation of the pipeline. Every order lands here first, and later steps will read from this table's change stream to trigger processing automatically.

  • Create a new file called dynamodb.tf inside your order-pipeline folder, and paste in the following code:
# Orders table with on-demand billing and streams for event processing
resource "aws_dynamodb_table" "orders" {
  name             = "${var.project_name}-orders"
  billing_mode     = "PAY_PER_REQUEST"
  hash_key         = "order_id"
  stream_enabled   = true
  stream_view_type = "NEW_IMAGE"

  attribute {
    name = "order_id"
    type = "S"
  }
}

What does this code do?

This creates a DynamoDB table named order-pipeline-orders with on-demand pricing, so you only pay for reads and writes you actually use.

The stream_enabled and stream_view_type settings turn on DynamoDB Streams, which captures every change to the table. You will wire these streams to Step Functions in a later step.

  • Save dynamodb.tf.

✔️ Awesome, I've got everything!

Great. Make sure you have saved dynamodb.tf before continuing.

ⓧ I'd like to double check the full code

resource "aws_dynamodb_table" "orders" {
  name             = "${var.project_name}-orders"
  billing_mode     = "PAY_PER_REQUEST"
  hash_key         = "order_id"
  stream_enabled   = true
  stream_view_type = "NEW_IMAGE"

  attribute {
    name = "order_id"
    type = "S"
  }
}

Each Lambda function's Python source file needs to be zipped before Terraform can upload it to AWS. Terraform's archive_file data source handles this automatically.

  • Create a new file called lambda.tf inside your order-pipeline folder, and paste in the following code:
# Package each Lambda function's source code into a zip for deployment
data "archive_file" "create_order" {
  type        = "zip"
  source_file = "${path.module}/lambda/create_order.py"
  output_path = "${path.module}/lambda/create_order.zip"
}

data "archive_file" "validate_order" {
  type        = "zip"
  source_file = "${path.module}/lambda/validate_order.py"
  output_path = "${path.module}/lambda/validate_order.zip"
}

data "archive_file" "process_payment" {
  type        = "zip"
  source_file = "${path.module}/lambda/process_payment.py"
  output_path = "${path.module}/lambda/process_payment.zip"
}

data "archive_file" "fulfill_order" {
  type        = "zip"
  source_file = "${path.module}/lambda/fulfill_order.py"
  output_path = "${path.module}/lambda/fulfill_order.zip"
}

What are archive_file data sources?

A data source in Terraform reads information without creating infrastructure. Here, archive_file zips each .py file into a .zip that the Lambda resource will upload to AWS.

  • Save lambda.tf.
  • Confirm both new files exist by running:
ls *.tf

You should see dynamodb.tf and lambda.tf alongside main.tf, outputs.tf, and variables.tf.

The Lambda functions need an IAM role to run under. This role grants them permission to write logs and access DynamoDB.

  • In lambda.tf, add the following code below the archive blocks:
# Shared execution role for all Lambda functions
resource "aws_iam_role" "lambda_role" {
  name = "${var.project_name}-lambda-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "lambda.amazonaws.com"
      }
    }]
  })
}

# Attach the managed policy for CloudWatch logging
resource "aws_iam_role_policy_attachment" "lambda_basic" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

What does this code do?

The aws_iam_role creates a role that Lambda can assume. The assume_role_policy tells AWS that only the Lambda service is allowed to use this role.

The aws_iam_role_policy_attachment connects the AWS-managed AWSLambdaBasicExecutionRole policy, which grants permission to write logs to CloudWatch.

  • Still in lambda.tf, add this inline policy below the attachment to grant DynamoDB access:
# Grant Lambda functions permission to read and write the orders table
resource "aws_iam_role_policy" "lambda_dynamodb" {
  name = "${var.project_name}-lambda-dynamodb"
  role = aws_iam_role.lambda_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Action = [
        "dynamodb:PutItem",
        "dynamodb:GetItem",
        "dynamodb:UpdateItem"
      ]
      Resource = aws_dynamodb_table.orders.arn
    }]
  })
}

Why an inline policy instead of another managed policy?

The DynamoDB permissions are scoped to one specific table (aws_dynamodb_table.orders.arn). An inline policy lets you restrict access to exactly the resource you need, following the principle of least privilege.

  • Save lambda.tf and format it by running:
terraform fmt lambda.tf

You should see lambda.tf printed in the output, confirming the file has valid HCL syntax.

Seeing a syntax error?

  • Make sure every opening { has a matching closing }. Nested jsonencode blocks are easy to mis-close.
  • Check that each resource and data block starts on its own line with no extra characters before it.
  • help me debug a terraform fmt error in my lambda.tf

Now define the Lambda function resources that use the role and archive files you just created.

  • In lambda.tf, add the create_order Lambda function below the DynamoDB policy:
# The create_order function writes incoming orders to DynamoDB
resource "aws_lambda_function" "create_order" {
  filename         = data.archive_file.create_order.output_path
  function_name    = "${var.project_name}-create-order"
  role             = aws_iam_role.lambda_role.arn
  handler          = "create_order.handler"
  runtime          = "python3.13"
  source_code_hash = data.archive_file.create_order.output_base64sha256

  environment {
    variables = {
      TABLE_NAME = aws_dynamodb_table.orders.name
    }
  }
}

Why does this function have an environment block?

The create_order function is the only Lambda that writes directly to DynamoDB. It needs the TABLE_NAME environment variable so the Python code can find the right table at runtime. The other three functions receive their data from Step Functions and do not interact with DynamoDB directly.

  • Add the remaining three Lambda functions below create_order in lambda.tf:
# These three functions will be invoked by Step Functions later
resource "aws_lambda_function" "validate_order" {
  filename         = data.archive_file.validate_order.output_path
  function_name    = "${var.project_name}-validate-order"
  role             = aws_iam_role.lambda_role.arn
  handler          = "validate_order.handler"
  runtime          = "python3.13"
  source_code_hash = data.archive_file.validate_order.output_base64sha256
}

resource "aws_lambda_function" "process_payment" {
  filename         = data.archive_file.process_payment.output_path
  function_name    = "${var.project_name}-process-payment"
  role             = aws_iam_role.lambda_role.arn
  handler          = "process_payment.handler"
  runtime          = "python3.13"
  source_code_hash = data.archive_file.process_payment.output_base64sha256
}

resource "aws_lambda_function" "fulfill_order" {
  filename         = data.archive_file.fulfill_order.output_path
  function_name    = "${var.project_name}-fulfill-order"
  role             = aws_iam_role.lambda_role.arn
  handler          = "fulfill_order.handler"
  runtime          = "python3.13"
  source_code_hash = data.archive_file.fulfill_order.output_base64sha256
}

What does this code do?

These three functions handle validation, payment, and fulfillment. They share the same IAM role and runtime as create_order, but they do not need the TABLE_NAME environment variable because they do not write to DynamoDB.

The source_code_hash attribute tells Terraform to redeploy the function whenever the source file changes.

  • Save lambda.tf and confirm the file is still valid by running:
terraform fmt lambda.tf

You should see lambda.tf printed again, confirming all the new code is valid HCL.

✔️ Awesome, I've got everything!

Great. Make sure lambda.tf is saved before continuing.

ⓧ I'd like to double check the full code

Your lambda.tf is too long to show in one block, so scroll up and confirm you have these sections in order:

  • Four archive_file data blocks (create_order, validate_order, process_payment, fulfill_order).
  • One aws_iam_role named lambda_role.
  • One aws_iam_role_policy_attachment named lambda_basic.
  • One aws_iam_role_policy named lambda_dynamodb.
  • Four aws_lambda_function resources (create_order with an environment block, then validate_order, process_payment, fulfill_order without one).
Define the API Gateway HTTP API and integration

With the Lambda functions defined, you need a public-facing endpoint that routes incoming HTTP requests to the create_order function.

  • Create a new file called api_gateway.tf inside your order-pipeline folder, and paste in the following code:
# HTTP API for receiving order requests
resource "aws_apigatewayv2_api" "orders_api" {
  name          = "${var.project_name}-api"
  protocol_type = "HTTP"
}

# Auto-deploying stage so changes go live immediately
resource "aws_apigatewayv2_stage" "default" {
  api_id      = aws_apigatewayv2_api.orders_api.id
  name        = "$default"
  auto_deploy = true
}

# Proxy integration forwards the full request to Lambda
resource "aws_apigatewayv2_integration" "create_order" {
  api_id                 = aws_apigatewayv2_api.orders_api.id
  integration_type       = "AWS_PROXY"
  integration_uri        = aws_lambda_function.create_order.invoke_arn
  payload_format_version = "2.0"
}

What does this code do?

The aws_apigatewayv2_api creates an HTTP API. HTTP APIs are simpler and cheaper than REST APIs, which is ideal for this pipeline.

The $default stage with auto_deploy = true means every change you make goes live immediately without a manual deploy step. The AWS_PROXY integration passes the full HTTP request to your Lambda function.

  • Add the route and Lambda permission below the integration in api_gateway.tf:
# Map POST /orders requests to the integration
resource "aws_apigatewayv2_route" "create_order" {
  api_id    = aws_apigatewayv2_api.orders_api.id
  route_key = "POST /orders"
  target    = "integrations/${aws_apigatewayv2_integration.create_order.id}"
}

# Grant API Gateway permission to invoke the create_order Lambda
resource "aws_lambda_permission" "api_gateway" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.create_order.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "${aws_apigatewayv2_api.orders_api.execution_arn}/*/*"
}

What does this code do?

The route tells API Gateway to forward any POST /orders request to the Lambda integration you defined above.

The aws_lambda_permission is required because Lambda blocks invocations from other AWS services by default. Without this resource-based policy, API Gateway would receive a 403 when trying to call your function.

  • Save api_gateway.tf and format all your Terraform files by running:
terraform fmt .

You should see the names of any files that were reformatted.

Seeing a syntax error from terraform fmt?

  • Check api_gateway.tf for missing closing braces. The source_arn line uses string interpolation with ${}, which requires the entire value to be inside double quotes.
  • Make sure the route_key value is exactly "POST /orders" with a space between the method and path.
  • help me debug a terraform fmt error in my api_gateway.tf

✔️ Awesome, I've got everything!

Great. Confirm api_gateway.tf is saved before continuing.

ⓧ I'd like to double check the full code

Scroll up and confirm your api_gateway.tf has these five resources in order:

  • aws_apigatewayv2_api named orders_api with protocol_type = "HTTP".
  • aws_apigatewayv2_stage named default with auto_deploy = true.
  • aws_apigatewayv2_integration named create_order with integration_type = "AWS_PROXY".
  • aws_apigatewayv2_route named create_order with route_key = "POST /orders".
  • aws_lambda_permission named api_gateway with principal = "apigateway.amazonaws.com".
Apply and test the API

All the Terraform files are in place. Before you can apply, there is one thing to handle.

Your outputs.tf file references a Step Functions state machine that you have not created yet. Terraform will refuse to plan if any output references an undeclared resource.

  • Open outputs.tf and find this block:
output "state_machine_arn" {
  description = "ARN of the Step Functions state machine"
  value       = aws_sfn_state_machine.order_processing.arn
}
  • Comment it out by adding # at the start of each line, so it looks like this:
# output "state_machine_arn" {
#   description = "ARN of the Step Functions state machine"
#   value       = aws_sfn_state_machine.order_processing.arn
# }
  • Save outputs.tf. You will uncomment this block in the next step when you create the state machine.
  • Deploy all your infrastructure by running:
terraform apply

What should I see?

Terraform displays a plan listing every resource it will create. You should see around 10-12 resources to add, including the DynamoDB table, IAM role, four Lambda functions, and the API Gateway resources.

Terraform will prompt you with Do you want to perform these actions? Type yes and press Enter. The apply takes about 1-2 minutes.

Once the apply finishes, Terraform prints the output values including your API endpoint URL.

  • Copy your API endpoint by running:
terraform output api_endpoint

You should see a URL like https://xxxxxxxx.execute-api.us-east-1.amazonaws.com/$default/orders.

Before you send a request, what do you think will happen when you POST a JSON order to this URL? Will the response include the order ID?

  • Send a test order by running:
curl -X POST "$(terraform output -raw api_endpoint)" \
  -H "Content-Type: application/json" \
  -d '{"customer": "Jane", "item": "Widget", "quantity": 2}'

You should see a JSON response confirming the order was created, with a unique order_id:

{"message": "Order created", "order_id": "a1b2c3d4-..."}

What just happened?

Your curl request hit API Gateway, which forwarded it to the create_order Lambda function. The function generated a UUID, built an order object, wrote it to DynamoDB, and returned the order ID.

The order is in Lambda's response, but is it actually in DynamoDB?

  • Verify the order was stored by scanning the table:
aws dynamodb scan --table-name $(terraform output -raw dynamodb_table_name)

You should see your order item in the Items array, with customer set to Jane, item set to Widget, and status set to RECEIVED.

Curl returning an error?

  • If you see Internal Server Error, check your Lambda function logs in CloudWatch. The most common cause is a typo in the TABLE_NAME environment variable or missing DynamoDB permissions.
  • If you see Not Found, verify that terraform output api_endpoint includes /orders at the end of the URL.
  • If the dynamodb scan returns empty Items, confirm the table name matches order-pipeline-orders and your CLI is configured for the same region as your Terraform provider.
  • help me debug my order pipeline API Gateway and Lambda setup

Now restore the output you commented out so it is ready for the next step.

  • In outputs.tf, find the commented-out state_machine_arn block:
# output "state_machine_arn" {
#   description = "ARN of the Step Functions state machine"
#   value       = aws_sfn_state_machine.order_processing.arn
# }
  • Remove the # from each line so the block is active again:
output "state_machine_arn" {
  description = "ARN of the Step Functions state machine"
  value       = aws_sfn_state_machine.order_processing.arn
}
  • Save outputs.tf. Do not run terraform apply yet. The state machine resource that this output references will be created in the next step.

Data is flowing from the outside world into DynamoDB. Next up, you will build a Step Functions state machine that processes each order through validation, payment, and fulfillment.

Build the Step Functions State Machine

Your API is accepting orders and storing them in DynamoDB. That part works. But right now those orders just sit there with a RECEIVED status, completely unprocessed.

You need an orchestration layer that picks up each order and runs it through validation, payment, and fulfillment in sequence. AWS Step Functions lets you define that multi-step workflow as a state machine. In this step, you'll build the state machine and test the workflow logic in isolation before wiring it to the data stream.

In this step, get ready to:
  • Define a Step Functions state machine with IAM permissions in Terraform.
  • Manually trigger a test execution and watch it complete in the console.
Define the Step Functions state machine in Terraform

Step Functions uses Amazon States Language (ASL) to define workflows as a series of states. Each state invokes a Lambda function, then passes its output to the next state in the chain.

  • Create a new file called step_functions.tf inside your order-pipeline folder.
  • Add the IAM role that Step Functions will assume by pasting this code:
# IAM role that Step Functions assumes to run the workflow
resource "aws_iam_role" "sfn_role" {
  name = "${var.project_name}-sfn-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "states.amazonaws.com"
      }
    }]
  })
}

What does this code do?

This creates an IAM role that the Step Functions service can assume. The states.amazonaws.com principal tells AWS that only Step Functions can use this role, following the same trust-policy pattern you used for the Lambda role in the previous step.

  • Below the role, add an IAM policy granting the state machine permission to invoke your three processing Lambda functions:
# Grant the state machine permission to invoke the processing Lambdas
resource "aws_iam_role_policy" "sfn_lambda" {
  name = "${var.project_name}-sfn-lambda"
  role = aws_iam_role.sfn_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Action = "lambda:InvokeFunction"
      Resource = [
        aws_lambda_function.validate_order.arn,
        aws_lambda_function.process_payment.arn,
        aws_lambda_function.fulfill_order.arn
      ]
    }]
  })
}

What does this code do?

This policy grants the state machine the lambda:InvokeFunction permission on exactly the three Lambda functions the workflow will call. The create_order Lambda is excluded because it handles API Gateway requests, not workflow steps.

  • Save step_functions.tf.
  • Confirm Terraform can parse the new IAM resources by running:
terraform plan

You should see a plan with 2 resources to add: the IAM role and the IAM policy.

Seeing a syntax error?

  • Check that the opening and closing braces in jsonencode({}) are balanced. Each { needs a matching } and each [{ needs a matching }].
  • Make sure the file is saved as step_functions.tf (not .tf.txt or another extension).
  • help me debug a Terraform plan error in step_functions.tf

Now you have IAM permissions sorted. Time to define the actual workflow.

  • Add the state machine resource below the IAM policy in step_functions.tf:
# Define the state machine using Amazon States Language
resource "aws_sfn_state_machine" "order_processing" {
  name     = "${var.project_name}-order-processing"
  role_arn = aws_iam_role.sfn_role.arn

  definition = jsonencode({
    Comment = "Order processing workflow"
    StartAt = "ValidateOrder"
    States = {
      ValidateOrder = {
        Type     = "Task"
        Resource = aws_lambda_function.validate_order.arn
        Next     = "ProcessPayment"
      }
      ProcessPayment = {
        Type     = "Task"
        Resource = aws_lambda_function.process_payment.arn
        Next     = "FulfillOrder"
      }
      FulfillOrder = {
        Type     = "Task"
        Resource = aws_lambda_function.fulfill_order.arn
        End      = true
      }
    }
  })
}

What is Amazon States Language?

ASL is a JSON-based language for defining Step Functions workflows. Each state in the States object is a step in the workflow. StartAt tells the state machine which state to run first.

Each Task state invokes a Lambda function via Resource, then hands off to the state named in Next. The final state uses End = true to mark the workflow as complete.

  • Save step_functions.tf.

✔️ Awesome, I've got everything!

Great. Make sure you've saved step_functions.tf with all three resource blocks before continuing.

ⓧ I'd like to double check the full code

Your step_functions.tf should contain three resource blocks in this order:

  • aws_iam_role "sfn_role" with the states.amazonaws.com service principal.
  • aws_iam_role_policy "sfn_lambda" granting lambda:InvokeFunction on the three processing Lambda ARNs.
  • aws_sfn_state_machine "order_processing" with an ASL definition chaining ValidateOrder, ProcessPayment, and FulfillOrder.

Scroll up and confirm each block matches the code you pasted earlier.

Test the state machine manually

The Terraform code defines the state machine, but it only exists in your files right now. You need to deploy it and run a test execution to prove the workflow logic works before wiring it to the data stream.

  • Deploy the state machine by running:
terraform apply -auto-approve

You should see 3 resources added: the IAM role, the IAM policy, and the state machine itself.

Seeing an error during apply?

  • If you see a permissions error, confirm your AWS CLI credentials are still active by running aws sts get-caller-identity.
  • If you see an invalid ASL definition error, check that every opening brace in jsonencode({}) has a matching closing brace.
  • help me debug a terraform apply error for Step Functions

The state machine is deployed. Now you'll trigger a test execution with a sample order payload that mimics what a DynamoDB stream record looks like.

Before you run the next command, what do you think will happen? Will all three states (ValidateOrder, ProcessPayment, FulfillOrder) complete successfully, or will one of them fail?

  • Start a test execution by running this command:
aws stepfunctions start-execution \
  --state-machine-arn $(terraform output -raw state_machine_arn) \
  --input '[{"dynamodb": {"NewImage": {"order_id": {"S": "test-001"}, "customer": {"S": "Jane"}, "item": {"S": "Widget"}, "quantity": {"N": "2"}, "status": {"S": "RECEIVED"}}}}]'

What does this command do?

  • aws stepfunctions start-execution triggers a new execution of the state machine.
  • --state-machine-arn uses terraform output to grab the ARN dynamically so you don't have to copy it.
  • --input passes a sample DynamoDB stream record. This mimics the format the validate Lambda expects, with the order data nested under dynamodb.NewImage.

You should see a JSON response with an executionArn and a startDate. This confirms the execution started. Now check the results in the console.

  • Open the AWS Management Console in your browser.
  • Search for Step Functions in the top search bar and open it.
  • Click on the order-pipeline-order-processing state machine.
  • Click on the most recent execution in the Executions list.

You should see the execution graph with all three states highlighted as successful: ValidateOrder, ProcessPayment, and FulfillOrder completed in sequence.

Why test manually first?

Testing the state machine in isolation proves the workflow logic and IAM permissions work correctly. If something breaks later when you wire up the DynamoDB stream, you'll know the issue is in the plumbing, not the workflow itself.

Execution shows a failed state?

  • Click on the failed state in the graph, then expand the Error section to see the exact error message.
  • If you see an access denied error, confirm the sfn_lambda policy lists all three Lambda ARNs in the Resource array.
  • If the --input JSON was malformed, try re-running the start-execution command carefully. Escaped quotes inside single quotes are required.
  • help me debug a failed Step Functions execution

The workflow logic is solid. ValidateOrder, ProcessPayment, and FulfillOrder all execute in sequence. But there's a gap. You still had to manually start the execution with a CLI command. In the next step, you'll close that gap by wiring DynamoDB Streams to Step Functions via EventBridge Pipes so every new order triggers the workflow automatically.

Wire DynamoDB Streams to Step Functions via EventBridge Pipes

In the last step, you proved your state machine can validate, process, and fulfill an order. But you had to trigger it manually with the AWS CLI every time.

In production, orders would pile up in DynamoDB while someone manually starts each execution. EventBridge Pipes fixes this by connecting DynamoDB Streams directly to Step Functions. When a new order lands in the table, the pipe automatically starts a state machine execution with zero custom code.

Why EventBridge Pipes instead of a glue Lambda?

Without EventBridge Pipes, you would write a Lambda function that reads from the DynamoDB stream and calls StartExecution on Step Functions. That is boilerplate code you have to maintain, version, and monitor.

EventBridge Pipes handles the same job as a managed service integration. No Lambda function to write, no deployment package to manage, and no concurrency limits to worry about.

In this step, get ready to:
  • Create an EventBridge Pipe with IAM permissions connecting DynamoDB Streams to Step Functions.
  • Deploy and test the fully automated end-to-end pipeline.
Create the EventBridge Pipe Terraform configuration

Your DynamoDB table already has streams enabled from the previous deployment. Every time a record is inserted, the stream captures the change event. Now you need to build the pipe that reads those events and forwards them to your state machine.

The pipe needs its own IAM role with two permissions: reading from the DynamoDB stream (source) and starting Step Functions executions (target).

  • Create a new file called pipes.tf in your order-pipeline project directory.
  • Add the IAM role that EventBridge Pipes will assume by pasting in this code:
# IAM role that EventBridge Pipes assumes to connect source and target
resource "aws_iam_role" "pipe_role" {
  name = "${var.project_name}-pipe-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "pipes.amazonaws.com"
      }
    }]
  })
}

What does this code do?

This creates an IAM role that the EventBridge Pipe will assume when it runs. The pipes.amazonaws.com service principal tells AWS that only EventBridge Pipes can use this role.

  • Below the role, add the source IAM policy that grants read access to the DynamoDB stream:
# Grant the pipe permission to read records from the DynamoDB stream
resource "aws_iam_role_policy" "pipe_source" {
  name = "${var.project_name}-pipe-source"
  role = aws_iam_role.pipe_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Action = [
        "dynamodb:DescribeStream",
        "dynamodb:GetRecords",
        "dynamodb:GetShardIterator",
        "dynamodb:ListStreams"
      ]
      Resource = aws_dynamodb_table.orders.stream_arn
    }]
  })
}

What does this policy allow?

These four DynamoDB Streams actions let the pipe discover shards, read records, and iterate through stream data. The policy is scoped to just your orders table stream ARN, following the principle of least privilege.

  • Save pipes.tf.
  • Verify Terraform can parse your new file by running:
terraform validate

You should see Success! The configuration is valid. in the output.

Seeing a validation error?

Check for missing commas between key-value pairs and unmatched braces in the jsonencode blocks. Each opening { and [ needs a matching close.

Help me fix a Terraform validation error in my pipes.tf IAM configuration

  • Below the source policy, add the target IAM policy that grants permission to start Step Functions executions:
# Grant the pipe permission to start Step Functions executions
resource "aws_iam_role_policy" "pipe_target" {
  name = "${var.project_name}-pipe-target"
  role = aws_iam_role.pipe_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect   = "Allow"
      Action   = "states:StartExecution"
      Resource = aws_sfn_state_machine.order_processing.arn
    }]
  })
}

What does this policy allow?

The states:StartExecution action is the single permission the pipe needs on the target side. It can start a new execution of your order-pipeline-order-processing state machine, but nothing else.

  • Below the target policy, add the pipe resource itself. This is the core of the integration:
# The pipe: reads INSERT events from DynamoDB Streams, triggers Step Functions
resource "aws_pipes_pipe" "orders_to_sfn" {
  depends_on = [aws_iam_role_policy.pipe_source, aws_iam_role_policy.pipe_target]
  name       = "${var.project_name}-orders-pipe"
  role_arn   = aws_iam_role.pipe_role.arn
  source     = aws_dynamodb_table.orders.stream_arn
  target     = aws_sfn_state_machine.order_processing.arn

  # Only pick up new records, process one at a time
  source_parameters {
    dynamodb_stream_parameters {
      starting_position = "LATEST"
      batch_size        = 1
    }

    # Only trigger on new orders (INSERT), not updates or deletes
    filter_criteria {
      filter {
        pattern = jsonencode({
          eventName = ["INSERT"]
        })
      }
    }
  }
}

What does this pipe do?

  • source points to the DynamoDB stream. Whenever a change event appears, the pipe picks it up.
  • target points to your Step Functions state machine. The pipe starts a new execution for each event.
  • starting_position = "LATEST" means the pipe only reads new events from this point forward, ignoring anything already in the stream.
  • The filter pattern {"eventName": ["INSERT"]} ensures only new orders trigger the workflow. Updates and deletes are ignored.
  • depends_on ensures the IAM policies are fully attached before the pipe tries to use them.
  • Save pipes.tf.

✔️ Awesome, I've got everything!

Great. Double check you have saved pipes.tf with all four resource blocks.

ⓧ I'd like to double check the full code

Scroll up and confirm your pipes.tf file contains these four blocks in order:

  • aws_iam_role "pipe_role" with the pipes.amazonaws.com service principal.
  • aws_iam_role_policy "pipe_source" with four DynamoDB Streams actions scoped to aws_dynamodb_table.orders.stream_arn.
  • aws_iam_role_policy "pipe_target" with states:StartExecution scoped to aws_sfn_state_machine.order_processing.arn.
  • aws_pipes_pipe "orders_to_sfn" with source, target, dynamodb_stream_parameters, and filter_criteria.
Deploy and test the end-to-end pipeline

Everything is defined. Time to deploy the pipe and see if orders flow through the pipeline automatically.

  • Deploy the new resources by running:
terraform apply

Terraform shows a plan to add 4 new resources (the IAM role, two policies, and the pipe). Type yes and press Enter to confirm.

Seeing a permissions error during apply?

If the pipe fails to create with an access denied error, make sure the depends_on line is present in the aws_pipes_pipe resource. Without it, Terraform may try to create the pipe before the IAM policies are attached.

Help me debug a Terraform apply error for my EventBridge Pipe

Once the apply completes, the pipe is live and listening for new DynamoDB stream events.

Before you test, think about what you expect to happen. You are about to POST an order to the same API endpoint you used before. Do you think a Step Functions execution will start on its own this time?

  • POST a new order to test the automatic trigger by running:
curl -X POST $(terraform output -raw api_endpoint) \
  -H "Content-Type: application/json" \
  -d '{"customer": "Jane", "item": "Keyboard", "quantity": 2}'

You should see a response like {"message": "Order created", "order_id": "..."}. That confirms the order landed in DynamoDB. Now check whether the pipe picked it up.

  • Open the AWS Management Console in your browser.
  • Search for Step Functions in the top search bar and select it.
  • Click the order-pipeline-order-processing state machine.
  • Look at the Executions list. You should see a new execution with a status of Succeeded that was triggered within the last minute.

That execution was started automatically by EventBridge Pipes. You did not run aws stepfunctions start-execution this time. The pipe read the INSERT event from the DynamoDB stream and kicked off the state machine on its own.

What should I see in the execution?

Click on the execution to see the visual workflow graph. All three states (ValidateOrder, ProcessPayment, FulfillOrder) should be green. The input tab shows the raw DynamoDB stream event that the pipe forwarded, which is why the validate_order Lambda parses the NewImage format.

No new execution appeared?

The pipe can take up to 30 seconds to detect and forward a new stream event. Wait a moment and refresh the page.

If nothing appears after a minute, check that the pipe is in an active state. Go to Amazon EventBridge in the console, select Pipes, and confirm order-pipeline-orders-pipe shows a status of Running.

Help me troubleshoot why my EventBridge Pipe is not triggering Step Functions

Your pipeline now runs end-to-end with zero manual intervention. Next up, you will make it production-ready by adding error handling and observability.

Add Error Handling and Observability

Your pipeline automatically processes every order that lands in DynamoDB. The happy path works beautifully.

But what happens when something goes wrong? A customer submits an order with missing fields. A payment service times out. Right now, any error crashes the entire workflow with no recovery and no way to see what happened. This step adds Retry and Catch blocks to the Step Functions definition, plus CloudWatch logging so you can debug failures in production.

In this step, get ready to:
  • Add Retry and Catch error handling to the Step Functions state machine definition.
  • Add CloudWatch logging for full execution visibility.
  • Test error handling by submitting an invalid order and watching it get caught gracefully.
Add Retry and Catch to the ASL definition

The Amazon States Language (ASL) definition that powers your state machine currently has no error handling. If any Lambda function throws an error, the entire execution fails immediately.

You will add two error-handling features. A Catch block on the ValidateOrder state redirects errors to a terminal failure state. A Retry block on the ProcessPayment state automatically retries transient failures before giving up.

  • In step_functions.tf, find the States = { block inside the definition argument of aws_sfn_state_machine.order_processing.
  • Find the ValidateOrder state. It currently looks like this:
      ValidateOrder = {
        Type     = "Task"
        Resource = aws_lambda_function.validate_order.arn
        Next     = "ProcessPayment"
      }
  • Replace the entire ValidateOrder block with this version that includes a Catch block:
      ValidateOrder = {
        Type     = "Task"
        Resource = aws_lambda_function.validate_order.arn
        Next     = "ProcessPayment"
        Catch = [{
          ErrorEquals = ["States.ALL"]
          Next        = "OrderFailed"
          ResultPath  = "$.error"
        }]
      }

What does the Catch block do?

When the validate_order Lambda raises any error, the Catch block intercepts it instead of crashing the whole workflow.

It stores the error details at $.error in the state output and redirects to OrderFailed, a terminal state you will add in a moment.

  • Now find the ProcessPayment block right below. It currently looks like this:
      ProcessPayment = {
        Type     = "Task"
        Resource = aws_lambda_function.process_payment.arn
        Next     = "FulfillOrder"
      }
  • Replace the entire ProcessPayment block with this version that includes a Retry block, and add the new OrderFailed state after the closing brace of FulfillOrder:
      ProcessPayment = {
        Type     = "Task"
        Resource = aws_lambda_function.process_payment.arn
        Next     = "FulfillOrder"
        Retry = [{
          ErrorEquals     = ["States.TaskFailed"]
          IntervalSeconds = 2
          MaxAttempts     = 2
          BackoffRate     = 2.0
        }]
      }
      FulfillOrder = {
        Type     = "Task"
        Resource = aws_lambda_function.fulfill_order.arn
        End      = true
      }
      OrderFailed = {
        Type  = "Fail"
        Error = "OrderValidationFailed"
        Cause = "The order failed validation"
      }

What do Retry and OrderFailed do?

The Retry block tells Step Functions to automatically retry the payment Lambda up to 2 times if it throws a States.TaskFailed error. It waits 2 seconds before the first retry and doubles the wait for each subsequent attempt (BackoffRate of 2.0).

The OrderFailed state is a terminal Fail state. When the Catch on ValidateOrder fires, the execution ends here with a clear error message instead of crashing silently.

  • Save step_functions.tf.
  • Verify the syntax is correct by running:
terraform validate

You should see Success! The configuration is valid. in your terminal.

Seeing a syntax error?

Check that every opening brace { and bracket [ has a matching closing brace or bracket. The Catch and Retry values use [{}] (an array of objects), which is easy to mistype.

Also confirm the OrderFailed block sits inside States = { at the same indentation level as ValidateOrder and FulfillOrder.

help me fix a Terraform syntax error in my Step Functions definition

Add CloudWatch logging and deploy

Error handling catches failures, but you also need visibility into what happened. A CloudWatch Log Group records every state transition, input, and output so you can debug production issues.

  • In step_functions.tf, add the following IAM policy after the existing aws_iam_role_policy "sfn_lambda" resource. This grants the state machine permission to write logs:
resource "aws_iam_role_policy" "sfn_logging" {
  name = "${var.project_name}-sfn-logging"
  role = aws_iam_role.sfn_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect = "Allow"
      Action = [
        "logs:CreateLogDelivery",
        "logs:CreateLogStream",
        "logs:GetLogDelivery",
        "logs:UpdateLogDelivery",
        "logs:DeleteLogDelivery",
        "logs:ListLogDeliveries",
        "logs:PutLogEvents",
        "logs:PutResourcePolicy",
        "logs:DescribeResourcePolicies",
        "logs:DescribeLogGroups"
      ]
      Resource = "*"
    }]
  })
}

Why does the logging policy use Resource = "*"?

Step Functions needs broad CloudWatch Logs permissions to create and manage log deliveries across your account. The actions like logs:CreateLogDelivery and logs:PutResourcePolicy operate at the account level, so they cannot be scoped to a single log group ARN.

  • Next, add the CloudWatch Log Group resource below the IAM policy you just added:
resource "aws_cloudwatch_log_group" "sfn_logs" {
  name              = "/aws/stepfunctions/${var.project_name}-order-processing"
  retention_in_days = 7
}
  • Now add a logging_configuration block to the aws_sfn_state_machine resource. Find the line role_arn = aws_iam_role.sfn_role.arn and add this block below it, before the definition argument:
  logging_configuration {
    log_destination        = "${aws_cloudwatch_log_group.sfn_logs.arn}:*"
    include_execution_data = true
    level                  = "ALL"
  }

What does the logging configuration do?

Setting level to ALL records every state transition. The include_execution_data flag captures the actual input and output at each step, so you can see exactly what data flowed through the pipeline.

The log destination ARN ends with :* because Step Functions requires the wildcard suffix to write to all log streams within the group.

  • Save step_functions.tf.
  • Deploy all the changes by running:
terraform apply
  • Review the plan. You should see resources to add (the log group and logging IAM policy) and a resource to update (the state machine). Type yes to confirm.

Wait for the apply to complete. You should see a message ending with Apply complete! showing resources added and changed.

Seeing an error during apply?

If Terraform reports a syntax error, run terraform validate first to pinpoint the exact file and line. Common issues include a missing comma between IAM policy actions or a misplaced logging_configuration block outside the aws_sfn_state_machine resource.

help me fix a Terraform apply error in my Step Functions logging configuration

Test error handling

The error handling is deployed. Time to prove it works by sending a deliberately bad order through the pipeline.

Your validate_order Lambda already raises a ValueError when it receives an item called INVALID. Before the Catch block, that error would have crashed the execution. Now it should route cleanly to the OrderFailed state.

Before you run the next command, take a moment to predict: what do you think will happen when you POST an order with "item": "INVALID"? Will the API return an error, or will the order still be created successfully?

  • Submit an invalid order by running:
curl -X POST $(terraform output -raw api_endpoint) \
  -H "Content-Type: application/json" \
  -d '{"customer": "Bob", "item": "INVALID", "quantity": 1}'

The API returns a 200 response with an order_id. The create_order Lambda accepts any order. The validation happens downstream in the state machine.

  • Open the AWS Step Functions console.
  • Click on the order-pipeline-order-processing state machine.
  • Look at the most recent execution. Click on it to open the execution details.

You should see the execution graph showing ValidateOrder highlighted in red, with the flow routed to OrderFailed instead of continuing to ProcessPayment. The error was caught gracefully.

What would have happened without Catch?

Without the Catch block, the ValueError from the validate Lambda would have caused the entire execution to fail with a generic States.TaskFailed error. There would be no clean error state, no structured error output, and no way to distinguish validation failures from other types of errors.

  • Now confirm the happy path still works. Submit a valid order by running:
curl -X POST $(terraform output -raw api_endpoint) \
  -H "Content-Type: application/json" \
  -d '{"customer": "Alice", "item": "Widget", "quantity": 2}'
  • Switch back to the Step Functions console and refresh the executions list.
  • Click on the newest execution. You should see it completed successfully through ValidateOrder, ProcessPayment, and FulfillOrder, all highlighted in green.
  • To check the logs, open the CloudWatch console.
  • Click Log groups in the left sidebar.
  • Find and click on /aws/stepfunctions/order-pipeline-order-processing.
  • Open the most recent log stream. You should see detailed JSON entries showing each state transition, including the input and output data at every step.

✔️ Awesome, I've got everything!

Great. Double check you have saved step_functions.tf with all the changes.

ⓧ I'd like to double check the full code

Your step_functions.tf file is now quite long. Scroll through it and confirm you have all of these resources in order:

  • aws_iam_role "sfn_role" with the states.amazonaws.com assume role policy.
  • aws_iam_role_policy "sfn_lambda" granting lambda:InvokeFunction on the three Lambda functions.
  • aws_iam_role_policy "sfn_logging" granting CloudWatch Logs permissions with Resource = "*".
  • aws_cloudwatch_log_group "sfn_logs" with retention_in_days = 7.
  • aws_sfn_state_machine "order_processing" with a logging_configuration block between role_arn and definition.
  • Inside the definition: ValidateOrder with Catch, ProcessPayment with Retry, FulfillOrder unchanged, and the new OrderFailed Fail state.

Your pipeline now handles errors gracefully and logs every state transition. Next up, you will take on the Secret Mission and extend the workflow even further.

Secret mission

Add a Parallel Notification Branch with SNS

Your pipeline processes orders end to end, but customers never hear about it. Add a Parallel state to your Step Functions workflow that fulfills orders and sends SNS notifications at the same time. This is the pattern real-world order systems use to keep customers in the loop.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

All the AWS resources in this project are serverless and cost nothing when idle. EventBridge Pipes has no free tier, but since it charges per request ($0.40 per million), zero traffic means zero charges.

Cost warning

While idle costs are effectively zero, any future orders you POST through the API will incur small charges from EventBridge Pipes ($0.40 per million requests) and Step Functions beyond the free tier of 4,000 state transitions per month. If you are done with the project, deleting everything is the safest way to avoid unexpected charges.

Resources you used:

  • DynamoDB table (order-pipeline-orders).
  • 4 Lambda functions (create-order, validate-order, process-payment, fulfill-order).
  • API Gateway HTTP API (order-pipeline-api).
  • Step Functions state machine (order-pipeline-order-processing).
  • EventBridge Pipe (order-pipeline-orders-pipe).
  • CloudWatch Log Group (/aws/stepfunctions/order-pipeline-order-processing).
  • IAM roles and policies (lambda-role, sfn-role, pipe-role).
  • SNS topic (order-pipeline-order-notifications) from the Secret Mission.
  • Local order-pipeline project directory on your Desktop.

Keep everything running

No action needed. Choose this if you are still actively building or want to experiment further with the pipeline.

  • All your resources are serverless. DynamoDB, Lambda, API Gateway, and Step Functions have generous free tiers that cover learning-level usage.
  • EventBridge Pipes charges $0.40 per million requests with no free tier, but with no traffic flowing through the pipe, you incur zero cost.
  • SNS has a free tier of 1 million publishes per month.
  • Your local order-pipeline directory stays on your Desktop for future reference.

Pause - I'll come back to this later

Serverless resources do not run continuously, so there is nothing to pause or stop. They only activate when a request arrives.

  • Your Lambda functions, Step Functions state machine, and EventBridge Pipe sit idle until an order is posted. There is no running process to shut down.
  • If you want to prevent any accidental charges, the only option is to delete the resources entirely using the Delete tab.

Delete - I don't want to use this again

Remove all project resources and start fresh. This is the recommended option after completing the project.

  • Open your terminal and navigate to the order-pipeline directory on your Desktop.
  • Destroy all AWS resources by running:
terraform destroy
  • Terraform displays a plan showing every resource it will delete. Type yes when prompted and press Enter to confirm.
  • Wait for Terraform to finish. You should see Destroy complete! followed by a count of resources destroyed.

What does terraform destroy remove?

This single command removes every AWS resource defined in your Terraform files: the DynamoDB table, all four Lambda functions, the API Gateway HTTP API, the Step Functions state machine, the EventBridge Pipe, the CloudWatch Log Group, all IAM roles and policies, and the SNS topic.

Any order data stored in DynamoDB is permanently deleted along with the table.

  • After the destroy completes, delete the local project directory. On macOS or Linux, run:
cd ~/Desktop
rm -rf order-pipeline
  • On Windows, run:
cd ~\Desktop
rmdir /s /q order-pipeline
  • Confirm the directory is gone by running ls on your Desktop. You should no longer see order-pipeline in the output.

Seeing errors during terraform destroy?

If a resource fails to delete due to a dependency error, run terraform destroy again. Terraform retries the deletion in the correct order.

If the EventBridge Pipe fails to delete, it may still be processing a batch. Wait a minute and run terraform destroy once more.

help me troubleshoot a terraform destroy error

Nice Work!

Nice Work!

You just built a production-style event-driven order processing pipeline on AWS, deployed and managed entirely through Terraform. From a single curl command, orders flow automatically through validation, payment, and fulfillment without you lifting a finger.

You learned how to:

  • Built an event-driven pipeline that automatically processes orders from API Gateway submission through DynamoDB, Step Functions orchestration, and Lambda processing.
  • Used EventBridge Pipes to connect DynamoDB Streams directly to Step Functions, eliminating the need for glue Lambda functions.
  • Defined a Step Functions state machine with Amazon States Language, including error handling with Retry and Catch to gracefully handle invalid orders.
  • Secret Mission: Added parallel workflow execution with SNS notifications, demonstrating production-ready patterns for simultaneous order fulfillment and customer communication.

Ready to quiz yourself?