Event-Driven Order Processing on AWS

Introduction

30 Second Summary

When you place an order online, dozens of systems need to react instantly. Inventory updates, payment processes, shipping notifications, and confirmation emails all fire at once without any single system waiting for the others.

In this project, you will build an event-driven order processing pipeline on AWS that receives orders through an API, routes them through a chain of decoupled services, and stores results in two different databases.

What You'll Build

An e-commerce order processing backend where placing an order triggers a chain of decoupled events across seven AWS services.

By the end of this project, you'll have:

• A REST API endpoint that receives order JSON and routes it through EventBridge for content-based event routing.
• An SNS topic fanning out to an SQS queue with a Dead Letter Queue for fault-tolerant message processing.
• A Python Lambda function that processes orders and writes to both DynamoDB and RDS MySQL.
• Secret Mission: Add real-time high-value order alerts using EventBridge numeric filtering and SNS email notifications.

Are there any prerequisites?

You will need an AWS account with free tier access, the AWS CLI installed and configured, VS Code installed, and basic Python knowledge.

Set Up Project Structure and AWS Resources

Before you can wire up an event-driven pipeline, you need the foundation in place. That means a verified AWS CLI connection, a local project folder, and the two databases your Lambda function will write to.

In this step, you will confirm your CLI access, scaffold the project files, and provision a DynamoDB table for fast key-value lookups and an RDS MySQL instance for SQL-based analytics.

In this step, get ready to:

• Verify your AWS CLI is configured and create the project folder.
• Create a DynamoDB table for order storage.
• Provision an RDS MySQL instance and create the order_summaries table.

Verify AWS CLI and create the project structure

Your AWS CLI needs to be configured with valid credentials so it can talk to your AWS account. Let's confirm that first.

• Open your terminal. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
• Type Terminal (macOS) or Command Prompt (Windows) and press Enter to open it.
• Verify your AWS CLI access by running this command:

aws sts get-caller-identity

✔️ I see my account ID and ARN

Your CLI is configured and ready. You should see JSON output containing your Account, UserId, and Arn. Move on to creating your project folder below.

ⓧ I see an error or "unable to locate credentials"

Your CLI is not configured yet. You need to run aws configure and provide your Access Key ID, Secret Access Key, default region, and output format.

• Run aws configure and enter your credentials when prompted.
• After configuring, run aws sts get-caller-identity again to confirm it works.

Find your credentials

Your Access Key ID and Secret Access Key are available in the AWS Console under IAM > Users > your username > Security credentials tab.

Now let's create the project folder structure on your Desktop.

• Move to your Desktop by running this command:

cd ~/Desktop

• Create your project folder and the Lambda subdirectory by running these commands:

mkdir -p order-pipeline/lambda
cd order-pipeline

• Create the requirements file with the pymysql dependency by running:

echo "pymysql==1.2.0" > lambda/requirements.txt

• Create the empty Lambda function file by running:

touch lambda/process_order.py

• Confirm the files exist by running:

ls lambda/

• You should see process_order.py and requirements.txt in the output.

• Open this project folder in VS Code by running:

code .

What is pymysql?

PyMySQL is a pure-Python MySQL client library. Because it has no compiled C dependencies, you can install it directly into a Lambda deployment package without needing Docker for cross-compilation.

Your Lambda function will use it to connect to the RDS MySQL database and write order summaries for analytics queries.

code command not found?

If code . does not work, open VS Code manually. Then click File > Open Folder and select the order-pipeline folder on your Desktop.

Help me open my project folder in VS Code.

Create the DynamoDB orders table

Your pipeline needs a fast database for storing orders by their unique ID. DynamoDB is perfect for this because it delivers single-digit millisecond reads when you query by partition key.

• Open the AWS Management Console in your browser and sign in.
• In the search bar at the top, type DynamoDB and select the DynamoDB service.
• In the left navigation pane, click Tables.
• Click Create table.
• For Table name, enter orders.
• For Partition key, enter order_id and leave the type as String.
• Leave the Sort key field empty.
• Under Table settings, keep Default settings selected. This uses on-demand capacity mode, which scales automatically and stays within free tier for low usage.
• Click Create table.

The table will show a status of Creating. Wait until it changes to Active before moving on. This usually takes less than 30 seconds.

Why on-demand capacity?

On-demand mode means DynamoDB automatically handles scaling. You pay only for the reads and writes you actually perform, with no need to pre-provision capacity units.

For a learning project with low traffic, this keeps you comfortably within the always-free tier of 25 GB storage and 25 read/write capacity units.

Table stuck in Creating status?

DynamoDB tables typically become active within seconds. If your table has been in Creating status for more than a few minutes, try refreshing the page. Check that you are viewing the correct AWS Region in the top-right corner of the Console.

My DynamoDB table is stuck in Creating status. How can I fix this?

Create the RDS MySQL instance and order_summaries table

While DynamoDB gives you fast key-value lookups, you also need a relational database for SQL queries like "total revenue this week" or "orders by region." RDS MySQL handles that analytics use case.

• In the AWS Console search bar, type RDS and select the RDS service.
• In the left navigation pane, click Databases.
• Click Create database.
• Select Standard create.
• For Engine type, choose MySQL.
• In Templates, choose Free tier (or Sandbox if your account shows that option instead).
• In Settings, enter orders-db for the DB instance identifier.
• Leave Master username as admin.
• Enter a master password and confirm it. Remember this password as you will need it later to connect.
• In Instance configuration, confirm the instance class is db.t3.micro (free tier eligible).
• In the Connectivity section, set Public access to Yes.
• Expand Additional configuration and enter orders_db for the Initial database name.
• Click Create database.

Why enable public access?

Public access allows your local machine (and your Lambda function outside a VPC) to connect directly to the RDS instance. This avoids the complexity of VPC and NAT gateway configuration for a learning project.

In a production environment, you would place both Lambda and RDS inside a VPC with security groups restricting access.

The RDS instance takes 5 to 15 minutes to reach Available status. While you wait, let's configure the security group so you can connect.

• Once the instance appears in the Databases list, click on orders-db to open its details.
• In the Connectivity & security tab, find the VPC security groups link and click on it.
• Click the Inbound rules tab, then click Edit inbound rules.
• Click Add rule.
• For Type, select MySQL/Aurora (port 3306).
• For Source, select My IP.
• Click Save rules.

Can't find the security group link?

Make sure you clicked on the orders-db instance name (not the checkbox). The Connectivity & security tab shows the security group as a clickable blue link under VPC security groups.

Help me find and edit the RDS security group inbound rules.

Now wait for the RDS instance status to show Available. Once it does, you can create the order_summaries table.

• In the orders-db detail page, copy the Endpoint value from the Connectivity & security tab.
• Connect to the database by running this command in your terminal:

mysql -h [[RDS_ENDPOINT="your-rds-endpoint.region.rds.amazonaws.com"]] -u admin -p

• When prompted, enter the master password you set during instance creation.
• Create the order_summaries table by running this SQL:

USE orders_db;

CREATE TABLE order_summaries (
    order_id VARCHAR(255) PRIMARY KEY,
    customer_email VARCHAR(255),
    item VARCHAR(255),
    amount DECIMAL(10,2),
    status VARCHAR(50)
);

What does this table store?

The order_summaries table stores the same order data that goes into DynamoDB, but in a relational format. This lets you run SQL queries for analytics, like calculating total revenue or filtering orders by status.

Your Lambda function will write to both databases simultaneously. DynamoDB handles fast lookups by order_id. RDS handles complex queries across multiple fields.

• Verify the table was created by running:

SHOW TABLES;

• You should see order_summaries listed in the output.
• Exit the MySQL client by typing:

EXIT;

mysql command not found?

If you do not have the MySQL client installed, you can install it with your package manager. On macOS, run brew install mysql-client. On Ubuntu/Debian, run sudo apt install mysql-client. On Windows, download the MySQL Shell from the official MySQL downloads page.

Alternatively, you can use AWS CloudShell (available in the Console toolbar) which has the MySQL client pre-installed.

Help me connect to my RDS MySQL instance.

Your foundation is solid. You have a verified CLI connection, a local project structure ready for your Lambda code, and two databases waiting to receive orders. Next up, you will create the EventBridge custom event bus and API Gateway endpoint that accept incoming orders and kick off the event chain.

Create EventBridge Bus and API Gateway

Your databases are ready to store orders. Now you need a way to receive them.

In this step, you will create the front door for your pipeline. An API Gateway endpoint accepts incoming order requests. An EventBridge custom event bus receives those orders as events and routes them to downstream services.

In this step, get ready to:

• Create an EventBridge custom event bus for order events.
• Build a REST API with a POST endpoint that sends events directly to EventBridge.
• Test the full flow by sending a curl request and verifying the event arrives.

Create the EventBridge custom event bus

EventBridge uses event buses to receive and route events. A custom bus keeps your order events separate from default AWS system events.

• In the AWS Console, search for EventBridge and open the service.
• In the left sidebar, click Event buses.
• Click Create event bus.
• Enter orders-event-bus as the name.
• Leave all other settings at their defaults and click Create.

What is a custom event bus?

Every AWS account has a default event bus that receives AWS service events (like EC2 state changes). A custom bus is your own channel for application-specific events.

By routing order events to orders-event-bus, you can write rules that only match your events without interference from system noise.

You should see orders-event-bus listed on the Event buses page with a status of Active.

Create the REST API and POST /orders endpoint

API Gateway will be the single entry point for all order requests. You will configure it to forward the order payload directly to EventBridge using an AWS Service integration.

Why a direct AWS Service integration?

API Gateway can call AWS services directly without a Lambda function in between. This reduces latency, cost, and moving parts. The mapping template transforms the incoming JSON into the format EventBridge expects.

• In the AWS Console, search for API Gateway and open the service.
• Click Create API.
• Under REST API (not REST API Private), click Build.
• Select New API.
• Enter OrdersAPI as the API name.
• Leave the endpoint type as Regional and click Create API.

Now create the resource and method:

• Click Create Resource.
• Enter orders as the Resource Name (the Resource Path auto-fills to /orders).
• Click Create Resource.
• With the /orders resource selected, click Create Method.
• For Method type, select POST.
• For Integration type, select AWS Service.
• For AWS Region, select your current region.
• For AWS Service, select EventBridge.
• For HTTP method, select POST.
• For Action, enter PutEvents.
• For Execution role, you need an IAM role ARN that allows events:PutEvents. Create one by following the steps in the callout below.
• Click Create Method.

How do I create the IAM execution role?

Open a new browser tab and navigate to the IAM Console. Create a new role with these settings:

• Trusted entity type: AWS service.
• Use case: API Gateway.
• After creating the role, add an inline policy that allows the events:PutEvents action on all resources (*).
• Name the role something like APIGatewayEventBridgeRole.
• Copy the role ARN and paste it into the Execution role field back in API Gateway.

Next, configure the mapping template that transforms the incoming order JSON into the EventBridge PutEvents format:

• On the POST method page, click the Integration Request tab.
• Click Edit on the Integration Request settings.
• Expand HTTP Headers. Add a header with name Content-Type and mapped from value 'application/x-amz-json-1.1' (including the single quotes).
• Add another header with name X-Amz-Target and mapped from value 'AWSEvents.PutEvents' (including the single quotes).
• Expand Mapping Templates.
• Click Add mapping template.
• For Content-Type, enter application/json.
• Paste the following mapping template into the template body:

{
  "Entries": [
    {
      "Source": "com.ecommerce.orders",
      "DetailType": "OrderPlaced",
      "Detail": "$util.escapeJavaScript($input.body)",
      "EventBusName": "orders-event-bus"
    }
  ]
}

What does this mapping template do?

This template transforms the raw JSON body from the API request into the format EventBridge expects for its PutEvents API.

• Source: com.ecommerce.orders identifies where the event came from.
• DetailType: OrderPlaced is the event category that rules will match on.
• Detail: the entire request body, escaped as a JSON string so EventBridge can parse it.
• EventBusName: routes the event to your custom bus instead of the default.

• Click Save to save the integration request configuration.

Now deploy the API so you can test it:

• Click Deploy API.
• For Stage, select New Stage.
• Enter dev as the stage name.
• Click Deploy.

After deployment, API Gateway shows your invoke URL. It follows this pattern:

https://{api-id}.execute-api.{region}.amazonaws.com/dev/orders

• Copy this URL. You will use it in the next substep to test the endpoint.

Don't see the invoke URL?

Make sure you deployed the API to the dev stage. Click Stages in the left sidebar, expand dev, and the invoke URL appears at the top.

Help me find my invoke URL

Test the API endpoint with a curl command

Time to verify the full flow. You will send a test order through the API and confirm it reaches EventBridge.

• Open your terminal and send a test order by running this command (replace the URL with your actual invoke URL):

curl -X POST [[API_INVOKE_URL="https://your-api-id.execute-api.your-region.amazonaws.com/dev/orders"]] -H "Content-Type: application/json" -d '{"order_id":"test-001","customer_email":"test@example.com","item":"Widget","amount":49.99}'

What does this command do?

• curl -X POST sends an HTTP POST request to your API endpoint.
• -H "Content-Type: application/json" tells the API the body is JSON.
• -d provides the order payload with an ID, email, item name, and amount.

You should receive a 200 response with a JSON body containing FailedEntryCount of 0 and an Entries array with an EventId. This confirms EventBridge accepted the event.

• To double-check in the Console, navigate back to EventBridge.
• In the left sidebar, click Event buses and select orders-event-bus.
• Click the Monitoring tab. You should see PutEvents invocation metrics showing at least one event received.

Getting a 403 or 500 error?

A 403 usually means the IAM execution role does not have the events:PutEvents permission. Go back to IAM and verify the inline policy is attached to the correct role.

A 500 often means the mapping template has a syntax error. Check that the Content-Type header is set to 'application/x-amz-json-1.1' (with single quotes) and the X-Amz-Target header is 'AWSEvents.PutEvents' (with single quotes).

Help me debug my API Gateway EventBridge integration

Your API Gateway is live and successfully routing orders to EventBridge. Next up, you will create the SNS topic and SQS queue that subscribe to these events and reliably buffer them for processing.

Create SNS, SQS, DLQ, and EventBridge Routing Rule

Your API Gateway is live and sending events to EventBridge. Now you need something to receive those events and deliver them reliably to a processor.

In this step, you will build the messaging layer that sits between EventBridge and your future Lambda function. This layer uses three AWS services working together: SNS fans out messages to subscribers, SQS buffers them for reliable processing, and a Dead Letter Queue catches any messages that fail repeatedly.

In this step, get ready to:

• Create the SQS queues (main queue and Dead Letter Queue) and SNS topic.
• Create an EventBridge rule that routes order events to SNS.
• Test the full flow from API Gateway through to SQS.

Create the SQS queues and SNS topic

You will create three resources: a Dead Letter Queue that catches failed messages, a main processing queue that uses the DLQ as its safety net, and an SNS topic that broadcasts events to the queue.

Start with the DLQ because the main queue needs to reference it during creation.

• In the AWS Console, search for SQS and open the Amazon SQS service.
• Click Create queue.
• For Type, keep Standard selected.
• For Name, enter order-processing-dlq.
• Leave all other settings as defaults.
• Scroll down and click Create queue.

What is a Dead Letter Queue?

A Dead Letter Queue is a holding area for messages that cannot be processed successfully. If a message fails a set number of times, SQS moves it to the DLQ instead of retrying forever.

This prevents a single bad message from blocking all other messages in the queue. You can inspect DLQ messages later to debug what went wrong.

Now create the main processing queue and attach the DLQ to it.

• Click Create queue again.
• For Type, keep Standard selected.
• For Name, enter order-processing-queue.
• Scroll down to the Dead-letter queue section and toggle it Enabled.
• In the Choose queue dropdown, select order-processing-dlq.
• Set Maximum receives to 3.
• Click Create queue.

Why Maximum receives is set to 3

Maximum receives means a message gets 3 attempts at being processed before SQS moves it to the DLQ. This gives your Lambda function a few retries to handle transient errors (like a brief network blip) while still catching messages that are genuinely broken.

Next, create the SNS topic that will broadcast events to the queue.

• In the AWS Console, search for SNS and open the Amazon SNS service.
• In the left navigation pane, click Topics.
• Click Create topic.
• For Type, select Standard.
• For Name, enter order-processing-topic.
• Leave all other settings as defaults and click Create topic.

Now subscribe the SQS queue to the SNS topic so that published messages flow into the queue.

• On the order-processing-topic details page, click Create subscription.
• For Protocol, choose Amazon SQS.
• For Endpoint, enter the ARN of your order-processing-queue.
• Click Create subscription.

Find your queue ARN quickly

Open the SQS Console in a new tab, click on order-processing-queue, and copy the ARN from the Details section. Paste it into the Endpoint field back in SNS.

Finally, update the SQS queue's access policy so SNS is allowed to deliver messages to it.

• Return to the SQS Console and click on order-processing-queue.
• Select the Access policy tab.
• Click Edit.
• In the Access policy JSON editor, add the following statement inside the existing Statement array:

{
  "Effect": "Allow",
  "Principal": {
    "Service": "sns.amazonaws.com"
  },
  "Action": "sqs:SendMessage",
  "Resource": "[[SQS_QUEUE_ARN=\"your order-processing-queue ARN\"]]",
  "Condition": {
    "ArnEquals": {
      "aws:SourceArn": "[[SNS_TOPIC_ARN=\"your order-processing-topic ARN\"]]"
    }
  }
}

What does this policy statement do?

This grants the SNS service permission to call sqs:SendMessage on your queue. The Condition block restricts it so only your specific SNS topic can send messages. Without this policy, SNS cannot deliver to the queue.

• Click Save.

Subscription shows PendingConfirmation?

If the subscription status says PendingConfirmation, the access policy on the queue was not set before subscribing. Update the policy (steps above), then delete and re-create the subscription.

Help me fix my SNS-to-SQS subscription that shows PendingConfirmation.

Create the EventBridge routing rule

With the messaging infrastructure in place, you need to tell EventBridge to route incoming order events to your SNS topic. You will create a rule that matches events with the source com.ecommerce.orders and detail type OrderPlaced.

• In the AWS Console, search for EventBridge and open the Amazon EventBridge service.
• In the left navigation pane, click Rules.
• In the Event bus dropdown at the top, select orders-event-bus.
• Click Create rule.
• For Name, enter route-orders-to-sns.
• For Event bus, confirm orders-event-bus is selected.
• For Rule type, select Rule with an event pattern.
• Click Next.

On the Build event pattern page, you will define which events this rule matches.

• Scroll down to the Event pattern section.
• Select Custom pattern (JSON editor).
• Paste the following event pattern:

{
  "source": ["com.ecommerce.orders"],
  "detail-type": ["OrderPlaced"]
}

What does this event pattern do?

EventBridge evaluates every event on the bus against this pattern. Only events where source equals com.ecommerce.orders AND detail-type equals OrderPlaced will trigger this rule. Everything else on the bus is ignored.

• Click Next.
• On the Select targets page, for Target type, select AWS service.
• For Select a target, choose SNS topic.
• For Topic, select order-processing-topic.
• Click Next.
• Skip the Tags page and click Next again.
• Review the rule summary and click Create rule.

You should see a success banner confirming the rule was created.

Don't see your event bus in the dropdown?

Make sure you are in the same AWS region where you created orders-event-bus. Check the region selector in the top-right corner of the Console.

Help me find my custom event bus in EventBridge.

Test the full event flow

The full chain is now connected: API Gateway sends events to EventBridge, EventBridge routes them to SNS, and SNS delivers them to SQS. Send a test order and verify the message lands in your queue.

• Send a test order through your API by running this command:

curl -X POST [[API_URL="your API Gateway invoke URL (e.g. https://{api-id}.execute-api.{region}.amazonaws.com/dev/orders)"]]/orders -H "Content-Type: application/json" -d '{"order_id":"test-002","customer_email":"test@example.com","item":"Gadget","amount":29.99}'

What does this command do?

This sends a POST request with a JSON order payload to your API Gateway endpoint. API Gateway transforms the body into an EventBridge event, EventBridge matches it against your rule, delivers it to SNS, and SNS forwards it to SQS. The entire chain executes in about one second.

• You should receive a 200 response from the API.
• Return to the SQS Console and click on order-processing-queue.
• Click Send and receive messages.
• Click Poll for messages.
• You should see at least one message in the list. Click on it to see the order JSON nested inside the SNS notification envelope.

Now verify the DLQ is empty (no failures have occurred yet).

• Go back to Queues and click on order-processing-dlq.
• Check that Messages available shows 0.

No messages in the queue?

Check these common causes:

• The EventBridge rule might target the wrong bus. Confirm the rule is on orders-event-bus, not the default bus.
• The SQS access policy might not allow SNS. Check the policy includes sqs:SendMessage with the correct aws:SourceArn condition.
• The event pattern might have a typo. Verify detail-type is hyphenated and matches OrderPlaced exactly.

Help me debug why messages are not arriving in my SQS queue from EventBridge and SNS.

Your messaging infrastructure is connected end-to-end. Next up, you will write the Lambda function that processes these orders from SQS and writes them to both DynamoDB and RDS.

Write and Deploy the Lambda Function

Your messaging pipeline is live. Orders flow from API Gateway through EventBridge and SNS into your SQS queue. But right now those messages just sit there with nothing to process them.

In this step, you will write a Python Lambda function that reads orders from the queue, then writes each order to both DynamoDB (for fast lookups) and RDS MySQL (for analytics queries). You will package the function with its dependency, deploy it to AWS, and wire it up to your SQS queue.

In this step, get ready to:

• Write the Python Lambda function that processes SQS messages.
• Package and deploy the function to AWS with the correct IAM permissions.
• Create the SQS event source mapping and verify end-to-end processing.

Write the Lambda function code

Your Lambda function has one job: pull order messages from SQS, parse the JSON, and write the order to two databases. DynamoDB gives you fast key-value lookups by order_id. RDS MySQL gives you SQL queries for analytics.

• Switch back to VS Code and open lambda/process_order.py.
• Add the imports, environment variables, and database setup by pasting this code:

import json
import os
import boto3
import pymysql

# Environment variables
DYNAMODB_TABLE = os.environ['DYNAMODB_TABLE']
RDS_HOST = os.environ['RDS_HOST']
RDS_USER = os.environ['RDS_USER']
RDS_PASSWORD = os.environ['RDS_PASSWORD']
RDS_DB_NAME = os.environ['RDS_DB_NAME']

# Initialize DynamoDB resource
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table(DYNAMODB_TABLE)


def get_rds_connection():
    """Create a connection to the RDS MySQL database."""
    return pymysql.connect(
        host=RDS_HOST,
        user=RDS_USER,
        password=RDS_PASSWORD,
        database=RDS_DB_NAME,
        cursorclass=pymysql.cursors.DictCursor
    )

What does this code do?

The top section reads five environment variables that you will configure on the Lambda function later. These keep database credentials out of your source code.

The get_rds_connection() function creates a fresh MySQL connection each time it is called. It uses pymysql, a pure-Python MySQL driver that works in Lambda without compiled dependencies.

• Below the get_rds_connection function, add the two database write functions:

def write_to_dynamodb(order):
    """Write the order to DynamoDB for fast key-value lookups."""
    table.put_item(
        Item={
            'order_id': order['order_id'],
            'customer_email': order['customer_email'],
            'item': order['item'],
            'amount': str(order['amount']),
            'status': 'processed'
        }
    )
    print(f"Written order {order['order_id']} to DynamoDB")


def write_to_rds(order):
    """Write the order summary to RDS MySQL for analytics."""
    connection = get_rds_connection()
    try:
        with connection.cursor() as cursor:
            sql = """
                INSERT INTO order_summaries (order_id, customer_email, item, amount, status)
                VALUES (%s, %s, %s, %s, %s)
            """
            cursor.execute(sql, (
                order['order_id'],
                order['customer_email'],
                order['item'],
                order['amount'],
                'processed'
            ))
        connection.commit()
        print(f"Written order {order['order_id']} to RDS")
    finally:
        connection.close()

What do these functions do?

The write_to_dynamodb function calls put_item to store the order as a key-value record. The amount is converted to a string because DynamoDB does not accept Python floats directly.

The write_to_rds function inserts the same order data into the order_summaries table you created earlier. The try/finally block ensures the database connection closes even if the insert fails.

• Finally, add the main handler function that Lambda invokes when messages arrive:

def lambda_handler(event, context):
    """Process SQS messages containing order events."""
    for record in event['Records']:
        # SQS message body contains the SNS notification
        sns_message = json.loads(record['body'])
        # The actual order detail is in the SNS Message field
        order = json.loads(sns_message['Message'])

        print(f"Processing order: {order['order_id']}")

        # Write to both databases
        write_to_dynamodb(order)
        write_to_rds(order)

    return {
        'statusCode': 200,
        'body': json.dumps({'message': f"Processed {len(event['Records'])} orders"})
    }

Why parse the message twice?

When EventBridge routes an event to SNS and SNS delivers to SQS, the SQS message body contains the SNS notification envelope. The actual order JSON is nested inside the Message field of that envelope. That is why the code calls json.loads twice: once to unwrap the SNS envelope, and once to parse the order detail.

• Save lambda/process_order.py.

Your function is complete. You will not see a visible change yet because the function needs to be deployed to AWS before it can process real messages.

Seeing red squiggles in VS Code?

Your editor may show import errors for boto3 and pymysql because they are not installed in your local environment. This is expected. These libraries are available in the Lambda runtime (boto3) and in the deployment package you will create next (pymysql).

Help me understand why my Lambda imports show errors locally

✔️ Awesome, I've got everything!

Great. Double check you have saved lambda/process_order.py before continuing.

ⓧ I'd like to double check the full code

Here is the complete lambda/process_order.py file:

import json
import os
import boto3
import pymysql

# Environment variables
DYNAMODB_TABLE = os.environ['DYNAMODB_TABLE']
RDS_HOST = os.environ['RDS_HOST']
RDS_USER = os.environ['RDS_USER']
RDS_PASSWORD = os.environ['RDS_PASSWORD']
RDS_DB_NAME = os.environ['RDS_DB_NAME']

# Initialize DynamoDB resource
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table(DYNAMODB_TABLE)


def get_rds_connection():
    """Create a connection to the RDS MySQL database."""
    return pymysql.connect(
        host=RDS_HOST,
        user=RDS_USER,
        password=RDS_PASSWORD,
        database=RDS_DB_NAME,
        cursorclass=pymysql.cursors.DictCursor
    )


def write_to_dynamodb(order):
    """Write the order to DynamoDB for fast key-value lookups."""
    table.put_item(
        Item={
            'order_id': order['order_id'],
            'customer_email': order['customer_email'],
            'item': order['item'],
            'amount': str(order['amount']),
            'status': 'processed'
        }
    )
    print(f"Written order {order['order_id']} to DynamoDB")


def write_to_rds(order):
    """Write the order summary to RDS MySQL for analytics."""
    connection = get_rds_connection()
    try:
        with connection.cursor() as cursor:
            sql = """
                INSERT INTO order_summaries (order_id, customer_email, item, amount, status)
                VALUES (%s, %s, %s, %s, %s)
            """
            cursor.execute(sql, (
                order['order_id'],
                order['customer_email'],
                order['item'],
                order['amount'],
                'processed'
            ))
        connection.commit()
        print(f"Written order {order['order_id']} to RDS")
    finally:
        connection.close()


def lambda_handler(event, context):
    """Process SQS messages containing order events."""
    for record in event['Records']:
        # SQS message body contains the SNS notification
        sns_message = json.loads(record['body'])
        # The actual order detail is in the SNS Message field
        order = json.loads(sns_message['Message'])

        print(f"Processing order: {order['order_id']}")

        # Write to both databases
        write_to_dynamodb(order)
        write_to_rds(order)

    return {
        'statusCode': 200,
        'body': json.dumps({'message': f"Processed {len(event['Records'])} orders"})
    }

Package and deploy the Lambda function

Lambda needs your code and its dependencies bundled into a single zip file. The boto3 library is pre-installed in the Lambda runtime, but pymysql is not. You need to include it in your deployment package.

• In your terminal, make sure you are inside the order-pipeline folder.
• Install pymysql into a local package directory by running:

pip install pymysql==1.2.0 -t ./package

What does this command do?

The -t ./package flag tells pip to install the library into a package subfolder instead of your system Python. This lets you bundle the dependency into the Lambda zip without affecting your local environment.

• Create the deployment zip containing both the dependency and your function code by running:

cd package && zip -r ../deployment.zip . && cd .. && zip deployment.zip lambda/process_order.py

What does this command do?

The first part (cd package && zip -r ../deployment.zip .) puts all the pymysql files into deployment.zip. The second part (zip deployment.zip lambda/process_order.py) adds your function code to the same zip. Lambda will unpack this zip when it runs your function.

• Confirm the zip file was created by running:

ls deployment.zip

You should see deployment.zip listed in the output.

Next, you need an IAM role that grants your Lambda function permission to read from SQS, write to DynamoDB, and log to CloudWatch.

• Open the AWS Console and navigate to IAM.
• In the left sidebar, click Roles.
• Click Create role.
• For Trusted entity type, select AWS service.
• For Use case, select Lambda.
• Click Next.
• In the Permissions policies search box, type AWSLambdaSQSQueueExecutionRole and select it.
• Clear the search box, then search for AmazonDynamoDBFullAccess and select it as well.
• Click Next.
• For Role name, enter lambda-order-processor-role.
• Click Create role.
• After the role is created, click on lambda-order-processor-role to open it.
• Copy the Role ARN from the top of the page. You will need it in the next command.

What permissions does this role grant?

The AWSLambdaSQSQueueExecutionRole policy lets Lambda receive and delete messages from SQS, plus write logs to CloudWatch. The AmazonDynamoDBFullAccess policy lets Lambda read and write items in any DynamoDB table in your account.

• Create the Lambda function by running this command in your terminal:

aws lambda create-function \
  --function-name process-order \
  --runtime python3.13 \
  --handler lambda.process_order.lambda_handler \
  --role [[ROLE_ARN="arn:aws:iam::123456789012:role/lambda-order-processor-role"]] \
  --zip-file fileb://deployment.zip

What does each flag do?

• --function-name sets the Lambda function name to process-order.
• --runtime python3.13 tells Lambda to use the Python 3.13 managed runtime.
• --handler lambda.process_order.lambda_handler tells Lambda where to find the entry point: the file at lambda/process_order.py and the function lambda_handler inside it.
• --zip-file fileb://deployment.zip uploads your deployment package from the local file.

• Set the environment variables your function needs by running:

aws lambda update-function-configuration \
  --function-name process-order \
  --environment "Variables={DYNAMODB_TABLE=orders,RDS_HOST=[[RDS_HOST="your-rds-endpoint.region.rds.amazonaws.com"]],RDS_USER=admin,RDS_PASSWORD=your-rds-password,RDS_DB_NAME=[[RDS_DB_NAME="your-database-name"]]}"

Find your RDS endpoint

In the AWS Console, go to RDS, click Databases, then click orders-db. The endpoint is shown under Connectivity & security. It looks like orders-db.xxxxxxxxxxxx.us-east-1.rds.amazonaws.com.

You should see a JSON response showing the updated function configuration with your environment variables listed.

Getting an InvalidParameterValueException?

Make sure the Role ARN you copied is correct and includes the full path (starting with arn:aws:iam::). Also confirm the role has finished creating. New IAM roles can take a few seconds to propagate.

Help me fix my Lambda creation error

Create the SQS trigger and test end-to-end

Your Lambda function is deployed but nothing is triggering it yet. You need to create an event source mapping that tells Lambda to poll your SQS queue for new messages.

• Get your SQS queue ARN by running:

aws sqs get-queue-attributes \
  --queue-url $(aws sqs get-queue-url --queue-name order-processing-queue --query QueueUrl --output text) \
  --attribute-names QueueArn \
  --query Attributes.QueueArn \
  --output text

Copy the ARN that is returned. It looks like arn:aws:sqs:us-east-1:123456789012:order-processing-queue.

• Create the event source mapping by running:

aws lambda create-event-source-mapping \
  --function-name process-order \
  --event-source-arn [[SQS_QUEUE_ARN="arn:aws:sqs:us-east-1:123456789012:order-processing-queue"]] \
  --batch-size 5

What is batch size?

The --batch-size 5 flag tells Lambda to pull up to 5 messages from SQS per invocation. This is more efficient than processing one message at a time because it reduces the number of Lambda invocations for high-volume queues.

You should see a JSON response with the event source mapping details and a state of Creating or Enabled.

• Wait about 30 seconds for the mapping to become active, then send a test order through your API by running:

curl -X POST [[API_URL="https://your-api-id.execute-api.your-region.amazonaws.com"]]/dev/orders \
  -H "Content-Type: application/json" \
  -d '{"order_id":"order-100","customer_email":"test@example.com","item":"Premium Widget","amount":49.99}'

Find your API URL

Your API URL is the same one you used in the previous step. You can find it in the API Gateway Console under Stages, then the dev stage. The invoke URL is shown at the top.

• Wait about 10 seconds for the message to flow through the pipeline, then check CloudWatch Logs by running:

aws logs filter-log-events \
  --log-group-name /aws/lambda/process-order \
  --filter-pattern "Processing order" \
  --query 'events[*].message' \
  --output text

You should see output containing Processing order: order-100 and messages confirming writes to DynamoDB and RDS.

• Verify the order landed in DynamoDB by opening the AWS Console.
• Navigate to DynamoDB, then click Tables.
• Click the orders table, then click Explore table items.
• You should see an item with order_id set to order-100 and status set to processed.

• Verify the order is in RDS by connecting to your MySQL instance and running:

SELECT * FROM order_summaries;

You should see a row with order_id of order-100, item of Premium Widget, and status of processed.

• Finally, return to the SQS Console and check the order-processing-queue. The Messages available count should be 0, confirming Lambda drained the queue.
• Check the order-processing-dlq as well. It should also show 0 messages, meaning no failures occurred.

CloudWatch Logs are empty?

If you do not see any log events, the event source mapping may still be initializing. Wait 60 seconds and resend the test order. Also confirm the mapping state is Enabled by running:

aws lambda list-event-source-mappings --function-name process-order

If the state shows Disabled, check that your Lambda execution role has the AWSLambdaSQSQueueExecutionRole policy attached.

Help me troubleshoot my Lambda SQS trigger

Your entire event-driven pipeline is now operational. An order hits your API, routes through EventBridge and SNS into SQS, and Lambda processes it into both databases. Next up, you will tackle the Secret Mission and add real-time alerts for high-value orders.

Secret mission

High-Value Order Alerts

Your pipeline processes every order the same way. But what if your business needs instant alerts when a big-ticket order comes in? In this secret mission, you will add a second EventBridge rule that uses numeric content-based filtering to detect orders above $100 and send a real-time email notification via SNS.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Most services in this project fall within the AWS always-free tier. The exception is your RDS instance, which is free for 6 months on new accounts (created after July 15, 2025) and may incur charges afterwards.

Cost warning

Your RDS MySQL db.t3.micro instance is free for 6 months on new AWS accounts. After that period, you will be charged for compute and storage. If you are not planning to continue this project, choose Pause or Delete below to avoid unexpected charges.

Resources you used:

• DynamoDB table orders (always free: 25 GB storage, 25 WCU, 25 RCU).
• RDS MySQL instance orders-db (free for 6 months on new accounts).
• Lambda function process-order (always free: 1,000,000 requests/month).
• API Gateway REST API OrdersAPI (always free: 1,000,000 calls/month).
• EventBridge custom event bus orders-event-bus with rules route-orders-to-sns and high-value-order-alert (free for custom events).
• SQS queues order-processing-queue and order-processing-dlq (always free: 1,000,000 requests/month).
• SNS topics order-processing-topic and high-value-order-alerts (always free: 1,000,000 publishes/month).
• Local project folder order-pipeline on your Desktop.

Keep everything running

No action needed. Choose this if you are still actively building or experimenting with your pipeline.

• All services except RDS remain within the always-free tier at low usage levels.
• Your RDS db.t3.micro instance is free for 6 months on new accounts. After that, it incurs hourly charges even when idle.
• Your local order-pipeline folder remains on your Desktop.

Pause - I'll come back to this later

Stop your RDS instance to avoid compute charges while keeping everything intact. Storage charges still apply after the free tier expires.

• Open the Amazon RDS console.
• In the navigation pane, choose Databases, then select orders-db.
• Choose Actions, then Stop temporarily.
• Acknowledge that the instance will restart automatically after 7 days, then click Stop temporarily.

Why does it restart after 7 days?

AWS automatically restarts stopped RDS instances after 7 consecutive days. If you need it stopped longer, you will need to stop it again each week or delete it entirely.

• All other services (Lambda, API Gateway, EventBridge, SQS, SNS, DynamoDB) are pay-per-use and will not incur charges while idle.

Delete - I don't want to use this again

Remove all project resources to ensure zero ongoing charges. Delete in this order to avoid dependency issues.

Delete the Lambda event source mapping and function

• Open the Functions page of the Lambda console.
• Select process-order.
• Choose Actions, Delete.
• Type confirm in the text input field and choose Delete.

What about the event source mapping?

Deleting the Lambda function also removes its event source mappings. You do not need to delete the SQS trigger separately.

Delete the API Gateway REST API

• Open the APIs page of the API Gateway console.
• Select OrdersAPI.
• Choose Actions, Delete.
• Choose Delete to confirm.

Delete the EventBridge rules and custom event bus

• Open the Amazon EventBridge console and navigate to Rules.
• Select event bus orders-event-bus from the event bus dropdown.
• Select the rule route-orders-to-sns, choose Delete, and confirm.
• Select the rule high-value-order-alert, choose Delete, and confirm.
• Navigate to Event buses, select orders-event-bus, and choose Delete.

Delete the SNS topics and subscriptions

• Open the Amazon SNS console and navigate to Topics.
• Select order-processing-topic, choose Delete, type delete me in the confirmation field, and confirm.
• Repeat for high-value-order-alerts.

What about the subscriptions?

Deleting an SNS topic automatically removes all subscriptions attached to it. You do not need to delete them separately.

Delete the SQS queues

• Open the Amazon SQS console and navigate to Queues.
• Select order-processing-queue, choose Delete, type confirm in the confirmation field, and confirm.
• Repeat for order-processing-dlq.

Delete the DynamoDB table

• Open the Tables page of the DynamoDB console.
• Select the orders table.
• Choose Delete.
• Enter delete in the text box and choose Delete table.

Delete the RDS instance

• Open the Amazon RDS console and navigate to Databases.
• Select orders-db.
• Choose Actions, then Delete.
• Uncheck Create final snapshot (not needed for a learning project).
• Check the acknowledgement box, type delete me in the confirmation field, and choose Delete.

Delete the IAM execution roles

• Open the Roles page of the IAM console.
• Search for and select the execution role you created for the Lambda function.
• Choose Delete, enter the role name to confirm, and choose Delete.
• Repeat for the API Gateway execution role (the role that allows events:PutEvents).

Delete the local project files

• Delete the local project folder by running this command:

cd ~/Desktop
rm -rf order-pipeline

On Windows, use:

cd ~\Desktop
Remove-Item -Recurse -Force order-pipeline

After completing these steps, all project resources are removed and you will incur zero ongoing charges.

That's a Wrap!

That's a Wrap!

Nice work! You just built a production-pattern event-driven order processing pipeline using seven AWS services working together.

You've learned how to:

• Build a REST API with API Gateway that feeds orders into an EventBridge custom event bus with content-based routing rules.
• Set up SNS fanout to SQS with a Dead Letter Queue for fault-tolerant message processing.
• Write a Python Lambda function triggered by SQS that writes to both DynamoDB for fast key-value lookups and RDS MySQL for SQL analytics.
• Secret Mission: Implemented real-time high-value order alerts using EventBridge numeric filtering and SNS email notifications.

Ready to quiz yourself?