Build a Fault-Tolerant Order Pipeline

Build an SQS and Lambda pipeline that isolates orders in a dead-letter queue.

Introduction

30 Second Summary

Online orders can arrive in seconds, but the work behind them can fail at any point. A reliable system keeps valid orders moving while setting failed orders aside for inspection.

In this project, you will build an order-processing pipeline with Amazon SQS and AWS Lambda. You will use Amazon CloudWatch Logs to observe repeated failures before a dead-letter queue isolates the failed order.

What You'll Build

Your finished demo processes `ORDER-1002` automatically, retries `FAIL`, and moves the failed message into `orders-dlq` for inspection.

By the end of this project, you'll have:

An automatic order flow where sending ORDER-1002 produces successful processing logs without manual polling.
A repeatable failure test where sending FAIL exposes retries across multiple Lambda invocations.
A failure-isolation proof where you poll orders-dlq and retrieve the failed message after its retries are exhausted.
Secret Mission: Create an Amazon CloudWatch alarm that changes to ALARM when a failed order reaches orders-dlq.

Are there any prerequisites?

This project requires an AWS account and a computer with a modern web browser.

Your AWS identity needs permission to manage Lambda functions, SQS queues, AWS Identity and Access Management (IAM) roles, CloudWatch logs, alarms, and billing preferences.

Before We Start

Before any hands-on work, lock in the failure path that keeps valid orders moving. An Amazon SQS source queue holds incoming orders for asynchronous processing by AWS Lambda. A separate dead-letter queue isolates repeatedly failing poison messages.

Set Your Region and Cost Guardrails

Your order pipeline depends on regional AWS services that need to work together. One consistent AWS Region keeps those resources in the same place.

Before any persistent resource exists, the AWS Management Console gives you a clear view of current Free Tier usage. Usage alerts add a cost guardrail before you start building.

In this step, get ready to:

Set one consistent Region for every regional project resource.
Review your current month-to-date Free Tier usage.
Confirm cost alerts plus the console access this project needs.

Sign in and set your Region

AWS keeps regional resources separate. The queue and function you create later must share one Region.

Your credentials stay under your control when you use the official AWS sign-in flow.

Navigate to the AWS Management Console sign-in page in your browser.
Sign in with your current AWS identity.
Choose US East (N. Virginia) from the Region selector in the top navigation bar.

You will see US East (N. Virginia) selected in the console header. Its Region code is us-east-1.

Review current Free Tier usage

Existing activity in your AWS account counts toward the same monthly allowances as this project. Billing and Cost Management shows how much usage your account has already accumulated.

You can relax on this billing screen because this check creates no paid resources.

Open Billing and Cost Management with the AWS Management Console search bar.
Choose Free Tier in the navigation pane.
Select MTD actual usage % as the sort column.
Review the rows in the sorted table for current month-to-date usage.

You will see the current month-to-date usage for eligible services. These percentages reveal how much of each allowance your account has already used.

Enable alerts and confirm service access

Free Tier usage alerts provide a second cost guardrail. Individual accounts receive an alert at 85 percent of a service limit.

Return to Billing and Cost Management from the Free Tier page.
Choose Billing preferences.
Choose Edit under Alert preferences.
Select Receive AWS Free Tier usage alerts.
Choose Update.

Account permissions determine which AWS consoles your current identity can use. This project relies on Amazon SQS, AWS Lambda, AWS Identity and Access Management (IAM), Amazon CloudWatch Logs, plus Amazon CloudWatch.

Open the Amazon SQS console with the AWS Management Console search bar.
Open the AWS Lambda console with the AWS Management Console search bar.
Open the IAM console with the AWS Management Console search bar.
Open the Amazon CloudWatch Logs console with the AWS Management Console search bar.
Open the Amazon CloudWatch console with the AWS Management Console search bar.

Each console should load under your current AWS identity. This confirms that your identity can reach the services used throughout the project.

Can't open a console?

A denied page means your current identity does not have the required access for that service. Ask an AWS administrator for an identity that can manage the resources listed in this project.

Help me identify which AWS console access is missing.

Before you refresh, consider whether you expect the Free Tier alert preference to stay selected.

Refresh the Billing preferences page.

You will see Receive AWS Free Tier usage alerts remains selected.

Return to Free Tier in the Billing and Cost Management navigation pane.

You will see your current month-to-date usage again. This confirms that your usage view plus its alert guardrail are available before you create resources.

You have handled the account guardrails early, before a single project resource exists.

Your account is ready for a cost-aware build. Next, you will create the source queue and test its basic message lifecycle.

Create and Test the Order Queue

Your Region is fixed for the project. Your Free Tier guardrails are ready.

Before automation can process orders, you need to understand how an Amazon SQS message queue handles one message. This manual test covers the full lifecycle from sending an order to deleting it.

In this step, get ready to:

Create a Standard source queue named orders-source.
Send ORDER-1001 to the source queue.
Receive the test order before deleting it from the queue.

Create the source queue

The orders-source queue holds orders until a consumer retrieves them. Its visibility timeout temporarily hides a retrieved message from other consumers.

The default visibility timeout is 30 seconds. That setting supports the retry behavior you explore later in the project.

Switch back to the Amazon SQS console from your access check.
Confirm the Region menu shows US East (N. Virginia).
Choose Create queue.
Keep Type set to Standard.
Enter orders-source in the Name field.
Confirm Visibility timeout shows 30 seconds.
Choose Create queue.

Your first project resource is live as orders-source waits for an order message.

Unable to Create the Queue?

Confirm the Region menu still shows US East (N. Virginia).
Ask your AWS administrator to confirm that your current identity can create Amazon SQS queues.

Get help diagnosing the queue creation problem.

Send and inspect a test order

Each queue message has a body that carries its useful value. The body ORDER-1001 gives you one order to trace through the queue.

Choose Send and receive messages on the orders-source queue page.
Enter ORDER-1001 under Message body.
Choose Send message.

The test order is now waiting inside orders-source.

Polling requests an available message from the queue. The Receive count records how many times Amazon SQS has retrieved that message.

The receive count can be easy to miss among the message details.

Choose Poll for messages.
Open the retrieved message.
Inspect the message body for ORDER-1001.
Locate the value under Receive count.

You should see ORDER-1001 in the message body. You should also see a value under Receive count.

Why Does Receive Count Matter?

Receive count shows that Amazon SQS delivered the queued order for retrieval. Later, repeated retrievals help you identify a message that keeps failing.

Delete the test order

Receiving a message lets you inspect it. Deleting that message completes the manual queue lifecycle.

Select the retrieved ORDER-1001 message.
Choose Delete.

The deletion removes the test order from orders-source.

Before you poll again, decide whether you expect Amazon SQS to retrieve the deleted order.

Choose Poll for messages once more.

You should see no retrieved messages. This confirms that orders-source is empty.

Still Seeing ORDER-1001?

Confirm that you selected the retrieved ORDER-1001 message before choosing Delete.
Wait for the deletion request to finish.
Choose Poll for messages again.

Ask why the deleted message still appears.

You have proved the complete lifecycle of an order message. Next, you will connect an automatic processor that consumes messages from orders-source.

Connect an Automatic Order Processor

Your orders-source queue can already hold an order until a consumer retrieves it. Your manual test proved that the queue lifecycle works.

Production orders need processing without someone polling the queue. In this step, an event source mapping connects AWS Lambda to Amazon SQS for automatic processing with loose coupling.

In this step, get ready to:

Create the order-processor function with Python 3.14.
Grant queue access through the function's execution role.
Connect the source queue to the function and verify the invocation in Amazon CloudWatch Logs.

Create the Lambda function

A Lambda function runs your order handler when it receives an event. The browser editor keeps this project inside the AWS Management Console.

Open the AWS Lambda console from the AWS Management Console service search.
Choose Create Event Function.

You'll see the function creation options for the selected Region.

Select Author from scratch.
Name the function order-processor.

The creation form now identifies the function that consumes your order messages.

Select Python 3.14 under Runtime.
Keep x86_64 under Architecture.

The function now has the runtime and architecture required by this project.

Keep the default execution role option selected.
Choose Create Event Function.

Your new function page opens with a browser-based source editor. That is the first resource in your automatic processing path.

Select lambda_function.py in the source editor.
Replace its contents with the code below:

def lambda_handler(event, context):
    for message in event["Records"]:
        body = message["body"]
        print(f"Received message: {body}")

        if body == "FAIL":
            raise ValueError("Invalid order message")

        print(f"Processed order: {body}")

    print("Batch complete")

What does this code do?

The lambda_handler() function receives the event that triggers the processor.
The event["Records"] loop handles every message delivered in the batch.
The message["body"] lookup extracts the order text for processing.
The printed messages make each processing stage visible in CloudWatch Logs.
The FAIL branch preserves an invalid order as a function error.

Apply the code changes by choosing Deploy in the DEPLOY section.
Confirm the function page indicates that the current code is deployed.

Function creation or deployment blocked?

Confirm that you edited lambda_function.py inside order-processor.
Compare every quote and indentation level with the full file below.
Ask your AWS administrator for an identity that can create Lambda functions if AWS denies the action.
Help me diagnose why my Lambda function cannot be created or deployed.

✔️ Awesome, I've got everything!

Your deployed lambda_function.py contains the complete order handler.

ⓧ I'd like to double check the full code

def lambda_handler(event, context):
    for message in event["Records"]:
        body = message["body"]
        print(f"Received message: {body}")

        if body == "FAIL":
            raise ValueError("Invalid order message")

        print(f"Processed order: {body}")

    print("Batch complete")

Grant permission to consume messages

The function's role acts as its identity inside AWS Identity and Access Management (IAM). The AWSLambdaSQSQueueExecutionRole policy grants the queue and logging permissions required by the consumer.

Permission changes can feel delicate. This managed policy keeps the added access focused on SQS consumption and function logging.

Switch back to the order-processor function page.
Choose Configuration.
Choose Permissions.
Choose the linked Role name.

The IAM role page shows the policies currently attached to your function's identity.

Choose Add permissions.
Choose Attach policies.
Search for AWSLambdaSQSQueueExecutionRole.
Select AWSLambdaSQSQueueExecutionRole.
Attach the selected policy to the role.
Confirm AWSLambdaSQSQueueExecutionRole appears in the role's permissions.

That permissions checkpoint is complete. Your function can now consume queue messages through its execution role.

What does the policy allow?

The policy lets the function receive messages from Amazon SQS. It also lets the function delete successfully processed messages.

Logging permissions send your handler's printed output to CloudWatch Logs. Those logs provide evidence that each order reached the processor.

Policy attachment denied?

Confirm that you selected the role linked from the order-processor permissions page.
Ask your AWS administrator for permission to attach the AWS managed policy if IAM blocks the action.
Help me troubleshoot why I cannot attach the SQS execution policy.

Connect the queue and process an order

The event source mapping polls orders-source for available messages. It invokes your function with each retrieved batch.

Return to the order-processor function from earlier.
Choose Add trigger under Function overview.

The trigger panel opens with the available event sources for your function.

Select SQS as the trigger source.
Choose orders-source as the SQS queue.

The trigger configuration now points at the source queue you tested earlier.

Keep Batch size set to 10.
Leave Enable trigger selected.

These settings allow the mapping to deliver up to ten available messages in one invocation.

Choose Add.
Confirm the SQS trigger appears as enabled in the function overview.

Your automatic connection is live. The queue can now invoke the processor without manual polling.

How does automatic polling work?

The event source mapping polls Amazon SQS on the function's behalf. Available messages arrive in the handler through event["Records"].

A successful batch response lets Lambda delete the processed messages from the queue. The function and queue stay independent while the mapping moves work between them.

Switch back to the Amazon SQS console from earlier.
Select orders-source.
Choose Send and receive messages.
Enter ORDER-1002 under Message body.
Choose Send message.

CloudWatch Logs records the output printed by each Lambda invocation. The newest log stream lets you confirm which path the handler completed.

Before you inspect the latest log stream, which success markers do you expect the handler to print after it reads ORDER-1002?

Return to the order-processor function.
Choose Monitor.
Choose View CloudWatch logs.
Open the latest log stream.

You'll see Processed order: ORDER-1002 followed by Batch complete.

Successful output missing?

Refresh the log streams after a short wait if the newest invocation is still arriving.
Confirm the SQS trigger remains enabled on order-processor.
Confirm orders-source and order-processor are both in us-east-1.
Help me find why ORDER-1002 is missing from my Lambda logs.

Your queue now hands valid orders to Lambda automatically. Next, you'll test how this pipeline behaves when processing fails.

Watch a Poison Message Break the Pipeline

You have a working asynchronous path from Amazon SQS to AWS Lambda. That is a strong milestone because ORDER-1002 reached the processor automatically.

A poison message triggers the same function error every time it is processed. Without a failure-isolation path, the 30-second visibility timeout makes the message available for another attempt.

You will send one poison message through the working pipeline. You will watch its failed invocations repeat in Amazon CloudWatch Logs.

In this step, get ready to:

Send FAIL through the source queue.
Observe at least two failed Lambda invocations in CloudWatch Logs.
Compare the repeated failure with the successful ORDER-1002 invocation.

Send the poison message

The function treats the exact body FAIL as an invalid order. This gives you a repeatable way to expose the pipeline's current failure behavior.

Return to the Amazon SQS console from earlier.
Select orders-source.
Choose Send and receive messages.
Enter FAIL under Message body.

Before you send it, do you expect this message to disappear after one processing attempt or return for another attempt?

Choose Send message.

The send confirmation proves that Amazon SQS accepted FAIL. Processing success still depends on the Lambda invocation.

Watch the retry loop

A failed invocation leaves the message undeleted. After the visibility timeout expires, Amazon SQS makes the message available to Lambda again.

This check takes at least one minute because you need to wait through two 30-second visibility-timeout cycles.

Return to the AWS Lambda console from earlier.
Select order-processor.
Choose Monitor.
Choose View CloudWatch logs.
Wait through at least two 30-second visibility-timeout cycles.
Refresh the log streams.
Open the newest log stream.
Inspect the visible events for Received message: FAIL.

You should see Received message: FAIL paired with Invalid order message in the failed invocation.

Open the next newest log stream if the attempts appear in separate streams.
Compare the timestamps or invocation boundaries for the failed attempts.

This is the intended shortfall. The source queue has no bounded failure-isolation path, so FAIL becomes eligible for another attempt after every failed cycle.

Only see one failed invocation?

Wait through one more 30-second visibility-timeout cycle.
Confirm that Enable trigger remains selected for the SQS event source mapping.
Check that the console Region still shows us-east-1.

If retries still do not appear, help me diagnose why order-processor only logged FAIL once.

Compare success with failure

A successful batch lets Lambda delete its SQS message. A thrown exception leaves the failed message available for another delivery cycle.

Locate the earlier invocation that contains Processed order: ORDER-1002.
Open its log stream.

You should see Processed order: ORDER-1002 followed by Batch complete in that successful invocation.

Before the final comparison, which message do you expect to reach Batch complete?

Return to the failed log events for FAIL.
Refresh the latest log events.
Compare their timestamps or invocation boundaries.

You should see Received message: FAIL followed by Invalid order message in at least two failed invocations. The earlier ORDER-1002 invocation reaches Batch complete once.

You have exposed the pipeline's uncontrolled retry behavior. Next, you will bound those retries and isolate the poison message in its own queue.

Isolate Failures with a Dead-Letter Queue

Your earlier Amazon CloudWatch Logs check proved that FAIL keeps returning to order-processor after each 30-second visibility timeout. Each retry consumes AWS Lambda capacity without moving the order toward success.

You will now add a dead-letter queue to isolate that poison message. A redrive policy caps the source queue at five receives before Amazon SQS moves the failed message aside.

In this step, get ready to:

Create orders-dlq with a 14-day message retention period.
Configure orders-source to move messages after five receives.
Verify that FAIL reaches orders-dlq after its bounded retries.

Create the dead-letter queue

A dead-letter queue gives unprocessable messages their own holding area. You will give this queue a 14-day retention period so failed orders remain available for investigation.

Your earlier usage check covers the risk here: this test adds only a handful of SQS requests.

Switch back to the Amazon SQS console from earlier.
Choose Create queue.
Keep Type set to Standard.
Enter orders-dlq in the Name field.
Set Message retention period to 14 days.
Choose Create queue.

You have cleared the first hurdle: orders-dlq is ready to receive failed orders.

Configure the redrive policy

The redrive policy tells the source queue when repeated delivery has reached its limit. Setting Maximum receives to 5 gives the poison message bounded retries before isolation.

Select orders-source from the queue list.
Choose Edit.
Find the Dead-letter queue section.
Enable Dead-letter queue.
Choose the ARN for orders-dlq in the dead-letter queue field.
Set Maximum receives to 5.
Choose Save.

The queue details should now show orders-dlq as the dead-letter queue. The maximum receive count should show 5.

Cannot Save the Redrive Policy?

Confirm that orders-source uses the Standard queue type.
Confirm that orders-dlq is in us-east-1.

Help me diagnose the redrive policy.

Verify failure isolation

The existing FAIL message may already be near the receive limit because of the retries you observed. The redrive policy applies when the message reaches its next receive attempt.

A fresh test can take several minutes because the 30-second visibility timeout controls when each retry begins.

Before you poll, which queue do you expect to hold FAIL after its fifth failed receive?

Allow the existing FAIL message to complete its remaining receive attempts.
Return to orders-dlq in the Amazon SQS console.
Choose Send and receive messages.
Choose Poll for messages.

You should see one retrieved message in orders-dlq.

Open the retrieved message to inspect its details.

Its Message body should show FAIL.

Leave the message in orders-dlq without choosing Delete.
Switch back to the order-processor log group from earlier.
Wait through one more 30-second visibility-timeout cycle.
Refresh the latest log streams.

The earlier Invalid order message entries remain as history. No new retry error appears for FAIL after it moves to orders-dlq.

Because you left the message undeleted, FAIL becomes visible in orders-dlq again after the wait.

Is the DLQ Still Empty?

Confirm that the SQS trigger on order-processor remains enabled.
Send one fresh FAIL message to orders-source from Send and receive messages.
Wait for five receive attempts before polling orders-dlq again.

Help me work out why the message has not moved.

You have closed the failure loop: poison orders now stop consuming Lambda retries after their bounded attempts.

Secret mission

Sound the Alarm When Orders Fail

Your dead-letter queue protects the order pipeline from repeated failures. Now create an alarm that makes a visible failed order impossible to overlook.

Clean Up Your Resources

Your AWS pipeline uses resources with monthly free-tier allowances. The options below let you keep everything active, pause order processing, or delete the stack.

Cost warning

This lab is expected to stay within the monthly free-tier allowances when total usage remains below the included limits.

Existing account usage counts toward the same allowances. Charges can occur if your combined monthly usage exceeds them.

Resources you used:

The AWS Lambda function order-processor with an enabled Amazon SQS event source mapping from orders-source using a batch size of 10.
The Standard source queue orders-source plus the Standard dead-letter queue orders-dlq containing the visible FAIL message.
The AWS Identity and Access Management (IAM) execution role linked to order-processor with AWSLambdaSQSQueueExecutionRole attached.
The Amazon CloudWatch Logs log group /aws/lambda/order-processor.
The Amazon CloudWatch alarm orders-dlq-has-messages.

Keep everything running

No action is needed while you are still practicing with the pipeline.

Review the Free Tier page regularly for current month-to-date usage.
Poll orders-dlq regularly for isolated failed messages.
Check orders-dlq-has-messages manually because the alarm has no notification action.

Pause - I'll come back to this later

Pausing disables Lambda polling while preserving every resource for another session.

Switch back to AWS Lambda in the AWS Management Console.
Select order-processor.
Select the SQS trigger for orders-source under Function overview.
Disable the event source mapping.

The event source mapping should show a disabled state. This confirms that AWS Lambda has stopped polling orders-source.

All other resources remain available until you return.

Delete - I don't want to use this again

Deleting this stack is permanent. Every resource can be rebuilt by following this project again.

Delete the Lambda Function

The function controls automatic order consumption. Its generated execution-role name is the only user-specific value you need for the final cleanup.

Switch back to AWS Lambda in the AWS Management Console.
Select order-processor.
Choose Configuration.
Choose Permissions.
Record the linked Role name here: your Lambda execution role name.

Good, the role name is captured. You can remove the consumer without losing track of its remaining IAM resource.

Choose Actions on the function page.
Choose Delete.
Type confirm in the confirmation field.
Choose Delete again.

You should no longer see order-processor in the function list. Its SQS event source mapping is removed with the function.

Delete the SQS Queues

The queues hold incoming orders plus isolated failures. Removing both clears all project messages.

Switch back to Amazon SQS.
Select orders-source.
Choose Delete.
Enter confirm.
Choose Delete.

The source queue should disappear from the queue list.

Select orders-dlq.
Choose Delete.
Enter confirm.
Choose Delete.

The dead-letter queue should disappear from the queue list. The isolated FAIL message is deleted with it.

Delete the Log Group

The log group stores the successful invocation plus the poison-message retry history.

Switch back to Amazon CloudWatch Logs.
Select /aws/lambda/order-processor from the log groups list.
Choose Actions.
Choose Delete log group(s).
Confirm the deletion in the dialog.

You should no longer see /aws/lambda/order-processor in the log groups list.

Delete the Alarm

The alarm monitors the DLQ metric that no longer has a queue to report on.

Switch back to Amazon CloudWatch.
Choose Alarms.
Choose All alarms.
Select orders-dlq-has-messages.
Choose Actions.
Choose Delete.
Confirm the deletion in the dialog.

You should no longer see orders-dlq-has-messages under All alarms.

Delete the IAM Role

The execution role remains after the function is deleted. Removing it completes the teardown.

Switch back to AWS Identity and Access Management (IAM).
Choose Roles.
Search for your Lambda execution role name.
Select the matching role.
Choose Delete.
Confirm the deletion in the dialog.

That completes the teardown. Your function, event source mapping, queues, logs, alarm, and generated execution role are removed.

Nice Work!

You did it! You built a fault-tolerant serverless order pipeline from Amazon SQS to AWS Lambda.

Your pipeline processes valid orders automatically. Poison messages now move into a dead-letter queue after bounded retries.

You've learned how to:

Built a visible asynchronous pipeline that consumes queued orders through an event source mapping. Applied an AWS Identity and Access Management (IAM) execution role with the permissions required for queue processing.
Used Amazon CloudWatch Logs to compare successful processing with repeated poison-message failures. This exposed visibility timeout behavior and automatic retries.
Added a redrive policy that moves failed orders into a dead-letter queue after five receives. The resulting design protects valid orders from uncontrolled failure loops.
Secret Mission: Created an Amazon CloudWatch alarm that entered ALARM when orders-dlq contained the visible FAIL message.

Ready to quiz yourself?