Zero-Downtime Deployments on AWS

Deploy a serverless app with canary routing and live rollback on AWS.

Introduction

30 Second Summary

Last Friday, someone on your team pushed a code update to the company's API. It had a bug. Within seconds, every single customer started seeing errors. and no one noticed it until today.

You're a Cloud Engineer, and your manager just handed you the incident report. Your first real assignment: make sure this never happens again. In this project, you will plan a serverless architecture, build it on AWS, deliberately recreate the Friday incident, and then fix it with canary deployments.

What You'll Build

When you are done, you'll have a deployment system that makes sure last Friday never happens again. You can push a code update, watch exactly which version handles each request, and prove that only 10% of users ever see untested code.

By the end of this project, you'll have:

  • An architecture diagram drawn using the same whiteboarding approach that AWS Solutions Architects use in real client engagements.
  • Two live URLs you can visit in your browser: a dev endpoint that always runs the latest code, and a prod endpoint protected by alias routing that only serves tested versions.
  • Real proof of canary deployments in action, with traffic split data visible in DynamoDB and CloudWatch showing which version handled which request.
  • Secret Mission: Race your fellow participants to roll back a broken production deployment in under 10 seconds, measuring your recovery time in real time.

What do I need for this workshop?

Grab a whiteboard or large piece of paper, some markers, and a phone or camera for photographing your architecture diagram. All AWS work happens in the browser, so no additional software is needed.

Before We Start

Your manager handed you the incident report from last Friday's outage. Before you touch the AWS console, take a moment to lock in the problem you're solving and what a successful fix looks like.

Architect Your Zero-Downtime Solution

The incident report tells the full story: someone deployed untested code, 100% of traffic hit it instantly, and every customer saw errors for 45 minutes. Your job is to design a serverless architecture on AWS that makes this impossible.

But jumping straight into the console is how mistakes happen. This is exactly how AWS Solutions Architects work in real client engagements: they show the architecture first, mapping out services and traffic flow before creating a single resource.

In this step, get ready to:
  • Draw a serverless architecture diagram using physical whiteboards
  • Discuss the blast radius problem and identify where weighted routing fits.
  • Photograph your completed architecture plan.
Confirm your AWS account is ready

The next step builds every resource in the AWS Management Console. You need an AWS account before you can create anything.

✔️ I can sign in

You're all set. Skip ahead to the next substep.

ⓧ I need to set one up

Creating an AWS account is free. New accounts receive $100 in credits on sign-up, with up to $100 more available as you explore services.

  • Open the AWS sign-up page.
  • Enter your email address and an account name, then click Verify email address.
  • Check your inbox for a verification code and enter it.
  • Create a strong root user password and click Continue.
  • Select the Free account plan.
  • Enter your contact information and accept the AWS Customer Agreement.
  • Add a payment method. AWS requires a card on file but does not charge you on the Free plan.
  • Verify your identity with a phone number when prompted.
  • Choose the Basic (Free) support plan and click Complete sign up.
  • Wait for the activation email. Activation usually takes a few minutes but can take up to 24 hours.
  • Once activated, sign in to the AWS Management Console to confirm your account is working.

Haven't received the activation email?

Check your spam or junk folder. If it has been more than 24 hours, contact AWS Support for help activating your account.

Help me troubleshoot my AWS account activation.

Draw your architecture diagram

Your architecture needs five components. Each one plays a specific role in handling traffic, routing deployments, and proving what happened.

Gather your team at the whiteboard and figure out how to diagram them using AWS Architecture Icons conventions: labeled boxes for each service, arrows for traffic and data flow, and dashed boundaries for logical groups.

  • Users/Internet — the traffic source. Every request to your application starts here.
  • Lambda Function URL — a stable HTTPS endpoint that never changes, no matter how many versions of your code exist behind it.
  • AWS Lambda function with a prod alias and numbered published versions (v1, v2). The alias is the traffic routing decision point that controls which version of your code handles each request.
  • DynamoDB table called deployment-log. This records every request so you have proof of which version handled which user.
  • CloudWatch — your monitoring layer for logs and alarms.

What are AWS Architecture Icons?

AWS Architecture Icons are the official icon set AWS provides for creating architecture diagrams. They are designed to be simple so you can easily use them in diagrams.

Solutions Architects use these conventions to communicate system designs with clients and teams. Labeled boxes represent services, arrows represent traffic or data flow, and dashed boundaries group related components.

Your diagram should show how a request flows from a user all the way to the database, and where monitoring fits in. There is no single correct layout, but every component and connection should be accounted for.

Discuss the blast radius problem

Now that your diagram shows the happy path, consider what happens when things go wrong. This discussion reveals the core problem your entire project solves.

  • Discuss as a team: what happens if you push a bad version and point the alias straight to it?

Every single user gets the broken code instantly. The only way to undo it is to manually point the alias back to the previous version. This is the blast radius problem.

  • Now discuss: what if you could send only 10% of traffic to the new version first?
  • Mark on your diagram where the weighted routing decision happens. It is at the alias level, between the Function URL and the published version of the function code.

What is blast radius?

Blast radius is the percentage of users affected when a deployment goes wrong. When an alias points directly to a new version, the blast radius is 100%.

Canary deployments reduce the blast radius by routing only a small percentage of traffic to the new version first. If the new version is healthy, you gradually increase the percentage. If it is broken, only a fraction of users were affected.

Photograph your completed diagram

Your architecture plan is complete. Before you move on, verify that your diagram has everything you need.

  • Check that your diagram includes all five components: Users, Lambda Function URL, Lambda function with versions and prod alias, DynamoDB, and CloudWatch.
  • Verify that arrows show the request flow from Users through the Function URL to the prod alias to a versioned function to DynamoDB.
  • Confirm that you have marked where the weighted routing decision happens (at the alias level).
  • Photograph your whiteboard diagram. You will compare it against the real AWS resources after you build them in the next step.

Keep your photo handy

Save the photograph somewhere easy to access. At the end of the next step, you will open it side by side with the AWS console to see how each box on the whiteboard maps to a real running resource.

Your architecture plan is on the whiteboard and photographed. Next up, you will bring every box on that diagram to life as a real AWS resource.

Build the Architecture

Your whiteboard diagram maps out the architecture that would have prevented last Friday's outage. Now it's time to make it real.

In this step, you'll create each component from your diagram as a live AWS resource. You'll set up a DynamoDB table for logging requests, deploy a Lambda function with the v1 application code, and create two separate endpoints for dev and prod.

In this step, get ready to:
  • Create a DynamoDB table and a Lambda function with the v1 application code.
  • Publish a version, create a prod alias, and set up dev and prod Function URLs.
  • Test both endpoints and verify request logs in DynamoDB.
Create the DynamoDB table and Lambda function

Every box on your whiteboard needs a real AWS resource behind it. You'll start with the database that logs every request, then build the function that serves your application.

  • For Table name, enter deployment-log.
  • For Partition key, enter request_id and leave the type as String.
  • Under Table settings, select Customize settings.
  • Under Read/write capacity settings, select On-demand.
  • Leave everything else as default, scroll down and click Create table.
  • Wait until the table status shows Active. This usually takes a few seconds.

Why on-demand capacity?

On-demand mode means DynamoDB automatically scales to handle whatever traffic you send it. You only pay for the requests you actually make, and there is no capacity to pre-configure.

For this workshop, on-demand keeps things simple and stays within the always-free tier.

SETUP LAMBDA

  • Select Author from scratch.
  • For Function name, enter canary-deploy-app.
  • For Runtime, select Python 3.14 (or the latest Python version available).
  • Leave Architecture as x86_64.
  • Click Create function.
  • You should see a green banner confirming the function was created successfully.

The code editor now shows some default code. You'll replace it with the v1 application code. This code handles incoming requests, logs them to DynamoDB, and returns a full-page SaaS-style dashboard with deployment info in a header bar.

  • In the code editor, select all the default code and delete it.
  • Paste the following imports and setup at the top of the file:
import json
import boto3
from datetime import datetime, timezone

dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table("deployment-log")

What does this setup do?

  • boto3 is the AWS SDK for Python. It comes pre-installed in the Lambda Python runtime, so no extra installation is needed.
  • dynamodb.Table("deployment-log") connects your function to the DynamoDB table you just created.
  • Below the setup, leave a blank line and paste the handler function:
def lambda_handler(event, context):
    # Capture request details for this invocation
    request_id = context.aws_request_id
    version = "1"
    timestamp = datetime.now(timezone.utc).isoformat()
    status = "healthy"

    # Log this request to DynamoDB
    table.put_item(
        Item={
            "request-id": request_id,
            "version": version,
            "timestamp": timestamp,
            "status": status,
        }
    )

What does this function do?

  • lambda_handler is the entry point that runs every time the function is invoked.
  • It captures a unique request_id from the Lambda context, along with the version number, a UTC timestamp, and a health status.
  • It writes all of these details to DynamoDB with put_item. This is how you track which version handled each request.
  • Directly below the code you just pasted, leave a blank line and add the HTML dashboard return block:
    # Build a SaaS-style dashboard page
    html = f"""<!DOCTYPE html>
<html><head><title>NextWork Deploy</title></head>
<body style="margin:0;font-family:system-ui,sans-serif;background:#0f172a;color:#e2e8f0">
<nav style="display:flex;align-items:center;gap:16px;padding:12px 24px;background:#1e293b;border-bottom:1px solid #334155">
<span style="font-size:14px;font-weight:600;color:#f8fafc">NextWork Deploy</span>
<span style="background:#1d4ed8;color:#fff;font-size:11px;padding:2px 8px;border-radius:9999px;font-weight:600">v{version}</span>
<span style="display:flex;align-items:center;gap:4px;font-size:12px;color:#4ade80"><span style="width:6px;height:6px;background:#4ade80;border-radius:50%;display:inline-block"></span>{status}</span>
<span style="font-size:11px;color:#64748b;margin-left:auto">{timestamp}</span></nav>
<main style="padding:32px 24px;max-width:900px;margin:0 auto">
<h1 style="font-size:22px;font-weight:600;margin:0 0 24px;color:#f8fafc">Dashboard</h1>
<div style="display:grid;grid-template-columns:repeat(3,1fr);gap:16px;margin-bottom:32px">
<div style="background:#1e293b;border-radius:10px;padding:20px"><div style="font-size:12px;color:#64748b;margin-bottom:4px">Total Requests</div><div style="font-size:28px;font-weight:700;color:#f8fafc">12,847</div><div style="font-size:12px;color:#4ade80;margin-top:4px">+14.2% from last week</div></div>
<div style="background:#1e293b;border-radius:10px;padding:20px"><div style="font-size:12px;color:#64748b;margin-bottom:4px">Avg Latency</div><div style="font-size:28px;font-weight:700;color:#f8fafc">42ms</div><div style="font-size:12px;color:#4ade80;margin-top:4px">-8.1% from last week</div></div>
<div style="background:#1e293b;border-radius:10px;padding:20px"><div style="font-size:12px;color:#64748b;margin-bottom:4px">Uptime</div><div style="font-size:28px;font-weight:700;color:#f8fafc">99.97%</div><div style="font-size:12px;color:#64748b;margin-top:4px">Last 30 days</div></div></div>
<div style="background:#1e293b;border-radius:10px;padding:20px">
<div style="font-size:14px;font-weight:600;margin-bottom:12px;color:#f8fafc">Active Services</div>
<div style="display:flex;justify-content:space-between;padding:8px 0;border-bottom:1px solid #334155;font-size:13px"><span>api-gateway</span><span style="color:#4ade80">Running</span><span style="color:#64748b">us-east-1</span></div>
<div style="display:flex;justify-content:space-between;padding:8px 0;border-bottom:1px solid #334155;font-size:13px"><span>auth-service</span><span style="color:#4ade80">Running</span><span style="color:#64748b">us-east-1</span></div>
<div style="display:flex;justify-content:space-between;padding:8px 0;font-size:13px"><span>worker-queue</span><span style="color:#4ade80">Running</span><span style="color:#64748b">us-east-1</span></div></div>
</main></body></html>"""

    # Return the dashboard as an HTML page
    return {
        "statusCode": 200,
        "headers": {"Content-Type": "text/html"},
        "body": html,
    }

What does the return block do?

  • The f"""...""" syntax creates a Python f-string that injects the version, status, and timestamp directly into the HTML.
  • The header bar at the top shows a version badge, a green status indicator, and the request timestamp. The rest of the page renders mock SaaS metrics (requests, latency, uptime) and a services list.
  • Content-Type: text/html tells the browser to render the response as a web page instead of raw text.
  • Click Deploy.

A green banner appears with the message "Successfully updated the function canary-deploy-app." That confirms your code is saved and ready.

✔️ Awesome, I've got everything!

Great. Double check you have clicked Deploy and seen the green success banner.

ⓧ I'd like to double check the full code

Scroll up and confirm you have all three code blocks in the editor, in this order:

  • The imports and setup block (import json, import boto3, from datetime, and the two dynamodb setup lines).
  • The lambda_handler function with the put_item call to DynamoDB.
  • The HTML dashboard return block, starting with html = f""" and ending with the return statement.

Make sure there is a blank line between the setup lines and the def lambda_handler line. The HTML return block should be inside the handler function, directly below put_item.

Seeing a syntax error after deploying?

  • Make sure you deleted all the default code before pasting. Leftover lines from the default template can cause syntax errors.
  • Check that the imports block and the handler function are not pasted on the same line. There should be a blank line between them.

Still stuck? Why is my Lambda function showing a syntax error after I pasted the v1 code?

The function is deployed, but it does not yet have permission to write to DynamoDB. You need to attach the DynamoDB access policy to the function's execution role.

  • Click the Configuration tab (below the code editor).
  • Click Permissions in the left sidebar.
  • Under Execution role, click the role name link. This opens the IAM console in a new tab.
  • On the role page, click Add permissions, then Attach policies.
  • Search for AmazonDynamoDBFullAccess, select the checkbox next to it, and click Add permissions.
  • You should see AmazonDynamoDBFullAccess listed under the role's attached policies before you close the IAM tab.
  • Close the IAM tab and return to the Lambda console.

Why does Lambda need a separate permission?

AWS services follow the principle of least privilege. A Lambda function can only access other AWS services if its execution role explicitly grants permission.

Without AmazonDynamoDBFullAccess, the function's code would run, but the table.put_item() call would fail with an access denied error.

🙋‍♀️ Don't see the role name link?

  • Make sure you are on the Configuration tab, not the Code tab. The Permissions sidebar only appears under Configuration.
  • The role name is a clickable blue link under the Execution role heading. It opens in a new browser tab.

Still stuck? How do I find and update my Lambda function's IAM execution role?

Publish version 1 and create your endpoints

Your function code is deployed to $LATEST, which is a mutable pointer that always reflects your most recent code. To protect production, you need an immutable snapshot of this code (a published version) and an alias that routes traffic to it.

  • In the Lambda console, make sure you are on the canary-deploy-app function page.
  • Click the Actions dropdown (upper-right area of the code editor) and select Publish new version.
  • Enter v1 stable as the description and click Publish.

You are taken to the version 1 page. This version is an immutable snapshot. The code inside it can never be changed.

What is a Lambda version?

A published version is a frozen copy of your function code and configuration. Once published, it never changes. This is what makes rollbacks instant. You can always point traffic back to a known-good version without redeploying anything.

The function itself ($LATEST) is mutable. Every time you click Deploy, $LATEST changes. A published version is a separate, numbered snapshot that stays locked even as $LATEST keeps moving forward.

  • Click the Action tab and click Create alias.
  • For Name, enter prod.
  • For Description, enter Production traffic.
  • For Version, select 1 from the dropdown.
  • Click Save.

The prod alias now points to version 1. This is the routing layer from your whiteboard diagram. Any traffic hitting this alias goes to version 1.

Now you need two separate URLs: one for prod (attached to the alias) and one for dev (attached to $LATEST). This matches the two endpoints on your whiteboard.

  • While still on the prod alias page, click the Configuration tab.
  • Click Function URL in the left sidebar.
  • Click Create function URL.
  • For Auth type, select NONE.
  • Click Save.
  • Copy the Function URL that appears and save it somewhere (a text file or sticky note). This is your prod URL.

What is a Lambda Function URL?

A Function URL gives your Lambda function a dedicated HTTPS endpoint. It follows the format https://<url-id>.lambda-url.<region>.on.aws. Because this URL is attached to the prod alias, it always routes to whatever version the alias points to.

  • Now create the dev URL. Go back to the main function page by clicking canary-deploy-app in the breadcrumb navigation at the top.
  • Make sure you are on the $LATEST version (not the alias or a version number).
  • Click the Configuration tab, then Function URL in the left sidebar.
  • Click Create function URL.
  • For Auth type, select NONE.
  • Click Save.
  • Copy this Function URL and save it alongside the first one. This is your dev URL.

You now have two endpoints. The prod URL routes through the alias to a published, immutable version. The dev URL always runs whatever code is in $LATEST.

Don't see the Function URL option?

  • For the prod URL, you must be on the prod alias page (not the main function page or a version page). Click the Aliases tab, then click prod to navigate there.
  • For the dev URL, you must be on the main function page showing $LATEST. Click the function name canary-deploy-app in the breadcrumb.

Still stuck? How do I create Function URLs on my Lambda alias and on $LATEST?

Test both endpoints and verify DynamoDB

Every component from your whiteboard diagram now has a real AWS resource behind it. Before moving on, you need to prove they all work together.

  • Open the prod URL in a new browser tab.

Before you look at the response, what do you expect to see? Think about what the lambda_handler function returns.

You should see a full-page dashboard that looks like a real SaaS app. The header bar in the top-left shows a v1 badge, a green healthy status, and a timestamp.

  • Open the dev URL in another browser tab.

You should see the same dashboard page with version 1 and status healthy in the header bar. Both URLs are running the same v1 code right now.

  • Now verify that DynamoDB is logging requests. Open the DynamoDB console.
  • Click Tables, then click deployment-log.
  • Click Explore table items.

You should see at least two items (one from each URL visit). Each item has four fields: request_id, version, timestamp, and status.

Why log every request to DynamoDB?

These log entries are your proof of which version handled which request. When you set up canary deployments later, you'll scan this table and count how many rows came from each version to verify the traffic split is real.

Compare what you see in the AWS console to your whiteboard diagram. Every box on the whiteboard now has a real running resource behind it.

✔️ I see the dashboard

You should see a full-page SaaS dashboard with v1 and healthy in the header bar on both your prod and dev URLs. You're all set.

ⓧ I'm getting an Internal Server Error

An Internal Server Error means the function code is running but crashing. Work through these fixes in order until the error clears.

Fix 1: Attach the DynamoDB policy
  • Open your Lambda function's Configuration tab.
  • Click Permissions in the left sidebar.
  • Click the execution role link under Execution role. This opens IAM in a new tab.
  • Confirm AmazonDynamoDBFullAccess appears under the attached policies.
  • If it is missing, click Add permissions, then Attach policies.
  • Search for AmazonDynamoDBFullAccess, select the checkbox, and click Add permissions.
  • Refresh your Function URL in the browser and check if the error clears.
Fix 2: Confirm the table name matches the code
  • Open the DynamoDB console and click Tables.
  • The table name must be exactly deployment-log (case-sensitive, with a hyphen). The code references this exact string in dynamodb.Table("deployment-log").
  • If the name is different, delete the table and recreate it with the correct name.
Fix 3: Confirm you clicked Deploy
  • Go to your Lambda function's Code tab.
  • If you see the default Hello from Lambda code instead of your v1 imports and handler, paste the correct code from the code blocks above.
  • Click Deploy.
  • Refresh your Function URL in the browser.
Fix 4: Republish if you published before deploying
  • If you published version 1 before clicking Deploy, that version captured the default starter code.
  • Deploy the correct code first (Fix 3 above).
  • Click Actions and select Publish new version to create version 2.
  • Click the Aliases tab.
  • Click prod, then click Edit.
  • Change the version to the new number and click Save.
  • Refresh your prod Function URL in the browser.
Fix 5: Check for a region mismatch
  • Open the Lambda console and navigate to your canary-deploy-app function.
  • Look at the region dropdown in the top-right corner of the console.
  • Now open the DynamoDB console and click Tables.
  • Check the region dropdown again. It must show the same region as your Lambda function.
  • If the regions are different, delete the resource that is in the wrong region and recreate it in the matching region.

Still seeing the error?

Still stuck? My Lambda Function URL is returning an Internal Server Error. How do I debug it?

Your architecture is live. You have a DynamoDB table logging every request, a Lambda function running v1, and two separate endpoints for dev and prod. Next up, you'll push a bad update and see firsthand what happens when something goes wrong.

Deploy a Bad Update and See the Blast Radius

Every component on your whiteboard diagram is now a real, running AWS resource. Both your dev and prod URLs return healthy responses, and DynamoDB logs every request.

Time to recreate the Friday incident on purpose. You need to experience the exact failure mode from the incident report firsthand. In this step, you will deliberately break your application, see how the alias protects production, then bypass that protection and feel the full blast radius of a bad deployment.

In this step, get ready to:
  • Deploy buggy code and see the alias protect production.
  • Promote the buggy code to production and experience blast radius.
  • Roll back production with a single alias change.
Deploy buggy code to $LATEST

The prod alias points to version 1, a published snapshot of your code. The $LATEST version is where you edit code before publishing. If you overwrite $LATEST with broken code, only the dev URL (which points to $LATEST) should be affected. The prod URL should stay safe behind the alias.

  • Return to the Lambda console and open your canary-deploy-app function.
  • Make sure you are on the $LATEST version. Click the function name canary-deploy-app in the breadcrumb at the top if you are on a version or alias page.
  • In the code editor, select all the existing code and delete it.
  • Paste in this buggy replacement code:
import json


def lambda_handler(event, context):
    raise Exception("Bug in version 2! This version is broken.")

What does this code do?

This code immediately raises an exception on every invocation. It does not import boto3, does not connect to DynamoDB, and does not return a response. Every request that hits this code will crash.

  • Click Deploy to save the new code to $LATEST.

Now test what happened. The dev URL points to $LATEST, which now has buggy code. What do you think the dev URL will return?

  • Refresh the dev URL tab in your browser.

You should see an error response. The dev URL returns an Internal Server Error because $LATEST now contains the buggy code that crashes on every request.

Now think about the prod URL. It points to the prod alias, which points to version 1. You only changed $LATEST. What do you think the prod URL will return?

  • Refresh the prod URL tab in your browser.

The prod URL still shows the healthy v1 dashboard page. The alias is protecting production from the bad code in $LATEST.

Why does the alias protect production?

When you published version 1, Lambda took an immutable snapshot of the code at that moment. The prod alias points to that snapshot, not to $LATEST. No matter how many times you overwrite $LATEST, version 1 never changes.

This is alias insulation. The dev URL runs the latest code (risky), while the prod URL only serves published, tested versions (safe).

Promote the bad code to production

The alias kept production safe from the broken $LATEST code. But what happens when someone publishes that broken code as a new version and points the alias straight to it? No canary. No health check. 100% of traffic switches instantly.

  • Return to the Lambda console on the canary-deploy-app function page.
  • Click the Actions dropdown (top-right area of the code editor) and select Publish new version.
  • Enter v2 buggy as the description and click Publish.

You are taken to the version 2 page. The buggy code is now a permanent, published version.

  • Click the Aliases tab, then click the prod alias name.
  • Click Edit.
  • Change the Version dropdown from 1 to 2.
  • Click Save.

Every user hitting the prod URL now gets routed to version 2. Before you test, what do you think happens when you refresh the prod URL?

  • Refresh the prod URL tab in your browser.You increase from value version.

The prod URL returns an error. Every single user hitting this URL now sees the broken response. There was no warning, no gradual rollout, no health check. This is the blast radius of a bad deployment: 100% of traffic is broken instantly.

  • Switch back to the DynamoDB console tab.
  • Open the deployment-log table and click Explore table items.

Where are the new entries?

You will NOT see new entries from the buggy version. The v2-buggy code crashes immediately on the raise Exception line, before it ever calls table.put_item(). This means you lost visibility into the failure.

Not only is production broken, but there is no record of it in your request log. In a real system, this silent failure makes debugging even harder.

Roll back the alias to version 1

The fix is simple. The alias is just a pointer. Changing it back to version 1 instantly restores production without redeploying any code.

  • Return to the Lambda console and open your canary-deploy-app function.
  • Click the Aliases tab, then click the prod alias name.
  • Click Edit.
  • Change the Version dropdown from 2 back to 1.
  • Click Save.

You just pointed the alias back to the healthy version 1. What do you expect to see when you refresh the prod URL?

  • Refresh the prod URL tab in your browser.

The prod URL works again instantly. The rollback took one dropdown change and a save. No code redeployment. No waiting.

But the damage was already done. Every user who hit the URL between the bad promotion and the rollback saw an error.

Prod URL still showing an error?

Your browser may be caching the error response. Try a hard refresh: Cmd+Shift+R (macOS) or Ctrl+Shift+R (Windows).

If you are still seeing errors, double-check that the prod alias Version is set to 1 in the Lambda console.

Why is my prod URL still returning an error after I rolled back the alias?

✔️ Awesome, I've got everything!

Great. Double check you have screenshots of all three states: dev broken with prod healthy, prod broken after the alias change, and prod recovered after rollback.

ⓧ I'd like to double check the full code

The $LATEST code in your Lambda function should match the v2-buggy code below. The prod alias should be back on version 1.

import json


def lambda_handler(event, context):
    raise Exception("Bug in version 2! This version is broken.")

The alias rollback saved production in seconds. But pointing the alias straight to an untested version is what broke things in the first place. Next, you will learn how canary deployments prevent this by sending only a fraction of traffic to a new version first.

Fix It with Canary Deployments

You just relived the Friday incident. You pointed the prod alias directly to version 2, and every user immediately got the broken code. That blast radius happened because the alias routed 100% of traffic to the untested version with no safety net.

Canary deployments fix this. Instead of routing all traffic to a new version at once, you send only a small percentage first. If the new version is healthy, you gradually increase the traffic. If it breaks, only a fraction of users are affected.

In this step, get ready to:
  • Deploy a fixed version 3 and configure weighted alias routing.
  • Verify the 90/10 traffic split with CloudWatch Log Analytics and DynamoDB.
  • Promote version 3 to 100% of production traffic.
Deploy the fixed code and configure weighted routing

The fix is a new version of your function code. It has the same structure as the v1 code you deployed earlier, with version set to "3" instead of "1". Right now, $LATEST still contains the buggy v2 code, so you need to replace it.

  • Switch back to the Lambda console tab with your canary-deploy-app function open.
  • Click the function name canary-deploy-app in the breadcrumb at the top to return to the $LATEST version.
  • In the code editor, select all the code and delete it.
  • Paste in the following code:
import json
import boto3
from datetime import datetime, timezone

# Connect to the DynamoDB request log table
dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table("deployment-log")


def lambda_handler(event, context):
    # Capture request details for this invocation
    request_id = context.aws_request_id
    version = "3"
    timestamp = datetime.now(timezone.utc).isoformat()
    status = "healthy"

    # Log this request to DynamoDB
    table.put_item(
        Item={
            "request_id": request_id,
            "version": version,
            "timestamp": timestamp,
            "status": status,
        }
    )

What does this code do?

  • Connects to the deployment-log DynamoDB table so the function can record every request it handles.
  • Inside lambda_handler, it captures the request ID, version number, timestamp, and a healthy status.
  • It writes those details to DynamoDB with put_item. This is how you get real proof of which version handled each request.
  • Directly below the code you just pasted, add this return block:
    # Build a SaaS-style dashboard page
    html = f"""<!DOCTYPE html>
<html><head><title>NextWork Deploy</title></head>
<body style="margin:0;font-family:system-ui,sans-serif;background:#0f172a;color:#e2e8f0">
<nav style="display:flex;align-items:center;gap:16px;padding:12px 24px;background:#1e293b;border-bottom:1px solid #334155">
<span style="font-size:14px;font-weight:600;color:#f8fafc">NextWork Deploy</span>
<span style="background:#1d4ed8;color:#fff;font-size:11px;padding:2px 8px;border-radius:9999px;font-weight:600">v{version}</span>
<span style="display:flex;align-items:center;gap:4px;font-size:12px;color:#4ade80"><span style="width:6px;height:6px;background:#4ade80;border-radius:50%;display:inline-block"></span>{status}</span>
<span style="font-size:11px;color:#64748b;margin-left:auto">{timestamp}</span></nav>
<main style="padding:32px 24px;max-width:900px;margin:0 auto">
<h1 style="font-size:22px;font-weight:600;margin:0 0 24px;color:#f8fafc">Dashboard</h1>
<div style="display:grid;grid-template-columns:repeat(3,1fr);gap:16px;margin-bottom:32px">
<div style="background:#1e293b;border-radius:10px;padding:20px"><div style="font-size:12px;color:#64748b;margin-bottom:4px">Total Requests</div><div style="font-size:28px;font-weight:700;color:#f8fafc">12,847</div><div style="font-size:12px;color:#4ade80;margin-top:4px">+14.2% from last week</div></div>
<div style="background:#1e293b;border-radius:10px;padding:20px"><div style="font-size:12px;color:#64748b;margin-bottom:4px">Avg Latency</div><div style="font-size:28px;font-weight:700;color:#f8fafc">42ms</div><div style="font-size:12px;color:#4ade80;margin-top:4px">-8.1% from last week</div></div>
<div style="background:#1e293b;border-radius:10px;padding:20px"><div style="font-size:12px;color:#64748b;margin-bottom:4px">Uptime</div><div style="font-size:28px;font-weight:700;color:#f8fafc">99.97%</div><div style="font-size:12px;color:#64748b;margin-top:4px">Last 30 days</div></div></div>
<div style="background:#1e293b;border-radius:10px;padding:20px">
<div style="font-size:14px;font-weight:600;margin-bottom:12px;color:#f8fafc">Active Services</div>
<div style="display:flex;justify-content:space-between;padding:8px 0;border-bottom:1px solid #334155;font-size:13px"><span>api-gateway</span><span style="color:#4ade80">Running</span><span style="color:#64748b">us-east-1</span></div>
<div style="display:flex;justify-content:space-between;padding:8px 0;border-bottom:1px solid #334155;font-size:13px"><span>auth-service</span><span style="color:#4ade80">Running</span><span style="color:#64748b">us-east-1</span></div>
<div style="display:flex;justify-content:space-between;padding:8px 0;font-size:13px"><span>worker-queue</span><span style="color:#4ade80">Running</span><span style="color:#64748b">us-east-1</span></div></div>
</main></body></html>"""

    # Return the dashboard as an HTML page
    return {
        "statusCode": 200,
        "headers": {"Content-Type": "text/html"},
        "body": html,
    }

What does the return block do?

This builds a full-page SaaS dashboard using a Python f-string and returns it with Content-Type: text/html. The browser renders it as a styled web page instead of raw data.

The header bar injects the version, status, and timestamp variables so each response reflects the current invocation. The rest of the page shows mock metrics and services.

  • Click Deploy.

Before you test, think about what you expect. The dev URL points to $LATEST. You just replaced the buggy code with healthy v3 code. What do you think the dev URL will return now?

  • Refresh your dev Function URL in the browser.

You should see a full-page SaaS dashboard with a v3 badge and healthy status in the header bar. The dev endpoint is working again.

Still seeing a 500 error on the dev URL?

  • Make sure you clicked Deploy after pasting the code. The code editor saves your changes, but Deploy is what publishes them to $LATEST.
  • Check that you are on the $LATEST version, not on a numbered version or the prod alias. The function name breadcrumb should not show a version number after it.
  • Verify you pasted both the put_item block and the return block. A missing return block causes a different error.

help me figure out why my dev URL still returns a 500 error after deploying v3 code

✔️ Awesome, I've got everything!

Great. Double check you see the v3 badge in the header bar when refreshing the dev URL, then continue below.

ⓧ I'd like to double check the full code

Your lambda_function.py in the code editor should contain these sections in order:

  • Three import lines: json, boto3, and datetime.
  • DynamoDB setup: dynamodb = boto3.resource("dynamodb") and table = dynamodb.Table("deployment-log").
  • The lambda_handler function with version = "3", a put_item call, and the HTML dashboard return block.

Scroll up and confirm each section is present. If anything is missing, paste it in, click Deploy, and refresh the dev URL.

Now publish this code as a permanent version and set up the canary routing.

  • Click the Actions dropdown in the upper-right area of the function page.
  • Select Publish new version.
  • Enter v3 fixed as the description.
  • Click Publish. You are taken to the version 3 page.
  • Click the Aliases tab.
  • Click the prod alias name.
  • Click Edit.
  • Change the Version dropdown to 1. After Step 3, this dropdown shows 2 (the buggy version). You want the proven-stable version 1 as the primary, receiving most of the traffic.
  • Expand the Weighted alias section.
  • Set Additional version to 3.
  • Set Weight (%) to 10.
  • Click Save.

What is a canary deployment?

You just configured a 90/10 traffic split. 90% of requests hitting the prod URL go to version 1 (the proven-stable version). Only 10% go to version 3 (the new code). This is a canary deployment.

The name comes from the old mining practice of sending a canary into a mine to detect danger. If the canary was fine, the miners followed. If version 3 is healthy, you promote it to 100%. If it breaks, only 10% of users were affected instead of everyone.

Generate traffic and verify the split

The weighted routing is configured, but you need real traffic to see it in action. You will send 50 requests to the prod URL and check where each one landed.

  • Open a terminal on your computer. On macOS, open Terminal. On Windows, open PowerShell. On Linux, open your preferred terminal app.
  • Generate 50 requests to your prod URL by running this command:

macOS

for i in $(seq 1 50); do curl -s [[PROD_URL="your prod Function URL"]] | grep -o '9999px[^>]*>v[0-9]*' | grep -o '[0-9]*$'; done

Windows

1..50 | ForEach-Object { $r = (curl.exe -s [[PROD_URL="your prod Function URL"]]) -join ''; if ($r -match '9999px[^>]*>v(\d+)') { $Matches[1] } }

Linux

for i in $(seq 1 50); do curl -s [[PROD_URL="your prod Function URL"]] | grep -o '9999px[^>]*>v[0-9]*' | grep -o '[0-9]*$'; done

Before you press Enter, think about what you expect. The alias sends 90% of traffic to version 1 and 10% to version 3. Out of 50 requests, roughly how many should show each version?

You should see a mix of 1 and 3 scrolling by. Most lines will show 1 and a few will show 3. The exact ratio varies because Lambda uses a probabilistic model, but you should see a rough 90/10 split.

✔️ I see a mix of 1 and 3

The weighted routing is working. Most lines show 1 and a few show 3. Continue below to verify the split in CloudWatch.

ⓧ I only see version 1

At low traffic volumes, Lambda's probabilistic routing can be uneven. Try running the loop again to generate another 50 requests. With 100 total requests, you are more likely to see version 3 appear.

Also confirm that the prod alias has the weighted routing configured. Return to the Lambda console, click the Aliases tab, and click prod. You should see version 1 as the primary and version 3 at 10% weight.

help me figure out why my weighted alias is not splitting traffic between version 1 and version 3

ⓧ I only see version 3

This happens when version 1 was published with different HTML than version 3. The curl command searches for a specific pattern in the SaaS dashboard header bar. If version 1 uses an older HTML template without that pattern, its responses produce no output and only version 3 lines appear.

To confirm, open the Lambda console, click the Versions tab, and click version 1. Check whether the code editor shows the same SaaS dashboard HTML (with the border-radius:9999px version badge) as version 3. If version 1 has older or different HTML, follow these steps to fix it:

  • Click the function name canary-deploy-app in the breadcrumb to return to $LATEST.
  • Select all the code in the editor and delete it.
  • Paste the v1 code from Step 2 (the same code as version 3, but with version = "1" instead of version = "3").
  • Click Deploy.
  • Click Actions, then Publish new version. This creates a new version number (e.g. version 4).
  • Click the Aliases tab, click prod, then click Edit.
  • Change the Version dropdown to your new version number (the one you just published with version = "1" code).
  • Set the Additional version back to 3 with Weight (%) at 10.
  • Click Save and run the curl loop again.

help me fix my version 1 code so the curl command can detect both versions in the traffic split

Now verify the split with CloudWatch Log Analytics, which can query your Lambda logs and count invocations per version.

  • Search for CloudWatch in the AWS console search bar and open it.
  • In the left sidebar, click Logs, then Log Analytics.
  • In the Search log groups... field near the top left, type /aws/lambda/canary-deploy-app and select it from the results.
  • Delete the default query in the query editor and paste in this query:
fields @message | filter @message like /START/ | parse @message "Version: *" as version | stats count() by version

What does this query do?

Lambda automatically writes a START log entry for every invocation that includes the version number. This query filters for those START lines, extracts the version number, and counts how many requests each version handled.

  • Set the time range to Last 1 hour using the dropdown above the query editor.
  • Click Run query.

The results table shows a count for each version. You should see a large count for version 1 [4] and a smaller count for version 3, roughly matching the 90/10 split you configured.

  • Return to the DynamoDB console. Open the deployment-log table and click Explore table items.
  • Scroll through the recent items and note the version field. You should see a mix of 1 and 3 entries matching the Log Analytics results.

Why check both CloudWatch and DynamoDB?

CloudWatch Log Analytics gives you an aggregate count. DynamoDB gives you the raw per-request data. In production, you would use the aggregate view (CloudWatch) for quick dashboards and the raw data (DynamoDB) for debugging individual requests. Having both gives you confidence the split is real.

Promote version 3 to full production

The Log Analytics data and DynamoDB entries confirm that version 3 is healthy. Time to shift 100% of traffic to it.

  • Return to the Lambda console and open your canary-deploy-app function.
  • Click the Aliases tab.
  • Click the prod alias name.
  • Click Edit.
  • Change the Version dropdown to 3.
  • Collapse or clear the Weighted alias section so there is no additional version.
  • Click Save.

What do you expect to see when you refresh the prod URL now? Will there still be a mix of versions, or only one?

  • Refresh the prod Function URL in your browser several times.

Every response should now show the version 3 dashboard. Refresh a few times to confirm you no longer see version 1.

Still seeing version 1 in some responses?

Make sure you removed the weighted alias by collapsing or clearing the Weighted alias section. If the additional version is still set, some traffic will continue going to the old version.

Also confirm the Version dropdown shows 3, not 1. Click Save again after making changes.

help me fix my prod alias so it routes 100% of traffic to version 3

Your canary deployment is complete. You safely tested version 3 with 10% of traffic, verified the split with real data, and promoted it to serve all users. Next up, you will add automated error detection so you know the moment something goes wrong.

Add Automated Error Detection

Your canary deployment controls the blast radius. But remember the incident report: it took 45 minutes to even realize something was wrong. Canary deployments only work if you know when something goes wrong.

A CloudWatch Alarm monitors a specific metric and changes state when a threshold is crossed. In this step, you will create an alarm on your Lambda function's Errors metric so you can watch it fire the moment a canary version starts failing.

In this step, get ready to:
  • Create a CloudWatch Alarm on your Lambda function's Errors metric.
  • Deploy a buggy canary version to trigger the alarm.
  • Watch the alarm fire in real time, then clean up the alias.
Create the CloudWatch Alarm

Right now, if a canary version starts failing, nobody knows unless they manually check. An alarm on the Lambda Errors metric automatically detects failures the moment they happen.

  • Search for CloudWatch in the AWS console search bar and select it.
  • In the left sidebar, click Alarms, then All alarms.
  • Click Create alarm.
  • Click Select metric.
  • In the Browse tab, click Lambda, then By Function Name.
  • Find canary-deploy-app and select the checkbox next to the Errors metric.
  • Click Select metric.I don't know how many tracks there are here

What is the Errors metric?

Lambda automatically tracks how many invocations end in an error. The Errors metric counts these failures over a time period you choose.

By setting a threshold of "greater than 0," your alarm fires the instant any invocation fails. In production, this alarm would trigger an automated rollback.

Now configure the alarm's conditions and name:

  • For Statistic, select Sum.
  • For Period, select 1 minute.
  • Under Conditions, select Greater than and enter 0 as the threshold.
  • Click Next.
  • On the Configure actions page, under Notification, click Remove to skip notifications. You do not need SNS for this project.
  • Click Next, procced without action
  • For Alarm name, enter deployment-error-alarm.
  • Click Next, then click Create alarm.

The alarm appears in the list with a state of Insufficient data. It will move to OK once it receives its first data point from your Lambda function.

Don't see the alarm?

Make sure you clicked Create alarm on the final review page. If you only clicked Next, the alarm was not saved.

Also confirm you are viewing Alarms > All alarms in the left sidebar, not a filtered view.

Help me find my CloudWatch Alarm in the console.

Deploy a buggy canary to test the alarm

To see the alarm fire, you need errors. You will deploy the same buggy code from Step 3 as a new canary version and send traffic to it.

  • Search for Lambda in the AWS console search bar and open it.
  • Click your canary-deploy-app function.
  • Make sure you are on the $LATEST version. If you see a version number or alias name in the breadcrumb, click the function name to return to $LATEST.
  • In the code editor, select all the code and delete it. Paste in the following buggy code:
import json


def lambda_handler(event, context):
    raise Exception("Bug in version 2! This version is broken.")

What does this code do?

This is the same buggy code you used in Step 3. It immediately raises an exception on every invocation. It does not write to DynamoDB, so failed requests leave no trace in your request log.

  • Click Deploy.
  • Open your dev URL in a new browser tab. You should see an error response (a 500 status). This confirms the buggy code is live on $LATEST.

Now publish this code as a new version and set up the canary split.

  • Click Actions and select Publish new version.
  • Enter v4 buggy as the description.
  • Click Publish.
  • Click the Aliases tab.
  • Click the prod alias name.
  • Click Edit.
  • Keep the Version dropdown set to 3.
  • Expand the Weighted alias section.
  • For Additional version, select 4.
  • For Weight (%), enter 10. This sends 90% of traffic to version 3 and 10% to version 4.
  • Click Save.

The canary is live. 10% of prod traffic now hits the buggy version 4, which will generate errors.

  • Open a terminal on your computer. On macOS, open Terminal. On Windows, open PowerShell. On Linux, open your preferred terminal app.
  • Send 100 requests to your prod URL by running this command:

macOS

for i in {1..100}; do curl -s "[[PROD_URL="your prod Function URL"]]"; echo; done | sort | uniq -c

Windows

1..100 | ForEach-Object { Invoke-WebRequest -Uri [[PROD_URL="your prod Function URL"]] -UseBasicParsing | Select-Object -ExpandProperty Content } | Group-Object | Select-Object Count, Name

Linux

for i in {1..100}; do curl -s "[[PROD_URL="your prod Function URL"]]"; echo; done | sort | uniq -c

What does this command do?

This loop sends 100 requests to your prod Function URL and prints each response. The sort | uniq -c part groups identical responses and counts them, so you see exactly how many requests hit each version.

With the 90/10 weighted alias, you should see roughly 90 successful responses from version 3 and roughly 10 error responses from buggy version 4. Those errors are exactly what your CloudWatch Alarm is watching for.

You should see two groups in the output: a large count of successful responses and a smaller count of error responses. The error responses came from buggy version 4.

Watch the alarm fire and clean up

Before you check the alarm, take a moment to predict. Do you think the alarm state has already changed, or is it still in OK (or Insufficient data)?

  • Search for CloudWatch in the AWS console search bar and open it.
  • In the left sidebar, click Alarms, then All alarms.
  • Watch the deployment-error-alarm. Within 1-2 minutes, its state should change to In alarm. The alarm state shows as a red banner

Why does it take a minute or two?

You configured the alarm with a 1-minute period. CloudWatch evaluates the Errors metric once per period. The alarm needs at least one full evaluation cycle with errors above the threshold before it transitions to the In alarm state.

  • Take a screenshot of the alarm showing the In alarm state.

The alarm detected the errors from your buggy canary version automatically. In production, this alarm would trigger an automated rollback or page an on-call engineer.

Now clean up the canary so prod is fully healthy again.

  • Search for Lambda in the AWS console search bar and open it.
  • Click your canary-deploy-app function.
  • Click the Aliases tab.
  • Click the prod alias, then click Edit.
  • Change the Version dropdown to 3.
  • Collapse or clear the Weighted alias section so there is no additional version.
  • Click Save.

Refresh your prod URL in the browser. You should see the version 3 dashboard with status healthy. The alarm will return to OK after the errors stop.

Alarm still showing In alarm?

The alarm evaluates on a 1-minute cycle. After you remove the buggy canary, it can take 1-2 more evaluation cycles with zero errors before the state transitions back to OK. Wait a couple of minutes and refresh the CloudWatch Alarms page.

If the alarm stays in In alarm for more than 5 minutes, confirm that the prod alias is pointing to version 3 with no weighted routing. Send a few test requests to your prod URL and verify they return healthy responses.

Help me troubleshoot my CloudWatch Alarm state.

Secret mission

The 10-Second Rollback

Everything you have built so far prevents bad code from reaching users. But bad deployments still happen. Race your fellow participants to roll back a broken production deployment as fast as possible, watching your endpoint recover in real time.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. All resources in this project run on the AWS always-free tier, so there are no ongoing costs or hourly charges.

Resources you used:

  • Lambda function canary-deploy-app (includes 4 published versions, the prod alias, and 2 Function URLs).
  • DynamoDB table deployment-log.
  • CloudWatch Alarm deployment-error-alarm.

Keep everything running

No action needed. All resources are serverless and on the always-free tier. They sit idle at zero cost when not in use.

  • Lambda includes 1,000,000 free requests per month.
  • DynamoDB includes 25 GB of free storage.
  • CloudWatch includes 10 free alarms.

Your resources will remain available for future experiments without any charges.

Pause - I'll come back to this later

All resources in this project are serverless. They only execute when invoked and cost nothing when idle, so there is no running process to shut down.

To remove public access while preserving your function code and published versions, delete the two Function URLs.

  • In the Lambda console, open your canary-deploy-app function.
  • Click the Configuration tab, then Function URL in the left sidebar.
  • Delete the Function URL attached to $LATEST.
  • Click the Aliases tab, then click prod.
  • Click the Configuration tab, then Function URL, and delete the Function URL attached to the prod alias.

Your function code, published versions, alias configuration, DynamoDB table, and CloudWatch Alarm will remain intact for when you return.

Delete - I don't want to use this again

Remove all project resources from your AWS account.

Delete the Lambda function:

  • Search for "Lambda" in the AWS console search bar and open it.
  • Click Functions in the left sidebar.
  • Select the checkbox next to canary-deploy-app.
  • Click Actions, then Delete.
  • Follow the confirmation prompt to complete the deletion. This also removes all published versions, the prod alias, and both Function URLs.

Delete the DynamoDB table:

  • Search for "DynamoDB" in the AWS console search bar and open it.
  • Click Tables in the left sidebar.
  • Select deployment-log.
  • Click Delete.
  • Follow the confirmation prompt to complete the deletion.

Delete the CloudWatch Alarm:

  • Search for "CloudWatch" in the AWS console search bar and open it.
  • In the left sidebar, click Alarms, then All alarms.
  • Select the checkbox next to deployment-error-alarm.
  • Click Actions, then Delete.
  • Confirm the deletion.

That's a Wrap!

That's a Wrap!

Nice work! Your manager's incident report has been answered. You designed, built, broke, and fixed a production-style deployment system on AWS. Last Friday's outage will never happen again.

You now have a live serverless application with canary deployments, automated error detection, and an architecture diagram to match.

You've learned how to:

  • Designed a serverless architecture on a whiteboard using the same approach AWS Solutions Architects use in real engagements, then built it live with Lambda, DynamoDB, and CloudWatch.
  • Experienced blast radius firsthand by recreating the Friday incident, then fixed it with canary deployments using weighted alias routing.
  • Added automated error detection with a CloudWatch Alarm that fires when a canary version starts generating errors, solving the 45-minute detection gap from the original incident.
  • Completed an optional Secret Mission to race your fellow participants through a 10-second production rollback.

Ready to quiz yourself?