Serverless File Pipeline with Terraform

Introduction

30 Second Summary

Every time you upload a photo to social media or submit a document to a website, something behind the scenes reacts to that file automatically. Cloud-powered businesses rely on event-driven pipelines that process, catalog, and notify in milliseconds.

In this project, you will build a serverless file processing pipeline on AWS using Terraform. When a file lands in an S3 bucket, a Lambda function extracts its metadata, stores it in DynamoDB, and sends you an email notification.

What You'll Build

A fully automated serverless pipeline that reacts to file uploads, processes metadata, stores it in a NoSQL database, and exposes a REST API for querying results.

By the end of this project, you'll have:

• A serverless pipeline that automatically processes files uploaded to S3, stores metadata in DynamoDB, and sends email notifications via SNS.
• A public REST API built with API Gateway and Lambda that returns all processed file metadata as JSON.
• A CloudWatch dashboard and alarm that monitors your pipeline's health and alerts you when errors occur.
• Secret Mission: An SQS dead-letter queue that catches failed processing attempts and triggers an automated alarm.

Are there any prerequisites?

This project assumes you already have the AWS CLI and Terraform CLI installed and configured on your machine. You also need basic familiarity with the AWS console.

For a text editor, Notepad or VS Code both work fine.

Set Up the Project Structure

Before you can deploy any AWS resources, Terraform needs a project directory with a specific file structure.

Each file has a defined role: one for infrastructure resources, one for input variables, and one for output values. In this step, you'll create the directory, configure the AWS provider, and initialize Terraform to download the required plugin.

In this step, get ready to:

• Create the project directory and file structure.
• Configure the Terraform AWS provider with version constraints.
• Run terraform init to download the AWS provider plugin.

Create the project directory and files

Terraform reads all .tf files in a directory to build your infrastructure configuration. You need a few standard files plus a lambda/ subdirectory for your Python function code.

• Move to your Desktop by running this command:

cd ~/Desktop

• Create your project folder and move into it by running these commands:

mkdir file-pipeline-terraform
cd file-pipeline-terraform

• Create the lambda/ subdirectory and your Terraform files by running:

mkdir lambda
touch main.tf variables.tf outputs.tf terraform.tfvars.example

• Confirm the files exist by running:

ls

You should see main.tf, variables.tf, outputs.tf, terraform.tfvars.example, and the lambda folder listed in the output.

touch not recognized?

On Windows PowerShell, touch is not a built-in command. Use New-Item main.tf, variables.tf, outputs.tf, terraform.tfvars.example instead, or create the files through your text editor's File > New File option.

Help me create empty files on Windows for my Terraform project

• Open the file-pipeline-terraform folder in your text editor so you can edit the files in the next substep.

Open from the terminal

If you use VS Code, run code . in your terminal to open the current folder directly. If you use Notepad or another editor, use File > Open Folder and navigate to Desktop/file-pipeline-terraform.

Configure the Terraform AWS provider

The provider block tells Terraform which cloud platform to target and which plugin version to download. You also need a variables file to keep your configuration flexible.

• Open main.tf in your text editor and paste this code:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
  required_version = ">= 1.0"
}

provider "aws" {
  region = var.aws_region
}

What does this code do?

• required_providers tells Terraform to download the hashicorp/aws plugin. The ~> 6.0 constraint means any version in the 6.x series is acceptable.
• required_version ensures anyone running this config has Terraform 1.0 or higher installed.
• provider "aws" configures the AWS plugin to deploy resources in the region specified by the aws_region variable (defined next).

• Save main.tf.
• Open variables.tf and paste this code:

variable "aws_region" {
  description = "AWS region for all resources"
  type        = string
  default     = "us-west-2"
}

variable "notification_email" {
  description = "Email address to receive file processing notifications"
  type        = string
}

What does this code do?

• aws_region has a default value of us-west-2, so Terraform uses it automatically unless you override it.
• notification_email has no default, so Terraform will prompt you for it at deploy time. This is the address that receives file-processing notifications later.

• Save variables.tf.
• Open terraform.tfvars.example and paste this reference template:

aws_region         = "us-west-2"
notification_email = "your-email@example.com"

Why a .tfvars.example file?

This file documents the expected variables without containing your real values. When you deploy later, you'll create a terraform.tfvars file (without .example) containing your actual email address.

• Save terraform.tfvars.example.
• Leave outputs.tf empty for now. You'll add output definitions in later steps.

✔️ Awesome, I've got everything!

Great. Double check you've saved all your files before moving on.

ⓧ I'd like to double check the full code

Here is what each file should contain at this point:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
  required_version = ">= 1.0"
}

provider "aws" {
  region = var.aws_region
}

variable "aws_region" {
  description = "AWS region for all resources"
  type        = string
  default     = "us-west-2"
}

variable "notification_email" {
  description = "Email address to receive file processing notifications"
  type        = string
}

aws_region         = "us-west-2"
notification_email = "your-email@example.com"

outputs.tf should be empty.

Initialize Terraform

Running terraform init downloads the AWS provider plugin and prepares your directory for use. This is the first command you run in any new Terraform project.

• Download the AWS provider plugin by running this command:

terraform init

What does this command do?

• Terraform reads the required_providers block in main.tf and downloads the matching AWS plugin from the Terraform registry.
• It creates a .terraform directory to store the plugin and a .terraform.lock.hcl file to lock the exact provider version.

You should see output ending with Terraform has been successfully initialized! followed by a suggestion to run terraform plan.

Seeing an error instead?

• If you see a credentials error, confirm your AWS CLI is configured by running aws sts get-caller-identity in your terminal. It should return your account ID.
• If Terraform reports a version conflict, make sure the version = "~> 6.0" line in main.tf matches exactly.
• If the command is not found, confirm Terraform is on your PATH by running terraform --version.

Help me troubleshoot terraform init errors

What is Infrastructure as Code?

Infrastructure as Code (IaC) means defining your cloud resources in text files instead of clicking through a console. Terraform reads these files and creates, updates, or destroys resources to match what you've described.

The benefit is reproducibility. You can spin up an identical environment on a different account, roll back changes with version control, or tear everything down with a single command.

Your project structure is ready and Terraform can talk to AWS. Next, you'll define the IAM roles that give your Lambda functions the exact permissions they need.

Define IAM Roles and Policies

Your Terraform project is initialized and ready to define resources. Before your Lambda functions can interact with other AWS services, they need permission to do so.

AWS uses IAM roles and policies to control what each service is allowed to do. In this step, you'll define a role that your Lambda functions can assume, and attach policies granting only the specific permissions your pipeline needs.

In this step, get ready to:

• Create an IAM role that Lambda functions can assume.
• Attach a managed policy for CloudWatch Logs access.
• Create a custom policy granting S3, DynamoDB, and SNS permissions.

Create the IAM role for Lambda

An IAM role is like an identity card for an AWS service. Your Lambda functions need a role that says "I am allowed to be invoked as a Lambda function." This is established through an assume role policy that trusts the Lambda service.

You'll also add a random_id resource that generates a unique suffix. This keeps your AWS resource names unique so they don't collide with other accounts.

• In main.tf, find the provider "aws" block at the bottom of the file.
• Add the following code below the provider block:

resource "random_id" "suffix" {
  byte_length = 4
}

# IAM Role for Lambda functions
resource "aws_iam_role" "lambda_role" {
  name = "file-pipeline-lambda-role-${random_id.suffix.hex}"

  # Allow the Lambda service to assume this role
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      }
    ]
  })
}

What does this code do?

• random_id "suffix" generates a random hex string (8 characters) that gets appended to resource names, preventing naming collisions across AWS accounts.
• aws_iam_role "lambda_role" creates the role itself. The assume_role_policy tells AWS that the Lambda service is trusted to use this role.
• sts:AssumeRole is the specific action that lets Lambda "put on" this role when it runs your functions.

• Save main.tf.

You won't see a visible change yet. The next chunk adds the policies that give this role its actual permissions.

Seeing a syntax error in your editor?

• Check that the jsonencode() block has matching curly braces and square brackets. Each opening brace must have a closing brace.
• Make sure you placed this code AFTER the closing brace of the provider "aws" block, not inside it.
• Help me fix a syntax error in my Terraform IAM role code.

Attach managed and custom IAM policies

The role exists, but it has zero permissions. You need to attach policies that spell out exactly what your Lambda functions can do.

You'll attach two policies. The first is an AWS-managed policy that grants CloudWatch Logs access so your functions can write logs. The second is a custom policy granting the exact S3, DynamoDB, and SNS permissions your pipeline needs.

Why least-privilege permissions?

You could attach a broad admin policy and your Lambda would work. But in production, granting only the exact actions a function needs is a security best practice called least-privilege. If a function is compromised, the damage is limited to only the resources you explicitly listed.

• In main.tf, add the following code below the aws_iam_role "lambda_role" resource you just created:

# Attach AWS-managed policy for CloudWatch Logs access
resource "aws_iam_role_policy_attachment" "lambda_basic" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

# Custom policy granting only the permissions this pipeline needs
resource "aws_iam_role_policy" "lambda_custom" {
  name = "file-pipeline-custom-policy"
  role = aws_iam_role.lambda_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = ["s3:GetObject"]
        Resource = "${aws_s3_bucket.uploads.arn}/*"
      },
      {
        Effect = "Allow"
        Action = ["dynamodb:PutItem", "dynamodb:Scan"]
        Resource = aws_dynamodb_table.file_metadata.arn
      },
      {
        Effect = "Allow"
        Action = ["sns:Publish"]
        Resource = aws_sns_topic.file_notifications.arn
      }
    ]
  })
}

What does this code do?

• aws_iam_role_policy_attachment "lambda_basic" attaches the AWS-managed AWSLambdaBasicExecutionRole policy. This gives your functions permission to write logs to CloudWatch.
• aws_iam_role_policy "lambda_custom" defines a custom inline policy with three permission statements, one for each service your pipeline touches.
• s3:GetObject lets the function read uploaded files. dynamodb:PutItem and dynamodb:Scan let it write and query metadata. sns:Publish lets it send notifications.
• The Resource fields reference other Terraform resources (S3 bucket, DynamoDB table, SNS topic) that you'll define in the next steps.

• Save main.tf.

You won't be able to validate this code yet. The custom policy references S3, DynamoDB, and SNS resources that you'll create in the next steps. You'll validate everything together when you run terraform plan in Step 8.

Editor showing errors about unknown resources?

• This is expected. References like aws_s3_bucket.uploads.arn point to resources you haven't defined yet. Your editor or linter may flag them, but they'll resolve once you add those resources in the next steps.
• If you see errors about mismatched braces or brackets instead, check that every { has a matching } and every [ has a matching ].
• Help me debug bracket matching in my Terraform IAM policy code.

✔️ Awesome, I've got everything!

Great. Double check you've saved main.tf before moving on.

ⓧ I'd like to double check the full code

Here is the complete main.tf file as it should look at the end of this step:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
  required_version = ">= 1.0"
}

provider "aws" {
  region = var.aws_region
}

resource "random_id" "suffix" {
  byte_length = 4
}

resource "aws_iam_role" "lambda_role" {
  name = "file-pipeline-lambda-role-${random_id.suffix.hex}"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "lambda_basic" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_iam_role_policy" "lambda_custom" {
  name = "file-pipeline-custom-policy"
  role = aws_iam_role.lambda_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = ["s3:GetObject"]
        Resource = "${aws_s3_bucket.uploads.arn}/*"
      },
      {
        Effect = "Allow"
        Action = ["dynamodb:PutItem", "dynamodb:Scan"]
        Resource = aws_dynamodb_table.file_metadata.arn
      },
      {
        Effect = "Allow"
        Action = ["sns:Publish"]
        Resource = aws_sns_topic.file_notifications.arn
      }
    ]
  })
}

Your Lambda functions now have a defined identity and a precise set of permissions. Next up, you'll create the S3 bucket and DynamoDB table that these permissions point to.

Create the S3 Bucket and DynamoDB Table

Your IAM role defines what the Lambda functions are allowed to do, but the pipeline's core resources don't exist yet.

In this step, you'll create the S3 bucket where files get uploaded, a DynamoDB table that stores file metadata, an SNS topic that sends email notifications, and the event trigger that connects S3 to your Lambda function.

In this step, get ready to:

• Create an S3 bucket and DynamoDB table for your pipeline's data layer.
• Set up SNS email notifications for processed files.
• Configure the S3 event trigger that invokes your Lambda function on new uploads.

Define the S3 bucket and DynamoDB table

Every event-driven pipeline starts with a trigger source and a data store. The S3 bucket is where files land, and the DynamoDB table is where your Lambda function will write metadata about each processed file.

• In main.tf, find the random_id resource block. Add the following code directly below it, before the aws_iam_role block:

# S3 Bucket for file uploads
resource "aws_s3_bucket" "uploads" {
  bucket = "file-pipeline-uploads-${random_id.suffix.hex}"
}

# DynamoDB Table for file metadata
resource "aws_dynamodb_table" "file_metadata" {
  name         = "file-metadata-${random_id.suffix.hex}"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "file_id"
  range_key    = "uploaded_at"

  attribute {
    name = "file_id"
    type = "S"
  }

  attribute {
    name = "uploaded_at"
    type = "S"
  }
}

What does this code do?

• aws_s3_bucket.uploads creates a bucket with a unique name using the random suffix you defined earlier. This prevents naming conflicts across AWS accounts.
• aws_dynamodb_table.file_metadata creates a NoSQL table with on-demand billing. You only pay for the reads and writes you actually use.
• hash_key (partition key) and range_key (sort key) together form a composite primary key. Each file gets a unique ID, and uploaded_at lets you query files in chronological order.
• PAY_PER_REQUEST means AWS automatically scales read and write capacity. This is ideal for unpredictable workloads like file uploads.

• Save main.tf.

You won't run terraform plan yet because the IAM policy still references resources that don't exist (like the SNS topic). You'll add those next.

Seeing a syntax error in your editor?

• Make sure each attribute block is inside the aws_dynamodb_table resource block, not outside it.
• Check that you placed this code between the random_id resource and the aws_iam_role resource. The order matters because the IAM policy references these resources.

Help me fix a syntax error in my DynamoDB Terraform resource.

Set up SNS email notifications

When a file is processed, you want to know about it. SNS (Simple Notification Service) sends an email notification every time your Lambda function successfully processes a file.

• Still in main.tf, add the following code below the DynamoDB table resource you just created, still above the aws_iam_role block:

# SNS Topic for notifications
resource "aws_sns_topic" "file_notifications" {
  name = "file-processing-notifications-${random_id.suffix.hex}"
}

resource "aws_sns_topic_subscription" "email" {
  topic_arn = aws_sns_topic.file_notifications.arn
  protocol  = "email"
  endpoint  = var.notification_email
}

What does this code do?

• aws_sns_topic.file_notifications creates a messaging channel that your Lambda function will publish to whenever it finishes processing a file.
• aws_sns_topic_subscription.email subscribes your email address to that topic using the email protocol. The email address comes from the notification_email variable defined in variables.tf.

When you deploy later, AWS will send a confirmation email to the address you provide. You'll need to click the confirmation link before notifications start arriving.

• Save main.tf.

You won't see a change yet. The SNS topic is defined but won't be active until you deploy with terraform apply in a later step.

Not sure what email to use?

The notification_email variable has no default value. You'll provide your email address in a terraform.tfvars file when you deploy. For now, just defining the resource is enough.

Help me understand how Terraform variables work with SNS email subscriptions.

Add the S3 bucket notification configuration

The S3 bucket exists, but it doesn't know to trigger anything when a file arrives. The bucket notification resource wires the s3:ObjectCreated:* event to your processor Lambda function.

• At the bottom of main.tf, below the aws_iam_role_policy resource, add:

resource "aws_s3_bucket_notification" "upload_trigger" {
  bucket = aws_s3_bucket.uploads.id

  lambda_function {
    lambda_function_arn = aws_lambda_function.processor.arn
    events             = ["s3:ObjectCreated:*"]
  }

  depends_on = [aws_lambda_permission.s3_invoke]
}

What does this code do?

• This resource configures the S3 bucket to fire an event whenever any object is created (s3:ObjectCreated:*). That event invokes the processor Lambda function.
• depends_on tells Terraform to wait until the Lambda permission (aws_lambda_permission.s3_invoke) exists before creating the notification. Without this, AWS would reject the notification because S3 doesn't have permission to call Lambda yet.
• Both aws_lambda_function.processor and aws_lambda_permission.s3_invoke don't exist yet. You'll create them in the next step. Terraform allows forward references in configuration files.

• Save main.tf.

Running terraform plan will still fail at this point because the Lambda function and permission resources haven't been defined yet. That's expected. You'll resolve this in the next step.

Worried about the forward references?

Terraform resolves all references at plan time, not when it reads the file. As long as every referenced resource exists somewhere in your .tf files before you run terraform plan, the configuration is valid. You'll add the missing Lambda resources in the next step.

Help me understand how Terraform handles forward references between resources.

✔️ Awesome, I've got everything!

Great. Double check you've saved main.tf with all three sections of new code in place.

ⓧ I'd like to double check the full code

Here's the complete main.tf as it should look at the end of this step:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
  required_version = ">= 1.0"
}

provider "aws" {
  region = var.aws_region
}

resource "random_id" "suffix" {
  byte_length = 4
}

# S3 Bucket for file uploads
resource "aws_s3_bucket" "uploads" {
  bucket = "file-pipeline-uploads-${random_id.suffix.hex}"
}

# DynamoDB Table for file metadata
resource "aws_dynamodb_table" "file_metadata" {
  name         = "file-metadata-${random_id.suffix.hex}"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "file_id"
  range_key    = "uploaded_at"

  attribute {
    name = "file_id"
    type = "S"
  }

  attribute {
    name = "uploaded_at"
    type = "S"
  }
}

# SNS Topic for notifications
resource "aws_sns_topic" "file_notifications" {
  name = "file-processing-notifications-${random_id.suffix.hex}"
}

resource "aws_sns_topic_subscription" "email" {
  topic_arn = aws_sns_topic.file_notifications.arn
  protocol  = "email"
  endpoint  = var.notification_email
}

resource "aws_iam_role" "lambda_role" {
  name = "file-pipeline-lambda-role-${random_id.suffix.hex}"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "lambda_basic" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_iam_role_policy" "lambda_custom" {
  name = "file-pipeline-custom-policy"
  role = aws_iam_role.lambda_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = ["s3:GetObject"]
        Resource = "${aws_s3_bucket.uploads.arn}/*"
      },
      {
        Effect = "Allow"
        Action = ["dynamodb:PutItem", "dynamodb:Scan"]
        Resource = aws_dynamodb_table.file_metadata.arn
      },
      {
        Effect = "Allow"
        Action = ["sns:Publish"]
        Resource = aws_sns_topic.file_notifications.arn
      }
    ]
  })
}

resource "aws_s3_bucket_notification" "upload_trigger" {
  bucket = aws_s3_bucket.uploads.id

  lambda_function {
    lambda_function_arn = aws_lambda_function.processor.arn
    events             = ["s3:ObjectCreated:*"]
  }

  depends_on = [aws_lambda_permission.s3_invoke]
}

Your pipeline's data layer and messaging infrastructure are in place. Next up, you'll write the Python Lambda function that processes uploaded files and wires everything together.

Write and Deploy the Processor Lambda

Your S3 bucket, DynamoDB table, SNS topic, and IAM permissions are all defined. Now you need the actual code that runs when a file hits the bucket.

In this step, you will write the Python function that processes uploaded files and then wire it into Terraform so the entire deployment is automated. The function extracts metadata from each S3 upload event, stores it in DynamoDB, and publishes a notification to SNS.

In this step, get ready to:

• Write the Python Lambda handler that processes S3 events.
• Define the Terraform resources that package and deploy the function.
• Grant S3 permission to invoke the Lambda function.

Write the processor Lambda function

AWS Lambda functions are small pieces of code that run on demand without managing servers. Your processor function will be triggered every time a file lands in the S3 bucket.

• Create the file lambda/processor.py inside your project's lambda/ subdirectory.
• Add the imports and module-level setup by pasting this code into the file:

import json
import os
import uuid
from datetime import datetime, timezone

import boto3

# Initialize AWS service clients
dynamodb = boto3.resource("dynamodb")
sns = boto3.client("sns")

# Read environment variables set by Terraform
TABLE_NAME = os.environ["DYNAMODB_TABLE"]
SNS_TOPIC_ARN = os.environ["SNS_TOPIC_ARN"]

What does this code do?

The boto3 library is included in the Lambda Python runtime, so you do not need to install it separately.

• dynamodb = boto3.resource("dynamodb") creates a high-level DynamoDB client for writing items to your table.
• sns = boto3.client("sns") creates a low-level SNS client for publishing notifications.
• os.environ["DYNAMODB_TABLE"] reads the table name injected by Terraform at deploy time. This keeps your code decoupled from specific resource names.

• Add the handler function below the module-level setup you just pasted:

def handler(event, context):
    table = dynamodb.Table(TABLE_NAME)

    for record in event["Records"]:
        bucket = record["s3"]["bucket"]["name"]
        key = record["s3"]["object"]["key"]
        size = record["s3"]["object"]["size"]
        event_time = record["eventTime"]

        file_id = str(uuid.uuid4())
        uploaded_at = datetime.now(timezone.utc).isoformat()

        # Store metadata in DynamoDB
        table.put_item(
            Item={
                "file_id": file_id,
                "uploaded_at": uploaded_at,
                "bucket": bucket,
                "file_name": key,
                "file_size": size,
                "event_time": event_time,
            }
        )

        # Send SNS notification
        sns.publish(
            TopicArn=SNS_TOPIC_ARN,
            Subject="File Processed",
            Message=f"File '{key}' ({size} bytes) was uploaded to '{bucket}' and processed successfully.",
        )

    return {"statusCode": 200, "body": json.dumps("Processing complete")}

What does the handler do?

When S3 triggers this function, it passes an event containing one or more records. Each record represents a single file upload.

• The function extracts the bucket name, file key (path), file size, and upload timestamp from the S3 event record.
• It generates a unique file_id using UUID and captures the current UTC time as uploaded_at. These become the partition key and sort key in DynamoDB.
• After writing the metadata, it publishes a human-readable message to SNS so you receive an email notification.

• Save lambda/processor.py.
• Confirm the file exists by running:

ls lambda/

• You should see processor.py listed in the output.

Don't see processor.py?

• Make sure you created the file inside the lambda/ subdirectory, not in the project root.
• Check that you saved the file after pasting the code.

Help me find my processor.py file

✔️ Awesome, I've got everything!

Great. Make sure you have saved lambda/processor.py before moving on.

ⓧ I'd like to double check the full code

import json
import os
import uuid
from datetime import datetime, timezone

import boto3

dynamodb = boto3.resource("dynamodb")
sns = boto3.client("sns")

TABLE_NAME = os.environ["DYNAMODB_TABLE"]
SNS_TOPIC_ARN = os.environ["SNS_TOPIC_ARN"]


def handler(event, context):
    table = dynamodb.Table(TABLE_NAME)

    for record in event["Records"]:
        bucket = record["s3"]["bucket"]["name"]
        key = record["s3"]["object"]["key"]
        size = record["s3"]["object"]["size"]
        event_time = record["eventTime"]

        file_id = str(uuid.uuid4())
        uploaded_at = datetime.now(timezone.utc).isoformat()

        # Store metadata in DynamoDB
        table.put_item(
            Item={
                "file_id": file_id,
                "uploaded_at": uploaded_at,
                "bucket": bucket,
                "file_name": key,
                "file_size": size,
                "event_time": event_time,
            }
        )

        # Send SNS notification
        sns.publish(
            TopicArn=SNS_TOPIC_ARN,
            Subject="File Processed",
            Message=f"File '{key}' ({size} bytes) was uploaded to '{bucket}' and processed successfully.",
        )

    return {"statusCode": 200, "body": json.dumps("Processing complete")}

Define the Lambda resources in Terraform

Terraform needs to package your Python file into a zip archive before it can deploy it to AWS. You will use a data source to create the zip and an aws_lambda_function resource to deploy it.

• Open main.tf and find the aws_iam_role_policy block that ends with three closing braces. Add the following code below it (before the existing aws_s3_bucket_notification block):

# Lambda: Processor function
data "archive_file" "processor" {
  type        = "zip"
  source_file = "${path.module}/lambda/processor.py"
  output_path = "${path.module}/lambda/processor.zip"
}

resource "aws_lambda_function" "processor" {
  filename         = data.archive_file.processor.output_path
  function_name    = "file-processor-${random_id.suffix.hex}"
  role             = aws_iam_role.lambda_role.arn
  handler          = "processor.handler"
  source_code_hash = data.archive_file.processor.output_base64sha256
  runtime          = "python3.12"
  timeout          = 30

  environment {
    variables = {
      DYNAMODB_TABLE = aws_dynamodb_table.file_metadata.name
      SNS_TOPIC_ARN  = aws_sns_topic.file_notifications.arn
    }
  }
}

What do these resources do?

• data "archive_file" "processor" zips your Python file into lambda/processor.zip every time you run terraform plan or terraform apply.
• source_code_hash tells Terraform to redeploy the function whenever the code changes.
• handler = "processor.handler" tells Lambda to call the handler function inside processor.py.
• The environment block injects the DynamoDB table name and SNS topic ARN as environment variables that your Python code reads at runtime.

• Save main.tf.

You will not see a change yet. The next substep adds the permission resource that completes the Lambda configuration.

Grant S3 permission to invoke the function

AWS uses resource-based policies to control which services can invoke a Lambda function. Without an explicit permission, S3 cannot trigger your processor even though the notification is configured.

• In main.tf, add the following resource directly below the aws_lambda_function block you just created (and before the existing aws_s3_bucket_notification block):

resource "aws_lambda_permission" "s3_invoke" {
  statement_id  = "AllowS3Invoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.processor.function_name
  principal     = "s3.amazonaws.com"
  source_arn    = aws_s3_bucket.uploads.arn
}

What does this permission do?

This resource tells AWS that s3.amazonaws.com is allowed to call lambda:InvokeFunction on your processor. The source_arn restricts the permission to only your specific S3 bucket so no other bucket can trigger this function.

The aws_s3_bucket_notification you defined earlier uses depends_on to wait for this permission before it creates the notification. This prevents a race condition where S3 tries to invoke the function before permission exists.

• Save main.tf.
• Verify your Terraform configuration is syntactically valid by running:

terraform validate

• You should see a message confirming the configuration is valid.

Seeing a validation error?

• Check that the data "archive_file" "processor" block appears before the aws_lambda_function resource that references it.
• Make sure your aws_lambda_permission block is placed before the aws_s3_bucket_notification block.
• Verify that lambda/processor.py exists on disk. The archive_file data source reads it at plan time.

Help me fix my Terraform validation error

✔️ Awesome, I've got everything!

Great. Double check you have saved main.tf before moving on.

ⓧ I'd like to double check the full code

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
  required_version = ">= 1.0"
}

provider "aws" {
  region = var.aws_region
}

resource "random_id" "suffix" {
  byte_length = 4
}

# S3 Bucket for file uploads
resource "aws_s3_bucket" "uploads" {
  bucket = "file-pipeline-uploads-${random_id.suffix.hex}"
}

# DynamoDB Table for file metadata
resource "aws_dynamodb_table" "file_metadata" {
  name         = "file-metadata-${random_id.suffix.hex}"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "file_id"
  range_key    = "uploaded_at"

  attribute {
    name = "file_id"
    type = "S"
  }

  attribute {
    name = "uploaded_at"
    type = "S"
  }
}

# SNS Topic for notifications
resource "aws_sns_topic" "file_notifications" {
  name = "file-processing-notifications-${random_id.suffix.hex}"
}

resource "aws_sns_topic_subscription" "email" {
  topic_arn = aws_sns_topic.file_notifications.arn
  protocol  = "email"
  endpoint  = var.notification_email
}

resource "aws_iam_role" "lambda_role" {
  name = "file-pipeline-lambda-role-${random_id.suffix.hex}"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "lambda_basic" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_iam_role_policy" "lambda_custom" {
  name = "file-pipeline-custom-policy"
  role = aws_iam_role.lambda_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = ["s3:GetObject"]
        Resource = "${aws_s3_bucket.uploads.arn}/*"
      },
      {
        Effect = "Allow"
        Action = ["dynamodb:PutItem", "dynamodb:Scan"]
        Resource = aws_dynamodb_table.file_metadata.arn
      },
      {
        Effect = "Allow"
        Action = ["sns:Publish"]
        Resource = aws_sns_topic.file_notifications.arn
      }
    ]
  })
}

# Lambda: Processor function
data "archive_file" "processor" {
  type        = "zip"
  source_file = "${path.module}/lambda/processor.py"
  output_path = "${path.module}/lambda/processor.zip"
}

resource "aws_lambda_function" "processor" {
  filename         = data.archive_file.processor.output_path
  function_name    = "file-processor-${random_id.suffix.hex}"
  role             = aws_iam_role.lambda_role.arn
  handler          = "processor.handler"
  source_code_hash = data.archive_file.processor.output_base64sha256
  runtime          = "python3.12"
  timeout          = 30

  environment {
    variables = {
      DYNAMODB_TABLE = aws_dynamodb_table.file_metadata.name
      SNS_TOPIC_ARN  = aws_sns_topic.file_notifications.arn
    }
  }
}

resource "aws_lambda_permission" "s3_invoke" {
  statement_id  = "AllowS3Invoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.processor.function_name
  principal     = "s3.amazonaws.com"
  source_arn    = aws_s3_bucket.uploads.arn
}

resource "aws_s3_bucket_notification" "upload_trigger" {
  bucket = aws_s3_bucket.uploads.id

  lambda_function {
    lambda_function_arn = aws_lambda_function.processor.arn
    events             = ["s3:ObjectCreated:*"]
  }

  depends_on = [aws_lambda_permission.s3_invoke]
}

Your processor Lambda is fully defined. Next up, you will set up SNS email notifications so you actually receive an alert every time a file is processed.

Build the Query API with API Gateway

Your pipeline can process files and send notifications, but there's no way to retrieve the data it stores.

In this step, you'll build a public REST API that lets anyone query all processed file metadata from DynamoDB. You'll use API Gateway (the HTTP API flavour) to expose a GET /files endpoint backed by a second Lambda function.

In this step, get ready to:

• Write a query Lambda function that scans DynamoDB and returns JSON.
• Create an HTTP API with a Lambda integration and a GET route.
• Grant API Gateway permission to invoke the function and define Terraform outputs.

Write the query Lambda function

The processor Lambda writes metadata into DynamoDB. Now you need a separate function whose only job is reading that data back out and returning it as a JSON response.

• Create a new file called query.py inside your lambda/ directory.
• Paste the following code into lambda/query.py:

import json
import os
from decimal import Decimal

import boto3

dynamodb = boto3.resource("dynamodb")
TABLE_NAME = os.environ["DYNAMODB_TABLE"]


class DecimalEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, Decimal):
            return int(obj) if obj % 1 == 0 else float(obj)
        return super().default(obj)


def handler(event, context):
    table = dynamodb.Table(TABLE_NAME)

    response = table.scan()
    items = response.get("Items", [])

    return {
        "statusCode": 200,
        "headers": {
            "Content-Type": "application/json",
            "Access-Control-Allow-Origin": "*",
        },
        "body": json.dumps({"files": items, "count": len(items)}, cls=DecimalEncoder),
    }

What does this code do?

DynamoDB stores numbers as Decimal objects, but Python's built-in json.dumps can't serialise them. The DecimalEncoder class converts each Decimal to a plain int or float before serialisation.

The handler function scans the entire DynamoDB table, wraps the results in a JSON body with a files array and a count, and returns it with CORS headers so browsers can call the endpoint directly.

• Save lambda/query.py.
• Confirm the file exists by running:

ls lambda/

You should see both processor.py and query.py listed in the output.

Only seeing processor.py?

Make sure you created the file inside the lambda/ subdirectory, not in the project root.

Check the filename is exactly query.py (no capital letters, no extra extension).

Help me figure out why my query.py file isn't showing up in the lambda directory.

Create the HTTP API with a Lambda integration

With the Python code ready, you need Terraform resources to package the function, deploy it to Lambda, and wire up an HTTP API in front of it.

• Open main.tf and scroll to the bottom, below the aws_s3_bucket_notification resource.
• Add the query Lambda function resources by pasting this code:

# Lambda: Query function
data "archive_file" "query" {
  type        = "zip"
  source_file = "${path.module}/lambda/query.py"
  output_path = "${path.module}/lambda/query.zip"
}

resource "aws_lambda_function" "query" {
  filename         = data.archive_file.query.output_path
  function_name    = "file-query-${random_id.suffix.hex}"
  role             = aws_iam_role.lambda_role.arn
  handler          = "query.handler"
  source_code_hash = data.archive_file.query.output_base64sha256
  runtime          = "python3.12"
  timeout          = 10

  environment {
    variables = {
      DYNAMODB_TABLE = aws_dynamodb_table.file_metadata.name
    }
  }
}

What does this code do?

• archive_file zips query.py into a deployment package that Lambda can accept.
• The aws_lambda_function resource deploys the zip with the python3.12 runtime and passes the DynamoDB table name as an environment variable.
• source_code_hash tells Terraform to redeploy the function whenever the zip contents change.

• Directly below the query Lambda resource, add the API Gateway resources:

# API Gateway HTTP API
resource "aws_apigatewayv2_api" "files_api" {
  name          = "file-metadata-api-${random_id.suffix.hex}"
  protocol_type = "HTTP"
}

resource "aws_apigatewayv2_stage" "default" {
  api_id      = aws_apigatewayv2_api.files_api.id
  name        = "$default"
  auto_deploy = true

  access_log_settings {
    destination_arn = aws_cloudwatch_log_group.api_gw.arn
    format = jsonencode({
      requestId      = "$context.requestId"
      sourceIp       = "$context.identity.sourceIp"
      requestTime    = "$context.requestTime"
      httpMethod     = "$context.httpMethod"
      routeKey       = "$context.routeKey"
      status         = "$context.status"
      responseLength = "$context.responseLength"
    })
  }
}

resource "aws_apigatewayv2_integration" "query_lambda" {
  api_id             = aws_apigatewayv2_api.files_api.id
  integration_uri    = aws_lambda_function.query.invoke_arn
  integration_type   = "AWS_PROXY"
  integration_method = "POST"
}

resource "aws_apigatewayv2_route" "get_files" {
  api_id    = aws_apigatewayv2_api.files_api.id
  route_key = "GET /files"
  target    = "integrations/${aws_apigatewayv2_integration.query_lambda.id}"
}

What's happening here?

• aws_apigatewayv2_api creates an HTTP API (lighter and cheaper than the older REST API type).
• The $default stage with auto_deploy = true means every route change is live immediately without a manual deployment step.
• The integration uses AWS_PROXY so the full HTTP request passes through to Lambda, and Lambda's response is sent directly back to the caller.
• The route GET /files maps incoming GET requests at the /files path to the query Lambda integration.

Why does the stage reference a log group that doesn't exist yet?

The access_log_settings block points to aws_cloudwatch_log_group.api_gw, which you'll define in the next step. Terraform resolves all references at plan time, so you'll add that resource before running terraform apply.

• Save main.tf.
• You won't see a change yet. You'll deploy everything together in Step 8 after the CloudWatch resources are in place.

Seeing red squiggles or syntax errors?

Make sure each resource block has matching opening and closing braces. A missing } is the most common cause of cascading errors in HCL.

Confirm that the query Lambda block appears after the S3 bucket notification resource and before any new section you paste next.

Help me find the syntax issue in my main.tf API Gateway resources.

Grant API Gateway permissions and define outputs

API Gateway needs explicit permission to invoke your Lambda function. Without it, every request would return a 500 error. You'll also define Terraform outputs so you can easily grab resource names and URLs after deployment.

• In main.tf, add the Lambda permission resource directly below the aws_apigatewayv2_route block:

resource "aws_lambda_permission" "api_gw_invoke" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.query.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "${aws_apigatewayv2_api.files_api.execution_arn}/*/*"
}

What does this permission do?

This grants the apigateway.amazonaws.com service principal permission to call lambda:InvokeFunction on your query function. The /*/* wildcard at the end covers any method and any route on the API.

• Save main.tf.
• Now open outputs.tf and replace its contents with the following output definitions:

output "s3_bucket_name" {
  description = "Name of the S3 upload bucket"
  value       = aws_s3_bucket.uploads.id
}

output "api_endpoint" {
  description = "URL to query processed file metadata"
  value       = "${aws_apigatewayv2_api.files_api.api_endpoint}/files"
}

output "dynamodb_table_name" {
  description = "Name of the DynamoDB metadata table"
  value       = aws_dynamodb_table.file_metadata.name
}

output "sns_topic_arn" {
  description = "ARN of the SNS notification topic"
  value       = aws_sns_topic.file_notifications.arn
}

output "dashboard_url" {
  description = "URL to the CloudWatch dashboard"
  value       = "https://${var.aws_region}.console.aws.amazon.com/cloudwatch/home?region=${var.aws_region}#dashboards:name=${aws_cloudwatch_dashboard.pipeline.dashboard_name}"
}

What do these outputs do?

Terraform outputs print useful values after terraform apply completes. Instead of digging through the AWS console to find your bucket name or API URL, Terraform hands them to you in the terminal.

The api_endpoint output constructs the full URL you'll call to query file metadata. The dashboard_url output references a CloudWatch dashboard you'll create in the next step.

• Save outputs.tf.
• You won't see any output yet. These values will appear after you run terraform apply in Step 8.

Editor highlighting errors in outputs.tf?

The dashboard_url output references aws_cloudwatch_dashboard.pipeline which doesn't exist yet. This is expected. You'll add that resource in the next step.

If you see other errors, check that every string interpolation uses the ${...} syntax and that quotes are not accidentally escaped.

Help me fix errors in my outputs.tf file.

✔️ Awesome, I've got everything!

Great. Double check you've saved both main.tf and outputs.tf before moving on.

ⓧ I'd like to double check the full code

Here is the complete lambda/query.py:

import json
import os
from decimal import Decimal

import boto3

dynamodb = boto3.resource("dynamodb")
TABLE_NAME = os.environ["DYNAMODB_TABLE"]


class DecimalEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, Decimal):
            return int(obj) if obj % 1 == 0 else float(obj)
        return super().default(obj)


def handler(event, context):
    table = dynamodb.Table(TABLE_NAME)

    response = table.scan()
    items = response.get("Items", [])

    return {
        "statusCode": 200,
        "headers": {
            "Content-Type": "application/json",
            "Access-Control-Allow-Origin": "*",
        },
        "body": json.dumps({"files": items, "count": len(items)}, cls=DecimalEncoder),
    }

Here is the complete main.tf:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
  required_version = ">= 1.0"
}

provider "aws" {
  region = var.aws_region
}

resource "random_id" "suffix" {
  byte_length = 4
}

# S3 Bucket for file uploads
resource "aws_s3_bucket" "uploads" {
  bucket = "file-pipeline-uploads-${random_id.suffix.hex}"
}

# DynamoDB Table for file metadata
resource "aws_dynamodb_table" "file_metadata" {
  name         = "file-metadata-${random_id.suffix.hex}"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "file_id"
  range_key    = "uploaded_at"

  attribute {
    name = "file_id"
    type = "S"
  }

  attribute {
    name = "uploaded_at"
    type = "S"
  }
}

# SNS Topic for notifications
resource "aws_sns_topic" "file_notifications" {
  name = "file-processing-notifications-${random_id.suffix.hex}"
}

resource "aws_sns_topic_subscription" "email" {
  topic_arn = aws_sns_topic.file_notifications.arn
  protocol  = "email"
  endpoint  = var.notification_email
}

resource "aws_iam_role" "lambda_role" {
  name = "file-pipeline-lambda-role-${random_id.suffix.hex}"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "lambda_basic" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_iam_role_policy" "lambda_custom" {
  name = "file-pipeline-custom-policy"
  role = aws_iam_role.lambda_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = ["s3:GetObject"]
        Resource = "${aws_s3_bucket.uploads.arn}/*"
      },
      {
        Effect = "Allow"
        Action = ["dynamodb:PutItem", "dynamodb:Scan"]
        Resource = aws_dynamodb_table.file_metadata.arn
      },
      {
        Effect = "Allow"
        Action = ["sns:Publish"]
        Resource = aws_sns_topic.file_notifications.arn
      }
    ]
  })
}

# Lambda: Processor function
data "archive_file" "processor" {
  type        = "zip"
  source_file = "${path.module}/lambda/processor.py"
  output_path = "${path.module}/lambda/processor.zip"
}

resource "aws_lambda_function" "processor" {
  filename         = data.archive_file.processor.output_path
  function_name    = "file-processor-${random_id.suffix.hex}"
  role             = aws_iam_role.lambda_role.arn
  handler          = "processor.handler"
  source_code_hash = data.archive_file.processor.output_base64sha256
  runtime          = "python3.12"
  timeout          = 30

  environment {
    variables = {
      DYNAMODB_TABLE = aws_dynamodb_table.file_metadata.name
      SNS_TOPIC_ARN  = aws_sns_topic.file_notifications.arn
    }
  }
}

resource "aws_lambda_permission" "s3_invoke" {
  statement_id  = "AllowS3Invoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.processor.function_name
  principal     = "s3.amazonaws.com"
  source_arn    = aws_s3_bucket.uploads.arn
}

resource "aws_s3_bucket_notification" "upload_trigger" {
  bucket = aws_s3_bucket.uploads.id

  lambda_function {
    lambda_function_arn = aws_lambda_function.processor.arn
    events             = ["s3:ObjectCreated:*"]
  }

  depends_on = [aws_lambda_permission.s3_invoke]
}

# Lambda: Query function
data "archive_file" "query" {
  type        = "zip"
  source_file = "${path.module}/lambda/query.py"
  output_path = "${path.module}/lambda/query.zip"
}

resource "aws_lambda_function" "query" {
  filename         = data.archive_file.query.output_path
  function_name    = "file-query-${random_id.suffix.hex}"
  role             = aws_iam_role.lambda_role.arn
  handler          = "query.handler"
  source_code_hash = data.archive_file.query.output_base64sha256
  runtime          = "python3.12"
  timeout          = 10

  environment {
    variables = {
      DYNAMODB_TABLE = aws_dynamodb_table.file_metadata.name
    }
  }
}

# API Gateway HTTP API
resource "aws_apigatewayv2_api" "files_api" {
  name          = "file-metadata-api-${random_id.suffix.hex}"
  protocol_type = "HTTP"
}

resource "aws_apigatewayv2_stage" "default" {
  api_id      = aws_apigatewayv2_api.files_api.id
  name        = "$default"
  auto_deploy = true

  access_log_settings {
    destination_arn = aws_cloudwatch_log_group.api_gw.arn
    format = jsonencode({
      requestId      = "$context.requestId"
      sourceIp       = "$context.identity.sourceIp"
      requestTime    = "$context.requestTime"
      httpMethod     = "$context.httpMethod"
      routeKey       = "$context.routeKey"
      status         = "$context.status"
      responseLength = "$context.responseLength"
    })
  }
}

resource "aws_apigatewayv2_integration" "query_lambda" {
  api_id             = aws_apigatewayv2_api.files_api.id
  integration_uri    = aws_lambda_function.query.invoke_arn
  integration_type   = "AWS_PROXY"
  integration_method = "POST"
}

resource "aws_apigatewayv2_route" "get_files" {
  api_id    = aws_apigatewayv2_api.files_api.id
  route_key = "GET /files"
  target    = "integrations/${aws_apigatewayv2_integration.query_lambda.id}"
}

resource "aws_lambda_permission" "api_gw_invoke" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.query.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "${aws_apigatewayv2_api.files_api.execution_arn}/*/*"
}

Here is the complete outputs.tf:

output "s3_bucket_name" {
  description = "Name of the S3 upload bucket"
  value       = aws_s3_bucket.uploads.id
}

output "api_endpoint" {
  description = "URL to query processed file metadata"
  value       = "${aws_apigatewayv2_api.files_api.api_endpoint}/files"
}

output "dynamodb_table_name" {
  description = "Name of the DynamoDB metadata table"
  value       = aws_dynamodb_table.file_metadata.name
}

output "sns_topic_arn" {
  description = "ARN of the SNS notification topic"
  value       = aws_sns_topic.file_notifications.arn
}

output "dashboard_url" {
  description = "URL to the CloudWatch dashboard"
  value       = "https://${var.aws_region}.console.aws.amazon.com/cloudwatch/home?region=${var.aws_region}#dashboards:name=${aws_cloudwatch_dashboard.pipeline.dashboard_name}"
}

Your query API is wired up and ready to serve file metadata. Next up, you'll add CloudWatch monitoring to keep an eye on the health of your entire pipeline.

Add CloudWatch Monitoring

Your pipeline now processes files and serves metadata through an API. But how do you know if something breaks at 2 AM?

In this step, you will add CloudWatch monitoring to your pipeline. Log groups capture function output, a metric alarm alerts you when errors occur, and a dashboard gives you a visual overview of the entire system's health.

In this step, get ready to:

• Create CloudWatch log groups for each Lambda function and API Gateway.
• Set up a metric alarm that notifies you when Lambda errors occur.
• Build a CloudWatch dashboard visualizing Lambda and DynamoDB metrics.

Create CloudWatch log groups

CloudWatch log groups are containers that store log output from your Lambda functions and API Gateway. Without explicit log groups, AWS creates them automatically with infinite retention, which can quietly accumulate storage costs.

• In main.tf, scroll to the bottom of the file (after the aws_lambda_permission.api_gw_invoke resource) and add the following log group resources:

# CloudWatch Log Groups
resource "aws_cloudwatch_log_group" "processor" {
  name              = "/aws/lambda/${aws_lambda_function.processor.function_name}"
  retention_in_days = 14
}

resource "aws_cloudwatch_log_group" "query" {
  name              = "/aws/lambda/${aws_lambda_function.query.function_name}"
  retention_in_days = 14
}

resource "aws_cloudwatch_log_group" "api_gw" {
  name              = "/aws/api-gw/${aws_apigatewayv2_api.files_api.name}"
  retention_in_days = 14
}

What does this code do?

Each log group is named to match the AWS service it captures logs from. The /aws/lambda/ prefix is the standard naming convention that Lambda uses to route its output.

Setting retention_in_days = 14 means logs older than 14 days are automatically deleted. This keeps storage costs predictable for a learning project.

The api_gw log group is referenced by the aws_apigatewayv2_stage.default resource you added in the previous step. Adding it here resolves that dependency.

• Save main.tf.

Seeing a reference error for api_gw?

The API Gateway stage resource references aws_cloudwatch_log_group.api_gw.arn in its access_log_settings block. Make sure your new log group resource name is exactly "api_gw" to match that reference.

Help me fix a CloudWatch log group reference error in my Terraform config.

Create a metric alarm for Lambda errors

A metric alarm watches a specific CloudWatch metric and triggers an action when a threshold is crossed. You will configure an alarm that fires when your processor Lambda encounters more than 1 error in a 5-minute window.

• In main.tf, add the following alarm resource below the log groups you just created:

# CloudWatch Alarm for Lambda errors
resource "aws_cloudwatch_metric_alarm" "processor_errors" {
  alarm_name          = "file-processor-errors-${random_id.suffix.hex}"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 1
  metric_name         = "Errors"
  namespace           = "AWS/Lambda"
  period              = 300
  statistic           = "Sum"
  threshold           = 1
  alarm_description   = "Alarm when file processor Lambda has errors"
  alarm_actions       = [aws_sns_topic.file_notifications.arn]

  dimensions = {
    FunctionName = aws_lambda_function.processor.function_name
  }
}

What does this alarm do?

• namespace = "AWS/Lambda" tells CloudWatch to look at Lambda's built-in metrics.
• metric_name = "Errors" tracks the count of failed invocations.
• period = 300 sets a 5-minute evaluation window (300 seconds).
• threshold = 1 means the alarm triggers if more than 1 error occurs in that window.
• alarm_actions points to your existing SNS topic, so you get an email when the alarm fires.

• Save main.tf.

Not sure where to place the alarm resource?

Place it directly after the three aws_cloudwatch_log_group resources. Terraform does not care about order within a file, but keeping related resources together makes your configuration easier to read.

Help me organize my CloudWatch resources in main.tf.

Create a CloudWatch dashboard

A dashboard gives you a single pane of glass to monitor your pipeline. You will create widgets that show Lambda invocations, errors, duration, and DynamoDB read/write capacity.

• In main.tf, add the following dashboard resource below the alarm you just created:

# CloudWatch Dashboard
resource "aws_cloudwatch_dashboard" "pipeline" {
  dashboard_name = "file-pipeline-${random_id.suffix.hex}"

  dashboard_body = jsonencode({
    widgets = [
      {
        type   = "metric"
        x      = 0
        y      = 0
        width  = 12
        height = 6
        properties = {
          metrics = [
            ["AWS/Lambda", "Invocations", "FunctionName", aws_lambda_function.processor.function_name],
            ["AWS/Lambda", "Errors", "FunctionName", aws_lambda_function.processor.function_name],
            ["AWS/Lambda", "Duration", "FunctionName", aws_lambda_function.processor.function_name]
          ]
          period = 300
          region = var.aws_region
          title  = "Processor Lambda Metrics"
        }
      },
      {
        type   = "metric"
        x      = 12
        y      = 0
        width  = 12
        height = 6
        properties = {
          metrics = [
            ["AWS/Lambda", "Invocations", "FunctionName", aws_lambda_function.query.function_name],
            ["AWS/Lambda", "Errors", "FunctionName", aws_lambda_function.query.function_name],
            ["AWS/Lambda", "Duration", "FunctionName", aws_lambda_function.query.function_name]
          ]
          period = 300
          region = var.aws_region
          title  = "Query Lambda Metrics"
        }
      },
      {
        type   = "metric"
        x      = 0
        y      = 6
        width  = 12
        height = 6
        properties = {
          metrics = [
            ["AWS/DynamoDB", "ConsumedReadCapacityUnits", "TableName", aws_dynamodb_table.file_metadata.name],
            ["AWS/DynamoDB", "ConsumedWriteCapacityUnits", "TableName", aws_dynamodb_table.file_metadata.name]
          ]
          period = 300
          region = var.aws_region
          title  = "DynamoDB Capacity"
        }
      }
    ]
  })
}

What does this dashboard display?

The dashboard has three widgets arranged in a grid. The first widget tracks your processor Lambda (invocations, errors, duration). The second widget tracks your query Lambda with the same metrics.

The third widget shows DynamoDB consumed capacity units. This tells you how much read and write throughput your table is actually using. All widgets use a 300-second (5-minute) period to match the alarm's evaluation window.

• Save main.tf.
• Verify your file has no syntax errors by running:

terraform validate

• You should see Success! The configuration is valid. in the terminal output.

Seeing a JSON encoding error?

The dashboard_body uses jsonencode() which is sensitive to HCL syntax inside it. Make sure you have commas between widget objects in the widgets array and that all brackets are properly closed.

Help me fix a JSON encoding error in my CloudWatch dashboard Terraform resource.

Why is there a dashboard cost?

AWS provides the first 3 CloudWatch dashboards for free. Since this project creates only one dashboard, it stays within the free allowance. If you already have 3 dashboards in your account, each additional one costs $3/month.

✔️ Awesome, I've got everything!

Great. Double check you have saved main.tf with all three CloudWatch sections (log groups, alarm, and dashboard).

ⓧ I'd like to double check the full code

Here is the complete main.tf file with all resources including the new CloudWatch monitoring:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 6.0"
    }
  }
  required_version = ">= 1.0"
}

provider "aws" {
  region = var.aws_region
}

resource "random_id" "suffix" {
  byte_length = 4
}

# S3 Bucket for file uploads
resource "aws_s3_bucket" "uploads" {
  bucket = "file-pipeline-uploads-${random_id.suffix.hex}"
}

# DynamoDB Table for file metadata
resource "aws_dynamodb_table" "file_metadata" {
  name         = "file-metadata-${random_id.suffix.hex}"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "file_id"
  range_key    = "uploaded_at"

  attribute {
    name = "file_id"
    type = "S"
  }

  attribute {
    name = "uploaded_at"
    type = "S"
  }
}

# SNS Topic for notifications
resource "aws_sns_topic" "file_notifications" {
  name = "file-processing-notifications-${random_id.suffix.hex}"
}

resource "aws_sns_topic_subscription" "email" {
  topic_arn = aws_sns_topic.file_notifications.arn
  protocol  = "email"
  endpoint  = var.notification_email
}

resource "aws_iam_role" "lambda_role" {
  name = "file-pipeline-lambda-role-${random_id.suffix.hex}"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "lambda_basic" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

resource "aws_iam_role_policy" "lambda_custom" {
  name = "file-pipeline-custom-policy"
  role = aws_iam_role.lambda_role.id

  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = ["s3:GetObject"]
        Resource = "${aws_s3_bucket.uploads.arn}/*"
      },
      {
        Effect = "Allow"
        Action = ["dynamodb:PutItem", "dynamodb:Scan"]
        Resource = aws_dynamodb_table.file_metadata.arn
      },
      {
        Effect = "Allow"
        Action = ["sns:Publish"]
        Resource = aws_sns_topic.file_notifications.arn
      }
    ]
  })
}

# Lambda: Processor function
data "archive_file" "processor" {
  type        = "zip"
  source_file = "${path.module}/lambda/processor.py"
  output_path = "${path.module}/lambda/processor.zip"
}

resource "aws_lambda_function" "processor" {
  filename         = data.archive_file.processor.output_path
  function_name    = "file-processor-${random_id.suffix.hex}"
  role             = aws_iam_role.lambda_role.arn
  handler          = "processor.handler"
  source_code_hash = data.archive_file.processor.output_base64sha256
  runtime          = "python3.12"
  timeout          = 30

  environment {
    variables = {
      DYNAMODB_TABLE = aws_dynamodb_table.file_metadata.name
      SNS_TOPIC_ARN  = aws_sns_topic.file_notifications.arn
    }
  }
}

resource "aws_lambda_permission" "s3_invoke" {
  statement_id  = "AllowS3Invoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.processor.function_name
  principal     = "s3.amazonaws.com"
  source_arn    = aws_s3_bucket.uploads.arn
}

resource "aws_s3_bucket_notification" "upload_trigger" {
  bucket = aws_s3_bucket.uploads.id

  lambda_function {
    lambda_function_arn = aws_lambda_function.processor.arn
    events             = ["s3:ObjectCreated:*"]
  }

  depends_on = [aws_lambda_permission.s3_invoke]
}

# Lambda: Query function
data "archive_file" "query" {
  type        = "zip"
  source_file = "${path.module}/lambda/query.py"
  output_path = "${path.module}/lambda/query.zip"
}

resource "aws_lambda_function" "query" {
  filename         = data.archive_file.query.output_path
  function_name    = "file-query-${random_id.suffix.hex}"
  role             = aws_iam_role.lambda_role.arn
  handler          = "query.handler"
  source_code_hash = data.archive_file.query.output_base64sha256
  runtime          = "python3.12"
  timeout          = 10

  environment {
    variables = {
      DYNAMODB_TABLE = aws_dynamodb_table.file_metadata.name
    }
  }
}

# API Gateway HTTP API
resource "aws_apigatewayv2_api" "files_api" {
  name          = "file-metadata-api-${random_id.suffix.hex}"
  protocol_type = "HTTP"
}

resource "aws_apigatewayv2_stage" "default" {
  api_id      = aws_apigatewayv2_api.files_api.id
  name        = "$default"
  auto_deploy = true

  access_log_settings {
    destination_arn = aws_cloudwatch_log_group.api_gw.arn
    format = jsonencode({
      requestId      = "$context.requestId"
      sourceIp       = "$context.identity.sourceIp"
      requestTime    = "$context.requestTime"
      httpMethod     = "$context.httpMethod"
      routeKey       = "$context.routeKey"
      status         = "$context.status"
      responseLength = "$context.responseLength"
    })
  }
}

resource "aws_apigatewayv2_integration" "query_lambda" {
  api_id             = aws_apigatewayv2_api.files_api.id
  integration_uri    = aws_lambda_function.query.invoke_arn
  integration_type   = "AWS_PROXY"
  integration_method = "POST"
}

resource "aws_apigatewayv2_route" "get_files" {
  api_id    = aws_apigatewayv2_api.files_api.id
  route_key = "GET /files"
  target    = "integrations/${aws_apigatewayv2_integration.query_lambda.id}"
}

resource "aws_lambda_permission" "api_gw_invoke" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.query.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "${aws_apigatewayv2_api.files_api.execution_arn}/*/*"
}

# CloudWatch Log Groups
resource "aws_cloudwatch_log_group" "processor" {
  name              = "/aws/lambda/${aws_lambda_function.processor.function_name}"
  retention_in_days = 14
}

resource "aws_cloudwatch_log_group" "query" {
  name              = "/aws/lambda/${aws_lambda_function.query.function_name}"
  retention_in_days = 14
}

resource "aws_cloudwatch_log_group" "api_gw" {
  name              = "/aws/api-gw/${aws_apigatewayv2_api.files_api.name}"
  retention_in_days = 14
}

# CloudWatch Alarm for Lambda errors
resource "aws_cloudwatch_metric_alarm" "processor_errors" {
  alarm_name          = "file-processor-errors-${random_id.suffix.hex}"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 1
  metric_name         = "Errors"
  namespace           = "AWS/Lambda"
  period              = 300
  statistic           = "Sum"
  threshold           = 1
  alarm_description   = "Alarm when file processor Lambda has errors"
  alarm_actions       = [aws_sns_topic.file_notifications.arn]

  dimensions = {
    FunctionName = aws_lambda_function.processor.function_name
  }
}

# CloudWatch Dashboard
resource "aws_cloudwatch_dashboard" "pipeline" {
  dashboard_name = "file-pipeline-${random_id.suffix.hex}"

  dashboard_body = jsonencode({
    widgets = [
      {
        type   = "metric"
        x      = 0
        y      = 0
        width  = 12
        height = 6
        properties = {
          metrics = [
            ["AWS/Lambda", "Invocations", "FunctionName", aws_lambda_function.processor.function_name],
            ["AWS/Lambda", "Errors", "FunctionName", aws_lambda_function.processor.function_name],
            ["AWS/Lambda", "Duration", "FunctionName", aws_lambda_function.processor.function_name]
          ]
          period = 300
          region = var.aws_region
          title  = "Processor Lambda Metrics"
        }
      },
      {
        type   = "metric"
        x      = 12
        y      = 0
        width  = 12
        height = 6
        properties = {
          metrics = [
            ["AWS/Lambda", "Invocations", "FunctionName", aws_lambda_function.query.function_name],
            ["AWS/Lambda", "Errors", "FunctionName", aws_lambda_function.query.function_name],
            ["AWS/Lambda", "Duration", "FunctionName", aws_lambda_function.query.function_name]
          ]
          period = 300
          region = var.aws_region
          title  = "Query Lambda Metrics"
        }
      },
      {
        type   = "metric"
        x      = 0
        y      = 6
        width  = 12
        height = 6
        properties = {
          metrics = [
            ["AWS/DynamoDB", "ConsumedReadCapacityUnits", "TableName", aws_dynamodb_table.file_metadata.name],
            ["AWS/DynamoDB", "ConsumedWriteCapacityUnits", "TableName", aws_dynamodb_table.file_metadata.name]
          ]
          period = 300
          region = var.aws_region
          title  = "DynamoDB Capacity"
        }
      }
    ]
  })
}

Your monitoring layer is in place. Next up, you will deploy the entire pipeline and test it end-to-end by uploading a file and watching the data flow through every service.

Deploy, Test, and Verify

All your Terraform configuration is written and ready. This step brings it to life by deploying every resource to AWS and proving the entire pipeline works end-to-end.

You will run terraform plan to preview what gets created, deploy with terraform apply, then trigger the pipeline by uploading a test file. By the end, you will have verified the full event-driven flow: S3 to Lambda to DynamoDB to SNS to API Gateway.

In this step, get ready to:

• Deploy all AWS resources with Terraform.
• Test the pipeline by uploading a file and verifying end-to-end processing.
• View the CloudWatch dashboard to confirm metrics are flowing.

Deploy the infrastructure with Terraform

Terraform needs your personal configuration values before it can deploy. The variables.tf file declares what values are needed, but you supply the actual values in a terraform.tfvars file.

• Create a new file called terraform.tfvars in your project directory (next to main.tf) and add the following content:

aws_region         = "us-west-2"
notification_email = "[[NOTIFICATION_EMAIL=\"your-email@example.com\"]]"

What does this file do?

Terraform automatically loads any file named terraform.tfvars and uses its values to fill in the variables declared in variables.tf. The notification_email is where you will receive SNS notifications when files are processed.

• Save terraform.tfvars.
• Preview everything Terraform will create by running this command:

terraform plan

Terraform will output a detailed list of every resource it intends to create. At the bottom, you should see a summary line like:

Plan: X to add, 0 to change, 0 to destroy.

What should I see?

The plan lists every resource with a + symbol next to it, meaning it will be created. You should see your S3 bucket, DynamoDB table, Lambda functions, API Gateway, IAM roles, SNS topic, CloudWatch resources, and more. The exact count depends on Terraform's internal representation of each resource.

Seeing errors in the plan?

If you see a variable-related error, check that your terraform.tfvars file is saved in the same directory as main.tf. Make sure the email address is wrapped in quotes.

If you see a provider error, confirm your AWS CLI credentials are configured correctly by running aws sts get-caller-identity.

Help me debug my terraform plan error.

• Deploy all resources to AWS by running:

terraform apply

What happens during apply?

Terraform will show the same plan and then prompt you with: "Do you want to perform these actions?" Only yes will be accepted to approve. Type yes and press Enter to proceed.

Deployment takes 1-2 minutes. Terraform creates each resource in the correct dependency order automatically.

• Type yes when prompted and press Enter.
• Wait for the apply to complete. You should see a final message showing all outputs including your S3 bucket name, API endpoint URL, and dashboard URL.

Save your output values

Copy the s3_bucket_name and api_endpoint values from the outputs. You will need them in the next substep to test the pipeline.

• Check your email inbox for a message from AWS with the subject "AWS Notification - Subscription Confirmation".
• Click the Confirm subscription link in the email.

Why do I need to confirm?

AWS requires email subscription confirmation to prevent spam. Until you click the link, the subscription stays in a "PendingConfirmation" state and no notifications will be delivered. This is expected behavior, not a Terraform issue.

Don't see the confirmation email?

Check your spam or junk folder. The email comes from no-reply@sns.amazonaws.com. If you still don't see it after a few minutes, verify that the email address in your terraform.tfvars is correct, then run terraform apply again.

Help me troubleshoot my SNS subscription confirmation.

Upload a test file and verify the pipeline

With the infrastructure deployed and the SNS subscription confirmed, you can now trigger the pipeline. Uploading any file to the S3 bucket fires the entire chain: Lambda processes the file, stores metadata in DynamoDB, and sends you an email.

• Create a small test file by running:

echo "Hello from my serverless pipeline" > test-file.txt

• Upload the test file to your S3 bucket by running:

aws s3 cp test-file.txt s3://[[S3_BUCKET="your-s3-bucket-name-from-output"]]/test-file.txt

Where do I find my bucket name?

Your bucket name was printed in the Terraform outputs after terraform apply. It looks like file-pipeline-uploads- followed by a random hex string. You can also retrieve it at any time by running:

terraform output s3_bucket_name

• Check your email inbox. Within a few seconds you should receive a notification with the subject "File Processed" containing the file name and size.
• Query your API endpoint to verify the metadata was stored in DynamoDB. Copy the api_endpoint value from your Terraform outputs and paste it into your browser's address bar.

You should see a JSON response containing the metadata for your uploaded file, including the file name, size, and timestamp.

What does the API response look like?

The response is a JSON object with a files array and a count field. Each file entry includes file_id, uploaded_at, file_name, file_size, and bucket. This is the exact data your processor Lambda extracted from the S3 event and stored in DynamoDB.

Not seeing the file in the API response?

Make sure you are hitting the full URL including the /files path. The URL looks like https://<api-id>.execute-api.<region>.amazonaws.com/files.

If the response shows an empty array, wait 10-15 seconds and refresh. Lambda cold starts can add a brief delay on the first invocation.

Help me debug why my API isn't returning file metadata.

View the CloudWatch dashboard

Your pipeline triggered successfully. Now let's verify that CloudWatch captured the metrics from that invocation.

• Retrieve your dashboard URL by running:

terraform output dashboard_url

• Copy the URL and paste it into your browser.
• You should see three widgets: Processor Lambda Metrics, Query Lambda Metrics, and DynamoDB Capacity.

What do the dashboard metrics show?

The Processor Lambda widget should show at least 1 invocation from your file upload. The Query Lambda widget shows invocations from when you hit the API endpoint. The DynamoDB widget shows read and write capacity consumed during those operations.

Metrics can take a few minutes to appear. If the widgets are empty, wait 3-5 minutes and refresh the page.

Dashboard showing no data?

CloudWatch metrics have a delay of a few minutes. If you just uploaded the file, wait and refresh. Make sure you are viewing the correct AWS region in the console (top-right corner should match us-west-2 or whichever region you set).

Help me figure out why my CloudWatch dashboard is empty.

Your entire serverless pipeline is deployed and verified. Every piece works: S3 triggers Lambda, Lambda writes to DynamoDB and publishes to SNS, and API Gateway exposes the data publicly. Next up, you will extend this pipeline with fault-tolerance in the Secret Mission.

Secret mission

Add a Dead-Letter Queue

Your pipeline handles the happy path well, but what happens when a Lambda invocation fails? In production, failed messages disappear silently unless you have a safety net. In this secret mission, you'll add an SQS dead-letter queue to catch failed invocations and a CloudWatch alarm that alerts you automatically when something goes wrong.

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

This project uses serverless AWS services that cost nearly nothing at rest. The only recurring charge is the CloudWatch dashboard at $3/month if left running beyond Free Tier credits.

Cost warning

The CloudWatch dashboard costs $3/month. All other resources (Lambda, DynamoDB, SNS, SQS, API Gateway) fall within AWS Always Free limits for the tiny amount of usage this project generates.

If you want to keep experimenting but avoid costs, delete just the dashboard resource. If you're done, run terraform destroy to remove everything in one command.

Resources you used:

• S3 bucket (file uploads).
• DynamoDB table (file metadata).
• 2 Lambda functions (processor and query).
• SNS topic and email subscription.
• API Gateway HTTP API.
• CloudWatch dashboard, log groups, and alarms.
• SQS dead-letter queue (Secret Mission).
• Local project directory on your Desktop.

Keep everything running

No action needed. Choose this if you're still actively building or experimenting with the pipeline.

• The serverless resources (Lambda, DynamoDB, SNS, SQS, API Gateway) cost nothing at rest within Always Free limits.
• The CloudWatch dashboard is the only recurring cost at $3/month.
• To keep everything except the dashboard, you can run terraform destroy -target=aws_cloudwatch_dashboard.pipeline to remove just that resource.
• Your local project files in the file-pipeline folder remain on your Desktop.

Pause - I'll come back to this later

Serverless architectures have no running instances to pause. There is nothing to stop or hibernate.

• Lambda functions only execute when triggered. They consume no resources while idle.
• DynamoDB in on-demand mode charges only for reads and writes. No traffic means no cost.
• If you want to reduce your bill to $0, delete the CloudWatch dashboard by running:

terraform destroy -target=aws_cloudwatch_dashboard.pipeline

• Type yes when prompted to confirm.
• Everything else stays deployed and ready for your next test upload.

Delete - I don't want to use this again

Remove all project resources and start fresh. This is the recommended option after completing the project.

• Empty your S3 bucket first (Terraform cannot delete a bucket that contains objects). Run this command:

aws s3 rm s3://$(terraform output -raw s3_bucket_name) --recursive

• Destroy all AWS resources managed by Terraform by running:

terraform destroy

• Terraform will display all resources it plans to delete. Type yes when prompted to confirm the destruction.
• Wait for Terraform to finish. You should see a message confirming all resources have been destroyed.
• Delete your local project files by running:

cd ~/Desktop
rm -rf file-pipeline

What does terraform destroy remove?

This single command removes all 7+ AWS resources defined in your main.tf file: the S3 bucket, DynamoDB table, both Lambda functions, the SNS topic and subscription, the API Gateway, all CloudWatch resources, and the SQS dead-letter queue.

This is the power of Infrastructure as Code. No manual clicking through the AWS console to find and delete each resource individually.

Nice Work!

Nice Work!

You just built a professional-grade serverless pipeline integrating 7 AWS services, all provisioned through Terraform.

You've learned how to:

• Deploy a complete event-driven architecture where S3 uploads trigger Lambda processing, DynamoDB storage, and SNS email notifications.
• Build a public REST API with API Gateway and Lambda that queries processed file metadata from a NoSQL database.
• Manage all infrastructure as code with Terraform, using the full plan, apply, and destroy lifecycle to provision and tear down cloud resources in seconds.
• Added fault-tolerance with an SQS dead-letter queue and automated CloudWatch alerting (Secret Mission).

Ready to quiz yourself?