Serverless File Pipeline with LocalStack

Build an event-driven S3, Lambda, and DynamoDB pipeline locally.

Introduction

30 Second Summary

Every time you upload a photo to a website or submit a document online, an automated system detects the new file, processes it, and stores information about it in a database. This pattern powers everything from social media uploads to order processing at major companies.

In this project, you will build that exact system on your own computer. You'll use LocalStack to simulate AWS locally, then wire together three core services into an automated pipeline that processes files the moment they're uploaded.

What You'll Build

A serverless file processing pipeline that automatically detects file uploads to S3, extracts metadata using a Lambda function, and stores it in a DynamoDB database.

By the end of this project, you'll have:

  • A running LocalStack environment simulating AWS on your local machine, with no real AWS account needed.
  • An event-driven pipeline where uploading a file to S3 automatically triggers a Lambda function that saves metadata to DynamoDB.
  • Hands-on experience with real AWS CLI commands that work identically in production environments.
  • Secret Mission: Add an SQS dead letter queue to catch and recover from processing failures.

Do I need an AWS account or prior cloud experience?

No. Everything in this project runs locally on your computer using LocalStack. You do not need a real AWS account, you will not incur any cloud costs, and no prior AWS experience is required.

This guide walks you through installing every tool from scratch. You just need Docker Desktop, Python, and VS Code already on your machine.

Before We Start

To build an automated file processing pipeline, you need three tools that work together: the AWS CLI (your remote control for cloud services), LocalStack (a fake AWS that runs on your computer), and awslocal (a shortcut that points your commands at the fake AWS instead of the real one).

In this step, you will install all three tools and confirm they are working. Every command you learn here is identical to what you would use in a real cloud job.

In this step, get ready to:
  • Install the AWS CLI, LocalStack, and awslocal on your machine.
  • Create a free LocalStack account and connect it to your CLI.
  • Start LocalStack and verify it responds to commands.
Install the AWS CLI

The AWS CLI is a program that lets you type commands to create and manage AWS resources. Think of it as a remote control for the cloud.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type Terminal (macOS) or cmd (Windows) and press Enter to open it.

Why a terminal?

The terminal is where you type text commands instead of clicking buttons. All AWS work (both local and real) happens here.

  • Install the AWS CLI by following the instructions for your operating system:

macOS

  • Download and install the AWS CLI by running these commands:
curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
sudo installer -pkg AWSCLIV2.pkg -target /

What does this command do?

The first line downloads the official AWS CLI installer to your current folder. The second line runs the macOS installer, which places the aws command on your system.

Your terminal will ask for your Mac password because sudo means "run with admin permissions". Type your password and press Enter. You will not see the characters as you type.

Windows

  • Download the installer from this URL in your browser: https://awscli.amazonaws.com/AWSCLIV2.msi
  • Run the downloaded file and click Next through each screen until it finishes.
  • Close any open Command Prompt windows after the install completes.

Why close the window?

Windows needs a fresh terminal to recognize newly installed programs. Closing and reopening ensures the aws command is available.

  • Open a fresh terminal and verify the install by running:
aws --version

✔️ I see a version number starting with aws-cli/2

The AWS CLI is installed and ready. Move on to the next substep.

ⓧ Command not found

  • On macOS, make sure you ran both the curl and sudo installer commands above.
  • On Windows, restart your computer and open a fresh Command Prompt.
  • Run aws --version again in the new terminal.

help me install the AWS CLI

Install LocalStack and awslocal

LocalStack is a program that pretends to be AWS on your computer. It listens for commands and responds just like the real AWS would.

The awslocal wrapper is a shortcut. Instead of typing long commands telling the AWS CLI to talk to your fake AWS, you type awslocal instead of aws and it handles the rest.

  • Install LocalStack by running:
pip install localstack
  • Install the awslocal wrapper by running:
pip install awscli-local[ver1]

What is awscli-local?

This package installs the awslocal command. It is a thin wrapper and substitute for the standard aws command, enabling AWS CLI commands within LocalStack without needing to add --endpoint-url to every command.

  • Check LocalStack installed correctly by running:
localstack --version

✔️ I see a version number

LocalStack is installed. Move on to creating your free account.

ⓧ Command not found

If pip install completed successfully but the CLI is not found, your PATH may not include the Python Scripts directory.

  • On Windows, close and reopen Command Prompt, then try again.
  • On macOS, close and reopen Terminal, then try again.
  • If it still fails, try running python -m localstack.cli.main --version to confirm it installed.

help me fix my LocalStack installation

  • Create a free LocalStack account (required even for free use):
  • Go to https://app.localstack.cloud in your browser.
  • Sign up with GitHub or email. This is completely free on the Hobby plan.
  • Once logged in, navigate to your account settings and find Auth Tokens.
  • Copy your auth token.

Why do I need an account?

As of March 2026, all LocalStack usage (including the free Hobby tier) requires an auth token. The Hobby plan gives you free access to basic service emulation for 30 AWS services. No credit card required.

  • Set your auth token by running this command (paste your actual token in place of the placeholder):
localstack auth set-token your-token-here

What does this command do?

This saves your auth token so LocalStack can verify your account each time it starts. You only need to do this once.

Start LocalStack and set up fake credentials

Now that everything is installed, you need to start LocalStack and tell the AWS CLI how to talk to it. Real AWS needs real passwords, but LocalStack accepts anything.

  • Make sure Docker Desktop is open and running. You should see its icon in your taskbar or menu bar.
  • Start LocalStack in the background by running:
localstack start -d

What does the -d flag mean?

The -d means "detached". LocalStack runs in the background so you can keep using the same terminal for other commands.

  • Wait for LocalStack to finish starting by running:
localstack wait -t 60

This waits up to 60 seconds for LocalStack to become ready. Once it finishes without an error, your fake AWS is running on port 4566.

Seeing a timeout or Docker error?

  • Make sure Docker Desktop is fully started (not just opening). Wait for the Docker icon to stop animating.
  • If you see a port conflict error, another program may be using port 4566. Close other development servers and try again.

help me start LocalStack

  • Set up fake credentials by running:
aws configure
  • The command will ask you four questions. Enter these values exactly:
  • AWS Access Key ID: type test and press Enter.
  • AWS Secret Access Key: type test and press Enter.
  • Default region name: type us-east-1 and press Enter.
  • Default output format: type json and press Enter.

Why are we using fake credentials?

Real AWS requires real access keys that cost money if misused. LocalStack does not validate credentials, so test works perfectly. You are building the same muscle memory without the risk.

  • Verify everything works end-to-end by running:
awslocal s3 ls

✔️ I see an empty response with no error

LocalStack is running and responding to your commands. The empty output is correct because you have not created any S3 buckets yet.

ⓧ I see an error

  • If you see "Could not connect to the endpoint URL", LocalStack may not be running. Run localstack start -d again and wait for localstack wait -t 60 to complete.
  • If you see "command not found" for awslocal, close and reopen your terminal so it picks up the new PATH.

help me connect to LocalStack

Your local AWS environment is ready. Next up, you will create your first cloud resources: an S3 bucket for file storage and a DynamoDB table for your database.

Install Your Tools

Building a serverless pipeline requires three tools working together: the AWS CLI for sending commands to AWS, LocalStack for simulating AWS on your computer, and awslocal for routing those commands to the local simulation.

In this step, you'll confirm each tool is installed and verify that LocalStack is running and responding to commands.

In this step, get ready to:
  • Confirm the AWS CLI is working.
  • Confirm LocalStack and awslocal are installed.
  • Verify LocalStack is running and responding to commands.
Confirm the AWS CLI is ready

The AWS CLI is a program that lets you type commands to create and manage AWS resources. Think of it as a remote control for the cloud.

  • Confirm the AWS CLI is installed by running this command:
aws --version

You should see output starting with aws-cli/2 followed by a Python version and your operating system.

What is the AWS CLI?

CLI stands for Command Line Interface. Instead of clicking through web pages in the AWS console, you type short commands to create buckets, deploy functions, and query databases.

Every command you learn with the AWS CLI works identically whether you're talking to LocalStack on your laptop or real AWS in production. This is why practicing locally builds directly transferable skills.

Don't see a version number?

If you see 'aws' is not recognized, close your terminal and open a fresh one. The installer updates your PATH, but existing terminals don't pick up the change automatically.

If that doesn't help, restart your computer and try again. Help me troubleshoot my AWS CLI installation.

Confirm LocalStack and awslocal are ready

LocalStack is a program that pretends to be AWS on your computer. It listens for commands and responds just like the real AWS would.

  • Check your LocalStack version by running:
localstack --version

You should see a version number like 4.5.0 or similar.

What is LocalStack?

LocalStack emulates over 30 AWS services on your machine using Docker containers. When you send it a command, it processes that command the same way real AWS would.

This means you can build and test AWS applications without an AWS account, without internet access, and without spending any money.

What is awslocal?

Awslocal is a thin wrapper around the aws command. Instead of typing aws --endpoint-url=http://localhost:4566 s3 ls every time, you just type awslocal s3 ls and it handles the routing for you.

The commands themselves are identical to real AWS commands. You build the same muscle memory you'd use in a production environment.

LocalStack command not found?

If your terminal doesn't recognize localstack, your Python Scripts directory may not be in your PATH. Close and reopen your terminal, then try again.

Help me fix my LocalStack installation.

Verify LocalStack is running and responding

LocalStack runs as a background process inside a Docker container. It needs Docker Desktop to be running and your auth token to be set before it can start.

  • Confirm LocalStack is responding to commands by running:
awslocal s3 ls

You should see empty output with no error message. That means LocalStack is running and listening for commands.

Why do I use fake credentials?

Real AWS requires access keys and secret keys to authenticate you. LocalStack doesn't enforce credentials, but the AWS CLI still expects them to be configured.

Your AWS CLI is configured with test as both the Access Key ID and Secret Access Key. LocalStack accepts any value here. The region is set to us-east-1 to match how most production AWS accounts are configured.

Seeing a connection error?

Make sure Docker Desktop is running. LocalStack needs Docker to be active in the background. Check your taskbar for the Docker whale icon.

If Docker is running but you still see errors, LocalStack may not have started yet. Help me troubleshoot my LocalStack connection.

Your local AWS environment is ready. Next up, you'll create your first AWS resources: an S3 bucket for storing files and a DynamoDB table for tracking metadata.

Create Your Storage and Database

Your tools are installed and LocalStack is running. You have a fake AWS environment on your own computer, ready to receive commands.

Now you will create the two core AWS resources for your pipeline: an S3 bucket to store uploaded files, and a DynamoDB table to log information about them. These are the storage and database layers that your Lambda function will connect to in the next step.

In this step, get ready to:
  • Create an S3 bucket for file storage.
  • Create a DynamoDB table for file metadata.
  • Upload a test file to verify your bucket works.
Create an S3 bucket

An S3 bucket is a named container for files. Think of it like a folder in the cloud where you store anything. The name must be unique, like a username.

  • Create a bucket called file-uploads by running this command:
awslocal s3 mb s3://file-uploads

What does this command do?

The mb stands for "make bucket". The s3:// prefix tells the CLI you are talking about S3.

Because you are using awslocal instead of aws, the command goes to your fake AWS (LocalStack) instead of the real cloud.

  • Verify the bucket exists by running:
awslocal s3 ls

You should see a line with today's date and the name file-uploads.

Don't see file-uploads listed?

  • Make sure LocalStack is still running. Run localstack status to check.
  • If LocalStack stopped, restart it with localstack start -d and wait with localstack wait -t 60 before trying again.
  • help me troubleshoot why my S3 bucket was not created in LocalStack
Create a DynamoDB table

DynamoDB is a database. Instead of rows and columns like a spreadsheet, it stores "items" (like index cards with information on them). Each item needs a unique identifier.

Your table will be called file-metadata and each item will have a unique text ID called file_id.

  • Create the table by running this command:
awslocal dynamodb create-table --table-name file-metadata --attribute-definitions AttributeName=file_id,AttributeType=S --key-schema AttributeName=file_id,KeyType=HASH --billing-mode PAY_PER_REQUEST

What does this command do?

In plain English, this command says: "Create a table called file-metadata. Each item has a unique text ID called file_id. Charge me per use instead of a fixed rate."

  • --table-name file-metadata names the table.
  • --attribute-definitions defines the data types. AttributeType=S means the ID is a text string.
  • --key-schema sets which attribute is the unique identifier. KeyType=HASH means it is the primary key.
  • --billing-mode PAY_PER_REQUEST means "charge per use" (not relevant locally, but this is how you would do it in production).
  • Verify the table exists by running:
awslocal dynamodb list-tables

You should see file-metadata inside a TableNames array in the JSON output.

Don't see file-metadata in the output?

  • Check for typos in the command. The table name must be exactly file-metadata (with a hyphen, not an underscore).
  • If you see an error about the table already existing, that means it was created successfully on a previous attempt. Run the list-tables command again to confirm.
  • help me troubleshoot why my DynamoDB table was not created in LocalStack
Upload a test file to S3

Before wiring everything together, confirm that your bucket can actually receive files.

  • Create a small test file by running:
echo "Hello AWS" > test-file.txt
  • Upload it to your bucket by running:
awslocal s3 cp test-file.txt s3://file-uploads/

What does this command do?

The cp stands for "copy". You are copying the file from your computer into the S3 bucket. This is the same command you would use in a real AWS job.

  • Confirm the file arrived by running:
awslocal s3 ls s3://file-uploads/

You should see test-file.txt listed in the output. Your bucket is working and ready to receive files.

Don't see test-file.txt in the bucket?

  • Make sure you ran the echo command first. The file needs to exist on your computer before you can upload it.
  • Check that you included the trailing slash in s3://file-uploads/ when running the copy command.
  • help me troubleshoot why my file did not upload to S3 in LocalStack

Your storage and database are ready. Next up, you will write a Lambda function that automatically processes files when they land in your bucket.

Write and Deploy Your Lambda Function

Your S3 bucket and DynamoDB table are both set up and ready. Now it's time to write the code that actually does the work when a file arrives.

A Lambda function is a piece of code that AWS runs for you. You don't need to set up a server. You just give AWS your code and tell it when to run it. Right now, nothing happens when a file lands in your bucket. You need a piece of code that wakes up automatically and processes each upload.

In this step, get ready to:
  • Write a Python function that reads file upload events and saves metadata to DynamoDB.
  • Package and deploy the function to LocalStack.
  • Test the function manually with a fake upload event.
Create the Lambda function code

A Lambda function is a Python file with a special function that AWS knows how to call. When something triggers it (like a file upload), AWS sends it an "event" containing details about what happened. Your function reads those details and saves them to the database.

  • Move to your Desktop by running this command:
cd ~/Desktop
  • Create your project folder and move into it by running these commands:
mkdir file-processor
cd file-processor
  • Confirm the folder exists by running:
ls

You should see an empty output (the folder is new). Your terminal is now inside the file-processor folder on your Desktop.

Seeing a 'directory already exists' error?

If you already have a file-processor folder on your Desktop from a previous attempt, delete it first or just run cd file-processor to move into the existing one.

Why isn't my folder being created?

  • Open this folder in VS Code by running:
code .

VS Code opens with file-processor shown in the title bar and an empty Explorer sidebar.

Getting 'code' command not found?

This means VS Code's command-line tool isn't in your system PATH. Open VS Code manually using Cmd+Space (macOS) or the Windows key (Windows), type VS Code, then use File > Open Folder to open your file-processor folder.

Help me fix the 'code' command not found error.

  • Create a new file called handler.py in VS Code.
  • In the Explorer sidebar, click the New File icon (a page with a plus sign).
  • Type handler.py and press Enter.

This file will contain all the code that runs when a file is uploaded. You'll write it in two parts: the setup code at the top, then the main function.

  • Add the imports and DynamoDB connection setup to handler.py by pasting this code:
import json
import boto3
import os
from datetime import datetime, timezone

# Connect to DynamoDB running in LocalStack
# boto3 is the Python tool for talking to AWS services
dynamodb = boto3.resource(
    "dynamodb",
    endpoint_url=os.environ.get("AWS_ENDPOINT_URL", "http://localhost:4566"),
    region_name="us-east-1",
    aws_access_key_id="test",
    aws_secret_access_key="test",
)

# Point to our table
table = dynamodb.Table("file-metadata")

What does this code do?

  • boto3 is the official Python library for talking to AWS services. It's how your code creates, reads, and manages AWS resources.
  • endpoint_url points boto3 at LocalStack (http://localhost:4566) instead of real AWS. This is what makes everything run locally.
  • The fake credentials (test/test) work because LocalStack doesn't validate them.
  • table = dynamodb.Table("file-metadata") connects to the DynamoDB table you created in the previous step.
  • Below the setup code, add the main function by pasting this code:


def lambda_handler(event, context):
    """
    This function runs automatically when a file is uploaded to S3.
    AWS sends it an 'event' that contains info about what happened.
    """
    print("Received event:", json.dumps(event))

    # The event contains a list of "Records" (one per file uploaded)
    for record in event["Records"]:
        # Pull out the details about the uploaded file
        bucket_name = record["s3"]["bucket"]["name"]
        file_key = record["s3"]["object"]["key"]
        file_size = record["s3"]["object"].get("size", 0)

        # Create a unique ID and timestamp
        timestamp = datetime.now(timezone.utc).isoformat()
        file_id = f"{file_key}-{timestamp}"

        # Build the item we will save to the database
        item = {
            "file_id": file_id,
            "bucket": bucket_name,
            "key": file_key,
            "size": file_size,
            "timestamp": timestamp,
            "status": "processed",
        }

        # Save it to DynamoDB
        table.put_item(Item=item)
        print(f"Stored metadata for {file_key} in DynamoDB")

    # Tell whoever called us that everything went fine
    return {
        "statusCode": 200,
        "body": json.dumps({"message": "Files processed successfully"}),
    }

What does this function do?

AWS calls lambda_handler automatically and passes it an event object. The event contains a list of records, one per file uploaded.

  • For each uploaded file, it pulls out the bucket name, file path (key), and file size.
  • It generates a unique file_id by combining the file path with a timestamp.
  • It saves all the metadata as an item in your file-metadata DynamoDB table.
  • It returns a success response (statusCode: 200) so the caller knows everything went fine.
  • Save handler.py by pressing Cmd+S (macOS) or Ctrl+S (Windows).

Seeing red underlines in VS Code?

If VS Code shows errors under boto3, that just means the library isn't installed in your local Python environment. That's fine. The code runs inside LocalStack's Lambda container, which has boto3 pre-installed.

Help me understand why VS Code shows errors in my Lambda code.

✔️ Awesome, I've got everything!

Great. Double check you've saved handler.py and move on to packaging it.

ⓧ I'd like to double check the full code

Here's the complete handler.py file for comparison:

import json
import boto3
import os
from datetime import datetime, timezone

# Connect to DynamoDB running in LocalStack
# boto3 is the Python tool for talking to AWS services
dynamodb = boto3.resource(
    "dynamodb",
    endpoint_url=os.environ.get("AWS_ENDPOINT_URL", "http://localhost:4566"),
    region_name="us-east-1",
    aws_access_key_id="test",
    aws_secret_access_key="test",
)

# Point to our table
table = dynamodb.Table("file-metadata")


def lambda_handler(event, context):
    """
    This function runs automatically when a file is uploaded to S3.
    AWS sends it an 'event' that contains info about what happened.
    """
    print("Received event:", json.dumps(event))

    # The event contains a list of "Records" (one per file uploaded)
    for record in event["Records"]:
        # Pull out the details about the uploaded file
        bucket_name = record["s3"]["bucket"]["name"]
        file_key = record["s3"]["object"]["key"]
        file_size = record["s3"]["object"].get("size", 0)

        # Create a unique ID and timestamp
        timestamp = datetime.now(timezone.utc).isoformat()
        file_id = f"{file_key}-{timestamp}"

        # Build the item we will save to the database
        item = {
            "file_id": file_id,
            "bucket": bucket_name,
            "key": file_key,
            "size": file_size,
            "timestamp": timestamp,
            "status": "processed",
        }

        # Save it to DynamoDB
        table.put_item(Item=item)
        print(f"Stored metadata for {file_key} in DynamoDB")

    # Tell whoever called us that everything went fine
    return {
        "statusCode": 200,
        "body": json.dumps({"message": "Files processed successfully"}),
    }
Package and deploy the function to LocalStack

Lambda needs your code in a zip file before it can run it. You also need to create an IAM role, which is like a permission badge that tells AWS what your code is allowed to do.

  • Switch back to your terminal and zip your handler file by running:

macOS

zip handler.zip handler.py

Windows

tar -a -cf handler.zip handler.py
  • Confirm the zip file was created by running:
ls

You should see both handler.py and handler.zip in the output.

  • Create an IAM role for your Lambda function by running:
awslocal iam create-role --role-name lambda-role --assume-role-policy-document "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"lambda.amazonaws.com\"},\"Action\":\"sts:AssumeRole\"}]}"

What is an IAM role?

In real AWS, a role is a permission badge that says what a piece of code is allowed to do. This command creates a badge called lambda-role and says "Lambda is allowed to wear it."

LocalStack doesn't actually enforce permissions, but you still need to create the role because real AWS requires it. This keeps your commands production-ready.

You should see a JSON response showing the role details including the RoleName as lambda-role.

  • Deploy your Lambda function to LocalStack by running:
awslocal lambda create-function --function-name file-processor --runtime python3.11 --role arn:aws:iam::000000000000:role/lambda-role --handler handler.lambda_handler --zip-file fileb://handler.zip

What does this command do?

  • --function-name file-processor names your function.
  • --runtime python3.11 tells Lambda which language version to use.
  • --role arn:aws:iam::000000000000:role/lambda-role attaches the permission badge you just created.
  • --handler handler.lambda_handler tells Lambda to call the lambda_handler function inside handler.py.
  • --zip-file fileb://handler.zip provides the zip file containing your code.
  • Verify the function deployed successfully by running:
awslocal lambda list-functions

You should see file-processor in the output with its runtime, handler, and role details.

Not seeing your function in the list?

Make sure you ran the create-function command from inside the file-processor folder (where handler.zip lives). If the zip file can't be found, the deployment fails silently.

Also confirm LocalStack is still running with localstack status. If it stopped, restart it with localstack start -d.

Help me troubleshoot my Lambda deployment.

Test the function manually

Before wiring everything together, you'll test the function by hand. You'll send it a fake event that looks like "a file was uploaded to S3."

  • Switch back to VS Code and create a new file called test-event.json in the file-processor folder.
  • Click the New File icon in the Explorer sidebar.
  • Type test-event.json and press Enter.
  • Paste this fake S3 event into test-event.json:
{
  "Records": [
    {
      "eventSource": "aws:s3",
      "eventName": "ObjectCreated:Put",
      "s3": {
        "bucket": {
          "name": "file-uploads"
        },
        "object": {
          "key": "test-file.txt",
          "size": 10
        }
      }
    }
  ]
}

What is this file?

This is a fake message that mimics what AWS sends to Lambda when a file is uploaded. It says: "a file called test-file.txt (10 bytes) was uploaded to the file-uploads bucket."

In production, AWS generates this automatically. For testing, you create it by hand.

  • Save test-event.json by pressing Cmd+S (macOS) or Ctrl+S (Windows).
  • Switch back to your terminal and invoke the function with your fake event by running:
awslocal lambda invoke --function-name file-processor --payload fileb://test-event.json output.json

You should see a response showing StatusCode: 200 in the terminal.

  • Check what your function returned by running:
cat output.json

You should see a JSON response containing "statusCode": 200 and "message": "Files processed successfully".

  • Confirm the function actually saved data to your DynamoDB table by running:
awslocal dynamodb scan --table-name file-metadata

You should see an item with the file name (test-file.txt), size, timestamp, and status processed. Your function works.

Not seeing any items in the scan?

Check the output.json file. If it shows an error instead of statusCode: 200, your function hit a problem.

Common causes: a typo in handler.py (especially in the table.put_item or dynamodb.Table lines), or the table name in your code doesn't match file-metadata. Double check using the full code tab above.

Help me debug my Lambda function output.

Your Lambda function is deployed and tested. Next up, you'll wire everything together so that uploading a file to S3 automatically triggers your function.

Connect Everything Together

In the last step, you built and tested your Lambda function by hand. You confirmed it can read file information and write it to DynamoDB.

But right now, all the pieces exist separately. Nothing connects them. In this step, you will tell S3: "whenever a new file lands in you, automatically run my Lambda function." This is the moment everything becomes automatic.

In this step, get ready to:
  • Tell S3 to trigger your Lambda function on every file upload.
  • Upload test files and watch the pipeline run automatically.
  • Confirm that DynamoDB received the processed results.
Tell S3 to trigger Lambda when files are uploaded

S3 can send notifications when things happen inside a bucket. You will create a configuration file that tells S3 which events to watch for and which Lambda function to call.

  • In VS Code, create a new file called s3-notification.json inside your file-processor folder.
  • Paste the following configuration into it:
{
  "LambdaFunctionConfigurations": [
    {
      "LambdaFunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:file-processor",
      "Events": ["s3:ObjectCreated:*"]
    }
  ]
}

What does this configuration say?

LambdaFunctionArn is the unique address of your Lambda function. It tells S3 exactly which function to call when something happens.

s3:ObjectCreated:* means "any time a new object is created in this bucket, no matter how it got there." The wildcard * covers all creation methods (uploads, copies, multipart uploads).

  • Save s3-notification.json.

✔️ Awesome, I've got everything!

Great. Double check you have saved your file before moving on.

ⓧ I'd like to double check the full code

{
  "LambdaFunctionConfigurations": [
    {
      "LambdaFunctionArn": "arn:aws:lambda:us-east-1:000000000000:function:file-processor",
      "Events": ["s3:ObjectCreated:*"]
    }
  ]
}
  • Apply this notification configuration to your file-uploads bucket by running this command:
awslocal s3api put-bucket-notification-configuration --bucket file-uploads --notification-configuration file://s3-notification.json

What does this command do?

put-bucket-notification-configuration attaches your notification rules to the file-uploads bucket. From now on, S3 knows to call your Lambda function whenever a file arrives.

  • Confirm the configuration is active by running:
awslocal s3api get-bucket-notification-configuration --bucket file-uploads

You should see a JSON response containing your Lambda function ARN (arn:aws:lambda:us-east-1:000000000000:function:file-processor) in the output.

Not seeing your Lambda ARN in the output?

  • Make sure you saved s3-notification.json before running the apply command. The CLI reads the file from disk.
  • Check that your terminal is in the file-processor folder where the file lives. Run dir to list files and confirm s3-notification.json appears.
  • Why isn't my S3 notification configuration showing my Lambda function?
Test the full pipeline

Everything is wired up. Now for the real test. You will upload files to S3 and your Lambda function should fire automatically without you doing anything else.

  • Create a couple of test files by running these commands:
echo "Project report content" > report.txt
echo "Meeting notes content" > notes.txt
  • Upload both files to your S3 bucket by running:
awslocal s3 cp report.txt s3://file-uploads/documents/report.txt
awslocal s3 cp notes.txt s3://file-uploads/documents/notes.txt

What's different this time?

In Step 2, when you uploaded test-file.txt, nothing happened automatically. You had to invoke Lambda by hand with a fake event.

This time, S3 will detect the new files and trigger your Lambda function on its own. No manual invocation needed.

Wait about 5 seconds. Lambda needs a moment to wake up and process the uploads.

Confirm the pipeline worked end-to-end

This is the moment of truth. If your pipeline is working, DynamoDB should now contain entries for both uploaded files.

  • Check your database for the new entries by running:
awslocal dynamodb scan --table-name file-metadata

You should see entries for both documents/report.txt and documents/notes.txt with their sizes and timestamps.

  • List your files in the bucket to confirm they arrived by running:
awslocal s3 ls s3://file-uploads/documents/

You should see report.txt and notes.txt listed in the output.

Don't see entries for report.txt and notes.txt in DynamoDB?

  • Wait a few more seconds and run the scan again. Lambda sometimes takes a moment to process.
  • Re-run the get-bucket-notification-configuration command from the previous substep. If the output is empty, your notification was not applied. Re-run the put command.
  • Make sure LocalStack is still running by checking localstack status. If it stopped, restart it with localstack start -d.
  • Why aren't my S3 uploads triggering my Lambda function automatically?

If you see those entries, congratulations. You just built an event-driven architecture pipeline. A file went into S3, that triggered Lambda, and Lambda wrote to DynamoDB. All automatically.

Your pipeline is fully operational. Every file that lands in your bucket now gets automatically processed and logged. Next up, you will add a safety net for when things go wrong.

Secret mission

Add a Dead Letter Queue for Failed Processing

When your Lambda function crashes, the event that triggered it just disappears into the void. In production, you never want to lose data. Can you add a safety net that catches failed events so you can investigate them later?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. This project runs entirely locally on your machine using LocalStack, so there are no ongoing costs.

Resources you used:

  • LocalStack running in Docker (contains your S3 bucket, Lambda function, DynamoDB table, SQS queue, and IAM role).
  • Project folder file-processor on your Desktop (contains handler.py, test-event.json, s3-notification.json, and handler.zip).
  • AWS CLI v2 (installed via MSI installer).
  • LocalStack CLI and awslocal (installed via pip).

Keep everything running

No action needed. Choose this if you are still actively building or want to experiment more with LocalStack.

  • LocalStack will keep running in the background until you stop it or restart your computer.
  • Docker Desktop will continue using memory while LocalStack is running.
  • Your free LocalStack account remains active at app.localstack.cloud.
  • The AWS CLI, LocalStack CLI, and awslocal remain installed on your machine for future projects.

Pause - I'll come back to this later

Shut down LocalStack to free up memory. Your project files stay on disk so you can restart later.

  • Stop LocalStack by running this command in your terminal:
localstack stop
  • This stops the Docker container. All AWS resources (S3 bucket, Lambda, DynamoDB table, SQS queue) are destroyed when LocalStack stops.
  • Your file-processor folder stays on your Desktop. When you want to resume, start LocalStack again and re-run the setup commands from Steps 2-4.

Delete - I don't want to use this again

Remove all project resources completely and free up disk space.

  • Stop LocalStack by running:
localstack stop
  • Delete the project folder by running:
cd ~/Desktop
rmdir /s /q file-processor

On macOS or Linux?

Use this command instead:

cd ~/Desktop
rm -rf file-processor
  • Uninstall LocalStack CLI and awslocal by running:
pip uninstall localstack -y
pip uninstall awscli-local -y
  • Uninstall the AWS CLI by opening Programs and Features in Windows (search appwiz.cpl in your Start menu). Select AWS Command Line Interface and click Uninstall.
  • Optionally, remove the stored fake credentials by running:
rmdir /s /q %UserProfile%\.aws

On macOS or Linux?

Use this command instead:

rm -rf ~/.aws

Nice Work!

Nice Work!

You just built a fully automated serverless file processing pipeline using the three most in-demand AWS services. Every single command you typed works identically on real AWS.

You've learned how to:

  • Set up a local AWS development environment using LocalStack and used the AWS CLI to create and manage cloud resources from your terminal.
  • Wrote, deployed, and tested a Python Lambda function that automatically processes file uploads and saves metadata to a DynamoDB table.
  • Wired S3, Lambda, and DynamoDB into an event-driven pipeline where uploading a file triggers automatic processing and database storage.
  • Secret Mission: Added error handling with an SQS dead letter queue to catch failed processing events. A production best practice that shows you think about what happens when things go wrong.

Ready to quiz yourself?