Deploy a Serverless Pipeline with AWS CLI

Deploy an S3-to-Lambda-to-DynamoDB pipeline entirely from your terminal.

Introduction

30 Second Summary

Every time you upload a photo to a cloud service, something happens behind the scenes. The file is automatically catalogued and stored without you clicking a single extra button.

In this project, you will build that same automatic file processing on AWS, deployed entirely from the AWS CLI. When a file lands in your storage bucket, a serverless function fires and writes its metadata to a database.

What You'll Build

You'll drop a file into an S3 bucket and seconds later query DynamoDB from your terminal to see that Lambda automatically extracted and stored the file's metadata.

By the end of this project, you'll have:

Drop any file into S3 and see its metadata record (filename, size, timestamp) appear in DynamoDB within seconds, powered by a fully event-driven pipeline.
Invoke your Lambda function from the terminal and read CloudWatch logs to confirm exactly what happened during execution.
Tear down every resource with a clean CLI cleanup sequence in the correct dependency order, proving the operational discipline production teams rely on.
Secret Mission: Add prefix filtering so the pipeline only fires for files uploaded to a specific folder.

Do I need any prerequisites?

You'll need an AWS account (free tier is fine). The AWS CLI will be installed and configured as part of Step 1.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters. This step is your moment to commit to the goal before any hands-on work begins.

Install and Configure the AWS CLI

Cloud engineers provision and manage infrastructure from the terminal. The AWS CLI is the primary tool for interacting with AWS services programmatically.

In this step, you will install AWS CLI v2 in your WSL2 Ubuntu terminal, configure it with your credentials, and verify that you can authenticate with AWS.

In this step, get ready to:

Install AWS CLI v2 in your WSL2 Ubuntu terminal.
Configure the CLI with your AWS credentials and default region.
Verify authentication by querying your AWS account identity.

Install AWS CLI v2

The AWS CLI needs to be installed before you can run any AWS commands. You will use your WSL2 Ubuntu terminal for the installation.

Open your WSL2 Ubuntu terminal.
Check if you already have the AWS CLI installed by running this command:

aws --version

Check what you see in your terminal:

✔️ I see version 2.x or higher

Your AWS CLI is already installed and up to date. Skip ahead to the next substep where you configure your credentials.

ⓧ I see version 1.x

You have an older version of the AWS CLI. You need version 2 for this project. Install AWS CLI v2 by running these commands:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip -o awscliv2.zip
sudo ./aws/install --update

What do these commands do?

This downloads the official AWS CLI v2 installer from Amazon, unzips it, and runs the install script with the --update flag to upgrade your existing v1 installation to v2.

Confirm the new version by running:

aws --version

You should see output starting with aws-cli/2. followed by version details.

ⓧ Command not found

The AWS CLI is not installed yet. Install AWS CLI v2 by running these commands:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

What do these commands do?

This downloads the official AWS CLI v2 installer from Amazon, unzips it, and runs the install script. The installer places the aws binary in /usr/local/bin/ so it is available system-wide.

If unzip is not found, install it first by running:

sudo apt update && sudo apt install unzip -y

After installing unzip, re-run the three commands above.
Confirm the installation by running:

aws --version

You should see output starting with aws-cli/2. followed by version details.

Still seeing command not found after installing?

Close and reopen your terminal, then try aws --version again. If it still fails, check that /usr/local/bin is in your PATH by running echo $PATH.

help me troubleshoot why aws --version is not working after installing the AWS CLI on WSL2 Ubuntu

Configure the CLI with your credentials

The CLI needs your AWS credentials to authenticate requests. You will create an access key in the AWS Console, then feed it to the CLI.

Where do I get my access keys?

Access keys are a pair of values (an Access Key ID and a Secret Access Key) that let the CLI prove your identity to AWS. You create them in the IAM console.

Open https://console.aws.amazon.com/iam/ and sign in.
In the left navigation pane, select Users.
Click your user name to open the user details page.
Select the Security credentials tab.
In the Access keys section, click Create access key.
Select Other as the use case, then click Next.
Click Create access key.
Click Download .csv file and save it in a secure location. You will not be able to see the Secret Access Key again after closing this page.

Keep this page open or have the downloaded file handy for the next step.

Configure the CLI with your credentials by running this command:

aws configure

The CLI will prompt you for four values. Enter them one at a time:

AWS Access Key ID: paste the Access Key ID from your downloaded CSV or the IAM page.
AWS Secret Access Key: paste the Secret Access Key from the same source.
Default region name: type us-east-1.
Default output format: type json.

What does aws configure do?

This command stores your credentials and preferences in a configuration file on your machine. Every future aws command reads these values automatically so you do not have to provide them each time.

The region us-east-1 tells the CLI which AWS data center to target by default. The json output format makes command results easy to read in your terminal.

🙋‍♀️ Accidentally pasted the wrong value?

Run aws configure again. It will prompt you for all four values and overwrite whatever was there before.

help me fix my AWS CLI configuration

Verify your configuration

Before you run this next command, what do you think will happen if your credentials are configured correctly?

Test that the CLI can authenticate with AWS by running this command:

aws sts get-caller-identity

You should see JSON output containing three fields: your Account ID, your User ARN, and your User ID. This confirms the CLI is installed, configured, and can talk to AWS.

What should I see?

The output looks something like this (your values will be different):

{
    "UserId": "AIDAEXAMPLEID",
    "Account": "123456789012",
    "Arn": "arn:aws:iam::123456789012:user/your-username"
}

If you see this structure with your own account details, your CLI is ready to go.

🙋‍♀️ Seeing an error instead of your account info?

If you see a message about invalid credentials or an inability to locate credentials, your Access Key ID or Secret Access Key may be incorrect. Run aws configure again and carefully re-paste both values from your downloaded CSV file.

If you see a network or timeout error, check that your internet connection is active and that no firewall is blocking outbound HTTPS traffic.

help me troubleshoot aws sts get-caller-identity errors

Your CLI is installed, configured, and authenticated. Next up, you will create an IAM role and deploy your first Lambda function entirely from this terminal.

Create the IAM Role and Deploy a Basic Lambda Function

Your AWS CLI is authenticated and ready to go. Now it's time to deploy your first piece of cloud infrastructure: a Lambda function.

Every Lambda function needs an IAM execution role before it can run. This role tells AWS which services Lambda is allowed to interact with. You'll create the role first, then write and deploy a simple function to prove the whole deployment pipeline works from your terminal.

In this step, get ready to:

Create an IAM execution role that Lambda can assume.
Write a basic Lambda function and package it for deployment.
Deploy the function and invoke it to confirm it runs in the cloud.

Create the IAM execution role

IAM roles define what permissions a service has. Lambda needs a trust policy that explicitly grants lambda.amazonaws.com permission to assume the role.

Switch back to your terminal and move to your Desktop by running this command:

cd ~/Desktop

Create a project folder and move into it by running these commands:

mkdir serverless-pipeline
cd serverless-pipeline

Create the trust policy file by running this command:

cat > trust-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Action": "sts:AssumeRole"
    }
  ]
}
EOF

What does this trust policy do?

A trust policy is a JSON document that defines which AWS service is allowed to "assume" (use) this role. This policy grants that permission exclusively to Lambda.

Without this trust policy, Lambda would be unable to take on the role's permissions, and your function would fail to execute.

Confirm the file was created by running:

ls

You should see trust-policy.json listed in the output.

Create the IAM role using your trust policy by running this command:

aws iam create-role --role-name lambda-file-processor-role --assume-role-policy-document file://trust-policy.json

What does this command do?

--role-name lambda-file-processor-role gives the role a descriptive name you'll reference in later commands.
--assume-role-policy-document file://trust-policy.json attaches your trust policy so Lambda can assume this role.

You should see JSON output containing the role details, including an Arn field. Copy this ARN value. You'll need it when deploying the Lambda function.

Seeing an access denied error?

Make sure the IAM user you configured in Step 1 has permission to create roles. The user needs the IAMFullAccess managed policy or equivalent permissions.
Check that trust-policy.json is in your current directory. Run ls to confirm.

Why is my create-role command failing?

Attach the AWSLambdaBasicExecutionRole managed policy to your role by running:

aws iam attach-role-policy --role-name lambda-file-processor-role --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

What does this policy grant?

AWSLambdaBasicExecutionRole is a managed policy that grants Lambda permission to write logs to CloudWatch. Without this, your function would run but you'd have no way to see its log output.

This command produces no output on success. That's normal.

Wait 10 seconds for IAM to propagate the role across AWS. Run this command to pause:

sleep 10

Why wait 10 seconds?

IAM changes are eventually consistent across AWS regions. If you try to use a brand-new role immediately, Lambda might not recognize it yet. A short pause avoids a confusing "role not found" error.

Write and package the Lambda function

With the role ready, you need a function to deploy. This is a minimal Python handler that returns a success message to prove deployment works.

Create your Lambda function file by running this command:

cat > lambda_function.py << 'EOF'
import json

def lambda_handler(event, context):
    return {"statusCode": 200, "body": json.dumps({"message": "Hello from file-processor!"})}
EOF

What does this function do?

lambda_handler is the entry point AWS Lambda calls when your function is triggered.
It returns a JSON response with a 200 status code and a simple message. This proves your function executes successfully.

Package the function into a zip file for deployment by running:

zip function.zip lambda_function.py

Confirm both files exist by running:

ls

You should see lambda_function.py, function.zip, and trust-policy.json in the output.

Missing function.zip?

Make sure lambda_function.py exists in your current directory before running zip.
If zip is not found, install it by running sudo apt install zip -y.

Why isn't my zip file being created?

Deploy and invoke the function

Your code is packaged and your role is ready. Time to deploy the function to AWS and prove it runs.

Deploy your Lambda function by running this command. Replace the --role value with the ARN you copied from the create-role output:

aws lambda create-function \
  --function-name file-processor \
  --runtime python3.14 \
  --handler lambda_function.lambda_handler \
  --role [[ROLE_ARN="arn:aws:iam::123456789012:role/lambda-file-processor-role"]] \
  --zip-file fileb://function.zip

What do these flags mean?

--function-name file-processor names your function in AWS.
--runtime python3.14 tells Lambda which language runtime to use.
--handler lambda_function.lambda_handler tells Lambda the file name and function name to call. The format is filename.function_name.
--zip-file fileb://function.zip uploads your local zip package. The fileb:// prefix tells the CLI to read the file as binary.

You should see JSON output with your function's details, including its ARN and runtime. This confirms the function was deployed successfully.

Seeing a role not found error?

Make sure you waited at least 10 seconds after creating the role. IAM needs time to propagate.
Double-check that the --role value is the full ARN (starts with arn:aws:iam::), not just the role name.
If you still see errors, run sleep 10 and try the command again.

Why can't Lambda find my IAM role?

Before you run the next command, what do you think will happen when you invoke this function? What status code do you expect to see?

Invoke your deployed function by running:

aws lambda invoke --function-name file-processor out.json

You should see output containing StatusCode 200. This confirms the invocation request succeeded.

Check your function's actual response by reading the output file:

cat out.json

You should see the JSON response: {"statusCode": 200, "body": "{\"message\": \"Hello from file-processor!\"}"}. Your code is running in the cloud, deployed entirely from your terminal.

Seeing an error in out.json?

If out.json contains an error message, check that your lambda_function.py matches the code above exactly. A typo in the function name or return structure causes runtime failures.
If the command itself fails with "function not found", confirm the function name is file-processor (with a hyphen, not an underscore).

Why is my Lambda invoke returning an error?

Your Lambda function is live and responding. Next up, you'll give it a real job: processing S3 file uploads and logging metadata to DynamoDB.

Add DynamoDB and Upgrade the Lambda

Your Lambda function is live, but right now it just says hello. It does not actually process anything.

In this step, you give the function a real job. You will create a DynamoDB table to store file metadata, grant your Lambda role permission to write to it, and upgrade the function code to parse S3 event records. When you invoke the function with a test event, data appears in the table.

In this step, get ready to:

Create a DynamoDB table and grant your Lambda role write access.
Rewrite the Lambda function to process S3 events and log metadata to DynamoDB.
Test the updated function with a sample S3 event payload.

Create the DynamoDB table and attach permissions

Your Lambda function needs somewhere to write data. A DynamoDB table gives you a fast, serverless data store that pairs naturally with Lambda.

Create the file-metadata table by running this command:

aws dynamodb create-table \
    --table-name file-metadata \
    --attribute-definitions AttributeName=id,AttributeType=S \
    --key-schema AttributeName=id,KeyType=HASH \
    --billing-mode PAY_PER_REQUEST

What does this command do?

--table-name file-metadata names the table.
--attribute-definitions defines a single attribute id of type String (S).
--key-schema makes id the partition key (HASH).
--billing-mode PAY_PER_REQUEST means you pay only when you read or write. No capacity planning needed.

You should see a JSON response containing "TableStatus": "CREATING". The table is ready within a few seconds.

Now you need to let your Lambda role write to this table. You will create a policy JSON file and attach it as an inline policy.

Create a file called dynamodb-policy.json in your project folder. Open it in your text editor and paste the following content:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:PutItem",
        "dynamodb:GetItem",
        "dynamodb:UpdateItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:*:table/file-metadata"
    }
  ]
}

What does this policy allow?

This grants the Lambda function permission to put, get, and update items in the file-metadata table. The wildcard in the account position means it works regardless of your specific account ID.

Save dynamodb-policy.json, then attach it to your Lambda role by running:

aws iam put-role-policy \
    --role-name lambda-file-processor-role \
    --policy-name dynamodb-write-policy \
    --policy-document file://dynamodb-policy.json

This command produces no output on success. If you see no error, the policy is attached.

Seeing an error about the file path?

Make sure your terminal is in the same folder as dynamodb-policy.json. Run ls to confirm the file is listed.
Check that the file prefix is file:// (two forward slashes), not fileb://.

Help me troubleshoot the put-role-policy command.

Upgrade the Lambda function code

Your current function just returns a hello message. The upgraded version parses incoming S3 event records and writes file metadata to DynamoDB.

Open lambda_function.py in your text editor. Delete all the existing code and replace it with:

import json
import boto3
import os
from datetime import datetime, timezone

# Connect to the DynamoDB table using the TABLE_NAME environment variable
dynamodb = boto3.resource("dynamodb")
table = dynamodb.Table(os.environ["TABLE_NAME"])

What does this code do?

boto3.resource("dynamodb") creates a connection to the DynamoDB service.
os.environ["TABLE_NAME"] reads the table name from an environment variable, keeping the function flexible.

Below the code you just pasted, add the handler function:

def lambda_handler(event, context):
    print("Received event:", json.dumps(event))

    for record in event["Records"]:
        bucket = record["s3"]["bucket"]["name"]
        key = record["s3"]["object"]["key"]
        size = record["s3"]["object"]["size"]
        event_time = record["eventTime"]

        item = {
            "id": f"{bucket}/{key}/{event_time}",
            "bucket": bucket,
            "key": key,
            "size": size,
            "event_time": event_time,
            "processed_at": datetime.now(timezone.utc).isoformat(),
        }

        table.put_item(Item=item)
        print(f"Stored metadata for {key} ({size} bytes)")

    return {"statusCode": 200, "body": json.dumps({"message": "Processed successfully"})}

What does the handler do?

It loops through each record in the incoming S3 event.
For each file, it extracts the bucket name, object key, size, and event time.
It builds a unique id from the bucket, key, and timestamp, then writes the item to DynamoDB.
It prints a log line so you can trace execution in CloudWatch later.

Save lambda_function.py.

✔️ Awesome, I've got everything!

Great. Double check you have saved the file before moving on.

ⓧ I'd like to double check the full code

Your lambda_function.py should contain these two sections in order:

The imports and table connection (import json through table = dynamodb.Table(os.environ["TABLE_NAME"])).
The lambda_handler function that loops through records and writes to DynamoDB.

Scroll up and confirm each block is present, in order.

Now zip and deploy the updated code, then set the environment variable so the function knows which table to use.

Create the deployment package and push the new code by running:

zip -f function.zip lambda_function.py

Deploy the updated code to Lambda by running:

aws lambda update-function-code \
    --function-name file-processor \
    --zip-file fileb://function.zip

You should see a JSON response with "LastUpdateStatus": "Successful" or "InProgress". Wait a few seconds for the update to complete.
Set the TABLE_NAME environment variable by running:

aws lambda update-function-configuration `
    --function-name file-processor `
    --environment "Variables={TABLE_NAME=file-metadata}"

You should see a JSON response that includes "Environment": { "Variables": { "TABLE_NAME": "file-metadata" } } confirming the variable is set.

Seeing a conflict error on update-function-configuration?

If you get a ResourceConflictException, the previous code update is still in progress. Wait 5-10 seconds and try the command again.

Help me fix a conflict error when updating Lambda configuration.

Test with a sample S3 event

Your function now expects an S3 event, not a generic test. You need to craft a test payload that mimics what S3 sends when a file is uploaded.

Create a file called test-event.json in your project folder. Open it and paste:

{
  "Records": [
    {
      "eventVersion": "2.1",
      "eventSource": "aws:s3",
      "awsRegion": "us-east-1",
      "eventTime": "2026-08-04T12:00:00.000Z",
      "eventName": "ObjectCreated:Put",
      "s3": {
        "bucket": {
          "name": "your-unique-bucket-name"
        },
        "object": {
          "key": "test-file.txt",
          "size": 1024
        }
      }
    }
  ]
}

What is this file?

This is a mock S3 event. It tells your function that a file called test-file.txt (1024 bytes) was uploaded to a bucket. The bucket name can be anything here because this is a test payload.

Save test-event.json.

Before you invoke, what do you think will happen? Will the function succeed and write a record to DynamoDB, or will something go wrong?

Invoke the function with the test event by running:

aws lambda invoke `
    --function-name file-processor `
    --payload fileb://test-event.json `
    out.json

You should see StatusCode: 200 in the terminal output.
Check the function's response by running:

Get-Content out.json

You should see {"statusCode": 200, "body": "{\"message\": \"Processed successfully\"}"}}.

That confirms the function ran without errors. Now verify the data actually landed in DynamoDB.

Scan the DynamoDB table by running:

aws dynamodb scan --table-name file-metadata

You should see a JSON response containing your test record with fields like key, size, event_time, and processed_at.

Seeing an empty Items array?

Check that out.json shows StatusCode 200 and the "Processed successfully" message. If it shows an error, your function code may have a typo.
Confirm the TABLE_NAME environment variable is set by running aws lambda get-function-configuration --function-name file-processor and checking the Environment section.
Make sure you waited at least 10 seconds after attaching the IAM policy before invoking. IAM propagation can cause early failures.

My DynamoDB scan is returning empty results after invoking my Lambda.

Your Lambda now has a real job. It parses S3 events and persists metadata to DynamoDB. Next up, you will wire S3 to trigger the function automatically whenever a file is uploaded.

Wire Up the S3 Trigger and Test End-to-End

In the previous step, you proved your Lambda function can process S3 event data and write to DynamoDB. But you invoked it manually with a test payload.

In a real pipeline, the function should fire automatically the moment a file lands in S3. This step wires that connection. You will also hit a real-world permissions error along the way, because S3 cannot invoke your function until you explicitly grant it permission.

In this step, get ready to:

Create an S3 bucket and wire it to trigger your Lambda function on every upload.
Upload a test file and verify the metadata record appears in DynamoDB automatically.
Inspect CloudWatch logs from the terminal to confirm Lambda executed successfully.

Create the S3 bucket and notification configuration

Your pipeline needs somewhere to receive file uploads. You will create an S3 bucket with a globally unique name, then prepare the notification configuration file that tells S3 which Lambda function to call.

Create your S3 bucket by running this command (replace the bucket name with something unique to you, like your initials and today's date):

aws s3 mb s3://[[BUCKET_NAME="your-unique-bucket-name-e.g.-jd-file-pipeline-20260804"]]

You should see a confirmation message like make_bucket: s3://your-bucket-name.

Why does my bucket name need to be unique?

S3 bucket names are globally unique across all AWS accounts. No two buckets anywhere in the world can share the same name. Using your initials plus the date makes collisions unlikely.

Create the notification configuration file by opening your text editor and saving the following as notification.json:

{
  "LambdaFunctionConfigurations": [
    {
      "LambdaFunctionArn": "arn:aws:lambda:us-east-1:YOUR_ACCOUNT_ID:function:file-processor",
      "Events": ["s3:ObjectCreated:*"]
    }
  ]
}

What does this configuration do?

This JSON tells S3 to invoke the file-processor Lambda function whenever any object is created in the bucket. The s3:ObjectCreated:* event type covers Put, Post, Copy, and multipart uploads.

You need to replace YOUR_ACCOUNT_ID in notification.json with your actual AWS account ID. Get it by running:

aws sts get-caller-identity --query Account --output text

Copy the 12-digit account ID from the output and paste it into notification.json in place of YOUR_ACCOUNT_ID.
Save notification.json.

Apply the notification configuration (and fix the permissions error)

With the bucket created and the configuration file ready, the next logical step is to apply the notification. But there is a catch that every cloud engineer hits the first time.

Try applying the notification configuration by running:

aws s3api put-bucket-notification-configuration --bucket [[BUCKET_NAME="your-unique-bucket-name"]] --notification-configuration file://notification.json

Before you run this, what do you think will happen? Will S3 accept the configuration, or will something go wrong?

You will see an error. S3 checks whether it has permission to invoke your Lambda function before it accepts the notification configuration. Since you have not granted that permission yet, the command fails.

Seeing this error? That is expected.

The error occurs because S3 verifies Lambda function permissions before applying the notification configuration. You need to explicitly grant S3 permission to invoke your function. This is a real-world IAM lesson: services do not automatically trust each other, even within your own account.

Grant S3 permission to invoke your Lambda function by running:

aws lambda add-permission --function-name file-processor --statement-id s3-invoke --action lambda:InvokeFunction --principal s3.amazonaws.com --source-arn arn:aws:s3:::[[BUCKET_NAME="your-unique-bucket-name"]]

What does this command do?

This adds a resource-based policy to your Lambda function that says: "The S3 service is allowed to invoke this function, but only from this specific bucket." The --source-arn flag restricts which bucket can trigger the function, preventing other accounts from invoking it.

You should see a JSON response containing the new policy statement with s3.amazonaws.com as the principal.

Now re-run the notification configuration command:

aws s3api put-bucket-notification-configuration --bucket [[BUCKET_NAME="your-unique-bucket-name"]] --notification-configuration file://notification.json

This time it succeeds silently (no output means success). Your S3 bucket is now wired to trigger Lambda on every upload.

Still seeing a permissions error?

Make sure the bucket name in your --source-arn flag matches the exact bucket name you created. Also confirm that notification.json contains your real 12-digit account ID, not the placeholder text.

help me fix the S3 notification permission error

Upload a test file and verify the pipeline end-to-end

The moment of truth. You will upload a real file to S3 and check whether the entire pipeline fires automatically: S3 triggers Lambda, Lambda processes the event, and a record appears in DynamoDB.

Create a small test file and upload it to your bucket by running:

echo "Hello from the pipeline" > testfile.txt
aws s3 cp testfile.txt s3://[[BUCKET_NAME="your-unique-bucket-name"]]/testfile.txt

You should see a line like upload: .\testfile.txt to s3://your-bucket/testfile.txt.

Wait about 5 seconds for Lambda to process the event, then scan DynamoDB to see the new record:

aws dynamodb scan --table-name file-metadata

Before you look at the output, what do you expect to see? If the pipeline worked, you should see a new record with your bucket name, the key testfile.txt, the file size, and a timestamp.

You should see at least two records in the output now: the test record from the previous step's manual invocation, and a new record with your real bucket name and testfile.txt as the key.

What should I see in the DynamoDB scan?

Each record has an id field in the format bucket/key/eventTime. Look for the entry where key is testfile.txt. The size field shows the file's byte count, and processed_at shows when Lambda ran.

Now check the CloudWatch logs to see what Lambda printed during execution. First, list the log streams:

aws logs describe-log-streams --log-group-name /aws/lambda/file-processor --order-by LastEventTime --descending --max-items 1

Copy the logStreamName value from the output (it looks like 2026/08/04/[$LATEST]abc123...). Then fetch the log events:

aws logs get-log-events --log-group-name /aws/lambda/file-processor --log-stream-name [[LOG_STREAM="paste-your-log-stream-name-here"]] --start-from-head

You should see log messages including Received event: followed by the S3 event JSON, and Stored metadata for testfile.txt confirming the function processed your file.

Not seeing a DynamoDB record for your uploaded file?

Wait 10-15 seconds and scan again. Lambda cold starts can add a few seconds of delay. If the record still does not appear, check that your notification.json has the correct account ID and that the bucket name matches everywhere.

help me debug why my S3 upload did not trigger Lambda

Your serverless pipeline is fully operational. A file dropped into S3 automatically triggers Lambda, which logs the event and writes metadata to DynamoDB. Next up, you will learn how to cleanly tear down every resource in the correct dependency order.

Clean Up All Resources

Your pipeline is working end-to-end. A file upload triggers Lambda, metadata lands in DynamoDB, and you proved it all from the terminal.

Leaving cloud resources running after you finish a project is how surprise AWS bills happen. Cloud engineers always clean up. In this step, you will systematically tear down every resource in the correct dependency order.

In this step, get ready to:

Remove the S3 trigger and delete the bucket.
Delete the Lambda function, DynamoDB table, and CloudWatch log group.
Detach policies and delete the IAM role.

Remove the S3 notification and delete the bucket

The S3 bucket currently has a notification configuration that triggers Lambda on every upload. You need to disable that trigger first, then remove all objects, then delete the bucket itself.

Clear the notification configuration from your bucket by running this command:

aws s3api put-bucket-notification-configuration --bucket [[BUCKET_NAME="your-unique-bucket-name"]] --notification-configuration '{}'

What does this command do?

Passing an empty JSON object '{}' as the notification configuration replaces the current Lambda trigger with nothing. This effectively disables all event notifications on the bucket.

Delete all objects from the bucket, then delete the bucket itself by running these commands:

aws s3 rm s3://[[BUCKET_NAME="your-unique-bucket-name"]] --recursive
aws s3 rb s3://[[BUCKET_NAME="your-unique-bucket-name"]]

What do these commands do?

aws s3 rm --recursive deletes every object inside the bucket. S3 requires a bucket to be empty before you can delete it.
aws s3 rb removes the now-empty bucket permanently.

Confirm the bucket is gone by running:

aws s3 ls

Your bucket name should no longer appear in the list.

Still seeing your bucket in the list?

Make sure you used the correct bucket name in the previous commands. Check for typos by comparing against the name you used throughout the project.

If you see a "BucketNotEmpty" error, run the aws s3 rm --recursive command again before trying aws s3 rb.

Help me delete my S3 bucket from the CLI.

Delete the Lambda function, DynamoDB table, and log group

With the S3 trigger removed, nothing will invoke the Lambda function anymore. Now you can safely delete the compute and storage resources.

Delete the Lambda function by running:

aws lambda delete-function --function-name file-processor

Delete the DynamoDB table by running:

aws dynamodb delete-table --table-name file-metadata

You should see a JSON response showing the table's description with a TableStatus of DELETING.

Delete the CloudWatch log group by running:

aws logs delete-log-group --log-group-name /aws/lambda/file-processor

Why delete the log group?

Lambda automatically creates a CloudWatch log group the first time the function runs. It persists even after you delete the function. Cleaning it up removes the last trace of your function's execution history.

🙋‍♀️ Seeing a ResourceNotFoundException?

If you see this error for the log group, it means your Lambda function may never have been invoked (so no log group was created). This is safe to ignore.

Help me troubleshoot deleting my Lambda resources.

Detach policies and delete the IAM role

IAM roles cannot be deleted while policies are still attached. You need to remove both the inline policy and the managed policy first.

Find the name of the inline policy attached to your role by running:

aws iam list-role-policies --role-name lambda-file-processor-role

You should see a PolicyNames array listing your inline policy name. Note this name for the next command.

Delete the inline policy from the role by running this command, replacing the policy name with what you saw in the previous output:

aws iam delete-role-policy --role-name lambda-file-processor-role --policy-name [[INLINE_POLICY_NAME="your-inline-policy-name from the list above"]]

Detach the managed execution policy by running:

aws iam detach-role-policy --role-name lambda-file-processor-role --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

Delete the IAM role itself by running:

aws iam delete-role --role-name lambda-file-processor-role

Why this specific order?

AWS requires you to remove all attached policies before deleting a role. If you try to delete the role first, the CLI returns an error saying the role has attached policies. This dependency-aware teardown order is the same pattern you would use in production scripts.

🙋‍♀️ Getting a DeleteConflict error?

This means a policy is still attached to the role. Run aws iam list-role-policies --role-name lambda-file-processor-role to check for remaining inline policies.

Also run aws iam list-attached-role-policies --role-name lambda-file-processor-role to check for remaining managed policies. Remove any you find before retrying the delete.

Help me delete my IAM role.

Before you check the result, predict: what do you expect to see when you try to get details on a deleted Lambda function?

Verify that the Lambda function is gone by running:

aws lambda get-function --function-name file-processor

You should see a ResourceNotFoundException error. This confirms the function no longer exists. Every resource from this project has been removed.

Secret mission

Add Prefix Filtering to the S3 Trigger

In production, Lambda functions rarely fire on every single upload to a bucket. Can you modify the notification configuration to only trigger on files uploaded to a specific folder?

Clean Up Your Resources

This project runs entirely within the AWS Always Free tier, so there are no ongoing costs for idle resources. Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

IAM role (lambda-file-processor-role) with attached policies.
Lambda function (file-processor).
S3 bucket (your uniquely named bucket).
DynamoDB table (file-metadata).
CloudWatch log group (/aws/lambda/file-processor).
Local project files (trust-policy.json, lambda_function.py, dynamodb-policy.json, notification.json, notification-filtered.json, test-event.json).
AWS CLI v2 installation and credentials configuration.

Keep everything running

No action needed. Choose this if you plan to continue experimenting with your serverless pipeline or want the AWS CLI available for future projects.

If you already cleaned up your AWS resources in Step 5 or the Secret Mission, there is nothing running in your account. You can re-deploy at any time using the same commands.
Your local project files remain on your Desktop for reference.
The AWS CLI stays installed and configured on your machine, ready for your next project.

Pause - I'll come back to this later

Shut down running processes to free up resources while keeping your files for later.

If you completed Step 5, your AWS resources are already deleted. No further cloud cleanup is needed.
Keep your local project files on the Desktop so you can re-deploy later without rewriting them.
Your AWS CLI configuration remains in place so you can run commands immediately next time.

Delete - I don't want to use this again

Remove all project resources and start fresh. Follow these steps to clean up everything.

If you already ran the cleanup commands in Step 5, your AWS resources are gone. Confirm by running:

aws lambda get-function --function-name file-processor

You should see a ResourceNotFoundException error, confirming the function no longer exists.

If the function still exists, re-run the full cleanup sequence from Step 5:

Remove the S3 bucket notification, delete all objects, then delete the bucket.
Delete the Lambda function, the DynamoDB table, and the CloudWatch log group.
Detach all policies from the IAM role, then delete the role.

Delete your local project files by running:

Remove-Item -Recurse -Force trust-policy.json, lambda_function.py, dynamodb-policy.json, notification.json, notification-filtered.json, test-event.json, myFunction.zip

To uninstall the AWS CLI v2, open Programs and Features by running:

appwiz.cpl

Select AWS Command Line Interface and choose Uninstall.
Confirm that you want to uninstall.

Removing your AWS credentials

Uninstalling the CLI does not remove your stored credentials. To delete them, remove the .aws folder in your user profile. This folder stores credentials shared across all AWS SDKs, so only delete it if you have no other AWS tools on your machine.

Remove-Item -Recurse -Force $env:USERPROFILE\.aws

That's a Wrap!

Nice work! You just deployed, tested, and tore down a complete serverless pipeline entirely from your terminal. That is exactly how cloud engineers operate AWS infrastructure in production.

You've learned how to:

Deploy a Python Lambda function from a local zip file and invoke it from the terminal, without ever opening the AWS Console.
Wire S3 event notifications to Lambda, creating a fully automated event-driven pipeline that processes file uploads in real time and writes metadata to DynamoDB.
Cleaned up all resources in the correct dependency order, proving the operational discipline that interviewers look for in cloud engineering candidates.
Secret Mission: Scoped the S3 trigger to only fire for files uploaded to a specific prefix, mirroring how production teams isolate processing pipelines on shared buckets.

Ready to quiz yourself?