Design a 10-Million-Object Store

Design a resilient object store and prototype S3-to-DynamoDB metadata indexing.

Introduction

30 Second Summary

A storage design can sound convincing until someone asks where ten million objects fit or what happens when a whole location fails. Without concrete answers, the design is only a sketch.

In this project, you will design a distributed object store for 10 million objects. You will prove one core path with a small AWS prototype that sends uploads from Amazon S3 through AWS Lambda into Amazon DynamoDB.

What You'll Build

Your final demo walks an interviewer from capacity estimates to a failure-ready multi-Region architecture before a real upload appears as searchable metadata in a separate table.

By the end of this project, you'll have:

A capacity model you can defend with explicit assumptions for storage, metadata, replication, and peak traffic across 10 million objects.
An architecture canvas that lets you trace PUT, GET, and LIST requests across partitioned metadata, replicated object data, background repair, and regional failover.
A live metadata-index prototype where uploading hello.txt creates a queryable DynamoDB item containing its bucket, key, byte size, event time, and sequencer.
Secret Mission: Make both Regions writable with immutable object versions plus a deterministic winner rule for concurrent writes.

Are there any prerequisites?

You need basic system design knowledge plus an AWS account with permission to use AWS CloudShell, AWS CloudFormation, Amazon S3, AWS Lambda, Amazon DynamoDB, and IAM. The browser-based setup needs no local installation on your Mac.

Before We Start

Before any hands-on work, commit to designing a 10-million-object store with reliability claims you can defend.

Set Up the Design and AWS Workspaces

A storage design is only defensible when every assumption can be traced on a shared canvas. The same discipline applies to the AWS workspace that later proves one slice of the design.

This step sets up a browser-based diagrams.net canvas. It also prepares AWS CloudShell for the prototype without creating any AWS project resources.

In this step, get ready to:

Open a blank architecture canvas with room for each design artifact.
Launch AWS CloudShell in us-east-1.
Verify your AWS identity plus the preinstalled deployment tools.

Create the architecture canvas

Your canvas will hold the reasoning behind the entire storage system. Keeping each future artifact in its own area makes the design easier to explain.

Paste https://app.diagrams.net/?libs=aws4&splash=0 into your browser's address bar.
Press Enter to load diagrams.net.
Create a blank diagram in the browser editor.
Reserve an open area for requirements.
Reserve an open area for capacity calculations.
Reserve an open area for architecture layers.
Reserve an open area for request flows.
Reserve an open area for the failure matrix.

Good start. Your blank canvas now has a clear home for every layer of the interview story.

Open CloudShell in us-east-1

An AWS Region determines where later prototype resources are created. This project uses us-east-1 throughout the deployment.

Open the AWS Management Console in a new browser tab.
Sign in with your AWS account.
Use the region selector in the console header to choose US East (N. Virginia).
Confirm the region selector now shows US East (N. Virginia).
Select the AWS CloudShell icon in the console header.
Wait for the terminal prompt to become available.

CloudShell uses your console session for authentication. That removes local Mac setup from this project.

Select the editor icon in the CloudShell toolbar.

You'll see the CloudShell visual editor pane. This confirms the browser editor is available for the project files you create later.

Switch back to the CloudShell terminal.

That is the browser workspace sorted. The terminal plus visual editor are ready in the correct AWS Region.

Confirm identity and tool access

The last setup check proves which AWS identity CloudShell is using. It also confirms that AWS SAM CLI plus Docker are available in the managed environment.

Before you run the checks, do you expect CloudShell to recognize your identity plus both tools?

Ask CloudShell to print the three workspace checks by running these commands:

aws sts get-caller-identity
sam --version
docker --version

What Do These Checks Prove?

The first command asks AWS for the identity attached to this CloudShell session.
The second command confirms the AWS SAM CLI executable is available.
The third command confirms the Docker executable is available.
Together these checks prove the later build can stay inside the browser-based shell.

The identity output names the active AWS account. The next command prints AWS SAM CLI version information.

The final command prints Docker version information. That clears the environment hurdle for the later prototype.

Missing Identity or Version Output?

Return to the AWS Management Console if the first command identifies an unexpected account.
Sign in with the AWS account you intend to use for this project.
Restart CloudShell from the console header if either version check fails.

If resource creation is denied in a later step, the signed-in identity needs permission to deploy the services listed in the project prerequisites.

Still blocked? Help me troubleshoot my AWS CloudShell identity or tool checks.

Your design canvas plus CloudShell workspace are ready. Next up, you'll test a naive single-node object store against the capacity and failure requirements.

Put the Naive Object Store on Trial

Your design workspace is ready. The blank diagrams.net canvas now becomes the test bench for your 10-million-object system.

A single server can appear to handle every required operation. This step tests that confidence against explicit capacity targets plus realistic failures.

In this step, get ready to:

Define the operations plus reliability targets that the design must satisfy.
Quantify the storage plus throughput created by 10 million objects.
Stress a single-node architecture against those targets.

Define the requirements

Functional requirements describe what clients need the object store to do. The architecture needs these success criteria before its components can be judged.

Double-click a blank area in the requirements section of the canvas.
Select Text from the shape selector.
Type Functional Requirements as the heading.
Add PUT: store new or updated object bytes beneath the heading.
Add GET: retrieve the exact bytes for an object beneath the PUT requirement.
Add DELETE: make an object unavailable beneath the GET requirement.
Add prefix LIST: return object keys that share a prefix beneath the DELETE requirement.

Good. Your canvas now states what the service must do before any architecture earns a pass.

Non-functional targets turn reliability expectations into measurable limits. They give you a consistent way to test designs during an interview.

Add Non-Functional Targets as a second heading in the requirements area.
Record 10,000,000 objects as the object-count target.
Record 1,000 peak reads per second as the read-traffic target.
Record 100 peak writes per second as the write-traffic target.

How Do the Reliability Targets Differ?

An availability target sets how often the service must successfully handle requests.
A durability target sets how reliably stored bytes must survive over time.
A latency target limits how long each operation can take.
A Recovery Point Objective (RPO) limits acceptable data loss after a disruption.
A Recovery Time Objective (RTO) limits how long restoration can take.

Write a numeric annual availability target for successful requests.
Write a durability target for retaining object bytes over time.
Write a latency target for each operation.
Write the maximum acceptable data-loss window as the RPO.
Write the maximum acceptable restoration time as the RTO.

Calculate the workload

A capacity model converts assumptions into quantities the architecture must carry. These calculations expose pressure that a simple box diagram can hide.

Add a text-only shape titled Capacity Assumptions to the calculations area.
Record 10,000,000 objects as the object-count assumption.
Record 1 MiB average object size as the object-size assumption.
Record 1 KiB metadata per object as the metadata-size assumption.
Record 1,000 peak reads per second as the read-rate assumption.
Record 100 peak writes per second as the write-rate assumption.

How Do the Calculations Work?

Object data uses 10,000,000 objects times 1 MiB. Dividing 10,000,000 MiB by 1,024 twice gives about 9.54 TiB.
Metadata uses 10,000,000 records times 1 KiB. Dividing 10,000,000 KiB by 1,024 twice gives about 9.54 GiB.
Read payload multiplies 1,000 reads per second by 1 MiB per object. The result is 1,000 MiB/s.
New write payload multiplies 100 writes per second by 1 MiB per object. The result is 100 MiB/s.

Add Logical object data: approximately 9.54 TiB to the capacity area.
Add Logical metadata: approximately 9.54 GiB beneath the object-data result.
Add Peak read payload: 1,000 MiB/s beneath the metadata result.
Add New write payload: 100 MiB/s beneath the read result.

Those four figures describe the logical workload before extra copies are added. Your design now has numbers that every architecture layer must defend.

Draw and break the naive design

The naive architecture centralizes the request path plus both storage responsibilities. That makes each central component a possible single point of failure.

Move to the architecture area on the diagrams.net canvas from earlier.
Drag four rectangle shapes from the General shape library into a left-to-right row.
Label the first rectangle Client.
Label the second rectangle API Server.
Label the third rectangle Metadata Database.
Label the fourth rectangle Disk Array.

The four boxes contain enough parts to describe every functional requirement. The trial now checks whether that path can survive the workload.

Connect Client to API Server by dragging from a directional arrow on the client shape.
Connect API Server to Metadata Database by dragging from a directional arrow on the server shape.
Connect API Server to Disk Array by dragging from another directional arrow on the server shape.

You should see one client path converging on one API process. That process reaches one metadata database plus one disk array.

Connectors Not Attached?

Hover over the source shape until its directional arrows appear.
Drag the connector again from one directional arrow.
Release the connector when the target shape outline turns blue.

Still stuck? Help me attach shapes with connectors in diagrams.net.

Before you put this design under load, do you think one central path can absorb every request plus every byte?

Trace all 1,000 peak reads per second through API Server.
Trace all 100 peak writes per second through API Server.
Compare 9.54 TiB of logical object data with the one Disk Array.
Compare 9.54 GiB of logical metadata with the one Metadata Database.

Every request converges on the API process. Every lookup converges on the metadata database.

Every object byte converges on the disk array. The design provides no independent path for distributing those pressures.

Trace a PUT while treating API Server as unavailable.
Trace a GET while treating Metadata Database as unavailable.
Trace a GET while treating Disk Array as unavailable.
Trace any operation while treating the entire row's Availability Zone as unavailable.

Each component failure stops a required path. An Availability Zone failure removes the entire service.

Add Scalability bottleneck / single point of failure beneath API Server.
Add Scalability bottleneck / single point of failure beneath Metadata Database.
Add Scalability bottleneck / single point of failure beneath Disk Array.

Before the final review, which component do you expect to hit its limit first under peak read pressure?

Recalculate the four capacity results from the five recorded assumptions.
Point to the component that receives every client request.
Point to the component that owns every metadata lookup.
Point to the component that owns every object byte.
State what remains available after losing the node.
State what remains available after losing the Availability Zone.

You should reproduce about 9.54 TiB of logical object data plus 9.54 GiB of logical metadata. You should also reproduce 1,000 MiB/s of peak read payload plus 100 MiB/s of new write payload.

The API process owns every request path. The metadata database owns every lookup path.

The disk array owns every object-byte path. Losing the node or its Availability Zone leaves no serving path.

You have exposed exactly where the naive object store breaks. Next, you will redesign those weak points using an architecture that can distribute the workload.

Split and Partition the Data and Metadata Planes

The capacity model from the last step exposed a hard limit in the naive design. Every object request still depends on one metadata database.

That database cannot scale or remain balanced as storage nodes join or fail. The next design needs smaller units that can move independently.

In this step, you'll build separate routing, metadata plane, and object-data plane layers. You'll partition point lookups across virtual shards.

Prefix listing will use an ordered index with its own scaling controls.

In this step, get ready to:

Separate request routing from metadata records and object bytes.
Map object identities onto movable virtual shards.
Trace separate PUT, GET, and prefix LIST request flows.

Separate the routing, metadata, and object-data layers

Your canvas shows about 9.54 TiB of object data beside 9.54 GiB of metadata. That difference gives each layer a separate scaling job.

Add a large container to the right of the naive diagram using the shapes in the left sidebar of diagrams.net.
Label the container Separated architecture.

You should now see a dedicated area where the scalable design can sit beside the naive version.

Place a client shape at the left edge of the new container.
Add three router shapes to the right of the client.

The three router shapes make horizontal scaling visible. Traffic has several entry points instead of one API process.

Label the router group Stateless request routers.
Add a Metadata service box to the right of the routers.

The metadata service now has its own boundary. The routers can locate records without owning them.

Add an Object-data service box below the metadata service.
Connect the client to the stateless request-router layer.

You should now see the incoming request path reach a scalable routing layer.

Connect the request-router layer to the metadata service.
Connect the request-router layer to the object-data service.

The diagram now separates small lookup records from large object payloads. Each service can scale around its own workload.

Add a note beside the metadata service that says Object identity = bucket name + object key.
Add a metadata note containing size, checksum, version, replica locations, and state.

Your router layer now holds no object bytes or metadata records. The metadata service records where each object version belongs.

Why Keep the Routers Stateless?

A stateless router can handle any incoming request because it owns no durable record. Failed routers can be replaced without moving object metadata.

New routers can also join behind the same entry point. Routing capacity grows independently from stored data.

Partition point lookups with virtual shards

The full object identity becomes the input to a partition key. Its high cardinality spreads requests more effectively than a field with only a few possible values.

A consistent hashing map assigns each identity to a virtual shard. Physical metadata nodes own groups of those shards.

Add a partitioning band between the stateless routers and the metadata service.
Label the band hash(bucket name + object key).

Every object identity now passes through the same deterministic mapping step.

Draw several virtual-shard segments beneath the hash label.
Draw several physical metadata nodes beneath the virtual shards.

You should see more virtual shards than physical nodes. That difference creates smaller ownership units that can move between nodes.

Connect each virtual shard to exactly one physical metadata node.
Add one new physical metadata node at the edge of the map.

The new node is visible without changing how object identities map to virtual shards.

Redirect only a subset of the virtual-shard ownership lines to the new node.

Only selected ownership lines should move. Existing identities keep their virtual-shard destinations while physical ownership is rebalanced.

Add a warning note that says Avoid low-cardinality partition keys that create hot partitions.
Add design-bucket as the sample bucket beside the hash input.

The warning captures the main partitioning risk. A small set of popular key values can concentrate traffic on too few nodes.

Add the sample object key photos/cat.jpg below the sample bucket.
Add the sample object key photos/dog.jpg below the first key.

The two photo keys share a bucket while retaining distinct full identities.

Add the sample object key reports/q3.pdf below the photo keys.
Trace each sample identity through the hash band to one virtual shard.

Each identity should have one deterministic shard destination. Two identities may share a shard because hashing does not promise a unique shard for every key.

Why Use Virtual Shards?

Directly mapping identities to physical nodes makes node changes disruptive. Virtual shards preserve the identity mapping while ownership moves in smaller pieces.

That extra layer gives operators finer control over load balancing. Rebalancing can target busy sections of the namespace.

Draw the PUT, GET, and LIST flows

Point operations target one exact object identity. Prefix listing must preserve key order across many matching identities.

Separate request lanes make that difference visible. They also expose the extra index work created by listing.

Add a horizontal lane labelled PUT beneath the partition map.
Draw the PUT path from the client through a stateless router to one metadata shard before one object replica set.

The write path should touch one shard for metadata placement. It should also reach one replica set for object bytes.

Add a second horizontal lane labelled GET beneath the write lane.
Draw the GET path from the client through a stateless router to one metadata shard before one object replica set.

The read path should use metadata to locate the object replica set. It never searches every storage node.

Add a third horizontal lane labelled LIST beneath the read lane.
Draw the LIST path from the client through a stateless router to an Ordered bucket-and-prefix listing index.

The listing path now has a dedicated index for sorted key discovery. Object storage stays outside that discovery scan.

Add an index-cost note that says Extra index write for each object change.
Add a hotspot-control note that says Split busy prefix ranges across index partitions.

The canvas now records the tradeoff explicitly. Fast ordered listing requires more writes plus controls for popular prefixes.

Why Does LIST Need Its Own Index?

Hash distribution gives point operations one direct destination. It does not preserve alphabetical or prefix order.

The listing index maintains bucket and prefix order for efficient discovery. Every object change must also update that index.

Before you trace the finished design, which request lane do you expect to avoid the object-data service?

Trace all three sample identities through the PUT lane.
Trace all three sample identities through the GET lane.
Trace the prefix photos/ through the LIST lane.

The PUT trace should touch one metadata shard plus one object replica set. The GET trace should do the same.

The photos/ listing should use the ordered index to return the two photo keys. The report key should remain outside the result.

Strong work. Your design now distributes point lookups while preserving an efficient path for ordered listing.

Flows Difficult to Follow?

Use one colour for metadata paths.
Use a second colour for object-data paths.
Place each request label directly beside its first connector.

Still seeing crossed paths? Help me simplify my PUT, GET, and LIST architecture flows without changing their destinations.

Your architecture can now spread metadata work without losing ordered listing. Next, you'll add independent replicas plus repair so node and Region failures do not erase that progress.

Design for Failure and Repair

Your diagrams.net canvas now spreads metadata traffic across movable virtual shards. Its separate data plane keeps large object bytes away from lookup records.

Scale alone does not protect the service when a replica disappears. A lost Availability Zone or Region can still interrupt requests or erase recent data. In this step, you will add quorum replication plus repair paths that make those tradeoffs explicit.

In this step, get ready to:

Distribute replicas across three Availability Zones under a quorum.
Build the detection-to-repair loop.
Define regional promotion plus failback in a failure matrix.

Distribute replicas and set quorum

A quorum lets the service keep operating after one replica becomes unavailable. Each replica set stores three copies while reads and writes consult two copies.

Return to the architecture area on your diagrams.net canvas from earlier.
Add three wide containers with the rectangle tool in the left toolbar.
Label the containers AZ 1 through AZ 3.
Duplicate the existing request-router group until the diagram contains three router groups.
Move one request-router group into each Availability Zone.
Represent each metadata shard as a three-replica set.
Duplicate the existing object-data replica until the diagram contains three object replicas.
Distribute every replica set so one copy sits in each Availability Zone.

How Quorum Keeps Service Moving

The N=3 setting places three copies across independent zones.
The W=2 setting accepts a write after two copies acknowledge it.
The R=2 setting consults two copies during a read.
The rule R + W > N makes the read set overlap the write set.

Add a quorum note beside each replica set containing N=3 / W=2 / R=2.
Add the inequality R + W > N below the quorum note.

Before you simulate the missing replica, do you expect either request path to lose quorum?

Mark one object replica as unavailable with a red X.
Trace a write through the two remaining replicas.
Trace a read through the two remaining replicas.

That is your first reliability win. One unavailable replica reduces redundancy without stopping the request path.

Add detection and repair paths

A quorum keeps requests moving during a limited fault. Repair mechanisms detect damaged copies and restore the intended replica count.

The repair loop combines health detection, hinted handoff, checksum verification, Merkle-tree anti-entropy comparison, plus background repair.

Add a Health detection box beside each replica set.
Connect each Health detection box to the replicas it monitors.
Add a Hinted handoff queue beside the request routers.
Draw the temporary-write path from Hinted handoff to a recovered replica.
Add Checksum verification to the read path.
Add Merkle-tree anti-entropy comparison between replica copies.
Add a Background repair arrow from healthy copies to a replacement replica.

How the Repair Loop Closes Gaps

Health detection identifies replicas that stop responding.
Hinted handoff holds temporary writes for a replica that is expected to return.
Checksum verification exposes damaged bytes before the service returns them.
Merkle-tree anti-entropy comparison finds differences between large replica sets.
Background repair copies healthy data until the replica set is complete again.

Availability means serving a request now. Durability means retaining the bytes over time.

Add an Availability = serve now note beside the quorum paths.
Add a Durability = retain bytes note beside the repair paths.

The failure matrix is the fiddliest part of this step because every row must separate current service from eventual recovery. It forces every reliability claim to have a visible consequence.

Create a failure matrix in the empty matrix area of your canvas.
Label its columns Failure / Reads / Writes / Risk / Recovery.
Add rows for Disk / Node / Metadata shard / Availability Zone / Region.
Complete the Disk row with Continue / Continue / Reduced redundancy / Checksum plus repair.
Complete the Node row with Continue / Continue / Hinted writes pending / Handoff plus repair.
Complete the Metadata shard row with Continue with two replicas / Continue with two replicas / Unavailable if quorum is lost / Restore replica.
Complete the Availability Zone row with Continue with two zones / Continue with two zones / Reduced fault margin / Rebuild after zone recovery.

The first four rows should preserve reads plus writes while two replicas remain healthy. Each row should also name the repair path that restores full redundancy.

Define regional failover

Global routing directs clients to one write-primary Region. Asynchronous replication copies accepted data to a secondary Region.

Because the copy trails the primary, the Recovery Point Objective (RPO) includes a replication-lag window. The Recovery Time Objective (RTO) includes failure detection plus promotion time.

Add a second large Region container beside the current architecture.
Label the current container Write-primary Region.
Label the new container Secondary Region.
Duplicate the router-to-repair architecture group into the secondary Region.
Add a Global routing component above both Regions.
Draw client paths from global routing to both regional entry points.
Mark only the primary regional path as Writes enabled.
Draw asynchronous replication arrows from the primary data planes to the matching secondary data planes.

What the Regional Tradeoff Means

An active-passive design avoids concurrent cross-Region write conflicts by sending writes to one Region. Promotion moves the write-primary role to the secondary Region.

Promotion cannot recover a write that never reached the secondary copy. Fast service recovery therefore does not make the RPO automatically zero.

Add a promotion arrow that turns the secondary Region into the write-primary after a confirmed regional outage.
Add a failback arrow that returns the write-primary role after replication catches up.
Add an RPO: replication-lag window note beside the replication path.
Add an RTO: detection plus promotion time note beside global routing.
Complete the Region row with May be unavailable until promotion / Paused until promotion / Replication-lag RPO window / Promote secondary plus fail back after catch-up.

Before you walk the completed matrix, do you expect every failure to preserve both reads and writes without stale data?

Walk the Disk row from failure through recovery.
Walk the Node row from failure through recovery.
Walk the Metadata shard row from failure through recovery.
Walk the Availability Zone row from failure through recovery.
Walk the Region row from failure through recovery.

You should be able to show that one disk or one node can disappear while a two-replica quorum remains. The same protection applies to one metadata-shard replica or one Availability Zone.

Region recovery resumes service after global routing promotes the secondary. A write inside the replication-lag window can be absent from that secondary copy.

Your architecture now explains how the service behaves when hardware or an entire failure domain disappears. Next up, you will deploy a thin AWS slice that proves object bytes and metadata can move through separate paths.

Deploy the Metadata Index Prototype

Your canvas now explains how the object store scales across shards. It also shows how the design survives failures across Availability Zones and Regions.

The final design needs a working seam between its object-data plane and metadata plane. This prototype uses AWS SAM to connect Amazon S3, AWS Lambda, and Amazon DynamoDB without building the full distributed storage system.

In this step, get ready to:

Create the infrastructure template and metadata indexer files.
Build and deploy the prototype in us-east-1.
Upload hello.txt and verify its metadata in DynamoDB.

Create the prototype files

The AWS CloudFormation template defines the object bucket, metadata table, and event-driven function. The function code turns each object-created event into a searchable metadata item.

Why AWS SAM and managed services?

A custom EC2 storage cluster would shift your attention to server provisioning and networking. AWS SAM keeps this prototype focused on the architectural seam you designed.

The prototype proves that object bytes and metadata can follow separate paths. Your canvas still owns the custom quorum, repair, and cross-Region design.

Switch back to the CloudShell terminal from earlier.
Open template.yaml in the visual editor by running this command:

edit template.yaml

What does this command do?

The edit command opens template.yaml in the CloudShell visual editor. Saving the editor creates the file in your persistent CloudShell home storage.

Paste the template header and bucket resource into template.yaml using this code:

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Thin-slice metadata index for an S3-like object store

Resources:
  ObjectBucket:
    Type: AWS::S3::Bucket
    Properties:
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true

What does this resource do?

The SAM transform lets CloudFormation expand serverless resources during deployment.
CloudFormation generates a unique physical name because ObjectBucket does not specify a fixed bucket name.
The public access block settings prevent public access through bucket policies or access control lists.

Press Cmd+S to save template.yaml.

The visual editor now shows the saved ObjectBucket resource with all four public access settings enabled.

Paste the metadata table resource below the ObjectBucket resource:

  MetadataTable:
    Type: AWS::DynamoDB::Table
    Properties:
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: object_id
          AttributeType: S
      KeySchema:
        - AttributeName: object_id
          KeyType: HASH

What does this resource do?

The PAY_PER_REQUEST billing mode charges for actual requests without reserving throughput.
The object_id string becomes the table's partition key.
Combining the bucket name with the object key gives the prototype a high-cardinality identity for each object.

Press Cmd+S to save the metadata table resource.

The saved template now shows MetadataTable below ObjectBucket.

Paste the function resource below the MetadataTable resource:

  Function1:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: indexer/
      Handler: app.lambda_handler
      Runtime: python3.13
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 10
      Environment:
        Variables:
          TABLE_NAME: !Ref MetadataTable
      Policies:
        - DynamoDBWritePolicy:
            TableName: !Ref MetadataTable
      Events:
        ObjectCreated:
          Type: S3
          Properties:
            Bucket: !Ref ObjectBucket
            Events: s3:ObjectCreated:*

What does this resource do?

The function runs the app.lambda_handler handler on the Python 3.13 runtime.
The TABLE_NAME environment variable receives the generated DynamoDB table name.
The DynamoDBWritePolicy grants the function permission to write metadata into that table.
The ObjectCreated event invokes the function for object-created events from the generated bucket.

Press Cmd+S to save the function resource.

The saved template now connects ObjectBucket to Function1 through the ObjectCreated event.

Paste the stack outputs below the Resources section:

Outputs:
  ObjectBucketName:
    Description: Generated bucket for prototype object bytes
    Value: !Ref ObjectBucket
  MetadataTableName:
    Description: Generated table for object metadata
    Value: !Ref MetadataTable
  IndexerFunctionName:
    Description: Generated Lambda metadata indexer
    Value: !Ref Function1

Why expose stack outputs?

CloudFormation generates physical names for the bucket, table, and function. These outputs give you a reliable way to find those names after deployment.

Press Cmd+S to save the completed template.yaml file.

The bottom of template.yaml now lists ObjectBucketName, MetadataTableName, and IndexerFunctionName.

Template sections in the wrong place?

Check that ObjectBucket, MetadataTable, and Function1 remain indented beneath Resources. Keep Outputs aligned with Resources.

Still unsure about the YAML structure? Help me compare my SAM template indentation with the required resource structure.

The indexer uses Python to decode the uploaded object key and write its metadata. Boto3 version 1.43.82 provides the DynamoDB client packaged with the function.

Return to the CloudShell terminal.
Open indexer/app.py in the visual editor by running this command:

edit indexer/app.py

What does this command do?

The visual editor creates the indexer path for the function source. The file name matches the app.lambda_handler handler declared in the template.

Paste the imports and DynamoDB client setup into indexer/app.py using this code:

import os
from urllib.parse import unquote_plus

import boto3


dynamodb = boto3.client("dynamodb")
table_name = os.environ["TABLE_NAME"]

What does this setup do?

The URL decoder converts the encoded object key from the S3 event into its readable form.
The DynamoDB client sends metadata writes to the service.
The table_name variable reads the table name supplied through TABLE_NAME.

Press Cmd+S to save the client setup.

The saved file now defines dynamodb and table_name before the event handler.

Paste the event handler below the table_name assignment:

def lambda_handler(event, _context):
    records = event.get("Records", [])
    indexed = 0

    for record in records:
        bucket_name = record["s3"]["bucket"]["name"]
        object_key = unquote_plus(record["s3"]["object"]["key"])
        size_bytes = str(record["s3"]["object"]["size"])
        sequencer = record["s3"]["object"]["sequencer"]
        object_id = f"{bucket_name}#{object_key}"

        dynamodb.put_item(
            TableName=table_name,
            Item={
                "object_id": {"S": object_id},
                "bucket_name": {"S": bucket_name},
                "object_key": {"S": object_key},
                "size_bytes": {"N": size_bytes},
                "event_name": {"S": record["eventName"]},
                "event_time": {"S": record["eventTime"]},
                "sequencer": {"S": sequencer},
            },
        )
        indexed += 1

    return {"indexed": indexed}

How does the indexer work?

The handler reads every record delivered in the S3 event.
The object_id combines the bucket name with the decoded object key.
The DynamoDB item stores the object location, byte size, event details, and sequencer.
The returned indexed count records how many event records the invocation processed.

Press Cmd+S to save the completed indexer/app.py file.

The saved handler now ends by returning the number of indexed records.

Indexer indentation looks uneven?

Keep the statements inside lambda_handler() indented by four spaces. Keep the put_item() call inside the for loop.

Need help checking the handler? Help me compare my Python indentation with the required lambda_handler structure.

Return to the CloudShell terminal.
Open indexer/requirements.txt in the visual editor by running this command:

edit indexer/requirements.txt

What does this command do?

The visual editor opens the dependency file beside the function source. AWS SAM reads this file during the build.

Pin the function dependency by pasting this line into indexer/requirements.txt.

boto3==1.43.82

Why package Boto3?

Packaging a pinned Boto3 release keeps the function's SDK version under your control. The build gathers this dependency from requirements.txt.

Press Cmd+S to save indexer/requirements.txt.

Your dependency file now contains the single Boto3 version pin required by the indexer.

✔️ Awesome, I've got everything!

Your three source files are ready. Double-check that each file is saved before building the application.

ⓧ I'd like to double check the full code

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Thin-slice metadata index for an S3-like object store

Resources:
  ObjectBucket:
    Type: AWS::S3::Bucket
    Properties:
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true

  MetadataTable:
    Type: AWS::DynamoDB::Table
    Properties:
      BillingMode: PAY_PER_REQUEST
      AttributeDefinitions:
        - AttributeName: object_id
          AttributeType: S
      KeySchema:
        - AttributeName: object_id
          KeyType: HASH

  Function1:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: indexer/
      Handler: app.lambda_handler
      Runtime: python3.13
      Architectures:
        - x86_64
      MemorySize: 128
      Timeout: 10
      Environment:
        Variables:
          TABLE_NAME: !Ref MetadataTable
      Policies:
        - DynamoDBWritePolicy:
            TableName: !Ref MetadataTable
      Events:
        ObjectCreated:
          Type: S3
          Properties:
            Bucket: !Ref ObjectBucket
            Events: s3:ObjectCreated:*

Outputs:
  ObjectBucketName:
    Description: Generated bucket for prototype object bytes
    Value: !Ref ObjectBucket
  MetadataTableName:
    Description: Generated table for object metadata
    Value: !Ref MetadataTable
  IndexerFunctionName:
    Description: Generated Lambda metadata indexer
    Value: !Ref Function1

Use this full-file reference to compare every resource, property, and output in template.yaml.

import os
from urllib.parse import unquote_plus

import boto3


dynamodb = boto3.client("dynamodb")
table_name = os.environ["TABLE_NAME"]


def lambda_handler(event, _context):
    records = event.get("Records", [])
    indexed = 0

    for record in records:
        bucket_name = record["s3"]["bucket"]["name"]
        object_key = unquote_plus(record["s3"]["object"]["key"])
        size_bytes = str(record["s3"]["object"]["size"])
        sequencer = record["s3"]["object"]["sequencer"]
        object_id = f"{bucket_name}#{object_key}"

        dynamodb.put_item(
            TableName=table_name,
            Item={
                "object_id": {"S": object_id},
                "bucket_name": {"S": bucket_name},
                "object_key": {"S": object_key},
                "size_bytes": {"N": size_bytes},
                "event_name": {"S": record["eventName"]},
                "event_time": {"S": record["eventTime"]},
                "sequencer": {"S": sequencer},
            },
        )
        indexed += 1

    return {"indexed": indexed}

Use this full-file reference to compare the imports, client setup, and event handler in indexer/app.py.

boto3==1.43.82

Use this reference to confirm the exact dependency pin in indexer/requirements.txt.

Build and deploy the prototype

The container build uses Docker to prepare dependencies for the Lambda runtime. This keeps the build independent of CloudShell's system Python version.

Before you build, ask yourself which three source files AWS SAM needs to discover.

Return to the CloudShell terminal.
Build the serverless application in a Lambda-like container by running:

sam build --use-container

What does this command do?

AWS SAM reads template.yaml and gathers the dependency from indexer/requirements.txt. Docker provides the build environment expected by the Lambda runtime.

The command finishes without a build error. That confirms SAM can resolve the template, function source, and packaged dependency.

Build did not complete?

Check that template.yaml is saved in the current CloudShell location. Confirm that both files exist inside the indexer folder.

If the container cannot start, confirm that Docker still responds in the CloudShell session. Help me diagnose why sam build cannot find my template, source folder, or Docker environment.

The deployment creates usage-based AWS resources in us-east-1. The prescribed test uses one small object and one function invocation, which keeps this experiment small.

The guided deployment asks for configuration before it creates the stack. Use these responses when each prompt appears.

Enter s3-like-object-store when asked for the stack name.
Enter us-east-1 when asked for the Region.
Allow AWS SAM to create the required IAM role.
Keep rollback enabled.
Save the deployment settings to samconfig.toml.
Start the guided deployment by running:

sam deploy --guided

What happens during deployment?

AWS SAM packages the application and submits the generated CloudFormation changes. CloudFormation creates the bucket, table, function, event connection, and IAM execution role.

Saving the guided settings creates samconfig.toml in CloudShell. Later deployments can reuse that configuration.

The deployment finishes with the s3-like-object-store stack in a successful state. Your object-data and metadata planes now have a working event-driven connection.

Deployment was denied?

Confirm that the active AWS identity can create CloudFormation, S3, Lambda, DynamoDB, and IAM resources. Resource creation fails when the signed-in identity lacks one of those permissions.

If CloudFormation rolls back, review the failed resource in the stack events before retrying. Help me understand which permission or resource caused my SAM deployment to fail.

Upload an object and inspect its metadata

CloudFormation generated unique physical names for the three resources. The stack outputs reveal those names without forcing you to search through every service.

Before you list the outputs, picture the three generated resource names you expect the stack to expose.

Display the generated stack outputs by running:

sam list stack-outputs --stack-name s3-like-object-store --output table

What does this command show?

The table displays the values exported by the stack. These values identify the generated bucket, metadata table, and indexer function.

Record the ObjectBucketName value here: your generated bucket name.
Record the MetadataTableName value here: your generated table name.

Create a small test file in CloudShell by running:

printf 'Hello from NextWork.\n' > hello.txt

What does this command do?

The command writes one short line into hello.txt. This gives the upload a small object whose size is easy to inspect.

CloudShell now contains hello.txt beside your SAM project files.

Select Actions in CloudShell.
Choose Download file.
Enter hello.txt in the file path field.
Choose Download.

The file is now available on your Mac for the console upload.

Use the AWS Management Console service search to open Amazon S3.
Select your generated bucket name from the bucket list.
Select the Objects tab.
Choose Upload.
Choose Add files.
Select hello.txt in the file picker.
Choose Open.
Choose Upload.

The upload status confirms that hello.txt reached the generated bucket. That object-created event now invokes Function1.

Use the AWS Management Console service search to open Amazon DynamoDB.
Select Tables.
Select your generated table name.

Before you inspect the table, ask yourself which uploaded object details the event consumer should have recorded.

Choose Explore table items.

What proves the prototype works?

One item has an object_id value ending in #hello.txt.
The bucket_name and object_key attributes identify the uploaded object.
The size_bytes attribute records the object size.
The event_time and sequencer attributes preserve event metadata.

That's the architectural seam proven. An S3 upload now produces an independently queryable DynamoDB metadata item through Lambda.

Metadata item is missing?

Wait briefly for the asynchronous event to reach Lambda. Refresh the table items view after the invocation has time to complete.

Confirm that you uploaded hello.txt to the bucket named by ObjectBucketName. Help me trace why my S3 upload did not create a DynamoDB metadata item.

Secret mission

Make Both Regions Writable

Your recovery design has one write-primary Region. In this mission, you will make both Regions writable while preserving immutable versions through a deterministic conflict policy.

Clean Up Your Resources

Your design canvas can stay as reference material. The deployed AWS prototype uses usage-based services, so decide whether to keep it running, pause practical usage, or delete it entirely.

Cost warning

Amazon S3, AWS Lambda, Amazon DynamoDB, and Amazon CloudWatch are usage-based services. The prescribed test is expected to cost a few cents or less in US East (N. Virginia).

Existing account usage can change your bill. Active-active replication exists only on your canvas, so it creates no cross-Region deployment charges.

Resources you used:

The s3-like-object-store AWS CloudFormation stack.
The generated Amazon S3 ObjectBucket containing hello.txt.
The generated DynamoDB MetadataTable.
The generated Lambda Function1 with its IAM execution role.
The Lambda log group in CloudWatch Logs.
The AWS SAM deployment artifacts in its managed S3 bucket.
The AWS CloudShell files template.yaml, indexer/app.py, indexer/requirements.txt, and samconfig.toml.

Keep everything running

No action needed. Choose this if you want to demonstrate the deployed prototype again or continue refining your storage design.

Retain the s3-like-object-store stack so the upload-to-metadata path remains available.
Keep your completed diagrams.net canvas for future system design practice.
Keep the CloudShell files so you can rebuild or update the prototype.
Allow for continuing storage charges from the deployed resources.

Pause - I'll come back to this later

Remove the test object to stop practical usage while preserving the deployed prototype. The serverless resources remain available, so persistent storage can still incur charges.

Return to Amazon S3 in us-east-1.
Select the generated ObjectBucket from earlier.
Open the Objects tab.
Select hello.txt.
Choose Delete.
Complete the deletion confirmation.
Stop uploading objects to prevent new metadata-indexing events.

You should see an empty Objects list. The stack remains deployed for your next demonstration.

Delete - I don't want to use this again

Remove the deployed AWS resources when you have finished using the prototype. You can also clear the project files from CloudShell home storage.

Deletion is permanent

Deleting the stack removes the live prototype, so save any evidence you want to keep first. Your completed design canvas remains untouched.

Return to Amazon S3 in us-east-1.
Select the generated ObjectBucket from earlier.
Open the Objects tab.
Select hello.txt if it is present.
Choose Delete.
Complete the deletion confirmation.

You should see an empty Objects list. This allows CloudFormation to remove the generated bucket.

The test invocation can leave a Lambda log group after the function is removed. Clear that log group before deleting the stack.

Return to Amazon CloudWatch in us-east-1.
Open the log groups page from the CloudWatch Logs navigation.
Select the log group associated with the generated Lambda function.
Delete the selected log group from its action menu.

You should no longer see that Lambda log group in the log groups list.

AWS SAM asks you to confirm the stack deletion. It also asks before removing the packaged deployment artifacts.

Switch back to AWS CloudShell from earlier.
Delete the stack and its deployment artifacts by running:

sam delete --stack-name s3-like-object-store --region us-east-1

What does this command remove?

AWS SAM deletes the s3-like-object-store CloudFormation stack. It also removes the packaged application artifacts associated with the deployment.

The stack deletion removes the generated S3 bucket, DynamoDB table, Lambda function, and IAM execution role.

Confirm each deletion prompt.
Return to the CloudFormation console in us-east-1 after the command finishes.
Confirm that s3-like-object-store no longer appears in the stack list.

Stack deletion stopped?

If the deletion reports that the bucket still contains objects, return to the generated ObjectBucket. Remove every remaining object before running the AWS SAM deletion command again.

If permissions stop the cleanup, your signed-in AWS identity needs permission to delete the stack resources. Ask for help diagnosing the cleanup failure.

Check the file paths

The next command permanently removes only the project files named below from your CloudShell home directory. Your diagrams.net canvas stays available.

Return to your CloudShell home directory and remove the project files by running:

cd ~
rm -rf template.yaml indexer samconfig.toml

What does this command remove?

The command removes template.yaml, the indexer directory, and samconfig.toml from CloudShell home storage.

Check the remaining CloudShell files by running:

ls

What should I see?

The output should no longer include template.yaml, indexer, or samconfig.toml. Your AWS prototype and its CloudShell source files are now removed.

Nice Work!

That's a substantial build! You turned a fragile single-node sketch into an interview-ready storage design with a live metadata-indexing prototype on AWS.

You've learned how to:

Built a defensible capacity model for 10 million objects from explicit assumptions. Used its results to expose the naive server's bottlenecks.
Designed a partitioned architecture that separates object bytes from metadata. Virtual shards route point lookups. An ordered index supports prefix LIST requests. Quorum replication preserves service during replica loss across Availability Zones. Checksum-based repair restores full health. The Regional recovery plan defines RPO explicitly. It also defines RTO explicitly.
Deployed an event-driven metadata prototype. Amazon S3 uploads invoke AWS Lambda. The function writes queryable object metadata to Amazon DynamoDB. Verified the complete path with hello.txt.
Secret Mission: Built an active-active Regional design with immutable object versions. Defined a deterministic winner rule for concurrent writes.

Ready to quiz yourself?