Secure File Vault: Encrypted Storage

Encrypt S3 files with your own KMS key and share them via pre-signed URLs.

Introduction

30 Second Summary

When you store sensitive files in the cloud, the provider encrypts them by default. But "encrypted" does not mean "under your control."

In this project, you will build a secure file vault on Amazon S3 where every uploaded file is encrypted with a key you own and control. You will upload files, prove they are encrypted with your key, share them through time-limited links, and set up automatic archival rules.

What You'll Build

Picture uploading a confidential document to your vault, immediately proving it is encrypted with your personal key, then generating a download link that works for exactly 60 seconds before self-destructing.

By the end of this project, you'll have:

  • A customer-managed encryption key in AWS KMS that gives you full control over who can encrypt and decrypt your vault's files.
  • Working pre-signed URLs that grant time-limited download access to encrypted files without exposing any credentials.
  • Automated lifecycle rules that move aging files to cheaper storage and delete them on schedule.
  • Secret Mission: A bucket policy that blocks all unencrypted uploads, enforcing encryption at the API level.

Are there any prerequisites?

You will need an AWS account with billing configured and the AWS CLI installed and configured with credentials.

Before We Start

Before we dive in, let's lock in what you're about to build and why it matters. This keeps the bigger picture in focus as you work through each step.

Set Up Your Encryption Key

S3 already encrypts every object you upload using AWS-managed keys. That sounds secure, but it means AWS controls the key lifecycle. You cannot audit who used the key, revoke it independently, or restrict its usage to specific principals.

In this step, you will create your own customer-managed KMS key. This gives you full control over who can encrypt and decrypt your vault's files.

In this step, get ready to:
  • Create a symmetric customer-managed KMS key using the AWS CLI.
  • Verify the key exists and note its Key ID and ARN.
  • Review the default key policy that was created.
Create your customer-managed KMS key

A symmetric encryption key is what Amazon S3 needs for server-side encryption. The aws kms create-key command creates one with sensible defaults: 256-bit AES-GCM, encrypt/decrypt usage, and key material generated by AWS.

  • Create your encryption key by running this command:
aws kms create-key

What does this command do?

With no parameters, create-key creates a symmetric encryption key (SYMMETRIC_DEFAULT spec, ENCRYPT_DECRYPT usage). The key material is generated inside KMS and never leaves the service unencrypted.

Because you are creating this key via the CLI (not the console), KMS applies a simple default key policy that gives your AWS account permission to use IAM policies to control access.

You should see a JSON response containing a KeyMetadata object. It includes your new key's KeyId and Arn.

  • Find the KeyId value in the output (it looks like 1234abcd-12ab-34cd-56ef-1234567890ab).
  • Copy this value and save it somewhere you can easily access, such as a text file or sticky note. You will use it throughout the project.
  • Also note the Arn value (it looks like arn:aws:kms:<region>:<account-id>:key/<key-id>). You will need this in the next step.

Save both values now

You will reference the Key ID and ARN in every remaining step. Paste them into a scratch file so you do not have to re-run commands to find them later.

Verify the key exists and is enabled

Before you trust a new key with your data, confirm it is active and customer-managed.

  • Before you run the next command, what do you expect to see for KeyState and KeyManager?
  • Verify your key by running this command (replace the placeholder with your actual Key ID):
aws kms describe-key --key-id [[KEY_ID="your-key-id-from-the-previous-step"]]

The output confirms two critical facts: "KeyState": "Enabled" means the key is ready for cryptographic operations. "KeyManager": "CUSTOMER" means you own and control this key, not AWS.

What is the difference between CUSTOMER and AWS?

An AWS-managed key (KeyManager: "AWS") is created automatically when you first use a service like S3. You cannot delete it, change its policy, or share it across accounts.

A customer-managed key (KeyManager: "CUSTOMER") gives you full control. You decide who can use it, when to rotate it, and when to delete it.

Not seeing Enabled state?

If KeyState shows something other than Enabled, confirm you are running the command in the same region where you created the key. Your CLI default region must match.

If you receive an error that the key was not found, double-check that you pasted the correct Key ID from the create-key output.

help me debug why my KMS key is not showing as Enabled

Review the default key policy

Every KMS key has exactly one key policy. This policy is the primary access control for the key. Since you created the key via the CLI without specifying a policy, KMS applied a minimal default.

  • View the key policy by running this command:
aws kms get-key-policy --key-id [[KEY_ID="your-key-id"]] --policy-name default --output text

What does this policy do?

The default key policy for a programmatically created key contains a single statement. It grants your AWS account's root principal (arn:aws:iam::<account-id>:root) permission to perform all KMS actions (kms:*) on this key.

This does not mean the root user can instantly use the key. It means your account can now use IAM policies to delegate access to specific users and roles. Without this statement, no IAM policy would have any effect on the key.

You should see a JSON policy document with the Sid value Enable IAM User Permissions. This confirms your key policy is in place and your account can delegate access as needed.

This key costs $1/month

Customer-managed KMS keys cost $1/month (prorated hourly). You will delete this key in the Before You Go section to stop charges. The 20,000 free-tier requests per month cover all the operations in this project.

Your encryption key is live and under your control. Next up, you will create an S3 bucket and wire this key into it so every file you upload is automatically encrypted with your key.

Create Your Encrypted Vault

You have a customer-managed KMS key ready, but your vault does not exist yet. Without configuring a bucket to use your key by default, uploads silently fall back to AWS-managed encryption.

That means you would have built a key but never actually used it. This step wires your key into a new S3 bucket so every file is protected by it automatically.

In this step, get ready to:
  • Create a new S3 bucket for your vault.
  • Configure SSE-KMS as the default encryption with your customer-managed key and S3 Bucket Keys enabled.
  • Upload a test file and confirm encryption headers show your key was used.
Create the S3 bucket

Your vault needs a home. S3 bucket names are globally unique, so you will choose a name that includes your own identifier.

  • Create your vault bucket by running this command:
aws s3api create-bucket --bucket [[BUCKET_NAME="your-unique-bucket-name-e.g.-secure-file-vault-yourname"]]

What does this command do?

This creates a new, empty S3 bucket in the us-east-1 region (the default). If your CLI is configured for a different region, you need to add --region and --create-bucket-configuration LocationConstraint=<your-region> to the command.

Bucket names must be globally unique across all AWS accounts. Adding your name or a random string avoids collisions.

  • You should see a Location field in the output confirming your bucket was created.

Seeing a BucketAlreadyExists error?

Someone else already owns that bucket name. Try a more unique name by appending random numbers or your initials.

If you see an error about LocationConstraint, your CLI is configured for a region other than us-east-1. Add --create-bucket-configuration LocationConstraint=<your-region> to the command.

Help me troubleshoot my S3 bucket creation error

Configure SSE-KMS as default encryption

Right now your bucket uses AWS-managed encryption by default. You need to switch it to use your customer-managed key so every upload is automatically encrypted with the key you control.

  • Set your bucket's default encryption to SSE-KMS with S3 Bucket Keys enabled by running this command:
aws s3api put-bucket-encryption \
    --bucket [[BUCKET_NAME="your-bucket-name"]] \
    --server-side-encryption-configuration '{
        "Rules": [
            {
                "ApplyServerSideEncryptionByDefault": {
                    "SSEAlgorithm": "aws:kms",
                    "KMSMasterKeyID": "[[KMS_KEY_ARN="your-kms-key-arn"]]"
                },
                "BucketKeyEnabled": true
            }
        ]
    }'

What does this command do?

  • SSEAlgorithm set to aws:kms tells S3 to use KMS-based encryption instead of its own managed keys.
  • KMSMasterKeyID points to YOUR customer-managed key ARN from Step 1.
  • BucketKeyEnabled set to true activates S3 Bucket Keys, which reduce KMS request costs by up to 99%.

What are S3 Bucket Keys?

Without Bucket Keys, S3 calls KMS for every single object operation. With Bucket Keys, S3 generates a short-lived bucket-level key from KMS and reuses it to derive data keys for multiple objects.

This dramatically reduces the number of KMS API calls and keeps costs low.

  • Verify the encryption configuration was applied by running:
aws s3api get-bucket-encryption --bucket [[BUCKET_NAME="your-bucket-name"]]
  • You should see output showing SSEAlgorithm as aws:kms, your key ARN under KMSMasterKeyID, and BucketKeyEnabled as true.

Encryption config not showing your key?

Double-check that you pasted the full ARN (starting with arn:aws:kms:) into the KMSMasterKeyID field. A typo in the ARN can cause the command to succeed silently but encrypt with the wrong key.

Help me fix my bucket encryption configuration

Upload a test file and verify encryption

Time to prove the encryption is working. You will upload a simple text file and then inspect its metadata to confirm your KMS key was used.

  • Create a test file locally by running:
echo "This is a confidential vault file." > test-file.txt
  • Upload the test file to your vault by running:
aws s3 cp test-file.txt s3://[[BUCKET_NAME="your-bucket-name"]]/test-file.txt
  • You should see an upload confirmation message.

Now for the real proof. Before you run the next command, what do you think the encryption headers will show? Will they reference the AWS-managed key or your customer-managed key?

  • Inspect the uploaded file's encryption metadata by running:
aws s3api head-object --bucket [[BUCKET_NAME="your-bucket-name"]] --key test-file.txt
  • In the output, confirm that ServerSideEncryption shows aws:kms and SSEKMSKeyId matches your key ARN from Step 1.

What does this prove?

The head-object response includes encryption metadata for the object. Seeing "ServerSideEncryption": "aws:kms" confirms KMS encryption is active.

The SSEKMSKeyId field matching your ARN proves it used YOUR key, not the default AWS-managed one. This is the difference between "encrypted by default" and "encrypted with a key I own and control."

SSEKMSKeyId does not match your key ARN?

Your put-bucket-encryption command may have had a typo in the KMSMasterKeyID value. Re-run the encryption configuration command with the correct ARN, then re-upload the file. Default encryption only applies to new uploads.

Help me fix my object's encryption key mismatch

Your vault is locked down. Every file uploaded to this bucket is now automatically encrypted with a key you own and control. Next up, you will learn how to share those encrypted files securely without making anything public.

Test Secure Access with Pre-signed URLs

Your vault is encrypted and locked down. Every file is protected by your customer-managed KMS key. But what happens when you need to share a file with someone who does not have AWS credentials?

Pre-signed URLs solve this by generating a temporary download link that expires after a set time. In this step, you will prove that direct access is blocked, then create a time-limited URL that grants access for exactly the window you choose.

In this step, get ready to:
  • Confirm that direct URL access to your encrypted file is denied.
  • Generate a pre-signed URL with a 60-second expiration.
  • Verify the URL works within the time window, then fails after expiration.
Attempt to access your file directly

Every S3 object has a direct URL. For a private bucket, requesting this URL without authentication should be denied. You will prove that your vault is truly private by constructing the URL and pasting it into a browser.

  • Construct your object's direct URL by combining your bucket name, region, and file name into this format:
https://[[BUCKET_NAME="your-bucket-name"]].s3.[[AWS_REGION="your-aws-region"]].amazonaws.com/test-file.txt

What is this URL?

This is the virtual-hosted style URL that S3 uses for every object. It includes your bucket name as a subdomain, your region, and the object key (file name). Without authentication credentials attached, S3 treats this as an anonymous request.

Before you paste this URL, what do you think will happen when an unauthenticated request hits your private bucket?

  • Copy the URL you constructed and paste it into your browser's address bar.
  • Press Enter to load the page.

You will see an XML error response containing AccessDenied. This confirms your bucket is private and no one can download files without proper authorization.

Seeing a different error?

If you see a NoSuchBucket error, double-check that your bucket name and region are correct in the URL. The region must match the region where you created the bucket.

If you see a page that says the page could not be loaded, make sure you used https:// at the start of the URL.

Still stuck? Help me troubleshoot why my S3 direct URL is not returning an AccessDenied error as expected.

Generate a pre-signed URL

Now that you have confirmed direct access is denied, you need a way to share this file temporarily. A pre-signed URL embeds your AWS credentials and an expiration timestamp directly into the URL, granting anyone with the link time-limited access.

  • Generate a pre-signed URL for your test file with a 60-second expiration by running this command:
aws s3 presign s3://[[BUCKET_NAME="your-bucket-name"]]/test-file.txt --expires-in 60

What does this command do?

The aws s3 presign command generates a URL signed with your AWS credentials. The --expires-in 60 flag sets the link to expire in 60 seconds.

Anyone with this URL can download the file within that 60-second window. No AWS account required. After expiration, the URL becomes useless.

The command outputs a long URL containing query parameters like X-Amz-Signature and X-Amz-Expires. These encode your credentials and the expiration time.

  • Quickly copy the entire pre-signed URL from your terminal output.
  • Paste it into your browser's address bar and press Enter.

Your file downloads successfully. This is the same file that returned AccessDenied moments ago, now accessible through a signed, time-limited link.

File not downloading?

If you see AccessDenied even with the pre-signed URL, you may have taken more than 60 seconds between generating it and pasting it. Run the aws s3 presign command again and paste the new URL immediately.

If you see a SignatureDoesNotMatch error, make sure you copied the entire URL without cutting off any characters at the end.

Need more help? Help me understand why my S3 pre-signed URL returns an error instead of downloading my file.

Verify the URL expires

The power of pre-signed URLs is that they self-destruct. Once the time window closes, the same URL becomes useless. You will now prove this by waiting for your URL to expire and trying it again.

  • Wait at least 60 seconds from when you generated the pre-signed URL.

Before you paste the expired URL, what do you expect to see this time?

  • Paste the same pre-signed URL back into your browser and press Enter.

You will see an XML error response with AccessDenied or a message indicating the request has expired. The URL no longer works.

Why does this matter?

Pre-signed URLs let you share encrypted files without making anything public and without giving anyone permanent access. The recipient gets a narrow time window, and after that the link is dead.

In production, you would set the expiration to match your use case. The maximum is 604800 seconds (7 days) when using the CLI with IAM user credentials.

Still seeing the file download?

Your browser may have cached the file from your earlier successful download. Try opening the URL in a private or incognito window instead. You can also clear your browser cache and retry.

If you are unsure whether 60 seconds have passed, generate a new pre-signed URL with --expires-in 10 for a shorter test window, wait 10 seconds, then try the expired URL.

Need help? My pre-signed URL still seems to work after it should have expired. Help me figure out why.

Your encrypted vault now has secure, time-limited sharing. Next up, you will automate cost management by configuring lifecycle rules that move old files to cheaper storage and delete them on schedule.

Automate Data Retention with Lifecycle Rules

Your vault files are encrypted and shareable via time-limited links. But right now, every file sits in S3 Standard storage indefinitely, accumulating costs even if no one ever accesses it again.

S3 Lifecycle rules automate cost management by moving aging files to cheaper storage classes and deleting them when their retention period ends. In this step, you will write a lifecycle configuration that transitions files to Glacier after 30 days and expires them after 365 days.

In this step, get ready to:
  • Create a lifecycle configuration JSON file with transition and expiration rules.
  • Apply the lifecycle configuration to your vault bucket.
  • Verify the rules are active via the CLI.
Create the lifecycle configuration file

S3 lifecycle configurations are JSON documents that tell AWS what to do with objects as they age. You will define two rules: one to transition objects to Glacier (cheap archival storage) after 30 days, and one to delete them entirely after 365 days.

  • Create a file called lifecycle.json by running this command:
cat > lifecycle.json << 'EOF'
{
  "Rules": [
    {
      "ID": "TransitionToGlacier",
      "Filter": {
        "Prefix": ""
      },
      "Status": "Enabled",
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "GLACIER"
        }
      ]
    },
    {
      "ID": "ExpireAfterOneYear",
      "Filter": {
        "Prefix": ""
      },
      "Status": "Enabled",
      "Expiration": {
        "Days": 365
      }
    }
  ]
}
EOF

What does this configuration do?

  • Rules is the top-level array that holds all your lifecycle rules.
  • The first rule (TransitionToGlacier) moves ALL objects in the bucket to Glacier storage after 30 days of creation.
  • The second rule (ExpireAfterOneYear) permanently deletes objects after 365 days.
  • Filter with an empty Prefix means the rule applies to every object in the bucket.
  • Status set to Enabled means the rule starts working immediately after you apply it.
  • Confirm the file was created by running:
cat lifecycle.json

You should see the full JSON configuration printed to your terminal, matching the content above.

Don't see the file content?

  • Make sure you copied the entire command including the EOF on its own line at the end.
  • If you see No such file or directory, check that your terminal is in the directory where you ran the cat > command.

Help me troubleshoot creating my lifecycle.json file.

✔️ Awesome, I've got everything!

Great. Make sure the JSON is saved and move on to applying it.

ⓧ I'd like to double check the full code

Your lifecycle.json file should contain exactly this:

{
  "Rules": [
    {
      "ID": "TransitionToGlacier",
      "Filter": {
        "Prefix": ""
      },
      "Status": "Enabled",
      "Transitions": [
        {
          "Days": 30,
          "StorageClass": "GLACIER"
        }
      ]
    },
    {
      "ID": "ExpireAfterOneYear",
      "Filter": {
        "Prefix": ""
      },
      "Status": "Enabled",
      "Expiration": {
        "Days": 365
      }
    }
  ]
}
Apply the lifecycle configuration to your bucket

Now that the configuration file exists locally, you need to push it to your S3 bucket so AWS starts enforcing the rules.

  • Apply the lifecycle configuration by running:
aws s3api put-bucket-lifecycle-configuration \
  --bucket [[BUCKET_NAME="your-bucket-name"]] \
  --lifecycle-configuration file://lifecycle.json

What does this command do?

The put-bucket-lifecycle-configuration subcommand uploads your local JSON file as the bucket's lifecycle policy. The file:// prefix tells the CLI to read the configuration from your local filesystem.

This command produces no output on success. If you see no error message and your terminal returns to the prompt, the configuration was applied successfully.

Seeing an error?

  • If you see MalformedXML or a JSON parsing error, double check your lifecycle.json for missing commas or brackets.
  • If you see NoSuchBucket, confirm you are using the correct bucket name from Step 2.
  • If you see an access denied error, confirm your CLI credentials have s3:PutLifecycleConfiguration permission on the bucket.

Help me fix my put-bucket-lifecycle-configuration error.

Verify the lifecycle rules are active

The apply command gave no output, so you need to query the bucket's configuration to prove the rules took effect.

  • Before you run the next command, predict: what two rules do you expect to see in the output, and what status should each have?
  • Retrieve the lifecycle configuration by running:
aws s3api get-bucket-lifecycle-configuration \
  --bucket [[BUCKET_NAME="your-bucket-name"]]

You should see JSON output containing your two rules. Confirm the output shows:

  • A rule with "ID": "TransitionToGlacier", "Status": "Enabled", and a transition to GLACIER at 30 days.
  • A rule with "ID": "ExpireAfterOneYear", "Status": "Enabled", and expiration at 365 days.

What happens next with these rules?

AWS evaluates lifecycle rules once per day. Objects that have been in the bucket for over 30 days will be queued for transition to Glacier. Objects older than 365 days will be queued for permanent deletion.

Your test-file.txt was just uploaded, so nothing will transition today. But the rules are in place and will activate automatically as your files age.

Not seeing your rules?

  • If you see NoSuchLifecycleConfiguration, the put command did not succeed. Re-run the put-bucket-lifecycle-configuration command from the previous substep and check for errors.
  • If you see rules but with different values, check your lifecycle.json for typos in the Days or StorageClass fields.

Help me verify my S3 lifecycle configuration.

Your vault now has automated data retention. Files are encrypted with your own key, shareable via time-limited links, and automatically archived and deleted on schedule. Next up: a secret mission to make your vault even more secure by blocking unencrypted uploads at the policy level.

Secret mission

Enforce Encryption with a Bucket Policy

Your bucket encrypts by default, but a careless API call could still upload an unencrypted object by explicitly overriding the default. Can you write a bucket policy that blocks ALL unencrypted uploads, then prove it works by testing both a denied and a successful upload?

Clean Up Your Resources

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. The KMS customer-managed key costs approximately $1/month (prorated hourly), while S3 storage for small test files stays within the free tier.

Cost warning

Your customer-managed KMS key costs $1/month, prorated hourly. Disabling the key does NOT stop this charge. To stop all costs, you must schedule the key for deletion (minimum 7-day waiting period) and delete the S3 bucket.

Resources you used:

  • 1 KMS customer-managed symmetric key ($1/month prorated hourly).
  • 1 S3 bucket (secure-file-vault) with default SSE-KMS encryption, lifecycle rules, and a bucket policy.
  • Test files uploaded to the bucket (test-file.txt).
  • Local JSON configuration files (lifecycle.json, bucket-policy.json).

Keep everything running

No action needed. Choose this if you are still actively experimenting with your encrypted vault or plan to build on it soon.

  • Your KMS key remains enabled and costs approximately $1/month.
  • S3 storage for small test files stays within the free tier (5 GB storage, 20,000 GET requests, 2,000 PUT requests per month for 12 months from account creation).
  • Lifecycle rules will automatically transition your objects to Glacier after 30 days and delete them after 365 days.

Pause - I'll come back to this later

Disable the KMS key so your encrypted objects become unreadable until you re-enable it. This prevents accidental usage but does not stop the $1/month charge.

  • Disable your KMS key by running:
aws kms disable-key --key-id your-key-id-here
  • While disabled, any attempts to upload, download, or generate pre-signed URLs for encrypted objects will fail.
  • To resume later, re-enable the key by running:
aws kms enable-key --key-id your-key-id-here

Disabled keys still cost money

A disabled key stays in your account and still costs $1/month. To fully stop charges, you must schedule the key for deletion (see the Delete tab). Scheduling for deletion is irreversible after the waiting period ends.

Delete - I don't want to use this again

Remove all project resources and stop all charges. This is irreversible once the KMS key deletion waiting period completes.

  • Delete all objects in your S3 bucket by running:
aws s3 rm s3://your-bucket-name --recursive
  • Delete the empty S3 bucket by running:
aws s3api delete-bucket --bucket your-bucket-name
  • Schedule your KMS key for deletion by running:
aws kms schedule-key-deletion --key-id your-key-id-here --pending-window-in-days 7

What happens after scheduling deletion?

The key enters a waiting period (7 days minimum, 30 days maximum). During this time the key is disabled and no charges apply. After the waiting period, the key is permanently deleted and cannot be recovered.

If you change your mind during the waiting period, you can cancel deletion with aws kms cancel-key-deletion --key-id your-key-id-here.

  • Delete the local configuration files by running:
rm lifecycle.json bucket-policy.json

Nice Work!

Nice Work!

You just built a production-grade encrypted file vault on AWS. Every file uploaded is encrypted with a key you own and control, shareable only through time-limited links, and automatically managed by lifecycle rules.

You've learned how to:

  • Create a customer-managed KMS key and wire it into an S3 bucket as the default encryption, proving every uploaded file is encrypted with YOUR key by inspecting object metadata.
  • Generate pre-signed URLs that grant time-limited download access to encrypted files without exposing AWS credentials.
  • Configure S3 Lifecycle rules to automatically transition files to Glacier cold storage and expire them on schedule, automating cost management.
  • (Secret Mission) Wrote and applied a bucket policy that blocks all unencrypted uploads, enforcing encryption at the API level.

Ready to quiz yourself?