Create S3 Buckets with Terraform

Completely new to Terraform? Let's start by launching an S3 bucket!

Introduction

⚡️ 30 Second Summary

Welcome to your introduction to Terraform with AWS! 🎉

In this project, you'll learn how to use Terraform to automate the deployment of an S3 bucket.

In the real world, automation is key to staying efficient in how you develop and deploy code. It's how DevOps teams cut out the manual work of creating new resources in their cloud environment, which reduces the chances of errors.

This project is highly relevant for roles like DevOps Engineer, DevSecOps Engineer or Platform Engineer - where it is a key responsibility to automate your cloud infrastructure.

In this project, get ready to...

🛠️ Install and configure Terraform.
🔑 Configure your AWS credentials in the terminal.
🪣 Create and manage S3 buckets with Terraform.
💎 Upload files to S3 using Terraform.

🎬 Want a complete demo of this project, from start to finish? Check out our walkthrough with Natasha

If you're up for a bit of a challenge, quiz yourself on the key concepts up ahead in this project.

Before we start Step #1...

Before we get started, it's important that you know what we're trying to do today...

Install Terraform

Let's kick things off by installing Terraform, our tool for building and managing infrastructure as code!

What is Terraform?

Terraform is a tool that helps you build and manage your cloud infrastructure using code.

Instead of setting up resources manually in the AWS console or CLI, you write a script that tells Terraform exactly what you want in your cloud infrastructure, like servers, databases, and networks. Terraform then automatically builds all of this for you, using your script as a blueprint.

💡 What is Infrastructure as Code (IaC)?

Infrastructure as Code (IaC) is the practice of describing your cloud setup (like your servers, storage, networks) in plain text files instead of clicking through a web console. When you run those files, the cloud builds everything exactly as you've written it written. Because the description lives in code, you can version-control it, share it with teammates, and recreate the same environment any time you need.

Terraform is a popular IaC tool because it understands all the big cloud providers - AWS, Azure, and Google Cloud. In the same configuration file, you can tell Terraform to create resources across any or all of them. Terraform's syntax is also designed to read almost like English, so you can pick it up quickly while still modeling complex setups once you’re ready.

In this step, get ready to:

Download and install Terraform.
Verify your installation.

Download and Install Terraform

Head to the official Terraform download page: https://developer.hashicorp.com/terraform/downloads

Download the Terraform package for your operating system. Make sure to select the correct version for your system (macOS, Windows, or Linux).
If you're not sure which option you should pick for your operating system:

MacOS

Select the Apple icon from the top left hand corner of your computer's menu bar.
Select About this Mac.
Note whether your Chip says Apple (pick ARM64) or Intel (pick AMD64).

Windows

Select the Start button.
Search for System Information.
Note whether your System Type says x64-based or ARM-based PC.

Linux

Open a terminal and run uname -m.
Match the output to the correct Linux package option.

Unzip the package in your local computer's Downloads folder.

Move the Terraform binary to your PATH.

What is a Terraform binary?

A Terraform binary is the core file that you run to use Terraform. It's called an executable file because your operating system needs it to execute commands/code that use the tool.

💡 What is PATH?

PATH is a shortlist of folders that your computer searches through when you run a command in the terminal.

For example, when you type terraform, your computer walks through that list until it finds a file named terraform and then runs it. If the program lives outside those folders, the command fails. Add a new folder path to PATH and it joins the checklist, so you can reference the folder without typing its full location.

The instructions for moving your Terraform binary depends on your operating system:

MacOS/Linux

Open your local computer's Terminal.
Navigate your terminal into the unzipped folder:

cd Downloads/[[FOLDERNAME="folder_name"]]

I ran into an error

Make sure you've replaced the placeholder i.e. folder_name with the name of your unzipped folder, e.g. terraform_1.10.2_darwin_arm64.

Still stuck? Ask the NextWork community!

Run pwd to check that you're at the right directory.

What does pwd do?

pwd stands for print working directory, and this command asks your server "where am I right now?" The terminal will show you the exact location of the directory (folder) you're in.

Run this command to add Terraform to your PATH:

sudo mv terraform /usr/local/bin/

I ran into an error

This command assumes that your PATH includes /usr/local/bin, but you can customize it if your locations are different. Run echo $PATH to see all the PATHS you could use for storing your Terraform file.

You might be asked to enter a password - enter the password you use to log into your computer.

Great work! You've just installed Terraform in your local computer. Now we'll need to verify we've installed it correctly.

Windows

Open System Properties:
- Windows 10: Search for Edit the system environment variables in the Start menu and select the shortcut that appears.
- Windows 11: Right-click on the Start button, select System. Scroll down and on the right side under Related settings, click on Advanced system settings.

What is System Properties?

System Properties is a Windows control panel that lets you change advanced system settings, including environment variables like PATH that tell Windows where to find programs.

In the System Properties window, select the Advanced tab and select Environment Variables.

What are Environment Variables?

Environment variables are settings that tell Windows how to behave. The PATH variable specifically is a list of folders where Windows looks for programs when you type a command in the terminal.

Scroll down in the System Variables section until you find the variable PATH
Select Edit.
In the Edit Environment Variable dialog, select New and enter the directory path: c:\terraform

Why are we adding c:\terraform to PATH?

By adding the terraform folder to PATH, Windows will know where to find the terraform program when you type terraform in any terminal. Without this, you'd have to type the full path every time.

Be sure to include a semicolon at the end of the previous path, i.e. c:\path;c:\terraform

Why the semicolon?

The semicolon separates different paths in the PATH variable. Windows uses this to distinguish between multiple folder locations where it should look for programs.

Launch a new terminal session for the settings to take effect.

Why do I need a new terminal?

Environment variable changes only take effect in new terminal sessions. Your current terminal is still using the old PATH settings, so you need to open a fresh one to see the changes.

Great work! You've just installed Terraform in your local computer. Now we'll need to verify we've installed it correctly.

Verify Terraform Installation

Run the following command:

terraform version

You should see the installed Terraform version in the terminal! This would only work if you've installed Terraform correctly.

I see an error message

Double-check that you've done all the Terraform installation steps for your operating system! You can also refer to the official Terraform documentation.
Make sure Terraform is correctly added to your system's PATH. Run which terraform or where terraform in the terminal. If Terraform is in your PATH, you should see a full path (e.g., /usr/local/bin/terraform); if you get an error, it's not in your PATH!
Still stuck? Ask the NextWork community!

Set up your Terraform project

Now that Terraform is installed, let’s create a project folder. This folder will store the files that tell Terraform what to build.

In this step, get ready to:

Create a project directory.
Set up your main configuration file.

Create Project Directory

Feeling warmed up to your terminal? Let's run a few more commands to set up your project directory:

MacOS/Linux

On your desktop, create a new directory named nextwork_terraform. You can do this by running the following command in the Terminal:

mkdir ~/Desktop/nextwork_terraform

Navigate to the nextwork_terraform directory by running the following command:

cd ~/Desktop/nextwork_terraform

Double check that you're at the right directory.

pwd

Create main.tf

Inside the nextwork_terraform directory, create a new file named main.tf. Run the following command:

touch main.tf

Windows

On your desktop, create a new directory named nextwork_terraform. You can do this by running this PowerShell command:

New-Item -Path "$env:USERPROFILE\Desktop\nextwork_terraform" -ItemType Directory

Next, navigate your terminal to the nextwork_terraform folder:

Set-Location "$env:USERPROFILE\Desktop\nextwork_terraform"

Let's double check that you're at the right folder.

Get-Location

Create main.tf

Inside the nextwork_terraform folder, create a new file named main.tf. Run the following command:

New-Item -Path "main.tf" -ItemType File

What is main.tf?

main.tf is a central file in a Terraform project.

It's where you write down what you want your infrastructure to look like, using Terraform's language. Think of it as the blueprint for building your cloud resources.

💡 How does Terraform use configuration files?

Terraform uses configuration files (like main.tf) to define and manage infrastructure. These files describe the desired state of your infrastructure, and Terraform figures out how to achieve that state.

Head to your local computer's Desktop, and find and open your new nextwork_terraform folder.

Define main.tf

Now, let's add some code to our main.tf file to tell Terraform what we want to build.

In this project, we want to build an S3 bucket.

In this step, get ready to:

Add Terraform code to main.tf

Add Code to main.tf

Open main.tf in a text editor like VS Code, NotePad++ or TextEdit.

Copy and paste the following code into main.tf:

provider "aws" {
  region = "[[AWS_REGION="your-region-code"]]"
}

resource "aws_s3_bucket" "my_bucket" {
  bucket = "nextwork-unique-bucket-[[YOURNAME="yourname"]]-[[RANDOMNUM="random-number"]]" # Make sure this bucket name is globally unique by typing a long random number
}

resource "aws_s3_bucket_public_access_block" "my_bucket_public_access_block" {
  bucket = aws_s3_bucket.my_bucket.id

  block_public_acls       = true
  ignore_public_acls      = true
  block_public_policy     = true
  restrict_public_buckets = true
}

Don't forget to save your changes to main.tf.

You might be wondering...

What does the code mean?

Provider "aws" This tells Terraform to use AWS. A "provider" is a plugin that lets Terraform work with a specific cloud service. The provider turns your configuration into API calls that create and manage your infrastructure.
Resource "aws_s3_bucket" "my_bucket" This creates an S3 bucket. my_bucket is an internal name that other blocks of code in main.tf will use to reference your bucket.
Resource "aws_s3_bucket_public_access_block" "my_bucket_public_access_block" This controls who can access your S3 bucket. All the settings like block_public_acls, ignore_public_acls, block_public_policy, and restrict_public_buckets are set to true to prevent public access to your bucket and its contents.

How are Terraform files structured?

Terraform stores its instructions in self-contained “blocks.” You might notice this in our main.tf file - the code is split into three blocks!

There are different kinds of blocks to do different kinds of things:

A provider block tells Terraform which cloud to work with e.g. AWS, Google Cloud, Azure. (this is in our `main.tf code!)
A resource block describes what to create, such as a storage bucket or virtual machine, and how it should look.

By keeping each piece of your infrastructure in its own block, the file stays easy to read and you can tweak one part without touching the rest of your setup!

Tip: The style of organizing code into individual blocks is called modularity.

Extra for Experts: are provider and resource the only types of blocks?

A variable block holds placeholder values you can swap in for different environments
An output block prints useful details (e.g. an S3 bucket’s URL) after a run finishes
A data block reads existing cloud resources so you can reference them without recreating anything
A module block pulls in reusable chunks of Terraform code from elsewhere

💎 Customize main.tf

Congratulations

Secret Mission unlocked!

Surprise! Welcome to this hidden secret mission.

Your mission, should you choose to accept it, is to add your own customizations to your S3 bucket with Terraform.

Visit the Terraform registry's documentation on S3 buckets.

What is the Terraform registry?

The Terraform registry is where you can find and read documentation about Terraform modules and providers.

In our case, we're visiting the registry to read documentation on the aws provider, where we'll find setup instructions, descriptions of each available resource, best practice tips and examples, and parameters for customization.

Developers can also use the registry to upload their Terraform configurations as modules. Other developers can then use those modules in their own configurations, so they won't have to write everything from scratch.

💡 What can I find in this aws_s3_bucket page?

The aws_s3_bucket in the Terraform documentation is all about how to use Terraform to create and manage AWS S3 buckets.

It covers how to configure these buckets, set their properties, and manage interactions with other AWS services.

From the documentation, choose an additional configuration that you might want to add to your S3 bucket or the object you've uploaded.
Write your configuration in main.tf.
Remember, this task is open-ended. There's no single correct answer, the goal is to experiment! 🧪
Don't forget to save your changes to main.tf.

You're too cool 🔥 Nice work taking on this open ended challenge and adding your own customizations!

Run your Terraform configuration

With our main.tf file ready, let's try running our Terraform configuration!

This means we'll need to do two things...

In this step, get ready to:

Initialize Terraform.
Plan your infrastructure changes i.e. preview how your configuration will change your AWS environment.

Initialize Terraform

Head back to your local terminal.
Initalize Terraform by running the following command:

terraform init

Let's break down the command and the terminal's response!

What does terraform init do?

Terraform init is the first command you run for a new Terraform project. It sets up your project by:

Downloading necessary plugins: It grabs the providers which handle the connection with cloud services like AWS.
Setting up the backend: Terraform needs to set up a backend that keep its own record of your infrastructure, so future commands can see what has already been created.
Preparing modules: If your setup uses reusable code blocks (called modules) this is where the code is downloaded from their source to replace placeholders in your code.
Creating a lock file: This file keeps track of the versions of everything Terraform is using, making sure that everyone on your team is using the same setup to avoid inconsistencies.

Extra for Experts: Breaking down the output

Extra for Experts: Can you break down the output for me?

This detailed output from terraform init tells you about each step Terraform is taking to get ready, like finding and installing the right AWS provider.

Initializing the backend... Terraform is preparing the backend where your state files will be stored. A state file in Terraform remembers the current state of your infrastructure, so Terraform knows what your infrastruture already has when it creates/manages resources.
Initializing provider plugins... Terraform is starting the process to install the provider plugins it needs to manage your infrastructure.
Finding latest version of hashicorp/aws... Terraform is searching for the latest version of the AWS provider plugin, which is necessary to manage AWS resources.
Installing hashicorp/aws v5.80.0... The AWS provider plugin (at version 5.80.0) is installing.
Installed hashicorp/aws v5.80.0 (signed by HashiCorp) The AWS provider plugin (at version 5.80.0) has finished installing. The plugin is verified with a signature from HashiCorp, which means it's a verified and official plugin.
Terraform has created a lock file .terraform.lock.hcl to record the provider selections it made above. Terraform generates a lock file to record the exact provider versions used during initialization. This file should be committed to version control to ensure consistency across all environments and Terraform executions.
Terraform has been successfully initialized! Your environment is now set up to use Terraform for managing your resources (yay!)
You may now begin working with Terraform... You can move on to running other Terraform commands! Terraform specifically recommends running terraform plan to review any changes it'll make to you infrastructure, which we'll do in a second.
If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you ever tweak the backend location or add new shared code, just rerun terraform init to refresh that setup.

Plan your Terraform configuration

Run the following command:

terraform plan

What does 'terraform plan' do?

terraform plan creates an execution plan, showing you what changes Terraform will make to your infrastructure based on your configuration files (like main.tf).

It shows what will be created, updated, or destroyed, so you can review and confirm the configuration before any real changes are made.

Oops! Do you see any errors?

Authentication error

You might get an error that says No valid credential sources found

Or you might see an error that tells you the security token requested included in the request is invalid

Why did we get this error?

This error means Terraform doesn't have the necessary credentials to access your AWS account. We need to configure AWS credentials for Terraform to use, so that it can create AWS resources on our behalf.

No worries! We'll go through exactly how to resolve this error in the next step. See you there!

Error about bucket name

Is your error about your bucket's name?

To troubleshoot, double check your main.tf file:

Have you updated your bucket name to be globally unique?
Have you removed the square brackets i.e. [] in your configuration?

Once you've fixed your bucket name error, don't forget to run the Terraform command again and check whether the terminal gives you a different output.

If you're still stuck, don't forget to ask the NextWork community.

I didn't get any errors

Great! First - check if you've saved your main.tf file. You'd automatically get no errors if there are no saved changes for Terraform to look at!

You can skip the next step (on setting up the AWS CLI) and head straight to 🪣 Step #6: Launch an S3 bucket now.

Set up AWS Credentials

Let's fix that error by setting up our AWS credentials.

This will let Terraform authenticate with AWS and manage our resources.

In this step, get ready to:

Install AWS CLI.
Generate AWS access keys.
Configure CLI with your access key.

What is AWS CLI?

The AWS CLI (Command Line Interface) is a powerful tool that lets you manage your AWS services from your terminal. Instead of having to use the AWS Management Console, you can now run text commands from your local machine.

We're using the CLI in this project so that Terraform can create AWS resources from your local computer.

Install the AWS CLI

The instructions for installing AWS CLI are different depending on your computer's operating system.

Pick the one that works for you.

MacOS

Open your local terminal (called Terminal) and run:

curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
sudo installer -pkg AWSCLIV2.pkg -target /

Your terminal might ask for your password. Enter your usual password for logging into your Mac and press Enter on your keyboard.

That's it, your terminal will start downloading AWS CLI straight away!

Windows

Open PowerShell and run:

What do these commands do?

The first command downloads the AWS CLI installer MSI file to your current directory. The second command runs the installer and waits for it to finish:

Invoke-WebRequest -Uri "https://awscli.amazonaws.com/AWSCLIV2.msi" -OutFile "AWSCLIV2.msi"
Start-Process msiexec.exe -ArgumentList "/i AWSCLIV2.msi" -Wait

That's it, your terminal will start downloading AWS CLI straight away!

Linux

Open your terminal and run:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

That's it, your terminal will start downloading AWS CLI straight away!

What is AWS CLI?

In this case, we'll need the CLI

Nice work! You've just installed AWS CLI.

While we wait for AWS CLI to install...

Verify CLI Installation

Once you've installed AWS CLI, open your terminal and run:

aws --version

You should see a version number for AWS CLI, which confirms you've installed it perfectly 🤙

Don't see a version number?

Make sure you've run aws --version without any typos or extra spaces.
Check out the AWS CLI installation page for your OS. Mac and Windows instructions offer a package you could install without needing the terminal.
Still stuck? Ask the NextWork community.

Configure AWS CLI

Run aws configure in your terminal to log in.
The terminal is now asking for your Access Key ID.

What is an access key?

Think of an access key as the AWS CLI way of logging in.

The CLI and the Management Console aren't connected, so the CLI won't know if you've already authenticated yourself by logging into the console. It needs its own way to authenticate you, and accepting an access key is one of the fastest ways to do it.

Set Up AWS Access Keys

Log in to the AWS Management Console as your IAM Admin user.
Head to the IAM console.

Select Users from the left hand navigation panel.
Select your IAM Admin user.

Select the Security credentials tab.

In the Access keys section, select Create access key.

Select Command Line Interface (CLI).

Wow, what are the other options about?

These options tell AWS how you'll use your access key.

For example, Local code is for scripts running on your machine, and Application on AWS is for services like EC2 or Lambda wanting to access other AWS resources.

Check the acknowledgement checkbox at the bottom of the page.

Why is there an acknowledgement checkbox?

AWS usually recommends using another service called IAM Identity Center instead of access keys for managing programmatic access (i.e. access over the terminal), because it’s newer and more secure.

This is especially true for larger teams, where you might need to manage CLI access for multiple people.

Select Next.

For the description, enter a short description like this:

Created to get programmatic access to AWS. Using this key for the NextWork project on Terraform.

Select Create access key.

Well done! Your access key is created, and AWS has set you up with an Access key and Secret access key.
🚨 Select Download .csv file. This is extremely important - you can't see the Secret Access Key again once you leave this page. Downloading the .csv file is how you can keep these credentials!

Uh oh, I've already left this page

Ah, classic! Don't worry, you'll just need to delete the key you've set up and create another one. Don't forget to stay on the page the second time round.

If you're stuck while deleting your access key, ask the NextWork community.

Configure the AWS CLI

Head back to your local terminal again - this time you have credentials handy to access AWS CLI. Nice!
- If your previous terminal sessions has closed, run aws configure again.
Copy the Access Key ID from the IAM console, and paste it in your terminal.

Uh oh, I've already closed that tab...

It happens! In that case, open the ECR-Access_accessKeys.csv file in your Downloads folder. You would've downloaded this file as your backup for credentials!

Paste the access key ID into your terminal.

Copy the Secret Access Key from the IAM console.

Paste the secret access key into your terminal.

Enter your AWS region code (e.g., us-east-1). Make sure it's the same region code as what's in us-east-1!

You can leave the output format blank and press Enter on your keyboard. The default format for the terminal responses from your AWS commands is JSON, and that works great.

Plan your Terraform configuration (again)

Now that we've configured our AWS credentials, let's try running our Terraform configuration again.

This time, it should work without the authentication error 👀

In your terminal, head back to your nextwork_terraform directory.

MacOS/Linux

cd ~/Desktop/nextwork_terraform

Windows

cd "$env:USERPROFILE\Desktop\nextwork_terraform"

Plan your Terraform configuration again.

terraform init
terraform plan

WOOO you're making moves! Nice work, you've just set up AWS CLI and access keys, which got terraform plan to finally work.

Launch an S3 Bucket with Terraform

Terraform? Initalized ✅

Your configuration? Planned ✅

For the grand finale in this project, it's time to apply your Terraform configuration to launch your S3 bucket!

In this step, get ready to:

Apply your Terraform configuration.
Verify that you've launched an S3 bucket (exciting!)

In your terminal, you should see a Terraform plan showing the resources that will be created.

What is this Terraform plan saying?

Check out the line starting with Plan: (near the bottom of the output) for a summary!

Terraform plans to add two new resources (the S3 bucket and its public access block), with no resources changed or destroyed.

💡 What is the warning at the bottom of the output?

This warning suggests using the -out option to save this execution plan. Saving the plan lets you apply exactly that plan later, which is helpful in situations where you might want to save a certain configuration before you make any changes.

Apply Terraform Configuration

Run the following command:

terraform apply

Type yes and press Enter to apply the changes.

What does terraform apply do?

terraform apply applies the changes you're written in your Terraform configuration. It creates, modifies, or deletes resources in your infrastructure based on your code. In our case, this command creates the S3 bucket in your AWS account.

💡 What would happen if I ran 'terraform apply' before init or plan?

If you run terraform apply before terraform init, you would run into an error because terraform init needs to set up your project first by downloading necessary plugins and setting up the state file, which is a file Terraform uses to track the current state of your infrastructure.

There actually aren't any errors if you skip terraform plan. This is because terraform apply will automatically run a plan and ask for confirmation before making any changes. But, skipping the explicit terraform plan means you might miss reviewing these changes in detail. It's best practice to run a plan to catch any potential errors or unexpected results before they affect your live infrastructure.

Verify S3 Bucket Creation

Log in to the AWS Management Console as your IAM Admin user.
Head to the S3 console.
You should see your newly created S3 bucket in the console!

Great work, you've just set up Terraform in your computer AND used it to launch an S3 bucket.

Secret mission

Welcome to your 🤫 exclusive 🤫 secret mission!

Your mission, should you choose to accept it, is to upload a file to your new S3 bucket using Terraform.

💎 In this secret mission, get ready to:

Update your Terraform configuration file.
Apply your Terraform file again.
Showcase your secret mission in your project documentation.

Upload an S3 object with Terraform

Delete your resources

Important

Make sure to delete all your resources when you’re done to avoid unexpected charges. Yup - if you don't delete all your resources, you might get charged!

Do you think you can delete the resources you've created today?

Yep, all done.

Your Amazon S3 bucket
Your AWS access key (optional)

I know, but also, I don't...

If you're feeling stuck (we've all been there!), here's a little guide:

Delete Resources with Terraform

In your terminal (still inside the nextwork_terraform directory), run the following command:

terraform destroy

Type yes and press Enter on your keyboard to confirm you want to destroy the resources.

How does terraform destroy work?

terraform destroy reverses the changes made by terraform apply. It deletes the resources in your Terraform configuration, bringing your infrastructure back to its previous state.

In our case, this will remove the S3 bucket and any objects inside.

Delete the IAM Access Key

If you've created an access key for this project, it's best practice to delete it.

In your IAM console, select Users from the left hand sidebar.
Select your IAM Admin user.
Select the Security credentials tab.
Scroll down to the Access keys section.
Next to your access key, select the Actions dropdown.
Select Delete.
In the deletion pop-up panel, Deactivate your key, enter the access key ID in the text input field, then select Delete.
Delete the access key .csv file in your local computer's Downloads folder.

Phew, that should be everything deleted now!

That's a wrap!

Congrats! You've just learned how to:

🛠️ Install and configure Terraform.
🔑 Configure your AWS credentials in the terminal.
🪣 Create and manage S3 buckets with Terraform.
💎 Upload files to S3 with Terraform.

Ready to quiz yourself? You got this! 💪

Terraform is a truly practical and used tool in cloud computing and DevOps. Now that you know how to use it, you can start to automate the deployment of other projects you've built, or AWS services you're learning about!

p.s. Does it say "Still tasks to complete!" at the bottom of the screen?

This means you still have screenshots left to upload, or questions left to answer!

Press Ctrl+F (Windows) or Command+F (Mac) on your keyboard.
Search for the text Return to later.
Jump straight to your incomplete tasks!
🙋‍♀️ Still stuck? Ask the community!