Build a Data Lake and Query It with SQL

Upload CSV data to S3 and query it with SQL using Athena and Glue.

Introduction

30 Second Summary

Data analysts run SQL queries all day, but setting up a database server just to analyze a few CSV files feels like overkill. What if you could drop files into cloud storage and query them with SQL directly?

In this project, you will build a data lake on Amazon S3 by uploading CSV sales datasets. You will query them with SQL using Amazon Athena, then use an AWS Glue Crawler to automatically discover and catalog new data.

What You'll Build

Imagine dropping a CSV file into a cloud folder, running a SQL query, and watching your sales data appear as a table you can download and open in Excel.

By the end of this project, you'll have:

  • A data lake on Amazon S3 holding two sales datasets you uploaded, ready to query with SQL at any time without a database server.
  • Hands-on experience running SQL queries (SELECT, WHERE, GROUP BY, JOIN) against raw CSV files in Amazon Athena, with downloadable results you can open in Excel.
  • An AWS Glue Crawler that automatically discovers your data's schema and catalogs it in the Glue Data Catalog. Drop a new dataset into S3 and query it instantly without writing any CREATE TABLE statements.
  • Secret Mission: Partition your data by region and compare query performance to understand cost optimization at scale.

Are there any prerequisites?

No prior AWS experience is needed. You will create your AWS account and set up everything from scratch during the project.

Before We Start

Before diving into AWS, let's take a moment to lock in what you're about to build and why it matters for your data analyst career.

Set Up Your AWS Account and Create an S3 Bucket

You know what you're building and why it matters for your data analyst career. Now it's time to get the tools in place.

Before you can store or query any data in the cloud, you need an AWS account and a storage bucket. This step gets you from zero to having a place to put your data.

In this step, get ready to:
  • Create your AWS account with the Free plan.
  • Create an S3 bucket to store your data lake files.
  • Organize your bucket with folders for data and query results.
Create your AWS account

Everything in this project runs on AWS. You need an account before you can access any AWS services.

  • Open your browser and navigate to aws.amazon.com.
  • Click Create an AWS Account in the top right corner.
  • Enter your email address and create a password.
  • Follow the sign-up wizard to provide your billing information.
  • When asked to select a support plan, choose the Free plan.

Will I be charged?

New AWS accounts receive up to $200 in Free Tier credits. The services used in this project cost fractions of a cent for the small CSV files you'll work with.

You will not be charged for this project.

After signing up, you'll land on the AWS Management Console. This is the central dashboard for all AWS services.

Don't see the console?

Check your email for a verification link from AWS. You may need to confirm your email address before the console becomes accessible.

If you already have an AWS account, sign in at console.aws.amazon.com instead.

Help me get into the AWS Management Console.

Create an S3 bucket

Amazon S3 (Simple Storage Service) is where your data lake will live. A bucket is a top-level container that holds files, called objects.

  • In the AWS Management Console, type S3 into the search bar at the top.
  • Select S3 from the search results.
  • In the left navigation pane, click General purpose buckets.
  • Click Create bucket.
  • In the Bucket name field, enter a unique name. Use the pattern sales-datalake-yourname-2026, replacing yourname with your actual name.
  • Record your bucket name here: your-bucket-name
  • For AWS Region, keep the default US East (N. Virginia) us-east-1.
  • Leave all other settings as their defaults.
  • Scroll to the bottom and click Create bucket.

Your new bucket appears in the S3 bucket list.

What is a data lake?

A data lake is a central storage location where you keep raw data files (like CSVs, JSON, or logs) and query them directly. Unlike a traditional database, you don't load data into a server first.

Your S3 bucket is the foundation of this data lake. You'll store CSV files here and query them with SQL later.

Why does my bucket name need to be unique?

Every S3 bucket name must be globally unique across all AWS accounts. Adding your name to the bucket name helps ensure no one else has already taken it.

Seeing a "Bucket name already exists" error?

Someone else already has a bucket with that name. Try adding extra characters to make it unique, such as your initials or a different number.

Help me pick a unique S3 bucket name.

Organize your bucket with folders

Your bucket needs two folders: one for your CSV datasets and one for query output. Keeping them separate makes your data lake organized from the start.

  • Click on your bucket name in the bucket list to open it.
  • Click Create folder.
  • Enter sales-data as the folder name.
  • Scroll down and click Create folder.

You should see the sales-data/ folder listed inside your bucket. Now create the second folder.

  • Click Create folder again.
  • Enter athena-results as the folder name.
  • Scroll down and click Create folder.

Why two folders?

The sales-data/ folder will hold your CSV datasets. The athena-results/ folder will store query output from Amazon Athena, the SQL query engine you'll use in the next step.

Before moving on, what do you expect to see inside your bucket right now?

You should see both sales-data/ and athena-results/ listed as folders inside your bucket. Your data lake now has a home.

Don't see both folders?

Make sure you're viewing the inside of your bucket, not the top-level bucket list. Click your bucket name to open it.

If a folder is missing, click Create folder and create the missing one.

Help me find or create my S3 folders.

Your data lake has a home on S3 with a clean folder structure. Next up, you'll upload your first dataset and run SQL queries against it using Amazon Athena.

Upload Sales Data and Query It with Athena

Your S3 bucket is set up with two empty folders, ready for data. Now it's time for the real payoff: uploading a sales dataset and querying it with SQL directly in the cloud, without setting up any database server.

Amazon Athena is a serverless query service that lets you run standard SQL against files stored in S3. You point Athena at your data, describe its structure, and start querying. In this step, you'll upload a CSV file, teach Athena how to read it, and run your first cloud SQL queries.

In this step, get ready to:
  • Upload a sales CSV file to your S3 bucket.
  • Create an Athena database and define an external table that points to your S3 data.
  • Run SQL queries against cloud-stored CSV files and download the results.
Create and upload your sales data

Before you can query anything, you need data in your bucket. You'll create a small CSV file with 10 rows of sales records, then upload it to the sales-data/ folder in S3.

  • Open a text editor on your computer. Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type TextEdit (macOS) or Notepad (Windows) and press Enter to open it.
  • Paste the following sales data into your text editor:
order_id,product,category,quantity,price,order_date
1001,Laptop,Electronics,2,999.99,2024-01-15
1002,Desk Chair,Furniture,1,249.99,2024-01-20
1003,Monitor,Electronics,3,349.99,2024-02-10
1004,Bookshelf,Furniture,2,129.99,2024-02-14
1005,Keyboard,Electronics,5,79.99,2024-03-01
1006,Desk Lamp,Furniture,4,45.99,2024-03-05
1007,Webcam,Electronics,2,89.99,2024-03-12
1008,Filing Cabinet,Furniture,1,199.99,2024-04-01
1009,Mouse,Electronics,10,29.99,2024-04-15
1010,Standing Desk,Furniture,1,549.99,2024-04-22

What is this data?

This CSV contains 10 fictional sales orders across two product categories: Electronics and Furniture. Each row has an order ID, product name, category, quantity sold, unit price, and order date.

The first line is the header row. Athena will need to skip this header when reading the data, which you'll configure later with a table property.

  • Save the file as sales_2024.csv to a location you can find easily, like your Desktop.

File saved as .csv.txt instead of .csv?

On Windows, Notepad may add a .txt extension automatically. When saving, change the Save as type dropdown to All Files (*.*) before saving. On macOS, make sure TextEdit is in plain text mode by clicking Format then Make Plain Text before saving.

Help me save my CSV file correctly.

  • Return to the S3 console in your browser.
  • Click on your bucket name (sales-datalake-yourname-2026).
  • Click on the sales-data/ folder to open it.
  • Click Upload.
  • Click Add files, select sales_2024.csv from your computer, then click Upload.

You should see a green success banner confirming the file was uploaded. Click Close to return to the folder view.

Your sales-data/ folder now shows sales_2024.csv listed as an object.

Configure Athena and create your sales table

With data in S3, you need to tell Athena where to find it and what shape it takes. This means setting up a query result location, creating a database, and defining an external table that maps your CSV columns to SQL types.

  • In the AWS Console search bar at the top, type Athena and click Amazon Athena from the results.

What is Amazon Athena?

Athena is a serverless SQL query service. You don't need to set up servers or load data into a database. Instead, you point Athena at files in S3, describe their structure, and run SQL queries directly against those files.

Athena charges $5 per TB of data scanned per query. For the tiny CSV files in this project, each query will cost a fraction of a cent.

  • If this is your first time visiting Athena, click Explore the query editor to open the query editor.
  • Click Edit Settings to configure where Athena stores query results.
  • In the Location of query result box, enter the S3 path to your athena-results/ folder:
s3://[[BUCKET_NAME="sales-datalake-yourname-2026"]]/athena-results/
  • Click Save.
  • Click Editor in the navigation bar to return to the query editor.

You should see the Athena query editor with an empty query pane on the right side. This is where you'll write and run SQL.

  • Create a database to organize your tables by pasting this SQL into the query editor and clicking Run:
CREATE DATABASE IF NOT EXISTS sales_analytics;

What does this query do?

This creates a logical namespace called sales_analytics in the AWS Glue Data Catalog. Athena uses the Glue Data Catalog as its metadata store, so every database and table you create in Athena is automatically registered there.

  • Confirm the database was created. In the left navigation pane, open the Database dropdown and select sales_analytics.

You should see sales_analytics appear in the Database dropdown. The Tables list below it is empty because you haven't created any tables yet.

  • With sales_analytics selected as your database, clear the query editor and paste the following CREATE EXTERNAL TABLE statement. Click Run:
CREATE EXTERNAL TABLE IF NOT EXISTS sales (
  order_id INT,
  product STRING,
  category STRING,
  quantity INT,
  price DOUBLE,
  order_date DATE
)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
LOCATION 's3://[[BUCKET_NAME="sales-datalake-yourname-2026"]]/sales-data/'
TBLPROPERTIES ("skip.header.line.count"="1");

What does this SQL do?

This statement creates an external table, which is a schema definition that points to data stored in S3. Athena does not move or copy the data. It reads directly from the S3 location at query time.

  • The column definitions (order_id INT, product STRING, etc.) map each CSV column to a SQL data type.
  • ROW FORMAT DELIMITED tells Athena to use the default LazySimpleSerDe to parse each row. FIELDS TERMINATED BY ',' means columns are separated by commas.
  • LOCATION points to the S3 folder containing your CSV files. Athena scans all files in that folder automatically.
  • TBLPROPERTIES ("skip.header.line.count"="1") tells Athena to skip the first line of each CSV file, since that line contains column headers, not data.

After the query completes, you should see a sales table appear under the Tables list in the left navigation pane.

Table not appearing?

Make sure sales_analytics is selected in the Database dropdown. If you ran the CREATE EXTERNAL TABLE with a different database selected, the table was created in that other database. Try refreshing the table list by clicking the refresh icon next to Tables.

Help me fix my CREATE TABLE statement in Athena.

Run SQL queries against your cloud data

Your table is defined and pointing at the CSV in S3. Time to see if it actually works. Before you run your first query, think about this: what do you expect to see when you query a CSV file through SQL?

  • Clear the query editor and paste the following query. Click Run:
SELECT * FROM sales LIMIT 5;

You should see 5 rows of your sales data displayed as a formatted table in the Results pane below the query editor. Each row shows an order ID, product name, category, quantity, price, and date. This is the moment where S3 and Athena click together. You just queried a raw CSV file sitting in cloud storage using standard SQL.

What is schema-on-read?

Traditional databases require you to load data into a structured format before querying it. Athena uses a schema-on-read approach: the data stays as a plain CSV in S3, and the schema (column names, types, delimiter) is applied at query time. This means you can query data without any loading step.

  • Now try a more powerful query. Clear the editor, paste the following, and click Run:
SELECT category, SUM(quantity * price) AS total_revenue
FROM sales
GROUP BY category
ORDER BY total_revenue DESC;

What does this query do?

This query calculates total revenue per product category by multiplying quantity by price for each order, then summing the results. GROUP BY category groups the rows by category. ORDER BY total_revenue DESC sorts the results so the highest-revenue category appears first.

You should see two rows: Electronics and Furniture, each with a total revenue figure. Electronics should have a higher total revenue.

  • Download the results as a CSV by clicking Download results above the Results pane.
  • Open the downloaded CSV file in Excel or any spreadsheet app to confirm the data is there.

Query returning zero results or errors?

If your query returns no data, double-check that the LOCATION in your CREATE EXTERNAL TABLE statement matches your actual bucket name exactly, including the trailing slash. Also confirm your CSV file is inside the sales-data/ folder, not at the bucket root.

If you see column values that look shifted (e.g., product names appearing in the quantity column), confirm that FIELDS TERMINATED BY ',' is in your table definition and that your CSV uses commas as delimiters.

Help me debug my Athena query results.

You just queried raw CSV files in the cloud with nothing but SQL and an S3 bucket. Next up, you'll upload a second dataset and discover why manually defining every table doesn't scale.

Hit the Wall with a Second Dataset

In the previous step, you uploaded a sales CSV to Amazon S3 and queried it with SQL using Amazon Athena. No database server, no infrastructure to manage. Just drop a file in the cloud and run SQL against it.

But your data lake only has one dataset right now. Real-world data analysts work with many datasets arriving on different schedules. In this step, you will add a second dataset and discover what breaks when you try to query it.

In this step, get ready to:
  • Create a second CSV dataset (product returns) and upload it to your S3 bucket.
  • Attempt to query the new data and discover why manual table definitions do not scale.
Create the returns data file

Your second dataset tracks product returns. These are the records a data analyst would use to understand why customers send items back and how much refund money is going out.

  • Press Cmd+Space (macOS) or the Windows key (Windows) to open your search bar.
  • Type TextEdit (macOS) or Notepad (Windows) and press Enter to open it.
  • Paste the following product returns data into a new file:
return_id,order_id,product,return_reason,return_date,refund_amount
R001,1001,Laptop,Defective,2024-02-01,999.99
R002,1003,Monitor,Wrong Item,2024-02-20,349.99
R003,1006,Desk Lamp,Damaged,2024-03-15,45.99
R004,1007,Webcam,Not as Described,2024-03-25,89.99
R005,1009,Mouse,Defective,2024-05-01,29.99

What does this data represent?

Each row is a product return. The columns capture which order was returned, what the product was, why the customer returned it, when the return happened, and how much the refund cost.

Notice this dataset has different columns from your sales data. That difference is going to matter in a moment.

  • Save the file as returns_2024.csv on your Desktop, using the same method you used for sales_2024.csv in Step 2.

Make sure it saves as .csv

On Windows, change the Save as type dropdown to All Files (*.*) before saving. Otherwise Notepad adds a .txt extension and Athena cannot parse the file.

On macOS, click Format then Make Plain Text before saving.

Upload the returns data to S3

Your returns data needs its own folder in S3, separate from the sales data. This keeps your data lake organized by dataset.

  • In the AWS Console, navigate to S3.
  • Click on your bucket name to open it.
  • Click Create folder.
  • Enter returns-data as the folder name.
  • Click Create folder to confirm.

You should see the returns-data/ folder listed alongside sales-data/ and athena-results/.

  • Click on the returns-data folder to open it.
  • Click Upload.
  • Click Add files and select returns_2024.csv from your Desktop.
  • Click Upload.

After the upload completes, you should see returns_2024.csv listed inside the returns-data/ folder.

Don't see the file in the folder?

  • Make sure you clicked into the returns-data folder before uploading. If you uploaded to the bucket root instead, the file will be outside the folder.
  • Check that the file saved with a .csv extension on your Desktop before uploading. A .txt file will still upload, but will cause issues later.
  • Why isn't my CSV file showing in my S3 folder?
Try to query the returns data

Your returns data is sitting in S3. In the previous step, you queried your sales data in Athena after creating a database and a table definition. The question now is: can Athena see this new dataset automatically?

Before you run the next query, pause and think. You uploaded returns_2024.csv to S3, but did you ever tell Athena about it? Did you create a table for it? What do you expect to happen?

  • In the AWS Console, navigate to Athena.
  • Make sure sales_analytics is selected as your database in the left panel.
  • Run the following query in the query editor:
SELECT * FROM returns LIMIT 5;

The query fails. Athena cannot find a table called returns because you never created one. Uploading a CSV file to S3 does not automatically make it queryable.

Why did this query fail?

In Step 2, you ran a CREATE EXTERNAL TABLE statement that told Athena everything about your sales data: the column names, their data types, the delimiter, and the S3 location. Without that definition, Athena has no idea the data exists.

To query your returns data right now, you would need to write another long CREATE EXTERNAL TABLE statement with the exact column names (return_id, order_id, return_reason...), their data types, the delimiter, and the new S3 location.

For two datasets, that is manageable. But imagine doing this for dozens of datasets arriving daily. Every new file means another manual CREATE EXTERNAL TABLE statement where one typo in a column name or data type silently breaks your queries.

This is the real limitation. Manual schema definition works at small scale, but it does not scale when new datasets keep arriving. Data analysts need a way to automate this.

You have felt the pain of manual schema definition firsthand. In the next step, AWS Glue steps in to solve this problem. Its Crawler will scan your S3 data, figure out the schema automatically, and make new datasets queryable without a single CREATE TABLE statement.

Automate Schema Discovery with AWS Glue Crawler

In the last step, you hit the wall. Your returns data is sitting in S3, but Athena can't see it without a CREATE EXTERNAL TABLE statement that spells out every column, data type, and delimiter.

AWS Glue Crawler fixes this. It scans your S3 data, figures out the schema automatically, and registers it in the Glue Data Catalog so Athena can query it instantly.

In this step, get ready to:
  • Create an IAM role so the Glue Crawler can access your S3 data.
  • Create and run a Glue Crawler that auto-discovers your returns data schema.
  • Query the auto-discovered table and join it with your manually defined sales table.
Create an IAM role for the Glue Crawler

AWS Glue needs permission to read files from your S3 bucket. You grant those permissions by creating an IAM role with the right policies attached.

  • In the AWS Console, type IAM in the top search bar and select IAM.
  • Click Roles in the left sidebar.
  • Click Create role.
  • Under Trusted entity type, select AWS service.
  • Under Use case, search for and select Glue.
  • Click Next.

What is an IAM role?

An IAM role is a set of permissions that an AWS service can assume. Instead of giving Glue your account credentials, you create a role that says "Glue is allowed to do these specific things."

This follows the principle of least privilege. The Crawler only gets the permissions it needs, nothing more.

  • In the Permissions policies search box, type AWSGlueServiceRole.
  • Check the box next to AWSGlueServiceRole.
  • Click Next.
  • Under Role name, enter GlueCrawlerRole.
  • Click Create role.

You should see a success banner confirming GlueCrawlerRole was created.

What does AWSGlueServiceRole do?

The AWSGlueServiceRole managed policy gives Glue the baseline permissions it needs to run crawlers, access the Data Catalog, and write logs. You still need a custom policy to let Glue read your specific S3 bucket.

Next, add an inline policy so the Crawler can read your S3 bucket.

  • In the Roles list, click on GlueCrawlerRole to open it.
  • Click Add permissions.
  • Select Create inline policy.
  • In the Policy editor section, select JSON.
  • Clear any existing content and paste the following policy:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:ListBucket"],
      "Resource": [
        "arn:aws:s3:::[[BUCKET_NAME=\"sales-datalake-yourname-2026\"]]",
        "arn:aws:s3:::[[BUCKET_NAME=\"sales-datalake-yourname-2026\"]]/*"
      ]
    }
  ]
}

What does this policy do?

s3:GetObject lets Glue read individual files in your bucket. s3:ListBucket lets Glue list the contents of your bucket to find files to crawl.

The Resource array scopes these permissions to your specific bucket only. Glue cannot touch any other bucket in your account.

  • Click Next.
  • Under Policy name, enter GlueCrawlerS3Access.
  • Click Create policy.

You should see GlueCrawlerS3Access listed under the role's Permissions policies alongside AWSGlueServiceRole.

Don't see both policies listed?

  • Make sure you opened GlueCrawlerRole before clicking Add permissions. If you created the inline policy from the wrong role, it will be attached to the wrong role.
  • Check that you selected JSON in the policy editor. If you used the visual editor, the policy may not match.
  • Confirm you replaced the bucket name placeholder with your actual bucket name in both Resource lines.

Still stuck? Help me troubleshoot my GlueCrawlerRole permissions.

Create and run a Glue Crawler

With the IAM role ready, you can create the Crawler itself. The Crawler will scan your returns-data/ folder in S3 and register what it finds in a new Glue database.

  • In the AWS Console, type Glue in the top search bar and select AWS Glue.
  • Click Crawlers in the left menu.
  • Click Create crawler.
  • Enter sales-returns-crawler as the crawler name.
  • Click Next.

What is a Glue Crawler?

A Glue Crawler scans data in an S3 folder, reads headers and sample rows, and figures out column names and data types automatically. It uses built-in classifiers to recognize formats like CSV, JSON, and Avro.

The Crawler then registers a table in the Glue Data Catalog so other services like Athena can query the data immediately.

  • Under Is your data already mapped to AWS Glue tables?, select Not yet.
  • Click Add a data source.
  • In the S3 path field, enter the path to your returns data folder:
s3://[[BUCKET_NAME="sales-datalake-yourname-2026"]]/returns-data/
  • Leave all other data source settings as defaults and click Add an Amazon S3 data source.
  • Click Next.
  • Under Existing IAM role, select GlueCrawlerRole.
  • Click Next.
  • Under Target database, click Add database.
  • Enter sales_analytics_glue as the database name.
  • Click Create database.
  • Back on the crawler setup page, select sales_analytics_glue from the Target database dropdown.
  • Click Next.
  • Keep the default schedule On demand.
  • Review your settings and click Create crawler.
  • Click Run crawler.

The crawler takes 1-2 minutes to complete. Wait for the State column to show Completed.

When the crawler finishes, you should see 1 in the Tables added column.

What just happened?

The Crawler scanned every file in your returns-data/ folder. It read the CSV headers and sample rows to figure out column names and data types.

It then registered a new table called returns_data in the sales_analytics_glue database. The table name comes from the S3 folder name, with hyphens converted to underscores.

Crawler failed or shows 0 tables added?

  • Check that the S3 path you entered matches your actual bucket name and includes the trailing slash after returns-data/.
  • Verify that GlueCrawlerRole has both the AWSGlueServiceRole managed policy and the GlueCrawlerS3Access inline policy attached.
  • Confirm that the inline policy references your actual bucket name in both Resource lines.

Still stuck? Help me troubleshoot my Glue Crawler run.

Query the auto-discovered table

The Crawler created a table called returns_data in the sales_analytics_glue database. Time to see if it actually works.

  • In the AWS Console, navigate back to Amazon Athena.
  • In the Athena query editor, select sales_analytics_glue from the Database dropdown on the left.

You should see a table called returns_data listed below the database dropdown.

Before you run your next query, think about this: do you expect to see your returns data, even though you never wrote a CREATE TABLE statement for it?

  • Run this query in the editor:
SELECT * FROM returns_data LIMIT 5;

Your returns data appears as a table. Five rows of return records, complete with column names and correct data types. All without writing a single line of DDL.

Schema-on-read in action

This is the schema-on-read pattern. Your raw CSV files sit in S3 unchanged. The Glue Data Catalog stores the schema separately, and Athena applies it at query time.

You can update the schema without touching the underlying data. This is what makes data lakes flexible compared to traditional databases where you define the schema before loading data.

Now try joining your manually-defined sales table with the auto-discovered returns table. These tables live in different databases, but Athena can query across both.

  • Run this cross-database JOIN query:
SELECT s.product, s.category, r.return_reason, r.refund_amount
FROM sales_analytics.sales s
JOIN sales_analytics_glue.returns_data r ON s.order_id = r.order_id;

What does this query show?

This joins the sales data (which you created manually with CREATE EXTERNAL TABLE in Step 2) with the returns data (which the Glue Crawler discovered automatically). The result shows which products were returned and why.

The sales_analytics.sales prefix tells Athena to look in the sales_analytics database, while sales_analytics_glue.returns_data points to the Glue-discovered database. Athena handles the cross-database join seamlessly.

You should see a table showing the product name, its category, the return reason, and the refund amount for each returned order.

Query returned no results?

  • Make sure you selected sales_analytics_glue as the database before running the SELECT query. If you still have sales_analytics selected, the returns_data table will not be found.
  • For the JOIN query, double check that you used the fully qualified table names: sales_analytics.sales and sales_analytics_glue.returns_data.

Still stuck? Help me troubleshoot my Athena cross-database JOIN query.

Secret mission

Partition Your Data by Region

What if your data lake held terabytes instead of a few kilobytes? Right now, every Athena query scans all your data. In this mission, you will restructure your sales data into region-based partitions and watch Athena skip entire folders it does not need. This is the same cost optimization technique used on production data lakes.

Clean Up Your Resources

Clean Up Your Resources

All the services you used in this project are serverless and only charge when actively used. Your S3 storage costs for small CSV files are negligible (well under $0.01/month), and nothing else incurs ongoing charges unless you run more queries or crawlers.

Decide whether to keep your resources running, pause them to come back later, or delete them entirely.

Resources you used:

  • Amazon S3 bucket sales-datalake-yourname-2026 with CSV files and Athena query results.
  • Athena database sales_analytics with the manually created sales table.
  • AWS Glue Data Catalog database sales_analytics_glue with auto-discovered tables.
  • AWS Glue Crawler sales-returns-crawler (and any additional crawlers you created in the Secret Mission).
  • IAM role GlueCrawlerRole with attached policies.
  • Local CSV files (sales_2024.csv, returns_2024.csv, and any partitioned CSV files).

Keep everything running

No action needed. Choose this if you plan to keep experimenting with your data lake.

  • Your S3 bucket stores only small CSV files, so storage costs are negligible.
  • Athena, Glue, and IAM do not charge anything when idle.
  • You will only incur costs if you run additional Athena queries ($5 per TB scanned) or Glue Crawler runs.

Pause - I'll come back to this later

There is nothing to pause. Every service you used is serverless and only charges when actively used. Your resources will sit idle at effectively zero cost until you return.

  • Leave everything as-is. No processes are running in the background.
  • When you return, open the Athena query editor and pick up where you left off.

Delete - I don't want to use this again

Remove all project resources to return your AWS account to a clean state. Follow each group below in order.

Drop the Athena databases
  • Navigate to Amazon Athena in the AWS Console.
  • In the query editor, run the following query to drop the manually created database and its table:
DROP DATABASE IF EXISTS sales_analytics CASCADE;
  • Then run this query to drop the Glue-created database and its auto-discovered tables:
DROP DATABASE IF EXISTS sales_analytics_glue CASCADE;

What does CASCADE do?

The CASCADE keyword tells Athena to drop all tables inside the database before dropping the database itself. Without it, the command fails if the database still contains tables.

Delete the Glue Crawler
  • Navigate to AWS Glue in the AWS Console.
  • Click Crawlers in the left navigation pane.
  • Select the checkbox next to sales-returns-crawler (and any additional crawlers you created for the Secret Mission).
  • Click Delete and confirm the deletion.
Delete the IAM role
  • Navigate to IAM in the AWS Console.
  • Click Roles in the left navigation pane.
  • Search for GlueCrawlerRole and select the checkbox next to it.
  • Click Delete at the top of the page.
  • Type the role name GlueCrawlerRole to confirm, then click Delete. The console automatically detaches managed policies and deletes inline policies for you.
Delete the S3 bucket
  • Navigate to Amazon S3 in the AWS Console.
  • In the General purpose buckets list, select the checkbox next to your bucket.
  • Click Empty.
  • Type permanently delete to confirm, then click Empty. Wait for the operation to complete.
  • Return to the bucket list, select your bucket again, and click Delete.
  • Type the bucket name to confirm, then click Delete bucket.
Delete local CSV files
  • Delete sales_2024.csv, returns_2024.csv, and any partitioned CSV files you created from your computer.

Nice Work!

Nice Work!

You just built a serverless data lake on AWS and queried it with SQL. That is the exact workflow used in AWS data analyst roles.

You've learned how to:

  • Build a data lake on Amazon S3 by uploading CSV datasets, then run SQL queries (SELECT, WHERE, GROUP BY, JOIN) directly against those files using Amazon Athena, with no database server required.
  • Define a table schema manually with a CREATE EXTERNAL TABLE statement and LazySimpleSerDe for CSV parsing, then hit the wall when a second dataset arrived and realize why manual schemas do not scale.
  • Automate schema discovery with an AWS Glue Crawler that scans your S3 data, registers it in the Glue Data Catalog, and makes it instantly queryable in Athena. You even joined data across manually-defined and auto-discovered tables.
  • Secret Mission: Partitioned your data by region using Hive-style folder structure and observed how Athena scans less data when queries target specific partitions, understanding the cost optimization pattern used at scale.

Ready to quiz yourself?