Investigate a Down Web Server on Linux

Diagnose and fix three real web server failures on a live Linux server.

Introduction

30 Second Summary

When a website goes down, someone has to figure out why and bring it back up. That person is a Site Reliability Engineer, and their most important tool is the Linux command line.

In this project, you will launch a Linux server on AWS and install a web server called Nginx. Then you will deliberately break it and use core Linux commands to find each problem and fix it.

What You'll Build

You will launch an EC2 instance running Amazon Linux 2023, connect to it remotely via SSH, and work through realistic web server troubleshooting scenarios.

By the end of this project, you'll have:

A running Amazon Linux 2023 EC2 instance you launched and connected to via SSH.
Hands-on experience with 20+ essential Linux commands for navigation, log analysis, and service management.
Two diagnosed and fixed web server issues, including permission errors and configuration syntax errors.
Secret Mission: Write a bash health check script that monitors your server's vital signs automatically.

Are there any prerequisites?

You need an AWS account (free tier eligible) and MobaXterm installed on your Windows computer. No prior Linux experience is needed.

Launch an EC2 Instance and Connect with MobaXterm

Every troubleshooting task starts with reaching the server. Before you can check logs, restart services, or dig into errors, you need a machine to connect to.

In this step, you'll launch a virtual Linux server on AWS and connect to it remotely. By the end, you'll have a live terminal prompt on your own cloud server.

In this step, get ready to:

Launch an Amazon Linux 2023 EC2 instance on AWS.
Download a key pair file for secure SSH access.
Connect to your server remotely using MobaXterm.

Launch an EC2 instance in the AWS Console

EC2 (Elastic Compute Cloud) lets you spin up virtual servers in AWS. You'll launch one running Amazon Linux 2023, a lightweight Linux distribution built specifically for AWS.

Open your browser and go to console.aws.amazon.com.
Sign in with your AWS account credentials.
Type EC2 in the search bar at the top of the page.
Click EC2 in the search results.
Click Launch instances (the orange button).

You'll see a single-page configuration form for your new server. Work through each section from top to bottom.

In the Name field, enter linux-lab.
In the Application and OS Images section, verify that Amazon Linux 2023 AMI is selected. It should be the default and is marked Free tier eligible.
In the Instance type section, verify that t2.micro is selected. It is also marked Free tier eligible.

Your instance needs a key pair for secure access. A key pair creates a private key file that only you have, which proves your identity when connecting to the server.

In the Key pair (login) section, click Create new key pair.
Enter linux-lab-key as the Key pair name.
Verify RSA is selected as the key pair type.
Verify .pem is selected as the file format.
Click Create key pair. Your browser downloads a file called linux-lab-key.pem automatically.
Save the .pem file somewhere you can find it easily, like your Desktop.

You can only download this file once

AWS does not store your private key. If you lose the linux-lab-key.pem file, you will not be able to connect to your instance. Move it somewhere safe right now.

Back on the launch form, scroll down to the Network settings section.
Verify that Allow SSH traffic from is checked and set to My IP.
Leave all other settings as default.
Click Launch instance.

You should see a green "Success" banner confirming your instance is launching.

What did I just set up?

An AMI (Amazon Machine Image) is the template for your server's operating system and software. Amazon Linux 2023 is a lightweight Linux distribution optimized for running on AWS.

A key pair works like a lock and key for your server. AWS keeps the lock (public key) on the instance, and you downloaded the key (.pem file). Without it, nobody can connect.

A security group acts as a firewall for your instance. You configured it to only allow SSH connections from your IP address, keeping everyone else out.

Find your instance's public IP address

Your instance needs about a minute to fully boot up. Once it's running, AWS assigns it a public IP address that you'll use to connect from your computer.

Click View all instances to go to the EC2 Instances dashboard.
Wait for the Instance state column to show Running with a green circle.
Wait for the Status check column to show 2/2 checks passed. If the status hasn't updated, click the refresh icon near the top of the table.
Click on your linux-lab instance to select it.
In the details panel below, find the Public IPv4 address field.
Copy this IP address. You'll need it in the next substep to connect.

Connect using MobaXterm

Your server is running, but you can't interact with it yet. You need to open a remote SSH connection from your computer to the instance.

Press the Windows key to open your search bar.
Type MobaXterm and press Enter to open it.
Click Session in the top left corner of the MobaXterm window.
Select SSH as the session type.
In the Remote host field, paste your EC2 instance's public IP address.
Check the Specify username checkbox.
Enter ec2-user in the username field.
Click the Advanced SSH settings tab.
Check the Use private key checkbox.
Click the browse button next to it and select the linux-lab-key.pem file you saved earlier.
Click OK to start the connection.
When prompted to accept the server's host key, click Yes. This is a one-time security check for first-time connections.

You should see a Linux terminal prompt showing something like [ec2-user@ip-xxx-xxx-xxx-xxx ~]$. You are now connected to your server.

What is SSH?

SSH (Secure Shell) is a protocol for securely connecting to remote computers over a network. It encrypts everything you type, so your commands and credentials stay private.

Every SRE uses SSH multiple times a day to access servers. The .pem file you downloaded is your SSH private key, which proves your identity to the server without needing a password.

Can't connect to your instance?

Make sure your instance state shows Running with 2/2 checks passed in the EC2 dashboard. If it's still initializing, wait another minute and try again.
Check that you selected the correct .pem file in MobaXterm's Advanced SSH settings and that the username is exactly ec2-user.
If you see "Connection timed out", your security group may not allow SSH from your current IP. Go back to EC2, select your instance, click the Security tab, and confirm port 22 is open to your IP address.

Help me troubleshoot my SSH connection to EC2

You're connected to a live Linux server. Next up, you'll explore the filesystem and learn the core navigation commands every SRE needs.

Explore the Linux Filesystem

You've launched a server and connected to it remotely. That's the essential first step for any SRE.

Now it's time to learn how to move around. Every troubleshooting task on a Linux server starts with knowing where things are and how to read them.

In this step, get ready to:

Navigate the filesystem using pwd, cd, and ls.
Read file contents with cat and less.
Understand file permissions and the sudo command.

Find your way around with pwd, cd, and ls

The terminal is your primary tool for navigating a Linux server. Every directory, config file, and log you'll ever need to find lives somewhere in the filesystem.

Check your current location by running this command:

pwd

You should see /home/ec2-user. This is your home directory on the server.

List the files in your current directory by running:

ls

The directory might appear empty. That's completely normal for a fresh server.

See hidden files and detailed information by running:

ls -la

Now you can see files that were invisible before, all starting with a dot.

What does ls -la show?

The -l flag shows a detailed list with permissions, owner, file size, and date modified. The -a flag includes hidden files.

Hidden files in Linux are any file whose name starts with a dot (like .bashrc). They're hidden from basic ls to keep things tidy, but they're often important configuration files.

Navigate to the root of the filesystem by running:

cd /
ls

You should see a list of top-level directories. These are the building blocks of the entire filesystem.

Key directories to know

/etc stores configuration files for the system and installed services.
/var holds variable data like logs and temporary files that change over time.
/home contains user home directories.
/tmp is for temporary files that may be deleted on reboot.

Explore the configuration directory by running:

cd /etc
ls

You should see a long list of configuration files. Almost every service on the server stores its settings here.

Jump back to your home directory by running:

cd ~

What is the ~ shortcut?

The tilde ~ always points to your home directory. No matter where you are in the filesystem, cd ~ takes you straight back to /home/ec2-user.

Read files with cat and less

Navigating directories is only half the picture. You also need to read what's inside the files you find.

Check what Linux distribution your server runs by running:

cat /etc/os-release

You should see Amazon Linux 2023 in the output. The cat command prints the entire contents of a file to your terminal.

Check your server's hostname by running:

cat /etc/hostname

This shows a short string identifying your server. The hostname is how your server identifies itself on a network.

Browse the list of user accounts on the system by running:

less /etc/passwd

How do I use less?

Use the arrow keys to scroll up and down.
Press q to quit and return to the terminal.

The less command is better than cat for longer files because it lets you scroll through the content page by page instead of dumping everything at once.

Look for the ec2-user entry in the list. This is the account you're logged in as.

Understand file permissions

Every file and directory in Linux has permissions that control who can read, write, or execute it. Permission errors are one of the most common causes of server issues.

View the permissions of a file by running:

ls -l /etc/hostname

The output looks something like -rw-r--r-- 1 root root .... That first chunk of characters is the permission string.

How do I read file permissions?

The first character indicates the type: - for a regular file, d for a directory.
The next nine characters are three groups of three: owner, group, and others.
Each group shows r (read), w (write), x (execute), or - (no permission).
In -rw-r--r--, the owner can read and write. The group and others can only read.

Confirm which user you are by running:

whoami

You should see ec2-user.

See your user ID and group memberships by running:

id

Why does your user ID matter?

Your user ID and group memberships determine which files you can access. When a file's permissions say "owner can read," Linux checks whether your user ID matches the file's owner.

If you don't have permission, the command fails. This is the exact mechanism behind the permission errors you'll diagnose later in this project.

Some files are restricted so that only the root (admin) user can read them. Linux provides a way to temporarily act as root using the sudo command.

Read a root-only file by running:

sudo ls -l /etc/shadow

The /etc/shadow file contains password data and is only readable by root. Without sudo, this command would fail with a "permission denied" error.

What is sudo?

The sudo command runs a single command as the root (admin) user. It stands for "superuser do."

You'll use sudo frequently as an SRE whenever you need to access system files, install software, or manage services.

Getting 'permission denied'?

Make sure you included sudo at the very start of the command.
The ec2-user account has sudo privileges by default on Amazon Linux. If you're still getting errors, double check you're logged in as ec2-user by running whoami.
Why am I getting permission denied when running commands on my EC2 instance?

You can now navigate the filesystem, read files, and understand who has access to what. Next up, you'll install a real web server and learn to manage it as a running service.

Install and Start a Web Server

You've explored the Linux filesystem and know how to navigate directories, read files, and check permissions. Now it's time to give your server a real job.

In this step, you'll install Nginx, a popular web server, and learn how to manage it as a running service. Package installation, service management, and health verification are tasks you'll repeat on every server you touch as an SRE.

In this step, get ready to:

Install Nginx using the dnf package manager.
Start the Nginx service and check its status with systemctl.
Verify the web server responds to requests.

Install Nginx using the package manager

Amazon Linux 2023 uses dnf as its package manager. It handles downloading, installing, and updating software packages for you.

Why Nginx instead of Apache?

Nginx is the more widely-used web server in modern SRE and DevOps environments. Its configuration syntax is clean and predictable, making errors easy to spot and fix.

Install Nginx by running this command:

sudo dnf install nginx -y

What does this command do?

sudo runs the command as the root (admin) user. Installing software requires admin privileges.
dnf install nginx tells the package manager to download and install the Nginx package.
-y automatically answers yes to any confirmation prompts.

Watch for Complete! at the end of the output. This confirms Nginx installed successfully.

Installation failed?

If you see Could not resolve host, your EC2 instance may not have internet access. Check your VPC and subnet settings in the AWS Console.
If you see Permission denied, make sure you included sudo at the start of the command.
Help me troubleshoot my dnf install command

Start Nginx and check its status

Installing a package doesn't automatically start it. In Linux, services are managed through systemctl, which lets you start, stop, restart, and check the status of any service.

Start the Nginx service by running this command:

sudo systemctl start nginx

Check that Nginx started successfully by running:

sudo systemctl status nginx

Look for active (running) highlighted in green in the output. This confirms the service is up.

What is systemctl?

systemctl is the standard tool for managing services on modern Linux systems. The key commands you'll use are start, stop, restart, and status.

When something breaks in production, systemctl status is always your first move.

Try the shorthand status check by running:

systemctl is-active nginx

This prints a single word: active or inactive. It's useful in scripts where you need a quick answer.

Nginx not showing as active?

Make sure you included sudo when running the start command. Starting a service requires admin privileges.
If the status shows failed, run sudo journalctl -u nginx -e to see what went wrong.
Help me troubleshoot my Nginx service

Verify the web server is responding

A service can report as active in systemctl but still fail to serve traffic. The real test is making an actual HTTP request.

Make an HTTP request to your own server by running:

curl http://localhost

You should see a wall of HTML output. That's the default Nginx welcome page, which confirms the server is serving content.

Check just the response headers by running:

curl -I http://localhost

Look for HTTP/1.1 200 OK at the top of the output. A 200 status code means the server processed your request successfully.

What is curl?

curl is a command-line tool for making HTTP requests. Think of it as a text-based web browser.

The -I flag tells it to fetch only the response headers, skipping the page content. You'll use curl to test APIs, check server health, and debug connectivity issues.

Check which ports are listening on your server by running:

sudo ss -tlnp

You should see Nginx listening on port 80. This is the default port for HTTP traffic.

What does ss -tlnp mean?

-t shows TCP connections only.
-l shows only listening (waiting for connections) ports.
-n shows port numbers instead of service names.
-p shows which process owns each port.

ss is the modern replacement for netstat. You'll use it constantly to check what's running on which port.

Curl showing an error?

If you see Connection refused, Nginx may not be running. Run sudo systemctl status nginx to check.
If you see Could not resolve host, make sure you typed localhost exactly. It refers to the server you're currently logged into.
Help me troubleshoot my curl connection

Your web server is up and running. In the next step, you'll break it on purpose and use log analysis to find and fix the problem.

Break It and Fix It Part 1 - Permission Errors and Log Analysis

Your Nginx web server is installed, running, and responding to requests. Now comes the real SRE training.

In production, deployments can accidentally change file permissions and silently break a running server. You will deliberately cause this exact problem, then use log analysis tools to track it down.

In this step, get ready to:

Deliberately break Nginx by removing file permissions.
Diagnose the problem using error logs, grep, and journalctl.
Restore the server and verify it is healthy again.

Break the web server by changing file permissions

File permissions control who can read, write, and execute each file on a Linux system. When Nginx tries to serve a file it cannot read, the request fails with a 403 Forbidden error.

Check the current permissions of the Nginx welcome page by running:

ls -l /usr/share/nginx/html/index.html

You should see a permission string like -rw-r--r-- at the start of the output. This means the owner can read and write the file, and everyone else can read it.

Remove all permissions from the file by running:

sudo chmod 000 /usr/share/nginx/html/index.html

What does chmod 000 do?

The chmod command changes file permissions. 000 removes all read, write, and execute permissions for every user on the system.

This simulates a real production issue where a deployment script accidentally changes file permissions. The server keeps running, but it can no longer access the files it needs to serve.

Test the web server again by running:

curl http://localhost

You should see HTML containing 403 Forbidden instead of the welcome page. The server is running, but it cannot read the file.

Still seeing the welcome page?

Make sure you included sudo before the chmod command. Without it, the command does not have permission to change the file.
Run ls -l /usr/share/nginx/html/index.html to check the current permissions. If they still show -rw-r--r--, the chmod command did not take effect.

Help me figure out why my chmod command did not work

Find the problem using log analysis

When something breaks, logs are your first stop. Nginx writes error details to /var/log/nginx/error.log.

View the last 5 lines of the Nginx error log by running:

sudo tail -5 /var/log/nginx/error.log

You should see a Permission denied error pointing to index.html.

What does tail do?

The tail command shows the end of a file. The -5 flag limits the output to the last 5 lines.

In SRE work, the most recent errors are almost always at the bottom of the log file. tail lets you skip straight to them.

Search specifically for permission errors by running:

sudo grep "denied" /var/log/nginx/error.log

What does grep do?

The grep command searches for text patterns inside files. Here it filters the error log to show only lines containing "denied".

In large log files with thousands of lines, grep lets you zero in on exactly what matters. It is one of the most powerful commands in Linux.

Check the systemd journal for Nginx entries by running:

sudo journalctl -u nginx --since "5 minutes ago"

What does journalctl do?

The journalctl command is the centralized log viewer for all services managed by systemd. Instead of hunting through individual log files, you can query everything from one place.

The -u nginx flag filters entries to just the Nginx service.
The --since "5 minutes ago" flag limits results to a specific time window.

Why three different log commands?

Each tool has a strength. tail is fast for seeing the most recent entries. grep is powerful for searching large files by keyword. journalctl gives you a single view across all services with time-based filtering.

Real SREs reach for all three depending on the situation. Now you know when to use each one.

Fix the problem and verify

Restore read permissions on the welcome page by running:

sudo chmod 644 /usr/share/nginx/html/index.html

What does 644 mean?

The number 644 gives the owner read and write permissions. It gives the group and everyone else read-only permissions.

This is the standard permission level for web server files. The server process needs to read the file, but nobody should be writing to it through the web.

Confirm the permissions are restored by running:

ls -l /usr/share/nginx/html/index.html

You should see -rw-r--r-- at the start of the output. The file is readable again.

Verify the welcome page is back by running:

curl http://localhost

You should see the full Nginx welcome page HTML again. The 403 Forbidden error is gone.

Still seeing 403 Forbidden?

Make sure you included sudo before chmod. Without root privileges, the permission change silently fails.
Run ls -l /usr/share/nginx/html/index.html and confirm the permissions show -rw-r--r--. If they still show ----------, the restore did not work.
Double-check that the file path is exactly /usr/share/nginx/html/index.html. A typo in the path means chmod targeted the wrong file.

Help me fix my Nginx 403 error after restoring permissions

You just completed your first troubleshooting loop: observe the symptom, check the logs, find the cause, fix it, and verify. This is the core workflow of real incident response.

Permission errors are just one type of server failure. Next up, you will break the Nginx configuration file itself and learn to diagnose config syntax errors.

Break It and Fix It Part 2 - Config Errors and Process Management

You just completed your first troubleshooting loop by diagnosing a permission error through log analysis. Your Nginx web server is back online and healthy.

Configuration errors are the leading cause of outages in production. In this step, you will deliberately break your Nginx configuration file, then use diagnostic tools to find and fix the exact problem.

In this step, get ready to:

Back up and intentionally break the Nginx configuration file.
Diagnose the failure using systemctl status, nginx -t, and journalctl.
Fix the configuration, restart the service, and confirm everything is running.

Back up and break the Nginx configuration

Why back up config files?

A backup lets you quickly restore the original configuration if something goes wrong. In production, you would also use version control for config files. For now, a simple file copy keeps you safe.

Create a backup of your Nginx configuration file by running this command:

sudo cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.backup

Confirm the backup was created by running:

ls /etc/nginx/nginx.conf.backup

You should see the file path /etc/nginx/nginx.conf.backup printed in the output.

Now you will edit the Nginx config file to introduce a syntax error. The nano text editor is not pre-installed on Amazon Linux 2023, so you need to install it first.

Install the nano text editor by running:

sudo dnf install nano -y

Open the Nginx configuration file in nano by running:

sudo nano /etc/nginx/nginx.conf

A text editor opens inside your terminal. You need to find a specific line and remove one character from it.

Press Ctrl+W to open the search bar inside nano.
Type worker_connections and press Enter to jump to that line.
Use the arrow keys to move your cursor to the semicolon at the end of worker_connections 1024;.
Delete the semicolon so the line reads worker_connections 1024 with no semicolon at the end.
Press Ctrl+O then Enter to save the file.
Press Ctrl+X to exit nano.

What is nano?

nano is a simple text editor that runs inside the terminal. The commands at the bottom of the screen (like ^O for save and ^X for exit) use the caret symbol ^ to mean the Ctrl key.

Your configuration file now has a syntax error. Time to see what happens when Nginx tries to load it.

Try restarting Nginx with the broken configuration by running:

sudo systemctl restart nginx

The command returns an error. Nginx refuses to restart because the configuration file has a syntax error.

Seeing an error? That is the plan

You broke the configuration on purpose. A single missing semicolon is all it takes to bring down a web server. Now you will use diagnostic tools to track down the problem, just like an SRE would during a real incident.

Diagnose the problem

When a service fails, you need diagnostic tools to understand exactly what went wrong. You will use three commands that give you progressively more detail about the failure.

Check the service status by running:

sudo systemctl status nginx

Look for failed in red text. The output also shows the last few log lines hinting at what went wrong.

Test the configuration syntax directly by running:

sudo nginx -t

What does nginx -t do?

This is the single most useful Nginx troubleshooting command. It checks the configuration file for syntax errors and reports the exact file path and line number where the problem is.

You should see a message pointing to the missing semicolon in /etc/nginx/nginx.conf. In production, this is always the first command you run when Nginx will not start.

Check the system journal for recent Nginx entries by running:

sudo journalctl -u nginx -e

What do these flags mean?

The -u nginx flag filters journal entries to just the Nginx service. The -e flag jumps to the end of the log so you see the most recent entries first.

Between these three commands, you now know exactly what went wrong and where. The nginx -t output told you the precise file and line number of the syntax error.

Fix the config and verify the service

You know the cause: a missing semicolon on the worker_connections line. Time to fix it and bring the server back online.

Open the config file again by running:

sudo nano /etc/nginx/nginx.conf

Press Ctrl+W, type worker_connections, and press Enter to find the broken line.
Add the semicolon back to the end of the line so it reads worker_connections 1024;.
Press Ctrl+O then Enter to save.
Press Ctrl+X to exit nano.

Confirm the syntax is now correct by running:

sudo nginx -t

You should see syntax is ok and test is successful in the output.

Restart the Nginx service by running:

sudo systemctl restart nginx

Confirm Nginx is active and running by running:

sudo systemctl status nginx

Look for active (running) in green text. Your service is back online.

See the running Nginx processes by running:

ps aux | grep nginx

What does ps aux | grep do?

The ps aux command lists every running process on the server. The pipe | sends that list to grep, which filters it down to only lines containing nginx.

This is one of the most common patterns in Linux troubleshooting. You will use it constantly to check whether a specific process is running.

Verify the web server is responding by running:

curl http://localhost

You should see the Nginx welcome page HTML in the output. Your second troubleshooting loop is complete.

Nginx still not starting?

Double check that the semicolon is at the end of the worker_connections 1024; line. Run sudo nginx -t again to check for any remaining syntax errors.
Make sure you saved the file in nano before exiting. Press Ctrl+O then Enter to save, then Ctrl+X to exit.
If the config file is beyond repair, restore your backup by running sudo cp /etc/nginx/nginx.conf.backup /etc/nginx/nginx.conf and try again.

Why is Nginx still not starting after I fixed the config?

You have now completed two full troubleshooting loops using core Linux diagnostic tools. These are the same tools that SREs use every day to keep production servers running.

Secret mission

Write a Server Health Check Script

Real SREs don't run diagnostic commands one at a time during an incident. They automate their checks into scripts that run everything at once. Can you build your own?

Before you go

Clean Up Your Resources

Decide whether to keep your resources running, pause them to come back later, or delete them entirely. Your EC2 instance is covered by the AWS Free Tier if your account is eligible, but it costs approximately $0.0116/hour if it is not.

Cost warning

If your AWS account was created before July 15, 2025, your t2.micro instance is covered under 750 free hours per month for 12 months. If your account was created on or after that date, you receive a credit-based free tier instead.

Either way, if you are done with the project, terminate the instance to avoid any accidental charges.

Resources you used:

1x EC2 t2.micro instance (Amazon Linux 2023).
1x Key pair (linux-lab-key).
1x Security group (auto-created during instance launch).
Nginx web server installed on the instance.

Keep everything running

No action needed. Choose this if you are still actively experimenting with Linux commands on your instance.

Your instance continues running and you can reconnect with MobaXterm at any time.
Be mindful of the 750-hour monthly free tier limit if your account is eligible.

Pause - I'll come back to this later

Stop the instance to pause billing for compute while keeping your setup intact. You can start it again later.

In the AWS Console, navigate to EC2, then Instances.
Select your linux-lab instance.
Click Instance state, then Stop instance.
When you start the instance again, the public IP address will change. You will need to update the IP in MobaXterm when reconnecting.

Delete - I don't want to use this again

Remove all project resources permanently. This stops all charges.

In the AWS Console, navigate to EC2, then Instances.
Select your linux-lab instance.
Click Instance state, then Terminate instance.
Navigate to EC2, then Key Pairs.
Select linux-lab-key and delete it.
Delete the linux-lab-key.pem file from your local computer as well.

Mission accomplished

That's a Wrap!

Nice work! You just went from zero Linux experience to diagnosing and fixing real server problems on a live remote machine.

You've learned how to:

Launched an EC2 instance and connected to it via SSH using MobaXterm.
Navigated the Linux filesystem, understood file permissions, and managed software with dnf and services with systemctl.
Diagnosed and fixed real server failures using log analysis tools like tail, grep, journalctl, and nginx -t.
Secret Mission: Wrote a bash health check script to automate server monitoring.

Ready to quiz yourself?